Maintenance
- Upgraded cluster add-ons
2025-12-08
- Migrated to new ECR Basic Scanning version
2025-12-08
- Upgrading EKS clusters to v1.33 + node auto repair feature
2025-11-13
- Upgraded Concourse CI to version 7.14.2
2025-10-13
- Upgraded cluster add-ons
2025-09-25
- OpenSearch maintenance: improved monitoring and upgrading to 2.19
2025-09-23
- Upgraded Concourse CI to version 7.14.1
2025-08-21
- Upgraded cluster add-ons
2025-08-06
- Upgraded cluster add-ons
2025-07-04
- Skyscrapers report on security incident
2025-07-02
- Loki label optimisations to improve performance, rollout finished
2025-06-25
- Fixed subnet & route association bug
2025-06-16
- Grafana CVE-2025-4123 mitigation
2025-06-10
- [ACTION REQUIRED] Upgraded cluster add-ons
2025-05-20
- [ACTION REQUIRED] Loki label optimisations to improve performance
2025-05-08
- Upgraded cluster add-ons
2025-04-17
- Upgrading Concourse CI to version 7.13.1
2025-04-11
- Upgrading Concourse CI to version 7.13.0
2025-04-08
- All clusters patched against IngressNightmare (CVE-2025-1097, CVE-2025-1098, CVE-2025-1974, CVE-2025-24513, CVE-2025-24514)
2025-03-25
- Upgraded cluster add-ons
2025-03-12
- Upgrading Concourse CI to version 7.12.1
2025-02-10
- Upgrading EKS clusters to v1.32
2025-02-05
- [ACTION REQUIRED] Upgraded cluster add-ons
2025-01-23
- Upgrading EKS clusters to v1.31
2025-01-07
- Cleanup of Teleport
2024-12-11
- Upgrading Concourse CI to version 7.12.0
2024-12-09
- Loki optimisations to mitigate recurring performance issues
2024-12-06
- [ACTION REQUIRED] Upgraded cluster add-ons
2024-12-03
- Upgraded Teleport to version 15.4.21
2024-11-12
- Maintenance: OpenTofu upgraded to 1.8.3
2024-10-15
- [ACTION REQUIRED] Upgraded cluster add-ons
2024-09-19
- Upgrading Concourse CI to version 7.11.2
2024-09-03
- Upgraded Teleport to version 15.4.16
2024-08-26
- [ACTION REQUIRED] Upgraded cluster add-ons
2024-08-12
- Upgraded Teleport to version 15.4.9
2024-07-22
- [ACTION REQUIRED] Upgraded cluster add-ons
2024-07-11
- Our documentation is now available on https://docs.skyscrapers.eu
2024-06-27
- Migrating Docker Images from DockerHub to GitHub Container Registry (GHCR)
2024-06-24
- Upgrading EKS clusters to v1.30
2024-06-19
- Upgraded cluster add-ons - hotfixes
2024-06-13
- Upgraded Teleport to version 15.3.7
2024-05-29
- Upgraded cluster add-ons
2024-05-23
- Switching from Terraform to OpenTofu
2024-05-23
- Improving self-hoster GitHub Actions runners reliability
2024-05-02
- Upgraded Teleport to version 15.2.2
2024-04-24
- [Action required] Upgraded cluster add-ons
2024-04-11
- Overhauled EKS access control
2024-04-03
- Upgrading EKS clusters to v1.29
2024-03-28
- Secret envelope encryption enabled on EKS
2024-03-14
- Migration of GitHub actions-runner-controller to gha-runner-scale-set
2024-03-08
- Upgraded cluster add-ons
2024-02-29
- Upgraded Teleport to version 15.0.1
2024-02-16
- Use direct AWS EKS AMI instead of SkS custom-build
2024-01-22
- Upgraded cluster add-ons
2024-01-19
- Karpenter now deployed by default and migrated system pool
2024-01-11
- Vault upgraded to 1.15.4
2023-12-14
- Upgrading EKS clusters to v1.28
2023-12-05
- Upgraded cluster add-ons
2023-11-07
- Upgraded Vertical Pod Autoscaler to version 1.0.0
2023-10-10
- Upgraded Teleport to version 14.0.1
2023-10-10
- Upgraded cluster add-ons
2023-10-05
- Support for GPU node pools on AWS (EKS)
2023-09-29
- Removed Calico as NetworkPolicies engine - this functionality has merged with the AWS VPC CNI
2023-09-29
- Upgrading K8s clusters to v1.27
2023-09-20
- New experimental component Karpenter for cost-effective node scaling
2023-09-11
- Upgraded Teleport to version 13.3.8
2023-09-07
- Upgraded Teleport to version 13.3.0
2023-08-04
- Disable kubernetes-dashboard by default
2023-07-19
- Upgraded cluster add-ons
2023-07-14
- Upgraded cluster add-ons
2023-07-07
- Dedicated system node pool + reduced system component footprint
2023-07-05
- Upgrading K8s clusters to v1.26
2023-06-22
- Vault upgraded to 1.13.3
2023-06-09
- Upgrading OpenSearch clusters to 2.5
2023-06-09
- Upgraded Teleport to version 13.0.3
2023-05-31
- Post-mortem - A word on Pods stuck with ContainerCreating status problems
2023-05-30
- Upgraded fluent-bit and moving to built-in loki output
2023-05-25
- Upgraded cluster add-ons for monitoring & logging
2023-05-11
- [Action required] Update external-dns behavior for private Route53 zones
2023-05-10
- Upgraded cluster add-ons
2023-05-04
- Istio upgraded to version 1.16.4
2023-04-17
- Upgraded Teleport to version 12.2.1
2023-04-11
- Defaulting to capacity-optimized for Spot nodepools
2023-03-31
- [Action required] Upgrades starting for K8s clusters to 1.25
2023-03-30
- Upgraded EKS cluster add-ons
2023-03-27
- Deletion prevention on OpenSearch S3 backups
2023-03-20
- Istio upgraded to version 1.16.3
2023-03-16
- Vault upgraded to 1.13.0
2023-03-13
- INFO - Empty response for external.metrics.k8s.io/v1beta1 errors with kubectl and helm
2023-03-01
- Post Mortem - Loki log loss
2023-02-28
- Upgraded Teleport to version 12.0.2
2023-02-24
- Upgraded cluster add-ons
2023-02-20
- Improving Loki performance & scalability
2023-02-13
- Node Termination Handler Slack notifications disabled by default (AWS EKS)
2023-02-03
- Upgraded K8s clusters to 1.24
2023-01-27
- Upgraded Teleport to version 11.1.4 for security fixes
2022-12-26
- Upgraded cluster add-ons
2022-12-21
- Vault upgraded to 1.12.2
2022-12-13
- New feature: Use External-DNS for managing custom DNS records
2022-12-13
- Istio upgraded to version 1.16.1
2022-12-13
- Upgraded Teleport to version 11.1.1 for security fix
2022-12-02
- Upgraded Teleport to version 11.0.3
2022-11-28
- New feature: Kubernetes descheduler
2022-11-21
- Upgraded cluster add-ons
2022-11-18
- Upgraded K8s clusters to 1.23
2022-11-17
- Upgraded ingress-nginx, fixing CVE-2022-32149, CVE-2022-27664 and CVE-2022-1996
2022-11-10
- Upgraded monitoring add-ons, fixing Grafana CVE-2022-32149
2022-11-09
- AWS EKS AMI recalled
2022-11-04
- Improved monitoring for the RDS snapshot cross-account replicator module
2022-10-28
- Vault upgraded to 1.12.0
2022-10-25
- CVE-2022-27665 patches
2022-10-18
- Concourse upgraded to 7.8.3
2022-10-18
- NLB compatibility in Nginx Ingress
2022-10-17
- Istio upgraded to version 1.15.2
2022-10-17
- Upgraded cluster add-ons
2022-10-06
- New feature: Jaeger tracing
2022-10-06
- Add support for hierarchical namespaces
2022-09-27
- Concourse migrated from ECS to K8s
2022-09-16
- Upgraded Teleport to version 10.1.4
2022-08-24
- Upgraded cluster add-ons
2022-08-10
- Upgraded AKS and EKS clusters to 1.22
2022-07-25
- Upgraded cluster add-ons
2022-07-12
- Upgraded Teleport to version 9.3.7
2022-06-23
- Calico NetworkPolicy controller upgraded on EKS
2022-06-23
- Major Nginx Ingress Controller upgrade
2022-06-14
- Replacing eventrouter component for persisting K8s events
2022-06-07
- Improving Loki performance and usability
2022-05-25
- Upgraded cluster add-ons
2022-05-12
- [Action required] Final call - Deprecated API removal, upgrade your Ingresses etc.
2022-05-05
- Improved node termination handling (AWS EKS)
2022-04-26
- Add support for GP3 volumes through the AWS EBS CSI driver
2022-04-13
- Internal refactor of monitoring addons
2022-04-08
- [Important] New 24/7 escalation phone numbers!
2022-04-05
- Upgrade Nginx Ingress, fixing CVEs
2022-04-04
- Upgrade to Grafana v8 - take 2
2022-04-01
- [Important / breaking] VPC renames on AWS
2022-03-29
- Istio upgraded to version 1.13.2
2022-03-21
- Concourse upgraded to 7.7.0
2022-03-21
- Vault upgraded to 1.9.4
2022-03-18
- Upgraded cluster add-ons
2022-03-17
- Horizontal event-based scaling with KEDA
2022-03-11
- Hotfix for Grafana and InfoInhibitor alert info
2022-03-03
- EKS component upgrades
2022-03-01
- Allow for more fine-tuning of cluster-autoscaler params
2022-03-01
- AKS component upgrades
2022-03-01
- Upgraded Grafana and Prometheus
2022-02-24
- Upgraded Teleport to version 8.2.0
2022-02-14
- Github Actions Runner Controller
2022-02-07
- AKS rollouts are now automated
2022-02-07
- Calico NetworkPolicy controller upgraded on EKS
2022-02-02
- VPA enabled by for metrics-server
2022-01-28
- Let's Encrypt revocations affecting TLS-ALPN-01 certificates
2022-01-27
- Adding support for the AWS Load Balancer controller
2022-01-27
- Standardizing on Fluent Bit and Loki updates
2022-01-21
- VPA enabled for Vault
2022-01-07
- Upgraded Teleport to version 8.0.7
2022-01-07
- Module updated for AWS OpenSearch and started upgrades
2022-01-07
- Monitoring for Grafana Loki in case of discarded logs
2021-12-16
- Add support for mixed node pools in EKS
2021-12-16
- Critical CVE-2021-44228 in Log4j - Check your application workloads!
2021-12-14
- Add support for AWS Secrets Manager in EKS
2021-12-10
- Upgraded cluster add-ons
2021-12-06
- Upgraded Teleport to version 8.0.0
2021-11-26
- Istio upgraded to version 1.12.0
2021-11-25
- Introducing alerts for Fluent Bit errors
2021-11-15
- Concourse upgraded to v7.5.0
2021-10-18
- A note on Let's Encrypt chain issues due to DST Root CA X3 expiry
2021-10-11
- Making our Terraform helper script public
2021-10-07
- Grafana security patch following High Severity CVE-2021-39226
2021-10-07
- RDS snapshots cross-account replication available
2021-10-06
- Upgraded cluster add-ons
2021-09-27
- Guaranteed QoS for all critical system and infrastructure Pods
2021-09-27
- Improved EC2 instance interruption notifications
2021-09-20
- Upgrade AKS and EKS clusters to 1.21. Actions to take!
2021-09-16
- Istio upgraded to version 1.11.2
2021-09-13
- Mute critical KubeAPIErrorBudgetBurn alerts
2021-09-10
- VPA enabled by default
2021-09-09
- Downgraded Grafana to v7.5
2021-09-07
- Cert-manager upgraded to 1.4.4
2021-09-06
- Vault upgraded to 1.8.2
2021-08-31
- Vault upgraded to 1.8.1
2021-08-26
- Upgraded cluster components & increased Pod density. Actions to take!
2021-08-10
- Reduced memory usage of Cluster Autoscaler
2021-08-04
- Upgraded Teleport to version 6.2.8
2021-08-02
- Concourse upgraded to v7.3.2
2021-07-22
- Using encryption at rest for Prometheus and Alertmanager
2021-07-19
- Pod Disruption Budget for CoreDNS
2021-07-06
- Using Telepresence in our Reference Solution
2021-06-24
- GP2-encrypted is now the default storageclass
2021-06-18
- Always use encryption at rest for OpenVPN storage
2021-06-14
- Support for RabbitMQ and AmazonMQ monitoring
2021-06-10
- Initial support for Milvus database
2021-06-09
- Upgrade AKS and EKS clusters to 1.20
2021-06-04
- Encrypted storage layer of Teleport
2021-06-04
- Istio dashboards in Grafana
2021-06-03
- Istio upgraded to version 1.10.0
2021-05-27
- Vault upgraded to 1.7.2
2021-05-25
- Upgraded several cluster components
2021-05-18
- Adding support for EFS backed Persistent Volumes in Kubernetes
2021-05-18
- Concourse upgraded to v7.2.0
2021-05-12
- Vault upgraded to 1.7.1
2021-05-11
- Updated Teleport to version 4.4.9
2021-05-11
- Extra alerts for excessive resource usage
2021-05-11
- Merging AKS and EKS reference solution codebases and improving automation
2021-05-10
- Adding support for Istio service mesh
2021-04-14
- Upgraded several cluster components (patch-level)
2021-04-06
- Vault upgraded to 1.7.0
2021-03-30
- Upgrade AKS and EKS clusters to 1.19
2021-03-30
- Monitoring upgrades
2021-03-23
- Concourse upgraded to v7.1.0
2021-03-18
- Upgraded several cluster components
2021-03-16
- Improved monitoring alerts on Slack
2021-03-10
- Fixed regression in Elasticsearch monitoring for Prometheus
2021-03-05
- You can now use local NVMe Instance Storage with your Pods on EKS
2021-03-04
- Option to run the K8s API private
2021-02-23
- New container to authenticate to aws-ecr
2021-02-16
- Upgraded several cluster components
2021-02-09
- Auto-assigned Elastic IPs for K8s nodes (optional)
2021-02-05
- Striving for more automation on the K8s reference solution
2021-02-02
- Grafana main dashboard updated
2021-01-26
- Automated AWS ELasticsearch Service backups to S3
2021-01-26
- Velero S3 backups replication
2021-01-21
- Monitoring upgrades
2021-01-20
- Concourse upgraded to v6.7.3
2021-01-19
- Vault upgraded to 1.6.1
2021-01-12
- Grafana main dashboard updated
2020-12-18
- Upgraded Dex version with security patch
2020-12-15
- New log shipper added - Fluent Bit
2020-12-14
- Configurable default certificate and default backend on nginx-ingress
2020-12-14
- Upgrade EKS to 1.18
2020-12-08
- Updated Teleport to version 4.4.5
2020-12-08
- Fixed issue with Vault certificate renewal
2020-12-04
- Fall component upgrades
2020-11-30
- Support for GPU node pools in Azure Kubernetes Service (AKS)
2020-11-18
- Support for Azure Kubernetes Service (AKS)
2020-11-06
- Concourse upgraded to v6.7.1
2020-11-03
- Improved monitoring for Kubernetes backups
2020-10-16
- Concourse upgraded to v6.6.0
2020-10-15
- Vault upgraded to 1.5.4
2020-10-12
- Use the Vertical Pod Autoscaler to make your life easier and optimise cluster costs
2020-10-08
- Bugfix - Concourse workers out of disk space and crashing
2020-09-29
- Kubernetes events now available in Loki
2020-09-17
- Updated Teleport to version 4.3.5
2020-09-15
- Grafana main dashboard updated
2020-09-11
- Concourse upgraded to v6.5.1
2020-09-04
- Vault upgraded to 1.5.3
2020-09-03
- Upgraded several cluster components
2020-09-03
- Feature: NodeLocal DNSCache
2020-09-03
- Monitoring for External-DNS
2020-08-27
- Disable Nginx server token
2020-08-14
- Updates for addons
2020-08-05
- Concourse upgraded to v6.4.0
2020-07-31
- Vault upgraded to 1.5.0
2020-07-22
- Upgrade EKS to 1.17
2020-07-22
- Support Loki log-based alerting and metrics in Grafana
2020-07-10
- Updated docker images
2020-07-03
- Upgraded all ECS clusters
2020-06-25
- Updated Teleport to version 4.2.10
2020-06-25
- Updates for authorization addons
2020-06-18
- Update of ingress-nginx
2020-06-17
- Upgraded monitoring plugins
2020-06-16
- Update of Velero
2020-06-16
- Migrating from kube2iam to IAM roles for Service Accounts (IRSA)
2020-06-16
- Vault upgraded to 1.4.2
2020-06-05
- Upgrade EKS to 1.16
2020-06-05
- Cert-manger updated to 0.15.1
2020-06-05
- Additional Concourse monitoring
2020-06-05
- Upgrade Concourse to version 6.2.0 and other improvements
2020-06-04
- Resizable persistent volumes
2020-05-20
- Monitoring RDS instances through Prometheus
2020-05-19
- Automated OpenVPN deployments
2020-05-19
- Upgrades to monitoring & logging components
2020-05-13
- Fix Services without Endpoints timeout instead of reject
2020-05-13
- Option to configure custom routes and endpoints in Alertmanager
2020-05-12
- Option to add an internal-only Ingress controller
2020-05-07
- Oauth-proxy security fix following High Severity CVE-20200-11052
2020-05-06
- Loki - Option to set custom labels in Promtail
2020-05-06
- Update of cluster components
2020-04-27
- Upgrade EKS to 1.15
2020-04-16
- Improved Slack notifications for Prometheus alerts
2020-04-16
- Adjusting the NodeFilesystemSpaceFillingUp Prometheus alert
2020-04-16
- Simplifying our ECS monitoring
2020-04-14
- Migration to Helm 3
2020-04-10
- Auto configuration for AWS ElasticSearch multi-az deployment
2020-04-09
- Help fight COVID-19 with your Kubernetes cluster
2020-04-01
- Upgrades to monitoring components
2020-03-31
- Cluster addons upgrades
2020-03-27
- Alert and documentation for NodeWithImpairedVolumes
2020-03-27
- Use NetworkPolicies
2020-03-26
- Upgrade of core EKS components
2020-03-26
- Upgrade Concourse to version 5.8.1
2020-03-25
- Upgrade Caddy to version 1.0.4 with ACMEv2
2020-03-23
- Upgrade Concourse to version 5.8.0
2020-03-17
- Vault on K8s
2020-02-25
- All secret data is now encrypted in our Kubernetes definition files
2020-02-21
- Bugfix - Velero backups failing on some clusters
2020-02-11
- Bugfix - Raise fs.inotify limits
2020-02-11
- Documentation on how to have feature environments on Concourse
2020-01-16
- Allow runing K8s nodes and Concourse workers in public subnets
2020-01-16
- Bugfix - loki-promtail wasn't scheduled on tainted nodes
2020-01-15
- Bugfix - Grafana instability, increased memory request/limit
2020-01-15
- Teleport setups and updates are now fully automated
2020-01-14
- Upgrade Concourse to version 5.7.2
2019-12-12
- Spot termination alerts on Slack
2019-12-12
- Improved Prometheus-based ElasticSearch monitoring
2019-12-12
- Fixed Kubernetes cluster-autoscaler ASG auto-detectionn
2019-12-12
- Several other K8s Reference Solution improvemens
2019-12-05
- Kubernetes monitoring upgrades
2019-12-05
- Heavily reduced resource reservations for our default cluster Add-ons
2019-12-05
- Better support for different AWS regions and AZs for our Kubernetes reference solution
2019-12-05
- Moving from Calico to the AWS VPC CNI for EKS clusters
2019-10-22
- Upgrade Vault to 1.2.3
2019-10-17
- Introducing Grafana Loki to the k8s reference solution
2019-10-17
- CVE-2019-14287
2019-10-17
- CVE-2019-11253
2019-10-17
- Upgrade to Terraform 0.12
2019-10-04
- CVE-2019-16276 Upgrade Concourse to 5.5.3
2019-10-01
- Improved Kubernetes clusters automation
2019-09-30
- Upgrade EKS to 1.14
2019-09-25
- Upgrade Concourse to version 5.5.1
2019-09-25
- Upgrade Calico to 3.8.2
2019-09-24
- Concourse docker-image deprecation and how to migrate to the new registry-image
2019-09-19
- Switch Terraform Dynamodb tables and Vault Dynamodb backend to pay per request
2019-09-17
- Upgrade kops-based clusters to Kubernetes 1.11.10
2019-09-05
- Upgrade to EKS 1.13
2019-09-02
- Upgrade Concourse to version 5.4.1
2019-09-02
- Prometheus-blackbox-exporter available as optional cluster addon
2019-08-29
- Concourse task that checks the status of the service after the deployment
2019-08-29
- Redshift monitoring via Prometheus
2019-08-28
- Neo4j monitoring via Prometheus
2019-08-26
- Fix for dashboards HTTP 500 error when refreshing token
2019-08-20
- A note on CVE-2019-11247
2019-08-19
- Kubernetes dashboards ERR_TOO_MANY_REDIRECTS bug
2019-08-12
- Add Bitbucket, GitLab and Google authentication to Concourse
2019-08-09
- Kubernetes add-on upgrades
2019-08-06
- We're moving to EKS
2019-08-01
- Cluster and Persistent Volume backups with Velero 1.0
2019-07-16
- SSO / OAuth2 overhaul
2019-07-09
- Support for Cognito in ElasticSearch
2019-06-06
- Adding Prometheus monitoring for Elasticsearch on ECS
2019-06-04
- Move to the AWS provided Kibana
2019-04-17
- Update kube2iam to 0.10.7
2019-04-16
- Upgrade Concourse to version 5
2019-04-10
- Increased monitoring alerts visibility
2019-04-08
- Upgrade to Kubernetes 1.11.9 [CVE-2019-1002100, CVE-2019-9946, CVE-2019-3874, CVE-2019-1002101]
2019-03-29
- Create simple AWS resources from K8s via the AWS Service Operator
2019-03-19
- Support for cronjob monitoring
2019-03-18
- Upgrade Kubernetes components
2019-03-06
- Improved monitoring alerts on Slack
2019-03-06
- Mongodb monitoring and dashboards
2019-02-21
- Improved etcd backups
2019-02-19
- CVE-2019-5736 - Rolling out patched runc
2019-02-18
- Use encrypted EBS volumes for etcd storage and (optionally) encrypt k8s node root volumes
2019-01-21
- Move to CoreDNS dns server and add gp2-encrypted StorageClass
2019-01-15
- Upgrade Vault to 1.0.1
2019-01-11
- Upgrade to Kubernetes 1.11.6 [updated]
2019-01-03
- Upgrade to Kubernetes 1.10.11 [updated]
2018-12-03
- Set resource reservations for kubelet and other system processes
2018-11-27
- Adding Prometheus monitoring for ECS
2018-11-27
- Updated Prometheus & Grafana monitoring stack - update
2018-11-21
- Updated Prometheus & Grafana monitoring stack
2018-11-19
- Set resource requests and limits for all infrastructure pods
2018-11-13
- Moving from kube-lego to cert-manager for automatic TLS certificates
2018-11-13
- Grafana Pods dashboard updated memory metrics
2018-11-05
- Releasing our user-level documentation repository
2018-10-30
- K8S upgrade to stretch
2018-10-23
- Vault data is now backed up
2018-09-28
- Teleport upgrade to 2.7.5
2018-09-28
- Kubernetes cluster-autoscaler enabled
2018-09-28
- Reduced number of NAT gateways
2018-09-25
- Kubernetes Infrastructure Tools Upgraded
2018-09-25
- Upgrade logging to Kibana and Elasticsearch 6.3
2018-09-20
- Concourse version 4.2.1 upgrade
2018-09-19
- Concourse version 4.1.0 upgrade
2018-09-13
- Upgrade to Kubernetes 1.10
2018-09-12
- Reduced logging verbosity for system components
2018-08-29
- Better networking monitoring
2018-08-28
- Added AWS Inspector on Kubernetes cluster nodes
2018-08-25
- OIDC proxy for Kubernetes dashboard and other endpoints
2018-07-31
- Upgrade to Concourse v3.14.1
2018-07-02