Changelog
This changelog lists all updates, improvements and new features our Engineering team develops for our Skyscrapers Reference Developer Platform. These are rolled out automatically to all DevOps-as-a-Service customers.
2022 Q3
- 2022-08-10
Maintenance
Upgraded cluster add-ons
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. We’ve already rolled these out to all non-production clusters in the past days. Production upgrades are scheduled to happen next week during business …
- 2022-07-25
Maintenance
Upgraded AKS and EKS clusters to 1.22
We have started rolling out AKS and EKS 1.22. This brings our supported AKS platforms to v1.22.11 and EKS to v1.22.10. Upon writing we have already upgraded all non-production clusters. Production clusters will follow next week after some extra validation. …
- 2022-07-12
Maintenance
Upgraded cluster add-ons
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. We’ve already started rolling these out to all non-production clusters. Production upgrades are scheduled to happen next week during business hours. …
2022 Q2
- 2022-06-23
Maintenance
Upgraded Teleport to version 9.3.7
We’ve upgraded all Teleport clusters and nodes from version 8.2.0 to 9.3.7. Teleport is a tool we mostly use internally to provide secure and auditted access to (EC2) instances and Kubernetes clusters. You can find more information on this release in …
- 2022-06-23
Maintenance
Calico NetworkPolicy controller upgraded on EKS
On AWS EKS clusters we use Calico for providing NetworkPolicy functionality as an optional feature. With these NetworkPolicies you can control the traffic flow within a Kubernetes cluster between Pods, Services and external resources. Calico, and the …
- 2022-06-14
Maintenance
Major Nginx Ingress Controller upgrade
We’re in the process of upgrading the Nginx Ingress Controller from the legacy v0.51.0 version to mainline v1.2.1. This is in preparation for the AKS and EKS upgrades to Kubernetes 1.22 which is following in the coming weeks. All non-production …
- 2022-06-07
Maintenance
Replacing eventrouter component for persisting K8s events
Kubernetes events are a great resource to debug and troubleshoot problems with workloads and other cluster components. The problem is that the K8s API stores them for only 1 hour. To be able to persist those events further in time, we used an open-source …
- 2022-05-25
Maintenance
Improving Loki performance and usability
Some of our customers have experienced performance-related limitations in our Loki & Fluent-bit setup, mainly on queries that require scanning a large volume of data. At the moment we run a monolith Loki architecture, so the single Loki Pod performs …
- 2022-05-12
Maintenance
Upgraded cluster add-ons
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. And it’s a big one! We’ve already rolled these out to all non-production clusters. Production upgrades are scheduled to happen on Monday 16/05 …
- 2022-05-05
Maintenance
[Action required] Final call - Deprecated API removal, upgrade your Ingresses etc.
We are preparing to upgrade our platforms to Kubernetes 1.22, which drops support for many deprecated API versions! This is the final call to make sure you’ve updated your manifests, Helm charts (*), etc. to make use of the newest APIs (mainly …
- 2022-04-26
Maintenance
Improved node termination handling (AWS EKS)
For a long time now, we’ve been using the AWS Node Termination Handler for catching Spot instance interruption notices, allowing Kubernetes to respond appropriately by draining these (spot) nodes before they are terminated. You may have noticed this …
- 2022-04-13
Maintenance
Add support for GP3 volumes through the AWS EBS CSI driver
You can now create GP3 Persistent Volumes through the gp3 and gp3-encrypted Storage Classes. This is in addition to the previously available GP2 (gp2, gp2-encrypted) and EFS (efs) volumes. These new gp3* Storage Classes are provided by the AWS EBS CSI …
- 2022-04-08
Maintenance
Internal refactor of monitoring addons
We’ve completely refactored how we manage our monitoring components, like Prometheus, Grafana and many exporters and alerts. For you, the platform user, nothing will change although there is some disruption in Grafana and Prometheus expected during …
- 2022-04-05
Maintenance
[Important] New 24/7 escalation phone numbers!
It has come to our attention that many of our 24/7 escalation phone numbers could only receive calls from domestic numbers. We have generated new telepohone numbers, so make sure to verify your new number in your repo’s README!
- 2022-04-04
Maintenance
Upgrade Nginx Ingress, fixing CVEs
We’ve rolled out an nginx-ingress update (v0.51.0) to all clusters, with the following fixes: This release upgrades Alpine to 3.14.4 and nginx to 1.19.10 Patches OpenSSL CVE-2022-0778 Patches Libxml2 CVE-2022-23308
- 2022-04-01
Maintenance
Upgrade to Grafana v8 - take 2
Last year we rolled out a major Grafana update, going from 7.5 to 8.1, but had to roll back because a small number of customers were impacted by a breaking change regarding SQL datasources. Meanwhile we’ve taken the time to investigate the breaking …
2022 Q1
- 2022-03-29
Maintenance
[Important / breaking] VPC renames on AWS
We’d like to inform you that we’re renaming and (re-tagging) many network related resources, like the VPC name, subnet names, route tables, etc. Normally this shouldn’t have any impact for you, however if you rely on the VPC name for …
- 2022-03-21
Maintenance
Istio upgraded to version 1.13.2
We have upgraded Istio on all clusters that use it. The version was upgraded from 1.12.0 to 1.13.2. You can check the full release notes here. We’ve taken the chance to also upgrade Kiali to the latest version, from 1.43.0 to 1.47.0 (changelog). The …
- 2022-03-21
Maintenance
Concourse upgraded to 7.7.0
We have upgraded our Concourse setups to the latest version 7.7.0. This new version brings several small features and bug fixes. You can check the full changelog in the Concourse releases page. Rolling out this upgrade requires us to replace all the …
- 2022-03-18
Maintenance
Vault upgraded to 1.9.4
All Vault setups have been updated from 1.9.0 to the latest patch version 1.9.4. This release contains mostly small improvements and bug fixes. For full details and other changes, please refer to the upstream Vault release notes: …
- 2022-03-17
Maintenance
Upgraded cluster add-ons
As part of our regular upgrade cycle, the following Kubernetes cluster components have been updated. We’ve already rolled these out to all non-production clusters. Production upgrades will happen on Monday 21/03 during business hours. Most of the …
- 2022-03-11
Maintenance
Horizontal event-based scaling with KEDA
As of today we offer KEDA as a default component for horizontally scaling your Pods. Horizontal Pod Autoscaling, based on CPU and Memory metrics, was already possible through the standard Kubernetes HPA. However our customers asked for more possibilities, …
- 2022-03-03
Maintenance
Hotfix for Grafana and InfoInhibitor alert info
There are no actions to take, and all changes have been rolled out to all environments. Grafana datasources missing Last Monday (28/02), we pushed the latest monitoring upgrades to production environments. Quite soon some customer noticed issues with …
- 2022-03-01
Maintenance
EKS component upgrades
As part of our regular upgrade cycle, the following AWS specific Kubernetes cluster components have been updated. We’ve already rolled these out to all non-production clusters. Production clusters will follow once we validated everything is stable. …
- 2022-03-01
Maintenance
Allow for more fine-tuning of cluster-autoscaler params
We have exposed more parameters to the cluster-autoscaler, allowing for more fine-grained control. initially, only the scale_down_utilization_threshold could be configured. Now this is extended with the following parameters: balance_similar_node_groups …