2026-11-18
November add-on upgrades: renamed GPU metrics and a fixed Secrets Store CSI driver
#add-on #kubernetes #eks #update #upgrade #component #observability
Our November round of add-on upgrades is rolling out: non-production clusters first, production after a soak period. Two changes are worth knowing about: the GPU metrics have new names, and the Secrets Store CSI driver is back on its latest release.
What changed
- GPU metrics have new names and units. The NVIDIA DCGM exporter 4.9.0 renames every GPU metric, and most units change: utilization is a ratio from 0 to 1 instead of a percentage, memory is in bytes, and clock speeds are in Hz. Our GPU dashboards already use the new names, and the GPU monitoring guide lists every old and new name.
- The Secrets Store CSI driver is on its latest release again. We had held it on 1.5 because 1.6.0 could delete the Secrets it syncs from AWS Secrets Manager. Version 1.6.2 fixes that.
- A new node image. EKS node image v20261107 replaces every node once. Karpenter moves your workloads within your node pools’ disruption budgets, as in any node rotation.
- Atlantis runs OpenTofu 1.13.9. Your
atlantis planandatlantis applyruns use OpenTofu 1.13.9 now, up from 1.13.1. - Small behaviour changes.
- Traefik now rejects BasicAuth passwords longer than 1024 bytes, and more than 500 values in a single request header.
- The oauth2-proxy in front of internal tools reports itself as not ready while its Redis is down.
- On clusters with the Mountpoint for S3 CSI driver, opening a file for writing fails with
ENOMEMonce the limit of files open for writing at the same time is reached.
What you need to do
If you built your own Grafana dashboards or Prometheus alerts on the DCGM_FI_DEV_* GPU metrics, update them to the new names. Check thresholds too: an alert on > 90 percent utilization becomes > 0.9, and it would never fire on the new ratio. Without GPU nodes, there’s nothing to do.
Add-on upgrades
- aikido-kubernetes-agent v1.27.0 (chart v2.16.0)
- alloy v1.22.0 (chart v1.16.0)
- argo-workflows v4.1.6 (chart v2.0.18)
- aws-ebs-csi-driver v1.68.0
- aws-efs-csi-driver v3.5.4
- aws-load-balancer-controller v3.6.3
- aws-mountpoint-s3-csi-driver v2.9.2
- Caps the number of files open for writing; see above.
- dcgm-exporter 4.9.0
- Renamed GPU metrics; see above.
- eks-node-monitoring-agent v1.7.7
- flux v2.10.0
- grafana-loki v3.7.10 (chart v21.2.3)
- Major chart version; your logs and log retention are not affected.
- karpenter v1.15.2
- kube-prometheus-stack v93.3.0 (Prometheus Operator v0.95.2)
- node-exporter and kube-state-metrics restart once with stricter security defaults.
- metrics-server v0.9.1
- oauth2-proxy v7.15.6 (chart v10.7.4)
- opentofu v1.13.9 (Atlantis)
- prometheus-blackbox-exporter v0.29.1 (chart v11.21.3)
- prometheus-cloudwatch-exporter v0.16.0 (chart v0.29.0)
- prometheus-elasticsearch-exporter v1.11.0 (chart v7.5.0, OpenSearch module 14.1.2)
- secrets-store-csi-driver v1.6.2
- Back on the latest release; see above.
- tailscale v1.104.7
- traefik v3.7.18 (chart v42.4.0)
- velero v1.18.2 (chart v12.2.5)
- vertical-pod-autoscaler v1.8.1 (chart v0.16.0)
- vpc-cni v1.23.3
We also picked up the latest EKS build revisions for CoreDNS, kube-proxy and the snapshot controller (packaging updates only, no version change).