Prometheus watches flux-apps, and backups skip what they can't restore

Two smaller changes that roll out with our next platform release: non-production clusters first, production after a soak period.

Prometheus picks up monitors and alerts in flux-apps

Prometheus only reads ServiceMonitors, PodMonitors and PrometheusRules from namespaces that have a prometheus label. The flux-apps namespace, where our golden paths deploy your applications, didn’t have it. So metrics and alerts you defined there were silently ignored, unless you’d asked us to add the label or added it by hand.

The flux-apps namespace now has the label on every cluster.

What you need to do: check the alerting rules you’ve put in flux-apps. Rules that were ignored until now start being evaluated, so an alert you wrote a while ago and never saw can start firing in your alerts channel. Objects still need their own prometheus label, as described in the Prometheus guide. Other namespaces you create yourself still need the label too.

Backups leave out what Velero never restores

Your nightly Velero backup no longer includes Kubernetes events. Velero never restores events anyway, so they only made backups bigger and slower.

You can now also leave other resource types out of the backup, through velero.excludedResources in your cluster definition. That helps when a cluster holds many objects that have no use after a restore, such as finished Argo Workflows runs, which can make the backup run out of memory. Ask us if you’d like to use it.

What you need to do: nothing.