Tempo setup

Overview

Grafana Tempo is an open-source, high-scale distributed tracing backend. It integrates seamlessly with Grafana for visualizing traces and helps you understand the behavior of your applications during deployments and in production.

Tempo is part of our observability stack and works alongside other monitoring tools to provide detailed insights into request flows across your distributed systems.

Prerequisites

Before enabling Tempo, ensure that:

• Your cluster has sufficient resources to run Tempo (default: 2Gi memory, 1 CPU per replica)
• You have access to modify your cluster definition file

Note: Enabling Tempo will automatically enable Alloy, which acts as the trace collection agent.

Configuration

To enable Tempo in your cluster, add or modify the following configuration in your cluster definition YAML file under spec:

observability:
  tempo:
    enabled: true
    replicas: 2
    resources:
      requests:
        memory: 2Gi
        cpu: 1
      limits:
        memory: 2Gi
    retention: 24h
    volumeSize: 10Gi

Configuration parameters

Deployment

After updating your cluster definition file:

1. Create a PR with your change
2. Apply the changes through your standard deployment process
3. Verify the deployment:

kubectl get pods -n observability | grep tempo

You should see Tempo pods running (number matches your replicas configuration).

Accessing traces in Grafana

Once Tempo is deployed:

1. Access your Grafana
2. Search for Tempo Operational
3. Start using the traces from Tempo

Additional resources

• Grafana Tempo documentation
• Grafana Tempo GitHub repository
• Distributed tracing best practices