GitHub Actions runner controller

You can dynamically deploy Github self-hosted runners on the Kubernetes clusters using the gha-runner-scale-set-controller. The controller allows developers to define either organization or repository runners, and it automatically handles the registration and de-registration of those.

The controller is disabled by default but we can quickly enable it upon request.

Github authentication

In order for the controller to authenticate to Github and manage the runners, it needs either a Github application or a Github Personal Access Token (PAT) with access to the Github organization. We highly encourage you to use the Github application method, as the PAT requires very broad and high level access to the Github organization, which poses a high security risk would the token be exposed. You can read more about how the controller authenticates with Github in the official documentation.

To configure the gha-runner-scale-set-controller with a Github application follow these steps:

1. Create a new Github App for your organization by following this URL (replace :org with your organization name):

   https://github.com/organizations/:org/settings/apps/new?url=http://github.com/actions-runner-controller/actions-runner-controller&webhook_active=false&public=false&administration=write&organization_self_hosted_runners=write&actions=read&checks=read

   You will create a Github App with the following permissions (already enabled if you’ve followed the link above):

   • Required Permissions for Repository Runners:
     • Repository Permissions
       • Actions (read)
       • Administration (read / write)
       • Checks (read) (to use Webhook Driven Scaling)
       • Metadata (read)
   • Required Permissions for Organization Runners:
     • Repository Permissions
       • Actions (read)
       • Metadata (read)
     • Organization Permissions
       • Self-hosted runners (read / write)

   Note

   All API routes mapped to their permissions can be found here if you wish to review*

2. You will see an App ID on the page of the GitHub App you created as follows, the value of this App ID will be used later.

3. Download the private key file by pushing the “Generate a private key” button at the bottom of the GitHub App page. This file will also be used later.

4. Go to the “Install App” tab on the left side of the page and install the GitHub App that you created for your account or organization.

5. When the installation is complete, you will be taken to a URL in one of the following formats, the last number of the URL will be used as the Installation ID later (For example, if the URL ends in settings/installations/12345, then the Installation ID is 12345).

   • https://github.com/settings/installations/${INSTALLATION_ID}
   • https://github.com/organizations/<org>/settings/installations/${INSTALLATION_ID}

6. Finally, register the App ID (APP_ID), Installation ID (INSTALLATION_ID), and downloaded private key file (PRIVATE_KEY_FILE_PATH) to Kubernetes as Secret in the arc-system namespace (remember to set the correct kubectl context before running this command):

   $ kubectl create secret generic github-actions-controller-manager \
       -n arc-system \
       --from-literal=github_app_id=${APP_ID} \
       --from-literal=github_app_installation_id=${INSTALLATION_ID} \
       --from-file=github_app_private_key=${PRIVATE_KEY_FILE_PATH}

Alternatively, you can write the Github app credentials in a secret in AWS Secrets Manager, and configure ARC accordingly to fetch the details from there. Note that you’ll need to have the AWS Secrets Manager CSI driver enabled in your cluster. If you want to take this route, omit the last step (6.) above, and instead create a secret in AWS Secrets Manager following this format:

{
    "github_app_id": "",
    "github_app_installation_id": "",
    "github_app_private_key": ""
}

After the secret is updated and contains the necessary info, add the secret’s ARN to your cluster definition file. This grants the controller’s IAM role read access to that secret:

...
spec:
  ghaRunnerController:
    awsSecretsManager:
      secretARN: <secret-arn>

Let Skyscrapers know the ARN as well: we also set it in the runner configuration below, so the controller mounts the secret.

Runner configuration

After authentication is configured you’ll be ready to start setting up the runners. Skyscrapers sets them up with Terragrunt/OpenTofu, using our arc stack, which deploys both the controller and the runner scale sets. Each entry in gha_runner_scale_sets becomes a runner scale set:

inputs = {
  gha_runner_controller = {
    # Only when the GitHub App credentials live in AWS Secrets Manager
    aws_secrets_manager = {
      secret_arn = "<secret-arn>"
    }
  }

  gha_runner_scale_sets = [
    {
      environment           = "production"
      github_config_url     = "https://github.com/<org>"
      runner_scale_set_name = "k8s-runners-production"

      min_runners = 0 # Number of idle runners kept alive, 0 scales to zero when no jobs are running
      max_runners = 5

      cpu    = "1"
      memory = "1.5Gi"

      # Namespaces where the runners get admin rights, e.g. to deploy with kubectl or Helm
      gha_admin_k8s_namespaces = ["production"]

      dind_enabled = false
    },
  ]
}

Once those runners are up and running you can verify the runners are online in the GitHub UI and you can update the jobs in your workflows to use the runners by setting the runs-on: key in the job definition to the name of the runner scale set (k8s-runners-production in this example).