Tailscale setup

Overview

This guide explains how to set up Tailscale integration with Skyscrapers-managed EKS clusters. Tailscale provides secure VPN connectivity to your Kubernetes clusters, enabling private network access to cluster resources and advertised routes.

The integration deploys Tailscale connectors as Kubernetes pods that:

  • Advertise the VPC CIDR block to your Tailnet
  • Provide secure, authenticated access to cluster resources
  • Support custom route advertisement
  • Can optionally function as an exit node

Account Setup

Note

Steps 1 and 2 are completed by the customer. From Step 3 onwards, Skyscrapers will handle the configuration and deployment.

1. Creating Your Tailscale Account

Before you can integrate Tailscale with your Skyscrapers-managed clusters, you need to set up a Tailscale account.

Signing Up with SSO

Important

If you want to make use of the multiple Tailnets feature (e.g., separate Tailnets for different environments) you can’t use GitHub SSO as it only supports a single Tailnet per identity provider.

  1. Visit Tailscale signup
  2. Select your identity provider (Google, Microsoft, Okta, etc.)
  3. Complete the SSO authentication flow
  4. Once authenticated, you’ll be directed to your Tailscale admin console
  5. Your organization’s tailnet will be automatically created

2. Granting Access to Skyscrapers

To allow Skyscrapers to provision and configure Tailscale for your clusters, you need to grant access to the Skyscrapers engineer who will be setting up the integration:

  1. Log into your Tailscale admin console
  2. Navigate to SettingsUsers
  3. Click Invite usersOutside of my org
  4. Add the Skyscrapers engineer’s email address
  5. Assign an appropriate role (typically Admin for initial setup)
  6. Click Send invites
  7. The Skyscrapers engineer will accept the invitation and gain access to your tailnet for configuration purposes
  8. You need to approve the Skyscrapers engineer’s access in the Tailscale admin console under Users

Important

Skyscrapers engineers will not connect to your tailnet as regular users. The integration uses OAuth-authenticated connector pods running in your Kubernetes clusters, which do not count as user seats on your Tailscale billing plan. This means:

  • Skyscrapers will not incur any additional user costs on your Tailscale account
  • You will only be charged for the actual users and devices from your organization that connect to the tailnet
  • The connector pods are treated as subnet routers, which are free in all Tailscale plans
  • Guest access for Skyscrapers engineers (if needed for initial setup) does not count toward user limits on paid plans

3. Account Linking

Skyscrapers is a Tailscale MSP partner and takes care of the account management and billing on your behalf. The Tailscale account used for your tailnet will be linked to Skyscrapers’ partner account for billing purposes.

For that the Skyscrapers engineer needs to link your Tailscale account to Skyscrapers’ partner account (at the moment this is a manual process with Tailscale support):

  1. In the Tailscale settings, select the Tailnet ID
  2. Share the Tailnet ID in the #ext-partner-tailscale Slack channel with the required information.

4. OAuth Client Setup

The Skyscrapers Tailscale integration requires two separate OAuth clients for different purposes:

In order to create OAuth clients you need to have configured an initial ACL policy in your Tailscale admin console. Otherwise you’ll get a permission denied error when trying to create the OAuth clients.

OAuth Client #1: Infrastructure as Code Management

This OAuth client is used by OpenTofu to manage Tailscale devices, routes, and auth keys in your Kubernetes clusters.

  1. In the Tailscale admin console, navigate to SettingsTrust credentials

  2. Click Add OAuth client or Generate OAuth client

  3. Provide a descriptive name (e.g., “Skyscrapers OpenTofu - EKS Clusters”)

  4. Under Scopes, select Custom scopes and configure the following permissions:

    Devices:

    • Core: Write; Add tag:terraform
    • Posture Attributes: Write
    • Routes: Write
    • Device Invites: Write

    Keys:

    • Auth Keys: Write; Add tag:terraform
    • OAuth Keys: Write
  5. Click Generate client or Save changes

  6. Save both the OAuth Client ID and OAuth Client Secret

OAuth Client #2: GitHub Actions Policy Management

This OAuth client is used by GitHub Actions to automatically manage and update your Tailscale ACL policy file.

  1. In the Tailscale admin console, navigate to SettingsTrust credentials

  2. Click Add OAuth client or Generate OAuth client

  3. Provide a descriptive name (e.g., “Skyscrapers GitHub Actions - Policy Management”)

  4. Under Scopes, select Custom scopes and configure the following permissions:

    General:

    • Policy File: Write ✓ (Read or modify tailnet policy file)
  5. Click Generate client or Save changes

  6. Save both the OAuth Client ID and OAuth Client Secret

Both OAuth clients should be created separately with their specific scopes to follow the principle of least privilege.

Configuration

Access Control List (ACL) Configuration

The Tailscale ACL policy file controls who can access what within your tailnet. This file is managed in a GitHub repository and automatically deployed via GitHub Actions using the GitHub OAuth Client created above.

See our ACL file docs on details on how to configure the Tailscale ACL.

Deployment Configuration

See our deployment docs on details on how to configure the deployment.

Deployment

Step 1: Apply Configuration

After updating the cluster definition, deploy the changes using the Concourse pipeline of the corresponding cluster(s)

Step 2: Monitor Deployment

The Tailscale connector pods will be deployed via Flux. Monitor the deployment:

kubectl get pods -n tailscale

Expected output:

NAME                                  READY   STATUS    RESTARTS   AGE
tailscale-connector-0-<hash>          1/1     Running   0          2m
tailscale-connector-1-<hash>          1/1     Running   0          2m

Step 3: Verify in Tailscale Admin Console

  1. Log into your Tailscale admin console
  2. Verify that new machines appear with hostname format: <cluster-name>-<index>
  3. Check that the advertised routes include:
    • Your VPC CIDR block
    • Any additional routes specified in extra_routes
  4. Ensure the machines are tagged with:
    • tag:k8s
    • tag:<cluster-name>
    • tag:terraform

Verification

Test Connectivity

Once deployed, test connectivity from your local machine (connected to the Tailnet):

# Ping a node in the cluster's VPC
ping 10.0.1.10

# Access the Kubernetes API (if advertised)
kubectl get nodes

# Connect to an endpoint directly (if using extra routes)
curl http://192.168.248.100:8080

Troubleshooting

Check Logs

If you encounter issues, check the connector pod logs:

kubectl logs -n tailscale -l app.kubernetes.io/name=tailscale-connector

Pods Not Starting

Symptom: Pods stuck in Pending or CrashLoopBackOff

Solutions:

  • Verify KMS-encrypted secret is correct
  • Verify the OAuth credentials have the correct permissions
  • Check that nodes have sufficient resources
  • Review pod logs for specific errors

Routes Not Appearing

Symptom: Advertised routes don’t appear in Tailscale admin console

Solutions:

  1. Verify route advertisement in pod logs:
kubectl logs -n tailscale <pod-name> | grep routes
  1. Check that OAuth client has routes:write permission
  2. In Tailscale admin console, ensure routes are approved (may require manual approval)

Authentication Failures

Symptom: authentication failed errors in logs

Solutions:

  • Verify OAuth credentials are correct
  • Regenerate OAuth client if necessary

Connection Drops

Symptom: Intermittent connectivity to cluster resources

Solutions:

  • Verify multiple replicas are running (should have 2 by default)
  • Check pod disruption budget: kubectl get pdb -n tailscale
  • Review pod restarts: kubectl get pods -n tailscale -o wide
  • Increase replica count or resources if needed

Additional Resources

Support

For issues or questions:

Last updated on