Setup

Get access to a tokenizer. You can run your own, as described in the quick start in the tokenizer repo: https://github.com/superfly/tokenizer/ .

Make the app.

• fly app create proxypilot -o personal

Get the SEAL_KEY for your tokenizer. My tokenizer is https://timkenizer.fly.dev and I get its seal key by running fly -a timkenizer ssh console -C "tokenizer -sealkey". Make sure your cli-config.json references the tokenizer you want to use.

Seal secrets and set them for the app. The wrap.sh script does this, provided you have the SEAL_KEY for your tokenizer, and values for tokens in the ANTHROPIC_API_KEY, OPENAI_API_KEY, and GH_TOKEN.

• ./wrap.sh

Now deploy the app. This will install a tlsproxy as a sidecar from timflyio/tlsproxy on dockerhub, which is built from https://github.com/timflyio/tlsproxy. Note: make sure to specify the image label, which is referenced in cli-config.json. Otherwise the deploy will generate an image label which doesn't match.

• fly deploy --image-label shell

Try it out. From the shell container you can use gh, but you have no access to the real github token, or acces to the injected CA key. Go ahead and search the filesystem for it.

• fly ssh console --container shell
• echo $GH_TOKEN
• gh auth status

In the sideproxy container you can access the sealed github token, but it does not have access to the actual github token being used:

• fly ssh console --container sideproxy
• echo $GH_TOKEN

Destroy it

• fly m list
• fly m destroy --force <machine-id-here>

How it works

This builds two containers in a machine. The first is a shell in the shell container. It has the gh binary and the GH_TOKEN set to a dummy value. The shell has an /etc/hosts entry associating api.github.com with ::1. Requests to https://api.github.com will be sent there, and received by the side proxy's listener.

The second container is the sideproxy container which is listening on ::1 port 443. It auto-generates TLS certificates based on the request's SNI, using its own CA. The shell container is configured to trust this CA. To process a request, the server uses the https://tokenizer.fly.io proxy, passing in the sealed secret token, which is encrypted/sealed to the tokenizers public key. The tokenizer receives the sealed secret, and extracts its rules, which only allow access to https://api.github.com, and only works from the proxyauth app (using fly-src auth), unsealing the github token into an authorization header. It proxies this request to the http://api.github.com with the authorization header.

Example

This example demonstrates how dummy tokens are automatically replaced when making requests to anthropic, github, and openai:

$ fly ssh console --container shell
Connecting to fdaa:9:1094:a7b:4ce:9c8:c214:2... complete
root@shell:/# cd
root@shell:~# env |egrep 'TOKEN|KEY'
ANTHROPIC_API_KEY=dummy
OPENAI_API_KEY=dummy
GH_TOKEN=dummy
root@shell:~# cat anthropic.sh
#!/bin/sh

curl https://api.anthropic.com/v1/messages \
     --header "x-api-key: $ANTHROPIC_API_KEY" \
     --header "anthropic-version: 2023-06-01" \
     --header "content-type: application/json" \
     --data \
'{
    "model": "claude-opus-4-20250514",
    "max_tokens": 1024,
    "messages": [
        {"role": "user", "content": "Hello, world"}
    ]
}'
root@shell:~# ./anthropic.sh
{"id":"msg_0197HHPbR13ALq22Ea6PWHzF","type":"message","role":"assistant","model":"claude-opus-4-20250514","content":[{"type":"text","text":"Hello! Welcome to our conversation. How are you doing today? Is there anything specific you'd like to talk about or any questions I can help you with?"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":10,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"output_tokens":35,"service_tier":"standard"}}
root@shell:~# cat anthropic.mjs
import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic({
  apiKey: 'my_api_key', // defaults to process.env["ANTHROPIC_API_KEY"]
});

const msg = await anthropic.messages.create({
  model: "claude-opus-4-20250514",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello, Claude" }],
});
console.log(msg);
root@shell:~# node anthropic.mjs
{
  id: 'msg_01JX4bu4uzef1wtKNqFUJ6Qj',
  type: 'message',
  role: 'assistant',
  model: 'claude-opus-4-20250514',
  content: [
    {
      type: 'text',
      text: "Hello! It's nice to meet you. How are you doing today?"
    }
  ],
  stop_reason: 'end_turn',
  stop_sequence: null,
  usage: {
    input_tokens: 10,
    cache_creation_input_tokens: 0,
    cache_read_input_tokens: 0,
    output_tokens: 18,
    service_tier: 'standard'
  }
}
root@shell:~# cat xanthropic.py
#!/usr/bin/env python3

import anthropic

client = anthropic.Anthropic() # uses os.environ.get("ANTHROPIC_API_KEY")
message = client.messages.create(
    model="claude-opus-4-20250514",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello, Claude"}
    ]
)
print(message.content)
root@shell:~# ./xanthropic.py
[TextBlock(citations=None, text="Hello! It's nice to meet you. How are you doing today?", type='text')]
root@shell:~# gh auth status
github.com
  ✓ Logged in to github.com account timflyio (GH_TOKEN)
  - Active account: true
  - Git operations protocol: https
  - Token: *****
root@shell:~# cat openai.sh
#!/bin/sh

curl https://api.openai.com/v1/responses \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -d '{
    	"model": "gpt-4o-mini",
        "input": "Write a one-sentence bedtime story about a unicorn."
    }'
root@shell:~# ./openai.sh
{
  "id": "resp_68702bbe11c08194a0b85f89e23a817a005703628be936ce",
  "object": "response",
  "created_at": 1752181694,
  "status": "completed",
  "background": false,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "max_output_tokens": null,
  "max_tool_calls": null,
  "model": "gpt-4o-mini-2024-07-18",
  "output": [
    {
      "id": "msg_68702bbe72548194b80bea23f850f2bd005703628be936ce",
      "type": "message",
      "status": "completed",
      "content": [
        {
          "type": "output_text",
          "annotations": [],
          "logprobs": [],
          "text": "As the silvery moon cast a gentle glow over the enchanted meadow, the brave little unicorn spread her shimmering wings and soared into the starry sky, making wishes come true for all the children asleep below."
        }
      ],
      "role": "assistant"
    }
  ],
  "parallel_tool_calls": true,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "summary": null
  },
  "service_tier": "default",
  "store": true,
  "temperature": 1.0,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "tool_choice": "auto",
  "tools": [],
  "top_logprobs": 0,
  "top_p": 1.0,
  "truncation": "disabled",
  "usage": {
    "input_tokens": 18,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 42,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 60
  },
  "user": null,
  "metadata": {}
}
root@shell:~# cat xopenai.py
#!/usr/bin/env python3

from openai import OpenAI
client = OpenAI()

response = client.responses.create(
    model="gpt-4o-mini",
    input="Write a one-sentence bedtime story about a unicorn."
)

print(response.output_text)
root@shell:~# ./xopenai.py
As the moonlight danced upon the meadow, a gentle unicorn named Lila spread her shimmering wings and soared into the starry sky, leaving a trail of dreams for children everywhere to follow as they drifted off to sleep.
root@shell:~# cat openai.mjs
import OpenAI from "openai";
const client = new OpenAI();

const response = await client.responses.create({
    model: "gpt-4.1",
    input: "Write a one-sentence bedtime story about a unicorn.",
});

console.log(response.output_text);
root@shell:~# node openai.mjs
Under a sky sprinkled with twinkling stars, a gentle unicorn named Lila danced through a field of glowing moonflowers, carrying sweet dreams to every sleeping child.
root@shell:~# exit

Notes

• flyctl does not support a fly.toml option for specifying an image label, which means it has to be specified each time fly deploy is run. If its not specified an older version of the image will be run instead of the latest built image. This is less than ideal. Flyctl should probably support a fly.toml field for this. Would be even better if the deploy image name could be populated into the containers spec automatically by flyctl.
• If we wanted to automate this more, we seal tokens on behalf of users at deploy time, and lock down the sealed token to a specific org/app/machine id, so that the token couldn't even be moved to another machine in the same app.
• The wrapped auth tokens are exposed to all containers via the new /.fly/api secrets endpoints. ie. curl --unix-socket /.fly/api "http://flaps/v1/apps/$FLY_APP_NAME/secrets?show_secrets=1". This has implications to any container that is trying to limit secrets access!