Add guidance to avoid exposing raw secret values in conversation #1855

xingyaoww · 2026-01-30T00:59:04Z

Summary

This PR adds guidance to the system prompt instructing agents to avoid viewing or exposing raw secret/token values.

Motivation

Conversation history may be logged, stored, or shared (e.g., for debugging, sharing sessions, or review). If an agent prints or echoes the full value of a secret (e.g., echo $GITHUB_TOKEN), those raw values could be exposed in the conversation logs, compromising security.

This change was discussed in a slack thread when an agent revealed the prefix of a GitHub token when asked if it had access to the token.

Changes

Updated system_message_suffix.j2 template to include new guidance:

Avoid exposing raw secrets: Never attempt to print, echo, or otherwise view the full value of secrets (e.g., avoid echo $GITHUB_TOKEN). The conversation history may be logged or shared, and exposing raw secret values could compromise security. Always pass secrets by reference using environment variables.
Added test assertions to verify the new guidance is included in the system message suffix when secrets are configured.

Testing

Ran existing tests for test_agent_context.py - all pass
Added assertions to verify the new guidance text is present

@xingyaoww can click here to continue refining the PR

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.13-nodejs22`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:47b6ba8-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-47b6ba8-python \
  ghcr.io/openhands/agent-server:47b6ba8-python

All tags pushed for this build

ghcr.io/openhands/agent-server:47b6ba8-golang-amd64
ghcr.io/openhands/agent-server:47b6ba8-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:47b6ba8-golang-arm64
ghcr.io/openhands/agent-server:47b6ba8-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:47b6ba8-java-amd64
ghcr.io/openhands/agent-server:47b6ba8-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:47b6ba8-java-arm64
ghcr.io/openhands/agent-server:47b6ba8-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:47b6ba8-python-amd64
ghcr.io/openhands/agent-server:47b6ba8-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-amd64
ghcr.io/openhands/agent-server:47b6ba8-python-arm64
ghcr.io/openhands/agent-server:47b6ba8-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-arm64
ghcr.io/openhands/agent-server:47b6ba8-golang
ghcr.io/openhands/agent-server:47b6ba8-java
ghcr.io/openhands/agent-server:47b6ba8-python

About Multi-Architecture Support

Each variant tag (e.g., 47b6ba8-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., 47b6ba8-python-amd64) are also available if needed

Update system prompt to instruct agents to never print, echo, or view the full value of secrets. Since conversation history may be logged or shared, exposing raw secret values could compromise security. Agents should always pass secrets by reference using environment variables. Co-authored-by: openhands <openhands@all-hands.dev>

github-actions · 2026-01-30T01:02:29Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
TOTAL	16462	4804	70%

report-only-changed-files is enabled. No files were changed during this commit :)

all-hands-bot

Code Review Summary

This PR addresses an important security concern, but the implementation introduces contradictory guidance that could confuse agents. The main issues: (1) conflicting examples about echoing tokens, (2) confusing "pass by reference" terminology, and (3) weak test coverage. See inline comments for specific suggestions.

all-hands-bot · 2026-01-30T06:24:18Z

openhands-sdk/openhands/sdk/context/prompts/templates/system_message_suffix.j2

@@ -30,6 +30,7 @@ You can also directly look up a skill's full content by reading its location pat
 * How to use secrets: Simply reference the secret key in your command (e.g., `echo ${GITHUB_TOKEN:0:8}` or `curl -H "Authorization: Bearer $API_KEY" https://api.example.com`). The system will detect the key name in your command text and export it as environment variable before it executes your command.


🟠 Important: This line shows echo ${GITHUB_TOKEN:0:8} as a valid example, but line 33 says "Never attempt to print, echo, or otherwise view the full value of secrets (e.g., avoid echo $GITHUB_TOKEN)".

While technically you distinguish between "partial" (first 8 chars) and "full" value, the guidance is confusing because:

Line 33 says "Never attempt to print, echo, or otherwise view" which sounds absolute

Users might think this example is now wrong

The real distinction (debugging partial values vs. viewing full secrets) is unclear

Suggestion: Either remove the echo ${GITHUB_TOKEN:0:8} example entirely, or explicitly clarify in line 33 that viewing prefixes for debugging is acceptable but viewing the full value is not.

all-hands-bot · 2026-01-30T06:24:18Z

openhands-sdk/openhands/sdk/context/prompts/templates/system_message_suffix.j2

 * How to use secrets: Simply reference the secret key in your command (e.g., `echo ${GITHUB_TOKEN:0:8}` or `curl -H "Authorization: Bearer $API_KEY" https://api.example.com`). The system will detect the key name in your command text and export it as environment variable before it executes your command.
 * Secret detection: The system performs case-insensitive matching to find secret keys in your command text. If a registered secret key appears anywhere in your command, its value will be made available as an environment variable.
 * Security: Secret values are automatically masked in command output to prevent accidental exposure. You will see `<secret-hidden>` instead of the actual secret value in the output.
+* Avoid exposing raw secrets: Never attempt to print, echo, or otherwise view the full value of secrets (e.g., avoid `echo $GITHUB_TOKEN`). The conversation history may be logged or shared, and exposing raw secret values could compromise security. Always pass secrets by reference using environment variables.


🟠 Important: The phrase "Always pass secrets by reference using environment variables" is confusing because:

The secrets ARE already environment variables

"Pass by reference" is programming language terminology that doesn't apply to shell environment variables

It's unclear what action this is recommending

Suggestion: Rephrase to something clearer like:

* Avoid exposing raw secrets: Never echo or print the full value of secrets to view them (e.g., avoid `echo $GITHUB_TOKEN`). The conversation history may be logged or shared, and exposing raw secret values could compromise security. Instead, use secrets directly in your commands (e.g., in curl headers, git URLs) where they serve their intended purpose.

all-hands-bot · 2026-01-30T06:24:18Z

openhands-sdk/openhands/sdk/context/prompts/templates/system_message_suffix.j2

 * Secret detection: The system performs case-insensitive matching to find secret keys in your command text. If a registered secret key appears anywhere in your command, its value will be made available as an environment variable.
 * Security: Secret values are automatically masked in command output to prevent accidental exposure. You will see `<secret-hidden>` instead of the actual secret value in the output.
+* Avoid exposing raw secrets: Never attempt to print, echo, or otherwise view the full value of secrets (e.g., avoid `echo $GITHUB_TOKEN`). The conversation history may be logged or shared, and exposing raw secret values could compromise security. Always pass secrets by reference using environment variables.
 * Refreshing expired secrets: Some secrets (like GITHUB_TOKEN) may be updated periodically or expire over time. If a secret stops working (e.g., authentication failures), try using it again in a new command - the system should automatically use the refreshed value. For example, if GITHUB_TOKEN was used in a git remote URL and later expired, you can update the remote URL with the current token: `git remote set-url origin https://${GITHUB_TOKEN}@github.com/username/repo.git` to pick up the refreshed token value.


🟡 Suggestion: This example also technically exposes the token value, since https://${GITHUB_TOKEN}@github.com/username/repo.git will:

Appear in git remote configs

Potentially appear in process lists

Be visible in shell history

This seems to contradict the guidance in line 33 about not exposing secrets. Consider adding a note that this is a necessary exception for git authentication, or showing an alternative like using credential helpers.

Note: This is a pre-existing issue in the template, not introduced by this PR, but worth addressing for consistency.

tests/sdk/context/test_agent_context.py

openhands-ai · 2026-01-30T07:03:27Z

Looks like there are a few issues preventing this PR from being merged!

GitHub Actions are failing:
- Run tests

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1855 at branch `feat/avoid-raw-token-exposure`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

enyst

Could you please tell, do I understand correctly, the issue is that the agent printed a partial secret, e.g. the first 4 or the first 8 characters?

xingyaoww · 2026-01-30T07:34:59Z

@enyst printing partial secret is ok-ish, this basically try to discourage the agent from doing echo $GITHUB_TOKEN so the WHOLE github token is persisted in the event history :(

xingyaoww · 2026-01-30T07:35:52Z

@OpenHands please address the unresolved review comments here - make sure all tests pass before you push
#1855 (review)

openhands-ai · 2026-01-30T07:36:00Z

I'm on it! xingyaoww can track my progress at all-hands.dev

- Remove confusing 'pass by reference' terminology - Remove echo example from 'How to use secrets' to avoid contradiction - Clarify that secrets should be used directly in commands (curl headers, git URLs) - Improve test to verify guidance is within CUSTOM_SECRETS section

openhands-ai · 2026-01-30T07:47:35Z

I've addressed all the review comments from PR #1855 and pushed the changes. Here's a summary:

Changes Made

1. Template (`system_message_suffix.j2`)

Removed confusing echo ${GITHUB_TOKEN:0:8} example from line 30 to avoid contradiction with the "avoid exposing raw secrets" guidance
Removed confusing "pass by reference" terminology - replaced with clearer guidance: "Instead, use secrets directly in commands where they serve their intended purpose (e.g., in curl headers or git URLs)"
Simplified the guidance from "Never attempt to print, echo, or otherwise view the full value" to "Never echo or print the full value" for clarity

2. Test (`test_agent_context.py`)

Improved test assertions to verify the guidance appears within the <CUSTOM_SECRETS> section specifically, not just anywhere in the result

Review Comments Addressed

✅ Line 30: Removed the echo ${GITHUB_TOKEN:0:8} example that contradicted line 33
✅ Line 33: Removed confusing "pass by reference" terminology and clarified the guidance
✅ Line 34: Pre-existing issue noted but not in scope of this PR
✅ Line 740: Improved test to verify guidance is in the correct section

Verification

All 2025 tests pass
Pre-commit hooks pass (ruff, pyright, etc.)
Changes pushed to feat/avoid-raw-token-exposure branch

View full conversation

openhands-sdk/openhands/sdk/context/prompts/templates/system_message_suffix.j2

enyst

Worth trying IMHO.

On the scenario it’s fixing, what LLM was it?

…essage_suffix.j2 Co-authored-by: Engel Nyst <engel.nyst@gmail.com>

xingyaoww · 2026-01-30T16:41:16Z

@enyst i think it didn't print the full key this time (it was opus), but i did remember sonnet printed a full key at some point, so couldn't hurt to get this in

xingyaoww marked this pull request as ready for review January 30, 2026 06:21

Merge branch 'main' into feat/avoid-raw-token-exposure

0b07a3d

xingyaoww requested review from enyst, neubig and rbren January 30, 2026 06:22

xingyaoww enabled auto-merge (squash) January 30, 2026 06:22

all-hands-bot reviewed Jan 30, 2026

View reviewed changes

enyst reviewed Jan 30, 2026

View reviewed changes

xingyaoww disabled auto-merge January 30, 2026 07:35

Merge branch 'main' into feat/avoid-raw-token-exposure

a4591ba

xingyaoww requested a review from enyst January 30, 2026 16:09

enyst reviewed Jan 30, 2026

View reviewed changes

openhands-sdk/openhands/sdk/context/prompts/templates/system_message_suffix.j2 Outdated Show resolved Hide resolved

enyst approved these changes Jan 30, 2026

View reviewed changes

Update openhands-sdk/openhands/sdk/context/prompts/templates/system_m…

81f5476

…essage_suffix.j2 Co-authored-by: Engel Nyst <engel.nyst@gmail.com>

Merge branch 'main' into feat/avoid-raw-token-exposure

c8f75cb

xingyaoww enabled auto-merge (squash) January 30, 2026 16:41

xingyaoww merged commit f6322c9 into main Jan 30, 2026
18 checks passed

xingyaoww deleted the feat/avoid-raw-token-exposure branch January 30, 2026 16:43

		@@ -30,6 +30,7 @@ You can also directly look up a skill's full content by reading its location pat
		* How to use secrets: Simply reference the secret key in your command (e.g., `echo ${GITHUB_TOKEN:0:8}` or `curl -H "Authorization: Bearer $API_KEY" https://api.example.com`). The system will detect the key name in your command text and export it as environment variable before it executes your command.

Add guidance to avoid exposing raw secret values in conversation #1855

Add guidance to avoid exposing raw secret values in conversation #1855

Uh oh!

Conversation

xingyaoww commented Jan 30, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Changes

Testing

Uh oh!

github-actions bot commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Code Review Summary

Uh oh!

all-hands-bot Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

all-hands-bot Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

all-hands-bot Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

openhands-ai bot commented Jan 30, 2026

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

xingyaoww commented Jan 30, 2026

Uh oh!

xingyaoww commented Jan 30, 2026

Uh oh!

openhands-ai bot commented Jan 30, 2026

Uh oh!

openhands-ai bot commented Jan 30, 2026

Changes Made

1. Template (system_message_suffix.j2)

2. Test (test_agent_context.py)

Review Comments Addressed

Verification

Uh oh!

Uh oh!

enyst left a comment

Choose a reason for hiding this comment

Uh oh!

xingyaoww commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xingyaoww commented Jan 30, 2026 •

edited by github-actions bot

Loading

github-actions bot commented Jan 30, 2026 •

edited

Loading

1. Template (`system_message_suffix.j2`)

2. Test (`test_agent_context.py`)

xingyaoww commented Jan 30, 2026 •

edited

Loading