feat: add /strands test command for TUI testing#718
Open
Hweinstock wants to merge 21 commits intoaws:mainfrom
Open
feat: add /strands test command for TUI testing#718Hweinstock wants to merge 21 commits intoaws:mainfrom
Hweinstock wants to merge 21 commits intoaws:mainfrom
Conversation
Hweinstock
had a problem deploying
to
e2e-testing
GitHub Actions
Failure
Hweinstock
had a problem deploying
to
e2e-testing
GitHub Actions
Failure
Hweinstock
force-pushed
the
feat/strands-test
branch
from
0b335d0 to
e6b631b
Compare
Hweinstock
had a problem deploying
to
e2e-testing
GitHub Actions
Failure
Hweinstock
had a problem deploying
to
e2e-testing
GitHub Actions
Failure
- Add tester mode to process-inputs.cjs (routes /strands test) - Add task-tester.sop.md with TUI testing instructions - Add tui-test-flows.md with 5 test flows - Add Node.js setup + build steps for tester mode in workflow - Wire TUI harness MCP server (stdio) into the Strands agent
Hweinstock
force-pushed
the
feat/strands-test
branch
from
e6b631b to
640f7f2
Compare
Hweinstock
force-pushed
the
feat/strands-test
branch
from
8769eba to
870e1ad
Compare
notgitika
requested changes
Apr 3, 2026
| npm install -g "$(npm pack | tail -1)" | ||
| mkdir -p /tmp/mcp-harness | ||
| cp dist/mcp-harness/index.mjs /tmp/mcp-harness/index.mjs | ||
| cd /tmp/mcp-harness && npm init -y && npm install node-pty @xterm/headless express |
Contributor
There was a problem hiding this comment.
low risk but can we pin the versions here?
Contributor
Author
There was a problem hiding this comment.
Are we worried about a breaking change from those packages? I would argue the maintenance burden of pinning and updating isn't worth the tradeoff for consistency.
| fetch-depth: 0 | ||
|
|
||
| - name: Add strands-running label | ||
| continue-on-error: true |
Contributor
Author
There was a problem hiding this comment.
The runner attempts to add a label to the issue/PR, which doesn't work when running from a fork. Since this step is non-critical, I think its safe to continue if we hit that error.
….repo, add comment on continue-on-error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a
/strands testcommand that uses the existing TUI harness MCP server to drive the CLI interactively and verify behavior on PRs. The Strands agent launches the CLI in pseudo-terminals, interacts with it, verifies expectations, and posts results as PR comments with screenshots of any failures.How it works
There are two modes: predefined flows, and dynamic flows.
Commenting
/strands test {prompt}on a PR will trigger the agent in "tester" mode. If a prompt is specified after the test, it will only run the specified flow (dynamic), otherwise it will run all predefined flows. Results are posted as a PR comment with pass/fail counts and failure screenshots. Successes are also verified with screenshots as artifacts on the GH actions run.Ex.
--> https://github.com/Hweinstock/agentcore-cli/actions/runs/23949720694
(see attached screenshots, not able to post comment since its running in my fork).
Related Issue
Closes #
Documentation PR
N/A
Type of Change
Testing
npm run test:unitandnpm run test:integnpm run typechecknpm run lintsrc/assets/, I rannpm run test:update-snapshotsand committed the updated snapshotsSee example run for testing.
Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.