Switch to fail-fast prompt waits after WaitUntilAsync in CLI E2E tests

[Copilot]

Description

Reviewed all WaitUntilAsync calls in Aspire.Cli.EndToEnd.Tests and added fail-fast error detection to those that wait for output from completing (non-interactive) commands but had no guard against the command producing an error prompt.

Motivation: When a CLI command exits with a non-zero code and the test is using WaitUntilAsync to look for specific success text, the wait spins until the full timeout (30–180 s) before the test fails. The fix uses WaitForSuccessPromptFailFastAsync after each text-check WaitUntilAsync, which fails immediately upon seeing an error prompt and gives faster feedback in CI.

Cases updated — all are completing (non-interactive) commands:

File	Command	Change
BannerTests.cs (3 sites)	aspire cache clear, aspire --banner, aspire --nologo --help	WaitForSuccessPromptAsync → WaitForSuccessPromptFailFastAsync
DoctorCommandTests.cs (2 sites)	aspire doctor (both tests)	WaitForSuccessPromptAsync → WaitForSuccessPromptFailFastAsync
AgentCommandTests.cs (3 sites)	aspire agent --help, aspire mcp --help, deprecated-config aspire doctor	WaitForSuccessPromptAsync → WaitForSuccessPromptFailFastAsync
CentralPackageManagementTests.cs (1 site)	aspire add	WaitForSuccessPromptAsync → WaitForSuccessPromptFailFastAsync

The WaitUntilAsync lambdas are kept focused on checking for expected output text; error-prompt detection is handled by the established WaitForSuccessPromptFailFastAsync helper.

Cases kept as-is (justified):

• SmokeTests.cs, JsReactTemplateTests.cs — aspire run is long-running; no completion prompt until Ctrl+C; already throws on regression case.
• PlaywrightCliInstallTests.cs (4×), AgentCommandTests.cs (4×) — interactive prompts mid-command; command is still running, no prompt yet.
• ConfigDiscoveryTests.cs — already checks for ERR: text in the lambda.
• StartStopTests.cs — WaitForAnyPromptAsync follows; both valid text outcomes checked.
• CliE2EAutomatorHelpers.cs:112 — initial bash prompt before prompt-counting is configured; no sequence number available.
• CliE2EAutomatorHelpers.cs:285, ProjectReferenceTests.cs — already handle both success and error paths.
• TypeScriptCodegenValidationTests.cs — expects failure; WaitForAnyPromptAsync follows.
• DescribeCommandTests.cs — simple echo/cat shell commands with 10 s timeout.

Checklist

• Is this feature complete?
  • Yes. Ready to ship.
  • No. Follow-up changes expected.
• Are you including unit tests for the changes and scenario tests if relevant?
  • Yes
  • No
• Did you add public API?
  • Yes
    • If yes, did you have an API Review for it?
      • Yes
      • No
    • Did you add <remarks /> and <code /> elements on your triple slash comments?
      • Yes
      • No
  • No
• Does the change make any security assumptions or guarantees?
  • Yes
    • If yes, have you done a threat model and had a security review?
      • Yes
      • No
  • No
• Does the change require an update in our Aspire docs?
  • Yes
    • Link to aspire.dev issue:
      • New issue
  • No

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 15934

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 15934"

[Copilot]

Pull request overview

This PR updates Aspire CLI end-to-end tests to use a fail-fast prompt wait helper after certain WaitUntilAsync output checks, aiming to reduce long timeouts when a CLI command exits non-zero and returns an error prompt.

Changes:

• Replaced WaitForSuccessPromptAsync(counter) with WaitForSuccessPromptFailFastAsync(counter) in several non-interactive command test flows.
• Applied the change across banner, doctor, agent help, and CPM “aspire add” scenarios.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 9 comments.

File	Description
tests/Aspire.Cli.EndToEnd.Tests/DoctorCommandTests.cs	Uses fail-fast prompt waiting after aspire doctor output checks.
tests/Aspire.Cli.EndToEnd.Tests/CentralPackageManagementTests.cs	Uses fail-fast prompt waiting after aspire add flow completion.
tests/Aspire.Cli.EndToEnd.Tests/BannerTests.cs	Uses fail-fast prompt waiting after banner/help output checks.
tests/Aspire.Cli.EndToEnd.Tests/AgentCommandTests.cs	Uses fail-fast prompt waiting after some help-output WaitUntilAsync checks.

[Copilot]

Switching to WaitForSuccessPromptFailFastAsync only checks for an ERR prompt after this WaitUntilAsync has already succeeded. If the command exits with an error before printing the expected banner text, WaitUntilAsync will still spin until its timeout. To truly fail fast on non-zero exits, consider adding an ERR-prompt check inside the WaitUntilAsync predicate (or wrap WaitUntilAsync in a helper that returns/throws when [N ERR: is observed).

[Copilot]

Same concern as above: this fail-fast prompt check runs only after WaitUntilAsync completes, so it won’t shorten timeouts when the command fails before producing the expected output. Consider integrating ERR prompt detection into the WaitUntilAsync predicate (or using a helper that does both).

[Copilot]

This fail-fast prompt wait happens after the help-output WaitUntilAsync, so it won’t prevent a full WaitUntilAsync timeout if the command errors before emitting HelpHint. If the goal is to fail immediately on non-zero exits, consider checking for an ERR prompt in the WaitUntilAsync loop as well.

[Copilot]

WaitForSuccessPromptFailFastAsync runs after the output-text WaitUntilAsync. If aspire doctor fails before emitting the expected text, the test will still wait the full WaitUntilAsync timeout. If you’re aiming to fail fast on [N ERR: prompts, consider adding ERR-prompt detection to the WaitUntilAsync predicate (or using a helper that combines both checks).

[Copilot]

Same pattern here: switching the prompt wait to fail-fast doesn’t help if WaitUntilAsync never becomes true due to an early command failure; it will still spin until the timeout. Consider detecting [N ERR: inside the WaitUntilAsync predicate (or using a wrapper helper) so failures surface immediately.

[Copilot]

Fail-fast prompt detection occurs only after WaitUntilAsync succeeds. If the help command fails before printing the expected text, WaitUntilAsync will still time out. If the intent is to avoid long timeouts on non-zero exits, consider adding an ERR-prompt check inside WaitUntilAsync (or use a helper that fails fast on ERR while waiting for output).

[Copilot]

Same concern: this only fails fast after WaitUntilAsync succeeds. If aspire mcp --help exits with an error before producing the expected tokens, the test will still wait the full WaitUntilAsync timeout. Consider integrating ERR prompt detection into the WaitUntilAsync predicate (or a wrapper helper).

[Copilot]

This fail-fast prompt wait comes after the output-text WaitUntilAsync. If aspire doctor fails before emitting the expected warning text, WaitUntilAsync will still run until timeout. To match the stated goal of fail-fast on ERR prompts, consider checking for [N ERR: within the WaitUntilAsync predicate as well.

[Copilot]

In this test, WaitUntilAsync above waits for either the version-selection prompt or a success prompt. If the command exits with an error prompt, WaitUntilAsync will still time out before reaching this fail-fast check. Consider updating the preceding WaitUntilAsync predicate to also return on an ERR prompt (and throw), or using a helper that waits for OK/ERR and fails fast on ERR while still allowing the optional version-selection interaction.

[radical]

@mitchdenny The automated reviewer flagged the same concern on all 9 changed sites: WaitForSuccessPromptFailFastAsync runs after WaitUntilAsync completes, so if the command errors before producing the expected text, WaitUntilAsync still spins for its full timeout (30–60s) before the fail-fast check ever executes.

This is technically correct — the current change only helps in the narrower case where the expected output text does appear but the command then exits with a non-zero code. The more common failure scenario (command errors immediately, never produces the expected text) isn't covered.

To truly fail fast, ERR-prompt detection would need to happen inside the WaitUntilAsync predicate, e.g.:

await auto.WaitUntilAsync(s =>
{
    if (s.ContainsErrPrompt(counter)) throw new Exception("Command failed unexpectedly");
    return s.ContainsText("expected output");
}, timeout: TimeSpan.FromSeconds(30));

Or better yet, a helper like WaitUntilWithFailFastAsync that wraps both checks so individual tests don't need to repeat the ERR-detection boilerplate.

-- copilot