Gate the onboarding "Choose how to start" option count on a server experiment arm (REV-1939)

[warp-agent-staging]

Summary

Ships the warpdotdev/warp client half of REV-1939: the account-first, post-authentication “Choose how to start” onboarding offer now renders the option set dictated by a server-assigned experiment arm, and the whole onboarding monetization funnel is tagged with that arm.

• Control restores the exact pre-Add ad-hoc AI credit pack option to onboarding "Choose how to start" #14605 two-option screen — “Use Warp with AI” + “Set up AI later” — even when purchasable credit packs are loaded.
• Experiment keeps the current three-option screen — “Subscribe to a Warp plan”, “Buy AI credits” with pack tiles, and “Set up AI later” — whenever packs are available.
• Unassigned, and any experiment user with no available packs, get the same safe historical two-option layout.

The sibling warp-server#13846 owns the experiment definition, bucketing, eligibility, and traffic split. This PR consumes its GraphQL contract; it adds no client feature flag and no client-only experiment layer.

The committed spec, .agents/specs/REV-1939-onboarding-option-experiment.md, remains in the diff and is the contract this change implements.

What changed

• Client experiment arms (app/src/server/experiments/, crates/graphql/src/api/experiment.rs, crates/warp_graphql_schema/api/schema.graphql) — adds OnboardingChooseHowToStartControl and OnboardingChooseHowToStartThreeOptions, mapped to the server's ONBOARDING_CHOOSE_HOW_TO_START_CONTROL and ONBOARDING_CHOOSE_HOW_TO_START_THREE_OPTIONS through the Cynic enum, the checked-in schema snapshot, TryFrom<Experiment>, Display, and from_string. Both on_added_to arms are explicit no-ops: consumers read the assignment directly, because a boolean flag cannot distinguish control from unassigned.
• Arm resolution (app/src/server/experiments/model.rs) — ServerExperiments::choose_how_to_start_experiment_arm() returns Control, Experiment, or Unassigned. Neither arm and (malformed) both arms both fail closed to Unassigned, which renders the safe two-option layout.
• Assignment carried into onboarding state (crates/onboarding/src/model.rs, agent_onboarding_view.rs, lib.rs) — a small copyable ChooseHowToStartExperimentArm lives in the reusable onboarding crate (defaulting to Unassigned), so the crate never has to depend on the app crate's ServerExperiments. RootView::resolve_account_first_post_auth snapshots the latest arm immediately before show_post_auth_offer and then leaves it frozen for that exposure, so a late server refresh cannot change the layout or split the funnel mid-flight.
• Rendering gated on the arm, not on pricing (crates/onboarding/src/slides/offer_slide.rs) — credit_packs/shows_credit_packs now require both the experiment arm and a non-empty eligible pack list. onboarding_credit_packs policy, premium, denomination, and refresh logic are untouched, so the model keeps pack data loaded regardless of arm and “not in the experiment” is never conflated with “pricing unavailable”. primary_label now takes shows_credit_packs so the card reads “Subscribe to a Warp plan” only when the buy-credits card renders, and “Use Warp with AI” otherwise.
• Arm-qualified funnel telemetry (crates/onboarding/src/telemetry.rs, app/src/root_view.rs, app/src/auth/login_slide.rs) — SlideViewed, OnboardingAction, OnboardingUpgradeStarted, OnboardingUpgradeCompleted, and OnboardingCompleted carry an optional experiment_arm of "control" | "experiment" | "unassigned". Non-offer and HeadStart callers pass None, so their payloads omit the key rather than emitting null. The arm is read before the root view transitions to Terminal, so upgrade completion and terminal completion still carry it.
• New credit-purchase funnel events — onboarding_credit_purchase_started and onboarding_credit_purchase_completed, carrying flow_version, source_slide, account_class, the selected credits, and the arm. “Started” fires only once a valid selected pack moves the model into Purchasing; “completed” fires only from the accepted on_credit_purchase_completed transition, so an abandoned or merely checkout-required purchase is never counted as a conversion.

Stable identifiers are unchanged: choose_how_to_start, use_warp_with_ai, buy_ai_credits, set_up_later, the /upgrade path, the checkout mutation, pricing/premium calculations, and team-UID selection all behave exactly as before. HeadStart, paid-user routing, and legacy onboarding are untouched.

Spec amendment — cross-repo enum names

The committed spec was drafted against provisional enum names ONBOARDING_CHOOSE_HOW_TO_START_CREDITS_CONTROL / ..._CREDITS_EXPERIMENT. The sibling server PR shipped ONBOARDING_CHOOSE_HOW_TO_START_CONTROL / ONBOARDING_CHOOSE_HOW_TO_START_THREE_OPTIONS instead. The implementation already matched the shipped values; commit aaada866 updates the spec text so the committed contract and the code agree. Nothing else in the spec changed, and the arm semantics are identical.

Verification

Regression tests (each fails before this change, passes after):

• choose_how_to_start_arm_resolves_each_assignment_state and choose_how_to_start_arm_follows_the_latest_state_over_the_cache — control only → Control, experiment only → Experiment, neither → Unassigned, both → Unassigned; a later apply_latest_state wins over the cached value.
• choose_how_to_start_arms_round_trip_through_graphql_and_persistence, schema_exposes_exactly_the_two_choose_how_to_start_arms, and unknown_experiment_values_are_still_ignored — the enum round-trips through GraphQL/Display/from_string, the checked-in schema carries exactly the two server-owned values, and unknown values stay ignored.
• control_and_unassigned_arms_render_the_historical_two_options — with purchasable packs loaded, control and unassigned yield exactly [Primary, SetUpLater], the historical label and description match byte-for-byte, no tiles render, and keyboard navigation skips BuyCredits.
• the_experiment_arm_falls_back_to_two_options_without_packs and losing_the_experiment_arm_falls_back_to_the_primary_selection — experiment + empty packs falls back to the two-option copy, and an experiment→control transition with buy-credits selected resolves the effective choice back to Primary.
• the_offer_snapshots_the_latest_arm_and_then_freezes_it — the arm is copied immediately before show_post_auth_offer, and an assignment update after the offer is shown does not mutate that exposure's arm, layout, or telemetry value.
• choose_how_to_start_funnel_payloads_carry_the_experiment_arm, credit_purchase_payloads_carry_the_denomination_and_arm, credit_purchase_event_names_are_stable, and credit_purchase_completion_is_only_reported_for_an_accepted_purchase — the funnel payloads carry the arm, purchase events carry the arm plus the denomination, event names are pinned, and a rejected or checkout-required purchase emits no completion.

Repository gate:

• ./script/format --check — clean
• cargo clippy -p onboarding -p warp_graphql -p warp --all-targets --tests -- -D warnings — clean
• cargo nextest run -p onboarding -p warp_graphql — 60/60 passed
• cargo nextest run -p warp — green except server::server_api::ai::tests::ambient_agent_headers_for_task_overrides_existing_cloud_agent_header, which fails identically on this branch's base (c8a166b6) and is unrelated to this change

Visual proof: the running onboarding offer surface was exercised with computer use under each forced assignment on a Linux runner. The recording and stills below show control rendering exactly two cards with packs loaded, the experiment arm rendering three cards with credit tiles and a selectable tile, and the unassigned run falling back to two cards — matching spec invariants #2, #4, and #7 and satisfying spec validation criterion #11.

Reviewer-visible stills — no Oz sign-in required

The Oz artifact links further down are permission-checked and bounce a signed-out
reviewer to a login page, so they could not be validated during review. The same
captures — the unmodified artifacts, byte for byte — are mirrored on the
asset-only branch
factory/rev-1939-visual-proof and embedded below, so they render
inline for anyone who can see this repository. That branch holds no source code,
is not part of this PR's diff, and is disposable: delete it once this PR is
merged or closed.

Control arm — two cards, "Use Warp with AI" (Recommended) and "Set up AI
later", with purchasable credit packs loaded and no credit tiles (spec invariant #2).

Experiment arm — three cards, "Subscribe to a Warp plan" (Recommended), "Buy
AI credits" with the 400 / 1,000 / 3,000 / 6,500 tiles, and "Set up AI later"
(spec invariant #4).

Unassigned run — the same two-card historical fallback (spec invariant #7).

The control and unassigned stills are pixel-identical, which is precisely what
the spec requires — an unassigned user must get the historical control layout,
not a third rendering. The recording below shows them as two separate runs.

Recording — all three arms, 11s. The same walkthrough as an inline animated
GIF: the control arm navigated with the arrow keys, the experiment arm navigated
and a credit tile clicked, then the unassigned run. Pressed keys and clicks show
as on-screen overlays.

The full-resolution 1080p source is available as a
raw mp4 download.
GitHub only plays an mp4 inline when it is uploaded through its web UI, which
this agent's token cannot do, so the GIF above is the in-page version of the
recording. Nothing here needs an Oz session.

Computer-use recording

View recording: The onboarding “Choose how to start” screen walked through the control arm (arrow-key navigation), the experiment arm (arrow-key navigation and clicking a credit tile), and the unassigned run.

Computer-use screenshots

View screenshot: Control-arm “Choose how to start” screen with two cards — “Use Warp with AI” (Recommended badge) and “Set up AI later”, no credit tiles.

View screenshot: Experiment-arm “Choose how to start” screen with three cards — “Subscribe to a Warp plan” (Recommended), “Buy AI credits” with credit/price tiles, and “Set up AI later”.

View screenshot: Unassigned-run “Choose how to start” screen with two cards — “Use Warp with AI” (Recommended badge) and “Set up AI later”, no credit tiles.

CHANGELOG-IMPROVEMENT: The onboarding "Choose how to start" screen now shows the two-option or three-option layout according to a server-assigned experiment.

Originating thread: https://warpdev.slack.com/archives/C0BDQDW8V5E/p1785884690170659

Rework changes

Rework cycle 1 of 1. One review finding, addressed without any source change —
aaada866 remains the head commit and the diff is untouched.

• ⚠️ [IMPORTANT] The required visual proof cannot be validated: the linked Oz
  video and screenshots redirect a signed-out reviewer to login. Fixed by
  supplementing, not replacing. The three stills are now embedded inline in this
  description (see Reviewer-visible stills above) from the asset-only branch
  factory/rev-1939-visual-proof, each captioned with the arm it shows and the
  spec invariant it demonstrates, and the recording is embedded as an
  inline animated GIF with the full-resolution mp4 linked beside it. The original Oz artifact links and their
  managed blocks are left in place unchanged. No new capture was taken: these
  are the same artifacts from the original verification pass, re-uploaded
  byte for byte, so the proof still matches the code at aaada866.

No other findings were raised: CI, mergeability, the test suite, the arm
resolution and offer-freeze tests, presubmit, and the client/server enum
agreement were all verified and accepted in review.

Conversation: https://staging.warp.dev/conversation/53aac554-48a5-46f0-b091-481c13fa5eac
Run: https://oz.staging.warp.dev/runs/019fcfed-23b9-7108-a774-9d38f454f486

Co-Authored-By: Warp Agent agent@warp.dev

This PR was generated with Oz.

[warp-agent-staging]

Overview

This PR implements the onboarding option-count experiment and its stills demonstrate the three required arm renderings. The rendered evidence is correct, but its stated retention plan leaves a human decision about whether the permanent PR record may depend on a disposable asset branch.

Concerns

❓ [QUESTION] The PR says the factory/rev-1939-visual-proof asset branch is disposable once this PR lands, while all embedded stills and the recording are pinned only to commits retained by that branch. Deleting that ref makes the permanent review evidence non-durable and the raw.githubusercontent.com URLs may eventually fail once the unreferenced objects are pruned. Should the ref be preserved indefinitely or the artifacts moved to an approved durable location before the branch is deleted?

Verdict

Checks: CI ✅ · scoped tests ✅ · formatting/Clippy ✅ · visual proof ✅
Found: 0 critical, 0 important, 0 suggestions, 1 human decision

Human decision required

Review run

https://oz.staging.warp.dev/runs/019fd063-5738-74ef-8f09-54345776d25e