docs: conformance status of the CLI against SPEC v0.2

[msampathkumar]

Conformance status: a2a-cli against SPEC.md v0.2 — all tiers

Audited commit: b741dc3 (main, 2026-08-28) · Audit date: 2026-09-01
Specification: SPEC.md v0.2, rev 2026-08-17 (Review — pre-Proposed)
Registry: COMPLIANCE.md

Status snapshot, not a conformance claim. Graded against main only, excluding open PRs #23 and #24. No TCK-conformant third-party agent was exercised, so per the Conformance model nothing here may be published as a tier claim. §5 shows what #23 already changes.

---

1. Goals

1. Establish a baseline. Give a per-requirement, evidence-linked map of where the implementation stands against every tier of v0.2 — not just Tier 1.
2. Separate "not built" from "built differently." A missing feature and a feature that diverges from its contract need different fixes; conflating them hides both.
3. Flag registry drift. Where COMPLIANCE.md describes a surface SPEC.md has since changed, record it — a stale row makes a conformant tool read as failing.
4. Produce a re-runnable artifact. Machine-readable YAML keyed by requirement ID, so this can be regenerated as work lands instead of re-argued.

This document measures; it does not plan. Remediation ordering, effort estimates and issue-filing live outside it.

2. Scope

In scope: all three capability tiers of SPEC.md v0.2 — Tier 1 Core, Tier 2 Standard, Tier 3 Advanced.

Explicitly out of scope for this report:

Excluded	Requirement IDs	Count	Why
Authentication	all A2ACLI_AUTH_* (Tier 1: 001–004 · Tier 2: 005–007 · Tier 3: 008–011)	11	Handled separately
Agent Skills	A2ACLI_SKILL_001, A2ACLI_SKILL_002	2	Tracked in #4

Exclusion rule: by requirement area, not by subject matter. A2ACLI_CARD_GET_003 (authenticated extended card) stays in scope because its area is CARD_GET; its auth dependency is noted in its row.

Total in scope: 53 of 66 requirements.

One in-scope item already has an issue and is reported here only for completeness: CARD_GET_005 (catalog resolution) → #19.

Disclosure. This audit was produced by the author of SPEC.md, measuring an implementation written largely by another maintainer. The specification is pre-Proposed (§15.1) and still moving — PR #23 amends it while this is being written — so a gap is at least as likely to mean the spec moved as the code is behind. §4.4 records three cases where the fault is on the specification side. Read the numbers as a drift map between two halves of the same project, not as a grade.

3. Summary

Graded fail-closed per Appendix A.3: only pass satisfies; partial and fail both block a tier.

Tier	In scope	✅ Pass	◐ Partial	❌ Fail	Pass rate	Satisfied?
Tier 1 — Core	33	16	10	7	48%	No
Tier 2 — Standard	12	6	1	5	50%	No
Tier 3 — Advanced	8	2	1	5	25%	No
Total	53	24	12	17	45%	No

The shape of the gap. Every A2A operation in Appendix A has a command behind it, three transports are wired, and go build ./... / go test ./... are green. What is missing is the layer the specification exists to standardise — the machine contract: deterministic exit codes, the JSON error envelope, blocking-wait-by-default, and task get --wait. Those four account for most of Tier 1's shortfall and are what let a script, a CI job, or a coding harness drive the tool unattended (§2.3, §6.1, §6.6).

Counting commands gives ~80% done. Counting requirements gives 43%. The difference is entirely the output and error contract — and for a CLI whose stated audience is programs, that contract is the product.

4. Details

Companion documents:

Document	Contents
report-partial-features.md	The 12 partial items, with the specific gap in each
compliance-partial.yaml	Same 12, machine-readable, keyed by requirement ID
report-failed-features.md	The 18 missing/failed items, split into unbuilt vs built-but-non-conformant
compliance-failed.yaml	Same 18, machine-readable

4.1 Passing (24)

Tier 1 (16). CARD_GET_001 · SEND_001 · SEND_003 · SEND_005 · TASK_GET_001 · TASK_CANCEL_001 · INTERACT_001 · INTERACT_003 · INTERACT_005 · OUT_001 · OUT_006 · TX_001 · TX_002 · TX_003 · CONFIG_001 · CLI_001

Tier 2 (6). TASK_LIST_001 · TASK_SUBSCRIBE_001 · TX_004 · CONFIG_002 · CLI_002 · PUSH_001

Tier 3 (2). TX_005 (gRPC) · DEMO_SERVER_001 (shipped as server --echo|--exec|--proxy, exceeding the requirement)

Notable strengths: card resolution handles all four reference forms; v0.3↔v1.0 compatibility is wired on both client and server; message parts are repeatable and order-preserving with correct --media-type binding; configuration precedence is complete and tested. TX_003 passes by delegation — the SDK's tenantTransportDecorator (a2aclient/factory.go:167, v2.5.0) applies the selected interface's declared tenant when the request carries none, and omits it when the interface declares none. The CLI's non-spec --tenant global can override a declared value, which is worth watching but is not a §13.1 breach on its own.

Caveat — four of these have no automated test. task cancel, task list, task subscribe, and config show output are covered by no test case, so "pass" rests on a code read (plus, for config show, a manual run). TASK_CANCEL_001's idempotency requirement in particular is unverified. Strictly these are closer to not_measured than pass; they are recorded as pass because the code path is unambiguous, and flagged here rather than silently upgraded.

4.2 Partial (12) — built, but diverging from contract

Tier	ID	One-line gap
1	SEND_002	--async works; the blocking-wait default it opts out of does not exist
1	SEND_004	Nothing dropped, but a file part shows [file: url], not name + media type + size
1	SEND_006	Ordering correct; --data-part - treats - as a literal, not stdin
1	INTERACT_002	Rejection surfaces correctly, but cannot "point at --debug" — no such flag
1	INTERACT_004	Ids printed; no resume command, no stop-and-report on interruption
1	TASK_POLL_003	Stops on INPUT_REQUIRED but not AUTH_REQUIRED
1	TASK_POLL_004	Falls back only on unsupported-capability, not on mid-stream failure
1	OUT_002	One document ✅, but a bare Task instead of the SendMessageResponse wrapper
1	OUT_005	Labels stable, no ANSI; artifacts are indented lines, not §11.2 blocks
1	EXIT_002	Exit 0 correct; no stderr warning on a FAILED/REJECTED outcome
2	VER_003	Capability checked for streaming; not for push or extended card
3	CARD_GET_003	--extended works; the public card's declared schemes are never consulted first

4.3 Missing or failed (17)

Built but non-conformant (4) — code exists and does the wrong thing:

Tier	ID	Defect
1	EXIT_001	Exit codes collapse to 0/1; 2 required, 3/4/5 reserved
1	OUT_003	-o json --stream pretty-prints instead of emitting JSONL
1	OUT_004	Failures print Error: %v; no envelope, zero A2ACLI_ERR_* codes in the tree
1	DEFAULT_001	4 of 6 baseline defaults shipped; blocked by SEND_002 + VER_001

Not built (13) — feature absent:

Tier	ID	Missing
1	TASK_POLL_001	task get --wait
1	TASK_POLL_002	--poll-interval / wait --timeout, bounded backoff
1	VER_001	--a2a-version; A2A-Version never signalled → servers assume legacy 0.3
2	CARD_GET_002	card get --validate
2	TASK_SUBSCRIBE_002	Stream resumption after disconnect
2	DOWNLOAD_001	task download — a file artifact can be named but never retrieved
2	OUT_007	--debug with raw protocol logging
2	CONFORM_001	conformance TCK smoke-check
3	CARD_GET_004	Agent Card signature verification
3	CARD_GET_005	Catalog / registry resolution (tracked in #19)
3	PUSH_002	Local webhook receiver — push-config registers a URL the CLI cannot host
3	CHAT_001	Interactive chat
3	VER_002	Declare server-required protocol extensions

4.4 Registry drift — COMPLIANCE.md lags SPEC.md

Three requirement rows describe a command surface SPEC.md has since changed. SPEC.md is authoritative for behavior (COMPLIANCE.md, About this document), and in each case the implementation already tracks it — so these are stale registry entries, not code defects and not genuine contradictions.

They matter for one practical reason: a row that names a flag which no longer exists reads as a failure the tool does not have. Anyone grading against the registry verbatim would mark SEND_006 failed for the wrong reason.

#	Registry row	Says	SPEC.md says	Code follows
1	A2ACLI_SEND_006 (L143)	--text / --file / --data	§10.2: --text-part / --file-part / --data-part	SPEC ✅
2	A2ACLI_CLI_001 (L173)	"-v/--version prints the tool version"	§7.2: -v, --verbose; --version takes no short form	SPEC ✅
3	A2ACLI_DEMO_SERVER_001 (L224)	demo-server	§7.1 table: server	SPEC ✅

Fix: resync the three rows to SPEC.md. Renumbering and rewording are still permitted while the specification is pre-Proposed (COMPLIANCE.md, Requirement identifiers); from the first Proposed version onward they freeze, so this is worth clearing before then. Row 3 keeps its DEMO_SERVER identifier regardless — identifiers are never reused, and tier membership is not encoded in them.

One stale sentence inside SPEC.md itself. §7.1's Naming paragraph still lists demo-server among the bare-verb commands while its own table two lines above says server. That is the source the registry row was written from, so fix it in the same pass.

One genuine specification gap (not drift — neither document is stale, the question is simply unanswered): §6.5 mandates wait-by-default, but A2A permits a direct Message reply that creates no task. SPEC.md should state that the wait is a no-op in that case, so implementers do not poll a nonexistent taskId.

Keeping it from recurring. All three rows drifted because SPEC.md was edited without a matching registry pass. Resyncing COMPLIANCE.md in the same change that touches SPEC.md — and, once the tool matures, generating the report from the registry rather than by hand — would close the loop. Specific concerns about in-flight spec edits belong on the PR making them, not here.

5. Effect of open PR #23

PR #23 feat: cli spec reconciliation part 2 (branch yarolegovich/cli-spec-reconcile-2) is unmerged but already improves five in-scope rows — three of them to a full pass, verified from its diff:

ID	main	With #23	Change
OUT_003	❌	✅	Compact encoder per event + --pretty opt-out
OUT_002	◐	✅	PrintSendResult wraps in StreamResponse
INTERACT_004	◐	✅	formatResumeHint() appended
VER_001	❌	◐	--a2a-version added, gates 0.3 vs 1.0 transports
DEFAULT_001	❌	❌	Version row fixed; wait-by-default still missing

It also fixes two §7.2 flag divergences (--polling-interval → --poll-interval at 2s; --svc-param accepts :) — note the rename ships no alias, so existing scripts using --polling-interval will break.

It amends SPEC.md too: adds --pretty to §7.2 and §11.3, and removes three normative clauses from §13.2 — the 1.0 negotiation floor, the "versions earlier than 1.0 MUST require an explicit opt-in" rule, and the independent MUST to "surface a version-unsupported error clearly rather than silently downgrading". Raised on the PR itself rather than argued here.

Projected after merge: 27 ✅ / 11 ◐ / 15 ❌ — 51% pass. Nothing in #23 touches exit codes, the error envelope, the wait engine, or polling.go, so every remaining item in §4.2 and §4.3 survives it.

6. Method and reproduction

Verified by execution: go build ./... (exit 0) and go test ./... — 43 test functions and 127 subtests, 170 passing test cases, 0 failing, across 2,630 lines of test code. Plus live runs of the built binary against the CLI's own a2a server --echo: exit codes, error shape, JSONL framing, --async, and config show redaction were observed, not inferred.

Test coverage is narrower than "all green" suggests. The module has 9 packages; 4 carry no test file at all — internal/output, internal/utils, internal/testutil, and the root main. internal/output is the notable one: it implements the entire OUT_* surface, including two requirements graded pass here, with no test of its own.

Verified by code reading only: requirements needing an agent this repo cannot produce — INPUT_REQUIRED/AUTH_REQUIRED interruption, mid-stream failure, a taskless Message reply, a signed card.

git clone https://github.com/a2aproject/a2a-cli && cd a2a-cli
git checkout b741dc33b0506008e438e11dbd3c602e3abe1a92
go build ./... && go test ./...
go build -o /tmp/a2a .
/tmp/a2a server --echo --port 18923 --quiet &

$ /tmp/a2a send --bogus-flag x >/dev/null 2>&1; echo $?
1                                     # §11.6 requires 2

$ /tmp/a2a card get https://nonexistent.invalid.example -o json
Error: failed to resolve agent card: …    # §11.4 requires the Appendix B envelope

$ /tmp/a2a send -a 127.0.0.1:18923 -o json --stream "hello"
{
  "task": {                           # §11.3 requires one object per physical line
…

A green test suite is not evidence of conformance here. TestSendStreaming decodes with json.NewDecoder, which accepts multi-line JSON — so the OUT_003 violation passes CI today. Recommend a spec_compliance_test.go keyed by requirement ID so COMPLIANCE.md rows map 1:1 to test names and this report becomes generated rather than hand-graded.

---

Appendix A — Machine-readable detail

Per-requirement evidence, file:line references and remediation notes. Schema: COMPLIANCE.md Appendix A.4; status vocabulary per A.2.

compliance-partial.yaml — the 12 partial rows

# a2a-cli — PARTIAL requirements against SPEC.md v0.2
#
# Schema: COMPLIANCE.md Appendix A.4. Status vocabulary (A.2):
#   pass | partial | fail | not_measured | na | withdrawn
# Every status other than `pass` carries a note (A.2).
#
# Scope: EXCLUDES Authentication (A2ACLI_AUTH_*) and Agent Skills (A2ACLI_SKILL_*).
# Exclusion is by requirement AREA, so A2ACLI_CARD_GET_003 remains in scope.
#
# This file lists ONLY the 12 `partial` rows. Companion: compliance-failed.yaml.
# Grading is a status snapshot from a code read plus live runs against the CLI's
# own echo server -- NOT a conformance claim (no TCK-conformant agent exercised).

report:
  tool: a2a-cli
  tool_version: dev
  language: Go
  repository: https://github.com/a2aproject/a2a-cli
  commit: b741dc33b0506008e438e11dbd3c602e3abe1a92
  report_date: 2026-09-01
  spec_version: "0.2"
  spec_revision: 2026-08-17
  a2a_versions: ["1.0"]
  agent_tested: "local `a2a server --echo` only; no third-party agent"
  agent_tck_conformant: "no"
  transports_covered: [HTTP+JSON]
  tier_claimed: none
  excluded_areas: [AUTH, SKILL]
  subset: partial

summary:
  in_scope_total: 53
  tier_satisfied: {tier_1: false, tier_2: false, tier_3: false}
  partial_total: 12
  by_tier:
    tier_1: 10
    tier_2: 1
    tier_3: 1

requirements:

  A2ACLI_SEND_002:
    tier: 1
    area: SEND
    spec: "§10.2, §6.5"
    requirement: "Blocking by default; --async returns identifiers immediately"
    status: partial
    note: >-
      --async is implemented and sets Config.ReturnImmediately. The blocking-wait
      default it opts out of does not exist: the non-stream path issues a single
      SendMessage with a nil Config and never polls to a terminal or interrupted
      state. No --wait, and no --return-immediately / --no-wait aliases.
    evidence: "internal/cli/send.go:95-105 (one-shot path), :144-152 (buildRequest)"
    remediation: "Route the default through the shared waiter added for TASK_POLL_001."

  A2ACLI_SEND_004:
    tier: 1
    area: SEND
    spec: "§10.2"
    requirement: "Renders produced artifacts; never silently discards a part"
    status: partial
    note: >-
      partsText renders text, data, file and raw parts, so nothing is silently
      dropped. But a file part is rendered as "[file: <url>]" or
      "[binary N bytes]" rather than reporting name, media type and size.
    evidence: "internal/output/output.go:293-324 partsText; :241-250 artifact loop"
    remediation: "Emit Filename, MediaType and byte length for file parts."

  A2ACLI_SEND_006:
    tier: 1
    area: SEND
    spec: "§10.2"
    requirement: "Part flags repeatable and order-preserving; --data-part - reads stdin"
    status: partial
    note: >-
      Ordering and repeatability are correct and covered by TestPartsParse.
      '--data-part -' does not read stdin: RawOrInline falls through to treating
      a non-file value as an inline literal, so '-' becomes the string "-".
    evidence: "internal/flagparse/parts.go:45-77 (ordering); :148-156 RawOrInline"
    remediation: "Special-case '-' to io.ReadAll(os.Stdin) before the file probe."

  A2ACLI_INTERACT_002:
    tier: 1
    area: INTERACT
    spec: "§8.1"
    requirement: >-
      Continue a task via --task-id, with or without --context-id; a rejected
      identifier fails non-zero, creates no new task, and points at --debug
    status: partial
    note: >-
      The identifier is sent against the task and a server rejection surfaces as
      a non-zero exit with no phantom task created. The tool cannot satisfy the
      final clause -- it has no --debug flag to point the caller at.
    evidence: "internal/cli/send.go:194-196"
    remediation: "Blocked on OUT_007 (--debug); then add the pointer to the error hint."

  A2ACLI_INTERACT_004:
    tier: 1
    area: INTERACT
    spec: "§8.2"
    requirement: >-
      Reports taskId / contextId / state on completion and interruption, in
      copy-pasteable form, and prints the exact command to resume
    status: partial
    note: >-
      Identifiers and state are printed in a stable, copy-pasteable form. No
      resume command is emitted, and there is no stop-and-report behaviour on an
      interrupted state.
    evidence: "internal/output/output.go:226-239 formatTask"
    remediation: "Resolved by PR #23, which adds formatResumeHint()."
    fixed_in_pr: 23

  A2ACLI_TASK_POLL_003:
    tier: 1
    area: TASK_POLL
    spec: "§9.1, §9.3"
    requirement: >-
      Stops immediately on interrupted states (INPUT_REQUIRED / AUTH_REQUIRED);
      treats TASK_STATE_UNSPECIFIED as neither terminal nor interrupted
    status: partial
    note: >-
      The loop tests `Terminal() || InputRequired` in both the guard and the
      event branch. AUTH_REQUIRED is missing, so a task pausing for
      authentication is polled until the deadline instead of returning to the
      caller. UNSPECIFIED handling is incidentally correct but untested.
    evidence: "internal/polling/polling.go:62 (loop guard), :82 (event branch)"
    remediation: "Add TaskStateAuthRequired to both conditions; add the sibling test."

  A2ACLI_TASK_POLL_004:
    tier: 1
    area: TASK_POLL
    spec: "§9.3"
    requirement: "When a wait prefers streaming, falls back to polling on stream failure"
    status: partial
    note: >-
      Fallback triggers only on a2a.ErrUnsupportedOperation (capability absent).
      A mid-stream transport failure aborts the run instead of degrading to
      polling, which §9.3 requires. Also inherits the AUTH_REQUIRED gap above.
    evidence: "internal/cli/send.go:73-93 (fallback), :80-82 (error class check)"
    remediation: "Broaden the fallback predicate to any stream-establishment or mid-stream error."

  A2ACLI_OUT_002:
    tier: 1
    area: OUT
    spec: "§11.3, App. B"
    requirement: >-
      -o json without --stream emits exactly one document, the terminal protocol
      object, never implicitly switched to its streamed JSONL form
    status: partial
    note: >-
      Cardinality is correct -- exactly one document. The shape is not:
      PrintSendResult prints the oneof MEMBER (a bare Task) rather than the
      SendMessageResponse wrapper Appendix B specifies. The same command under
      --stream DOES wrap, so one command emits two incompatible top-level shapes
      and a consumer must sniff for `id` vs `messageId`.
    evidence: "internal/output/output.go:117-131 PrintSendResult"
    remediation: "Resolved by PR #23, which wraps in a2a.StreamResponse{Event: result}."
    fixed_in_pr: 23

  A2ACLI_OUT_005:
    tier: 1
    area: OUT
    spec: "§11.2, §6.1"
    requirement: >-
      text floor -- one Label: value per line, stable labels, no control
      sequences; block content under its own Label: line closed by a blank line
    status: partial
    note: >-
      Labels are stable across invocations and no terminal control sequences are
      emitted. Block content is not implemented: rendered artifacts and history
      appear as indented lines beneath a heading rather than as §11.2 blocks
      closed by a blank line, so a reader cannot skip a block whole.
    evidence: "internal/output/output.go:241-261 (artifact and history loops)"
    remediation: "Emit each artifact under its own label line, terminated by a blank line."

  A2ACLI_EXIT_002:
    tier: 1
    area: EXIT
    spec: "§6.6, §11.6"
    requirement: >-
      Tool execution and agent outcome stay decoupled; a non-success or paused
      outcome SHOULD be named in a stderr warning
    status: partial
    note: >-
      The decoupling holds -- a conducted and reported turn exits 0 regardless of
      the agent's verdict, verified against the local echo server. The stderr
      warning naming a FAILED / REJECTED / INPUT_REQUIRED outcome is absent; a
      tier claim promotes that SHOULD to required.
    evidence: "internal/cli/root.go:71-82; internal/cli/send.go:102-105"
    remediation: "Warn on stderr when the terminal state is not COMPLETED."

  A2ACLI_VER_003:
    tier: 2
    area: VER
    spec: "§13.3"
    requirement: >-
      Verifies a capability on the Agent Card before invoking a capability-gated
      operation (streaming, push notifications, extended card)
    status: partial
    note: >-
      Done for streaming: handleStreaming checks card.Capabilities.Streaming and
      returns ErrUnsupportedOperation. No equivalent guard before push-config
      operations or before fetching the extended card.
    evidence: "internal/cli/send.go:157-159"
    remediation: "Apply the same guard in task_push_config_*.go and card_get.go."

  A2ACLI_CARD_GET_003:
    tier: 3
    area: CARD_GET
    spec: "§10.1, §12.4"
    requirement: >-
      Authenticated extended Agent Card, fetched only via a security scheme
      advertised on the public card
    status: partial
    note: >-
      --extended fetches the extended card via client.GetExtendedAgentCard and
      renders it. The qualifier is unmet: the public card's declared
      securitySchemes are never consulted before a credential is presented.
      Retained in scope because its area is CARD_GET, though the unmet half is
      authentication-related.
    evidence: "internal/cli/card_get.go:64 (flag), :68-81 getExtendedAgentCard"
    remediation: "Read securitySchemes from the public card and gate the fetch on a match."

compliance-failed.yaml — the 17 failed rows

# a2a-cli — FAILED / MISSING requirements against SPEC.md v0.2
#
# Schema: COMPLIANCE.md Appendix A.4. Status vocabulary (A.2):
#   pass | partial | fail | not_measured | na | withdrawn
# Every status other than `pass` carries a note (A.2).
#
# Scope: EXCLUDES Authentication (A2ACLI_AUTH_*) and Agent Skills (A2ACLI_SKILL_*).
# Exclusion is by requirement AREA.
#
# This file lists ONLY the 18 `fail` rows. Companion: compliance-partial.yaml.
# `kind` is a local annotation, not part of the COMPLIANCE.md schema:
#   non_conformant -> code exists and behaves against the requirement
#   not_built      -> the feature is absent

report:
  tool: a2a-cli
  tool_version: dev
  language: Go
  repository: https://github.com/a2aproject/a2a-cli
  commit: b741dc33b0506008e438e11dbd3c602e3abe1a92
  report_date: 2026-09-01
  spec_version: "0.2"
  spec_revision: 2026-08-17
  a2a_versions: ["1.0"]
  agent_tested: "local `a2a server --echo` only; no third-party agent"
  agent_tck_conformant: "no"
  transports_covered: [HTTP+JSON]
  tier_claimed: none
  excluded_areas: [AUTH, SKILL]
  subset: failed

summary:
  in_scope_total: 53
  tier_satisfied: {tier_1: false, tier_2: false, tier_3: false}
  failed_total: 17
  by_kind:
    non_conformant: 4
    not_built: 13
  by_tier:
    tier_1: 7
    tier_2: 5
    tier_3: 5

requirements:

  # ─────────── Built but non-conformant (5) ───────────

  A2ACLI_EXIT_001:
    tier: 1
    area: EXIT
    spec: "§11.6, App. D"
    kind: non_conformant
    requirement: "Implements exit statuses 0, 1 and 2; reserved statuses carry documented meanings"
    status: fail
    note: >-
      Execute() returns only 0 or 1. An unknown flag exits 1 where §11.6 requires
      2; an unresolvable card exits 1 where 3 is reserved. Verified by running the
      built binary. A caller cannot distinguish failure classes without parsing
      stderr prose -- the exact outcome §11.6 exists to prevent.
    evidence: "internal/cli/root.go:71-82"
    remediation: "Return a typed error carrying an exit status; map per App. D."

  A2ACLI_OUT_004:
    tier: 1
    area: OUT
    spec: "§11.4, App. B, App. D"
    kind: non_conformant
    requirement: >-
      Errors are machine-readable and consistent across transports; protocol
      failures carry the A2A error name, CLI-local failures an A2ACLI_ERR_* code
    status: fail
    note: >-
      Every failure renders as "Error: %v" on stderr in both -o text and -o json.
      No error envelope, no hint field, and the string A2ACLI_ERR_ appears zero
      times in the source tree.
    evidence: "internal/cli/root.go:77-80"
    remediation: >-
      Add internal/clierr with the eight App. D codes; classify at call sites;
      render {error:{code,message,hint,a2aCode}} on stdout under -o json.

  A2ACLI_OUT_003:
    tier: 1
    area: OUT
    spec: "§11.3, §11.4, App. B"
    kind: non_conformant
    requirement: >-
      -o json --stream emits JSONL: one complete JSON object per line, flushed as
      produced, final line carrying the terminal object
    status: fail
    note: >-
      PrintJSON calls enc.SetIndent("", "  ") unconditionally, so each streamed
      event spans roughly 20 physical lines. Verified by running the binary
      against the local echo server. A stream-terminating error is also not
      emitted as a final error line.
    evidence: "internal/output/output.go:60-65 PrintJSON, via PrintEvent :86"
    remediation: "Resolved by PR #23: compact encoder per event plus a --pretty opt-out."
    fixed_in_pr: 23

  A2ACLI_DEFAULT_001:
    tier: 1
    area: DEFAULT
    spec: "§6.5"
    kind: non_conformant
    requirement: "Ships the baseline defaults, each overridable, and exposes them via --help"
    status: fail
    note: >-
      Four of six rows ship: transport server-preference order, text output,
      concise detail, TLS verification on (gRPC only). Missing: wait-on-completion
      (SEND_002) and protocol version (VER_001). App. A.3 rule 3 forbids a pass
      while any breakdown row is not shipped-and-overridable.
    evidence: "internal/cli/root.go:114-124"
    remediation: "Unblocked by SEND_002 and VER_001."

  # ─────────── Not built (13) ───────────

  A2ACLI_TASK_POLL_001:
    tier: 1
    area: TASK_POLL
    spec: "§9.3"
    kind: not_built
    requirement: "Polling path available -- one-shot task get, plus task get --wait"
    status: fail
    note: >-
      task get registers only --history. §7.1 states polling is not a separate
      command but is task get --wait. The polling engine exists yet is reachable
      only as a send --stream fallback, so a job started with --async cannot be
      waited on at all.
    evidence: "internal/cli/task_get.go:64; internal/polling/polling.go:32 (used only at send.go:86)"
    remediation: "Expose the shared waiter as task get --wait."

  A2ACLI_TASK_POLL_002:
    tier: 1
    area: TASK_POLL
    spec: "§9.3"
    kind: not_built
    requirement: "--poll-interval and --timeout honored; bounded backoff; no busy-loop; interruptible"
    status: fail
    note: >-
      No --poll-interval: the flag is --polling-interval, send-local, defaulting
      to 5s against the RECOMMENDED 2s. The loop uses a fixed time.After with no
      backoff. The global --timeout is a per-request deadline, not a wait
      deadline, and no A2ACLI_ERR_TIMEOUT is reported on expiry.
    evidence: "internal/cli/send.go:116; internal/polling/polling.go:63-68"
    remediation: "Add both flags to the shared waiter with exponential backoff and a wait deadline."

  A2ACLI_VER_001:
    tier: 1
    area: VER
    spec: "§13.2"
    kind: not_built
    requirement: >-
      A2A-Version signaled explicitly on every request; negotiates only within
      1.x, never below 1.0; no silent downgrade
    status: fail
    note: >-
      No --a2a-version flag and no version service parameter set by the CLI. A2A
      §3.6.2 makes an empty value mean 0.3, so every request is silently treated
      as legacy by a conformant server. The only occurrence of the name in the
      tree is an env-mapping test for a flag that does not exist.
    evidence: "internal/clicfg/binder_test.go:34 (only occurrence)"
    remediation: >-
      Partly resolved by PR #23, which adds --a2a-version and gates 0.3 vs 1.0
      transport registration; confirm the header is emitted on every request.
    fixed_in_pr: 23

  A2ACLI_CARD_GET_002:
    tier: 2
    area: CARD_GET
    spec: "§10.1"
    kind: not_built
    requirement: "card get --validate -- validate the Agent Card against the A2A schema"
    status: fail
    note: "Flag absent; card get registers only --extended."
    evidence: "internal/cli/card_get.go:64"

  A2ACLI_TASK_SUBSCRIBE_002:
    tier: 2
    area: TASK_SUBSCRIBE
    spec: "§9.4, §9.2"
    kind: not_built
    requirement: "Stream resumption after disconnect; the first event re-delivers the full Task"
    status: fail
    note: >-
      No reconnect loop. WithInactivityTimeout cancels the context on silence
      rather than resubscribing, so a dropped connection ends the run.
    evidence: "internal/cli/task_subscribe.go:26-58; internal/utils/timeout.go:28-60"

  A2ACLI_DOWNLOAD_001:
    tier: 2
    area: DOWNLOAD
    spec: "§7.1"
    kind: not_built
    requirement: "task download -- save task artifacts to disk"
    status: fail
    note: >-
      Not implemented. Combined with SEND_004, a file artifact can be named but
      never retrieved, which makes file-producing agents unusable end to end.

  A2ACLI_OUT_007:
    tier: 2
    area: OUT
    spec: "§7.2"
    kind: not_built
    requirement: "--debug enables diagnostic logging to stderr, including raw protocol messages"
    status: fail
    note: >-
      No --debug flag. --verbose emits a handful of '#'-prefixed lines, not wire
      traffic. Also blocks INTERACT_002, which must point callers at --debug.
    evidence: "internal/cli/root.go:64-68 (logf), :122 (--verbose)"

  A2ACLI_CONFORM_001:
    tier: 2
    area: CONFORM
    spec: "§7.1"
    kind: not_built
    requirement: "conformance -- smoke-check a live agent against the A2A TCK"
    status: fail
    note: "Not implemented; no conformance command in the tree."

  A2ACLI_CARD_GET_004:
    tier: 3
    area: CARD_GET
    spec: "§10.1"
    kind: not_built
    requirement: >-
      Agent Card signature verification per A2A §8.4.3; reports verified /
      unverifiable / absent and never presents an unverified card as verified
    status: fail
    note: "Not implemented. No JWS or RFC 8785 canonicalization path exists."

  A2ACLI_CARD_GET_005:
    tier: 3
    area: CARD_GET
    spec: "§10.1"
    kind: not_built
    requirement: "Catalog / registry resolution wherever --agent-card takes a reference"
    status: fail
    note: "Not implemented. ALREADY TRACKED in issue #19 -- do not re-file."
    tracked_in_issue: 19

  A2ACLI_PUSH_002:
    tier: 3
    area: PUSH
    spec: "§9.2"
    kind: not_built
    requirement: "Local webhook receiver able to accept push notifications"
    status: fail
    note: >-
      Not implemented. task push-config can register a callback URL the CLI
      itself cannot host, so the loop cannot be closed locally.

  A2ACLI_CHAT_001:
    tier: 3
    area: CHAT
    spec: "§8.1, §7.1"
    kind: not_built
    requirement: "Interactive chat carrying contextId and active taskId across turns"
    status: fail
    note: >-
      Not implemented. Note §6.1 and §11.2 require any such mode to be gated by
      terminal detection and to auto-degrade to text when stdout is not a TTY.

  A2ACLI_VER_002:
    tier: 3
    area: VER
    spec: "§13.3, A2A §4.6"
    kind: not_built
    requirement: "Declares server-required protocol extensions the tool supports"
    status: fail
    note: "Not implemented; the CLI cannot talk to an agent that requires an extension."

# Required by COMPLIANCE.md A.3 rule 3: DEFAULT_001 cannot pass
# unless every row below is shipped: yes AND overridable: yes.
default_001_breakdown:
  transport_server_preference_order: {shipped: yes, overridable: yes}
  task_completion_wait_by_default:   {shipped: no,  overridable: no, note: "SEND_002"}
  output_human_readable_text:    {shipped: yes, overridable: yes}
  detail_level_concise:      {shipped: yes, overridable: yes}
  protocol_version_highest_mutual:   {shipped: no,  overridable: no, note: "VER_001"}
  transport_security_tls_on:     {shipped: yes, overridable: yes, note: "--insecure covers gRPC only"}

[yarolegovich]

This is not the way to test conformance. Please provide a list of commands that were run, what were the expectations and what were the observed outputs. Then a discussion is possible whether a cli needs to be fixed or a requirement relaxed. Thanks.

[yarolegovich]

Verified by code reading only: requirements needing an agent this repo cannot produce — INPUT_REQUIRED/AUTH_REQUIRED interruption, mid-stream failure, a taskless Message reply, a signed card.

ok, so no commands were run

[msampathkumar]

Thank you for prompt review @yarolegovich 👍

Update: Authentication & Success tests are made out of scope for this report.

[msampathkumar]

Based on the repos I have raised a list of feature request. A couple of these feature are WIP in PR#23 and they are tagged accordingly.

PR	Priority	Description
#28	P0	Distinct exit codes (0/1/2) so callers can tell a bad invocation from a real failure.
#29	P0	Machine-readable JSON error output under -o json.
#30	P1	task get --wait to poll an async task to completion.
#31	P1	--bearer/--api-key credential flags with A2ACLI_* env vars.
#32	P2	--a2a-version to select the protocol version — implemented in PR #23.
#33	P2	task download to save task artifacts to disk.
#34	P2	Interactive chat for multi-turn agent sessions.
#35	P2	Print the exact resume command when a task pauses — implemented in PR #23.
#36	P1	Fix: a blocking wait ignores AUTH_REQUIRED and hangs until timeout.