TESTING.md

Testing

This SDK ships a test suite that runs against the E2E sandbox (api.sandbox.e2e.gr4vy.app). It covers the full merchant transaction lifecycle and the back-office configuration surface, and reaches every API operation.

Note for contributors: this repository is generated by Speakeasy. Everything under tests/, scripts/, and this file is not regenerated and is safe to edit. The test-only utils package is made importable from every shard directory by tests/conftest.py (which adds tests/ to sys.path), so this does not rely on pyproject.toml pythonpath — which Speakeasy regenerates.

Prerequisites

• Python 3.10+ and Poetry.
• The signing key for the E2E sandbox. Provide it either way:
  • Export it: export PRIVATE_KEY="$(cat /path/to/private_key.pem)", or
  • Save it as private_key.pem in the repo root (git-ignored).

tests/utils/environment.py reads PRIVATE_KEY first, then falls back to private_key.pem.

Running the tests

poetry install

# Everything (offline unit tests + live E2E)
poetry run pytest tests

# A single shard (just pass the directory — pytest runs that path)
poetry run pytest tests/flows
poetry run pytest tests/processing
poetry run pytest tests/backoffice
poetry run pytest tests/test_auth.py tests/test_webhooks.py   # offline

# A single test module or test
poetry run pytest tests/flows/test_transaction_lifecycle.py
poetry run pytest tests/flows/test_transaction_lifecycle.py::test_authorize_capture_refund

How it is organised

tests/
  conftest.py             # `merchant` fixture (one isolated merchant per module) + sys.path bootstrap
  utils/
    environment.py        # merchant-isolation harness + client builder (PRIVATE_KEY / private_key.pem)
    client.py             # JsonInterceptorClient: forward-compat injector + HTTP-call recorder
    merchant.py           # TestMerchant: merchant-scoped client + identifiers
    fixtures.py           # mock-card inputs, addresses, cart items, unique ids
    checkout_fields.py    # raw PUT /checkout/sessions/{id}/fields + authorize() helper
    poll.py               # bounded polling for eventually-consistent reads
    reach.py              # "endpoint reached + cleanly rejected (4xx)" assertion; fails on 5xx
    gen.py                # seeded generators for the property-based tests
  flows/                  # headline narratives (transaction + buyer lifecycles)
  processing/             # per-resource CRUD + actions (transactions, refunds, payment methods,
                          #   buyers, gift cards, digital wallets, payment links, checkout sessions)
                          #   + test_property.py (bounded "uncertainty" testing)
  backoffice/             # merchant accounts, payment services, 3DS scenarios, reports, payouts,
                          #   audit logs, account updater
  test_auth.py            # offline JWT tests
  test_webhooks.py        # offline signature tests

Isolation & parallelism

Each E2E test module provisions its own merchant account (random id) with a mock-card payment service via the module-scoped merchant fixture, so modules share no state and the suite is safe to shard. CI shards by directory — each job runs one of tests/flows, tests/processing, tests/backoffice (and the offline pair). The shard map lives in .github/workflows/ci.yaml, which Speakeasy does not regenerate, so regeneration can't break it.

The mock-card connector ("test mode")

The deterministic test connector approves card 4111 1111 1111 1111 with a future expiry and CVV 123 (see fixtures.py). Operations that need real external state (live wallet provisioning, real network tokens, a payout-capable PSP, a configured 3DS or gift-card service) can't reach a 2xx in the mock env; those are exercised with reach.reaches(), which asserts the endpoint was reached and cleanly rejected with a 4xx. A 5xx fails the test — that means we sent something the API couldn't handle, which is a real defect to surface.

Forward-compatibility

The test HTTP client injects a random unknown field into every JSON object response, proving the SDK tolerates forward-compatible payloads. Disable it for debugging with GR4VY_NO_INJECT=1.

Property-based ("uncertainty") testing

tests/processing/test_property.py explores generated inputs (amounts, currency, mixed-case metadata) from a fixed seed. Override the seed with FC_SEED; the run count per property is bounded (gen.RUNS) to keep live-sandbox cost low. We use a small self-contained generator rather than an external dependency so the suite stays hermetic.

Endpoint-reach coverage

When GR4VY_TRACK_HTTP=1 is set, the test HTTP client logs the method + path of every request to coverage/http/*.jsonl. scripts/endpoint_coverage.py builds the operation catalogue from the generated src/gr4vy/*.py resource modules (each synchronous self._build_request(method=..., path=...) call) and reports how many operations were reached by a real HTTP request — newly generated (but untested) endpoints show up as "not reached". CI runs this once, un-sharded, and posts the result as a sticky PR comment. It is a report, not a merge gate.

The script reads every coverage/http/*.jsonl it finds, so clear the directory first when running locally — otherwise stale logs from a previous run inflate the reported reach. (CI starts from a clean checkout, so this only matters locally.)

rm -rf coverage/http            # drop stale logs from previous runs
GR4VY_TRACK_HTTP=1 poetry run pytest tests
poetry run python scripts/endpoint_coverage.py

CI

• lint runs pylint -j=0 tests src once on Python 3.13.
• e2e runs the three E2E shards on Python 3.13; offline runs the offline suite on Python 3.10–3.14.
• ci-complete is the single stable required check (it needs lint + every shard) and performs the one-time auto-approve of SDK-regen PRs. Point branch rulesets at ci-complete, not the per-shard matrix jobs.
• coverage posts the endpoint-reach comment and is not a merge dependency.
• merge (auto-merge of bot regen PRs) depends only on ci-complete.