Generate the SP-API SDK with oagen (standalone client, models, resources), Amazon plugin, benchmarks

[dbritto-dev]

Summary

Replaces the hand-written requests client with an SDK generated from Amazon's API models by an oagen emitter (codegen/). oagen is a framework for building SDK generators, not a generator: it parses the spec into a typed IR and the Python emitter in codegen/src/python/ (TypeScript) turns that IR into files. The emitter is built after the reference prompt for an oagen Python emitter (five modules, the verified 0.30.2 ground truth, vitest over an inline fixture and an end-to-end test of the output as the definition of done), the repository is laid out like workos/openapi-spec (committed spec, src/policy/, thin oagen.config.ts, sdk:* scripts that are plain oagen invocations), and src/amzn_selling_partner/sdk/ is the generated, standalone SDK (client, HTTP layer on httpx2 with the retry/throttling policy, errors, pydantic models, resources), committed so installing the package pulls in no generator. Everything Amazon-specific that is not in the specs (regions, LWA auth with Restricted Data Tokens and grantless scopes, document helpers, notification models) is a plugin on top (plugins.amazon_spapi). The 0.1.x entry points (client, reports, vendor, utils) keep working as wrappers.

Design notes and decisions are in docs/PLAN.md; the breaking changes are in MIGRATION.md; docs/UPDATING_SPECS.md describes a spec bump and the generator; codegen/README.md lists the generator commands and the oagen API discrepancies met on the way. The docs lead with the async client and use the SDK's enums and models in every example.

Review decisions applied: Python 3.10+; the package name stays amzn_selling_partner; pydantic v2 for schemas/validation/parsing; build-time generation; pytest-benchmark instead of a hand-written harness, no nightly job; oagen's resolved method names with operationHints for collisions; sync and async clients with the same surface (with_options(); http_client= injects any httpx2 client with its own defaults honoured; the SDK's own clients come from a factory method whose public products, DefaultHttpxClient, DefaultAsyncHttpxClient and DefaultAioHttpClient, are what a caller configures and injects, httpx2 being the default and aiohttp an explicit choice), auto-pagination helpers, header parameters and non-JSON bodies, py.typed; one models package per API version; the security checks job (bandit + safety) kept as on main; no wrapper scripts, testing lives in the test suites; ruff is the only linter and formatter and ty the only type checker; releases bump minor unless the branch commits the component to .github/release-bump; the HTTP layer delegates to httpx2 (request building, base URL, encoding, pooling, connection retries) rather than rebuilding it.

What is in the branch

• codegen/spec/open-api-spec.yaml – the one OpenAPI 3 document every oagen command runs against, committed like the example's spec/open-api-spec.yaml. npm run spec:build writes it from Amazon's 67 Swagger 2.0 files and 23 notification schemas (swagger2openapi, #ref/dangling-reference repairs, components namespaced per API version, operations tagged with their service, sample AWS key IDs in Amazon's example URLs redacted because GitHub's push protection rejects them). spec/petstore.yaml is the same for the test fixtures. CI fails on drift.
• codegen/src/policy/ – the resolution policy behind a barrel: operation-hints.ts (the 29 colliding derived names), mount-rules.ts (the pricing v0 split), transforms.ts (transformSpec: alias inlining, inline-object hoisting, name protection against oagen's name cleaner; schemaNameTransform; operationIdTransform). oagen.config.ts is the thin shim that spreads the emitter plugin, the policy and emitterOptions.python (the sdkBehavior overrides). test/policy.test.ts checks every hint names an operation and every mount rule a service of the committed spec.
• codegen/src/python/ – the emitter, as the reference prescribes: types.ts (TypeRef → annotation, exhaustive with assertNever), models.ts (models + enums: every model of a package in models/<pkg>/__init__.py, required fields first, X | None = None, snake_case attributes with wire aliases; enums in models/<pkg>/enums.py with __str__ = str.__str__), resources.ts (<Service>Resource / Async<Service>Resource, one method per resolved operation; path params, body and required params positional, optional ones keyword-only; params/headers built explicitly, then self._http.request(...); iter_<method> for the 69 paginated operations; SERVICES/OPERATIONS registry), client.ts (client.py with Client/AsyncClient, one lazily created resource per service, latest-version aliases and with_options(); __init__.py; errors.py from the error policy; _http.py built on httpx2: the httpx2 client comes from a factory method (_create_client, one product per sync/async subclass) whose products are the public factories DefaultHttpxClient, DefaultAsyncHttpxClient (httpx2's transport, connection retries from the policy, transport options build the transport) and DefaultAioHttpClient (the aiohttp transport, whose httpx-targeted implementation is driven through an adapter to httpx2's transport interface); SDK-created clients carry the base_url and every request goes through build_request, so joining, URL/query/header encoding and merging are httpx2's, and a user-supplied http_client keeps its own defaults; the module adds only the status-code retries with backoff and Retry-After from the SDK behavior in the IR, per-operation token buckets, the Auth hook, decoding through cached TypeAdapters and paginate/apaginate), index.ts (assembles the Emitter; its formatCommand runs ruff over every file oagen writes, so generation has no formatting step of its own); src/plugin.ts exports { emitters, extractors, smokeRunners }. npm scripts: spec:build, sdk:parse, sdk:resolve, sdk:generate (into a clean output dir), generate, sdk:diff, test, typecheck, build.
• Definition of done, verified – npm run generate produces both SDKs from clean output directories and a second run changes nothing; npm test (37 vitest tests: inline fixture spec written to a temp file for field ordering, nullable rendering, enum emission, header params and non-JSON bodies; the tutorial's tasks-api.yml; spec build, policy and helpers) and tsc --noEmit are clean; tests/test_sdk_end_to_end.py runs the generated Amazon SDK over httpx2.MockTransport, sync and async (a list with an enum query parameter whose value, not its name, reaches the wire, a create with a body model, a 404), tests/petstore_sdk covers every operation shape and a Hypothesis fuzz test encodes parameters through real requests; no generated file is hand-edited (CI regenerates and fails on drift).
• Amazon plugin – SellingPartner(Client) / AsyncSellingPartner: regional servers + Marketplace, LWA auth (refresh-token and client_credentials grants, single-flight cache, pluggable store, RDTs via the Tokens API looked up by operationId, with_rdt() opt-in), document download/upload helpers, notification model registry, sandbox example helpers.
• sandbox_tests.py – runs every operation through both clients over httpx2.MockTransport using the x-amzn-api-sandbox examples read from the raw model files.
• benchmarks/ – pytest-benchmark suite over httpx2.MockTransport asserting the generated method stays within 1.15× of an equivalent hand-written httpx2 call (both build the request with httpx2 on a client with base_url; the margin is the request context, throttle lookup, retry bookkeeping and typed error mapping the generated path adds); runs in CI on every push.
• CI – a codegen job (Node 24, oagen's declared engine: npm ci --ignore-scripts, npm run build, npm run typecheck, npm test, npm run generate, git diff --exit-code over the spec and the generated code), the Python matrix 3.10–3.13 (ruff, ty check over hand-written and generated code, pytest, benchmarks, wheel content check) and the security job from main (nox -s security_test: bandit over the package, generated code included, plus safety over the dependencies, Python 3.10–3.13). bandit's false positives in generated code (the pagination token_param= keyword, enum members named like credentials, the retry jitter) are marked with targeted # nosec comments by the emitter, so real findings still fail the job.
• Release – every push to main bumps the minor version (uv version --bump minor --no-sync) unless the branch committed major, minor or patch to .github/release-bump; the release commit removes the file, so the next push is back to minor. This branch commits major, so merging it releases the next major however the merge is done (merge commit, squash or rebase).

Numbers (this container)

Tests	120 pytest, 37 vitest, ty check clean (generated code included), ruff clean, bandit clean
Generated method vs equivalent hand-written httpx2 call (best of rounds)	sync 1.07–1.10×, async 1.10–1.11× (limit 1.15×)
Sandbox examples, all 55 APIs	1964 passed / 70 failed of 2034 cases; every failure is an Amazon example that violates its own schema; orders and listings_items are clean and run in CI
Generated files	233 (Amazon) + 13 (petstore fixtures), identical on a second npm run generate

Decisions to confirm (docs/PLAN.md)

1. Package and distribution names unchanged (amzn-selling-partner / amzn_selling_partner); Python 3.10+ (so __str__ = str.__str__ rather than StrEnum).
2. Method names are oagen's resolved names (list_orders, get_order); sdk.resources.OPERATIONS maps Amazon's operationIds to them. Collisions are settled in src/policy/operation-hints.ts.
3. LWA-only auth; boto3 / SigV4 dropped (compat shims warn or raise).
4. One resource and one models package per API version; enums are str enums; generated code and the merged spec committed, CI fails on drift.
5. The RDT table (plugins/_amazon/rdt.py) was written without access to developer-docs.amazon.com; please verify against the Tokens API use-case guide.
6. The benchmark limit moved from 1.10× to 1.15× when the generated path and the baseline were put on the same httpx2 build_request path; keeping 1.10× would mean a fast path that bypasses build_request for SDK-created clients.

Packaging note

The wheel ships Python only (plus py.typed and oagen's .oagen-manifest.json, which lets oagen generate prune stale files); the spec submodule and codegen/spec/ are generator inputs. codegen/ needs Node 24 and npm ci --ignore-scripts (oagen depends on tree-sitter grammars for its compat extractors, which the generator never imports).

🤖 Generated with Claude Code

https://claude.ai/code/session_01RDH3P3vmiBCWwaDYnvpWHn

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.