parity: per-service AWS audit of all 163 services

.claude/skills/gopherstack-gates/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: gopherstack-gates
+description: Pick the narrowest correct build/test/lint command for a gopherstack change and decode golangci-lint/CI failures without trial and error. Use whenever about to run tests or lint, before pushing, when a linter fires (fieldalignment, cyclop, gocognit, exhaustive, mnd, gochecknoglobals, depguard, forbidigo, sloglint, nolintlint), when writing a test that waits on async state, or when integration tests fail mysteriously (missing build-linux). Also use to predict what CI will do before opening a PR.
+---
+
+# gopherstack gates
+
+## What changed → narrowest command
+
+| change | command |
+|---|---|
+| One service's Go code | `go test -race ./services/<svc>/...` |
+| One service's lint | `golangci-lint run ./services/<svc>/...` |
+| Wire-shape confidence | `go test -race -run TestSDKCompleteness ./services/<svc>/...` plus that service's roundtrip tests |
+| Shared type in `pkgs/` changed | add `go build ./...` to the above — catches breaks in every consumer |
+| Anything touching persistence/snapshot | run that service's `persistence_test.go` explicitly |
+| Full gate before push | `make lint` → `make test` → `make build-linux` → `make integration-test`, in that order |
+
+## Makefile targets
+
+| target | runs | prereq |
+|---|---|---|
+| `build` | `go build` with UI embed | `ui-build` |
+| `build-linux` | `CGO_ENABLED=0 GOOS=linux go build -o bin/gopherstack` | none |
+| `lint` | `golangci-lint run --timeout 20m ./...` + `go vet -vettool=$(go tool -n mulint-vet) ./...` + `go tool govulncheck ./...` | `install-deps ui-lint ui-fmt ui-check` |
+| `lint-fix` | `fieldalignment -fix ./...` then `golangci-lint run --fix ./...` | `install-deps ui-lint-fix ui-fmt-fix` |
+| `test` | `go tool gotestsum --format pkgname -- -race -shuffle on -short ./...` | none |
+| `integration-test` | `gotestsum ... -race -shuffle on -timeout 10m ./test/integration/...` | **`build-linux`** — the Docker image copies `bin/gopherstack`, so a stale/missing binary makes integration tests fail against old code with no obvious error |
+| `terraform-test` | `gotestsum -v -race -parallel 8 -timeout 45m ./test/terraform/...` | `install-tofu` |
+| `e2e-test` | `gotestsum ... -tags=e2e ./test/e2e/...` | `ui-build` |
+| `total-coverage` | unit+integration+terraform+e2e merged into coverage.out/.html | `build-linux` |
+| `docs` | `go run ./cmd/gendocs` — regenerates per-service README.md, root README table, `.badges/*.svg` | none |
+| `all` | `make lint-fix && make total-coverage` | — |
+
+If you edit code and then run `make integration-test` without `make build-linux`
+first, you are testing the OLD binary. Always rebuild first, or just run
+`make integration-test` (it depends on `build-linux` already) rather than
+invoking gotestsum directly.
+
+## golangci-lint gotchas that actually bite
+
+`.golangci.yml` is a large (~65 linter) config. These are the ones that
+routinely surprise agents:
+
+| gotcha | detail |
+|---|---|
+| `fieldalignment` | force-enabled via `settings.govet.enable: [fieldalignment]` on top of `enable-all: true`. Bites on every new struct. Fix: `fieldalignment -fix ./...` (also runs as part of `make lint-fix`), don't hand-reorder fields |
+| golines, not `lll` | max line length is **120**, enforced by the `golines` formatter, not a linter check. Run `gofmt`/the formatter rather than manually wrapping |
+| `exhaustive` | fires on both `switch` AND `map` literals over enum-like types. A `default:` case satisfies it — add one instead of enumerating every case if the switch isn't meant to be exhaustive |
+| `mnd` (magic numbers) | bare numeric literals need a named const, even small ones in non-test files |
+| `gochecknoglobals` | package-level `var` needs `//nolint:gochecknoglobals // <reason>` — see account's `operationNames` map for the accepted pattern |
+| `depguard` | bans `github.com/golang/protobuf`, `satori/go.uuid`, `gofrs/uuid` (pre-v5), `math/rand` in non-test files, `log` in non-main files |
+| `forbidigo` | bans `fmt.Print*` and ad-hoc `slog.Default()`/`slog.New()` outside `pkgs/logger` — route all logging through `pkgs/logger` |
+| `sloglint` | `no-global: all`, `context: scope` — no global loggers, pass context-scoped ones |
+| `nolintlint` | any `//nolint:` needs an explanation AND names a specific linter (`require-explanation: true`, `require-specific: true`). Only `funlen`/`gocognit`/`golines` may skip the explanation (`allow-no-explanation`) |
+| `_test.go` files | broadly waive ireturn/bodyclose/cyclop/dupl/errcheck/funlen/gocognit/goconst/gosec/noctx/wrapcheck — don't fight these in tests |
+
+**Disabled — don't waste time appeasing them**: `nlreturn` ("too strict") and
+`lll` (superseded by golines). If a linter output mentions either, something
+is misconfigured in your local run, not a real gate.
+
+## Banned-nolint rule
+
+`//nolint:cyclop`, `//nolint:gocyclo`, `//nolint:gocognit`, `//nolint:funlen`
+are banned by project convention (not CI-enforced — a human/agent convention).
+Current count is 0; keep it that way. Verify:
+
+```bash
+grep -rn "nolint:cyclop\|nolint:gocyclo\|nolint:gocognit\|nolint:funlen" --include="*.go" .
+```
+
+When one of these linters fires, decompose — never re-suppress. Patterns
+used in this repo:
+
+- flat routing switch too complex → `map[routeKey]string` built via `sync.OnceValue`
+- long validate-then-build function → extract `validateXInput`/`resolveXLocked` helpers
+- big test function → `t.Helper()` sub-helpers
+
+## synctest / require.Eventually rule
+
+`time.Sleep` **inside** a `synctest.Test(t, func(t *testing.T){...})` bubble
+is fine — the bubble has a fake clock that advances instantly. The banned
+pattern is an *unbubbled* `time.Sleep` used to await async state outside a
+bubble. Real Docker/loopback I/O can't run in a bubble (not durably
+blocking), so those use `require.Eventually(t, condFn, timeout, tick, msg)`
+instead.
+
+```go
+synctest.Test(t, func(t *testing.T) {
+    b := NewInMemoryBackend(t.Context(), "123456789012", "us-east-1")
+    created, _ := b.CreateCluster(...)
+    time.Sleep(clusterTransitionDelay + time.Millisecond) // fake clock inside bubble
+    // assert ACTIVE
+})
+```
+
+(services/eks/async_lifecycle_test.go:35-58)
+
+`mulint` (declared as a go.mod `tool`, invoked separately by `make lint` via
+`go vet -vettool=...`, NOT a golangci-lint plugin) catches mutex misuse —
+recursive locks, copied locks — relevant given the coarse-lock convention in
+`pkgs-catalog.md`.
+
+## What CI runs (predict failures before opening a PR)
+
+`.github/workflows/ci.yml`: `ui-lint` → `lint` → `modernize` (`go fix -diff
+./...`) → `govulncheck` → `codeql` → `unit-tests` (4-way sharded:
+`-race -shuffle on -short -timeout 5m`, split by `awk "NR % 4 == chunk"` over
+`go list ./...`) → `build` (static linux binary, tags `netgo osusergo
+static_build`) → `integration-tests` (needs `build`; downloads the built
+artifact, never builds inline, 4-way sharded by discovered `^func Test`
+names, `-timeout 10m`) → `terraform-tests`.
+
+Branch protection on `main`: 12 required contexts (lint, unit, e2e, ui-lint,
+ui-test, integration, terraform, modernize, govulncheck, build, CodeFactor,
+CodeQL), `enforce_admins=true`, no bypass. If your change doesn't pass
+locally with `make lint && make test && make build-linux &&
+make integration-test`, it will not pass CI either.

.claude/skills/gopherstack-map/SKILL.md

@@ -0,0 +1,151 @@
+---
+name: gopherstack-map
+description: Navigate the gopherstack repo (161 AWS service emulators under services/, plus pkgs/, cli.go, test/, ui/) without re-discovering structure via Glob/Grep every session. Use whenever asked to find where a service is registered, what operations it supports, its routing style, its parity grade, which pkgs/ package to reuse, or generally "where is X" / "how does service Y work" / "map this service" for any gopherstack service. Trigger before opening a service directory cold, and run scripts/svcmap.sh <service> for a compact structural summary instead of manually listing+grepping files.
+---
+
+# gopherstack map
+
+161 per-service dirs share a small set of shapes. Reading this table beats
+re-deriving it with Glob/Grep.
+
+## Directory taxonomy
+
+| path | contents |
+|---|---|
+| `services/<svc>/` | one emulator: `handler.go` + `store.go` + op-family files + `PARITY.md` + generated `README.md` |
+| `pkgs/` | 32 shared infra packages (table below) |
+| `cmd/gendocs` | regenerates per-service READMEs + `.badges/*.svg` from PARITY.md (`make docs`) |
+| `test/e2e/` | SDK-driven e2e |
+| `test/integration/` | SDK-driven, Docker-container-backed — the real parity proof |
+| `test/terraform/` | terraform-provider-aws acceptance-style |
+| `ui/` | Svelte console source, `ui/src/routes/<service>/` |
+| `dashboard/static/spa/` | built UI output (gitignored but `.keep`-protected, load-bearing) |
+| `.beads/` | beads/Dolt issue tracker data |
+| `.badges/` | 5 committed SVGs regenerated by `make docs` |
+| `docs/` | hand-written narrative (architecture/, services/, docker.md, migration.md, quickstart.md) — distinct from generated per-service READMEs |
+| root `parity.md` | 1773-line live audit punch-list/planning doc — NOT a per-service manifest |
+| `services/_PARITY_TEMPLATE.md` | the PARITY.md schema |
+
+Also present: `internal/`, `modules/`, `proto/`, `examples/`, `bench/`, `scripts/`,
+`assets/`, `demo/`, `bin/` (build output).
+
+## Entry points
+
+- `main.go` — 5 lines, calls `Run()`.
+- `cli.go` (~9160 lines): `Run()` at line 1747 (Kong CLI parsing, then service
+  wiring). `echo.New()` at 2110. `initializeServices` at 2628,
+  `initIndependentServices` at 2660 — this is where every provider is
+  constructed. Chaos middleware `registry.Use(chaos.Middleware(faultStore))`
+  at 7626; IAM enforcement middleware at 7639.
+- Cross-service wiring (e.g. `wireTaggingGrafana` at 6992) happens in `Run()`
+  AFTER all providers exist. That's why a service needing a sibling backend
+  resolves it lazily on first request (see `cross_service.go`/`crossservice.go`
+  below), not at Init time.
+
+## Where do I look for X
+
+| question | answer |
+|---|---|
+| Is service X registered, and how? | `cli.go`, `initIndependentServices` (2660) — grep the service name |
+| What ops does X support? | `GetSupportedOperations()` in `services/X/handler.go` |
+| What's X's parity status? | `services/X/PARITY.md` frontmatter, `overall: A|B` field |
+| What's X's wire shape for op Y? | `services/X/wire.go` — doc comment cites the SDK source line that confirms it |
+| How does X convert domain↔wire? | `services/X/wire_convert.go` |
+| Where does X validate + mutate state? | the family file, e.g. `workspaces.go`, not `handler_workspaces.go` |
+| Where does X register its in-memory tables? | `services/X/store_setup.go`, `registerAllTables` |
+| Does X support tagging? | `services/X/tags.go` + `handler_tags.go`, or absent |
+| Does X reach into another service's backend? | `services/X/cross_service.go` or `crossservice.go` — a structurally-matched `siblingServices` interface |
+| Does X have chaos-injectable async transitions? | `services/X/chaos_transitions.go` |
+| Quick structural summary of X | run `scripts/svcmap.sh X` (below) |
+
+## Service file-role table
+
+Verified against grafana, networkmanager, account.
+
+| file | role |
+|---|---|
+| `handler.go` | `Handler` struct, `NewHandler`, `service.Registerable` methods (`Name`/`GetSupportedOperations`/`Reset`/`RouteMatcher`/`MatchPriority`/`ExtractOperation`/`ExtractResource`), route dispatch, `Handler()` echo entrypoint, `handleError`/`classifyError`, path+query helpers |
+| `handler_<family>.go` | HTTP handlers for one resource family: decode → call backend → marshal. Pure translation, no business logic |
+| `<family>.go` (e.g. `workspaces.go`) | backend methods `(b *InMemoryBackend) CreateWorkspace(...)`: validate, lock, mutate state |
+| `store.go` | `InMemoryBackend` struct, `NewInMemoryBackend`, `Reset`, ID generators, ARN builders |
+| `store_setup.go` | `registerAllTables` — one `store.Register(b.registry, "name", store.New(keyFn))` + `AddIndex` per collection |
+| `wire.go` | wire-shape DTO structs mirroring SDK field names/casing, doc-comment citing the SDK source |
+| `wire_convert.go` | `toXWire`/`fromXWire` mappers |
+| `errors.go` | sentinel errors, `apiError` struct, constructors (`notFoundError`/`conflictError`/`validationError`/`quotaError`) |
+| `consts.go` | path-segment and JSON-key literals reused 3+ times (goconst-driven) |
+| `provider.go` | `Provider` implementing `service.Provider` (Name, Init) — what `cli.go` registers |
+| `persistence.go` | `backendSnapshot` struct + Snapshot/Restore |
+| `tags.go` / `handler_tags.go` | TagResource/UntagResource/ListTagsForResource/TaggedResources |
+| `cross_service.go` / `crossservice.go` | lazy sibling-service lookups |
+| `chaos_transitions.go` | fault-injection hooks for async state-machine transitions |
+
+Older services may instead have `interfaces.go` (a `StorageBackend` interface)
+and a flat `operationHandlers` map instead of family-file backend methods.
+
+## Three routing shapes
+
+Detect by grepping `handler.go`:
+
+| shape | example | detect | how it works |
+|---|---|---|---|
+| nested path-segment switch | grafana | `routeRequest(` in handler.go | `routeRequest` returns `(op, dispatchFunc)` from a switch on path segments |
+| declarative route table | networkmanager | `routeTable()` in handler.go | `[]route{fn, op, method, pattern}` with `:Param` wildcard capture; one `<family>Routes()` per handler file, concatenated by `routeTable()` |
+| fixed-path op map | account | `operationHandlers[` in handler.go | `POST /OperationName`; `operationNames` + `operationHandlers` are method-expression maps keyed by path |
+
+## pkgs/ catalog (32 packages)
+
+**Locking rule** (the one that matters): lock granularity follows INVARIANT
+granularity, not data-structure granularity. Use one coarse
+`lockmetrics.RWMutex` per service backend, `mu.Lock("OpName")` labelled for
+metrics — AWS ops are cross-map transactions (FK validation + resource write
++ index update atomically; `Snapshot()` needs one consistent view).
+`safemap.Map[K,V]` is ONLY for genuinely isolated single-map state (token
+stores, caches, janitor-swept stores) — never backend resource maps, since
+per-map locks risk torn cross-map state. Never scatter raw `sync.Mutex`.
+
+| pkg | use for |
+|---|---|
+| arn | build AWS ARNs |
+| awserr | shared sentinel errors |
+| awsmeta | request-scoped AWS metadata |
+| awstime | wire-format timestamps (`Epoch()` for epoch-seconds) |
+| chaos | fault-injection middleware |
+| collections | type-safe slice/map helpers |
+| config | centralized AWS config |
+| container | runtime-agnostic container layer |
+| ctxval | type-safe context values |
+| dns | embedded DNS server |
+| docker | Docker integration |
+| dynamoattr | DynamoDB attribute values |
+| events | event system |
+| handler | HTTP helpers (WriteJSON, operation-context keys) |
+| httputils | body read/caching |
+| inithooks | startup user-script hooks |
+| lockmetrics | instrumented RWMutex |
+| logger | project-wide slog wrapper — ALL logging goes here |
+| page | generic `Page[T]` pagination + opaque tokens |
+| persistence | pluggable snapshot/restore |
+| portalloc | central port allocator |
+| ptrconv | nil-safe pointer deref |
+| safemap | isolated concurrency-safe map |
+| sdkcheck | SDK op-surface completeness |
+| service | registry/middleware, CloudTrail-capture chokepoint |
+| store | generic keyed collection `Table[V]` |
+| strs | case-fold string helpers |
+| tags | concurrency-safe resource tag map |
+| telemetry | observability observers |
+| testleak | goroutine-leak detection for tests |
+| version | build-time version info |
+| worker | standard background-work primitive |
+
+## scripts/svcmap.sh
+
+`scripts/svcmap.sh <service>` prints a ~20-line structural summary: file
+list with line counts, detected routing shape, op count, PARITY.md
+`overall:` grade + audit date, which of persistence/tags/cross_service/
+chaos_transitions files exist, and test files. Run it before reading a
+service cold:
+
+```bash
+.claude/skills/gopherstack-map/scripts/svcmap.sh grafana
+```