Skip to content

Add optional concurrent per-image verification - #199

Draft
bdehamer wants to merge 3 commits into
mainfrom
bdehamer-parallel-image-verification
Draft

Add optional concurrent per-image verification#199
bdehamer wants to merge 3 commits into
mainfrom
bdehamer-parallel-image-verification

Conversation

@bdehamer

Copy link
Copy Markdown
Contributor

Summary

Implements the first idea from #195: verify the images in a multi-image admission request concurrently instead of one at a time.

Today Validate loops over r.Request.Keys and calls BundleFromName for each image on the shared request context, so a slow fetch for one image can consume most of the shared deadline and starve the images processed after it. This PR adds an opt-in, bounded worker pool for the outer, per-image loop so total wall-clock time approaches the max of the per-image fetches rather than the sum.

Per the request, scope is deliberately narrow:

  • ✅ Parallelize the outer, per-image loop only.
  • ❌ Do not parallelize attestation-bundle retrieval within a single image — each image still fetches its bundles serially (DoBundleFromName is untouched).
  • ✅ Gated behind a start-up flag that defaults to the existing serial behavior.

Changes

  • -image-concurrency flag (cmd/aaop/aaop.go), default 1 (serial). Values > 1 cap the number of images verified simultaneously; values < 1 are rejected at start-up.
  • provider.WithConcurrency(n) option (pkg/provider/provider.go). New is now variadic and defaults to serial; n < 1 is clamped to 1.
  • Extracted the per-image body into a validateImage helper. Validate dispatches to a serial path (unchanged, byte-for-byte behavior including the verification-error short-circuit) or a bounded worker pool (sync.WaitGroup.Go + a semaphore channel).
  • Ordering and semantics preserved: response items keep request/key order, and a verification error still yields a request-level SystemError (the first error in key order wins, so results are deterministic in either mode).

Behavior notes

Testing

  • go test -race ./... passes.
  • golangci-lint run reports 0 issues.
  • New provider tests: serial/parallel result parity, a deterministic barrier-based test proving images are processed concurrently (a serial impl would deadlock and fail), a bounded-concurrency test asserting the pool never exceeds the limit, WithConcurrency clamping, and the system-error path under concurrency.

Follow-ups / not included

  • The flag is not wired into the Helm chart, consistent with the existing bundle-* tuning flags (also chart-less). Happy to add chart plumbing if desired.

Refs #195

Opened as a draft for early feedback on the approach (flag shape, default concurrency bound).

When a single admission request contains multiple images (e.g. a
multi-container pod), the provider validated them one at a time on the
shared request context, so a slow fetch for one image could starve the
images processed after it (issue #195).

Add an opt-in, bounded worker pool that verifies the request's images in
parallel. Only the outer, per-image loop is parallelized; each image
still fetches its own attestation bundles serially. The behavior is
gated behind a new `-image-concurrency` start-up flag that defaults to 1
(the existing serial behavior). The per-image logic is extracted into a
`validateImage` helper, and result ordering plus the verification-error
short-circuit are preserved regardless of serial or parallel execution.

Refs #195

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in, bounded concurrent image verification while preserving serial behavior by default.

Changes:

  • Adds WithConcurrency and ordered concurrent validation.
  • Adds the -image-concurrency startup flag.
  • Adds concurrency, bounds, parity, and error-path tests.
Show a summary per file
File Description
pkg/provider/provider.go Implements concurrent per-image validation.
pkg/provider/provider_test.go Tests concurrency behavior and limits.
cmd/aaop/aaop.go Adds and validates the concurrency flag.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread pkg/provider/provider.go
A panic from the pluggable fetcher/verifier in a wg.Go worker goroutine
would crash the entire provider process, since net/http only recovers
panics in the request goroutine (the serial path). Recover inside each
worker and convert the panic into that image's indexed system error, so
opt-in concurrency cannot turn one malformed request or dependency panic
into a process-wide outage. Adds a panic-path test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread pkg/provider/provider.go
The returned option previously clamped n < 1 inside the closure, mutating
its captured variable on every application. Reusing one such option
across concurrent New calls would race on that shared state. Normalize n
once before creating the closure so applying an Option is read-only and
safely reusable.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants