feat(template): isolate template-build compression with [template_build] switch

[huajq]

What

Adds a dedicated [template_build] compression section (compression_enabled / compression_algorithm / compression_workers, all off by default). When enabled, template builds capture snapshots with both memory layers and the sealed rootfs read-write layer written as ZFile-compressed overlaybd layers. Pause/capture of ordinary running sandboxes is unchanged.

Why

The only pre-existing compression switch, [memory_snapshot].compression_enabled, covers memory layers only (rootfs seals always stay raw) and is shared by every capture path — so compressing template-built artifacts forced operators to also pay compression CPU on every production pause. Template-built snapshots are the distribution vehicle for sandbox images; compressing their memory and rootfs delta layers shrinks repository storage and cross-node transfer, and that choice should be decoupled from the runtime pause policy.

Related issue

N/A — no tracking issue.

Scope and non-goals

Included: the [template_build] config section; compression-override plumbing in the template-build capture path; ZFile recontainerization of the sealed rootfs layer; the delta-filename convention extension; unit and integration tests; documentation.

Intentionally excluded: the orchestrator pause/capture path (memory layers still consult only [memory_snapshot], rootfs seals stay raw); extra drives (stay raw); ublk-daemon and the RestackSnapshot RPC (the daemon still seals raw); publish/resume/P2P code (existing zfile compatibility is reused); no per-API compression knob.

Design and behavior changes

• New TemplateBuildConfig registered as the [template_build] section. The shared algorithm enum is renamed to OverlaybdCompressionAlgorithm (serde wire values lz4/zstd unchanged).
• FirecrackerSandbox gains snapshot_compression_override, set only by the template runner; snapshot_to_dir prefers the override for both the memory output and the rootfs seal output.
• At staging time the daemon seals the upper into a raw snapshot.commit as before (the live runtime keeps referencing it); the main process then recontainerizes only the staged artifact into snapshot.zfile.commit via the existing compact_layers machinery. The over-budget compaction path also honors the requested seal output.
• local_layer.rs learns the snapshot.zfile.commit delta filename (shared by the posixfs/OSS/ACR publish paths).
• Behavior change: template builds no longer consult [memory_snapshot].compression_enabled. If that switch is on but [template_build] is off, template artifacts stay raw.

Compatibility and operations

• Public API or generated protocol: no changes (no generated code touched).
• Configuration or defaults: new optional [template_build] section; every key has a default (off / lz4 / 1), existing configs load unchanged.
• Snapshot manifest, artifact layout, or storage format: staged rootfs configs may reference the new snapshot.zfile.commit name; ZFile is the standard overlaybd layer format that resume paths already read natively. Managed-layer bytes of template-built snapshots may now be ZFile (already true for memory layers).
• Upgrade and rollback: the switch defaults off, so upgrades are unaffected. Rollback is safe — resume does not depend on filename matching, so older binaries can resume and re-capture compressed snapshots.
• Host requirements, permissions, ports, or dependencies: no changes.
• Known minor effect: P2P uuid-based advertisement skips ZFile seal layers (best-effort path; falls back to repository storage).

Validation

• make fmt
• make clippy
• make test-unit
• Relevant Rust integration tests
• make -C services test (required when services/ changes)
• Generated clients/server regenerated with the documented make target
• Documentation updated
• Benchmarks or performance comparison completed

Commands and results:

make fmt && make clippy
  # both clean, no warnings

make test-unit
  # 792 passed; 2 failed — both pre-existing environment issues, unrelated
  # to this change:
  # - image::resolver::discover_overlaybd_referrer_*: needs registry network
  #   access; passes once the registry is reachable
  # - privileges::scoped_spawn_clears_child_capabilities: fails only when run
  #   as root; passes as a non-root user (CI runs unit tests as non-root)

# Integration test on a Linux KVM host, with [template_build].compression_enabled=true
# and [memory_snapshot].compression_enabled=false (to prove isolation):
cargo test -p agentenv --test integration -- \
  --exact snapshot::built_and_derived_snapshot_can_be_launched
  # ok — the test asserts the resolved rootfs and memory newest lowers of both
  # the base and the derived template build probe as ZFile, covering
  # build -> publish -> resolve -> resume -> re-capture end to end

# Same test with [memory_snapshot].compression_enabled=true and [template_build] off:
  # ok — template artifacts are unaffected by the memory switch

Skipped checks and reasons: no benchmark — the non-template hot path is untouched (one extra Option check per capture); the compression cost (one extra read+write pass of the sealed layer) only applies to template-build captures with the switch on, by design. services/ and generated code untouched, so those checks are N/A.

Risks and reviewer notes

• Behavior change to call out: template builds no longer honor [memory_snapshot].compression_enabled; deployments relying on the old coupling must set [template_build] explicitly.
• Key files: src/sandbox/firecracker/overlaybd_snapshot.rs (staging recontainerization), src/sandbox/firecracker/sandbox.rs (override precedence), src/image/local_layer.rs (filename convention).
• On the over-budget path (32+ runtime-owned layers) the sealed layer is recontainerized and then merged again by compaction — correct but slightly wasteful; left as-is deliberately.

Checklist

• The PR contains one coherent change and no unrelated formatting or refactoring.
• New behavior is covered by tests, or I explained why testing is impractical.
• Logs and examples contain no credentials, tokens, or private registry information.
• I did not manually edit generated code without updating its source and regenerating it.

[github-actions]

🔍 OpenCodeReview found 4 issue(s) in this PR.

• ✅ Successfully posted inline: 3 comment(s)
• ❌ Failed to post inline: 1 comment(s)

---

📄 src/cfg.rs (L444-L448)

⚠️ GitHub could not post this as an inline comment: Lines 444-448 could not be resolved (outside PR diff hunks)

This renames a public type and therefore breaks source compatibility for downstream users that import or name MemorySnapshotCompressionAlgorithm, even though the memory-snapshot field still exposes the same domain and behavior. Internal call sites were migrated, but external users cannot be. Preserve a deprecated alias for the old name (or defer the rename to a breaking release).

💡 Suggested Change

Before:

pub enum OverlaybdCompressionAlgorithm {
    #[default]
    Lz4,
    Zstd,
}

After:

pub enum OverlaybdCompressionAlgorithm {
    #[default]
    Lz4,
    Zstd,
}

#[deprecated(note = "use OverlaybdCompressionAlgorithm")]
pub type MemorySnapshotCompressionAlgorithm = OverlaybdCompressionAlgorithm;

[github-actions]

TemplateBuildConfig is the only newly added nested config type not included in impl_config_default!. As a result, callers cannot construct it with TemplateBuildConfig::default() or use struct-update syntax, unlike MemorySnapshotConfig and the other nested configuration types. Add it to the default-implementation list to keep the public configuration API consistent and make future field additions less disruptive to callers.

Suggestion:

Suggested change

	pub struct TemplateBuildConfig {
	#[config(default = false)]
	pub compression_enabled: bool,
	#[config(default = "lz4")]
	pub compression_algorithm: OverlaybdCompressionAlgorithm,
	/// Number of blocking threads used to compress 4KiB blocks within a
	/// layer. 1 = sequential (identical output layout at any value).
	#[config(default = 1)]
	pub compression_workers: usize,
	}
	pub struct TemplateBuildConfig {
	#[config(default = false)]
	pub compression_enabled: bool,
	#[config(default = "lz4")]
	pub compression_algorithm: OverlaybdCompressionAlgorithm,
	/// Number of blocking threads used to compress 4KiB blocks within a
	/// layer. 1 = sequential (identical output layout at any value).
	#[config(default = 1)]
	pub compression_workers: usize,
	}
	// Also add `TemplateBuildConfig` to `impl_config_default!(...)` below.

[github-actions]

When the runtime-owned suffix exceeds the compaction budget, appended_layer has already been compressed to ZFile above and is then fed into compact_layers(..., seal_output) with the rest of the suffix. Although mixed ZFile input is supported, this decompresses and recompresses the freshly sealed layer, doubling expensive compression I/O for long-lived/derived template chains. Decide whether suffix compaction is needed before recontainerizing: compact the raw appended layer directly when over budget, and create snapshot.zfile.commit only for the non-compaction path. Add a test covering the over-budget branch.

[github-actions]

This check does not exercise the new compression path under the repository's default configuration (template_build.compression_enabled defaults to false), so a normal integration-test run can pass even if the builder never propagates compression. Add a dedicated test setup/configuration that enables template-build compression rather than making the only end-to-end assertion a no-op. That test should also open the ZFile and compare options().algo with template_build.compression_algorithm; is_zfile alone would not catch an implementation that ignores the configured LZ4/Zstd choice.