Skip to content

Commit f624e3b

Browse files
sjarmakclaude
andcommitted
fix: verifier pipefail crash, k8s false-pass guard, sg_only clone manifests
P1: Wrap grep|wc pipelines with (grep ... || true) in all 13 new refactor verifiers to prevent crash under set -euo pipefail when grep finds no matches (exit 1). P0: Add commit-count threshold guard (>5) in k8s-score-normalizer verifier to prevent false pass when image checkout is ahead of origin/master (22 release-branch commits counted as agent work). P2: Add clone manifests to all 23 new sg_only Dockerfiles so sgonly_verifier_wrapper can clone the mirror repo at verify time instead of running against an empty workspace. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 11cb88a commit f624e3b

File tree

37 files changed

+116
-64
lines changed

37 files changed

+116
-64
lines changed

benchmarks/ccb_feature/cilium-policy-audit-logger-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/cilium--v1.16.5", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/cilium-policy-quota-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/cilium--v1.16.5", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/curl-http3-priority-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/curl--09e25b9d", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/django-rate-limit-middleware-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/django--674eda1c", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/envoy-custom-header-filter-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/envoy--v1.33.0", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/numpy-rolling-median-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/numpy--v2.2.2", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/pandas-merge-asof-indicator-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/pandas--v2.2.3", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/prometheus-silence-bulk-api-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/prometheus--ba14bc4", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/pytorch-gradient-noise-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/pytorch--d18007a1", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

benchmarks/ccb_feature/terraform-compact-diff-fmt-feat-001/environment/Dockerfile.sg_only

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ RUN git init && \
1515

1616
RUN mkdir -p /logs/agent /logs/verifier
1717

18+
RUN echo '{"workdir": "/workspace", "repos": [{"mirror": "sg-evals/terraform--v1.10.3", "target_dir": "."}]}' > /tmp/.sg_only_clone_manifest.json
19+
1820
RUN touch /tmp/.sg_only_mode
1921

2022
RUN (adduser --disabled-password --gecos '' claude 2>/dev/null || true) && \

0 commit comments

Comments
 (0)