Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions .github/workflows/mutation-testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
name: Backend Mutation (Stryker.NET)
if: ${{ !inputs.frontend_only }}
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 180
steps:
- name: Checkout
uses: actions/checkout@v7
Expand All @@ -46,21 +46,26 @@ jobs:
dotnet-version: 8.0.x

- name: Install Stryker.NET
run: dotnet tool install --global dotnet-stryker
run: dotnet tool install --global dotnet-stryker --version 4.16.0

- name: Validate Stryker.NET configuration
shell: pwsh
run: ./scripts/ci/Test-StrykerConfig.ps1 -SelfTest

- name: Restore solution
run: dotnet restore backend/Taskdeck.sln

- name: Run Stryker.NET
working-directory: backend
run: dotnet stryker --config-file stryker-config.json
working-directory: backend/tests/Taskdeck.Domain.Tests
run: dotnet stryker --config-file ../../stryker-config.json --output ../../StrykerOutput
Comment thread
Chris0Jeky marked this conversation as resolved.

- name: Upload Stryker report
if: always()
uses: actions/upload-artifact@v7
with:
name: stryker-net-report
path: backend/StrykerOutput/**/reports/
if-no-files-found: error
retention-days: 30

frontend-mutation:
Expand Down
8 changes: 2 additions & 6 deletions backend/stryker-config.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
{
"stryker-config": {
"project": "Taskdeck.Domain.csproj",
"test-projects": [
"tests/Taskdeck.Domain.Tests/Taskdeck.Domain.Tests.csproj"
],
"solution": "Taskdeck.sln",
"mutation-level": "Standard",
"reporters": [
"html",
Expand All @@ -19,7 +15,7 @@
},
"language-version": "latest",
"verbosity": "info",
"ignored-methods": [],
"excluded-mutations": []
"ignore-methods": [],
"ignore-mutations": []
Comment thread
Chris0Jeky marked this conversation as resolved.
}
}
4 changes: 4 additions & 0 deletions docs/IMPLEMENTATION_MASTERPLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ Companion Active Docs:
- `docs/MANUAL_TEST_CHECKLIST.md`
- `docs/GOLDEN_PRINCIPLES.md`

## Delivery update (2026-07-27, mutation-testing runway)

- **Bounded backend mutation evidence (`#1500`, PR `#1503`, maintainer-held):** repaired the Domain Stryker.NET lane by pinning 4.16.0, validating its config/workflow contract with a 25-check preflight, running from the single Domain test-project context, capping the job at 180 minutes, and failing artifact upload when no report exists. Backend-only run [30236307062](https://github.com/Chris0Jeky/Taskdeck/actions/runs/30236307062) on exact workflow head `307add004fbe142321a6ec11be21fab708824d5d` completed in 192 seconds over 3,682 mutants (2,351 killed, 576 survived, 2 timed out, 753 skipped), for a 70.75% baseline. Its two-file `stryker-net-report` artifact is 874,386 bytes with SHA-256 `0e8a9a41b8cd484b6c267bd914c57cda0ffa973f59d8989e89038157605f21c8`. The score is calibration evidence, not a merge gate; expansion/ratcheting remains deferred until the existing 3-4-run policy baseline exists.

## Delivery update (2026-07-27, test substrate and Windows API diagnostics)

- **Dockerless Testcontainers gate (`#1518`):** the PostgreSQL collection fixture now evaluates `DockerAvailableCheck` before `PostgreSqlBuilder.Build`, restoring the documented graceful-skip behavior on hosts without Docker. Public-constructor and injected-factory regressions prove the unavailable path cannot construct a container, while a timed-out Docker CLI probe is killed as a process tree and boundedly reaped. The local exact tree finishes with 7 passed / 28 skipped / 0 failed and no surviving `docker info`; at reviewed head `6650370b`, the full serialized backend passed 7,465 tests with 33 intentional skips and hosted Container Integration proved the positive lifecycle with 35 passed / 0 skipped / 0 failed. Four DCO-signed-off commits merged as `ff1ee066`; exact-main Required CI `30257236213` and CodeQL `30257235538` are green.
Expand Down
6 changes: 5 additions & 1 deletion docs/MANUAL_TEST_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -1054,7 +1054,11 @@ Status legend: `[ ]` = not yet performed, `[x]` = verified.
19. [ ] Verify mutation testing workflow is NOT listed in `ci-required.yml` (non-blocking, nightly only).
20. [ ] Verify `StrykerOutput` directories are in `.gitignore`.
21. [ ] Run `npm run mutation:test` locally to validate frontend Stryker setup (after `npm install`).
22. [ ] Run `dotnet stryker --config-file stryker-config.json` locally in `backend/` to validate backend setup.
22. [ ] Validate the backend Stryker setup from the repository root.
- Run: `powershell -NoProfile -File scripts/ci/Test-StrykerConfig.ps1 -SelfTest`
- Run: `cd backend/tests/Taskdeck.Domain.Tests`
- Run: `dotnet stryker --config-file ../../stryker-config.json --output ../../StrykerOutput`
- Expected: the preflight passes and Stryker uses only `Taskdeck.Domain.Tests` for the `Taskdeck.Domain` mutation run.

### Z8. Platform: PWA and Offline Readiness (PR #802 — merged)

Expand Down
3 changes: 3 additions & 0 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

Last Updated: 2026-07-27

Backend mutation-testing runway repair (2026-07-27, `#1500`):
- **The scheduled Domain mutation lane now reaches a bounded, artifact-backed result instead of failing before execution.** Stryker.NET is pinned to 4.16.0; a 25-check PowerShell preflight rejects obsolete schema keys, wrong solution/test context, workflow timeout drift, and non-failing artifact uploads; and the workflow runs from `Taskdeck.Domain.Tests` with a 180-minute ceiling. Backend-only run [30236307062](https://github.com/Chris0Jeky/Taskdeck/actions/runs/30236307062) on exact workflow head `307add004fbe142321a6ec11be21fab708824d5d` completed in 192 seconds: 3,682 mutants created, 2,351 killed, 576 survived, 2 timed out, 753 skipped, score 70.75%. The non-empty two-file `stryker-net-report` artifact is 874,386 bytes (SHA-256 `0e8a9a41b8cd484b6c267bd914c57cda0ffa973f59d8989e89038157605f21c8`). The score remains a non-blocking calibration signal; PR `#1503` and merge remain maintainer-held because the workflow changes operational CI posture.

Dockerless Testcontainers gate repair (2026-07-27, `#1518`):
- **The PostgreSQL integration collection now reaches its documented graceful-skip path before Testcontainers validates Docker.** `PostgresContainerFixture` checks `DockerAvailableCheck` before constructing the container, so a Dockerless host runs seven host-native integration tests and reports all 28 container-dependent tests skipped instead of throwing from the fixture constructor. The real public-constructor and injected-factory regressions prove that the unavailable path does not construct or dispose a phantom container. A timed-out Docker CLI probe is killed with its process tree and given a bounded reap before the check returns; the local exact tree finished with 7 passed / 28 skipped / 0 failed and zero `docker info` processes before or after. At reviewed head `6650370b`, the full serialized backend passed 7,465 tests with 33 intentional skips and hosted Container Integration proved the positive startup/connection/teardown path with 35 passed / 0 skipped / 0 failed. Four DCO-signed-off commits merged as `ff1ee066`; exact-main Required CI run `30257236213` and CodeQL run `30257235538` are green.

Expand Down
23 changes: 16 additions & 7 deletions docs/TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -642,18 +642,24 @@ CI: `reusable-visual-regression.yml` in extended CI (testing/visual label). Uplo

### Mutation Testing (TST-05, `#90`/`#796`)

Backend (Stryker.NET): targets `Taskdeck.Domain` with `Taskdeck.Domain.Tests`. Thresholds: break=60, high=80.
Backend (Stryker.NET 4.16.0): targets `Taskdeck.Domain` from the `Taskdeck.Domain.Tests` project context. Thresholds: break=0, low=60, high=80. The checked-in preflight rejects obsolete config keys and solution-context/workflow drift before the long mutation run.
Frontend (Stryker JS): targets `captureStore`, `boardStore`, and `board/*.ts` submodules with vitest runner.

Run commands:
```bash
# Backend
cd backend && dotnet tool install dotnet-stryker && dotnet stryker
powershell -NoProfile -File scripts/ci/Test-StrykerConfig.ps1 -SelfTest
# PowerShell 7/Linux runner equivalent: pwsh -File scripts/ci/Test-StrykerConfig.ps1 -SelfTest
dotnet tool install --global dotnet-stryker --version 4.16.0
cd backend/tests/Taskdeck.Domain.Tests
dotnet stryker --config-file ../../stryker-config.json --output ../../StrykerOutput
# Frontend
cd frontend/taskdeck-web && npm run mutation:test
```

CI: `mutation-testing.yml` runs weekly (Sunday 04:00 UTC) + manual dispatch. Non-blocking, reports uploaded as artifacts. Policy at `docs/testing/MUTATION_TESTING_POLICY.md`.
CI: `mutation-testing.yml` runs weekly (Sunday 04:00 UTC) + manual dispatch. The backend job has a finite 180-minute ceiling for the full Domain mutation set and fails if no report artifact exists. Mutation score remains non-blocking. Policy at `docs/testing/MUTATION_TESTING_POLICY.md`.

Verified baseline: backend-only run [30236307062](https://github.com/Chris0Jeky/Taskdeck/actions/runs/30236307062) on exact workflow head `307add004fbe142321a6ec11be21fab708824d5d` completed in 192 seconds. Stryker created 3,682 mutants; 2,351 were killed, 576 survived, 2 timed out, and 753 were skipped, for a 70.75% score. The uploaded two-file `stryker-net-report` artifact is 874,386 bytes (SHA-256 `0e8a9a41b8cd484b6c267bd914c57cda0ffa973f59d8989e89038157605f21c8`).

### Container Integration Tests (TST-06, `#91`/`#804`)

Expand Down Expand Up @@ -812,9 +818,12 @@ Mutation testing is available as a non-blocking quality signal for detecting wea
### Running locally

```bash
# Backend (requires dotnet-stryker global tool)
cd backend
dotnet stryker --config-file stryker-config.json
# Backend (from the repository root; uses the workflow-pinned tool version)
powershell -NoProfile -File scripts/ci/Test-StrykerConfig.ps1 -SelfTest
# PowerShell 7/Linux runner equivalent: pwsh -File scripts/ci/Test-StrykerConfig.ps1 -SelfTest
dotnet tool install --global dotnet-stryker --version 4.16.0
cd backend/tests/Taskdeck.Domain.Tests
dotnet stryker --config-file ../../stryker-config.json --output ../../StrykerOutput

# Frontend
cd frontend/taskdeck-web
Expand All @@ -823,7 +832,7 @@ npm run mutation:test

### CI

Weekly workflow (Sunday 04:00 UTC) + manual dispatch via `.github/workflows/mutation-testing.yml`. Reports uploaded as artifacts.
Weekly workflow (Sunday 04:00 UTC) + manual dispatch via `.github/workflows/mutation-testing.yml`. The backend job has a 180-minute ceiling and missing backend reports fail artifact upload.

### Policy and triage

Expand Down
1 change: 1 addition & 0 deletions docs/agentic/FAILURE_LEDGER.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Rows sharing a surface and first tracking issue in `future_fix` show only their
| 2026-07-26 | non_blocking_risk | agent/tool-command-composition | Resolution record for the repeated #1490 PowerShell/native command-composition failures; the original recurrence remains in append-only history | No workaround remains; use the copyable safe forms and classify future failures through the ledger process | #1490 resolved by PR #1491 merge 61f718af941c56c31b8b7595333b9debd6e47da8: the MCP tooling guide now covers collected foreach output, optional rg exits, safe mu... | resolved |
| 2026-07-26 | blocker | agent/python-launcher | Resolution staged for #1487: exact Windows handlers and agent-utility permissions use py -3 -B, POSIX permissions retain python3 -B, and smoke children use the ... | Until the T4-class hook configuration is maintainer-merged, continue invoking py -3 -B explicitly on native Windows | #1487 resolution is staged on the human-held PR with sequential CPython 3.14 and 3.13 smoke coverage; append the final resolved record after merge and issue clo... | open |
| 2026-07-26 | blocker | agent/powershell-deny-floor | The repo PreToolUse deny policy is matched only to Bash, so native PowerShell tool commands do not traverse the repository's destructive-command policy | Treat the current configured-handler smoke as Bash-payload-only proof; rely on existing tool permissions and do not claim native-PowerShell interception | #1497: add reviewed T4 native-PowerShell deny-policy coverage with direct allow and deny canaries before claiming interception | open |
| 2026-07-27 | non_blocking_risk | ci/mutation-testing-backend | Resolution record for the 16-run backend mutation evidence outage; the original failures remain in append-only history. | No workaround remains for producing bounded Domain mutation evidence; retain the non-blocking score posture while collecting the policy's 3-4-run calibration wi... | #1500 / PR #1503 repaired the lane at exact workflow head 307add004fbe142321a6ec11be21fab708824d5d. Backend-only run 30236307062 completed in 192 seconds with a... | resolved |
| 2026-07-27 | pre_existing_noise | test/sqlite-concurrency | Correction to the #1373 resolution record: Required API Integration (windows-latest) on PR #1505 source head 6ee17a4849427d679722a9d1e034d4622536f4fe returned H... | Keep every recurrence red and use the bounded #1512 correlation/exception-classification diagnostics to capture the first server-side cause; do not retry the re... | #1373 resolved the proven #1282 SQLite registration-parity cause; successor #1512 remains open after its diagnostic-only slice until a recurrence identifies the... | open |
| 2026-07-27 | blocker | test/testcontainers-dockerless-skip | Taskdeck.Integration.Tests constructed its PostgreSQL Testcontainer in the collection-fixture constructor, so Testcontainers validated an unavailable Docker end... | Run the focused non-container projects while #1518 is in review; do not retry a known-hanging Docker probe merely to obtain a local full-suite verdict | #1518 defers PostgreSqlBuilder.Build until after DockerAvailableCheck succeeds, proves the unavailable path never invokes the factory, and retains Docker-backed... | resolved |

Expand Down
2 changes: 2 additions & 0 deletions docs/agentic/failure_ledger.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,7 @@
{"ts":"2026-07-25T23:48:15Z","class":"blocker","surface":"agent/python-launcher","failure":"Native-Windows python and python3 commands resolve to unusable Microsoft Store aliases, so configured agent hooks and documented failure-ledger utilities do not run even though CPython is installed behind py -3","workaround":"Invoke py -3 -B explicitly on Windows and python3 -B on POSIX; do not install another interpreter or mutate global PATH","future_fix":"#1487: make platform launchers explicit across configured handlers, permissions, smoke execution, and mirrored guidance","status":"open"}
{"ts":"2026-07-26T10:56:00Z","class":"blocker","surface":"agent/python-launcher","failure":"Resolution staged for #1487: exact Windows handlers and agent-utility permissions use py -3 -B, POSIX permissions retain python3 -B, and smoke children use the active sys.executable -B","workaround":"Until the T4-class hook configuration is maintainer-merged, continue invoking py -3 -B explicitly on native Windows","future_fix":"#1487 resolution is staged on the human-held PR with sequential CPython 3.14 and 3.13 smoke coverage; append the final resolved record after merge and issue closure","status":"open"}
{"ts":"2026-07-26T11:36:55Z","class":"blocker","surface":"agent/powershell-deny-floor","failure":"The repo PreToolUse deny policy is matched only to Bash, so native PowerShell tool commands do not traverse the repository's destructive-command policy","workaround":"Treat the current configured-handler smoke as Bash-payload-only proof; rely on existing tool permissions and do not claim native-PowerShell interception","future_fix":"#1497: add reviewed T4 native-PowerShell deny-policy coverage with direct allow and deny canaries before claiming interception","status":"open"}
{"ts":"2026-07-27T00:00:00Z","class":"non_blocking_risk","surface":"ci/mutation-testing-backend","failure":"All 16 recorded scheduled Mutation Testing runs failed to produce backend Stryker evidence; the latest runs 30191449339 and 29676656172 both stopped because Stryker.NET 4.16.0 rejects ignored-methods and excluded-mutations as invalid schema keys.","workaround":"Treat the frontend lane as independent evidence, but do not claim backend mutation coverage until a backend-only exact-head run completes and uploads a non-empty report artifact.","future_fix":"#1500 pins Stryker.NET 4.16.0, uses ignore-methods and ignore-mutations, and adds a local preflight that rejects obsolete keys; resolve this row only after the PR exact-head backend-only dispatch completes mutation execution with a non-empty Stryker artifact.","status":"open"}
{"ts":"2026-07-27T04:12:28Z","class":"non_blocking_risk","surface":"ci/mutation-testing-backend","failure":"Resolution record for the 16-run backend mutation evidence outage; the original failures remain in append-only history.","workaround":"No workaround remains for producing bounded Domain mutation evidence; retain the non-blocking score posture while collecting the policy's 3-4-run calibration window.","future_fix":"#1500 / PR #1503 repaired the lane at exact workflow head 307add004fbe142321a6ec11be21fab708824d5d. Backend-only run 30236307062 completed in 192 seconds with a 70.75% score over 3,682 mutants and uploaded a non-empty 874,386-byte two-file report artifact (SHA-256 0e8a9a41b8cd484b6c267bd914c57cda0ffa973f59d8989e89038157605f21c8).","status":"resolved"}
{"ts":"2026-07-27T06:50:27Z","class":"pre_existing_noise","surface":"test/sqlite-concurrency","failure":"Correction to the #1373 resolution record: Required API Integration (windows-latest) on PR #1505 source head 6ee17a4849427d679722a9d1e034d4622536f4fe returned HTTP 500 for 2 of 5 concurrent card-create requests while Ubuntu passed; an identical-head rerun and local stress passed, so the causal exception and any SQLite/SQLITE_BUSY classification remain unproven","workaround":"Keep every recurrence red and use the bounded #1512 correlation/exception-classification diagnostics to capture the first server-side cause; do not retry the request, quarantine the test, or accept HTTP 500 as success","future_fix":"#1373 resolved the proven #1282 SQLite registration-parity cause; successor #1512 remains open after its diagnostic-only slice until a recurrence identifies the root cause and a narrow corrective fix is proven by the full backend and exact-head Windows/Ubuntu gates","status":"open"}
{"ts":"2026-07-27T09:20:44Z","class":"blocker","surface":"test/testcontainers-dockerless-skip","failure":"Taskdeck.Integration.Tests constructed its PostgreSQL Testcontainer in the collection-fixture constructor, so Testcontainers validated an unavailable Docker endpoint before DockerAvailableCheck could mark the 28 container tests skipped; this blocked the serialized backend gate on Dockerless developer hosts","workaround":"Run the focused non-container projects while #1518 is in review; do not retry a known-hanging Docker probe merely to obtain a local full-suite verdict","future_fix":"#1518 defers PostgreSqlBuilder.Build until after DockerAvailableCheck succeeds, proves the unavailable path never invokes the factory, and retains Docker-backed CI as the positive-lifecycle gate","status":"resolved"}
Loading
Loading