scope the destructive audit tool to hypotheses, and stop using it for coverage #573

New issue

Open

opened 2026-09-21 23:40:09 +00:00 by hexajon · 0 comments

hexajon commented

2026-09-21 23:40:09 +00:00

Owner

Part of #563. Depends on #572, which builds the thing that replaces half of what this tool is currently asked to do.

.claude/agents/adr-0029-surface-audit.md is a repo-scoped audit tool that reads the codebase against ADR 0029 and reports findings. It is committed and it stays. Its brief currently asks it to do two jobs, and it is only capable of one.

What it does well

It finds real defects. In a single pass it surfaced the TreeIdentical short-circuit (internal/service/git/branch_merge_state.go:32-38), the orchestrate refusal that names the surface still allowing the refused act (internal/mcp/tools_orchestrate.go:131-141), the unproven remote branch delete (internal/mcp/tools_branches.go:68-70), and the two-policy contradiction between the pin test and the record. Each of those was confirmed by reading the source afterwards. Every issue under #563 started as one of its findings.

What it cannot do

Anything requiring completeness or a count.

Three passes over the same list (internal/parity/destructive_remote_pin_test.go:126-155) reported 22, 22, and 25 ungated rows. There are 24.
One pass applied the "a rationale comment is not a proof" rule correctly to branches-remote/delete and then relied on the same kind of comment to drop issues/comment-delete, issues/attachment-delete, prs/comment-delete, and prs/review-comment-delete from the same report.
Operations get absorbed into aggregate counts ("34+ operations absent from the table"), which reads as coverage and names nothing a reader can act on.

Rewriting the brief does not fix this. Three rewrites changed which defects appeared and never made the count stable.

What to change

Say in the brief that the tool produces hypotheses, and that no output of it is a coverage statement. Ban aggregate findings: an operation is named or it is not reported.
Remove the completeness language. Once #572 lands, coverage comes from the generated inventory and the gate, and the brief points at that instead of asking for a sweep.
Require a per-operation verdict line for any list the tool reads, or an explicit statement that it did not enumerate. Silence and a total are both currently acceptable outputs and neither should be.
State that a finding is unconfirmed until read in the source by whoever acts on it. Two reproductions it reported were filed as #564 and #566 on its word.

Acceptance criteria

The brief says what the tool is for and, in the same place, what it must never be trusted for, with the counting evidence above.
No output of the tool is presented as coverage anywhere.
After #572, coverage questions route to the generated inventory and the brief says so.
Any change to the brief is tested by running it before the change is committed. Three brief edits reached this branch untested, and two of the three did not do what their commit messages claimed.

Part of #563. Depends on #572, which builds the thing that replaces half of what this tool is currently asked to do. `.claude/agents/adr-0029-surface-audit.md` is a repo-scoped audit tool that reads the codebase against ADR 0029 and reports findings. It is committed and it stays. Its brief currently asks it to do two jobs, and it is only capable of one. ## What it does well It finds real defects. In a single pass it surfaced the `TreeIdentical` short-circuit (`internal/service/git/branch_merge_state.go:32-38`), the orchestrate refusal that names the surface still allowing the refused act (`internal/mcp/tools_orchestrate.go:131-141`), the unproven remote branch delete (`internal/mcp/tools_branches.go:68-70`), and the two-policy contradiction between the pin test and the record. Each of those was confirmed by reading the source afterwards. Every issue under #563 started as one of its findings. ## What it cannot do Anything requiring completeness or a count. - Three passes over the same list (`internal/parity/destructive_remote_pin_test.go:126-155`) reported 22, 22, and 25 ungated rows. There are 24. - One pass applied the "a rationale comment is not a proof" rule correctly to `branches-remote/delete` and then relied on the same kind of comment to drop `issues/comment-delete`, `issues/attachment-delete`, `prs/comment-delete`, and `prs/review-comment-delete` from the same report. - Operations get absorbed into aggregate counts ("34+ operations absent from the table"), which reads as coverage and names nothing a reader can act on. Rewriting the brief does not fix this. Three rewrites changed which defects appeared and never made the count stable. ## What to change 1. Say in the brief that the tool produces hypotheses, and that no output of it is a coverage statement. Ban aggregate findings: an operation is named or it is not reported. 2. Remove the completeness language. Once #572 lands, coverage comes from the generated inventory and the gate, and the brief points at that instead of asking for a sweep. 3. Require a per-operation verdict line for any list the tool reads, or an explicit statement that it did not enumerate. Silence and a total are both currently acceptable outputs and neither should be. 4. State that a finding is unconfirmed until read in the source by whoever acts on it. Two reproductions it reported were filed as #564 and #566 on its word. ## Acceptance criteria - The brief says what the tool is for and, in the same place, what it must never be trusted for, with the counting evidence above. - No output of the tool is presented as coverage anywhere. - After #572, coverage questions route to the generated inventory and the brief says so. - Any change to the brief is tested by running it before the change is committed. Three brief edits reached this branch untested, and two of the three did not do what their commit messages claimed.

hexajon added a new dependency

2026-09-21 23:40:14 +00:00

#563 Epic: the destructive surface has outgrown ADR 0029's classification table

hexajon added a new dependency

2026-09-21 23:40:17 +00:00

#572 the destructive surface has no enumerable source of truth

hexajon referenced this issue

2026-09-21 23:49:09 +00:00

chore(agents): track repo-scoped agent definitions, add the destruction-model auditor #574

hexajon referenced this issue from a commit

2026-09-21 23:49:10 +00:00

chore(agents): add a repo-scoped auditor for the destruction model