sixfold-space/madtea

SHA256

Fork 0

test: fail when hand-written madt_ call strings name an unregistered tool or action #860

Merged

hexajon merged 2 commits from issue-706-call-strings-drift into main

2026-10-21 03:53:34 +00:00

hexajon commented

2026-10-21 03:53:32 +00:00

Owner

Summary

What was wrong

Hand-written madt_ call strings had no drift guard. The hook scripts in hooks/scripts/ and the Go string literals of the MCP server (instructions, help topics, prompts, refusal text) print calls such as madt_worktrees(action="list"). Nothing checked those strings against the tools the server serves. A renamed or removed tool or action left stale text behind.

The first version of the guard found two stale strings and one removed tool name:

The release prompt in internal/mcp/prompts.go passed action="get" to madt_status. That tool takes no action argument.
The whoami help topic in internal/mcp/tools_help.go named madt_auth_permissions, a tool that no longer exists.

It also showed a gap in how I read the tool surface. madt_log accepts action=list, but the guard took its action sets from two registry maps, so it treated madt_log as a tool with no action argument. That made the valid madt_log(action="list") strings in hooks/scripts/check-git-cli.sh and hooks/scripts/check-git-local.sh look stale.

What changed

New internal/mcp/call_string_drift_test.go. It scans every file in hooks/scripts/*.sh and the string literals (not the comments) of the non-test Go files in internal/mcp/. It checks each madt_ reference against the live server with every domain enabled:
- scanCallStrings and scanGoStringLiterals find the references.
- liveCallSurface reads the served input schema of each tool. No action property means no action argument. An action enum gives the action set. If there is no enum, it falls back to the action manifest that the tool records in the registry. If there is neither, staleCallStrings reports every call string to that tool, because it cannot check them.
- staleCallStrings reports a reference to an unregistered tool, an action on a tool with no action argument, and an unknown action.
internal/mcp/tools_localgit.go: madt_log now publishes its action set. logActions holds the set, logInputSchema adds it as the enum of the action property, and the handler checks the action against the same slice.
internal/mcp/prompts.go: the release prompt now says "Call madt_status" with no action argument.
internal/mcp/tools_help.go: the whoami topic no longer names madt_auth_permissions.
The madt_log(action="list") hook strings stay as they were, because they are valid.

Tests

TestCallStringDrift fails when any scanned text names a tool or action that the live server does not register.
TestCallStringDriftBites plants stale strings (unknown tool, unknown action in the shell, Go and prose forms, an action on a tool with no actions, an unknown action on madt_log, a tool with an open action set, and a stale string in a Go literal) and checks that the guard reports each one. It also checks that valid text, madt_* and a bare madt_ prefix produce no report.
TestCallStringAllowlistIsCurrent fails when an entry in callStringNotATool no longer appears in any scanned text, or names a tool that is now registered.

What this does not do

This covers the first two acceptance criteria of #706: a gate check for stale tool and action names in the hook scripts and the MCP text. The third criterion, the worktree guard printing a call that every MCP client can make, is a wording decision. I have not made it here. Refs #706.

./scripts/gate.sh passes on this commit.

Refs #706

## Summary ## What was wrong Hand-written `madt_` call strings had no drift guard. The hook scripts in `hooks/scripts/` and the Go string literals of the MCP server (instructions, help topics, prompts, refusal text) print calls such as `madt_worktrees(action="list")`. Nothing checked those strings against the tools the server serves. A renamed or removed tool or action left stale text behind. The first version of the guard found two stale strings and one removed tool name: - The release prompt in `internal/mcp/prompts.go` passed `action="get"` to `madt_status`. That tool takes no action argument. - The `whoami` help topic in `internal/mcp/tools_help.go` named `madt_auth_permissions`, a tool that no longer exists. It also showed a gap in how I read the tool surface. `madt_log` accepts `action=list`, but the guard took its action sets from two registry maps, so it treated `madt_log` as a tool with no action argument. That made the valid `madt_log(action="list")` strings in `hooks/scripts/check-git-cli.sh` and `hooks/scripts/check-git-local.sh` look stale. ## What changed - New `internal/mcp/call_string_drift_test.go`. It scans every file in `hooks/scripts/*.sh` and the string literals (not the comments) of the non-test Go files in `internal/mcp/`. It checks each `madt_` reference against the live server with every domain enabled: - `scanCallStrings` and `scanGoStringLiterals` find the references. - `liveCallSurface` reads the served input schema of each tool. No `action` property means no action argument. An `action` enum gives the action set. If there is no enum, it falls back to the action manifest that the tool records in the registry. If there is neither, `staleCallStrings` reports every call string to that tool, because it cannot check them. - `staleCallStrings` reports a reference to an unregistered tool, an action on a tool with no action argument, and an unknown action. - `internal/mcp/tools_localgit.go`: `madt_log` now publishes its action set. `logActions` holds the set, `logInputSchema` adds it as the enum of the `action` property, and the handler checks the action against the same slice. - `internal/mcp/prompts.go`: the release prompt now says "Call madt_status" with no action argument. - `internal/mcp/tools_help.go`: the `whoami` topic no longer names `madt_auth_permissions`. - The `madt_log(action="list")` hook strings stay as they were, because they are valid. ## Tests - `TestCallStringDrift` fails when any scanned text names a tool or action that the live server does not register. - `TestCallStringDriftBites` plants stale strings (unknown tool, unknown action in the shell, Go and prose forms, an action on a tool with no actions, an unknown action on `madt_log`, a tool with an open action set, and a stale string in a Go literal) and checks that the guard reports each one. It also checks that valid text, `madt_*` and a bare `madt_` prefix produce no report. - `TestCallStringAllowlistIsCurrent` fails when an entry in `callStringNotATool` no longer appears in any scanned text, or names a tool that is now registered. ## What this does not do This covers the first two acceptance criteria of #706: a gate check for stale tool and action names in the hook scripts and the MCP text. The third criterion, the worktree guard printing a call that every MCP client can make, is a wording decision. I have not made it here. Refs #706. `./scripts/gate.sh` passes on this commit. Refs #706

hexajon added 2 commits

2026-10-21 03:53:32 +00:00

test: fail when hand-written madt_ call strings name an unregistered tool or action bb03f92920

Add a drift guard that scans the hook scripts and the string literals of
the MCP server package for madt_ tool names and action arguments, and
checks each one against the live server with every domain enabled.

The guard found four stale strings, now fixed:
- the log steer and advice in check-git-cli.sh and check-git-local.sh
  passed action=list to madt_log, which takes no action argument;
- the release prompt passed action="get" to madt_status, which takes no
  action argument;
- the whoami help topic named madt_auth_permissions, a removed tool.

Refs #706

fix: read call-string action sets from the served tool schemas

drift / govulncheck (pull_request) Has been skipped

Details

drift / forgejo-smoke (pull_request) Has been skipped

Details

drift / surface-diff (pull_request) Successful in 51s

Details

drift / govulncheck (push) Has been skipped

Details

drift / surface-diff (push) Successful in -39s

Details

drift / forgejo-smoke (push) Has been skipped

Details

ci / gate (push) Successful in 19m11s

Details

ci / gate (pull_request) Failing after 45m54s

Details

05822ab2f2

The drift guard took the action sets from two registry maps, so it treated
madt_log as a tool with no action argument. madt_log accepts action=list.

The guard now reads each served input schema: no action property means no
action argument, an action enum gives the set, and a recorded manifest is the
fallback. A tool that takes an action but publishes no set is reported.
madt_log now publishes its action set as the enum of its action property.

Restore the madt_log(action="list") hook strings, which were valid.

Refs #706

hexajon merged commit 05822ab2f2 into main

2026-10-21 03:53:34 +00:00

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

sixfold-space/madtea!860

No description provided.

Rows
Columns