madt_finish/merge masks real Gitea merge errors as a "transient flake" and auto-retries non-retryable ones

hexajon commented

2026-08-27 01:06:11 +00:00

Owner

Problem

On a merge failure, madt_finish and madt_prs action=merge report a generic message like:

merge failed: Gitea was temporarily unavailable ("try again later") after 4 attempts - the PR is ready to merge (mergeable=true), so this is a transient server flake: retry 'madt_finish' or 'madtea pr merge'

and auto-retry (4 attempts). But the underlying Gitea error is specific and actionable, and several are NOT retryable:

This branch is already included in the target branch. There is nothing to merge.
There is no key available to sign this commit.
This branch is out-of-date with the base branch.

Retrying these never succeeds, and the "transient flake" relabel hides the real cause from the user.

Concrete incident

A madt_finish returned HTTP 500 on merge - but the merge had actually SUCCEEDED server-side (the commit and its Closes #N trailers landed on the default branch; the linked issues closed). madtea reported the finish as failed. A retry then created a REDUNDANT PR for a branch already in the target. Merging that PR failed with the real error "nothing to merge", which madtea again reported as a generic "try again later" flake and retried.

Asks

Surface the real Gitea error message verbatim instead of relabeling it a "transient flake".
Classify retryable vs non-retryable errors. Do NOT auto-retry non-retryable ones (nothing-to-merge, no-signing-key, out-of-date-with-base, 4xx generally).
After a 5xx during merge, re-check whether the merge actually landed (PR merged, or branch already included in base) BEFORE reporting failure or retrying - to avoid creating a duplicate PR on an already-merged branch.

## Problem On a merge failure, `madt_finish` and `madt_prs action=merge` report a generic message like: > merge failed: Gitea was temporarily unavailable ("try again later") after 4 attempts - the PR is ready to merge (mergeable=true), so this is a transient server flake: retry 'madt_finish' or 'madtea pr merge' and auto-retry (4 attempts). But the underlying Gitea error is specific and actionable, and several are NOT retryable: - `This branch is already included in the target branch. There is nothing to merge.` - `There is no key available to sign this commit.` - `This branch is out-of-date with the base branch.` Retrying these never succeeds, and the "transient flake" relabel hides the real cause from the user. ## Concrete incident A `madt_finish` returned HTTP 500 on merge - but the merge had actually SUCCEEDED server-side (the commit and its `Closes #N` trailers landed on the default branch; the linked issues closed). madtea reported the finish as failed. A retry then created a REDUNDANT PR for a branch already in the target. Merging that PR failed with the real error "nothing to merge", which madtea again reported as a generic "try again later" flake and retried. ## Asks 1. Surface the real Gitea error message verbatim instead of relabeling it a "transient flake". 2. Classify retryable vs non-retryable errors. Do NOT auto-retry non-retryable ones (nothing-to-merge, no-signing-key, out-of-date-with-base, 4xx generally). 3. After a 5xx during merge, re-check whether the merge actually landed (PR merged, or branch already included in base) BEFORE reporting failure or retrying - to avoid creating a duplicate PR on an already-merged branch.

hexajon referenced this issue

2026-08-27 05:52:55 +00:00

fix(finish): surface real merge errors verbatim, classify non-retryable, re-check landed state after a 5xx #463

hexajon referenced this issue from a commit

2026-08-27 05:52:55 +00:00

fix(finish): surface real merge errors verbatim, classify non-retryable, re-check landed state after a 5xx (#456)

hexajon added the

status/needs-decision

label

2026-08-27 05:53:20 +00:00

hexajon commented

2026-08-27 05:53:21 +00:00

Author

Owner

Landed in PR #463 - all three asks: verbatim surfacing (the real Gitea body, no "transient flake" relabel), non-retryable classification (already-included / nothing-to-merge, no-signing-key, and out-of-date-with-base break immediately, no retry), and the post-5xx landed re-check (a merge that landed server-side reports success and opens no duplicate PR; the check gates every retry).

One refinement I want your call on. I put "out-of-date with the base branch" in the non-retryable set per the list in this issue, so it now fails verbatim. But the existing behind-base 405 path returns the same verbatim cause PLUS a ReconcileSteer - the copy-pasteable update-branch-and-remerge remedy - and out-of-date has exactly that remedy: updating the branch is the fix. So today the out-of-date failure is less actionable than the behind-base one.

I recommend adding the ReconcileSteer to the out-of-date verbatim failure: keep it non-retryable (no blind retry, no silent auto-merge of a tree the local gate did not validate) but hand the caller the update-branch remedy, so it reaches parity with the behind-base path. The other non-retryable reasons (nothing-to-merge, no-signing-key) stay bare - no update remedy applies to them. Say the word and I will add it.

(The alternative - route out-of-date to auto-update-and-remerge like a transient - reverses the non-retryable classification here and would silently merge a tree the local gate did not validate. I would not, but the call is yours.)

Landed in PR #463 - all three asks: verbatim surfacing (the real Gitea body, no "transient flake" relabel), non-retryable classification (already-included / nothing-to-merge, no-signing-key, and out-of-date-with-base break immediately, no retry), and the post-5xx landed re-check (a merge that landed server-side reports success and opens no duplicate PR; the check gates every retry). One refinement I want your call on. I put "out-of-date with the base branch" in the non-retryable set per the list in this issue, so it now fails verbatim. But the existing behind-base 405 path returns the same verbatim cause PLUS a ReconcileSteer - the copy-pasteable update-branch-and-remerge remedy - and out-of-date has exactly that remedy: updating the branch is the fix. So today the out-of-date failure is less actionable than the behind-base one. I recommend adding the ReconcileSteer to the out-of-date verbatim failure: keep it non-retryable (no blind retry, no silent auto-merge of a tree the local gate did not validate) but hand the caller the update-branch remedy, so it reaches parity with the behind-base path. The other non-retryable reasons (nothing-to-merge, no-signing-key) stay bare - no update remedy applies to them. Say the word and I will add it. (The alternative - route out-of-date to auto-update-and-remerge like a transient - reverses the non-retryable classification here and would silently merge a tree the local gate did not validate. I would not, but the call is yours.)

hexajon referenced this issue

2026-08-27 07:31:37 +00:00

feat(finish): schedule auto-merge-on-green when a merge is gated on required CI checks #464

hexajon referenced this issue

2026-08-27 17:18:59 +00:00

hooks/tests: release-gate.test.sh still scores a crashed gate as "allow" - and the audit scope is an open question #417

hexajon commented

2026-09-08 19:15:45 +00:00

Author

Owner

Decided: add the ReconcileSteer remedy to the out-of-date-with-base failure, keeping it non-retryable.

The out-of-date failure now returns the verbatim Gitea cause PLUS the copy-pasteable update-branch-and-remerge block, matching the behind-base 405 path (internal/service/workflow/merge.go, where behind-base already attaches ReconcileSteer). It stays in the non-retryable set (internal/gitea/client_errors.go): no blind retry, and no auto-merge of a tree the local gate did not validate. Nothing-to-merge and no-signing-key stay bare, since no update remedy applies to them.

The three original asks landed in PR #463; this closes the one refinement. Ready to build.

Decided: add the ReconcileSteer remedy to the out-of-date-with-base failure, keeping it non-retryable. The out-of-date failure now returns the verbatim Gitea cause PLUS the copy-pasteable update-branch-and-remerge block, matching the behind-base 405 path (`internal/service/workflow/merge.go`, where behind-base already attaches ReconcileSteer). It stays in the non-retryable set (`internal/gitea/client_errors.go`): no blind retry, and no auto-merge of a tree the local gate did not validate. Nothing-to-merge and no-signing-key stay bare, since no update remedy applies to them. The three original asks landed in PR #463; this closes the one refinement. Ready to build.

hexajon removed the

status/needs-decision

label

2026-09-08 19:15:46 +00:00

hexajon referenced this issue from a pull request that will close it,

2026-09-09 22:09:45 +00:00

fix(finish): attach the reconcile steer to the out-of-date-with-base merge failure, still non-retryable (#456) #475

hexajon referenced this issue from a commit

2026-09-09 22:09:46 +00:00

fix(finish): attach the reconcile steer to the out-of-date-with-base merge failure, still non-retryable (#456)

hexajon closed this issue

2026-09-09 22:09:48 +00:00