ci: the forgejo-smoke job has never run #542

New issue

Open

opened 2026-09-17 03:06:00 +00:00 by hexajon · 0 comments

hexajon commented

2026-09-17 03:06:00 +00:00

Owner

drift.yml's forgejo-smoke job spins up Forgejo in a sidecar container, provisions an admin and token, and runs roughly 60 CLI commands end to end. It is the only check that exercises madtea against a live forge rather than against a vendored spec.

It is gated on:

if: ${{ vars.RUN_FORGEJO_SMOKE == 'true' && github.event_name != 'pull_request' }}

This repository defines no Actions variables at all, so the first clause is false on every event. The job is skipped at evaluation time and never queued. As far as I can tell it has never executed here.

Why it matters

Every other forge check compares madtea against a static artifact: the vendored OpenAPI specs in docs/api-specs/. forgejo-smoke is the only one that answers "does this endpoint actually work", as opposed to "is this endpoint documented". Its own comment makes that distinction: it catches "endpoint that's documented but doesn't actually work on Forgejo".

That is a real and distinct class of failure, and nothing currently covers it.

It also compounds #538: the vendored Gitea spec turned out to be a minor version stale with 8 missing paths, and the live check that might have surfaced that class of drift was switched off.

Note on the gate itself

The gate is deliberate and the reasoning is sound. The job needs a host Docker socket to provision the first admin user, which a public runner must not provide, and the != pull_request clause keeps a socket-privileged job unreachable from fork PRs. None of that should change.

The question is only whether the variable gets set on an instance whose runner can support it, or whether the job should be removed rather than sit permanently dead.

Ask

Decide one of:

Set RUN_FORGEJO_SMOKE on an instance whose runner exposes the socket, and confirm the job actually runs and passes.
Remove the job, and record that live-forge verification is not covered.

A job that is permanently skipped is worse than no job, because the workflow reads as though live verification exists.

Acceptance

Either the job has demonstrably run and passed at least once, or it is gone.
If it stays, a permanently-skipped state is visible rather than silent.

Related: #538 (vendored specs are not validated against a live install), #540 (a failing scheduled run notifies nobody).

`drift.yml`'s `forgejo-smoke` job spins up Forgejo in a sidecar container, provisions an admin and token, and runs roughly 60 CLI commands end to end. It is the only check that exercises madtea against a live forge rather than against a vendored spec. It is gated on: ```yaml if: ${{ vars.RUN_FORGEJO_SMOKE == 'true' && github.event_name != 'pull_request' }} ``` This repository defines **no Actions variables at all**, so the first clause is false on every event. The job is skipped at evaluation time and never queued. As far as I can tell it has never executed here. ## Why it matters Every other forge check compares madtea against a **static** artifact: the vendored OpenAPI specs in `docs/api-specs/`. `forgejo-smoke` is the only one that answers "does this endpoint actually work", as opposed to "is this endpoint documented". Its own comment makes that distinction: it catches "endpoint that's documented but doesn't actually work on Forgejo". That is a real and distinct class of failure, and nothing currently covers it. It also compounds #538: the vendored Gitea spec turned out to be a minor version stale with 8 missing paths, and the live check that might have surfaced that class of drift was switched off. ## Note on the gate itself The gate is deliberate and the reasoning is sound. The job needs a host Docker socket to provision the first admin user, which a public runner must not provide, and the `!= pull_request` clause keeps a socket-privileged job unreachable from fork PRs. None of that should change. The question is only whether the variable gets set on an instance whose runner can support it, or whether the job should be removed rather than sit permanently dead. ## Ask Decide one of: * Set `RUN_FORGEJO_SMOKE` on an instance whose runner exposes the socket, and confirm the job actually runs and passes. * Remove the job, and record that live-forge verification is not covered. A job that is permanently skipped is worse than no job, because the workflow reads as though live verification exists. ## Acceptance * Either the job has demonstrably run and passed at least once, or it is gone. * If it stays, a permanently-skipped state is visible rather than silent. Related: #538 (vendored specs are not validated against a live install), #540 (a failing scheduled run notifies nobody).

hexajon added the

bug

label

2026-09-17 03:06:00 +00:00

hexajon referenced this issue

2026-09-17 04:50:00 +00:00

COMPAT.md asserts platform behaviour that nothing verifies, and the claims are duplicated into shipped strings #543

hexajon referenced this issue

2026-09-17 05:51:01 +00:00

Epic: madtea's external contracts are asserted, not verified #544

hexajon added a new dependency

2026-09-17 05:51:05 +00:00