sdk-quality-gate

---
name: sdk-quality-gate
description: Use before declaring any Supabase C# SDK change done or opening a PR, and as the Verify step of every flow. Runs the committed scripts/quality-gate/gate.sh script — the mechanized gauntlet (build/analyzers with a warning baseline, format on changed files, tests, a line-coverage baseline and E2E/acceptance tests, plus public-API diff and vulnerability-scan signals) — and reports its verdict. This is the deterministic "is it done" check; do not report a change as done without a PASS.
---

# Skill: SDK quality gate

The gauntlet is a program, not a procedure. Run it and report what it says.

The script lives at `scripts/quality-gate/gate.sh` — committed alongside the code
so contributors and CI run the identical command, not just agents. Paths below
are relative to the repository root; the script itself is location-independent.

```
scripts/quality-gate/gate.sh <package>              # full gate — before a PR
scripts/quality-gate/gate.sh <package> --fast       # inner loop only
scripts/quality-gate/gate.sh <package> --mutation   # evaluate a Stryker report
```

There are two modes. `--fast` is the inner loop only (build, format, tests) — the
fast local red/green cycle; the default is the full gate and adds security, the
public-API check, E2E and a line-coverage baseline. `<package>` is a directory —
e.g. `packages/Gotrue`, or `packages` for the whole solution. Run with
`bash <path>` if the executable bit is unset.

**Mutation runs in its own workflow, on every PR.** `mutation.yml` runs Stryker
`--since:origin/master` on changed packages, then `gate.sh packages --mutation`
fails on any survivor on a changed line, and SKIPs when every mutant on the changed
lines failed to compile. To check locally before pushing:

```
(cd packages/<Name>/<Name>.Tests && STRYKER_MUTATION=true dotnet stryker --since:origin/master)
scripts/quality-gate/gate.sh packages/<Name> --mutation
```

In a `git worktree`, `--since` finds no changes: pass the changed files with
`-m '**/<File>.cs'` instead. Never trust a run that tested zero mutants.

**CI runs the same command.** The `Build and Test` workflow
(`.github/workflows/build-and-test.yml`) invokes `scripts/quality-gate/gate.sh`
on every push and pull request, so a local PASS and a CI PASS mean the same
thing. The blocking-stage verdict here is the one that gates the merge.

## Reading the verdict

| Verdict | Exit | Meaning |
|---|---|---|
| `PASS` | 0 | Blocking stages green. Report done; surface the signals. |
| `PARTIAL` | 0 | `--fast` only. **Not** sufficient to declare done. |
| `INCOMPLETE` | 2 | A blocking stage could not run. Unverified ≠ verified. |
| `FAIL` | 1 | A blocking stage failed. |

Blocking stages `[B]` decide the verdict: build/analyzers, format, tests,
line coverage, public-API declared, **and E2E/acceptance**. A failing E2E test blocks the merge exactly like a failing unit
test — there is no green build with a red test.

**Test execution differs by mode.** `--fast` runs the inner loop only
(`TestCategory!=E2E`) — no coverage measured, verdict is `PARTIAL`. The full gate,
when the local/CI Supabase stack is reachable, runs every test **in one
unfiltered pass** — unit, contract and E2E together, under stage id `2` — so a
red unit test no longer prevents E2E from executing in the same run (they either
both ran, or neither did). That same run is also the coverage source: `2b`
measures line coverage across the whole suite, not just the inner loop, so
"coverage" always means the full picture. **Stage `7` doesn't appear in the
report at all when this happens** — id `2`'s own per-package rows (labeled
"Tests (Unit + Contract + E2E)") already carry the full outcome, so a separate
E2E row would only restate it, and imprecisely at that (a package with zero E2E
tests has nothing to "fold in"). If the stack is unreachable, the gate falls back
to running the inner loop alone for local feedback, and both E2E (`7`) and
coverage (`2b`) SKIP with a reason — "full" coverage isn't measurable without the
E2E half of the suite, and a check that couldn't run must never read as passed.
Either way, a stack that can't be reached holds the gate at `INCOMPLETE`.

Dropped intentionally: a package with zero `[TestCategory("E2E")]` tests used to
get its own non-blocking signal row under `7` ("no E2E tests in this package").
Now that E2E runs folded into the unfiltered stage `2`, that distinction isn't
separately observable without a second test-discovery pass, so it's gone — not a
regression to chase, just a note so its absence isn't a surprise.

Signal stages `[s]` never fail the build: the **public-API diff**, because the
maintainer is the merge gate on breaking changes — a break may be intended — and
the **dependency-vulnerability scan**, because a vulnerability may have no fix
available (QUALITY_RUBRIC §4).

Stages are skipped only when an earlier failure makes them impossible or
meaningless — a failed build blocks format, tests, E2E and coverage; a stack
that can't be reached blocks E2E and coverage specifically (tests still run via
the inner-loop fallback). Nothing else suppresses a stage, and anything not run
is still reported as `SKIP` with its reason.

## Reporting

Paste the summary table verbatim. Never paraphrase a stage result, and never
report a stage as passing when the script recorded `SKIP`. Detail is in
`<package>/.gate/report.json` and the per-stage logs it references.

Add the triage a number alone doesn't carry:

- **Public-API diff** — additive, or a break needing sign-off + a major plan +
  `[Obsolete]` + a `MIGRATION_vN.md` entry.

## On failure — fix the cause, not the check

Not acceptable repairs:

- `#pragma warning disable`, or lowering a severity in `.editorconfig`
- re-categorising a test so a filter skips it
- deleting or `[Ignore]`-ing a red test
- retrying or quarantining a flaky E2E — flakiness is a design defect
- hand-editing `.gate-baseline.json` the wrong way (warnings up, coverage down)
  to make a stage pass

If a blocking stage fails for a reason outside the change's scope, stop and
report it. Do not work around it and continue.

## Baselines

`<package>/.gate-baseline.json` is committed and holds the discovered project
paths, the warning count per code, and line coverage. The warning baseline only
moves *down* (fewer is better); coverage only *up* (more is better,
`coverage.hermeticLine`, a single percentage). Either direction, moving the number the
wrong way means editing the file by hand, so the regression appears in code review.

**Writing the baseline is opt-in.** A plain `gate.sh` run is read-only — it reads
the baseline to produce a verdict but never touches the file, so a contributor's
run can't silently rewrite a committed baseline. Pass `--overwrite-baseline` to
create it (a package's first run) or to update it. On CI this is a post-merge
maintainer concern: PR runs stay read-only, and only the push-to-master run
passes `--overwrite-baseline` and commits any improvement back to master.

**Coverage is compared with a ~1% tolerance band, not an exact match.** Even
hermetic (unit+contract) coverage isn't byte-reproducible on a loaded CI runner —
async continuations (retry/backoff `Task.Delay`, cancellation races, background
timers) get scheduled differently run to run, flipping a few covered lines. The
band absorbs that jitter symmetrically: a drop within it passes, a rise within it
doesn't move the baseline (so it never locks onto a lucky peak the next run can't
reach), and a real regression — an untested method, far more than 1% — still fails.

**Coverage has no ceiling.** It must always be at or above the best ever
recorded — never capped at, say, 95%, because a cap reopens exactly the
regression window the baseline exists to close: once at a cap, a large chunk of
fully untested new code could land without the aggregate dipping below it,
especially against a large existing denominator. If coverage nears 100% and the
remaining gaps are genuinely low-value to test (a guard clause, an unreachable
`default:` arm), the answer is `[ExcludeFromCodeCoverage]` on those specific
lines — not a gate-level threshold. That's a per-line, reviewed-in-diff opt-out;
a reviewer sees it and can push back, unlike a global cap that quietly loosens
the bar for everyone forever.

Optional keys: `formatBaseRef` (default: `origin/HEAD`, then `main`/`master`)
for the changed-file comparison, and `e2eHealthUrl` if the local stack isn't on
the default port.

When a `--overwrite-baseline` run reports `baseline lowered` or `baseline raised`,
commit the file. A read-only run (the default) never emits those — it won't have
written anything to commit.

## What the script cannot check

These stay human judgment. Walk them; don't recall them:

- [ ] Behavior asserted through the public surface; testability friction fixed by
      changing the design, not by exposing internals.
- [ ] Approved wire-shape snapshots are *right* — the contract tests only prove
      the current output still matches them, not that the approved bytes are correct.
- [ ] `CancellationToken` genuinely honored through the call path, not just
      accepted at the entry point.
- [ ] Orchestration reads as named intent; failures surface as the package's
      typed exception (v8 error model, CONVENTIONS §9).
- [ ] Public surface minimal and documented; entry points carry an `<example>`.
- [ ] Parity deviations noted in the PR description.
- [ ] Commits follow `type(scope): description`. The scope names the affected
      product — `auth`, `storage`, `postgrest`, `realtime`, `functions`, `core`,
      `supabase` — and is omitted for cross-cutting changes (append `!` before
      the colon for breaking changes, e.g. `feat(functions)!:`).

Every item that becomes mechanizable — an analyzer, an architecture test, a
`TestConventions` rule — should move out of this list and into the script. The
list is meant to shrink.