Was that security fix
actually a fix?
Fix Gate checks any security-fix pull request, whoever or whatever wrote it, on four questions, and posts the answers as a check run. Rule-based, on your own runner, one workflow file.
Why
Patch generation has outpaced patch verification.
In August 2026, 1Password tested 6,080 security patches written by two frontier models: 26% were fully correct, and 54% left the vulnerability in place, introduced a new one, or both (their write-up). GitHub counts an Autofix as successful only when the alert is gone, no new alert appeared, the file still parses and the tests still pass. Fix Gate applies that same bar to every fix pull request, and reports each of the four parts separately.
We built it because we needed it ourselves: when we read all 162 fixes our own engine proposed on eleven public repositories, 95 were wrong, and none had broken a file's syntax. A fix that compiles can still be wrong. The gate is how we check our own work, now open to anyone's.
What it checks
Four questions. Four honest answers.
Each row of the check run is pass, fail, nothing to check, or not run. A check that could not run is shown as not run. It is never counted as a pass.
Targeted finding gone
Railo's rules run over the base and head versions of every changed file. A finding reported at base, inside the lines the pull request changed, must no longer match at head. Name the finding with the rule and file inputs, or let the gate infer it.
Limit: Findings from other tools (CodeQL, Snyk, a pentest) are not re-run; the gate says so instead of guessing.
Nothing new in the changed lines
A finding at head with no counterpart at base, sitting in the changed lines, fails the check. A pre-existing finding that only shifted lines does not.
Limit: Only the changed files are scanned. The rest of the repository is not the pull request's work.
Changed files still parse
Python, JavaScript, TypeScript, Java and PHP are parse-checked. A file that already failed to parse at base is not held against the change.
Limit: Go, Ruby and C# are reported as not checked, not as passed.
Tests did not newly fail
Railo's container has none of your project's dependencies, so this comes from your own test job: pass its result in as tests_result.
Limit: Without that input the row says not run. It never says passed.
Setup
One file. Under five minutes.
- 1
Add the file.
Commit it as .github/workflows/fix-gate.yml. It uses the token every workflow already has.
- 2
Open a pull request that fixes a finding.
From Copilot Autofix, Snyk, an editor assistant, or a person. Fix Gate scans the changed files at base and at head.
- 3
Read the check.
A check run named Fix Gate shows the four rows, with the rule and line of anything still present or new.
name: Fix Gate
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
checks: write
jobs:
fix-gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
persist-credentials: false
- uses: IWEBai/fix-gate-action@v0
To fill the tests row, run your test job first and add needs: [test] with tests_result: ${{ needs.test.result }} under the Fix Gate step.
Inputs
| Input | What it does |
|---|---|
| rule | Rule id of the finding the pull request fixes. With file, pins the targeted finding instead of inferring it. |
| file | Path of the file the finding is in. |
| line | Line of the finding at base, to pick between several in one file. |
| require_resolved | true: fail when no base finding was resolved at all, so a non-fix cannot pass as a fix. Default false. |
| tests_result | Your test job's outcome: success, failure or skipped. Pass ${{ needs.test.result }}. |
| timeout | Seconds for the two scans. Default 120. A timeout is reported as not run. |
| fail_on_failure | false: a failed gate reports but does not fail the job. Default true. |
What it is not
Three lines we do not cross.
Runs on your runner.
The scans happen inside your GitHub Actions job. No file content leaves it. One usage ping carries the repository name, the mode and counts; set RAILO_TELEMETRY=0 to turn it off.
Reads, never writes.
The workflow needs contents: read and checks: write. Fix Gate opens nothing, pushes nothing and changes nothing in your repository.
Rules, not a model.
The verdict comes from running the same rules twice and comparing, plus a parser. No language model reads your code or writes the verdict.
Fix Gate is not a scanner replacement and not a test suite. It judges one pull request against the base it branched from, with Railo's rules and a parser. It is in beta: the rules it carries are the ones behind Railo's published failure-rate study, and they will miss things. The check run always shows the rule and line behind a fail, so a reviewer can judge it.
Gate says not verified?
Railo closes security findings as reviewed pull requests, each one checked the same way, with an evidence pack at the end. One week, fixed price.