CI gate docs

The CI gate checks up to 50 URLs, usually of a pull request's preview deployment, and answers pass or fail against the site's last full scan. It is part of the Team and Agency plans of the web app; the web app docs cover sites and scans.

How a check passes

• The baseline is the site's latest finished manual or scheduled scan. Each checked page is compared with the baseline page that has the same path and query, so a preview host is judged against production.
• A check fails when a page has a critical or serious issue that its baseline page does not have, or when a page could not be checked. Moderate and minor issues are counted but never fail a check.
• Issues you ignored in the web app do not count.
• With no baseline yet (no finished scan of the site), every critical or serious issue counts as new.

API tokens

Create a token under Settings → API tokens in the web app, with a label such as the repository name. It starts with cbt_ and is shown once; we store only its hash. Store it as a secret in your CI, for example a GitHub repository secret named CURBCUT_TOKEN. Revoke it in settings when you no longer need it. A token sees only its own workspace's sites and checks.

Start a check

POST https://app.saffo.dev/api/v1/checks with the token as a bearer token and a JSON body:

• urls: 1 to 50 absolute http or https URLs, all on one origin.
• site (optional): the site's id or home URL. Without it, the site is the verified site whose origin matches the URLs. Pass it when the URLs are on a preview host.

curl -sS -X POST https://app.saffo.dev/api/v1/checks \
  -H "authorization: Bearer $CURBCUT_TOKEN" \
  -H "content-type: application/json" \
  -d '{"site": "https://www.example.com/", "urls": ["https://preview-42.example.com/", "https://preview-42.example.com/pricing/"]}'

Answer, HTTP 202:

{
  "id": "3f6c2b8e-0a4d-4f1b-9c7e-5d2a1b0c9e8f",
  "status": "queued",
  "url": "https://app.saffo.dev/scans/3f6c2b8e-0a4d-4f1b-9c7e-5d2a1b0c9e8f",
  "poll": "https://app.saffo.dev/api/v1/checks/3f6c2b8e-0a4d-4f1b-9c7e-5d2a1b0c9e8f"
}

Errors come as { "error": "<reason>" } with these statuses:

Get the result

GET https://app.saffo.dev/api/v1/checks/<id> with the same token. Poll every 10 seconds or so until status is done, failed or cancelled. pass is null until then, and false when the check failed or was cancelled.

curl -sS https://app.saffo.dev/api/v1/checks/3f6c2b8e-0a4d-4f1b-9c7e-5d2a1b0c9e8f \
  -H "authorization: Bearer $CURBCUT_TOKEN"

{
  "id": "3f6c2b8e-0a4d-4f1b-9c7e-5d2a1b0c9e8f",
  "status": "done",
  "pass": false,
  "url": "https://app.saffo.dev/scans/3f6c2b8e-0a4d-4f1b-9c7e-5d2a1b0c9e8f",
  "pages": 2,
  "totals": { "critical": 0, "serious": 1, "moderate": 3, "minor": 0 },
  "newIssues": [
    {
      "ruleId": "button-name",
      "impact": "serious",
      "page": "https://preview-42.example.com/pricing/",
      "selector": "#plans > button.toggle",
      "criteria": ["4.1.2"]
    }
  ],
  "pageErrors": []
}

newIssues lists at most 50 issues; the scan page at url has all of them. pageErrors lists pages that could not be checked, as { "page", "error" }. A failed check also has an error field.

Preview hosts

URLs on another origin than the site's (a preview deployment) are accepted only when that origin serves the site's verification token at /.well-known/curbcut-verification.txt, the same file as for site verification. Add the file to your build output so every preview has it, and pass site.

Quota per plan

The count resets on the first day of each calendar month (UTC). A check counts when it is queued.

GitHub Actions

The Curbcut action is not published to a public repository yet. Until it is, use one of the two ways below; both need only bash, curl and jq, which GitHub-hosted runners have. Store the token as the repository secret CURBCUT_TOKEN first.

A step in your workflow

This workflow checks two pages of a pull request's preview. Change the URLs to your own preview host.

name: Accessibility
on: pull_request

jobs:
  curbcut:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - name: Curbcut check
        env:
          CURBCUT_TOKEN: ${{ secrets.CURBCUT_TOKEN }}
          CURBCUT_SITE: https://www.example.com/
          CURBCUT_URLS: |
            https://preview-${{ github.event.number }}.example.com/
            https://preview-${{ github.event.number }}.example.com/pricing/
        run: |
          set -euo pipefail
          api=https://app.saffo.dev
          auth="$RUNNER_TEMP/curbcut-auth"
          # The token goes to curl through a file, so it never shows in the process list.
          (umask 077; printf 'authorization: Bearer %s\n' "$CURBCUT_TOKEN" > "$auth")
          body=$(jq -n --arg urls "$CURBCUT_URLS" --arg site "$CURBCUT_SITE" \
            '{urls: [$urls | splits("\\s+") | select(length > 0)]} + (if $site == "" then {} else {site: $site} end)')
          code=$(curl -sS -o "$RUNNER_TEMP/curbcut.json" -w '%{http_code}' -X POST "$api/api/v1/checks" \
            -H @"$auth" -H 'content-type: application/json' --data "$body")
          if [ "$code" != 202 ]; then
            echo "::error::Curbcut check not accepted (HTTP $code): $(jq -r '.error // empty' "$RUNNER_TEMP/curbcut.json")"
            exit 1
          fi
          poll=$(jq -r .poll "$RUNNER_TEMP/curbcut.json")
          result=$(jq -r .url "$RUNNER_TEMP/curbcut.json")
          echo "Curbcut check: $result"
          deadline=$(( $(date +%s) + 15 * 60 ))
          while :; do
            sleep 10
            curl -sS --fail -o "$RUNNER_TEMP/curbcut.json" -H @"$auth" "$poll"
            case "$(jq -r .status "$RUNNER_TEMP/curbcut.json")" in done|failed|cancelled) break ;; esac
            if [ "$(date +%s)" -ge "$deadline" ]; then echo "::error::Curbcut check timed out: $result"; exit 1; fi
          done
          jq -r '.newIssues[]? | "::error title=\(.ruleId) (\(.impact))::\(.page) \(.selector)"' "$RUNNER_TEMP/curbcut.json"
          jq -r '.pageErrors[]? | "::error title=Page not checked::\(.page): \(.error)"' "$RUNNER_TEMP/curbcut.json"
          if [ "$(jq -r '.pass' "$RUNNER_TEMP/curbcut.json")" != true ]; then
            echo "::error::Curbcut check failed: $result"
            exit 1
          fi
          echo "Curbcut check passed: $result"

A composite action in your repository

To reuse the check in several workflows, save this file as .github/actions/curbcut-check/action.yml in your repository. Its inputs and outputs are the ones the published action will have.

name: Curbcut check
description: Checks URLs with the Curbcut CI gate and fails on new critical or serious WCAG issues.
inputs:
  token:
    description: Curbcut API token (cbt_...). Pass it from a secret.
    required: true
  urls:
    description: URLs to check, one per line (1 to 50, one origin).
    required: true
  site:
    description: Site id or home URL; needed when the URLs are on a preview host.
    required: false
    default: ""
  api-url:
    description: Base URL of the web app.
    required: false
    default: https://app.saffo.dev
  timeout-minutes:
    description: How long to wait for the result.
    required: false
    default: "15"
outputs:
  pass:
    description: "true when the check passed, else false"
    value: ${{ steps.check.outputs.pass }}
  result-url:
    description: The check's page in the web app.
    value: ${{ steps.check.outputs.result-url }}
runs:
  using: composite
  steps:
    - id: check
      shell: bash
      env:
        CURBCUT_TOKEN: ${{ inputs.token }}
        CURBCUT_URLS: ${{ inputs.urls }}
        CURBCUT_SITE: ${{ inputs.site }}
        CURBCUT_API_URL: ${{ inputs.api-url }}
        CURBCUT_TIMEOUT_MINUTES: ${{ inputs.timeout-minutes }}
      run: |
        set -euo pipefail
        api="${CURBCUT_API_URL%/}"
        work=$(mktemp -d)
        trap 'rm -rf "$work"' EXIT
        (umask 077; printf 'authorization: Bearer %s\n' "$CURBCUT_TOKEN" > "$work/auth")
        body=$(jq -n --arg urls "$CURBCUT_URLS" --arg site "$CURBCUT_SITE" \
          '{urls: [$urls | splits("\\s+") | select(length > 0)]} + (if $site == "" then {} else {site: $site} end)')
        code=$(curl -sS -o "$work/resp" -w '%{http_code}' -X POST "$api/api/v1/checks" \
          -H @"$work/auth" -H 'content-type: application/json' --data "$body")
        if [ "$code" != 202 ]; then
          echo "::error::Curbcut check not accepted (HTTP $code): $(jq -r '.error // empty' "$work/resp")"
          echo "pass=false" >> "$GITHUB_OUTPUT"
          exit 1
        fi
        poll=$(jq -r .poll "$work/resp")
        result=$(jq -r .url "$work/resp")
        echo "result-url=$result" >> "$GITHUB_OUTPUT"
        echo "Curbcut check: $result"
        deadline=$(( $(date +%s) + CURBCUT_TIMEOUT_MINUTES * 60 ))
        while :; do
          sleep 10
          curl -sS --fail -o "$work/resp" -H @"$work/auth" "$poll"
          case "$(jq -r .status "$work/resp")" in done|failed|cancelled) break ;; esac
          if [ "$(date +%s)" -ge "$deadline" ]; then
            echo "::error::Curbcut check timed out: $result"
            echo "pass=false" >> "$GITHUB_OUTPUT"
            exit 1
          fi
        done
        jq -r '.newIssues[]? | "::error title=\(.ruleId) (\(.impact))::\(.page) \(.selector)"' "$work/resp"
        jq -r '.pageErrors[]? | "::error title=Page not checked::\(.page): \(.error)"' "$work/resp"
        pass=$(jq -r 'if .pass == true then "true" else "false" end' "$work/resp")
        echo "pass=$pass" >> "$GITHUB_OUTPUT"
        if [ "$pass" != true ]; then
          echo "::error::Curbcut check failed: $result"
          exit 1
        fi
        echo "Curbcut check passed: $result"

Use it in a workflow:

name: Accessibility
on: pull_request

jobs:
  curbcut:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Curbcut check
        id: curbcut
        uses: ./.github/actions/curbcut-check
        with:
          token: ${{ secrets.CURBCUT_TOKEN }}
          site: https://www.example.com/
          urls: |
            https://preview-${{ github.event.number }}.example.com/
            https://preview-${{ github.event.number }}.example.com/pricing/
      - name: Result
        if: always()
        run: echo "pass=${{ steps.curbcut.outputs.pass }} ${{ steps.curbcut.outputs.result-url }}"

The job fails when the check fails, fails to start or times out, and each new issue shows as an annotation on the run. Wait for your preview deployment to finish before this step, for example with needs: on the deploy job.

Limits

Automated checks find only part of all WCAG failures. A passing check means no new critical or serious issue was found on the URLs checked, not that the pages are accessible.