Your prompt changed. Did quality?
Drift-CI runs your LLM evals on every pull request, fails the build on regressions, and tracks them to resolution.
MIT-licensed core · 7 providers · works with any CI
how it works
From pull request to resolved
Every PR runs your eval suite
The GitHub Action sends each case to your model and scores it against its committed baseline.
Analyzing 140 cases across 6 providers…
[████████████████████] 100%
✗ Regression: threshold exceeded in 1 case. Exiting 1 — merge blocked.
❌ 1 regression detected
A regression fails the build
When a score drops past your threshold the check fails, and a PR comment lists exactly which cases dropped and by how much.
An issue opens, ready to triage
Every regressed case becomes a tracked issue. Assign it, discuss it, and resolve it — it reopens if the case regresses again.
Fix it, and watch it recover
Compare runs case by case to confirm the fix, and watch quality and cost trend back over time.
Set up in two minutes
- 1Create a project and an ingest token
- 2Add DRIFT_PROJECT_TOKEN to your repo secrets
- 3Add the Action — your next PR is gated
- uses: Drift-CI/drift-ci@v1
with:
provider: anthropic
api-key: ${{ secrets.ANTHROPIC_API_KEY }}
dashboard-url: https://drift-ci.com
dashboard-token: ${{ secrets.DRIFT_PROJECT_TOKEN }}Built on an open-source engine
providers
Every provider, one suite
Anthropic, OpenAI, Azure, Bedrock, Gemini, Vertex AI, and Ollama. Swap models and compare behaviour without rewriting a single case.
evaluators
An evaluator for every check
exact-match, json-schema, cosine-similarity, LLM-judge, refusal-detection, rubric-checklist, and safety-classifier — chained per case.
baselines
Baselines as code, not hidden state
One git-committed JSON file per case, with secrets redacted before anything touches disk. Review drift as a diff, in your PR.
ci-gate
A real merge gate
A regression exits 1 and fails the build. A transient-error storm exits 2 — never mistaken for a regression. Deterministic by design.
Free and open source. Hosted when you scale.
The core engine is MIT-licensed. The hosted dashboard is free for one developer; Team adds analytics, cost tracking and tracked regressions.
Free
$0/forever
The hosted dashboard for one developer. Point the CLI or GitHub Action at it and keep every run.
- Included:Hosted dashboard & run history
- Included:Ingest from the CLI & GitHub Action
- Included:Run comparison & case diffs
- Included:1 member
- Not included:Org analytics & cost tracking
- Not included:Regression issues with comments & assignment
Team
$49/mo
Shared run history, drift and cost analytics, and tracked regressions for a growing team.
- Included:Everything in Free
- Included:Up to 5 members
- Included:Roles: admin, member, viewer
- Included:Org analytics & cost tracking
- Included:Regression issues with comments & assignment
- Coming soon:Slack & email alertsSoon
Enterprise
Custom
Security, scale and control for organizations running AI in production.
- Included:Everything in Team
- Included:Unlimited members
- Included:Custom contract & invoicing
- Coming soon:SSO / SAMLSoon
- Coming soon:Audit logs & retention controlsSoon
- Coming soon:Private deploymentSoon
Know before your users do.
Add Drift-CI to a workflow in minutes and gate every pull request on real behaviour.