use cases

Use cases

The jobs Drift-CI is built for, from catching a prompt regression in a pull request to spotting silent provider changes.

• LLM drift monitoring with scheduled eval runsProblem: Model aliases move, and serving changes can shift outputs. Your prompts didn't change, but the answers can.Outcome: A scheduled run of your suite against a pinned model catches behaviour changes the day they happen.
• Test LLM model upgrade candidates before you switchProblem: New models ship often. Switching blind means discovering regressions in production.Outcome: Run the same suite against the candidate model in a PR, see per-case deltas, and switch only when it holds up.
• Prompt regression testing on every pull requestProblem: A prompt tweak that fixes one case silently breaks five others, and you find out from users.Outcome: Every PR that touches a prompt runs your suite, compares scores to committed baselines, and fails the check on a real regression.
• How to test LLM JSON output against a schemaProblem: Downstream code expects valid JSON with the right fields. One model or prompt change and parsing fails in production.Outcome: Every case asserts the output validates against a per-case JSON schema, and optionally checks exact field values.