Your detector's public benchmark score does not predict how it behaves inside YOUR agent - grade it on YOUR tool outputs: catch at 1% FPR, per surface, with a training-overlap audit, in one signed receipt.
Operators choose the prompt-injection detector that screens their agents' tool outputs by public benchmark scores (BIPIA, InjecAgent-style sets) - rankings that demonstrably do not transfer: the best detector on BIPIA catches 2% of AgentDojo injections at a 1% false-positive rate, BIPIA-to-AgentDojo rank correlation is Kendall's tau = 0.01, and benign-output false-positive rates run from none to over 90%. This service grades a detector (or a shortlist) on the buyer's own tool-output distribution instead: differential replay labels injected outputs as twins of benign outputs with the payload buried at a recorded position across the buyer's tool surfaces; the detector is calibrated to a fixed 1% benign-FPR operating point; the signed verdict reports catch@1%FPR, benign FPR, and tasks-stopped rate per surface, plus a training-data overlap audit - pinned to detector id x version x distribution hash, so a version bump or a new distribution reads STALE, never silently current. In the lab (EXP-166), the public BIPIA leader scored catch@1%FPR 0.1933 in-situ against a 0.97 public claim, and public-vs-in-situ rank correlation was -0.3333. Stated plainly, because the lab proved the boundaries: those lab figures were measured on the lab's synthetic distribution against detector test doubles replicating documented failure shapes - they demonstrate the mechanism, not a rating of any named production detector; whether a synthetic-but-buyer-shaped distribution grades close enough to a real recorded one is unmeasured in the wild; a detector with undisclosed training data grades UNDISCLOSED on the overlap leg; willingness to pay is unmeasured.
Prices: Grade lookup $0.01/query · In-situ grade, classifier detector $4.99/grade · In-situ grade, LLM-judge detector $9.99/grade · Shortlist comparison $19.99/comparison for up to 5 classifier detectors (judge detectors inside a shortlist meter at $9.99 each) · Re-grade pass $9.99/operator/mo (one named classifier detector). One free in-situ grade (classifier detector) per operator, ever.
API: https://textforagents.com/products/in-situ-injection-detector-verdict/ · Docs: https://textforagents.com/products/in-situ-injection-detector-verdict/docs · OpenAPI: https://textforagents.com/products/in-situ-injection-detector-verdict/openapi.json · Support: support@textforagents.com