Advanced
API Performance
We test our API endpoints regularly to ensure they are returning responses reliably. We are actively working to increase bandwidth and reduce latency. Note that your mileage may vary as compared to our testing setup. These numbers are from March 2025.
| Evaluator Name | Latency (in s) | QPS | Max Input Tokens | Average Input Tokens (for benchmarking) |
|---|---|---|---|---|
| answer-relevance-small-2024-07-23 | 2.18 | TBD | 124k | ~30 |
| answer-relevance-large-2024-07-23 | 3.83 | TBD | 124k | ~30 |
| context-relevance-small-2024-07-23 | 2.55 | TBD | 124k | ~180 |
| context-relevance-large-2024-07-23 | 3.93 | TBD | 124k | ~180 |
| context-sufficiency-small-2024-07-23 | 3.38 | TBD | 124k | ~180 |
| context-sufficiency-large-2024-07-23 | 6.87 | TBD | 124k | ~180 |
| hallucination-small-2024-07-23 | 2.58 | TBD | 124k | ~180 |
| hallucination-large-2024-07-23 | 3.80 | TBD | 124k | ~180 |
| lynx-2024-07-23 | 1.32 | TBD | 124k | ~180 |
| judge-small-2024-08-08 | 1.68 | TBD | 124k | ~200 |
| judge-large-2024-08-08 | 3.16 | TBD | 124k | ~200 |
| glider-2024-12-11 | 2.44 | TBD | 8K | ~200 |
| toxicity-2024-10-27 | 2.52 | TBD | 512 | ~20 |
| pii-2024-05-31 | 0.26 | TBD | 16k | ~20 |
| phi-2024-05-31 | 0.32 | TBD | 16k | ~20 |
| nlp-2024-05-16 | 0.25 | TBD | 32k | ~50 |
| exact-match-2024-05-31 | 0.33 | TBD | 32k | ~50 |