Retired

GLIDER

The LLM evaluation platform documented on this site is retired and no longer offered. GLIDER is retired as a product. The research paper stays public on patronus.ai.

Patronus AI is a research lab developing training data and simulated environments for frontier AI. Its work spans reinforcement learning environments and digital world models for agents operating in software.

A small, explainable judge model for grading LLM interactions

GLIDER is a small, explainable 3B judge model that grades LLM interactions on user-defined criteria, with fine-grained scores and highlighted text spans. In the paper it was trained on 685 domains and 183 criteria, reached performance comparable to LLMs 17x its size, and its scores showed 91.3% human agreement.

Where to find the paper

Read it on arXiv (opens in a new tab) or as a PDF (opens in a new tab). The current home for GLIDER is its research page on patronus.ai (opens in a new tab).

GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking

Darshan Deshpande, Selvan Sunitha Ravi, Sky CH-Wang, Bartosz Mielczarek, Anand Kannappan, Rebecca Qian. Dec 18, 2024.

Darshan Deshpande, Selvan Sunitha Ravi, Sky CH-Wang, Bartosz Mielczarek, Anand Kannappan, Rebecca Qian. GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking. arXiv:2412.14140, 2024.