Evaluations are optional justifications attached to eval results, typically generated by a LLM. Explanations can be produced by the same LLM Judge that is generating the eval result, or an independent LLM.

Patronus evaluators support explanations by default, with the option to toggle explanations off.

Configure Explanations for Evaluators

The explain strategy parameter controls when evaluation result explanations are generated. Here are the available options:

never: No explanations are generated for any evaluation results
on-fail: Only generates explanations for failed evaluations
on-success: Only generates explanations for passed evaluations
always: Generates explanations for all evaluations

Important notes:

Not all evaluation criteria support explanations
The explain_strategy parameter overrides the explain parameter
For optimizing latency in online workflows, it's recommended to either use explain_strategy="never" or explain_strategy="on-fail" to reduce the number of generated explanations

Example

Here's an example of setting the explain strategy in a request:

{
    "evaluators": [
        {
            "evaluator": "judge-small",
            "profile_name": "patronus:is-concise", 
            "explain_strategy": "never"
        }
    ]
}

Configure Explanations Globally

You can also configure explanations globally with the explain flag in the evaluation API request. This applies globally to all evaluators, if you are using multiple evaluators.

The following values are available

True: Always generate explanations
False: No explanations are generated for any evaluation results
on-fail: Only generates explanations for failed evaluations
on-success: Only generates explanations for passed evaluations

from patronus import Client

client = Client(api_key="YOUR_API_KEY")
result = client.evaluate(
    evaluator="lynx-small",
    criteria="patronus:hallucination",
    evaluated_model_input="What is the largest animal in the world?",
    evaluated_model_output="The giant sandworm.",
    evaluated_model_retrieved_context="The blue whale is the largest known animal.",
    tags={"scenario": "onboarding"},
    explain=True
)