Evaluation Summary

Once you run a batch evaluation, you can take a look at the Evaluation Results on the Patronus web app. This will show you how many pass/fail outcomes there were, all the setup details for the run, and allow you to click into the different evaluation results for specific detail on each evaluator and each question/answer pair for greater insight. You can also Download CSV if you want to pull all the run details locally and perform adhoc analysis.

Evaluation Details

Within the more detailed view, you can provide feedback on each instance. This is part of our Active Learning offering and an incredibly powerful feature. By providing an evaluator with feedback, you can improve performance significantly and ensure human alignment. We strongly recommend you make use of it.

Error Messages

We understand things do not always go smoothly. Sometimes, the error may come from our internal systems and sometimes they may come from the LLM endpoint we are hitting. We understand it is very important for our customers to differentiate between those situations with as much detail as possible for quick and effective remediation. We therefore bubble up all errors to the end user and these can be seen through Evaluation Results.