Upload a Dataset

You can use Patronus to manage your suite of test [Datasets] (https://app.patronus.ai/datasets). Datasets in the UI currently have a limit of 30,000 rows. For datasets that are larger, we recommend loading the datasets locally or with a Pandas DataFrame. We accept .csv and .jsonl files with any text based schema.

When uploading datasets, you have the option to map fields in your dataset to fields in Patronus Datasets objects. This includes the following:

• evaluated_model_input: The prompt provided to your LLM agent or input to a task defined in the experimentation framework
• evaluated_model_output: The output of your LLM agent or task
• evaluated_model_retrieved_context: Metadata accessible by your LLM agent or used by the task defined in the experimentation framework
• evaluated_model_gold_answer: The gold answer or expected output

Mapping Multiple Fields

If you have multiple fields that map to one field in the Datasets schema, you can drag and drop multiple fields into one. This will create a JSON wrapper around the two fields, and store the result as string.

Once you upload the dataset, you'll see it in the Datasets view along with our off-the-shelf datasets.

To run an experiment with this dataset, reference the dataset using the id field, eg. in the above example it would be d-r2b062mkrngzkx5a.

financebench_dataset = client.remote_dataset("d-jxrisvlp1hgf786h")
 
client.experiment(
    "Finance Project",
    data=financebench_dataset,
    task=task,
    evaluators=[evaluator], # Replace with your evaluators
    experiment_name="GPT-4o-mini"
)

All datasets in the platform can be downloaded. See working with datasets for more information.