Upload a Dataset

You can use Patronus to manage your suite of test Datasets. Datasets in the UI currently have a limit of 30,000 rows. For datasets that are larger, we recommend loading the datasets locally or with a Pandas DataFrame. We accept .csv and .jsonl files with any text based schema.

When uploading datasets, you have the option to map fields in your dataset to fields in Patronus Datasets objects. This includes the following:

• task_input: The prompt provided to your LLM agent or input to a task defined in the experimentation framework
• task_output: The output of your LLM agent or task
• task_context: Metadata accessible by your LLM agent or used by the task defined in the experimentation framework
• gold_answer: The gold answer or expected output

Mapping Multiple Fields

If you have multiple fields that map to one field in the Datasets schema, you can drag and drop multiple fields into one. This will create a JSON wrapper around the two fields, and store the result as string.

Once you upload the dataset, you'll see it in the Datasets view along with our off-the-shelf datasets.

To run an experiment with this dataset, reference the dataset using its name or ID:

from patronus.datasets import RemoteDatasetLoader
from patronus.experiments import run_experiment
 
# Load a dataset from the Patronus platform using its name
financebench_dataset = RemoteDatasetLoader("financebench")
 
# Or load by ID
# financebench_dataset = RemoteDatasetLoader(by_id="d-jxrisvlp1hgf786h")
 
experiment = run_experiment(
    dataset=financebench_dataset,
    task=task,
    evaluators=[evaluator], # Replace with your evaluators
    experiment_name="Finance Project"
)

All datasets in the platform can be downloaded. See Using Datasets for more information.