Create a project

To run an experiment, first create a project. Multiple experiments can be run in the same project. You can create a new project in the UI in Experiments -> Projects. You can obtain the corresponding Project ID in the URL by clicking on the project. This is in the format https://app.patronus.ai/projects/project-id.

You can also create projects with the https://api.patronus.ai/v1/projects endpoint.

Create an experiment

Evaluations can be run within an experiment or standalone. To create an experiment ID, send a POST request to https://api.patronus.ai/v1/experiments.

Full Code Example

// Step 1: Create the project 
const projectOptions = {
    method: 'POST',
    headers: {
        accept: 'application/json',
        'content-type': 'application/json',
        'X-API-KEY': ""
    },
    body: JSON.stringify({ name: 'test-project' }),
};

let projectId: string | undefined;

fetch('https://api.patronus.ai/v1/projects', projectOptions)
    .then((res) => res.json())
    .then((res) => {
        console.log('Project created:', res);
        projectId = res.id;

        // Step 2: Now create the experiment
        return fetch('https://api.patronus.ai/v1/experiments', {
            method: 'POST',
            headers: {
                accept: 'application/json',
                'content-type': 'application/json',
                'X-API-KEY': ""
            },
            body: JSON.stringify({ project_id: projectId, name: 'test-experiment' }),
        });
    })
    .then((res) => res.json())
    .then((res) => {
        console.log('Experiment created:', res);
        const experimentId = res.experiment.id;

        // Step 3: Now evaluate the experiment
        return fetch('https://api.patronus.ai/v1/evaluate', {
            method: 'POST',
            headers: {
                'X-API-KEY': "",
                'content-type': 'application/json',
            },
            body: JSON.stringify({
                evaluators: [
                    {
                        evaluator: 'lynx',
                        criteria: 'patronus:hallucination',
                    },
                ],
                evaluated_model_input: 'What color is the ocean?',
                evaluated_model_retrieved_context: 'The ocean is blue',
                evaluated_model_output: 'The ocean is red',
                experiment_id: experimentId,
            }),
        });
    })
    .then((res) => res.text())
    .then((result) => {
        console.log('Evaluation result:', result);
    })
    .catch((err) => {
        console.error('Error:', err);
    });