Our Python SDK got smarter. We developed a Typscript SDK too. We are updating our SDK code blocks. Python SDKhere.Typscript SDKhere.
Description

Kubernetes Installation Steps

Requirements

  • Docker login credentials (reach out to us if you were not provided docker images).
  • Access to a Kubernetes cluster:
    • Minimum node requirements
      1x t3.medium
      1x m5.xlarge
  • Helm [https://helm.sh/docs/intro/install/]
  • Example configurations:
    • We will provide example helm charts for the configuration.

Download images from Docker

First, log into Docker Hub with the credentials provided to you to access the Docker images in the private registry. 🐳

 
docker login --username [USERNAME] --password [PASSWORD] [PROD REGISTRY SERVER]

Download the images provided to you. These should have the format registry.onprem.patronus.ai/<repository>:<tag>. For example, this command will pull the patronus-backend image with the 2024-08-14 tag.

 
docker pull registry.onprem.patronus.ai/patronus-backend:2024-08-14

If image pull was successful, you should see the following by running docker image ls:

 
REPOSITORY                                     TAG          IMAGE ID       CREATED         SIZE
registry.onprem.patronus.ai/patronus-backend   2024-08-14   7b30fe3251d7   3 weeks ago     617MB

We have the following private and public docker images.

Private Docker Images:

  • patronus-backend
  • patronus-migrate
  • patronus-evaluation-api
  • patronus-model-proxy-api
  • patronus-accounts-api
  • patronus-app
  • patronus-admin-portal
  • evaluators images, eg. evaluator-sdk-judge

Public docker images:

  • Hasura: [docker.io/hasura/graphql-engine:v2.26.0]
  • Vouch-proxy: [quay.io/vouch/vouch-proxy:latest]
  • PostgreSQL: [docker.io/bitnami/postgresql:16.0.0-debian-11-r13]
  • Redis: [docker.io/bitnami/redis:7.2.3-debian-11-r0]

Deploy Helm Charts

Now that you have successfully pulled docker images, we are ready to deploy helm charts! :chart_with_upwards_trend:

  1. Verify that you can connect to your kubernetes cluster. You will be installing Patronus AI services into new namespace.
 
❯ kubectl get pods
No resources found in default namespace.
  1. Install Patronus helm registry:
 
❯ helm repo update && helm repo add patronus-ai https://patronus-ai.github.io/helm-charts
"patronus-ai" has been added to your repositories

You should now be able to see patronus-stack Helm charts:

 
❯ helm search repo patronus-ai
NAME                            CHART VERSION   APP VERSION             DESCRIPTION
patronus-ai/patronus-stack      0.8.3           2025-02-06-17-31        A Helm chart for Kubernetes
  1. Update these parameters in the values.yaml file you were provided, following the patronus-stack chart examples. If you were not sent example .yaml files, please reach out to our team.

These are the values you will need to fill in:

  • values.yaml:
    • <GLOBAL_IMAGE_TAG> Specifies the image tag to use for deployments. Typically indicates the version of the container image.
    • <GLOBAL_IMAGE_REGISTRY> Specifies the registry where the container images are hosted.
    • <GLOBAL_ADMIN_SECRET> The key used for Fernet encryption. This key should be securely stored and managed.
    • <GLOBAL_FERNET_KEY> The key used for Fernet encryption. This key should be securely stored and managed (base64 encoded).
    • <GLOBAL_POSTGRESQL_POSTGRES_PASSWORD> The password for the PostgreSQL admin user.
    • <GLOBAL_POSTGRESQL_PASSWORD> The password for accessing PostgreSQL.
    • <GLOBAL_REDIS_PASSWORD> The password for accessing the Redis service.
    • <POST_INSTALL_IMAGE_TAG> Image tag to be used on post-install step.
    • <POST_INSTALL_CUSTOMER_NAME> customer name.
    • <PATRONUS_APP_HOST> to the URL of the frontend application eg. patronus-app.[your domain].
    • <PATRONUS_DOMAIN> Domain.
    • <PATRONUS_BACKEND_API_HOST> to the backend API URL, eg. patronus-backend-api.[your domain].
    • <PATRONUS_ADMIN_PORTAL_HOST> Optional if we are exposing Admin-portal dashborad, eg. patronus-admin-portal.[your domain].
    • <GOOGLE_API_CLIENT_ID> Optional if we are using Google values example
    • <GOOGLE_CLIENT_SECRET> Optional if we are using Google values example
  1. Install patronus-stack chart:
 
❯ helm -n <your namespace> upgrade patronus-stack --create-namespace --install \
    --values ./values.yaml \
patronus-ai/patronus-stack

You should see the following if successful:

 
Release "patronus-stack" has been upgraded. Happy Helming!
NAME: patronus-stack
LAST DEPLOYED: Mon Sep  2 23:46:43 2024
NAMESPACE: <your namespace>
STATUS: deployed
REVISION: 0

Validate your deployment

Run kubectl get svc -n <your_namespace>

The output should look something like this:

 
NAME                                      TYPE           CLUSTER-IP       EXTERNAL-IP           PORT(S)                      AGE
evaluator-sdk-answer-relevance            NodePort       172.20.44.58     <none>                8000:31128/TCP               25d
evaluator-sdk-context-relevance           NodePort       172.20.175.73    <none>                8000:30294/TCP               25d
evaluator-sdk-context-sufficiency         NodePort       172.20.44.181    <none>                8000:30274/TCP               25d
evaluator-sdk-exact-match                 NodePort       172.20.78.145    <none>                8000:31473/TCP               25d
evaluator-sdk-glider                      NodePort       172.20.147.38    <none>                8000:31553/TCP               25d
evaluator-sdk-hallucination               NodePort       172.20.173.158   <none>                8000:32608/TCP               25d
evaluator-sdk-judge                       NodePort       172.20.51.132    <none>                8000:32456/TCP               25d
evaluator-sdk-judge-mm                    NodePort       172.20.122.175   <none>                8000:30303/TCP               25d
evaluator-sdk-lynx                        NodePort       172.20.222.139   <none>                8000:30443/TCP               25d
evaluator-sdk-nlp                         NodePort       172.20.60.161    <none>                8000:30323/TCP               25d
evaluator-sdk-phi                         NodePort       172.20.135.150   <none>                8000:31550/TCP               25d
evaluator-sdk-pii                         NodePort       172.20.119.59    <none>                8000:31098/TCP               25d
evaluator-sdk-toxicity                    NodePort       172.20.98.178    <none>                8000:31112/TCP               25d
evaluator-sdk-toxicity-bert               NodePort       172.20.97.13     <none>                8000:30572/TCP               25d
patronus-stack-nginx                      LoadBalancer   172.20.76.160    *****.amazonaws.com   80:31215/TCP,443:31464/TCP   125d
patronus-stack-patronus-accounts-api      NodePort       172.20.41.212    <none>                80:30978/TCP                 125d
patronus-stack-patronus-admin-portal      NodePort       172.20.89.195    <none>                3000:31685/TCP               125d
patronus-stack-patronus-app               NodePort       172.20.33.240    <none>                3000:30438/TCP               125d
patronus-stack-patronus-backend           NodePort       172.20.139.87    <none>                80:31273/TCP                 125d
patronus-stack-patronus-evaluation-api    NodePort       172.20.121.169   <none>                80:32699/TCP                 125d
patronus-stack-patronus-hasura            ClusterIP      172.20.174.132   <none>                8080/TCP                     125d
patronus-stack-patronus-model-proxy-api   NodePort       172.20.255.174   <none>                80:32303/TCP                 125d
patronus-stack-postgresql                 ClusterIP      172.20.30.58     <none>                5432/TCP                     125d
patronus-stack-postgresql-hl              ClusterIP      None             <none>                5432/TCP                     125d
patronus-stack-redis-headless             ClusterIP      None             <none>                6379/TCP                     125d
patronus-stack-redis-master               ClusterIP      172.20.131.208   <none>                6379/TCP                     125d
patronus-stack-redis-replicas             ClusterIP      172.20.80.14     <none>                6379/TCP                     125d
patronus-stack-vouch                      ClusterIP      172.20.61.19     <none>                9090/TCP                     125d

Congratulations! You have now deployed Patronus AI in your environment. 🦄

On this page