Tracing & Logging With OpenTelemetry

You can use Patronus OpenTelemetry collector to export logs and traces, enabling features such as annotations and visualizations.

We do not convert your logs or spans, allowing you to send logs in any format. However, logs sent in unsupported formats may not be compatible with some features. Therefore, we strongly recommend adhering to our semantics for evaluation logs.

By design, you can store any standard OpenTelemetry logs and traces.

Follow our Integration guide to start using tracing.

Integration

To integrate your OTel logs and traces with our platform, configure your system to export data to our collection endpoint. Together with this, an API key is required. Optionally, you may provide a project name and an app. These details need to be passed in the request headers.

1. Setup OTel SDK

You can install and setup OTel SDK following the instruction for selected language. Once you have your SDK installed you can configure exporter to use Patronus Collector. You will also need to install otlp exporter.

Make sure you install api, sdk, and otlp exporter

pip install opentelemetry-api
pip install opentelemetry-sdk
pip install opentelemetry-exporter-otlp

2. Configure OTLP Exporter

Patronus infrastructure host Otel Collector for Opentelemetry integration. You can setup your sdk exporter to export the data to Patronus collector.

export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.patronus.ai:4317"
export OTEL_EXPORTER_OTLP_HEADERS='x-api-key=<YOUR_API_KEY>'

Remember to replace "YOUR_API_KEY"

Once you configure your OTLP Exporter your logs and traces should be sent to Patronus platform.

3. Initiate your tracer or/and logger

Depends on your language selection, you have to do following steps:

In this section you have code parts responsible for each step. You would need to merge those code blocks to have fully working example. For ready to use examples please go to Examples section.

STEP I: Initialize Provider

# Tracing:
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.trace import set_tracer_provider
 
trace_provider = TracerProvider()
set_tracer_provider(trace_provider)
 
# Logging:
from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry._logs import set_logger_provider
 
logger_provider = LoggerProvider()
set_logger_provider(logger_provider)

STEP II: Create Processor using OTLP exporter

# Tracing:
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor
 
trace_processor = BatchSpanProcessor(OTLPSpanExporter())
trace_provider.add_span_processor(trace_processor)
 
# Logging:
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
 
logs_processor = BatchLogRecordProcessor(OTLPLogExporter(insecure=False))
logger_provider.add_log_record_processor(logs_processor)

STEP III: Get logger/tracer

# Tracing:
tracer = trace_provider.get_tracer("my.tracer")
 
# Logging:
logger = logger_provider.get_logger("my.logger2")

STEP IV: Use your logger/tracer

# Tracing:
with tracer.start_as_current_span("my.span") as span:
    print("Let's trace again!")
 
# Logging:
from typing import Any
from opentelemetry import trace
from opentelemetry.sdk._logs import LogRecord
from opentelemetry._logs import SeverityNumber
from time import time_ns
 
# Helper method to use emit a log
def log_to_patronus(body: Any):
    # In case of using tracing as well this will create a log in current span context
    span_context = trace.get_current_span().get_span_context()
 
    logger.emit(
        record=LogRecord(
            trace_flags=span_context.trace_flags,
            timestamp=time_ns(),
            trace_id=span_context.trace_id,
            span_id=span_context.span_id,
            body=body,
            severity_number=SeverityNumber.INFO
        )
    )
 
log_to_patronus("Let's trace this!")

4. OTel + Patronus SDK

When you use Patronus evaluation (via SDK or API), we automatically create a Span and Log for each evaluation. You can wrap these within your trace context using tracing.

You must initiate your tracer first

from patronus import Client
client = Client()
 
# Start Opentelemetry tracing
with tracer.start_as_current_span("span-name") as span:
    # Run evaluation
    result = client.evaluate(
        evaluator="lynx-small",
        criteria="patronus:hallucination",
        evaluated_model_input="What is the largest animal in the world?",
        evaluated_model_output="The giant sandworm.",
        evaluated_model_retrieved_context="The blue whale is the largest known animal.",
    )

Code above will create your trace context and add Patronus evaluation Span and Log to your trace

Examples

Tracing example

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
 
# Initializa Tracer and Span Exporter
trace_provider = TracerProvider()
trace_processor = BatchSpanProcessor(OTLPSpanExporter())
trace_provider.add_span_processor(trace_processor)
trace.set_tracer_provider(trace_provider)
tracer = trace.get_tracer("Demo Tracing")
 
with tracer.start_as_current_span("my-span") as span:
    span.set_attribute("operation.name", "Demo")
    print("Hello world")

Logging example

from time import time_ns
from typing import Any
 
from opentelemetry import trace
from opentelemetry._logs import set_logger_provider, SeverityNumber
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from opentelemetry.sdk._logs import LogRecord, LoggerProvider
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
 
logger_provider = LoggerProvider()
set_logger_provider(logger_provider)
logs_processor = BatchLogRecordProcessor(OTLPLogExporter(insecure=False))
logger_provider.add_log_record_processor(logs_processor)
 
logger = logger_provider.get_logger("my.logger")
 
# Helper method to use emit a log
def log_to_patronus(body: Any):
    # In case of using tracing as well this will create a log in current span context
    span_context = trace.get_current_span().get_span_context()
 
    logger.emit(
        record=LogRecord(
            trace_flags=span_context.trace_flags,
            timestamp=time_ns(),
            trace_id=span_context.trace_id,
            span_id=span_context.span_id,
            body=body,
            severity_number=SeverityNumber.INFO
        )
    )
log_to_patronus("Hello universe!")

Tracing with Patronus SDK (evaluate)

from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from patronus import Client
 
trace_provider = TracerProvider()
trace_processor = BatchSpanProcessor(OTLPSpanExporter())
trace_provider.add_span_processor(trace_processor)
trace.set_tracer_provider(trace_provider)
tracer = trace.get_tracer("Demo Tracing")
 
client = Client()
 
# Start Opentelemetry tracing
with tracer.start_as_current_span("span-name") as span:
    # Run evaluation within your trace context
    result = client.evaluate(
        evaluator="lynx-small",
        criteria="patronus:hallucination",
        evaluated_model_input="What is the largest animal in the world?",
        evaluated_model_output="The giant sandworm.",
        evaluated_model_retrieved_context="The blue whale is the largest known animal."
    )

Tracing & Logging With OpenTelemetry

On this page