Percival
Error Taxonomy
Complete list of errors Percival can detect
Percival can detect 20+ failure modes in agentic traces out of the box. These errors are organized into three main categories: reasoning errors, system execution errors, and planning and coordination errors.
Base error taxonomy
Here are the agentic errors that Percival can catch:
| Category | Sub-category | Error Type | Brief Description |
|---|---|---|---|
| Reasoning Errors | Hallucinations | Language-only | Fabricated content without using tools |
| Reasoning Errors | Hallucinations | Tool-related | Invented tool outputs or capabilities |
| Reasoning Errors | Information Processing | Poor Information Retrieval | Retrieved or cited information irrelevant to the task |
| Reasoning Errors | Information Processing | Tool Output Misinterpretation | Misread or mis-applied a tool's result |
| Reasoning Errors | Decision Making | Incorrect Problem Identification | Misunderstood the overall or local task |
| Reasoning Errors | Decision Making | Tool Selection Errors | Chose an inappropriate tool for the job |
| Reasoning Errors | Output Generation | Formatting Errors | Produced malformed code / data or wrong structure |
| Reasoning Errors | Output Generation | Instruction Non-compliance | Ignored or deviated from the given instructions |
| System Execution Errors | Configuration | Tool Definition Issues | Tool was mis-declared (e.g. search declared as calculator) |
| System Execution Errors | Configuration | Environment Setup Errors | Missing keys, permissions, or other setup problems |
| System Execution Errors | API Issues | Rate Limiting | Exceeded quota (HTTP 429) |
| System Execution Errors | API Issues | Authentication Errors | Invalid or missing credentials (HTTP 401/403) |
| System Execution Errors | API Issues | Service Errors | Upstream failure (HTTP 500) |
| System Execution Errors | API Issues | Resource Not Found | Endpoint or asset missing (HTTP 404) |
| System Execution Errors | Resource Management | Resource Exhaustion | Ran out of memory / disk / other resources |
| System Execution Errors | Resource Management | Timeout Issues | Model timed out during execution |
| Planning and Coordination Errors | Context Management | Context Handling Failures | Context is not retained or used correctly |
| Planning and Coordination Errors | Context Management | Resource Abuse | A resource is unnecessarily used or called repeatedly |
| Planning and Coordination Errors | Task Management | Goal Deviation | Orchestrator deviates from the intended plan |
| Planning and Coordination Errors | Task Management | Task Orchestration | Assignment of tasks to the wrong sub-agents |
Custom error taxonomies
You can extend Percival's base taxonomy with domain-specific errors tailored to your application. This is especially useful for:
- Domain-specific applications (medical, legal, financial, sales)
- Compliance requirements (HIPAA, PCI-DSS, regulatory constraints)
- Product-specific failure modes
- Company policy violations
To learn how to create custom error taxonomies, see: Custom error taxonomies with Percival