Control and Measurement for LLM automations
The Root Signals SDK streamlines the evaluation of your LLM pipelines, to yield insights about their effectiveness. While many tools exist for building the pipelines themselves, quantifying their performance can be challenging.
Root Signals addresses this gap with carefully crafted universal evaluators based on cutting edge LLM research, as well as a framework for systematically adding your own.
Easily integrate with your CI/CD process for continuous monitoring to ensure the performance stays within acceptable limits.
Contents:
- Quickstart
- SDK features using concrete examples
- Root Signals evaluators
- Adjust evaluator behavior
- Retrieval Augmented Generation (RAG) evaluation
- Minimal Skill
- Simple Skill
- Skill with a validator
- Skill with reference data
- Skill with a data loader
- Evaluator Skill and minimal version of it
- Use OpenAI client for chat completions
- Evaluate your LLM pipeline by grouping validators to a Skill
- Add a model
- Authentication and setup
- API Reference