root.judges¶

Classes¶

`AJudge`	Async wrapper for a single Judge.
`Judge`	Wrapper for a single Judge.
`Judges`	Judges API

Module Contents¶

class root.judges.AJudge¶

Bases: root.generated.openapi_aclient.models.judge.Judge

Async wrapper for a single Judge.

For available attributes, please check the (automatically generated) superclass documentation.

async arun(*, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, expected_output: str | None = None, _request_timeout: int | None = None, _client: root.generated.openapi_aclient.ApiClient) → root.generated.openapi_aclient.models.judge_execution_response.JudgeExecutionResponse¶

Asynchronously run the judge.

Parameters:

response (str) – LLM output to evaluate
request (Optional[str]) – The prompt sent to the LLM. Optional.
contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators
functions (Optional[List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]]) – Optional functions to execute
expected_output (Optional[str]) – Optional expected output
_request_timeout (Optional[int]) – Optional timeout for the request
_client (root.generated.openapi_aclient.ApiClient)

Return type:

root.generated.openapi_aclient.models.judge_execution_response.JudgeExecutionResponse

client_context: root.utils.ClientContextCallable¶

class root.judges.Judge¶

Bases: root.generated.openapi_client.models.judge.Judge

Wrapper for a single Judge.

For available attributes, please check the (automatically generated) superclass documentation.

run(*, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, expected_output: str | None = None, _request_timeout: int | None = None, _client: root.generated.openapi_client.ApiClient) → root.generated.openapi_client.models.judge_execution_response.JudgeExecutionResponse¶

Run the judge.

Parameters:

response (str) – LLM output to evaluate
request (Optional[str]) – The prompt sent to the LLM. Optional.
contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators
functions (Optional[List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]]) – Optional functions to execute
expected_output (Optional[str]) – Optional expected output
_request_timeout (Optional[int]) – Optional timeout for the request
_client (root.generated.openapi_client.ApiClient)

Return type:

root.generated.openapi_client.models.judge_execution_response.JudgeExecutionResponse

client_context: root.utils.ClientContextCallable¶

class root.judges.Judges(client_context: root.utils.ClientContextCallable)¶

Judges API

Note

The construction of the API instance should be handled by accessing an attribute of a root.client.RootSignals instance.

Parameters:: client_context (root.utils.ClientContextCallable)

async adelete(judge_id: str, *, _request_timeout: int | None = None, _client: root.generated.openapi_aclient.ApiClient) → None¶

Asynchronously delete the judge.

Parameters:

judge_id (str) – The judge to be deleted.
_request_timeout (Optional[int])
_client (root.generated.openapi_aclient.ApiClient)

Return type:

None

async aget(judge_id: str, *, _request_timeout: int | None = None, _client: root.generated.openapi_aclient.ApiClient) → AJudge¶

Asynchronously get a judge by ID.

Parameters:

judge_id (str) – The judge to be fetched.
_request_timeout (Optional[int])
_client (root.generated.openapi_aclient.ApiClient)

Return type:

AJudge

async alist(*, limit: int = 100) → AsyncIterator[AJudge]¶

Asynchronously iterate through the judges.

Parameters:: limit (int) – Number of entries to iterate through at most.
Return type:: AsyncIterator[AJudge]

async arun(judge_id: str, *, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, expected_output: str | None = None, _request_timeout: int | None = None, _client: root.generated.openapi_aclient.ApiClient) → root.generated.openapi_aclient.models.judge_execution_response.JudgeExecutionResponse¶

Asynchronously run a judge directly by ID.

Parameters:

judge_id (str) – ID of the judge to run
response (str) – LLM output to evaluate
request (Optional[str]) – The prompt sent to the LLM. Optional.
contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators
functions (Optional[List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]]) – Optional functions to execute
expected_output (Optional[str]) – Optional expected output
_request_timeout (Optional[int]) – Optional timeout for the request
_client (root.generated.openapi_aclient.ApiClient)

Return type:

root.generated.openapi_aclient.models.judge_execution_response.JudgeExecutionResponse

async aupdate(judge_id: str, *, name: str | None = None, evaluator_references: List[root.generated.openapi_aclient.models.evaluator_reference_request.EvaluatorReferenceRequest] | None = None, _request_timeout: int | None = None, _client: root.generated.openapi_aclient.ApiClient) → AJudge¶

Asynchronously update an existing judge.

Parameters:

judge_id (str) – The judge to be updated.
name (Optional[str]) – New name for the judge
evaluator_references (Optional[List[root.generated.openapi_aclient.models.evaluator_reference_request.EvaluatorReferenceRequest]]) – New list of evaluator references
_request_timeout (Optional[int])
_client (root.generated.openapi_aclient.ApiClient)

Return type:

AJudge

delete(judge_id: str, *, _request_timeout: int | None = None, _client: root.generated.openapi_client.ApiClient) → None¶

Delete the judge.

Parameters:

judge_id (str) – The judge to be deleted.
_request_timeout (Optional[int])
_client (root.generated.openapi_client.ApiClient)

Return type:

None

get(judge_id: str, *, _request_timeout: int | None = None, _client: root.generated.openapi_client.ApiClient) → Judge¶

Get a judge by ID.

Parameters:

judge_id (str) – The judge to be fetched.
_request_timeout (Optional[int])
_client (root.generated.openapi_client.ApiClient)

Return type:

Judge

list(*, limit: int = 100, _client: root.generated.openapi_client.ApiClient) → Iterator[Judge]¶

Iterate through the judges.

Parameters:

limit (int) – Number of entries to iterate through at most.
_client (root.generated.openapi_client.ApiClient)

Return type:

Iterator[Judge]

run(judge_id: str, *, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, expected_output: str | None = None, _request_timeout: int | None = None, _client: root.generated.openapi_client.ApiClient) → root.generated.openapi_client.models.judge_execution_response.JudgeExecutionResponse¶

Run a judge directly by ID.

Parameters:

judge_id (str) – ID of the judge to run
response (str) – LLM output to evaluate
request (Optional[str]) – The prompt sent to the LLM. Optional.
contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators
functions (Optional[List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]]) – Optional functions to execute
expected_output (Optional[str]) – Optional expected output
_request_timeout (Optional[int]) – Optional timeout for the request
_client (root.generated.openapi_client.ApiClient)

Return type:

root.generated.openapi_client.models.judge_execution_response.JudgeExecutionResponse

update(judge_id: str, *, name: str | None = None, evaluator_references: List[root.generated.openapi_client.models.evaluator_reference_request.EvaluatorReferenceRequest] | None = None, _request_timeout: int | None = None, _client: root.generated.openapi_client.ApiClient) → Judge¶

Update an existing judge.

Parameters:

judge_id (str) – The judge to be updated.
name (Optional[str]) – New name for the judge
evaluator_references (Optional[List[root.generated.openapi_client.models.evaluator_reference_request.EvaluatorReferenceRequest]]) – New list of evaluator references
_request_timeout (Optional[int])
_client (root.generated.openapi_client.ApiClient)

Return type:

Judge

client_context¶