root.objectives

Classes

AObjective

Wrapper for a single Objective.

Objective

Wrapper for a single Objective.

Objectives

Objectives API

Versions

Version listing (sub)API

Module Contents

class root.objectives.AObjective

Bases: root.generated.openapi_aclient.models.objective.Objective

Wrapper for a single Objective.

For available attributes, please check the (automatically generated) superclass documentation.

async arun(*, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, _request_timeout: int | None = None) → root.generated.openapi_aclient.models.validator_execution_result.ValidatorExecutionResult

Asynchronously run all validators associated with the objective.

Parameters:

  • response (str) – LLM output.

  • request (Optional[str]) – The prompt sent to the LLM. Optional.

  • contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators

  • functions (Optional[List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]])

  • _request_timeout (Optional[int])

Return type:

root.generated.openapi_aclient.models.validator_execution_result.ValidatorExecutionResult

class root.objectives.Objective

Bases: root.generated.openapi_client.models.objective.Objective

Wrapper for a single Objective.

For available attributes, please check the (automatically generated) superclass documentation.

run(*, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, _request_timeout: int | None = None) → root.generated.openapi_client.models.validator_execution_result.ValidatorExecutionResult

Run all validators associated with the objective.

Parameters:

  • response (str) – LLM output.

  • request (Optional[str]) – The prompt sent to the LLM. Optional.

  • contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators

  • functions (Optional[List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]])

  • _request_timeout (Optional[int])

Return type:

root.generated.openapi_client.models.validator_execution_result.ValidatorExecutionResult

class root.objectives.Objectives(client: Awaitable[root.generated.openapi_aclient.ApiClient] | root.generated.openapi_client.ApiClient)

Objectives API

Note

The construction of the API instance should be handled by accesing an attribute of a root.client.RootSignals instance.

Parameters:

client (Union[Awaitable[root.generated.openapi_aclient.ApiClient], root.generated.openapi_client.ApiClient])

async acreate(*, intent: str | None = None, validators: List[root.validators.AValidator] | None = None, test_dataset_id: str | None = None, _request_timeout: int | None = None) → AObjective

Asynchronously create a new objective and return its ID.

Parameters:

  • intent (Optional[str]) – The intent of the objective.

  • validators (Optional[List[root.validators.AValidator]]) – An optional list of validators.

  • test_dataset_id (Optional[str]) – The ID of the test dataset

  • _request_timeout (Optional[int])

Return type:

AObjective

async adelete(objective_id: str, *, _request_timeout: int | None = None) → None

Asynchronously delete the objective from the registry.

Parameters:

  • objective_id (str) – The objective to be deleted.

  • _request_timeout (Optional[int])

Return type:

None

async aget(objective_id: str, *, _request_timeout: int | None = None) → AObjective

Asynchronously get an objective by ID.

Parameters:

  • objective_id (str) – The objective to be fetched.

  • _request_timeout (Optional[int])

Return type:

AObjective

async alist(*, intent: str | None = None, limit: int = 100) → AsyncIterator[root.generated.openapi_aclient.models.objective_list.ObjectiveList]

Asynchronously iterate through the objectives.

Note

The call will list only publicly available global objectives and those objectives available to the organzation(s) of the user.

Parameters:

  • intent (Optional[str]) – Specific intent the returned objectives must match.

  • limit (int) – Number of entries to iterate through at most.

Return type:

AsyncIterator[root.generated.openapi_aclient.models.objective_list.ObjectiveList]

async arun(objective_id: str, *, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, _request_timeout: int | None = None) → root.generated.openapi_aclient.models.validator_execution_result.ValidatorExecutionResult

Asynchronously run all validators associated with an objective.

Parameters:

  • response (str) – LLM output.

  • request (Optional[str]) – The prompt sent to the LLM. Optional.

  • contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators

  • objective_id (str)

  • functions (Optional[List[root.generated.openapi_aclient.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]])

  • _request_timeout (Optional[int])

Return type:

root.generated.openapi_aclient.models.validator_execution_result.ValidatorExecutionResult

async aupdate(objective_id: str, *, intent: str | None = None, validators: List[root.validators.AValidator] | None = None, test_dataset_id: str | None = None, _request_timeout: int | None = None) → AObjective

Asynchronously update an existing objective.

Parameters:

  • objective_id (str) – The objective to be updated.

  • intent (Optional[str]) – The intent of the objective.

  • validators (Optional[List[root.validators.AValidator]]) – An optional list of validators.

  • test_dataset_id (Optional[str])

  • _request_timeout (Optional[int])

Return type:

AObjective

create(*, intent: str | None = None, validators: List[root.validators.Validator] | None = None, test_dataset_id: str | None = None, _request_timeout: int | None = None) → Objective

Create a new objective and return its ID.

Parameters:

  • intent (Optional[str]) – The intent of the objective.

  • validators (Optional[List[root.validators.Validator]]) – An optional list of validators.

  • test_dataset_id (Optional[str]) – The ID of the test dataset

  • _request_timeout (Optional[int])

Return type:

Objective

delete(objective_id: str, *, _request_timeout: int | None = None) → None

Delete the objective from the registry.

Parameters:

  • objective_id (str) – The objective to be deleted.

  • _request_timeout (Optional[int])

Return type:

None

get(objective_id: str, *, _request_timeout: int | None = None) → Objective

Get an objective by ID.

Parameters:

  • objective_id (str) – The objective to be fetched.

  • _request_timeout (Optional[int])

Return type:

Objective

list(*, intent: str | None = None, limit: int = 100) → Iterator[root.generated.openapi_client.models.objective_list.ObjectiveList]

Iterate through the objectives.

Note

The call will list only publicly available global objectives and those objectives available to the organzation(s) of the user.

Parameters:

  • intent (Optional[str]) – Specific intent the returned objectives must match.

  • limit (int) – Number of entries to iterate through at most.

Return type:

Iterator[root.generated.openapi_client.models.objective_list.ObjectiveList]

run(objective_id: str, *, response: str, request: str | None = None, contexts: List[str] | None = None, functions: List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest] | None = None, _request_timeout: int | None = None) → root.generated.openapi_client.models.validator_execution_result.ValidatorExecutionResult

Run all validators associated with an objective.

Parameters:

  • response (str) – LLM output.

  • request (Optional[str]) – The prompt sent to the LLM. Optional.

  • contexts (Optional[List[str]]) – Optional documents passed to RAG evaluators

  • objective_id (str)

  • functions (Optional[List[root.generated.openapi_client.models.evaluator_execution_functions_request.EvaluatorExecutionFunctionsRequest]])

  • _request_timeout (Optional[int])

Return type:

root.generated.openapi_client.models.validator_execution_result.ValidatorExecutionResult

update(objective_id: str, *, intent: str | None = None, validators: List[root.validators.Validator] | None = None, test_dataset_id: str | None = None, _request_timeout: int | None = None) → Objective

Update an existing objective.

Parameters:

  • objective_id (str) – The objective to be updated.

  • intent (Optional[str]) – The intent of the objective.

  • validators (Optional[List[root.validators.Validator]]) – An optional list of validators.

  • test_dataset_id (Optional[str])

  • _request_timeout (Optional[int])

Return type:

Objective

client
versions
class root.objectives.Versions(client: Awaitable[root.generated.openapi_aclient.ApiClient] | root.generated.openapi_client.ApiClient)

Version listing (sub)API

Note that this should not be directly instantiated.

Parameters:

client (Union[Awaitable[root.generated.openapi_aclient.ApiClient], root.generated.openapi_client.ApiClient])

async alist(objective_id: str) → root.generated.openapi_aclient.models.paginated_objective_list.PaginatedObjectiveList

Asynchronously list all versions of an objective.

Parameters:

objective_id (str) – The objective to list the versions for

Return type:

root.generated.openapi_aclient.models.paginated_objective_list.PaginatedObjectiveList

list(objective_id: str) → root.generated.openapi_client.models.paginated_objective_list.PaginatedObjectiveList

List all versions of an objective.

Parameters:

objective_id (str) – The objective to list the versions for

Return type:

root.generated.openapi_client.models.paginated_objective_list.PaginatedObjectiveList