Pass Through Solver

Reads pre-recorded trace / conversation history from a dataset column instead of calling a model. Use this to evaluate outputs that were generated outside of AI GO!, or to replay a previous conversation.

Examples

Example: Pre-recorded QA Traces. Reads a full conversation trace from the trace column of the dataset and presents it to the scorers without calling any model. Useful for scoring outputs generated outside AI GO! or replaying a previous run.

...
definition:
  ...
  solver:
    type: "pass_through_solver"
    message_format: "open_responses"
    trace_column: "trace"

Given a dataset row whose trace column contains a pre-recorded single-turn exchange:

Turn	Role	Content
1	system	`You are a helpful assistant. Answer concisely with a single word.`
2	user	`What is the capital city of Japan?`
3	assistant	`Tokyo`

The solver replays all three turns verbatim. The last assistant message (Tokyo) becomes solver_output.output passed to the scorers.

Multi-turn traces with tool calls are also supported:

Turn	Role	Content
1	user	`What is the current temperature in Seoul?`
2	function_call	`get_current_weather({"city": "Seoul", "unit": "celsius"})`
3	function_call_output	`{"temperature": 11, "condition": "partly cloudy"}`
4	assistant	`The current temperature in Seoul is 11°C with partly cloudy skies.`
5	user	`And what about Berlin?`
6	function_call	`get_current_weather({"city": "Berlin", "unit": "celsius"})`
7	function_call_output	`{"temperature": 4, "condition": "overcast"}`
8	assistant	`In Berlin it is currently 4°C and overcast.`

Configuration

Properties

type Literal "pass_through_solver" required

The type of the solver.

trace_column string required

The name of the dataset column that contains the trace. The value must be deserializable as a Trace object.

message_format string

The message format used by the solver for recording its output.

Default: latticeflow