Caching
Running evaluations can be expensive - running a task may require many model queries and significant computation. AI GO! automatically caches task results so that later evaluations can reuse them, and work is never repeated unnecessarily.
Cache Policies
You stay in control through a per-evaluation cache policy, which determines both whether previous results are reused and whether new results are cached for future use. Each evaluation declares its own cache policy via the cache_policy field in the evaluation config (see CachePolicy reference). The policy applies to all task results in that evaluation.
| Policy | Description |
|---|---|
reuse (default) | Reuse cached task results from previous evaluations when available. |
update | Ignore existing cached task results, but cache the new results for future reuse. |
no-cache | Ignore existing cached task results and do not cache the new results. |
How Reuse Works
When cache_policy: reuse is set, AI GO! checks each task result against previously cached results, potentially doing full reuse or partial reuse of cached results.
Full Reuse
A task result is reused as-is - samples, solver outputs, scores, actions, and metrics - when a previous task result exists with an identical task specification:
- The same task
- The same dataset version
- The same solver definition, config values, and versions of any models it uses
- The same scorer definitions, config values, and versions of any models they use
- The same action rules and their config values
- The same metric definitions and their config values
- The same run config (ignoring
cache_policy)
No computation or model queries are performed for that task result.
Partial Reuse
If a previous task result matches on everything above except the scoring section - that is, the scorer definitions, their config values or model versions, the action rules, or the metric definitions have changed - AI GO! reuses the samples and solver outputs from the cached result, then recomputes scores, actions, and metrics from scratch.
By avoiding re-running the solver, all the model queries the solver would do are avoided too.
No Match
If no matching cached task result exists, then the task specification is executed fully and the result is stored as reusable for future evaluations (unless no-cache is set).
A cached task result is not used if it previously failed or was cancelled.