📘

Secrets store sensitive values (API keys, authentication tokens, certificates) that your evaluations need at runtime. AI GO! keeps them server-side so they do not appear in plain text in your YAML configs or evaluation logs by mistake.

Once secrets are stored in AI GO! (via lf secret add or inline in YAML — see Manage Secrets), you can reference them in your configuration using the << secrets.NAME >> placeholder syntax. AI GO! resolves these placeholders at runtime, so the actual values never appear in your config files or evaluation logs.

Syntax

Reference a secret by wrapping its name with << secrets. >>:

<< secrets.SECRET_NAME >>

Where Secrets Can Be Used

Model Connection Config

Secrets are most commonly used to inject API keys and authentication headers into model connection configs.

api_key

models:
  - key: my-model
    config:
      connection_type: custom_connection
      adapter:
        key: "latticeflow$openai_chat_completion"
      url: "https://api.openai.com/v1/chat/completions"
      api_key: "<< secrets.OPENAI_KEY >>"
      model_key: "gpt-4.1-mini"
    secrets:
      OPENAI_KEY: $OPENAI_API_KEY

custom_headers

models:
  - key: my-model
    config:
      connection_type: custom_connection
      url: "https://my-endpoint.example.com/v1/chat/completions"
      custom_headers:
        X-Auth-Token: "<< secrets.AUTH_TOKEN >>"
        X-Api-Version: "2024-01"
    secrets:
      AUTH_TOKEN: $MY_AUTH_TOKEN

environment

For custom inference models that need environment variables at runtime:

models:
  - key: custom-model
    config:
      connection_type: custom_inference
      environment:
        HF_TOKEN: "<< secrets.HF_TOKEN >>"
    secrets:
      HF_TOKEN: $HUGGING_FACE_TOKEN

TLS trusted_ca

models:
  - key: my-model
    config:
      connection_type: custom_connection
      url: "https://internal-endpoint.corp.net/v1/chat/completions"
      tls_context:
        validation_context:
          trusted_ca: "<< secrets.CA_CERT >>"
    secrets:
      CA_CERT: $MY_CA_CERTIFICATE

Task Definitions

Secret placeholders can also appear inside task definitions — for example, in custom Python scorer snippets or other templated fields. AI GO! resolves all << secrets.NAME >> placeholders in the entire task definition at execution time.

secrets:
  TARGET_FIELD: "is_correct"
  CONTENT_KEY: "message"

tasks:
  - key: my-task
    scorers:
      - type: python
        compute_scores_snippet: |
          def compute_scores(sample, solver_output):
              content = solver_output.output["choices"][0]["<< secrets.CONTENT_KEY >>"]["content"]
              return {"<< secrets.TARGET_FIELD >>": content.strip() == "YES"}
        metrics:
          - key: accuracy
            type: mean
            field: << secrets.TARGET_FIELD >>

Traceability of Secrets Used in Evaluations

Secrets are snapshotted the same way as any other entity used in an evaluation. That means that if an evaluation was run with a secret (used in a model or task), even if the secret was later updated, the original value of the secret is preserved for traceability purposes, and it is possible to find out which secret value the evaluation was run with

Exporting Evaluations with Secret Values

By default, if you run lf export eval, the secret values are not exported. However, to achieve full traceability, admin users can use the --with-secrets flag to export evaluation results with the secret values.

lf export eval --id <eval ID> --output ./results --with-secrets