Frequency

Computes the relative frequency of each distinct value found in a score field across all samples. Use this to understand the distribution of categorical or discrete outputs - for example, how often a labeler assigns each label, or how many samples received each rating. Unlike other metrics, the output metric names are derived from the distinct values in the field rather than a fixed name property.

Output

One metric per distinct value in the field, named {field}_{value}, each holding the fraction of samples with that value. All frequencies sum to 1.0.

Output Metric Names

The metric keys are the string representations of the distinct values found in the score field at runtime. For example, if the field is called 'valid' and contains "yes" and "no" values, the output metrics will be named valid_yes and valid_no.

Examples

Example: Label Distribution. Measures how often a model assigns each sentiment label, producing one frequency metric per distinct label value.

scorers:
  - type: labeler_via_model
    model_key: "<< config.judge_model >>"
    system_prompt: "Classify the sentiment of the response."
    user_prompt: "Response: {{ model_output.choices[0].message.content }}"
    valid_labels:
      - positive
      - neutral
      - negative
    metrics:
      - type: frequency
        field: label

This produces three metrics: label_positive, label_neutral, and label_negative, each holding the fraction of samples assigned that label.

Configuration

Properties

type Literal "frequency" required

The type of the metric.

field string, TemplateValue required

The field over which to compute the relative frequency.

key string

Unique identifier assigned to the entity in AI GO!.