Precision

Computes precision (TP / (TP + FP)) from per-sample true positive and false positive counts. Use this with scorers that produce counts - for example, a scorer that computes how many items in the model's output were actually correct and how many were not. Pair with Recall and F1 Score for a full picture of overlap quality.

Output

A single metric named precision by default, or the value of name if provided. Value is in the [0, 1] range; 1.0 means no false positives.

Examples

Example: Function Call Overlap. A Python scorer checks which required function calls the model made. Precision measures how many of the calls the model made were actually required.

scorers:
  - type: python
    compute_scores_snippet: !include "function_overlap_scorer.py"
    # scorer returns: num_true_positives, num_false_positives, num_false_negatives
    metrics:
      - type: precision
        num_true_positives_field: num_true_positives
        num_false_positives_field: num_false_positives
        name: Call Precision
      - type: recall
        num_true_positives_field: num_true_positives
        num_false_negatives_field: num_false_negatives
        name: Call Recall
      - type: f1_score
        num_true_positives_field: num_true_positives
        num_false_positives_field: num_false_positives
        num_false_negatives_field: num_false_negatives
        name: Call F1

Configuration

Properties

type Literal "precision"

Type

num_true_positives_field string required

The field that contains the number of true positives.

num_false_positives_field string required

The field that contains the number of false positives.

name string

The name given to the metric value. If not specified, it is precision.

key string

Unique identifier assigned to the entity in AI GO!.