Guides

Model Adapters

📘

To understand the AI GO! model input/output types, see Model I/O Types.

The model adapter consists of two parts with different responsibilities:

  • Input Adapter: Converts the AI GO! model input into a body of the request understood by the model endpoint.
  • Output Adapter: Converts a the body of the model endpoint response to an AI GO! model output.

Model adapters are defined using two Jinja templates.

Quick Links

Identity Adapter
If the model endpoint is already AI GO!-compatille. A starting point when defining new model adapters.
CLI
Learn about CLI commands.
Model Adapters Reference
Learn about model adapter definition in the reference.

Integrations

OpenAI Model
Learn how to integrate an OpenAI model.
Dify Model
Learn how to integrate an Dify RAG model.
RAGFlow Model
Learn how to integrate an RAGFlow model.
Streaming OpenAI Model
Learn how to integrate an OpenAI model with a (SSE) streaming response.

How it Works?

Both input and output transforms receive as input the information about the model, that can be used when defining the Jinja template. It contains the following properties:

Model Information Type
from pydantic import BaseModel

class ModelInfo(BaseModel):
    model_key: str
    modality: Modality
    task: Task

Input Transform

The input adapter is a function that maps the input in AI GO! format to the body of the HTTP request as expected by the model endpoint. It is defined as a Jinja template with signature equivalent to:

def process_input(model_input: dict, model_info: ModelInfo) -> str: ...

where:

  • model_input is the model input in AI GO! format (see Model I/O Types) given as a Jinja dictionary,
  • model_info is the information about the model,
  • and the return type is the request body as a string sent to the endpoint.

Output Transform

The output adapter is a function that maps the body of the response of the model endpoint to the model output format expected by AI GO!. It is defined as a Jinja template with signature equivalent to:

def process_output(body: str, model_info: ModelInfo) -> str: ...

where:

  • body is the response body given a string (if it is a JSON string, use {% set body = body | fromjson %} to parse it into a dictionary),
  • model_info is the information about the model,
  • and the return is the model output in AI GO! format (see Model I/O Types) given as a JSON string.

Jinja Filters

Jinja templates can use different Jinja filters to do data manipulation. In addition to built-in filters defined by Jinja, AI GO! supports the filters listed in the table below.

Filter

Description

Example

Built-in

{{ input | tojson }}

fromjson

Parses a JSON string.

{% set body = body | fromjson %}

Now nested data can be accessed:

body.choices[0]

json_escape

Makes a string safe to be inserted into a JSON string. This is needed because inserting a raw string that contains quotes or newlines into a JSON string results in invalid output.

"model": "{{model_info.model_key | json_escape}}"

Usage

Commands

Define the model adapter in a YAML file adapter.yaml.

display_name: "<display_name>"
key: "<key>"
# Optional: description: "<description>"
# Optional: long_description: "<long_description>"
task: "<task>"
process_input:
  language: "jinja"
  source_code: <input_transform>
process_output:
  language: "jinja"
  source_code: <output_transform>

Create the model adapter as defined in adapter.yaml. This will persist the model adapter in AI GO!.

  • If there is no model adapter with the key provided in the YAML file, a new model adapter is created.
  • If a model adapter with the key provided in the YAML file already exists, the model adapter is updated if the data has changed.
lf model-adapter add -f 'adapter.yaml'          # Create/update a single model adapter from a YAML file.
lf model-adapter add -f 'model_adapter/*.yaml'  # Create/update multiple adapters in a directory with YAML files.

List all model adapters.

lf model-adapters

Delete the model adapter by key.

lf model-adapter delete 'my-adapter'

Export the model adapter by key.

lf model-adapter export 'my-adapter'                    # Export to STDOUT as JSON
lf model-adapter export 'my-adapter' -o 'adapter.yaml'  # Export into a YAML file

Testing

Model adapters can be tested as part of testing the model.