AI GO! expects that the integrated models conform to its API - i.e. use the expected model input/output types.

AI GO! allows the user to adapt the input/output types of the model to the expected format without the need to modify its endpoints. This is done by model adapters.

ML Tasks

Model I/O types depends on the model's task.

Task	Description	Key
Chat completion	Task for LLM that generate a textual response from a list of messages comprising a conversation. See more here.	`chat_completion`
Retrieval-Augmented Generation (RAG)	Chat completion task where the output of the LLM is enhanced by additional information from the knowledgebase.	`chat_completion`
Custom	For models of any other task that is not supported. There is no schema on the input/output types. These outputs will be passed unchanged to the evaluators.	`custom`

The sections below define the expected types for each task as a Python (JSON) example and as a Python TypedDict.

Chat Completion

{
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hi"},
    ]
}

class ChatCompletionInput(TypedDict):
    messages: list[ChatCompletionMessage]

class ChatCompletionMessage(TypedDict):
    role: Role
    content: str
    refusal: NotRequired[str]
    
Role = Literal["user", "system", "assistant"]

Retrieval-Augmented Generation (RAG)

{
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of Switzerland?"},
    ]
}

class RAGCompletionInput(TypedDict):
    messages: list[ChatCompletionMessage]

class ChatCompletionMessage(TypedDict):
    role: Role
    content: str
    refusal: NotRequired[str]
    
Role = Literal["user", "system", "assistant"]