Skip to main content

LLM Backends

The Level 3 LLM call is a stateless operation that sends messages and tool definitions to a language model and returns the response. Covia supports multiple backends via the LangChain adapter.

Available Backends​

OperationProviderNotes
v/ops/langchain/openaiOpenAIDefault. Supports structured outputs (strict: true).
v/ops/langchain/anthropicAnthropicClaude models via native Anthropic API.
v/ops/langchain/ollamaOllamaLocal models. Requires Ollama running on the venue host.
v/ops/langchain/ollama_qwen3Ollama (Qwen3)Qwen3-specific configuration for Ollama.

Configuration​

Set the backend via llmOperation in agent config:

{
  "operation": "agent:create",
  "input": {
    "agentId": "claude-agent",
    "config": {
      "llmOperation": "v/ops/langchain/anthropic",
      "model": "claude-sonnet-4-20250514",
      "systemPrompt": "You are a helpful assistant."
    }
  }
}

OpenAI​

The default backend. Supports all OpenAI models and structured outputs.

{
  "llmOperation": "v/ops/langchain/openai",
  "model": "gpt-4o"
}

Structured outputs: When an agent uses typed outputs (Goal Tree outputs config) or responseFormat, the OpenAI backend enables strictJsonSchema and strictTools to guarantee schema conformance.

Secrets required: OPENAI_API_KEY — set via the secrets manager:

{ "operation": "secret:set", "input": { "key": "OPENAI_API_KEY", "value": "sk-..." } }

Anthropic​

Native Anthropic API integration for Claude models.

{
  "llmOperation": "v/ops/langchain/anthropic",
  "model": "claude-sonnet-4-20250514"
}

Secrets required: ANTHROPIC_API_KEY

Ollama​

Run local models via Ollama. The Ollama server must be accessible from the venue host.

{
  "llmOperation": "v/ops/langchain/ollama",
  "model": "llama3.1"
}

No API key required — Ollama runs locally.

Message Format​

All backends use the same message format:

{
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "What is the weather?" },
    { "role": "assistant", "content": "I'll check that for you.", "toolCalls": [...] },
    { "role": "tool", "id": "call_123", "name": "weather_check", "content": "{...}" }
  ],
  "tools": [...],
  "responseFormat": { "name": "Response", "schema": {...} }
}

Response:

{
  "role": "assistant",
  "content": "The weather is sunny.",
  "toolCalls": null
}

If toolCalls is present, the Level 2 adapter executes them and calls the LLM again.