AI Providers

Koyal ships with 55+ AI provider presets. Providers are configured via environment variables and selected at runtime — no rebuild required.

Configuration

# Primary provider (used for all agent ticks unless overridden)
AI_GATEWAY=anthropic
AI_MODEL=claude-sonnet-4-6

# API keys (only the ones you use are required)
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GEMINI_API_KEY=...
GROQ_API_KEY=gsk_...

Or configure via the dashboard: Settings → AI → Gateway and Settings → AI → Model.

Supported Providers

Tier 1 — Directly integrated

Provider	Env Var	Notes
OpenAI	`OPENAI_API_KEY`	GPT-4o, o1, o3, GPT-4.1-mini
Anthropic	`ANTHROPIC_API_KEY`	Claude 3.5, Claude 4 family
Google Gemini	`GEMINI_API_KEY`	Gemini 2.0 Flash, 1.5 Pro
AWS Bedrock	`AWS_*`	Claude, Llama, Titan, Nova

Tier 2 — OpenAI-compatible endpoints

Provider	Gateway ID	Notes
DeepSeek	`deepseek`	R1, V3, Coder
Groq	`groq`	Llama 3.3, Mixtral, Gemma
Together AI	`together`	Llama, Qwen, WizardLM
OpenRouter	`openrouter`	200+ models via one key
Mistral	`mistral`	Large, Nemo, 7B
Perplexity	`perplexity`	sonar-pro, sonar-research
xAI	`xai`	grok-3, grok-2
Fireworks	`fireworks`	Llama, Mixtral
Cerebras	`cerebras`	Fast inference
Lepton AI	`lepton`	Llama, Mistral
Cohere	`cohere`	Command R+
AI21	`ai21`	Jamba
Replicate	`replicate`	Community models
HuggingFace	`huggingface`	Inference API

Plus 35+ more in the catalog (crates/koyal-providers/src/catalog.rs).

Provider Selection

resolve_provider() in koyal-providers selects the active provider:

Check AI_GATEWAY env / config setting
If not set, try providers in priority order based on available API keys
On failure, activate circuit breaker and try next in failover chain
Log every provider call with tokens, latency, and cost estimate

Model Router

The model router enables failover chains:

# config.toml (example)
ai_gateway = "anthropic"
ai_model = "claude-sonnet-4-6"

# Failover: if anthropic is down, try openai
[[ai_profiles]]
gateway = "openai"
model = "gpt-4o"
priority = 2

Circuit breaker: after 3 consecutive failures, a provider is bypassed for 60 seconds.

Encrypted Credentials

Credentials are stored encrypted with AES-256-GCM:

# Credentials are read from env at startup and stored encrypted
# Never written to plain text on disk
koyal ai profile-list    # list stored profiles
koyal ai health          # test all configured providers

Per-Provider Health Monitoring

koyal ai health
# Checks each configured provider with a minimal API call
# Reports latency, token counts, and status

koyal ai budget --limit 100
# Shows token usage and estimated cost per provider

The dashboard System page shows provider health in real time via WebSocket updates.

Usage Tracking

Every AI call is tracked in UsageStore:

{
  "provider": "anthropic",
  "model": "claude-sonnet-4-6",
  "purpose": "agent_tick",
  "agent": "trader1",
  "inputTokens": 1240,
  "outputTokens": 380,
  "latencyMs": 1823
}

Access via GET /api/usage or koyal ai budget.

Keyboard shortcuts

Koyal Docs