Skip to content
The Gateway

AI assistant

The Gateway exposes two distinct surfaces under /api/v1/ai/:

  1. /ai/policy — the actual AI assistant. Forwards a natural-language query to a local Ollama instance, returns a structured policy action.
  2. /ai/validate — a rule-based reference validator (no LLM involved). Checks a proposed object against the SQLite store: missing profile, undefined privacy class, duplicate VLAN ID, etc.

Both are optional. Ollama integration is set up by the full install profile.

/ai/policy — the assistant

Section titled “/ai/policy — the assistant”

Request:

{ "query": "Block YouTube for the kids group between 22:00 and 07:00" }

The daemon prepends its system prompt (which describes the supported resources and API conventions), forwards to Ollama (default model configurable via ai_default_model in kv_settings), and parses the model’s JSON output back into a typed action:

{
  "query":       "Route YouTube through the EU tunnel",
  "model":       "llama3:8b",
  "explanation": "Route YouTube domains through the wg-eu-01 WireGuard tunnel, fail-closed.",
  "action": {
    "type":     "create",
    "resource": "routing-policy",
    "endpoint": "/api/v1/routing/policies",
    "method":   "POST",
    "fields": {
      "name":         "youtube-via-eu",
      "priority":     80,
      "source_type":  "domain",
      "source_value": "*.youtube.com",
      "hops":         [{"type":"vpn","target":"wg-eu-01"}],
      "fail_policy":  "fail-closed"
    }
  },
  "latency_ms": 2843,
  "raw":        "...the raw model output, for debugging..."
}

The system prompt teaches the LLM the real routingPolicy shape (source_type, source_value, hops) so the fields block is structurally valid — but you still read the explanation, sanity-check the fields, then curl the suggested endpoint. The assistant cannot mutate state directly. Per-policy time-of-day matching isn’t a real source_type, so a query like “between 22:00 and 07:00” will typically come back without a working time-window (the LLM may hallucinate one; the validator catches it on submit).

Identical (model, query) pairs are cached server-side for a short TTL — repeated questions don’t burn Ollama cycles.

/ai/validate — the reference checker

Section titled “/ai/validate — the reference checker”

This isn’t AI. It’s a small set of rules:

{
  "action":   "create",
  "resource": "device",
  "name":     "laptop-alice",
  "fields":   { "profile": "high-privacy", "privacy_class": "paranoid", "trust_class": "operator" }
}

Response:

{
  "errors":      ["profile \"high-privacy\" does not exist"],
  "warnings":    [],
  "suggestions": ["Did you mean \"high_privacy\"? Profiles in DB: balanced, paranoid, streaming."]
}

Use it from a UI form’s onChange before allowing submit, or from CI to lint a config bundle.

Why local

Section titled “Why local”

Three reasons The Gateway uses a local LLM by default:

  1. Privacy. The assistant sees your policy structure, tunnel names, and device IDs. None of this should leave the box.
  2. Latency. A local Ollama on the same Pi answers in a few seconds; a cloud round-trip is much slower in censored / high-latency environments.
  3. Availability. The assistant should work even when the WAN is down.

You can point at a remote endpoint via kv_settings.ai_ollama_url if you’d rather use a hosted model.

API surface

Section titled “API surface”
Method Path Notes
POST /api/v1/ai/policy Natural-language → structured action (Ollama)
POST /api/v1/ai/validate Rule-based reference check (no LLM)
GET /api/v1/ai/status Is Ollama reachable? Is a model loaded?
GET/PUT /api/v1/ai/config Endpoint URL, default model
POST /api/v1/ai/service Start / stop the local Ollama service
GET /api/v1/ai/models/running Loaded models
POST /api/v1/ai/models/load Load a model
POST /api/v1/ai/models/unload Unload a model from memory (file stays on disk)
POST /api/v1/ai/models/pull Pull a new model from the Ollama registry
DELETE /api/v1/ai/models/{name} Delete a model from disk

Next steps

Section titled “Next steps”

If you want to use the assistant:

  1. Pick an endpoint. Either run Ollama on the same box (a 7B model is fine on a Pi 4 for short prompts) or point at a remote one via ai_ollama_url. See Settings.
  2. Always run /ai/validate on the proposed object. The rule-based check catches dangling references, ID collisions and missing fields the LLM cheerfully hallucinates.
  3. Treat the output as a draft, not a commit. Review the JSON, apply via the normal /api/v1/routing/policies endpoint, then gwctl evaluate the affected device to confirm.
  4. Skip it entirely if you don’t want it. Nothing else in the daemon depends on the assistant — leaving ai_ollama_url empty disables the feature with no side effects.
Section titled “Related”