Broker Routing Engine

The Broker routes tasks to the most cost-effective handler. It tries cheaper options first before falling back to expensive LLM calls.

Routing Cascade

Incoming Task
     │
     ▼
┌─────────────┐
│ Skill Match  │ ← Pattern matching against registered skills
│ (free, <1ms) │
└──────┬──────┘
       │ no match
       ▼
┌─────────────┐
│ WASM Handler │ ← Execute local WASM binary
│ (~$0.001)    │
└──────┬──────┘
       │ no handler
       ▼
┌─────────────┐
│ LLM Fallback │ ← Send to language model API
│ (~$0.03+)    │
└─────────────┘

How Skill Matching Works

The broker compares the incoming task against registered skill patterns:

skills:
  - name: summarize
    handler: ./skills/summarize.wasm
    patterns:
      - summarize
      - summary
      - tldr
  - name: classify
    handler: ./skills/classify.wasm
    patterns:
      - classify
      - categorize
      - label

When a task like "summarize the quarterly report" arrives, the broker:

  1. Tokenizes the task string
  2. Matches against skill patterns
  3. Calculates a confidence score (0.0 - 1.0)
  4. Routes to the skill if confidence >= threshold (default 0.7)

Testing Routes

Preview where a task would be routed without executing:

naos broker route "summarize the quarterly report"

Output:

ROUTING DECISION
Task:       "summarize the quarterly report"
Route:      skill → summarize
Confidence: 0.93
Est. Cost:  $0.00
Est. Time:  <1ms

Configuration

broker:
  enabled: true
  routing:
    preferSkill: true
    llmAsLastResort: true
    confidenceThreshold: 0.7