Skip to content

[Unified Routing] Outstanding Tasks and Future Enhancements #554

Closed
#556
Closed
[Unified Routing] Outstanding Tasks and Future Enhancements#554
#556

Description

@kimiko-terraphim
kimiko-terraphim
opened on Feb 22, 2026

Overview

This issue tracks outstanding tasks and future enhancements for the unified routing system (terraphim_router + terraphim_spawner).

Completed

  • terraphim_router crate with capability-based routing
  • 4 routing strategies (CostOptimized, LatencyOptimized, CapabilityFirst, RoundRobin)
  • ProviderRegistry with Markdown/YAML frontmatter support
  • terraphim_spawner crate for agent process management
  • Health checking with 30s heartbeat
  • Output capture with @mention detection
  • @mention routing for decentralized coordination
  • FallbackRouter with 4 fallback strategies
  • RouterMetrics for observability
  • KnowledgeGraphRouter for smart routing
  • Comprehensive integration tests
  • Examples (basic + advanced)
  • Documentation

High Priority -- DONE (PR #556)

1. terraphim_persistence Integration

  • PersistedProviderRegistry with Persistable trait
  • Feature-gated behind persistence feature
  • Write-through persistence to DeviceStorage
  • 7 persistence tests with real DeviceStorage (memory backend)

2. OpenTelemetry Integration

  • Replace log with tracing for structured spans
  • Privacy helpers (prompt_preview, prompt_hash)
  • Spans on routing engine, fallback router, all strategies
  • OTel export feature-gated for future use

3. terraphim_llm_proxy Integration

  • RouterBridgeLlmClient connecting router to LlmClient execution
  • Capability-based provider selection (Ollama, OpenRouter)
  • Single-provider optimization (skip routing overhead)
  • 11 bridge tests covering strategy mapping, routing, fallback
  • Removed dead RoutedLlmClient pass-through wrapper

Medium Priority -- DONE (PR #556)

4. Dynamic Strategy Selection

  • Context-aware strategy selection via RoutingContext.strategy_override
  • PreferenceFilter for cost/latency constraints
  • WeightedStrategy for deterministic A/B testing
  • StrategyRegistry with factory pattern for named lookup
  • 11 new strategy tests

5. Advanced Health Checking

  • CircuitBreaker with Closed/Open/HalfOpen state machine
  • CircuitBreakerConfig (failure_threshold, cooldown, success_threshold)
  • HealthHistory with sliding window and success rate calculation
  • Degraded health status for gradual degradation detection
  • 15 new circuit breaker and health history tests

6. Agent Lifecycle Management

  • Graceful shutdown (SIGTERM -> grace period -> SIGKILL)
  • Non-blocking try_wait() for exit status polling
  • AgentPool with checkout/release/drain semantics
  • Configurable max restart attempts
  • 5 new lifecycle tests

Low Priority -- DONE (branch feat/router-tracing)

7. WebSocket Support for Agents

  • Broadcast channel (capacity 256) for live agent output streaming
  • subscribe_output() on AgentHandle for callers
  • WebSocket server endpoint (future terraphim_server task)
  • Browser-based agent UI

8. Provider Discovery

  • Dynamic provider registration with RegistryEvent notifications
  • File-watching hot-reload via notify (feature-gated: file-watch)
  • 500ms debounced reload, optional change notifications
  • mDNS/Bonjour discovery (deferred -- too heavyweight)
  • Service mesh / load balancer integration

9. Security Hardening

  • ResourceLimits (memory, CPU, file size, open files) via rlimit
  • Structured audit events via tracing target
  • Full sandboxing via terraphim_firecracker (sub-2s boot, VM pooling)
  • Secret management integration

Technical Debt -- DONE (branch feat/router-tracing)

Code Quality

  • Property-based tests (proptest) for strategies, keywords, circuit breaker
  • Criterion benchmarks for routing performance
  • Spawner migrated from log to tracing
  • Dead code warnings resolved (accessors added)
  • Extract common traits between Router and Spawner
  • Unify error types across crates

Related PRs

Milestone

v2.0 Unified Routing Release

Activity

  1. AlexMikhalev
    added 2 commits that reference this issue on Feb 22, 2026

    feat(terraphim_router): replace log with tracing for structured obser…

    fc76fb9

    feat(terraphim_router): add feature-gated persistence for ProviderReg…

    5120abf

  2. AlexMikhalev commented on Feb 22, 2026

    @AlexMikhalev
    AlexMikhalev
    on Feb 22, 2026
    Contributor

    Low-Priority Tasks + Technical Debt -- DONE (branch feat/router-tracing)

    6 additional commits pushed covering all remaining items:

    Commit A: Cleanup and Tracing Migration (d9d410b6)

    • Migrated terraphim_spawner from log to tracing with structured fields
    • Added output_capture() accessor and info_span!("spawner.spawn")
    • Added source_path() accessor and remove_provider() to ProviderRegistry

    Commit B: Security Hardening (261afa39)

    • ResourceLimits struct (max_memory_bytes, max_cpu_seconds, max_file_size_bytes, max_open_files)
    • Applied via pre_exec hook using nix::sys::resource::setrlimit (unix-only)
    • AuditEvent enum with structured audit logging via tracing::info!(target: "terraphim_spawner::audit", ...)
    • Note: Full sandboxing already available via terraphim_firecracker (sub-2s boot, VM pooling)

    Commit C: WebSocket Support (cdaedb4d)

    • broadcast::Sender<OutputEvent> (capacity 256) for live agent output streaming
    • subscribe_output() on AgentHandle for callers to receive events
    • Server integration (WebSocket endpoint) is a future terraphim_server task

    Commit D: Provider Discovery (2c33faaa)

    • RegistryEvent enum (ProviderAdded, ProviderRemoved, ProvidersReloaded) via broadcast channel
    • ProviderDirectoryWatcher using notify for hot-reload (feature-gated: file-watch)
    • 500ms debounced reload, optional change notifications
    • mDNS/Bonjour deferred as too heavyweight for this scope

    Commit E: Property-Based Tests (3f3f9dd7)

    • proptest for strategy invariants (cost/latency selection correctness, round-robin cycling)
    • Keyword extraction fuzz testing (never panics, returns subset of all capabilities)
    • Circuit breaker state machine testing, health history success rate always in [0.0, 1.0]

    Commit F: Benchmarks (e7959f25)

    • Criterion 0.8 benchmarks: keyword_extraction, strategy_selection, full_routing
    • 4 strategies x 4 provider counts + 4 prompt types x 3 provider counts
    • Run: cargo bench --bench routing_bench -p terraphim_router

    Test Summary

    • Router: 44 unit + 3 integration + 1 doc test = 48 passing
    • Spawner: 32 tests passing
    • Total: 80 tests, all green

    Remaining Future Work

    • WebSocket server endpoint integration (terraphim_server)
    • mDNS/Bonjour discovery
    • Service mesh / load balancer integration
    • Extract common traits between Router and Spawner
    • Unify error types across crates
  3. AlexMikhalev
    closed this as completedin #556on Feb 22, 2026
  4. AlexMikhalev
    added 2 commits that reference this issue on Apr 25, 2026

    feat: complete unified routing system (tasks 1-9 + technical debt) (#556

    cdb411d

    feat: complete unified routing system (tasks 1-9 + technical debt) (#556

    f55a1ab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions