⬡ K9 — Personal AI Assistant

Autonomous, Stateful, and Kinetic AI with Multi-Tier Memory, Agentic Planning, and Iron Man HUD

Overview • Architecture • Memory System • Agent Pipeline • Skills & Tools • Interface & Voice • Quickstart • Docs

---

🌐 Overview

Most local AI assistants are simple stateless script launchers: they freeze while fetching web data, suffer from total amnesia on follow-up questions, cannot resolve pronouns like "Who is he?", and expose basic console outputs.

K9 is engineered as an Operating System for Personal Cognition:

• 🧠 True Contextual Continuity: Retains conversational memory across exchanges. You can ask "Who founded SpaceX?", follow up with "Where was he born?", and K9 dynamically resolves pronouns through real-time entity tracking.
• 💾 4-Tier Memory Architecture: Incorporates rolling short-term buffers, permanent deduplicated JSON facts, process-scoped entity graphs, and 384-dimensional FAISS semantic vector recall.
• ⚡ Asynchronous & Non-Disruptive: Long-running operations (web scraping, weather, multi-tool executions) run in background worker threads without freezing speech or UI responsiveness.
• 🛡️ Plan-Validate-Execute Lifecycle: Queries are evaluated by an autonomous PlannerAgent, checked against safety and privacy policies by PlanValidator, and dispatched by an ExecutionAgent.
• 🖥️ Kinetic Iron Man HUD Terminal: Full-screen cybernetic TUI built on prompt_toolkit, featuring live telemetry badges, scroll inspection mode, command history, and audio waveform synchronization.

---

🏗️ System Architecture

K9 is organized into clear architectural subsystems that separate sensory inputs, orchestration, reasoning agents, memory tiers, and physical actuators.

flowchart TB
    subgraph Inputs ["  Sensory Inputs  "]
        KB["Keyboard Input (TUI Field)"]
        MIC["Microphone Stream (Faster-Whisper / Google)"]
        WAKE["Wake Word Sentinel ('K9' Pattern)"]
    end

    subgraph NervousSystem ["  Asynchronous Nervous System  "]
        EB["EventBus (Priority Queue)"]
        TM["TaskManager (Background Workers & Cron)"]
    end

    subgraph CognitiveCore ["  Cognitive Core (Brain)  "]
        BRAIN["Brain (Main Orchestrator)"]
        ROUTER["Router (Deterministic Fast-Path + LLM Fallback)"]
        
        subgraph AgentSwarm ["Agent Pipeline"]
            PLANNER["PlannerAgent (Query Decomposition)"]
            VALIDATOR["PlanValidator (Safety & Privacy Gate)"]
            EXECUTOR["ExecutionAgent (Parallel Execution)"]
        end
    end

    subgraph MemoryTiers ["  4-Tier Memory Subsystem  "]
        M1["Tier 1: Entity Memory (Pronoun Resolution)"]
        M2["Tier 2: Rolling Conversation Buffer (10-15 Turns)"]
        M3["Tier 3: Permanent Facts (Active Pointer JSON)"]
        M4["Tier 4: Episodic Vector Store (FAISS 384-Dim)"]
    end

    subgraph SkillRegistry ["  Extensible Tool Registry  "]
        T_WEB["Web Search (Tavily, Serper, Exa, Brave)"]
        T_WEATHER["Weather (Open-Meteo Live API)"]
        T_MEM["Memory Search & Store"]
        T_SYS["System Control (Apps & Volume)"]
        T_GEN["Generic Response (K9 Persona)"]
    end

    subgraph Actuators ["  Actuators & Display  "]
        TUI["TuiRenderer (Kinetic Iron Man HUD)"]
        TTS["Vocal Engine (ElevenLabs / EdgeTTS / pyttsx3)"]
    end

    %% Input flows
    KB --> EB
    MIC --> WAKE
    WAKE --> EB
    
    %% Nervous system to Brain
    EB --> BRAIN
    TM <--> BRAIN

    %% Core pipeline
    BRAIN --> ROUTER
    ROUTER --> PLANNER
    PLANNER --> VALIDATOR
    VALIDATOR --> EXECUTOR
    EXECUTOR --> SkillRegistry

    %% Memory bindings
    ROUTER <--> M1
    BRAIN <--> M2
    SkillRegistry <--> M3
    BRAIN <--> M4

    %% Outputs
    BRAIN --> TUI
    BRAIN --> TTS

Loading

---

🧠 The 4-Tier Cognitive Memory System

K9 solves the "AI amnesia" bottleneck by separating memory into four specialized cognitive layers:

graph LR
    subgraph QueryFlow ["Incoming Turn"]
        Q["User Query"]
    end

    subgraph T1 ["Tier 1: Entity Memory"]
        E1["In-Memory Graph<br/>• Pronoun Resolution ('he' -> 'Elon')<br/>• Gender & Entity Confidence (1.0)"]
    end

    subgraph T2 ["Tier 2: Conversational Context"]
        E2["Rolling Context Buffer<br/>• Last 10-15 Turns (~8,000 Chars)<br/>• Preserves 'Yes/No' Continuity"]
    end

    subgraph T3 ["Tier 3: Permanent State"]
        E3["Atomic JSON Memory<br/>• Normalization Index (No Dupes)<br/>• Versioned Active Pointer System"]
    end

    subgraph T4 ["Tier 4: Semantic Recall"]
        E4["Deep Episodic Vectors<br/>• FAISS + all-MiniLM-L6-v2<br/>• Importance Scoring (Scorer >= 0.65)"]
    end

    Q --> T1
    T1 --> T2
    T2 --> T4
    T4 --> T3

Loading

Memory Layers Comparison

Memory Layer	Storage Medium	Lifecycle	Latency	Key Function
1. Entity Memory	Process-Scoped Heap	Current Session	<1 ms	Resolves pronouns ("Who is he?") using context confidence matching
2. Rolling Conversation	Volatile RAM Buffer	Current Session	<2 ms	Retains the last 10–15 dialogue turns to support follow-up questions
3. Permanent Facts	Atomic Versioned JSON	Persistent (memory/)	~5 ms	Stores explicit facts ("My flight is at 8 AM") with deduplication index
4. Deep Episodic Memory	FAISS Vector Store	Persistent (memory/)	~12 ms	Embeds technical states & summaries via sentence-transformers for recall

Tip

Data Integrity (Active Pointer System): Every memory save writes a new immutable timestamped version (e.g. 2026-09-11_v1740900.json) and atomically flips meta.json. If a sudden power cut occurs during writing, K9 gracefully falls back to the previous snapshot without corruption.

---

🤖 Multi-Agent Planning & Routing

K9 utilizes a three-phase deliberation cycle to turn natural language commands into verified actions:

sequenceDiagram
    autonumber
    actor User
    participant Router as Multi-Phase Router
    participant Planner as PlannerAgent
    participant Validator as PlanValidator
    participant Executor as ExecutionAgent
    participant Skills as Tool Registry
    participant Brain as K9 Brain

    User->>Router: "Check weather in Tokyo and tell me about its airport"
    Note over Router: Structural Clause Splitter identifies 2 sub-goals
    Router->>Planner: Sub-goal 1: "Check weather in Tokyo"
    Planner-->>Router: Plan: [weather(city="Tokyo")]
    Router->>Planner: Sub-goal 2: "tell me about its airport"
    Planner-->>Router: Plan: [web_search(query="Tokyo Haneda Narita airport")]
    
    Router->>Validator: Validate compound execution plan
    Note over Validator: Confirms tool schemas, privacy policies & safety constraints
    Validator-->>Router: Validation: OK (Confidence: 0.98)
    
    Router->>Executor: Execute verified steps
    par Parallel Execution
        Executor->>Skills: weather(city="Tokyo")
        Executor->>Skills: web_search(query="Tokyo airport")
    end
    Skills-->>Executor: Structured Result Payloads
    Executor-->>Brain: Aggregate ToolResults
    Brain-->>User: Synthesize unified witty response via HUD + Vocal Engine

Loading

---

🛠️ Skills & Tool Ecosystem

All capabilities in K9 are implemented as modular, dynamically registered tools conforming to the Tool contract:

skills/
├── generic_response.py  # Small talk, banter, and witty personality responses
├── memory_search.py     # Semantic & factual memory queries
├── memory_store.py      # Permanent state, fact, and rule storage
├── weather.py           # Real-time weather, forecasts & ambient conditions
├── web_search.py        # Multi-provider live web retrieval (Tavily, Serper, Exa)
├── system_control.py    # Native OS application launcher & control
├── time.py              # Timezone, clock, and calendar arithmetic
├── safe_response.py     # Guardrail refusal for medical/legal/financial advice
└── async_test.py        # Background task & stress validation tool

Search Provider Cascade

K9 does not rely on a single search API. If one provider hits rate limits or latency spikes, it automatically cascades:

$$\text{Tavily} \longrightarrow \text{Serper.dev} \longrightarrow \text{Exa.ai} \longrightarrow \text{Brave Search} \longrightarrow \text{SerpAPI}$$

---

🖥️ Kinetic HUD Terminal & Voice

K9 features a custom Iron Man-inspired terminal interface powered by prompt_toolkit.

┌────────────────────────────────────────────────────────────────────────────┐
│ ⬡ K9 // ADVANCED KINETIC INTERFACE // ACTIVE SESSION     ● SYSTEM NOMINAL  │
├────────────────────────────────────────────────────────────────────────────┤
│                                                                            │
│  [SYSTEM] Core systems online. Episodic memory primed. EventBus active.   │
│                                                                            │
│  You : Who founded SpaceX and what is his net worth?                       │
│  ◈ [ROUTER] Intent: multi-phase query → PlannerAgent (confidence: 0.98)    │
│  K9  : SpaceX was founded in 2002 by Elon Musk. Current net worth is      │
│        estimated at ~$210B. Entity locked: [Elon Musk: Male (conf=1.00)].  │
│                                                                            │
│  You : Where was he born?                                                  │
│  K9  : Elon Musk was born in Pretoria, South Africa.                      │
│                                                                            │
├────────────────────────────────────────────────────────────────────────────┤
│ ▸ what tasks are currently running in the background?                      │
├────────────────────────────────────────────────────────────────────────────┤
│ [F1] HELP  [TAB] SCROLL MODE  [PGUP/DN] 15L  [CTRL+C] EXIT │ ONLINE │ TTS  │
└────────────────────────────────────────────────────────────────────────────┘

Keyboard Controls

Key Shortcut	Action	Description
Enter	Send Command	Dispatches command to K9 and clears prompt
↑ / ↓	Command History	Cycle through previously executed commands
Tab	Scroll Mode	Focuses chat window for line-by-line inspection
PgUp / PgDn	Fast Scroll	Jump 15 lines up or down through session logs
Esc	Exit Scroll Mode	Snap view back to the latest incoming message
Ctrl+C	Shutdown	Gracefully flushes vector memory and exits

---

🚀 Quickstart

1. Clone & Setup Environment

# Clone the repository
git clone https://github.com/parmarth-kumar/K9.git
cd K9

# Create and activate Python virtual environment
python -m venv .venv

# Windows:
.venv\Scripts\activate

# Linux/macOS:
source .venv/bin/activate

# Install required packages
pip install -r utilities/requirements.txt

2. Configure Environment Variables

Create your local .env file from the provided template:

cp .env.example .env

Open .env and configure your API keys:

# Required for primary LLM reasoning (Groq is recommended for ultra-low latency)
GROQ_API_KEYS=gsk_your_groq_key_here

# Optional: Secondary fallback LLM
OPENAI_API_KEYS=sk-proj-your_openai_key_here

# Required for Web Intelligence (at least one recommended)
TAVILY_API_KEYS=tvly-your_tavily_key_here
SERPER_API_KEYS=your_serper_key_here

# Audio Output (Optional: Defaults to local offline Windows SAPI pyttsx3)
ELEVENLABS_API_KEYS=sk_your_elevenlabs_key_here

3. Launch K9

python main.py

Note

On startup, K9 performs a self-validating bootstrap test, initializes the FAISS vector index, warms up embedding dimensions, and presents the full-screen kinetic HUD.

---

📚 Deep-Dive Documentation

Every component in K9 is comprehensively documented in the repository:

• 📑 Complete System Workflow: End-to-end mermaid lifecycle, data schemas, and event loop.
• 🧠 Memory Architecture Spec: Detailed mechanics of short-term, fact, entity, and episodic memory.
• 🔄 Contextual Continuity in Chat: Pronoun resolution and conversational state maintenance.
• ⚡ Asynchronous Parallel Tasks: Non-blocking worker threads, heartbeat updates, and the TaskManager.
• 🎙️ STT & TTS Strategic Ecosystem: Benchmarks for Deepgram, Whisper, ElevenLabs, and Cartesia.
• 🛡️ Offline-First Gatekeeper Plan: Air-gapped local model execution using Ollama & Kokoro.
• 👂 Wake Word Detection Guide: Implementing OpenWakeWord sentinel listeners.
• 🗺️ Engineering Roadmap: Feature progression across upcoming milestones.

---

Built with pride for high-performance personal computing.

_{K9 Personal AI Assistant • MIT Licensed • 2026}