PEQRouter

PEQRouter is a private, efficient, and quality-aware router for generative AI. For each prompt, it chooses where inference should run: on a phone, on a PC, or in the cloud.

The goal is simple: keep sensitive work local, use the smallest capable device when practical, and reach for the cloud when a request needs more model quality. PEQRouter combines local privacy checks, prompt-level quality estimates, and current device telemetry to make that choice.

Destination	Current runtime	Best suited for	Privacy behavior
Phone	Qwen3-0.6B through the GenieX Android app	Fast, low-energy requests	Remains local; never sent to cloud
PC	Qwen3-VL-4B-Instruct on a Snapdragon X-Elite NPU	Stronger local inference	Remains local; never sent to cloud
Cloud	Llama-3.3-70B through Cirrascale	Requests that need the most capability	Used only for prompts not marked sensitive

Presentation

How it works

1. Analyze the prompt locally for sensitive information and likely task difficulty.
2. Remove unavailable or privacy-incompatible destinations.
3. Check which remaining models are likely to meet the quality requirement.
4. Rank them using the selected latency, energy, and quality profile.
5. Return the decision or dispatch it to an explicitly enabled live executor.

flowchart LR
    P[Prompt] --> A[Local analysis]
    T[Device telemetry] --> R[Privacy and quality gates]
    A --> R
    R --> S[Profile-based ranking]
    S --> D{Selected destination}
    D --> PH[Phone]
    D --> PC[PC]
    D --> CL[Cloud]

Loading

Routing is safe to explore: execution is simulated unless the corresponding --live-phone, --live-pc, or --live-cloud option is supplied. Routing alone never enables live execution, and diagnostics retain privacy categories rather than matched prompt text.

Project snapshot

PEQRouter currently includes:

• a Python routing library and CLI;
• simulated and live executors for all three destinations;
• an Android phone server and a Snapdragon X-Elite PC server;
• four routing profiles: balanced, low-latency, low-energy, and high-quality;
• an optional prompt-level quality and answer-length estimator calibrated on 106 prompts; and
• latency, token, throughput, and energy observations where the runtime exposes enough data.

The calibrated device baselines are:

Device	Model	Throughput	Energy	Samples
Phone	Qwen3-0.6B	94.22 tok/s	0.0412 J/token	106
PC	Qwen3-VL-4B-Instruct (X-Elite NPU)	20.19 tok/s	0.4447 J/token	106
Cloud	Llama-3.3-70B	33.37 tok/s*	2.2475 J/token*	100

* Cloud throughput is end-to-end, including network and queueing time. Cloud energy is derived from the accelerator's rated 75 W TDP rather than measured request-level power. Phone and PC figures come from device calibration; see Design and implementation for the measurement boundaries.

Each profile expresses a different product preference:

Profile	Latency	Energy	Quality
balanced	0.500	0.333	0.167
low-latency	0.850	0.075	0.075
low-energy	0.150	0.800	0.050
high-quality	0.100	0.100	0.800

When the 106 calibration prompts are routed with healthy telemetry from a PC, the profiles produce the following distribution:

Profile	Phone	PC	Cloud
balanced	48	31	27
low-latency	51	8	47
low-energy	52	45	9
high-quality	0	15	91

These are in-sample calibration results, not a held-out evaluation. They illustrate how the profiles change routing behavior.

Quick start

PEQRouter requires Python 3.11 or newer.

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e .

Route a prompt without executing it:

python -m peqrouter route `
  --origin phone `
  --prompt "Compare three routing strategies" `
  --profile balanced `
  --scenario healthy

Route and execute with the built-in simulators:

python -m peqrouter run `
  --origin pc `
  --prompt-file request.txt `
  --scenario pc-congested `
  --json

For sensitive prompts, prefer stdin or --prompt-file so the text is not stored in shell history:

"Summarize the profile for John Smith" | python -m peqrouter route --origin pc

Live executors require their own server or credentials and remain opt-in. Setup instructions for the phone, PC, and cloud are in the design document.

Documentation

• Design and implementation — routing policy, telemetry, executors, estimator, APIs, and energy methodology
• Roadmap — concise list of deferred work
• Android app — phone runtime setup
• Benchmarks — calibration and evaluation workflows

Tests

python -m unittest discover -s tests -v

The suite is offline and uses mocked network calls. See the design document for optional dependencies and live smoke testing.

Contributors

• Kaining Zhou (zkn.ms@outlook.com)
• Shreyas Donti (shreyasdonti15@gmail.com)
• Leo Lu (leo.hz.lu@gmail.com)
• Aadesh Madnaik (aadesh.madnaik@gmail.com)
• Rex Remigius Stephen Jothi (rexremigius1@gmail.com)