A Raspberry Pi 4B edge application that uses PIR, light (LDR), and ultrasonic sensors to throttle YOLOv8 inference resolution and frame rate — reducing idle power draw and CPU/SoC thermals, without a GPU and without sacrificing detection quality.
Running a continuous, fixed-resolution computer-vision loop on an ARM SoC like the Raspberry Pi 4B keeps the CPU under sustained load — drawing high power and holding the SoC at an elevated temperature for the entire session — even though most of the time the scene is empty or static. The usual fixes (a heatsink, an AI-accelerator hat, or a weaker model) either add hardware cost or sacrifice accuracy.
SAGE-Vision makes compute proportional to scene demand: cheap sensors gate when and how hard the YOLO model runs, so the node idles near-free when nothing is happening and scales inference resolution to subject distance when it is. Measured against an always-on baseline, this cuts average whole-Pi power by 24 % (4.1 W → 3.1 W) and steady-state SoC temperature by 7 °C (57 °C → 50 °C) with no meaningful loss in detection quality — at zero additional hardware cost beyond sensors already on the bench.
The node runs fully offline on the Pi alone. Sensors wire directly to the 40-pin GPIO header (read by the pigpio background process — there is no microcontroller in the live path); a USB camera supplies frames; a two-thread core (sensor harvester + adaptive vision/FSM) does the work, with optional background threads for telemetry. Inference runs on tflite-runtime with INT8 YOLOv8-nano models.
| Component | Part | Role |
|---|---|---|
| Compute | Raspberry Pi 4B | edge inference node |
| Camera | Raspberry Pi official USB camera | video frames |
| Motion | HC-SR501 PIR | wake / presence signal |
| Light | LM393 comparator module | dark/bright gate for CLAHE |
| Distance | HC-SR04 ultrasonic | subject distance → model selection |
| Power measurement | Inline USB-C power meter (display) | whole-Pi power for the energy figure — read by hand, off the Pi |
| Sensor | Signal | Pi pin | Notes |
|---|---|---|---|
| HC-SR501 PIR | OUT | GPIO 17 (pin 11) | 5V supply; output already 3.3V-safe |
| LM393 light | DO | GPIO 27 (pin 13) | 3.3V supply; HIGH = dark on this module (LM393 polarity varies — set by LDR_DARK_LEVEL); hardware hysteresis via onboard pot |
| HC-SR04 | TRIG | GPIO 23 (pin 16) | direct connection |
| HC-SR04 | ECHO | GPIO 24 (pin 18) | via 1kΩ/2kΩ voltage divider (steps the 5V echo down to 3.3V) |
| Rail | Powers | Pi pins |
|---|---|---|
| 5V | HC-SR501, HC-SR04 | 2, 4 |
| 3.3V | LM393 | 1, 17 |
| GND | all sensor grounds + divider leg | 6, 9, 14, 20, 25, 30, 34, 39 |
Whole-Pi power is measured with a standalone inline USB-C power meter that plugs between the wall charger and the Pi's USB-C power port and shows live volts/amps/watts on its own display. It taps nothing on the GPIO header and runs no code — the watts are read by hand off its display and noted alongside each benchmark run. This replaces the earlier INA219 shunt rig (no soldering, no CC-resistor bring-up, no I²C).
See
docs/HARDWARE_CONNECTIONS.mdfor the meter placement, anddocs/TESTING.mdfor how power is logged during a run.
Each sensor has a distinct role, and the readings are filtered before use:
- Ultrasonic (primary): each reading is spike-rejected (a jump beyond
MAX_PLAUSIBLE_JUMP_CMis dropped unless it persists for several samples) then median-filtered over 5 samples, to absorb multipath bounce off walls/furniture. - PIR: a motion / wake trigger only.
- LDR (LM393): the CLAHE low-light gate (digital dark/bright).
Keep-awake is a logical OR; the expensive state is not. Any one signal — PIR motion, an ultrasonic deviation from the static background (|distance − background| > SONAR_BG_DELTA_CM, ~30 cm), or a YOLO person-detection (the vision vote) — keeps the node awake; absence is declared only when all three have been quiet for PRESENCE_TIMEOUT_S (~12 s). The vision vote is what distinguishes a still occupant (no motion, but detected) from an empty room — the failure mode of motion-only systems.
Why a deviation, not an absolute distance? A rangefinder never sees an empty room — a wall, furniture, or a sensor artifact always returns something, so an absolute
distance < thresholdis permanently true and the node can never declare absence. Sonar presence is therefore background subtraction: a person is a change from the learned empty-scene distance, and any constant reading is absorbed into the background and stops holding the node awake.
But the three signals are not equally trustworthy, so they do not equally unlock the expensive ACTIVE-HI state (tiered presence):
| Signal | Confirms | Unlocks ACTIVE-HI? |
|---|---|---|
| Vision (confident person) | a person | yes — holds HI, and arbitrates the others |
| Proximity (a sonar deviation from background) | a real object (maybe furniture) | only a time-limited HI probe — demoted to LO if vision stays silent |
| PIR (motion) | a heat/motion change (maybe sunlight) | no — wakes + keeps a cheap LO probe only |
This bounds the cost of any single false positive (a spurious PIR from sunlight) to the cheap LO probe, while still giving a genuinely far person the high-resolution look needed to detect them. (Static furniture no longer even reaches this stage — it is absorbed into the sonar background and stops voting presence entirely.) And because the LO probe keeps running inference, a person who later (re-)appears is re-detected by the vision vote and escalated back to HI — the demotion is self-correcting.
Inference is controlled through a 5-state finite state machine.
- SLEEP — no inference; the loop polls at ~0.5 s watching for a wake signal. Exits to STANDBY when the PIR fires or the ultrasonic reading deviates from the learned background by more than
SONAR_WAKE_DELTA_CM(~45 cm — a real change in the scene, not a constant echo), held long enough to pass the debounce gate. - STANDBY — a transitional state entered on waking; no inference. Polls fast (~0.05 s) to clear a brief warm-up (
STANDBY_WARMUP_S, ~200 ms), then always enters ACTIVE-LO (it no longer branches on distance — see ACTIVE-LO for why). - ACTIVE-LO — object present and close (
distance < HILO_CLOSE_CM, ~120 cm). Runs the 320×320 model: a close subject is large in frame, so low resolution suffices and is cheap. The loop is frame-rate-capped (~0.15 s) — a cheap model only saves power if the rate is also capped, otherwise it pegs the CPU just like the always-on baseline. ACTIVE-LO is also the wake-entry state (the node always exits STANDBY into ACTIVE-LO regardless of distance, so the first frames after a wake are captured immediately at low resolution; a far subject is then escalated to ACTIVE-HI by the gate) and the wind-down state (ACTIVE-HI drops here when presence has been quiet for over ~2 s). Exits to ACTIVE-HI if the target moves far, or to SLEEP once presence is absent (> ~12 s). - ACTIVE-HI — object present but far (
distance ≥ HILO_FAR_CM, ~160 cm). Runs the 640×640 model: a distant/small subject needs the higher resolution, but it is expensive (~1 s/frame on the Pi), so the loop is paced slow (~0.40 s). Exits to ACTIVE-LO when the target comes close or presence goes quiet. - WATCHDOG — entered from any state immediately on sensor-health failure: a
pigpiofault, no valid ultrasonic echo for > 2 s, or ≥ 3 consecutive dropped echo readings. Captured frames are CLAHE-preprocessed and inferred with the 640×640 model (worst-case assumption: far and dark). Exits to STANDBY only after the sensors stay healthy for ~1.5 s, so a marginal/flaky sensor cannot flap WATCHDOG ↔ ACTIVE.
Hysteresis note: the close (
HILO_CLOSE_CM, ~120 cm) and far (HILO_FAR_CM, ~160 cm) thresholds differ on purpose — the 120–160 cm gap is a hysteresis band that, together with the transition gate below, stops the model flickering HI↔LO at the boundary.
Wake-latency note: on every wake the node enters ACTIVE-LO first, so the first detection appears as soon as a fast 320 frame completes (~0.7–0.9 s after the wake signal) instead of waiting on a slow 640 frame (~1.5 s). The node prints a
[WAKE] first inference … ms after wake signalline so this latency can be measured directly (test/analyze_log.pyreports it).
State decisions are debounced by a single timed TransitionGate: an edge commits only when its condition has held continuously for hold_s and the current state has been occupied for at least dwell_s. Timing the streak (rather than counting loop ticks) makes the debounce behave identically regardless of how fast the loop runs in each state.
The gate guards every flicker-prone edge:
- HI ↔ LO — condition held
HILO_HOLD_S(~0.75 s) and ≥HILO_DWELL_S(~1.5 s) in-state. - SLEEP wake — hysteretic (wake needs a larger background deviation,
SONAR_WAKE_DELTA_CM~45 cm; keep-awake needs only the smallerSONAR_BG_DELTA_CM~30 cm) and confirmed (WAKE_HOLD_S), so a stray PIR pulse can't wake the node. - WATCHDOG recovery — sticky: leave only after the sonar stays healthy for
WATCHDOG_RECOVER_HOLD_Sand the minimum dwell elapses, so a marginal sensor can't flap the failsafe.
Inference runs on tflite-runtime (the lightweight CPU interpreter — not ultralytics/torch, which are too heavy for the Pi), using full-integer INT8 YOLOv8-nano models. INT8 is chosen for the Pi 4B's ARM Neon SIMD unit, which does INT8 multiply-accumulate faster than FP32 and at lower memory bandwidth.
Because a full-integer INT8 model bakes its input resolution in at export time, the adaptive 320/640 switch is achieved by loading two models (one per resolution) and selecting the matching interpreter per FSM state. The pre/post-processing that ultralytics would do internally is reimplemented by hand in NumPy/OpenCV: letterbox → quantize → invoke → dequantize → decode the YOLOv8 head → NMS → map boxes back to the original frame's pixels. Model export to .tflite is done off-device.
Orthogonal to the 5 states, the captured frame is enhanced when the LDR reports darkness (HIGH output on this module). The BGR frame is converted to YUV and CLAHE is applied to the Y (brightness) channel only, leaving chroma (U, V) untouched so colour is not distorted — then converted back.
tileGridSize = (8, 8)— splits the frame into an 8×8 grid (64 tiles, ~80×60 px each on a 640×480 frame), equalising each tile against its own local histogram (bilinearly interpolated across tiles to avoid blocky seams).clipLimit = 2.0— the contrast cap. The histogram has 256 bins (8-bit Y); OpenCV clips each bin atclipLimit × (tile_pixels / 256)≈ 2× the average bin height (~4800 px/tile ÷ 256 ≈ 19 px → clip at ~38 px), then redistributes the excess. This bounds the slope of the equalisation curve, which stops near-flat dark regions from amplifying sensor noise.2.0is a deliberately mild value.
WATCHDOG forces CLAHE on regardless (worst-case low-light assumption).
To manage the Pi's limited compute and prevent jitter in frame capture and inference, work is split across threads and pinned to specific cores with os.sched_setaffinity:
| Core | Responsibility |
|---|---|
| 0 | OS tasks + (optional) snapshot writing and cloud telemetry upload |
| 1 | Sensor I/O (GPIO harvester) |
| 2 & 3 | Camera capture, CLAHE, FSM-based inference (tflite-runtime), and GUI — run sequentially |
The implementation uses up to 6 threads — 4 run by default, plus 2 that start only with their opt-in flags:
- Main — launches the worker threads and parks until a shutdown event or
Ctrl+C. - GPIO harvester — continuously samples PIR, LM393 (LDR), and HC-SR04 (ultrasonic).
- Vision engine — camera capture, CLAHE, YOLOv8 inference, and GUI.
- pigpio echo callback — runs in pigpio's own real-time thread (the ISR equivalent), timestamping ultrasonic echo edges.
- Cloud uploader (opt-in,
--cloud) — uploads telemetry to ThingSpeak. - Snapshot writer (opt-in,
--snapshots) — writes detection frames to the SD card.
Cross-thread communication:
| Channel | Type | Between |
|---|---|---|
echo_lock |
mutex | pigpio echo callback ⇄ GPIO harvester |
| shared sensor state | mutex | GPIO harvester ⇄ vision engine |
_latest_record_lock |
mutex | vision engine ⇄ cloud uploader |
_snapshot_queue |
bounded queue (drop-oldest) | vision engine ⇄ snapshot writer |
The hot threads (sensors, vision) never block on slow I/O — they hand work to the background threads via a latest-value snapshot (sensor state, telemetry record) or a bounded queue (snapshots). This scales across cores despite Python's GIL because the heavy sections (TFLite invoke(), OpenCV, socket I/O) release the GIL and run as true native parallel work.
Every loop the vision thread assembles one telemetry record (a dict) and fans it out to non-blocking sinks (so a sink can never stall the FSM). The record:
| Field | Description |
|---|---|
state |
FSM state (SLEEP / STANDBY / ACTIVE-LO / ACTIVE-HI / WATCHDOG) |
model_res |
inference resolution / model used (320, 640, or --- when idle) |
latency_ms |
per-frame inference duration |
cpu_pct, cpu_temp_c |
CPU utilisation % and core temperature °C |
distance_cm |
median ultrasonic distance |
detections |
list of (label, confidence%) |
The default sink is the terminal sink, which prints one fixed-format line per loop to the Pi's own console — so the system runs fully offline:
[14:22:07] ACTIVE-LO | model 320 | lat 28.4ms | cpu 47.0% | temp 58.1C | dist 95.3cm | dets: Student/Person(94.2%)
--cloud— a background thread POSTs latency / CPU-temp / distance to a ThingSpeak channel every 20 s (the free-tier rate limit).emit_telemetry()only stashes the latest record, so the network call never runs in the inference loop. The write key is read from a git-ignored.env. Seedocs/SETUP.md, Phase 4.--snapshots— on a person detection, a worker thread writes a JPEG of the frame to./snapshots/(ring-buffered to a file cap, filename carries timestamp/class/conf/state).
Both are opt-in and add Wi-Fi/disk activity, so keep them off during measured benchmark runs (they would bias the meter's whole-Pi reading).
For demos the node opens a local window on the Pi's HDMI monitor (default; pass --headless to disable it for remote deployment). It shows the live feed with blue detection boxes and a HUD header above the unobstructed video:
SAGE-Vision ACTIVE-LO ← state name colour-coded (green active / grey idle / red watchdog)
Model 320 | Objects: 3 | 28 ms | 31 FPS
cpu 47% | 58C | dist 95 cm
During SLEEP/STANDBY the video area shows a "SYSTEM IDLE" placeholder. Keys: q quits cleanly, f toggles fullscreen. The GUI runs inside the vision thread (Cores 2 & 3) and adds negligible cost.
The headline metric — energy draw — is measured with a standalone inline USB-C power meter on the Pi's incoming 5V USB-C feed (see Hardware & Wiring), so it reads the whole-Pi power draw. The meter shows live watts on its own display; the value is read by hand and logged alongside each benchmark run rather than flowing through the telemetry — the node itself does no power sensing. Power instrumentation is only for benchmarking; the node runs identically without the meter.
When a subject appears, the node is in SLEEP and inference only begins after it exits the low-power state — so the scene is missed for the duration of that exit. We define wake-up latency as the interval from the wake signal to the first completed inference (the first available detection):
T_wake = T_sample + T_confirm + T_warmup + T_infer
| Term | Meaning | Constant | Value |
|---|---|---|---|
T_sample |
delay to observe the wake signal (SLEEP poll period; uniform 0–period → mean = period/2) | SLEEP loop ≈ 0.5 s | ~0.25 s (mean) |
T_confirm |
wake debounce hold (signal must persist) | WAKE_HOLD_S |
0.5 s |
T_warmup |
STANDBY warm-up | STANDBY_WARMUP_S |
0.2 s |
T_infer |
first inference time | 320 (LO-first) or 640 (HI-first) | ~0.19 s / ~0.76 s |
The node measures the deterministic part — T_confirm + T_warmup + T_infer — directly: it timestamps the wake signal and prints [WAKE] first inference … ms after wake signal, which test/analyze_log.py aggregates (mean / min / max). The T_sample term (sampling jitter before the signal is observed) adds a further ≤ 0.5 s on top.
Effect of wake-into-low-resolution-first. Entering ACTIVE-LO (320) on every wake instead of ACTIVE-HI (640) changes only T_infer — the other three terms are identical — so the improvement is exactly the difference in first-frame inference time:
ΔT_wake = T_infer(640) − T_infer(320) ≈ 0.76 − 0.19 = 0.57 s
| Wake strategy | T_infer |
T_wake (mean) |
|---|---|---|
| HI-first (640) — previous | ~0.76 s | 0.25 + 0.5 + 0.2 + 0.76 = ~1.71 s |
| LO-first (320) — current | ~0.19 s | 0.25 + 0.5 + 0.2 + 0.19 = ~1.14 s |
→ ~0.57 s faster to first detection (~33 % lower wake latency), with the largest gain for distant subjects, which previously paid the full 640 cost on the very first frame. The inference figures use published Pi-4 YOLOv8n timings (~760 ms at 640, ~190 ms at 320). On the benchmark hardware the measured LO-first wake latency is 1340 ms mean (960–1710 ms), the maximum being a one-time cold-start on the first inference after boot.
- INT8 YOLOv8-nano accuracy. The quantized nano model is weaker on small, distant, or heavily occluded objects than larger detectors. Future work: more efficient models that raise precision without raising power. (The 640 model's earlier INT8 calibration saturation was fixed by re-exporting with COCO128 calibration.)
- Stationary, occluded occupant. Presence can still miss someone who is simultaneously motionless (no PIR), out of the detector's reach (heavy occlusion or darkness), and lingering long enough to be absorbed into the sonar background. A person-specific static sensor such as mmWave radar would close this gap.
- Non-monitorable sensors + ultrasonic noise. PIR and LM393 faults cannot be detected — a dead pin reads as a quiet/lit room, invisible to the WATCHDOG (only the ultrasonic sensor is observable). Ultrasonic noise can also cause occasional spurious wakes, which trims the realised energy saving. Automated sensor-health diagnostics would help.
- Single-point ranging. One ultrasonic distance cannot represent scenes with multiple objects at different depths. Future work: object-level adaptive resolution driven from the camera itself.
SAGE-Vision/
├── rpi_edge/
│ ├── pi_edge_node.py # Main adaptive inference node (reads GPIO sensors)
│ ├── yolo_tflite.py # Lightweight tflite-runtime YOLOv8 detector
│ ├── requirements.txt # Pi Python dependencies
│ ├── yolov8n_320_int8.tflite # INT8 TFLite model — 320×320 (ACTIVE-LO)
│ └── yolov8n_640_int8.tflite # INT8 TFLite model — 640×640 (ACTIVE-HI / WATCHDOG)
├── test/
│ ├── test_baseline_edge.py # Unoptimised control benchmark (terminal + GUI)
│ └── analyze_log.py # Parse a captured telemetry log into per-run metrics
├── docs/
│ ├── SETUP.md # Installation and execution guide
│ ├── TESTING.md # Benchmarking and validation procedure
│ └── HARDWARE_CONNECTIONS.md # Wiring tables for all sensors + power-meter placement
├── .env.example # Template for the ThingSpeak key (copy to .env)
├── .gitignore
├── LICENSE # MIT license (project code)
└── README.md # This file — project overview & architecture
- docs/SETUP.md — installation and execution (HDMI or VNC display), plus the optional
--cloudsetup. - docs/HARDWARE_CONNECTIONS.md — full wiring for every sensor and where the inline USB-C power meter goes.
- docs/TESTING.md — benchmarking the adaptive node against the baseline.
| Decision | Rationale |
|---|---|
| On-Pi terminal telemetry + local GUI, offline by default | No network in the live path, no frame-encoding overhead, no second machine |
| Telemetry record + non-blocking sinks | Terminal always; cloud/snapshot sinks drop in without touching the FSM |
Direct GPIO sensors via pigpio |
Removes the ESP32 and serial link; hardware-timestamped echo edges keep distance accurate |
| Two pinned threads + core affinity | Isolates GPIO sensor I/O jitter from the inference loop's timing |
| Sensor-fused presence (PIR ∨ sonar-deviation ∨ vision vote) | A still person is invisible to PIR alone; fusion prevents false SLEEP. Sonar votes on a deviation from background, not an absolute distance, so a wall/phantom can't pin the node awake |
| Timed TransitionGate on every edge | Debounce behaves identically regardless of per-state loop pace; kills flicker |
| Two fixed-resolution INT8 models | Full-integer export bakes input size; switching models is the adaptive resolution |
| INT8 TFLite over FP32 PyTorch | 2–4× lower inference time on ARM Neon; no GPU required |
| CLAHE on luma over global brightness | Preserves local contrast for detection; global boost washes out fine edges |
This project's source code is released under the MIT License — you're free to use, modify, and distribute it with attribution.
Note on the model weights: the bundled
yolov8n_*_int8.tflitefiles are derived from Ultralytics YOLOv8, which is licensed AGPL-3.0. The MIT license above covers this project's own code; redistributing or deploying the YOLOv8-derived weights may carry AGPL-3.0 obligations. For non-AGPL use, see Ultralytics' commercial licensing.