GPS positioning correction for assisted navigation of visually impaired people on university campuses — no extra hardware, just a phone.
Raw GPS is usually accurate to 5-20 meters, which isn't tight enough to safely guide someone who can't see the path in front of them. Dedicated indoor-positioning hardware (BLE beacons, LIDAR) can close that gap, but it's expensive to install and maintain across an entire campus.
4u takes a different approach: an admin walks the campus once, photographing landmark "anchor points" (entrances, intersections, elevators, etc.) with their exact coordinates. From then on, when a user is navigating, their phone's live photo is matched against that anchor-point database using a visual embedding model, and the match is fused with the phone's raw GPS via a Kalman filter to produce a corrected position — typically under 5 meters of error, using hardware everyone already carries.
- Visual GPS correction — matches a live photo against indexed anchor-point photos (EigenPlaces place-recognition embeddings + Qdrant similarity search) and fuses the match with raw GPS via an Unscented Kalman filter that also learns the GPS's own slowly drifting offset.
- Trip planning & voice navigation — say or pick a destination, confirm with Start route, and get an auto-computed route over admin-built walkability connections. Spoken turn-by-turn guidance in steps ("in about 10 steps, turn left"), with direction from the gyroscope and the walking direction rather than the compass alone.
- Admin anchor-point tooling — capture photos + coordinates, review and verify submissions, build the walkability graph between points, and manage universities, buildings and users — from the app, or from the admin web app in a browser (
/admin/on the server). - Session review — browse every user's past navigation sessions grouped by user, re-watch a session's logged trail against its planned route, and read the feedback comments they left.
- Admin "test as user" — run the full trip-planning → navigation → feedback flow on your own admin account, no second test account needed.
- Accessibility — designed for TalkBack, a spoken first-run tutorial, a large Repeat-instruction button, high contrast, adjustable text size, and a fully localized UI (English/Spanish).
- Version support — the newest app release and the one before it both work; older ones are asked to update.
A Python/Dart monorepo: a single Flutter app (serving both a user and an admin role) talks to seven independent FastAPI microservices through one API gateway, backed by Supabase (Postgres + Storage) and Qdrant (vector search).
| Path | What it is |
|---|---|
application/ |
The Flutter mobile app — both roles, one codebase. |
backend-data-collection/ |
Receives raw walk photos + GPS from the app. |
backend-map-management/ |
Admin CRUD: places, buildings, anchor points, connections. |
backend-route-management/ |
Computes and stores navigation routes. |
backend-user-management/ |
Profiles, roles, preferences. |
backend-navigation-management/ |
Navigation sessions, GPS/correction logs, feedback. |
backend-ai-training/ |
Embedding extraction + Qdrant indexing/search. |
backend-positioning/ |
Fuses GPS + a visual match into a corrected position. |
packages/ |
Shared Python library (DB/vector-store clients, auth). |
db_schema/ |
Hand-maintained Postgres table definitions. |
gateway/ |
The nginx reverse proxy every client talks to. |
deployment/phase1-ec2/ |
Scripts for the live EC2 deployment. |
| Layer | Technology |
|---|---|
| Mobile & admin web app | Flutter (Dart) |
| Backend | Python, FastAPI, uv |
| Visual embeddings | PyTorch (EigenPlaces ResNet50; EfficientNet-B0 still selectable) |
| Vector search | Qdrant |
| Position fusion | Unscented Kalman filter (filterpy) |
| Database & storage | Supabase (Postgres + S3-compatible storage) |
| CI/CD | GitHub Actions |
git clone https://github.com/MiguelCock/4u.git
cd 4u
# one-time: create a Supabase project, apply db_schema/*.sql, create two
# Storage buckets, and a Qdrant Cloud (or self-hosted) instance
# fill in each backend service's .env from its .env.example, then:
docker compose up --build
cd application
cp .env.example .env # fill in Supabase + the gateway URL
flutter pub get
flutter runFor the full walkthrough — exact schema order, Storage bucket setup, running every service, and a manual end-to-end test script — see docs/SETUP.md.
- Using the app to get around (for users and the people helping them):
docs/user-guide/(Spanish/English). - Using the admin flow (capturing anchor points, verifying them, building routes, the web app):
docs/admin-guide/(English/Spanish). - Using the app as a developer:
application/README.md. - Tuning the position correction and guidance from test walks:
docs/TUNING.md.
Each component has its own CLAUDE.md with the real architecture detail — how it's built, what it depends on, and its known gaps. Start with the root CLAUDE.md for the full repository map and conventions.
Actively developed. See the repo's Issues for what's in progress and known gaps.
Branch off develop, open a PR against it — CI runs the matching component's tests automatically. main tracks tagged releases only.
Add a line under Unreleased in CHANGELOG.md describing what changed for the people using the app (Added / Changed / Fixed / Removed). When releasing, the release PR renames that heading to the new version and date. The release workflow publishes that section as the GitHub release description, and it refuses to publish a version that has no section.