Tile-Generator

C++ toolset for generating optimized vector map tiles from OpenStreetMap PBF files, targeting ESP32-based GPS navigators.

Features

• Sparse Index (bitmap + rank): O(1) tile lookup over the bounding box. A 1-bit-per-cell coverage bitmap plus a popcount rank table locates each compact 8-byte entry — empty cells (sea, gaps) resolve with a single small read instead of wasting an index slot.
• High-Performance C++ Engine: OSM PBF parsing and tile generation using GEOS, GDAL, and Libosmium.
• Efficient Binary Format: Packed NPK2 containers with Delta+ZigZag+VarInt coordinate encoding.
• Memory-Mapped Storage: Uses mmap for feature storage, allowing processing of large PBF files with minimal RAM.
• Multi-threaded Processing: Parallel tile generation leveraging all available CPU cores.
• Z-Order Management: 4-pass rendering pipeline supported in binary format.

Generation Process

The generator operates in multiple passes to ensure topological consistency and optimal packaging:

1. Pass 1 (Relations): Scans PBF for administrative boundaries and water multipolygons.
2. Pass 2 (Features): Extracts nodes and ways, applying semantic filtering and layer assignment.
3. Pass 3 (Water): Integrates global water polygons from external Shapefiles (using GDAL/OGR).
4. Pass 4 (Tiles): Parallel clipping, simplification (GEOS), and NPK2 packaging.

Dependencies & Installation

Required Libraries

• Libosmium: PBF parsing (installs Protozero and the zlib/bzip2/Expat headers it builds against).
• GEOS: Geometry operations (clipping, simplification, unions).
• GDAL/OGR: Shapefile support for ocean water polygons.
• zlib, bzip2, Expat: PBF decompression, required by CMake and pulled in by the line below.
• nlohmann/json: features.json parsing (optional: CMake fetches it if the system package is missing).
• CMake 3.14+ and a C++17 compiler: GCC 9+ or Clang.

Installation (Ubuntu/Debian)

sudo apt-get update
sudo apt-get install -y build-essential cmake \
    libosmium2-dev libgeos-dev libgdal-dev nlohmann-json3-dev \
    libbz2-dev zlib1g-dev libexpat1-dev

Compilation

mkdir build && cd build
cmake ..
make -j$(nproc)

Usage

./nav_generator <osm_pbf_file> <output_dir> <features_json> [--zoom min-max] [--water-shp <path>]

--zoom defaults to 6-17. The features_json argument is mandatory: it maps OSM tags to colour, priority and minimum zoom — see docs/features_json_format.md. The repository ships features.json as a ready-to-use example.

Example

./nav_generator andorra-latest.osm.pbf ./NAVMAP features.json --zoom 10-15

Ocean Water Polygons (optional)

Download the pre-computed water polygons shapefile (once, ~540 MB):

wget https://osmdata.openstreetmap.de/download/water-polygons-split-4326.zip
unzip water-polygons-split-4326.zip

Generate with oceans:

./nav_generator input.pbf output features.json --zoom 6-17 \
    --water-shp water-polygons-split-4326/water_polygons.shp

The generator reads the PBF bounding box and only loads water polygons intersecting the extract area. See WATER.md for details.

Visualization & Debugging

The project includes a Python-based simulator to validate maps before flashing them to the ESP32.

Prerequisites

• Python 3.x
• Pygame: sudo apt install python3-pygame

Usage

python3 tile_viewer.py <output_dir> --lat <latitude> --lon <longitude> --config features.json

Map controls

Key / Action	Effect
Arrow keys / Mouse drag	Pan map
[ / ]	Zoom out / in
Scroll wheel	Zoom out / in
B	Toggle background color (white/black)
F	Toggle polygon fill
G	Toggle tile grid
S / L	Toggle stats / legend panels
P	Cycle routing profile (WALK → BIKE → CAR)
Q / Escape	Quit

A* routing overlay

If a ROUTE/ directory with profile subfolders (CAR/, BIKE/, WALK/) exists alongside the nav dir (generated by route_generator), the viewer supports interactive routing. The active profile defaults to WALK and is switchable in-app:

Action	Effect
Left-click on map	Set route origin (green marker)
Right-click on map	Set route destination (red marker) + compute A* route (blue track)
Route button / P	Cycle profile WALK → BIKE → CAR and recompute
R	Clear route and reload tiles

On each right-click (or profile change) the viewer loads every cell in the bounding box covering origin and destination (+1 cell margin) and runs A* over that subgraph, honouring turn restrictions and the turn penalty. The sidebar shows the map info on the left column and the route summary (active profile, origin/destination coordinates, nodes + distance in km, A* time) on the right column.

See docs/tile_viewer.md for the full routing usage and CLI flags (--route-dir, --route-profile).

Route Generator

route_generator builds A* routing graph files from the same OSM PBF. It runs independently from tile generation — use it with vector tiles, PNG tiles, or any map source.

A single run generates all three routing profiles at once:

ROUTE/CAR/ROUTE.bin
ROUTE/BIKE/ROUTE.bin
ROUTE/WALK/ROUTE.bin

Usage

route_generator <input.pbf> <output_dir>

Internally the graph is partitioned into 0.05°×0.05° subcells with an index — readers load only the cells needed for the route on-demand.

Example

# Build tile generator and route generator
cd build && cmake .. && make -j$(nproc)

# Generate all three routing profiles for Andorra
./route_generator andorra-251227.osm.pbf .
# → ./ROUTE/CAR/ROUTE.bin
# → ./ROUTE/BIKE/ROUTE.bin
# → ./ROUTE/WALK/ROUTE.bin

# Copy to SD card alongside tiles
rsync -av NAVMAP/ /media/sdcard/NAVMAP/
rsync -av ROUTE/  /media/sdcard/ROUTE/

For the full binary format and profile speed tables see docs/route_generator.md.

Docker

Prerequisites

Docker Engine and Compose are the only packages the host needs: the compiler, GEOS, GDAL and Libosmium all live in the image, so the apt-get line above only applies when building without Docker.

sudo apt-get update
sudo apt-get install -y docker.io docker-compose   # Ubuntu/Debian: Compose v1, `docker-compose`
sudo usermod -aG docker "$USER"                    # run docker without sudo; log out and back in
docker-compose version                             # sanity check

Ubuntu 24.04 and Debian 12 ship Compose v1 as docker-compose, and the v2 package (docker-compose-v2, or docker-compose-plugin from Docker's own apt repository, https://docs.docker.com/engine/install/) answers to that same name, so the commands in this section use docker-compose throughout; on v2 you can write docker compose instead — same file, same arguments.

The image carries the full toolchain, so the first container run compiles both binaries into build-docker/ when they are missing. It only builds when a binary is absent, so after editing src/ rebuild with docker-compose run --rm generator cmake --build build-docker -j$(nproc). The repository is mounted at its own path: every input and output stays exactly where it is today. Run the commands from the repository root.

docker-compose build
docker-compose run --rm generator nav_generator <input.pbf> <output_dir> <features_json> [--zoom min-max] [--water-shp <path>]
docker-compose run --rm generator route_generator <input.pbf> <output_dir>

Web service

The web service draws the area on a map and runs the whole chain for it — download, merge, clip and the generators — so no osmium command has to be typed. It listens on 127.0.0.1:8080 only.

docker-compose build web          # once, or whenever the Dockerfile changes
docker-compose up -d web          # serves the page on 127.0.0.1:8080
docker-compose logs -f web        # follow the service log
docker-compose restart web        # after editing web/*.py
docker-compose down

A change under web/static/ needs no restart (the browser picks it up on reload). If the daemon answers permission denied while trying to connect to the Docker daemon socket, run newgrp docker once or prefix the commands with sg docker -c "...".

See docs/web_service.md for the workflow on the page, the cache rules, the API and where the figures and the logs end up.

Viewer

The viewer service opens tile_viewer.py on the host display through the mounted X11 socket:

xhost +si:localuser:"$USER"
docker-compose run --rm viewer python3 tile_viewer.py <nav_dir> --lat <lat> --lon <lon> [--config <features_json>]
xhost -si:localuser:"$USER"

Data sources

• Map data: © OpenStreetMap contributors, under the Open Database License.
• Region index and PBF extracts from Geofabrik.
• The map is drawn with Leaflet (BSD-2-Clause), vendored in web/static/vendor/.
• The base map is Natural Earth 1:10m (public domain), vendored as web/static/basemap.geojson; no setup step builds it, see docs/basemap.md to regenerate it.

Internal Format Details

• Tile container: NPK2 (Sparse index: coverage bitmap + rank table + compact 8B entries, O(1) lookup) — docs/bin_tile_format.md
• Tile internal format: NAV1 (Geometry + Text labels), 6-byte tile header
• Color palette: per-pack global RGB565 table; features store a 1-byte color index
• Feature header: 8 fixed bytes + varint coord_count/payload_size
• Coordinates: Web Mercator, 12-bit tile-relative space (0-4096)
• Routing graph: ROUTE.bin (header 32B + index 20B/cell + nodes 8B + edges 12B interleaved per cell; node coords as int16 cell-relative offsets; appended turn-restriction table via_node/in_edge/out_edge; surface bits in edge flags) — docs/route_generator.md

---

Author

Jordi Gauchía (jgauchia@jgauchia.com)