Skip to content

Preserve OSM one-way direction during generation - #105

Merged
pfeiferj merged 2 commits into
pfeiferj:mainfrom
FrogAi:codex/preserve-one-way-generation
Sep 7, 2026
Merged

pfeiferj merged 2 commits into
pfeiferj:mainfrom
FrogAi:codex/preserve-one-way-generation

Conversation

@FrogAi

@FrogAi FrogAi commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

The upstream generator records only oneway=yes, so legacy 1, reverse -1, and untagged roundabouts/motorways could lose their fixed direction in tiles. This change accepts those static cases, reverses stored nodes for -1, and swaps both directional speed pairs with the reversal. Explicit primary-tag overrides remain effective.

The existing one-way boolean and ordered node list remain the representation. A narrow runtime fix also lets a closed one-way be entered at its repeated endpoint in stored order.

Deploy the runtime fix before publishing regenerated tiles. Existing tiles need regeneration to gain the newly handled tags. This change does not add conditional or dynamic one-way direction support.

Representation, rationale, reproducible checks

Representation and tag behavior

The existing tile format already stores a one-way boolean and an ordered node list. Normalizing fixed reverse ways during generation lets the runtime continue interpreting stored forward order as the legal direction. No schema change is needed.

Primary oneway value Other tags Generated one-way flag Stored node order
yes or 1 Any True Original
-1 Any True Reversed
Absent/empty junction=roundabout or highway=motorway True Original
Absent/empty Ordinary road False Original
no or 0 Including roundabout/motorway False Original
reversible, alternating, or unknown explicit value Any False, retaining existing behavior Original
Absent/empty motorway_link or junction=circular alone False, retaining existing behavior Original

The defaults run only when the primary tag is absent/empty. They do not override an explicit primary value. The fixed-direction interpretation follows the OSM one-way documentation; conditional/dynamic direction support is outside this change.

Reverse-way metadata

Directional speed tags are relative to OSM node order. Reversing only the nodes would attach the original forward speed to the wrong stored direction, so both directional pairs are swapped together.

Field Before normalization Stored for oneway=-1
Nodes A → B → C C → B → A
Forward / backward numeric speed tags 20 / 80 km/h 80 / 20 km/h, converted by the existing parser
Forward / backward conditional tags 25 @ (Mo-Fr) / 70 @ (Mo-Fr) 70 @ (Mo-Fr) / 25 @ (Mo-Fr)
ID, name, reference, lanes, hazard Original metadata Unchanged
Nondirectional speed, advisory speed, conditional speed Original metadata Unchanged

A closed way repeats its first node at the end of the list. The upstream runtime's endpoint check classifies that shared endpoint as the end, rejecting stored-forward entry on a one-way loop. The narrow fix recognizes the shared endpoint of a closed one-way as forward. Open-way endpoints and closed two-way classification retain their existing behavior.

Verification

Checked head bfcfe77, tree aa581653eda08616b2511c7d0e4c2c664323368d, against upstream 7201c6b4b4ec1b0b9ea21daa8c05b80fdd7e01ee. The production diff is confined to generation and closed-endpoint direction.

The complete reproducer below creates all 20 synthetic OSM ways, runs osmium add-locations-to-ways, calls real GenerateOffline, reads the packed tile, and checks flags, every coordinate, both directional speed pairs, nondirectional metadata, bounds, highway class, endpoint classification, and legal/wrong-way bearings for the reverse way. A second test constructs a two-road tile and calls actual NextWay for both a closed one-way and a closed two-way.

flowchart LR
  A[20 tagged OSM XML ways] --> B[osmium adds node locations]
  B --> C[Located PBF]
  C --> D[GenerateOffline]
  D --> E[Packed tile]
  E --> F[Runtime way accessors]
  F --> G[Node order, metadata and GPS direction checks]
Loading
Check Upstream base Current implementation
20-way generation/read assertions Fails the missing fixed-direction and closed-endpoint cases Pass
NextWay into closed one-way No next way, ID 0 Loop ID 2, forward
NextWay into closed two-way Loop ID 2, backward Loop ID 2, backward
Focused race test and package vet Not run for this comparison Pass

Executed on 2026-09-05 with Linux amd64, Go 1.25.1, osmium 1.16.0, and libosmium 2.20.0. The actual run used an already populated module cache, GOPROXY=off, GOSUMDB=off, and a container with networking disabled. Source copies and generated fixtures were isolated from the branch and production data. No production tiles were regenerated or distributed, and no device driving replay was performed. These tests establish the static generation/runtime cases listed above.

Combined validation: exact source tree a4c306906627db3ac7a8ab768651c8628d55465a combines #101 9d61f06a1288ec4ea6f74f7d56a3057444316a13, #103 20e7c25b054b6399360676a7f539a39b4fbf855c, #105 bfcfe77be066634e36054327b20cfa6541063b54, #107 8e5e677d1196838069e9665d4e9d962bcc1e116b, #116 6fd5bbd6cf617c24a7fefd5e302fd36688a1a63b, #136 30e8ce98ea7a4c8401dbb5bfc62120c84fc689e4. The only overlapping file is settings/download.go; the resolution retains #136's selected-row loop and #107's progress publication inside it.

On 2026-09-05, combined Linux amd64 tests (including the scratch regression fixtures), race checks, vet and build passed. Under ARM64 emulation, the existing Makefile build stage (make GO_CAPNP_PATH=/usr/local/go-capnp/std), committed repository tests, vet and both CLI help commands passed with Go 1.25.1; go.mod/go.sum stayed unchanged and the resulting executable is AArch64. The ARM64 run does not include the extra amd64 scratch tests. It used an isolated retained build image, not a new dependency-install/image rebuild or physical device. No production archive payload or live params were accessed. #105 still requires runtime-first rollout before regenerated tiles are distributed.

Reproduce from a fresh clone

Save the two complete blocks below beside each other as reproducer_test.go and reproduce.sh. Requirements are Linux, Bash, Git, tar, Go 1.25.1, and osmium. The runner archives the pinned upstream and current revisions into separate temporary directories, adds only the test file to each copy, runs the assertions, then runs the current focused race check and package vet.

The setup command downloads only repository/module dependencies. The actual fixture requires no network or external OSM data. Populate a module cache for both revisions, then the runner explicitly disables dependency fetching:

git clone https://github.com/FrogAi/mapd.git mapd
export GOTOOLCHAIN=local
export GOMODCACHE="$(go env GOMODCACHE)"
for commit in 7201c6b4b4ec1b0b9ea21daa8c05b80fdd7e01ee bfcfe77be066634e36054327b20cfa6541063b54; do
  seed=$(mktemp -d)
  git -C mapd archive "$commit" | tar -x -C "$seed"
  (cd "$seed" && go mod download)
done
bash reproduce.sh mapd

reproduce.sh:

#!/usr/bin/env bash
set -euo pipefail
repository=$(cd "$1" && pwd)
fixture_dir=$(cd "$(dirname "$0")" && pwd)
output_dir=${PUBLIC_OUTPUT:-$(mktemp -d "${TMPDIR:-/tmp}/mapd-pr105.XXXXXX")}
mkdir -p "$output_dir"
output_dir=$(cd "$output_dir" && pwd)
export GOTOOLCHAIN=local GOPROXY=off GOSUMDB=off
export GOCACHE="$output_dir/build-cache" GOPATH="$output_dir/gopath"
mkdir -p "$GOCACHE" "$GOPATH"
test "$(go env GOVERSION)" = go1.25.1
go version
osmium --version | head -n 2
for entry in upstream:7201c6b4b4ec1b0b9ea21daa8c05b80fdd7e01ee current:bfcfe77be066634e36054327b20cfa6541063b54; do
  name=${entry%%:*}
  commit=${entry#*:}
  source="$output_dir/$name"
  mkdir "$source"
  git -C "$repository" archive "$commit" | tar -x -C "$source"
  cp "$fixture_dir/reproducer_test.go" "$source/maps/public_reproducer_test.go"
  cd "$source"
  set +e
  go test -mod=readonly -buildvcs=false -count=1 -timeout=90s -v -run 'TestPublic' ./maps > "$output_dir/$name.txt" 2>&1
  status=$?
  set -e
  printf '%s %s exit=%s\n' "$name" "$commit" "$status"
  if [ "$name" = upstream ]; then test "$status" -eq 1; else test "$status" -eq 0; fi
done
cd "$output_dir/current"
go test -mod=readonly -buildvcs=false -race -count=1 -timeout=90s -run 'TestPublic' ./maps > "$output_dir/current-race.txt" 2>&1
go vet -mod=readonly ./maps > "$output_dir/current-vet.txt" 2>&1
printf 'current race/vet passed\nresults: %s\n' "$output_dir"

reproducer_test.go:

package maps

import (
	"fmt"
	"math"
	"os"
	"os/exec"
	"path/filepath"
	"strings"
	"testing"

	"capnproto.org/go/capnp/v3"
	"pfeifer.dev/mapd/cereal/log"
	"pfeifer.dev/mapd/cereal/offline"
	m "pfeifer.dev/mapd/math"
)

func TestPublicOneWayGeneration(t *testing.T) {
	cases := []struct {
		name, oneway, highway, junction string
		oneWay, reverse, closed         bool
	}{
		{"forward", "yes", "residential", "", true, false, false},
		{"legacy forward", "1", "residential", "", true, false, false},
		{"reverse", "-1", "residential", "", true, true, false},
		{"explicit no", "no", "residential", "", false, false, false},
		{"legacy no", "0", "residential", "", false, false, false},
		{"ordinary default", "", "residential", "", false, false, false},
		{"roundabout default", "", "residential", "roundabout", true, false, true},
		{"motorway default", "", "motorway", "", true, false, false},
		{"roundabout override", "no", "residential", "roundabout", false, false, true},
		{"motorway override", "no", "motorway", "", false, false, false},
		{"roundabout legacy override", "0", "residential", "roundabout", false, false, true},
		{"motorway legacy override", "0", "motorway", "", false, false, false},
		{"reverse roundabout", "-1", "residential", "roundabout", true, true, true},
		{"motorway link default", "", "motorway_link", "", false, false, false},
		{"circular junction default", "", "residential", "circular", false, false, true},
		{"reversible roundabout", "reversible", "residential", "roundabout", false, false, true},
		{"alternating motorway", "alternating", "motorway", "", false, false, false},
		{"unknown explicit value", "unknown", "motorway", "", false, false, false},
		{"closed forward", "yes", "residential", "", true, false, true},
		{"closed two-way", "no", "residential", "", false, false, true},
	}
	positions := []m.Position{m.NewPosition(0.1, 0.1), m.NewPosition(0.1, 0.11), m.NewPosition(0.11, 0.11)}
	var source strings.Builder
	source.WriteString("<osm version=\"0.6\">\n")
	for i := range cases {
		for j, position := range positions {
			fmt.Fprintf(&source, "<node id=\"%d\" lat=\"%.7f\" lon=\"%.7f\"/>\n", i*3+j+1, position.Lat(), position.Lon())
		}
	}
	for i, test := range cases {
		fmt.Fprintf(&source, "<way id=\"%d\">\n", i+1)
		for j := range positions {
			fmt.Fprintf(&source, "<nd ref=\"%d\"/>\n", i*3+j+1)
		}
		if test.closed {
			fmt.Fprintf(&source, "<nd ref=\"%d\"/>\n", i*3+1)
		}
		for _, tag := range [][2]string{
			{"name", test.name}, {"oneway", test.oneway}, {"highway", test.highway}, {"junction", test.junction},
			{"ref", "R1"}, {"hazard", "curve"}, {"lanes", "2"}, {"maxspeed", "60"}, {"maxspeed:advisory", "40"},
			{"maxspeed:forward", "20"}, {"maxspeed:backward", "80"}, {"maxspeed:conditional", "50 @ (Mo-Fr)"},
			{"maxspeed:forward:conditional", "25 @ (Mo-Fr)"}, {"maxspeed:backward:conditional", "70 @ (Mo-Fr)"},
		} {
			if tag[1] != "" {
				fmt.Fprintf(&source, "<tag k=\"%s\" v=\"%s\"/>\n", tag[0], tag[1])
			}
		}
		source.WriteString("</way>\n")
	}
	source.WriteString("</osm>\n")
	directory := t.TempDir()
	inputXML := filepath.Join(directory, "ways.osm")
	inputPBF := filepath.Join(directory, "ways.osm.pbf")
	if err := os.WriteFile(inputXML, []byte(source.String()), 0o600); err != nil {
		t.Fatal(err)
	}
	if output, err := exec.Command("osmium", "add-locations-to-ways", inputXML, "-o", inputPBF).CombinedOutput(); err != nil {
		t.Fatalf("create located PBF fixture: %v\n%s", err, output)
	}
	box := m.Box{MinPos: m.NewPosition(0, 0), MaxPos: m.NewPosition(0.25, 0.25)}
	settings := OfflineSettings{Box: box, InputFile: inputPBF, OutputDirectory: filepath.Join(directory, "offline")}
	GenerateOffline(settings)
	data, err := os.ReadFile(GenerateBoundsFileName(Area{Box: box}, settings))
	if err != nil {
		t.Fatal(err)
	}
	tile := ReadOffline(data)
	if !tile.Loaded || tile.Ways.Len() != len(cases) {
		t.Fatalf("loaded = %t, ways = %d, want %d", tile.Loaded, tile.Ways.Len(), len(cases))
	}
	for i, test := range cases {
		t.Run(test.name, func(t *testing.T) {
			way := tile.Ways.At(i)
			if way.OneWay() != test.oneWay {
				t.Errorf("oneway = %t, want %t", way.OneWay(), test.oneWay)
			}
			wantNodes := append([]m.Position{}, positions...)
			if test.closed {
				wantNodes = append(wantNodes, positions[0])
			}
			if way.Nodes.Len() != len(wantNodes) {
				t.Fatalf("node count = %d, want %d", way.Nodes.Len(), len(wantNodes))
			}
			for j := range wantNodes {
				index := j
				if test.reverse {
					index = len(wantNodes) - j - 1
				}
				node := way.Nodes.At(j)
				if !node.Equals(wantNodes[index]) {
					t.Errorf("node %d = %v, want %v", j, node, wantNodes[index])
				}
			}
			wantForward, wantBackward := 20*0.277778, 80*0.277778
			wantForwardConditional, wantBackwardConditional := "25 @ (Mo-Fr)", "70 @ (Mo-Fr)"
			if test.reverse {
				wantForward, wantBackward = wantBackward, wantForward
				wantForwardConditional, wantBackwardConditional = wantBackwardConditional, wantForwardConditional
			}
			if math.Abs(way.MaxSpeedForward()-wantForward) > 1e-9 || math.Abs(way.MaxSpeedBackward()-wantBackward) > 1e-9 {
				t.Errorf("directional speeds = %g/%g, want %g/%g", way.MaxSpeedForward(), way.MaxSpeedBackward(), wantForward, wantBackward)
			}
			if way.ConditionalMaxSpeedRaw(true) != wantForwardConditional || way.ConditionalMaxSpeedRaw(false) != wantBackwardConditional {
				t.Errorf("conditional speeds = %q/%q", way.ConditionalMaxSpeedRaw(true), way.ConditionalMaxSpeedRaw(false))
			}
			if way.Id() != int64(i+1) || way.WayName() != test.name || way.WayRef() != "R1" || way.Hazard() != "curve" || way.Lanes() != 2 {
				t.Error("non-directional metadata changed")
			}
			if math.Abs(way.MaxSpeed()-60*0.277778) > 1e-9 || math.Abs(way.AdvisorySpeed()-40*0.277778) > 1e-9 ||
				way.MaxSpeedConditional() != "50 @ (Mo-Fr)" {
				t.Error("non-directional speeds changed")
			}
			wantBox := m.Box{MinPos: positions[0], MaxPos: positions[2]}
			gotBox := way.Box()
			if !gotBox.Equals(wantBox) || way.HighwayClass() != HighwayClassFromTag(test.highway) {
				t.Error("bounds or highway class changed")
			}
			wantForwardAtStart := !test.closed || test.oneWay
			if way.IsForwardFrom(way.Nodes.At(0)) != wantForwardAtStart {
				t.Errorf("forward at first node = %t, want %t", way.IsForwardFrom(way.Nodes.At(0)), wantForwardAtStart)
			}
			if !test.closed && way.IsForwardFrom(way.Nodes.At(way.Nodes.Len()-1)) {
				t.Error("open way endpoint classified as forward")
			}
			if test.name == "reverse" {
				_, segment, err := capnp.NewMessage(capnp.SingleSegment(nil))
				if err != nil {
					t.Fatal(err)
				}
				location, err := log.NewGpsLocationData(segment)
				if err != nil {
					t.Fatal(err)
				}
				location.SetLatitude(0.1)
				location.SetLongitude(0.105)
				for _, bearing := range []float32{90, 270} {
					location.SetBearingDeg(bearing)
					match, err := way.OnWay(location, 1)
					if err != nil || match.OnWay != (bearing == 270) {
						t.Errorf("reverse oneway bearing %g: match = %+v, err = %v", bearing, match, err)
					}
				}
			}
		})
	}
}

func TestPublicClosedEndpoint(t *testing.T) {
	for _, oneWay := range []bool{false, true} {
		message, segment, err := capnp.NewMessage(capnp.SingleSegment(nil))
		if err != nil {
			t.Fatal(err)
		}
		stored, err := offline.NewRootOffline(segment)
		if err != nil {
			t.Fatal(err)
		}
		stored.SetMinLat(0)
		stored.SetMinLon(0)
		stored.SetMaxLat(2)
		stored.SetMaxLon(2)
		ways, err := stored.NewWays(2)
		if err != nil {
			t.Fatal(err)
		}
		coordinates := [][]m.Position{
			{m.NewPosition(1, 0.99), m.NewPosition(1, 1)},
			{m.NewPosition(1, 1), m.NewPosition(1, 1.01), m.NewPosition(1.01, 1.01), m.NewPosition(1, 1)},
		}
		for i, points := range coordinates {
			way := ways.At(i)
			way.SetId(int64(i + 1))
			way.SetName("road")
			way.SetMinLat(1)
			way.SetMinLon(points[0].Lon())
			way.SetMaxLat(1)
			way.SetMaxLon(points[1].Lon())
			if i == 1 {
				way.SetOneWay(oneWay)
				way.SetMaxLat(1.01)
			}
			nodes, err := way.NewNodes(int32(len(points)))
			if err != nil {
				t.Fatal(err)
			}
			for j, point := range points {
				nodes.At(j).SetLatitude(point.Lat())
				nodes.At(j).SetLongitude(point.Lon())
			}
		}
		data, err := message.MarshalPacked()
		if err != nil {
			t.Fatal(err)
		}
		tile := ReadOffline(data)
		incoming := tile.Ways.At(0)
		next, err := incoming.NextWay(&tile, true)
		t.Logf("closed oneWay=%v, next ID=%d, forward=%v, error=%v", oneWay, next.Way.Id(), next.IsForward, err)
		if err != nil || next.Way.Id() != 2 || next.IsForward != oneWay {
			t.Errorf("closed endpoint not traversable in expected direction")
		}
	}
}

@FrogAi
FrogAi force-pushed the codex/preserve-one-way-generation branch from a249fc9 to c8769f6 Compare August 10, 2026 03:03
@FrogAi
FrogAi force-pushed the codex/preserve-one-way-generation branch from c8769f6 to def5848 Compare August 10, 2026 03:21
@FrogAi
FrogAi force-pushed the codex/preserve-one-way-generation branch from def5848 to 07a4cfc Compare September 4, 2026 21:28
FrogAi added a commit to FrogAi/mapd that referenced this pull request Sep 4, 2026
Retain the original PR commits and the tested rewrite. The resulting
file tree is identical to 07a4cfc.
Replace the earlier implementation with the simplified version.
@FrogAi
FrogAi force-pushed the codex/preserve-one-way-generation branch from df7d771 to bfcfe77 Compare September 4, 2026 21:50
@pfeiferj
pfeiferj merged commit 92a22aa into pfeiferj:main Sep 7, 2026
1 check passed
@FrogAi
FrogAi deleted the codex/preserve-one-way-generation branch September 8, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants