test(v3): claim-validation for booking invariants and S7-API-1

.github/workflows/ci.yml

@@ -30,10 +30,21 @@ jobs:
       - uses: actions/setup-go@v5
         with:
           go-version: '1.22'
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '20'
       - name: Unit tests
         run: |
           go test ./...
           go test -race ./...
+      - name: Seed inventory
+        env:
+          MYSQL_HOST: 127.0.0.1:3306
+          MYSQL_USER: root
+          MYSQL_PASSWORD: root
+          MYSQL_DATABASE: ticketdb
+          REDIS_HOST: 127.0.0.1:6379
+        run: go run . seed
       - name: Integration tests
         env:
           MYSQL_HOST: 127.0.0.1:3306
@@ -42,3 +53,8 @@ jobs:
           MYSQL_DATABASE: ticketdb
           REDIS_HOST: 127.0.0.1:6379
         run: go test -tags=integration ./integration/...
+      - name: Verification lab UI build
+        run: |
+          cd verification-lab/web
+          npm ci
+          npm run build

Makefile

@@ -1,4 +1,4 @@
-.PHONY: test test-race test-integration compose-config seed
+.PHONY: test test-race test-integration compose-config seed run-api run-worker dev
 
 test:
 	go test ./...
@@ -14,3 +14,13 @@ compose-config:
 
 seed:
 	go run . seed
+
+run-api:
+	go run .
+
+run-worker:
+	go run . worker
+
+# API and persist worker are separate processes; serve alone does not drain the bookings stream.
+dev:
+	trap 'kill 0' INT TERM; go run . worker & go run . & wait

README.md

@@ -10,10 +10,20 @@ When many users hit `/book` at once, classic row-level booking patterns contend
 
 ![Architecture](./architecture.png)
 
+<<<<<<< HEAD
+Flow (current implementation):
+1. Client sends `POST /book` with `X-User-Id`.
+2. Redis Lua allocation is used for atomic allocation + idempotent replay behavior.
+3. Booking events are written to a Redis Stream.
+4. A separate **worker** process (`go run . worker`) consumes the stream and persists sold ticket ownership to MySQL.
+
+**Important:** the API server (`go run .`) does **not** run the persist consumer. A serve-only deployment accepts bookings into Redis but does not write `sold` rows to MySQL until a worker is running.
+=======
 1. Client `POST /book` with `X-User-Id` (optional `X-API-Key` if configured)
 2. Redis Lua: `HGET` prior booking → or `LPOP` + `HSET` + `XADD`
 3. Stream consumer writes sold state to MySQL
 4. Replay path re-enqueues persist if Redis says booked but MySQL lags
+>>>>>>> origin/main
 
 ## Tech stack
 
@@ -26,7 +36,18 @@ When many users hit `/book` at once, classic row-level booking patterns contend
 | Load | k6 (`test.js`), legacy wrk Lua (`test.lua`) |
 | UI | React + Vite dashboards under `frontend/` (Dashboard, Login, Queue Simulation) |
 
+<<<<<<< HEAD
+## API surface
+| Method | Endpoint | Purpose |
+|---|---|---|
+| `POST` | `/book` | Attempt booking for one user (`X-User-Id` required) |
+| `GET` | `/tickets/count` | Remaining tickets in Redis queue |
+| `GET` | `/metrics` | Request/success/failure counters |
+| `GET` | `/healthz` | Liveness probe |
+| `GET` | `/readyz` | Readiness probe (Redis + MySQL; consumer check is worker-only in a split deployment) |
+=======
 ## Key engineering decisions
+>>>>>>> origin/main
 
 - **Redis owns hot allocation**; MySQL owns durable sold state (eventual consistency by design)
 - **Idempotent user booking**: same `user_id` retries return the same `ticket_id` with HTTP 200
@@ -70,6 +91,49 @@ docker compose up
 # API: http://localhost:8080
 ```
 
+<<<<<<< HEAD
+This starts:
+- `mysql` (MySQL 8)
+- `redis` (Redis 7)
+- one-shot `seed` service (`go run . seed`)
+- `server` service (`go run .`) — HTTP API only
+- `worker` service (`go run . worker`) — stream consumer (required for MySQL persistence)
+
+Default API endpoint: `http://localhost:8080`
+
+## Local development
+If you already run MySQL and Redis locally:
+
+```bash
+go run . seed    # run once on empty DB
+go run . worker  # persist consumer (run alongside API)
+go run .         # start API server
+```
+
+Or run API + worker together: `make dev`
+
+Read-only drift check (no Redis/MySQL mutations; exit `0` clean, `1` drift, `2` error):
+
+```bash
+go run . reconcile
+```
+
+## Configuration
+Environment variables used by runtime and compose setup:
+
+| Variable | Purpose |
+|---|---|
+| `HTTP_PORT` | HTTP server port (default `8080`) |
+| `MYSQL_HOST` | MySQL host:port |
+| `MYSQL_USER` | MySQL username |
+| `MYSQL_PASSWORD` | MySQL password |
+| `MYSQL_DATABASE` | MySQL database name |
+| `REDIS_HOST` | Redis host:port |
+| `TICKET_API_KEY` | Optional API key for protected booking access |
+| `TICKET_CORS_ORIGIN` | Allowed CORS origin (default `*`) |
+
+## Testing and verification
+=======
 Local (MySQL + Redis already running):
 
 ```bash
@@ -79,6 +143,7 @@ go run . reconcile   # optional
 ```
 
 ## Testing
+>>>>>>> origin/main
 
 ```bash
 make test

commands.go

@@ -0,0 +1,34 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"os"
+
+	"ticketengine/internal/reconcile"
+)
+
+func runSeedCommand() {
+	log.Println("Running inventory seed (explicit command)...")
+	initMySQL()
+	initRedis()
+	if err := SeedInventory(); err != nil {
+		log.Fatalf("seed failed: %v", err)
+	}
+	log.Println("Seed complete.")
+}
+
+func runReconcileCommand() {
+	initMySQL()
+	initRedis()
+	rep, err := RunReconciliation(context.Background())
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "RECONCILE ERROR: %v\n", err)
+		os.Exit(reconcile.ExitOperational)
+	}
+	fmt.Print(rep.Format())
+	if rep.HasDrift() {
+		os.Exit(reconcile.ExitDrift)
+	}
+}

commands_test.go

@@ -0,0 +1,80 @@
+package main
+
+import (
+	"context"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/alicebob/miniredis/v2"
+	"github.com/go-redis/redis/v8"
+	"gorm.io/driver/sqlite"
+	"gorm.io/gorm"
+
+	"ticketengine/internal/persist"
+)
+
+func setupCommandTestDeps(t *testing.T) {
+	testDB, err := gorm.Open(sqlite.Open("file:cmdtest?mode=memory&cache=shared"), &gorm.Config{})
+	if err != nil {
+		t.Fatalf("sqlite: %v", err)
+	}
+	if err := testDB.AutoMigrate(&Ticket{}); err != nil {
+		t.Fatalf("migrate: %v", err)
+	}
+	db = testDB
+
+	mr, err := miniredis.Run()
+	if err != nil {
+		t.Fatalf("miniredis: %v", err)
+	}
+	t.Cleanup(mr.Close)
+	redisClient = redis.NewClient(&redis.Options{Addr: mr.Addr()})
+}
+
+func TestServeDoesNotStartConsumer(t *testing.T) {
+	setupCommandTestDeps(t)
+
+	var started atomic.Bool
+	old := runPersistConsumer
+	runPersistConsumer = func(ctx context.Context, c *persist.Consumer) {
+		started.Store(true)
+	}
+	t.Cleanup(func() { runPersistConsumer = old })
+
+	bootstrapAPIServer()
+
+	if started.Load() {
+		t.Fatal("serve bootstrap must not start persist consumer")
+	}
+}
+
+func TestWorkerCommandStartsConsumer(t *testing.T) {
+	setupCommandTestDeps(t)
+
+	started := make(chan struct{}, 1)
+	old := runPersistConsumer
+	runPersistConsumer = func(ctx context.Context, c *persist.Consumer) {
+		started <- struct{}{}
+		<-ctx.Done()
+	}
+	t.Cleanup(func() { runPersistConsumer = old })
+
+	ctx, cancel := context.WithCancel(context.Background())
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		startPersistWorkerLoop(ctx)
+	}()
+	t.Cleanup(cancel)
+
+	select {
+	case <-started:
+	case <-time.After(3 * time.Second):
+		t.Fatal("worker did not start persist consumer")
+	}
+	cancel()
+	wg.Wait()
+}

docker-compose.yaml

@@ -84,6 +84,29 @@ services:
         condition: service_healthy
     command: ["sh", "-c", "go mod download && go run ."]
 
+  worker:
+    image: golang:1.22-bookworm
+    restart: unless-stopped
+    working_dir: /app
+    volumes:
+      - .:/app
+      - go_mod_cache:/go/pkg/mod
+    environment:
+      GOPROXY: https://proxy.golang.org,direct
+      MYSQL_HOST: mysql:3306
+      MYSQL_USER: ticket
+      MYSQL_PASSWORD: ticket
+      MYSQL_DATABASE: ticketdb
+      REDIS_HOST: redis:6379
+    depends_on:
+      seed:
+        condition: service_completed_successfully
+      mysql:
+        condition: service_healthy
+      redis:
+        condition: service_healthy
+    command: ["sh", "-c", "go mod download && go run . worker"]
+
 volumes:
   mysql_data:
   redis_data:

docs/V2_PROGRESS.md

@@ -121,15 +121,57 @@ No service uses `build:`; images are pulled only (documented upstream tags).
 
 **Objective:** Redis Stream + consumer group; remove GoChannel for bookings.
 
-**Status:** `not_started`
+### Tasks
+
+| Task | Status |
+|------|--------|
+| Redis Stream publish on allocate | complete |
+| Consumer group + MySQL `sold` update | complete |
+| Worker/API split (V3 phase 1) | complete |
+| PEL reclaim (`XAUTOCLAIM`) + shutdown drain (V3 phase 2) | complete |
+| DLQ after delivery cap (V3 phase 3) | complete |
+
+### Acceptance criteria
+
+- [x] Bookings persist via Redis Stream consumer (not GoChannel).
+- [ ] Crash mid-handler: pending entry reclaimed; ticket becomes `sold` once (`TestPersistRecovery` — requires MySQL+Redis; SKIP in agent VM 2026-10-05).
+- [x] Duplicate delivery is idempotent (same user/ticket `UPDATE` guard).
+- [x] DLQ for poison messages after `MaxDeliveryAttempts` (`TestPersistDLQ`).
+
+### Verification log
+
+| Command | Result | Date |
+|---------|--------|------|
+| `go test -count=1 -race ./internal/persist/` | PASS | 2026-10-05 |
+| `go test -tags=integration -count=1 -race -run TestPersistRecovery ./integration/` | SKIP (no Redis on agent VM); run in CI/Compose | 2026-10-05 |
+| `go test -count=1 -race ./...` | PASS | 2026-10-05 |
+| `go vet ./...` | PASS | 2026-10-05 |
+
+### Status
+
+`in_progress` — streams, reclaim, and recovery test landed; DLQ (V3 phase 3) remains.
 
 ---
 
 ## Phase 4 — Inventory Reconciliation
 
-**Objective:** Rebuild rules; no destructive reset of buyer state.
+**Objective:** Read-only operator reconcile (`go run . reconcile`); no destructive reset of buyer state.
 
-**Status:** `not_started`
+**Status:** `in_progress` (V3 Slice 6 — read-only drift report)
+
+### Reconcile command (Slice 6)
+
+- **Read-only:** no Redis/MySQL/stream/PEL mutations (queue rebuild remains `seed` / `TICKET_ENGINE_REDIS_SYNC` on serve only).
+- **Checks:** Redis `tickets` queue vs MySQL `available` (adjusted for in-flight stream/hash allocations), `hash:user` vs MySQL ownership, duplicate buyers, orphan allocations, stream pending (EXPECTED / STALE / SUSPICIOUS / UNKNOWN), DLQ length.
+- **Exit codes:** `0` clean, `1` drift, `2` operational error (DB/Redis/query failure).
+- **Output:** `RECONCILE RESULT: CLEAN` or `DRIFT DETECTED` with sections.
+
+### Verification log (Slice 6)
+
+| Command | Result | Date |
+|---------|--------|------|
+| `go test -count=1 -run TestReconcile ./internal/reconcile/...` | PASS | 2026-10-05 |
+| `go test -tags=integration -count=1 -run TestReconcileDrift ./integration/...` | SKIP if no Redis/MySQL | 2026-10-05 |
 
 ---
 
@@ -170,3 +212,25 @@ No service uses `build:`; images are pulled only (documented upstream tags).
 **Objective:** End-to-end reviewer sign-off.
 
 **Status:** `not_started`
+
+---
+
+## V3 — Worker/API split (Composer phase 1)
+
+**Status:** `complete` — `go run . worker` is the only persist consumer entrypoint; Compose/Makefile/CI start worker alongside serve.
+
+## V3 — PEL reclaim and shutdown (Composer phase 2)
+
+**Status:** `complete` — periodic `XAUTOCLAIM`, pending `0` replay, graceful drain with pending log on SIGTERM; `TestPersistRecovery` + `internal/persist` reclaim tests.
+
+## V3 — DLQ (Composer phase 3)
+
+**Status:** `complete` — `MaxDeliveryAttempts` then `XADD` `bookings.dlq` + `XACK`; `TestPersistDLQ` + unit DLQ test.
+
+## V3 — Worker scale-out (Composer phase 4)
+
+**Status:** `complete` — `TICKET_PERSIST_CONSUMER_NAME` / `HOSTNAME` consumer names, bounded parallel handlers, worker-sized MySQL pool (`TICKET_ENGINE_ROLE=worker`), `/readyz` fails when stream pending exceeds `MaxReadyPendingCount`; `TestReadyzStreamPending`, `TestTwoWorkersNoDoubleSell`.
+
+## V3 — Redis-only replay (Composer phase 5)
+
+**Status:** `complete` — idempotent `/book` replay uses `hash:user` in `ensurePersisted` (no MySQL read); `TestBookReplaySkipsMySQL`.