fmc: streamline the FMC target and run it at 100 MHz on cosmo

.gitignore

@@ -51,4 +51,4 @@ vivado*.str
 **/*.svg.bkp
 
 # cobble local env
-BUILD.vars
+BUILD.vars

cos_soak_100.json

@@ -0,0 +1,53 @@
+{
+  "seed": 478,
+  "steps": [
+    {
+      "divisor": 2,
+      "freq_mhz": 100.0,
+      "ok": true,
+      "udp_timeouts_retried": 9,
+      "phases": {
+        "A_const_reads": {
+          "ok": true,
+          "values": [
+            "0x000001de"
+          ]
+        },
+        "B_scratch": {
+          "ok": true,
+          "patterns": 68,
+          "mismatches": []
+        },
+        "C_stress": {
+          "ok": true,
+          "write_burst": 1024,
+          "read_words": 1024,
+          "mismatches": []
+        },
+        "D_soak": {
+          "ok": true,
+          "ops": 14969449,
+          "mismatches": []
+        },
+        "E_throughput": {
+          "ok": true,
+          "wire_write_MBps": 1.228,
+          "wire_read_MBps": 1.003,
+          "single_read_us": 226.7,
+          "fmc_read_MBps": 11.41,
+          "fmc_read_ns_per_word": 351,
+          "fmc_read_fixed_ns_per_word": 309,
+          "fmc_write_MBps": 50.17,
+          "fmc_write_ns_per_word": 80,
+          "note": "fmc_* numbers are bus line rate (delta-timed block ops); wire_* include per-word network cost and are insensitive to the FMC clock"
+        }
+      },
+      "btr1": "0x0010ffff",
+      "clkdiv_verified": true,
+      "sp_ticks_before": 63172,
+      "sp_ticks_after": 3664254,
+      "sp_rebooted": false
+    }
+  ],
+  "restored_baseline_ok": true
+}

hdl/ip/vhd/fmc_if/docs/fmc_if.adoc

@@ -2,8 +2,8 @@
 :toc: left
 :numbered:
 :icons: font
-:revision: 1.0
-:revdate: 2024-04-24
+:revision: 2.0
+:revdate: 2026-09-10
 
 = FMC Target interface
 
@@ -37,6 +37,83 @@ The AXI state machine monitors the transaction FIFO (and the WDATA FIFO in the
 case of writes) and wiggles the AXI signals appropriately to issue the
 read/write transactions.
 
+=== Pacing contract
+
+The wait line is held asserted by default and released under FSM control.
+After the SP samples the wait line released at rising edge N, its current
+data beat is valid on the bus at edge N+1, and each further release-sampled
+edge advances one beat; on reads the SP samples data on the same edge it
+samples the released wait. This cadence was proven on hardware; the FSM
+and the simulation model both encode it.
+
+All FMC inputs pass through dedicated per-pin capture registers (clocked by
+fmc_capture_clk, a later-phased MMCM sibling on boards that have one), so
+the FSM runs one cycle behind the bus; NWAIT pacing absorbs the delay.
+A 32-bit write occupies 6 fmc_clk cycles of bus time (address capture,
+dispatch, two arm cycles, two back-to-back beats); the AXI write is posted
+and completes behind the SP's back. A 32-bit read stalls through the
+CDC/AXI round trip and then streams its two beats on consecutive cycles.
+The `extra_beat_setup` generic re-inserts a dead cycle between read beats
+for a board that cannot close single-cycle data-out timing (the constraints
+must then carry matching setup-2 multicycle exceptions).
+
+=== Clocking, or how 100 MHz timing closes
+
+The wait line is sampled by the SP on every rising edge, so it is a
+single-cycle output path with no multicycle tricks available: the FPGA has
+period - tsu(NWAIT) - trace, about 6 ns at 10 ns, to get from its clock pin
+through a flop and back out. A plain IBUF+BUFG spends 3.5-6.5 ns of that on
+clock insertion alone, so at 100 MHz the internal FMC clock instead comes
+from an MMCM in phase-alignment mode (BUFG in the feedback path), which
+nulls the insertion delay and leaves the whole budget for the flop and
+output buffer. The output/enable/nwait flops are additionally packed into
+the IOBs, which is why data_out_hiz is one registered bit per pin in OBUFT
+T polarity.
+
+Inputs get the opposite treatment. The SP launches on its falling edge, so
+data arrives late in the cycle and the valid window at the pin is only a
+few nanoseconds wide; worse, STA charges the full corner spread of the MMCM
+clock network against a virtual clock with no pessimism removal. Rather
+than fight that, the MMCM produces a second output for the capture
+registers, phased later than the FSM clock (+135 deg vs +45 deg, see
+fmc_pll_ip.tcl), which parks the sampling instant comfortably inside the
+data-valid window. The capture registers are the only logic on that clock;
+the FSM consumes their outputs on its own clock a cycle later, which is
+where the "FSM runs one cycle behind the bus" above comes from. The +45 deg
+FSM/output phase is itself pinned by output hold on the NWAIT/data pins
+(earlier violates hold), so the two phases are not adjustable
+independently of the analysis in cosmo_timing.xdc.
+
+Both phases are specified in degrees, and the MMCM implements phase as
+VCO-referenced taps, so they are true fractions of the period: at 50 MHz
+the same bitstream places every edge proportionally later and every margin
+only grows. The constraints are therefore written once at 10 ns as a
+strict superset of all slower rates. The MMCM's VCO multiplier (M=12) is
+chosen so the VCO stays in range from a 50 MHz input through a 100 MHz
+one, letting one bitstream lock at any SP CLKDIV setting; a clock-stop
+monitor in the fabric holds the MMCM in reset while the SP's clock is
+absent and relocks it on return. A board without a clock-capable pin for
+the FMC clock (grapefruit) skips the MMCM, ties fmc_capture_clk to
+fmc_clk, and is bounded by the plain-BUFG budget to 66.67 MHz.
+
+=== Robustness
+
+* NWAIT timeout: a transaction that would stall the SP longer than
+  `timeout_cycles` (a wedged AXI responder, full FIFOs that never drain, or
+  a transaction start the FSM never observed -- chip select pinned with no
+  address latch seen, e.g. a miscaptured NADV) is abandoned: reads return
+  0xFFFF poison beats, writes are swallowed, and the sticky `timeout_count`
+  output increments. Data from a timed-out read that later arrives is
+  discarded before any live read is served. Without the missed-start case
+  the SP would hang on its bus with nothing to free it, which from software
+  looks like a silent death followed by a watchdog reboot.
+* Contention tripwire: the tristate control is re-derived from the live NOE/NE
+  pins every cycle it could drive, and `contention_count` increments if the
+  enable is ever up while the SP owns the bus (NADV or NWE low).
+* A mid-transaction chip-select abort cleans up within a few cycles; a write
+  whose transaction was already queued is completed with zero filler so the
+  AXI side never wedges waiting on write data.
+
 == Sim Env
 
 The simulation environment has a model of the STM32's FMC peripheral with some
@@ -50,18 +127,19 @@ testbench to properly interop with our AXI-Lite controller interface.
 
 == Current status
 
-This has been simulated against FMC models for single 32-bit read/writes. No
-bursting is supported currently. Only 32bit access with no consecutive access is
-implemented.
-
+Simulated against the FMC model for single 32-bit read/writes at 50, 66.67,
+and 100 MHz fmc_clk against a 125 MHz AXI clock, including back-to-back
+traffic, FIFO-full backpressure, slow and wedged AXI responders, timeout
+recovery, mid-transaction aborts, reset between transactions, and a
+randomized soak (`buck2 run //hdl/ip/vhd/fmc_if:fmc_tb`). No bursting beyond
+the two beats of a 32-bit access is supported; that is a deliberate scope
+decision. The model implements the WAITCFG=1/DATLAT=0 pacing contract,
+blocking writes via wait_until_idle, the ES0491 dummy read cycles, and
+one-shot abort injection.
 
 == Open Actions
 
-* Support multi-word transactions on FMC interface 
-
-* Support byte-enables FMC - AXI for <32bit access 
-
-* Clean up FMC model, especially around writes and figuring out blocking
+* Support byte-enables FMC - AXI for <32bit access
 
 == Future Investigation/Next steps
 

hdl/ip/vhd/fmc_if/docs/fmc_verification_plan.adoc

@@ -0,0 +1,203 @@
+:showtitle:
+:toc: left
+:numbered:
+:icons: font
+:revdate: 2026-09-10
+
+= FMC speed-up verification plan
+
+Verification for the FMC line-rate work: streaming FSM + input capture stage,
+cosmo at 100 MHz behind the fmc_pll MMCM, grapefruit at 66.67 MHz. Each phase
+gates the next; a failure stops the progression at a known-good configuration.
+Companion pieces: `fmc_if.adoc` (design + pacing contract),
+`tools/fmc_sweep/fmc_sweep.py` (the sweep driver), and the hubris revs
+(demo-server fixes, then the gated CLKDIV flip).
+
+Throughout: FMC_CLK divisor N means FMC_CLK = 200 MHz / N, so divisor 4 =
+50 MHz (shipping), 3 = 66.67 MHz, 2 = 100 MHz.
+
+== Phase 0: desk checks (no hardware)
+
+. Simulation regression: `buck2 run //hdl/ip/vhd/fmc_if:fmc_tb` -- all 19
+  tests, which cover single and back-to-back reads/writes, varied addresses,
+  FIFO backpressure, slow and wedged AXI responders, NWAIT timeout recovery
+  (including a missed transaction start), mid-transaction aborts, reset
+  between transactions, and randomized soaks, exercised at 50/66.67/100 MHz
+  fmc_clk against the 125 MHz AXI clock.
+  The harness fails any test on bus contention (model and DUT driving in the
+  same phase).
+. Timing closure: build both bitstreams and read the post-route summaries --
+  the build does NOT fail on timing, the reports are the authority:
++
+----
+buck2 build //hdl/projects/cosmo_seq:cosmo_seq //hdl/projects/grapefruit:grapefruit
+grep -A6 "Design Timing Summary" buck-out/.../cosmo_seq_route_timing.rpt
+grep -A6 "Design Timing Summary" buck-out/.../grapefruit_route_timing.rpt
+----
++
+Required: WNS and WHS both positive on both boards, zero failing endpoints,
+and the Inter Clock Table must show nonzero endpoint counts for every FMC
+class (fmc_virt_clk -> capture clock, capture -> FSM clock, FSM clock ->
+fmc_virt_clk). A class with zero endpoints means a constraint silently
+stopped matching (renamed cell or clock) and the "pass" is vacuous.
+Margins land near +0.1 setup / +0.02 hold on cosmo at 10 ns and somewhat
+looser on grapefruit at 15 ns, moving a few tens of picoseconds per rebuild;
+cosmo's output hold is the phase-limiting margin and the one to watch.
+. Datasheet re-verification (once, on paper): confirm against DS12110 that
+  the XDC's SP timing numbers (td(CLKL-NExL)=1, td(CLKL-AV)=2.5,
+  td(CLKL-ADV)=3, tsu(NWAIT-CLKH)=3, th(CLKH-NWAIT)=2) match the synchronous
+  multiplexed PSRAM tables at the relevant conditions, and that 100 MHz
+  FMC_CLK is within spec at VOS1. The constraints inherited these values
+  from the 66.67 MHz era and have not been re-read against rev-current
+  silicon docs.
+. hubris demo-server rev: land before any sweep. Runtime timing control
+  does not exist without it -- the task previously lacked the FMC controller
+  register region and faulted on every setter call (reported by hiffy as
+  success), and the setters themselves clobbered sibling BTR1 fields via
+  `btr1.write()`. It also adds the BTR1/BCR1 readback ops the sweep uses to
+  verify each divisor change and the block-transfer ops behind the phase-E
+  line-rate numbers.
+
+== Phase 1: grapefruit at 50 MHz (new RTL, old frequency)
+
+Goal: prove the streamlined FSM + capture stage is a drop-in at the
+shipping frequency, before any clock change. The FPGA is NWAIT-paced, so no
+behavioral difference is expected.
+
+. Build the archive with the new bitstream
+  (`buck2 run //tools/fpga_releaser:cli -- --fpga grapefruit --hubris <path>`
+  or the usual release flow) with hubris still at CLKDIV=3: the
+  demo-server rev included, the CLKDIV flip not included.
+. Flash, confirm hubris boots and the FPGA loads (spartan7-loader token
+  checks pass, grapefruit_seq comes up).
+. Baseline functional pass:
++
+----
+./tools/fmc_sweep/fmc_sweep.py --ip <lladdr> --interface <if> \
+    --archive <archive.zip> --json gf_baseline.json
+----
++
+This runs phases A-E at divisor 4 only: constant-register reads with gaps,
+scratch write/readback (info scratchpad at +0x10; fpga_checksum at +0xC is
+off-limits -- hubris uses it to decide whether to reprogram the FPGA),
+back-to-back write bursts into the scratchpad verified by ordering/last
+value, double-pass read sweeps of the post-code buffer (an external capture
+mem: readable, but AXI writes to it are dropped, so bulk
+write-verification is the simulation suite's job, not the bench's), a
+500-op seeded soak, and a throughput measurement. All must pass.
+. Record the phase-E numbers as the throughput baseline.
+. Leave a longer soak running with `--soak-seconds 3600` (the soak is
+  otherwise op-counted, and 500 UDP round trips finish in seconds).
+  Acceptance: zero mismatches, no UDP timeouts.
+
+== Phase 2: grapefruit sweep to 66.67 MHz
+
+. `--sweep 4,3` first. Phase ordering inside each step is deliberate:
+  constant reads come first because the SP never drives the bus during read
+  data phases, so a timing miss at a new frequency shows up as wrong data,
+  never as a driver fight. Do not reorder.
+. If divisor 3 passes: rerun with `--sweep 4,3,2`. Divisor 2 (100 MHz) is
+  expected to FAIL on grapefruit -- its clock enters on non-clock-capable
+  F17 with ~6 ns of detour insertion delay and no MMCM. The point is to
+  record where the cliff is and confirm the tool's failure handling:
+  it must stop at the first failing phase, restore divisor 4, and re-verify
+  phase A (`restored_baseline_ok: true` in the JSON).
+. Acceptance for 66.67 MHz: full A-E pass plus a >= 1 hour soak
+  (`--soak-seconds 3600`) at divisor 3,
+  and phase-E throughput ~1.33x the baseline batched numbers.
+
+== Phase 3: cosmo at 50 MHz (MMCM bitstream, old frequency)
+
+Goal: prove the MMCM clocking infrastructure -- dual-frequency lock, the
+clock-stop monitor, reset gating -- before touching the frequency.
+
+. Flash the new archive (CLKDIV still 3). First FMC access happening at all
+  proves the MMCM locked at a 50 MHz input (the VCO sits at its 600 MHz
+  minimum here -- this IS the risky corner of the dual-frequency config, so
+  a boot-loop or dead FMC points straight at lock failure; see the rollback
+  note below).
+. Run the full fmc_sweep at divisor 4; record baseline JSON.
+. Regenerate the ILA against the new netlist (the old fmc_ila.tcl net names
+  are stale: the clock is now `clk_fmc_fmc_pll`, the enable regs are
+  `data_out_hiz_int_reg[*]`, and `mark_debug` already tags fmc_state,
+  axi_state, txn, timeouts, contentions). Probe at minimum: fmc_state,
+  NWAIT, CS/ADV/WE/OE IBUFs, DA in/out, timeouts, contentions, and the
+  monitor's mmcm_reset/locked.
+. With the ILA, verify on live traffic:
+  * NWAIT cadence matches the model: one release-sample per beat, two
+    consecutive released samples per 32-bit access, wait re-asserted between
+    transactions.
+  * ES0491 dummy read cycles appear after bursts and provoke no state
+    change.
+  * `timeouts == 0` and `contentions == 0` after every soak. Nonzero
+    contention is a stop-ship: it means the pacing contract disagrees with
+    real silicon somewhere.
+. Robustness loops (the monitor/reset machinery):
+  * SP warm reset under active traffic, >= 50 iterations scripted. Each
+    time: FMC clock stops, monitor must hold the MMCM in reset, relock on
+    clock return, and peek/poke must work immediately after the SP is back.
+    No FPGA reprogram in this loop -- the bitstream rides through.
+  * FPGA reprogram (SP-driven, SP idle on FMC), >= 10 iterations, traffic
+    resumes cleanly each time.
+  * Full power cycle x5.
+  * Acceptance: zero hangs (an SP stuck in an FMC access = the timeout
+    machinery or reset gating failed), counters clean afterward.
+
+== Phase 4: cosmo sweep to 100 MHz
+
+. `--sweep 4,3,2` with the lab-image demo-server. All phases must pass at
+  every step; there is no acceptable-failure divisor on cosmo.
+. One-hour soak (`--soak-seconds 3600`) at divisor 2, counters zero.
+. MMCM phase margin sweep: rebuild bitstream variants with the phases
+  moved +/-18.75 deg = 5 MMCM taps = ~0.52 ns (the phase grid is
+  45/CLKOUT_DIVIDE = 3.75 deg per tap; edit `fmc_pll_ip.tcl`: FSM/output
+  phase 26.25/45/63.75, capture phase 116.25/135/153.75 -- move them
+  together first, then independently if a failure needs isolating). Run
+  the divisor-2 soak on each variant. Acceptance: the shipped phases pass
+  with both neighbors also passing, demonstrating >= ~0.5 ns of real
+  margin beyond STA. If a neighbor fails, the shipped point is marginal:
+  investigate before un-gating, don't ship on one passing point.
+. Re-verify the SP warm-reset loop at divisor 2 (relock now returns to a
+  1200 MHz VCO -- the other edge of the range).
+
+== Phase 5: end-to-end throughput
+
+. fmc_sweep phase E reports two families of numbers. `fmc_*` values come
+  from delta-timed server-side block ops (demo-server network ops 17-19:
+  checksum block reads and a fixed-address fill), where the round trip and
+  per-packet costs cancel and what remains is bus accesses -- these are the
+  line-rate measurements and must scale with the divisor (~1.33x going
+  4 -> 3, ~2x going 4 -> 2). `wire_*` values come from plain batched
+  peek/poke: ~9 network bytes per word swamps the ~0.4 us bus
+  access, so they are expected NOT to move with the divisor -- they are a
+  network/health reference, not a bus measurement. An `fmc_*` number that
+  fails to scale with the divisor means the divisor did not actually
+  change (or the block ops hit an error, which the checksum cross-checks
+  catch).
+. Timed cosmo-hf host-flash operations (the hot path: one status poll + one
+  data read per u32) before/after -- same image content, hubris timestamps
+  or a host-side stopwatch around a full read-back. Expected ~1.6-1.8x at
+  100 MHz; writes limited by flash program time, so use read-back as the
+  metric.
+. The phase-E `fmc_*` table across divisors goes in the PR description.
+
+== Phase 6: un-gate the hubris CLKDIV rev
+
+Per board, only after that board's phases pass: land the hubris rev that
+raises the boot-time divider (cosmo CLKDIV=1, grapefruit CLKDIV=2) in the
+same archive as its validated bitstream -- the archive carries both, so
+image and clock setting move atomically and no mixed-field state exists.
+
+Rollback at any point is the one-line CLKDIV revert: every bitstream in this
+series is NWAIT-paced and runs at 50 MHz unchanged. The single caveat is a
+cosmo MMCM that fails to lock at one input frequency (dual-VCO corner): that
+is a bitstream-level rollback, which is why Phase 3 proves the 50 MHz lock
+corner before anything else depends on it.
+
+== Open observability item
+
+`timeout_count` / `contention_count` are ports (left open at the tops) and
+mark_debug signals. Wiring them into a spare info/debug register would make
+the Phase 3/4 counter checks possible from software instead of the ILA and
+would let fmc_sweep gate every step on them; worth doing before the Phase 4
+soaks.