From 332eb95dc5a163242052ce02cfef760cbf169175 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 17:01:28 +0000 Subject: [PATCH 1/8] D-PUZZLE-0 step 3: crossword propagation as Cartesian-addressed masking MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Probe: cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs. Once compiled, propagation reads only integer lanes and WordId masks: - WHERE: cell = Morton8x8 code; slot position = FacetTier (slot:offset); a crossing is equality of two cell codes, evaluated once at compile time into one u16 lane cross[slot:offset] = (other slot:offset). - WHAT: WordId = DeepNSM-v2 PaletteVocab::from_frequency_ranked over academic_20k.csv (18,555 ids; vocab.rs's 18,559 counts surfaces before lowercasing). German: DeReKo top 20k from DEREKO_PATH, nothing committed. - letters: no canonical Moore symbol space exists, so a probe-local code read from the vocabulary's alphabet (English a-z plus é). - candidates: P(len, offset, letter) bitsets over WordIds; a placed word ANDs P into each crossing slot (ndarray::simd mask_and_assign), popcount 0 = contradiction, 1 = forced; queue to fixed point. - state: CE64 bits 59..63 only, the shared GIVEN/FORCED/ENTAILED/CANDIDATE. Grid legality (symmetry, run length, connectivity, black ratio) is row-mask arithmetic, checked against a cell-by-cell oracle. Creation (random fill, uniqueness) backtracks over the same masks; solving is the fixed point only, and equals steps 2/2b's cell-scan fixed point on created puzzles. deepnsm-v2 becomes a dev-dependency; guid-v3-tail is already a default contract feature, so no feature set changes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- crates/cognitive-shader-driver/Cargo.toml | 7 + .../crossword_mask_propagation_probe.rs | 1407 +++++++++++++++++ 2 files changed, 1414 insertions(+) create mode 100644 crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs diff --git a/crates/cognitive-shader-driver/Cargo.toml b/crates/cognitive-shader-driver/Cargo.toml index f7c682232..e9bcc55dd 100644 --- a/crates/cognitive-shader-driver/Cargo.toml +++ b/crates/cognitive-shader-driver/Cargo.toml @@ -163,6 +163,10 @@ test = true name = "crossword_real_words_probe" test = true +[[example]] +name = "crossword_mask_propagation_probe" +test = true + [dependencies] lance-graph-contract = { path = "../lance-graph-contract" } # Read-only ontology registry (Phase 7, v4 plan). default-features = false so the @@ -233,3 +237,6 @@ lab = ["serve", "grpc", "with-engine", "with-planner"] # D-CTX-0: the one Fisher-Z calibrator, for the probe only. Production code # borrows the calibrated bytes; it never depends on the calibrator. bgz-tensor = { path = "../bgz-tensor" } +# D-PUZZLE-0 step 3: crossword candidates are DeepNSM-v2 `WordId`s, assigned by +# its own `PaletteVocab::from_frequency_ranked`. Probe-only. +deepnsm-v2 = { path = "../deepnsm-v2" } diff --git a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs new file mode 100644 index 000000000..13f179da3 --- /dev/null +++ b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs @@ -0,0 +1,1407 @@ +//! D-PUZZLE-0, step 3: is crossword propagation the same substrate operation +//! as Sudoku's? +//! +//! The question is not whether a crossword can be solved. It is whether, once a +//! puzzle is compiled, propagation is `mask → intersect → popcount → promote → +//! expose → adjacent masks → repeat`, with nothing crossword-shaped left on the +//! hot path. Steps 2 and 2b (`crossword_population_fold_probe`, +//! `crossword_real_words_probe`) re-derive candidates from a cell array on every +//! step; they stay as the oracle here. +//! +//! # Three coordinate systems, kept apart +//! +//! | what | encoding | where it comes from | +//! |---|---|---| +//! | WHERE a letter sits | cell = `Morton8x8::from_xy(col, row)` (u16) | `lance_graph_contract::morton8x8` | +//! | WHICH slot position | `(slot:offset)` = `FacetTier { hi: slot, lo: offset }.as_u16()` | `facet::FacetTier`, the `(group:member)` reading | +//! | WHAT word | `WordId` (u16) from DeepNSM-v2 `PaletteVocab::from_frequency_ranked` | `deepnsm_v2::vocab` | +//! | what is BELIEVED | CE64 bits 59..63, the shared GIVEN 20 / FORCED 21 / ENTAILED 22 / CANDIDATE 4 | `shared/population_fold.rs` | +//! +//! A crossing is equality of two cell codes. The contract stores no topology +//! (`morton8x8.rs`: "no neighbour list and no stored edge"), so the equality is +//! evaluated once, at compile time: positions are sorted by cell code and every +//! equal pair becomes `cross[slot:offset] = (other slot:other offset)`, one u16 +//! per position. The runtime reads that lane; it never sees the grid. +//! +//! # Letters +//! +//! No canonical Moore symbol space exists to borrow: every Moore in the repo is +//! a direction table, a grid-edge validity byte or a palette-law operand, and +//! none of them may be read as a letter. So a letter is a probe-local code: the +//! sorted alphabet of the admitted words, code `k + 1` for the `k`-th letter, +//! `0` = no letter. The alphabet is read from the vocabulary, never assumed: +//! English gives 27 (a–z and the é of `sauté`, `cliché`), German its own. +//! +//! # Vocabularies (DeepNSM-v2) +//! +//! - **English** (committed): `crates/deepnsm/word_frequency/academic_20k.csv`, +//! column `word`, lowercased, into `PaletteVocab::from_frequency_ranked`: +//! exactly the vocabulary DeepNSM-v2 builds (`examples/genre_shapes.rs`), +//! 18,555 distinct ids (`vocab.rs` quotes 18,559 distinct surfaces; four +//! collapse when lowercased). Words with a hyphen or apostrophe keep their id but +//! are not crossword words. +//! - **German** (runtime only): DeepNSM-v2 builds no German vocabulary and the +//! repository holds none. With `DEREKO_PATH` set, the 20,000 most frequent +//! DeReKo-2014 forms (lowercased, proper nouns and non-alphabetic forms +//! dropped, frequencies summed) go through the same `from_frequency_ranked`. +//! CC BY-NC 3.0; nothing derived is committed. ä ö ü ß stay letters of their +//! own (no `ae`/`ss` folding: the vocabulary has none); text is assumed NFC, +//! and a form with a combining mark is refused because it is not alphabetic. +//! +//! # The candidate population +//! +//! For a language, a word length `L`, an offset `i` and a letter code `x`, +//! `P(L, i, x)` is a bitset over that language's `WordId`s. A slot's +//! candidates start as `P_all(L)` and every exposed crossing letter ANDs one +//! `P` into them (`ndarray::simd::mask_and_assign`); `popcount_batch_u64` +//! decides: 0 = contradiction, 1 = forced, more = still candidates. A forced +//! word exposes its letters to its crossings through the `cross` lane, and the +//! queue runs to a fixed point. Backtracking is used only to CREATE puzzles and +//! to check uniqueness; solving is the fixed point alone. +//! +//! Run: `cargo run --release -p cognitive-shader-driver --example crossword_mask_propagation_probe` +//! German too: `DEREKO_PATH=/path/DeReKo-2014-II-MainArchive-STT.100000.freq cargo run ...` +//! Tests: `cargo test -p cognitive-shader-driver --example crossword_mask_propagation_probe` + +use std::collections::HashMap; +use std::hint::black_box; +use std::time::{Duration, Instant}; + +use deepnsm_v2::vocab::{PaletteVocab, WordId}; +use lance_graph_contract::class_view::ClassId; +use lance_graph_contract::facet::FacetTier; +use lance_graph_contract::morton8x8::Morton8x8; +use ndarray::simd::{mask_and_assign, popcount_batch_u64}; + +#[path = "shared/population_fold.rs"] +mod population_fold; +use population_fold::{ + admit, check_three_ways, declarations, per_group, report, Lane, Rng, CANDIDATE, ENTAILED, + FORCED, GIVEN, +}; + +/// Same class as steps 2 and 2b: the same domain. +const CROSSWORD_CLASS: ClassId = 0x0907; + +/// NYT rules: shortest word; most black squares (a sixth). +const MIN_WORD: usize = 3; +const MAX_BLACK_DIVISOR: usize = 6; +/// Longest slot (NYT 15×15) and the per-slot stride of every position lane. +const MAX_LEN: usize = 15; +const STRIDE: usize = 16; +/// No crossing at this position. +const NONE: u16 = u16::MAX; +/// No word placed in this slot. +const UNSET: WordId = WordId::MAX; +/// German forms kept. +const GERMAN_VOCAB: usize = 20_000; + +const ACADEMIC: &str = include_str!("../../deepnsm/word_frequency/academic_20k.csv"); + +// ─────────────────────────────── vocabulary ─────────────────────────────── + +/// The language a lexicon and a puzzle belong to. A puzzle is only ever +/// solved against its own language's populations. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Lang { + En, + De, +} + +/// Cold side: strings and the letter codebook. Construction and checks only. +struct Cold { + vocab: PaletteVocab, + letters: Vec, +} + +impl Cold { + fn code(&self, c: char) -> Option { + self.letters.binary_search(&c).ok().map(|i| i as u8 + 1) + } + + fn word(&self, w: WordId) -> &str { + self.vocab.word(w).expect("word id in range") + } +} + +/// Hot side: integers and masks only. No `String`, no `char`. +struct Hot { + lang: Lang, + blocks: usize, + /// Alphabet size + 1 (code 0 = no letter). + codes: usize, + /// Per `WordId`: crossword length, 0 = not a crossword word. + len: Vec, + /// `[id * STRIDE + offset]` = letter code. + spell: Vec, + /// By length: every crossword word of that length. + all: Vec>, + /// By length: `[(offset * codes + code) * blocks ..]` = `P(L, offset, code)`. + at: Vec>, +} + +impl Hot { + fn pop(&self, len: usize, offset: usize, code: u8) -> &[u64] { + let i = (offset * self.codes + code as usize) * self.blocks; + &self.at[len][i..i + self.blocks] + } + + fn letter(&self, w: WordId, offset: usize) -> u8 { + self.spell[w as usize * STRIDE + offset] + } +} + +/// A crossword word: `MIN_WORD..=MAX_LEN` letters, every one alphabetic and +/// lowercase. +fn admissible(w: &str) -> bool { + let n = w.chars().count(); + (MIN_WORD..=MAX_LEN).contains(&n) && w.chars().all(|c| c.is_alphabetic() && !c.is_uppercase()) +} + +/// Build both sides from a frequency-ranked word list (most frequent first). +fn build_lexicon(lang: Lang, ranked: &[String]) -> (Hot, Cold) { + let mut vocab = PaletteVocab::new(); + vocab.from_frequency_ranked(ranked.iter().map(String::as_str)); + let n = vocab.len(); + assert!(n < UNSET as usize, "WordId::MAX is the empty marker"); + let mut letters: Vec = (0..n) + .filter_map(|id| vocab.word(id as WordId)) + .filter(|w| admissible(w)) + .flat_map(str::chars) + .collect(); + letters.sort_unstable(); + letters.dedup(); + assert!(letters.len() < 255, "letter codes are one byte"); + let cold = Cold { vocab, letters }; + let codes = cold.letters.len() + 1; + let blocks = n.div_ceil(64); + let mut hot = Hot { + lang, + blocks, + codes, + len: vec![0; n], + spell: vec![0; n * STRIDE], + all: vec![Vec::new(); MAX_LEN + 1], + at: vec![Vec::new(); MAX_LEN + 1], + }; + for l in MIN_WORD..=MAX_LEN { + hot.all[l] = vec![0; blocks]; + hot.at[l] = vec![0; l * codes * blocks]; + } + for id in 0..n { + let w = cold.word(id as WordId); + if !admissible(w) { + continue; + } + let l = w.chars().count(); + hot.len[id] = l as u8; + let bit = 1u64 << (id % 64); + hot.all[l][id / 64] |= bit; + for (i, c) in w.chars().enumerate() { + let code = cold.code(c).expect("letter in codebook"); + hot.spell[id * STRIDE + i] = code; + hot.at[l][(i * codes + code as usize) * blocks + id / 64] |= bit; + } + } + (hot, cold) +} + +/// English: DeepNSM-v2's academic vocabulary, built as `genre_shapes` builds it. +fn english_ranked() -> Vec { + ACADEMIC + .lines() + .skip(1) + .filter_map(|l| l.split(',').nth(3)) + .map(str::to_lowercase) + .filter(|w| !w.is_empty()) + .collect() +} + +/// German: the `GERMAN_VOCAB` most frequent DeReKo forms (`form, lemma, tag, +/// freq`), lowercased, `NE` and non-alphabetic forms dropped, summed. +fn german_ranked(text: &str) -> Vec { + let mut f: HashMap = HashMap::new(); + for line in text.lines() { + let c: Vec<&str> = line.split('\t').collect(); + let [form, _, tag, freq] = c.as_slice() else { + continue; + }; + if *tag == "NE" { + continue; + } + let w = form.to_lowercase(); + if w.is_empty() || !w.chars().all(char::is_alphabetic) { + continue; + } + *f.entry(w).or_insert(0.0) += freq.trim().parse::().unwrap_or(0.0); + } + let mut r: Vec<(String, f64)> = f.into_iter().collect(); + r.sort_by(|a, b| b.1.total_cmp(&a.1).then_with(|| a.0.cmp(&b.0))); + r.truncate(GERMAN_VOCAB); + r.into_iter().map(|(w, _)| w).collect() +} + +// ─────────────────────────────── layout ─────────────────────────────── + +/// A board as row masks: bit `c` of `rows[r]` = cell `(r, c)` is white. +#[derive(Clone, Debug, PartialEq)] +struct Grid { + side: usize, + rows: Vec, +} + +/// Every white bit of a line lies in a run of at least three: OR of the +/// three-wide windows covers the line. +fn line_runs_ok(w: u16) -> bool { + let t = w & (w >> 1) & (w >> 2); + (t | (t << 1) | (t << 2)) == w +} + +impl Grid { + #[cfg(test)] + fn from_rows(rows: &[&str]) -> Self { + let rows: Vec = rows + .iter() + .map(|r| { + r.bytes() + .enumerate() + .fold(0u16, |m, (c, b)| m | (u16::from(b == b'.') << c)) + }) + .collect(); + Self { + side: rows.len(), + rows, + } + } + + fn full_line(&self) -> u16 { + ((1u32 << self.side) - 1) as u16 + } + + fn cols(&self) -> Vec { + (0..self.side) + .map(|c| (0..self.side).fold(0u16, |m, r| m | (((self.rows[r] >> c) & 1) << r))) + .collect() + } + + fn reverse(&self, w: u16) -> u16 { + w.reverse_bits() >> (16 - self.side) + } + + fn blacks(&self) -> usize { + self.side * self.side + - self + .rows + .iter() + .map(|r| r.count_ones() as usize) + .sum::() + } + + /// 180° symmetry: row `r` reversed is row `side - 1 - r`. + fn symmetric(&self) -> bool { + (0..self.side).all(|r| self.reverse(self.rows[r]) == self.rows[self.side - 1 - r]) + } + + /// Every across and down run is at least `MIN_WORD` long. + fn runs_ok(&self) -> bool { + self.rows + .iter() + .chain(self.cols().iter()) + .all(|&w| line_runs_ok(w)) + } + + /// One white region: grow from the first white cell by row-mask dilation + /// until nothing changes. + fn connected(&self) -> bool { + let Some(r0) = self.rows.iter().position(|&w| w != 0) else { + return false; + }; + let full = self.full_line(); + let mut reach = vec![0u16; self.side]; + reach[r0] = self.rows[r0] & self.rows[r0].wrapping_neg(); + loop { + let mut changed = false; + for r in 0..self.side { + let up = if r > 0 { reach[r - 1] } else { 0 }; + let down = if r + 1 < self.side { reach[r + 1] } else { 0 }; + let grown = (reach[r] | (reach[r] << 1) | (reach[r] >> 1) | up | down) + & full + & self.rows[r]; + if grown != reach[r] { + reach[r] = grown; + changed = true; + } + } + if !changed { + return reach == self.rows; + } + } + } + + fn is_nyt_valid(&self) -> bool { + self.symmetric() + && self.runs_ok() + && self.connected() + && self.blacks() * MAX_BLACK_DIVISOR <= self.side * self.side + } + + /// Symmetric black pairs drawn until the grid is NYT-valid with at least + /// one black square. + fn random_nyt(side: usize, rng: &mut Rng) -> Self { + let cells = side * side; + loop { + let mut g = Self { + side, + rows: vec![((1u32 << side) - 1) as u16; side], + }; + for i in 0..cells.div_ceil(2) { + if rng.below(8) == 0 { + for j in [i, cells - 1 - i] { + g.rows[j / side] &= !(1 << (j % side)); + } + } + } + if g.blacks() > 0 && g.is_nyt_valid() { + return g; + } + } + } +} + +/// A compiled puzzle: everything the runtime may read. +#[derive(Clone)] +struct Puzzle { + lang: Lang, + /// Per slot: its length. + len: Vec, + /// `[slot * STRIDE + offset]`: the crossing `(slot:offset)` tile, or `NONE`. + cross: Vec, + /// `[slot * STRIDE + offset]`: the Morton cell code. Construction and the + /// oracle only; propagation never reads it. + cell: Vec, +} + +impl Puzzle { + fn slots(&self) -> usize { + self.len.len() + } +} + +/// Compile a grid: slots from the row and column masks, each position's cell +/// from `Morton8x8::checked_offset`, crossings from equal cell codes. +fn compile(grid: &Grid, lang: Lang) -> Puzzle { + let mut len = Vec::new(); + let mut cell = Vec::new(); + for (across, lines) in [(true, grid.rows.clone()), (false, grid.cols())] { + for (line, &w) in lines.iter().enumerate() { + let mut c = 0; + while c < grid.side { + if w >> c & 1 == 0 { + c += 1; + continue; + } + let start = c; + while c < grid.side && w >> c & 1 == 1 { + c += 1; + } + let l = c - start; + if l < 2 { + continue; + } + let (x, y) = if across { (start, line) } else { (line, start) }; + let (dx, dy) = if across { (1i8, 0i8) } else { (0, 1) }; + let origin = Morton8x8::from_xy(x as u8, y as u8); + let mut codes = [NONE; STRIDE]; + for (off, code) in codes.iter_mut().enumerate().take(l) { + *code = origin + .checked_offset(dx * off as i8, dy * off as i8) + .expect("on the board") + .code(); + } + len.push(l as u8); + cell.extend_from_slice(&codes); + } + } + } + assert!(len.len() < 255, "slot ids are one byte"); + let mut pos: Vec<(u16, u8, u8)> = len + .iter() + .enumerate() + .flat_map(|(s, &l)| (0..l).map(move |o| (s, o))) + .map(|(s, o)| (cell[s * STRIDE + o as usize], s as u8, o)) + .collect(); + pos.sort_unstable(); + let mut cross = vec![NONE; len.len() * STRIDE]; + for w in pos.windows(2) { + if w[0].0 == w[1].0 { + let tile = |(_, s, o): (u16, u8, u8)| FacetTier { hi: s, lo: o }.as_u16(); + cross[w[0].1 as usize * STRIDE + w[0].2 as usize] = tile(w[1]); + cross[w[1].1 as usize * STRIDE + w[1].2 as usize] = tile(w[0]); + } + } + Puzzle { + lang, + len, + cross, + cell, + } +} + +// ─────────────────────────────── runtime ─────────────────────────────── + +/// Why a solve did not start or did not finish. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Stop { + /// The puzzle and the populations are different languages. + Language, + /// A slot was left with no candidate, or two placed words disagree. + Contradiction(u8), +} + +/// Per-slot candidate masks over `WordId` and the placed word, if any. +#[derive(Clone)] +struct State { + blocks: usize, + cand: Vec, + placed: Vec, + events: usize, +} + +impl State { + fn new(hot: &Hot, puz: &Puzzle) -> Result { + if hot.lang != puz.lang { + return Err(Stop::Language); + } + let b = hot.blocks; + let mut cand = vec![0u64; puz.slots() * b]; + for (s, &l) in puz.len.iter().enumerate() { + cand[s * b..(s + 1) * b].copy_from_slice(&hot.all[l as usize]); + } + Ok(Self { + blocks: b, + cand, + placed: vec![UNSET; puz.slots()], + events: 0, + }) + } + + fn mask(&self, s: usize) -> &[u64] { + &self.cand[s * self.blocks..(s + 1) * self.blocks] + } + + fn count(&self, s: usize) -> u64 { + popcount_batch_u64(self.mask(s)) + } + + fn place(&mut self, s: usize, w: WordId) { + let b = self.blocks; + self.cand[s * b..(s + 1) * b].fill(0); + self.cand[s * b + w as usize / 64] = 1 << (w % 64); + self.placed[s] = w; + } + + /// Promote every unplaced slot already down to one candidate; refuse any + /// with none. + fn seed(&mut self, queue: &mut Vec) -> Result<(), Stop> { + for s in 0..self.placed.len() { + if self.placed[s] == UNSET { + match self.count(s) { + 0 => return Err(Stop::Contradiction(s as u8)), + 1 => { + let w = first_bit(self.mask(s)); + self.place(s, w); + queue.push(s as u8); + } + _ => {} + } + } + } + Ok(()) + } +} + +fn first_bit(m: &[u64]) -> WordId { + let (b, w) = m + .iter() + .enumerate() + .find(|(_, &w)| w != 0) + .expect("non-empty mask"); + (b * 64 + w.trailing_zeros() as usize) as WordId +} + +fn bits(m: &[u64]) -> Vec { + let mut out = Vec::new(); + for (b, &word) in m.iter().enumerate() { + let mut w = word; + while w != 0 { + out.push((b * 64 + w.trailing_zeros() as usize) as WordId); + w &= w - 1; + } + } + out +} + +/// The fixed point. Each queued slot exposes its word's letters through the +/// `cross` lane; an unplaced crossing slot ANDs `P(len, offset, letter)` into +/// its mask and is promoted at popcount 1. Reads only `Hot` and the compiled +/// `len` / `cross` lanes. +fn propagate(hot: &Hot, puz: &Puzzle, st: &mut State, queue: &mut Vec) -> Result<(), Stop> { + let b = st.blocks; + while let Some(s) = queue.pop() { + let s = s as usize; + let w = st.placed[s]; + for off in 0..puz.len[s] as usize { + let c = puz.cross[s * STRIDE + off]; + if c == NONE { + continue; + } + let (t, j) = ((c >> 8) as usize, (c & 0xFF) as usize); + let x = hot.letter(w, off); + if st.placed[t] != UNSET { + if hot.letter(st.placed[t], j) != x { + return Err(Stop::Contradiction(t as u8)); + } + continue; + } + let m = &mut st.cand[t * b..(t + 1) * b]; + mask_and_assign(m, hot.pop(puz.len[t] as usize, j, x)); + st.events += 1; + match popcount_batch_u64(m) { + 0 => return Err(Stop::Contradiction(t as u8)), + 1 => { + let w2 = first_bit(m); + st.placed[t] = w2; + queue.push(t as u8); + } + _ => {} + } + } + } + Ok(()) +} + +/// Solve from the givens: place them, seed singles, run to the fixed point. +fn solve(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result { + let mut st = State::new(hot, puz)?; + let mut queue = Vec::new(); + for &(s, w) in givens { + st.place(s as usize, w); + queue.push(s); + } + st.seed(&mut queue)?; + propagate(hot, puz, &mut st, &mut queue)?; + Ok(st) +} + +// ─────────────────────────────── creation ─────────────────────────────── + +/// Depth-first fill over the same masks, most-constrained slot first, random +/// order within it. `budget` counts tried words. Creation only. +fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Option> { + fn go(hot: &Hot, puz: &Puzzle, st: State, rng: &mut Rng, budget: &mut usize) -> Option { + let Some(s) = (0..puz.slots()) + .filter(|&s| st.placed[s] == UNSET) + .min_by_key(|&s| st.count(s)) + else { + return Some(st); + }; + let mut cs = bits(st.mask(s)); + rng.shuffle(&mut cs); + for w in cs { + if *budget == 0 { + return None; + } + *budget -= 1; + let mut next = st.clone(); + next.place(s, w); + let mut q = vec![s as u8]; + if propagate(hot, puz, &mut next, &mut q).is_ok() { + if let Some(done) = go(hot, puz, next, rng, budget) { + return Some(done); + } + } + } + None + } + let mut st = State::new(hot, puz).ok()?; + let mut q = Vec::new(); + st.seed(&mut q).ok()?; + propagate(hot, puz, &mut st, &mut q).ok()?; + go(hot, puz, st, rng, budget).map(|st| st.placed) +} + +/// Complete fills consistent with `givens`, up to `cap`; `None` when the +/// budget runs out. The uniqueness oracle for creation. +fn count_fills( + hot: &Hot, + puz: &Puzzle, + givens: &[(u8, WordId)], + cap: usize, + budget: &mut usize, +) -> Option { + fn go(hot: &Hot, puz: &Puzzle, st: State, cap: usize, budget: &mut usize) -> Option { + let Some(s) = (0..puz.slots()) + .filter(|&s| st.placed[s] == UNSET) + .min_by_key(|&s| st.count(s)) + else { + return Some(1); + }; + let mut n = 0; + for w in bits(st.mask(s)) { + if *budget == 0 { + return None; + } + *budget -= 1; + let mut next = st.clone(); + next.place(s, w); + let mut q = vec![s as u8]; + if propagate(hot, puz, &mut next, &mut q).is_ok() { + n += go(hot, puz, next, cap - n, budget)?; + if n >= cap { + break; + } + } + } + Some(n) + } + match solve(hot, puz, givens) { + Ok(st) => go(hot, puz, st, cap, budget), + Err(_) => Some(0), + } +} + +/// A created puzzle: its compiled layout, its solution and the givens that make +/// the solution unique. +struct Created { + puz: Puzzle, + solution: Vec, + givens: Vec<(u8, WordId)>, +} + +/// Grid, random fill, then givens in random order until unique. `None` when +/// the budget runs out; the caller draws again. +fn create(hot: &Hot, side: usize, rng: &mut Rng, budget: usize) -> Option { + let grid = Grid::random_nyt(side, rng); + let puz = compile(&grid, hot.lang); + let mut b = budget; + let solution = random_fill(hot, &puz, rng, &mut b)?; + let mut order: Vec = (0..puz.slots() as u8).collect(); + rng.shuffle(&mut order); + let mut givens = Vec::new(); + for s in order { + givens.push((s, solution[s as usize])); + let mut b = budget; + if count_fills(hot, &puz, &givens, 2, &mut b)? == 1 { + return Some(Created { + puz, + solution, + givens, + }); + } + } + unreachable!("every slot given is unique") +} + +// ─────────────────────────────── the lane ─────────────────────────────── + +/// Content lanes beside the epistemic edges: the claim "slot holds word". +#[derive(Default)] +struct Claims { + slot: Vec, + word: Vec, +} + +/// One instance's claims at the fixed point: given and forced slots one claim +/// each, an unplaced slot its true word as ENTAILED and every other surviving +/// word as CANDIDATE. +fn emit(c: &Created, st: &State, lane: &mut Lane, claims: &mut Claims) { + for s in 0..c.puz.slots() { + let given = c.givens.iter().any(|g| g.0 as usize == s); + let truth = c.solution[s]; + let mut push = |state, w: WordId, lane: &mut Lane| { + lane.push(state); + claims.slot.push(s as u8); + claims.word.push(w); + }; + if given { + assert_eq!(st.placed[s], truth); + push(GIVEN, truth, lane); + lane.expected[0] += 1; + } else if st.placed[s] != UNSET { + assert_eq!(st.placed[s], truth, "a forced word is the true word"); + push(FORCED, truth, lane); + lane.expected[1] += 1; + } else { + let alive = bits(st.mask(s)); + assert!(alive.contains(&truth), "the law never rules out the truth"); + push(ENTAILED, truth, lane); + lane.expected[2] += 1; + for w in alive.into_iter().filter(|&w| w != truth) { + push(CANDIDATE, w, lane); + lane.expected[3] += 1; + } + } + } + lane.close_group(); +} + +// ─────────────────────────────── oracles ─────────────────────────────── + +/// Every cell shared by two placed words carries the same letter, checked +/// from the Morton cell lanes alone (never from `cross`). +fn cell_consistent(hot: &Hot, puz: &Puzzle, placed: &[WordId]) -> bool { + let mut at: HashMap = HashMap::new(); + for (s, &w) in placed.iter().enumerate() { + if w == UNSET { + continue; + } + for off in 0..puz.len[s] as usize { + let x = hot.letter(w, off); + if *at.entry(puz.cell[s * STRIDE + off]).or_insert(x) != x { + return false; + } + } + } + true +} + +/// Steps 2/2b's method, as the reference: letters on a cell array, every +/// unplaced slot's candidates re-scanned from the vocabulary STRINGS each +/// round, singles placed until nothing changes. Returns the placed word per +/// slot, or `None` on contradiction. (Slot-indexed on purpose: it mirrors the +/// step-2 loop it stands in for.) +#[allow(clippy::needless_range_loop)] +fn cell_scan_solve(cold: &Cold, puz: &Puzzle, givens: &[(u8, WordId)]) -> Option> { + let words: Vec<(WordId, Vec)> = (0..cold.vocab.len() as WordId) + .map(|id| (id, cold.word(id))) + .filter(|(_, w)| admissible(w)) + .map(|(id, w)| (id, w.chars().collect())) + .collect(); + let mut board: HashMap = HashMap::new(); + let mut placed = vec![UNSET; puz.slots()]; + let put = |s: usize, w: &[char], board: &mut HashMap| -> bool { + (0..w.len()).all(|o| *board.entry(puz.cell[s * STRIDE + o]).or_insert(w[o]) == w[o]) + }; + for &(s, w) in givens { + let chars: Vec = cold.word(w).chars().collect(); + if !put(s as usize, &chars, &mut board) { + return None; + } + placed[s as usize] = w; + } + loop { + let mut changed = false; + for s in 0..puz.slots() { + if placed[s] != UNSET { + continue; + } + let l = puz.len[s] as usize; + let fits: Vec<&(WordId, Vec)> = words + .iter() + .filter(|(_, w)| { + w.len() == l + && (0..l).all(|o| { + board + .get(&puz.cell[s * STRIDE + o]) + .is_none_or(|&c| c == w[o]) + }) + }) + .collect(); + match fits.len() { + 0 => return None, + 1 => { + if !put(s, &fits[0].1, &mut board) { + return None; + } + placed[s] = fits[0].0; + changed = true; + } + _ => {} + } + } + if !changed { + return Some(placed); + } + } +} + +// ─────────────────────────────── benchmark ─────────────────────────────── + +fn median(mut v: Vec) -> f64 { + v.sort_by(f64::total_cmp); + v[v.len() / 2] +} + +/// Size sweep: how big can puzzles be created, and how long does solving take. +fn sweep(name: &str, hot: &Hot, cold: &Cold) { + println!("\n{name}: creation and solving by board size (budget 200,000 tried words per attempt, 20 s per size)"); + println!( + " {:>4} {:>7} {:>6} {:>7} {:>9} {:>11} {:>11} {:>9} {:>11}", + "side", + "puzzles", + "slots", + "givens", + "solved", + "create ms", + "solve us", + "events", + "scan us" + ); + let mut rng = Rng(0x5EED); + for side in [5, 7, 9, 11, 13, 15] { + let t0 = Instant::now(); + let (mut made, mut tried) = (Vec::new(), 0usize); + let mut create_ms = Vec::new(); + while t0.elapsed() < Duration::from_secs(20) && made.len() < 40 { + tried += 1; + let t = Instant::now(); + if let Some(c) = create(hot, side, &mut rng, 200_000) { + create_ms.push(t.elapsed().as_secs_f64() * 1e3); + made.push(c); + } + } + if made.is_empty() { + println!( + " {side:>4} {:>7} (none created in {tried} attempts within 20 s)", + 0 + ); + continue; + } + let (mut solve_us, mut scan_us, mut events, mut solved) = (Vec::new(), Vec::new(), 0, 0); + for c in &made { + let t = Instant::now(); + let st = + black_box(solve(hot, &c.puz, &c.givens)).expect("a created puzzle is consistent"); + solve_us.push(t.elapsed().as_secs_f64() * 1e6); + events += st.events; + if st.placed.iter().all(|&w| w != UNSET) { + solved += 1; + } + assert!(cell_consistent(hot, &c.puz, &st.placed)); + if side <= 9 { + let t = Instant::now(); + let scan = cell_scan_solve(cold, &c.puz, &c.givens).expect("consistent"); + scan_us.push(t.elapsed().as_secs_f64() * 1e6); + assert_eq!(scan, st.placed, "mask fixed point == cell-scan fixed point"); + } + } + let n = made.len(); + let slots = made.iter().map(|c| c.puz.slots()).sum::() as f64 / n as f64; + let givens = made.iter().map(|c| c.givens.len()).sum::() as f64 / n as f64; + let scan = if scan_us.is_empty() { + "-".to_string() + } else { + format!("{:.0}", median(scan_us)) + }; + println!( + " {side:>4} {n:>4}/{tried:<3} {slots:>6.1} {givens:>7.1} {:>8.0}% {:>11.1} {:>11.1} {:>9.1} {:>11}", + 100.0 * solved as f64 / n as f64, + median(create_ms), + median(solve_us), + events as f64 / n as f64, + scan + ); + } +} + +/// Filter micro-benchmark: one slot pattern (length + revealed letters) as a +/// mask AND chain vs a direct scan over the vocabulary's spell lane. +fn filter_bench(name: &str, hot: &Hot) { + let mut rng = Rng(0xF17); + let words: Vec = (0..hot.len.len() as WordId) + .filter(|&w| hot.len[w as usize] as usize >= 5) + .collect(); + let patterns: Vec<(usize, Vec<(usize, u8)>)> = (0..2000) + .map(|_| { + let w = words[rng.below(words.len() as u64) as usize]; + let l = hot.len[w as usize] as usize; + let k = 1 + rng.below(3) as usize; + let fixed = (0..k) + .map(|_| { + let o = rng.below(l as u64) as usize; + (o, hot.letter(w, o)) + }) + .collect(); + (l, fixed) + }) + .collect(); + let mut buf = vec![0u64; hot.blocks]; + let mut claims = 0u64; + let t = Instant::now(); + for (l, fixed) in &patterns { + buf.copy_from_slice(&hot.all[*l]); + for &(o, x) in fixed { + mask_and_assign(&mut buf, hot.pop(*l, o, x)); + } + claims += popcount_batch_u64(black_box(&buf)); + } + let mask_ns = t.elapsed().as_nanos() as f64; + let t = Instant::now(); + let mut scanned = 0u64; + for (l, fixed) in &patterns { + scanned += (0..hot.len.len()) + .filter(|&id| { + hot.len[id] as usize == *l + && fixed.iter().all(|&(o, x)| hot.spell[id * STRIDE + o] == x) + }) + .count() as u64; + } + let scan_ns = t.elapsed().as_nanos() as f64; + assert_eq!(black_box(claims), scanned, "mask filter == direct scan"); + let p = patterns.len() as f64; + println!( + " {name}: candidate filter over {} patterns, {:.1} surviving claims each", + patterns.len(), + claims as f64 / p + ); + println!( + " mask AND chain + popcount: {:>9.0} ns/pattern {:>7.2} ns/claim", + mask_ns / p, + mask_ns / claims as f64 + ); + println!( + " direct scan of the spell lane: {:>5.0} ns/pattern {:>7.2} ns/claim", + scan_ns / p, + scan_ns / claims as f64 + ); +} + +/// About a million claims at one size, folded the shared three ways. +fn lane_run(name: &str, hot: &Hot, side: usize) { + let decl = declarations(CROSSWORD_CLASS); + admit(&decl, CROSSWORD_CLASS).expect("the crossword class declares the canonical reading"); + let mut rng = Rng(0x1A4E); + let (mut lane, mut claims, mut slots) = (Lane::default(), Claims::default(), Vec::new()); + let t = Instant::now(); + while lane.edges.len() < 1_000_000 { + let Some(c) = create(hot, side, &mut rng, 200_000) else { + continue; + }; + let st = solve(hot, &c.puz, &c.givens).expect("consistent"); + slots.push(c.puz.slots()); + emit(&c, &st, &mut lane, &mut claims); + } + println!( + "\n {name}: {} claims from {} {side}x{side} puzzles, built in {:.2?}", + lane.edges.len(), + lane.groups, + t.elapsed() + ); + let counts = check_three_ways(&lane, &decl, CROSSWORD_CLASS); + let asserted: Vec = per_group(&lane, GIVEN.bit() | FORCED.bit() | ENTAILED.bit()) + .into_iter() + .map(|n| n as usize) + .collect(); + assert_eq!(asserted, slots); + println!(" every puzzle: one asserted claim per slot (group fold)"); + report(&lane, &decl, CROSSWORD_CLASS, &counts); +} + +fn run_language(name: &str, hot: &Hot, cold: &Cold) { + let words = hot.len.iter().filter(|&&l| l > 0).count(); + println!( + "\n=== {name}: {} WordIds, {} crossword words, {} letters ({})", + cold.vocab.len(), + words, + cold.letters.len(), + cold.letters.iter().collect::() + ); + filter_bench(name, hot); + sweep(name, hot, cold); + lane_run(name, hot, 7); +} + +fn main() { + println!("D-PUZZLE-0 step 3: crossword propagation as Cartesian-addressed population masking"); + let (hot, cold) = build_lexicon(Lang::En, &english_ranked()); + run_language("English, DeepNSM-v2 academic_20k", &hot, &cold); + match std::env::var("DEREKO_PATH") { + Ok(path) => { + let text = std::fs::read_to_string(&path).expect("DEREKO_PATH is readable"); + let (hot, cold) = build_lexicon(Lang::De, &german_ranked(&text)); + run_language("German, DeReKo-2014 top 20k (runtime only)", &hot, &cold); + } + Err(_) => println!("\nGerman skipped: set DEREKO_PATH to the DeReKo-2014 .freq file"), + } +} + +#[cfg(test)] +mod tests { + use super::population_fold::{count_in, queries, raw5, UNKNOWN_CAUSES}; + use super::*; + use causal_edge::layout::EPISTEMIC_MASK; + + fn english() -> (Hot, Cold) { + build_lexicon(Lang::En, &english_ranked()) + } + + fn words(ws: &[&str]) -> Vec { + ws.iter().map(|w| w.to_string()).collect() + } + + /// One across slot (row 0) crossing one down slot (column 1) at its + /// second letter. + fn plus() -> Grid { + Grid::from_rows(&["...", "#.#", "#.#"]) + } + + fn word(cold: &Cold, w: &str) -> WordId { + cold.vocab.id(w).expect("in the fixture") + } + + /// The English vocabulary is DeepNSM-v2's: 18,555 distinct ids in + /// frequency order, and the hyphen/apostrophe rows keep ids but no length. + #[test] + fn english_is_the_deepnsm_v2_academic_vocabulary() { + let (hot, cold) = english(); + assert_eq!(cold.vocab.len(), 18_555); + assert_eq!(cold.word(0), "the"); + let id = word(&cold, "so-called"); + assert_eq!(hot.len[id as usize], 0); + // The letters come from the data: a-z plus the é of `sauté` and + // `cliché` (DeepNSM-v2 lowercases and does not fold accents). + let mut az: Vec = ('a'..='z').collect(); + az.push('é'); + assert_eq!(cold.letters, az); + assert_eq!(hot.len[word(&cold, "cliché") as usize], 6); + } + + /// The letter codes round-trip: every crossword word decodes from its + /// spell lane back to its own string. + #[test] + fn every_word_decodes_from_its_letter_codes() { + let (hot, cold) = english(); + let mut n = 0; + for id in 0..cold.vocab.len() as WordId { + let l = hot.len[id as usize] as usize; + if l == 0 { + continue; + } + let back: String = (0..l) + .map(|o| cold.letters[hot.letter(id, o) as usize - 1]) + .collect(); + assert_eq!(back, cold.word(id)); + n += 1; + } + assert!(n > 10_000); + } + + /// Indexed positional populations equal a direct scan over the vocabulary + /// strings for random patterns. + #[test] + fn positional_populations_equal_a_string_scan() { + let (hot, cold) = english(); + let mut rng = Rng(2); + for _ in 0..500 { + let l = MIN_WORD + rng.below((MAX_LEN - MIN_WORD + 1) as u64) as usize; + let k = rng.below(4) as usize; + let fixed: Vec<(usize, char)> = (0..k) + .map(|_| { + let o = rng.below(l as u64) as usize; + ( + o, + cold.letters[rng.below(cold.letters.len() as u64) as usize], + ) + }) + .collect(); + let mut m = hot.all[l].clone(); + for &(o, c) in &fixed { + mask_and_assign(&mut m, hot.pop(l, o, cold.code(c).unwrap())); + } + let scan: Vec = (0..cold.vocab.len() as WordId) + .filter(|&id| { + let w: Vec = cold.word(id).chars().collect(); + admissible(cold.word(id)) + && w.len() == l + && fixed.iter().all(|&(o, c)| w[o] == c) + }) + .collect(); + assert_eq!(bits(&m), scan, "pattern {l} {fixed:?}"); + } + } + + /// A slot narrowed to one candidate is promoted. + #[test] + fn a_single_candidate_is_forced() { + let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "cow", "dog", "ant"])); + let puz = compile(&plus(), Lang::En); + let (across, down) = (0u8, 1u8); + assert_eq!(puz.len, [3, 3]); + let st = solve(&hot, &puz, &[(across, word(&cold, "cat"))]).unwrap(); + assert_eq!(st.placed[down as usize], word(&cold, "ant")); + } + + /// A forced word constrains EVERY crossing slot, not just the first. + #[test] + fn a_placed_word_masks_every_crossing() { + let (hot, cold) = build_lexicon( + Lang::En, + &words(&["cat", "cow", "cub", "toe", "tea", "dog", "ant"]), + ); + let puz = compile(&Grid::from_rows(&["...", ".#.", ".#."]), Lang::En); + assert_eq!(puz.len, [3, 3, 3]); + let st = solve(&hot, &puz, &[(0, word(&cold, "cat"))]).unwrap(); + let first = |s: usize| -> Vec { + bits(st.mask(s)) + .into_iter() + .map(|w| cold.word(w).chars().next().unwrap()) + .collect() + }; + assert!(first(1).iter().all(|&c| c == 'c') && first(1).len() == 3); + assert!(first(2).iter().all(|&c| c == 't') && first(2).len() == 2); + } + + /// No candidate left is a contradiction, never a silent empty slot. + #[test] + fn an_empty_slot_is_a_contradiction() { + let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "cow", "dog"])); + let puz = compile(&plus(), Lang::En); + assert_eq!( + solve(&hot, &puz, &[(0, word(&cold, "cat"))]).err(), + Some(Stop::Contradiction(1)) + ); + } + + /// The crossing comes from the compiled lane. Re-pointing one crossing + /// lets an incompatible word survive, and the cell oracle sees it; with + /// the lane emptied nothing propagates at all. + #[test] + fn crossings_are_read_from_the_compiled_lane() { + let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "cow", "dog", "ant"])); + let puz = compile(&plus(), Lang::En); + let given = [(0u8, word(&cold, "cat"))]; + let ok = solve(&hot, &puz, &given).unwrap(); + assert!(cell_consistent(&hot, &puz, &ok.placed)); + + // across offset 1 really crosses down offset 0; re-point the pair + // (both directions) at down offset 1. + let mut wrong = puz.clone(); + assert_eq!(wrong.cross[1], FacetTier { hi: 1, lo: 0 }.as_u16()); + wrong.cross[1] = FacetTier { hi: 1, lo: 1 }.as_u16(); + wrong.cross[STRIDE] = NONE; + wrong.cross[STRIDE + 1] = FacetTier { hi: 0, lo: 1 }.as_u16(); + let bad = solve(&hot, &wrong, &given).unwrap(); + assert_eq!(bad.placed[1], word(&cold, "cat")); + assert!(!cell_consistent(&hot, &wrong, &bad.placed)); + + let mut cut = puz.clone(); + cut.cross.fill(NONE); + let none = solve(&hot, &cut, &given).unwrap(); + assert_eq!(none.placed[1], UNSET); + assert_eq!(none.events, 0); + } + + /// Crossing tiles are symmetric and land on the same Morton cell. + #[test] + fn every_crossing_names_the_same_cell_both_ways() { + let mut rng = Rng(4); + for side in [5, 7, 9, 15] { + let puz = compile(&Grid::random_nyt(side, &mut rng), Lang::En); + for s in 0..puz.slots() { + for o in 0..puz.len[s] as usize { + let c = puz.cross[s * STRIDE + o]; + assert_ne!(c, NONE, "an NYT grid checks every square"); + let (t, j) = ((c >> 8) as usize, (c & 0xFF) as usize); + assert_eq!(puz.cell[t * STRIDE + j], puz.cell[s * STRIDE + o]); + assert_eq!( + puz.cross[t * STRIDE + j], + FacetTier { + hi: s as u8, + lo: o as u8 + } + .as_u16() + ); + } + } + } + } + + /// Languages are separate populations: German keeps ä ö ü ß as letters, + /// the same WordId names different words, and a puzzle is refused against + /// the other language's populations. + #[test] + fn languages_do_not_share_ordinals_or_populations() { + let (en, en_cold) = build_lexicon(Lang::En, &words(&["the", "and", "house", "street"])); + let (de, de_cold) = build_lexicon( + Lang::De, + &words(&["der", "und", "haus", "straße", "über", "größe", "mädchen"]), + ); + assert_ne!(en_cold.word(0), de_cold.word(0)); + for c in ['ä', 'ö', 'ü', 'ß'] { + assert!(de_cold.code(c).is_some() && en_cold.code(c).is_none()); + } + let de_puz = compile(&plus(), Lang::De); + assert_eq!(State::new(&en, &de_puz).err(), Some(Stop::Language)); + assert!(State::new(&de, &de_puz).is_ok()); + } + + /// Bits 59..63 carry state only: every claim edge is zero outside them, + /// its code is one of the four shared states, and two different words in + /// the same state have the identical edge. + #[test] + fn ce64_carries_state_never_content() { + let (hot, _) = english(); + let mut rng = Rng(7); + let (mut lane, mut claims) = (Lane::default(), Claims::default()); + while lane.groups < 5 { + if let Some(c) = create(&hot, 5, &mut rng, 200_000) { + let st = solve(&hot, &c.puz, &c.givens).unwrap(); + emit(&c, &st, &mut lane, &mut claims); + } + } + let codes: Vec = population_fold::STATES + .iter() + .map(|s| u32::from(s.raw())) + .collect(); + for e in &lane.edges { + assert_eq!(e.0 & !EPISTEMIC_MASK, 0); + assert!(codes.contains(&raw5(*e))); + } + let cands: Vec = (0..lane.edges.len()) + .filter(|&i| raw5(lane.edges[i]) == u32::from(CANDIDATE.raw())) + .collect(); + let (a, b) = (cands[0], cands[1]); + assert_ne!(claims.word[a], claims.word[b]); + assert_eq!(lane.edges[a], lane.edges[b]); + // reserved codes 24..31 never appear + assert_eq!(count_in(&lane.edges, !0u32 << 24), 0); + assert_eq!(count_in(&lane.edges, UNKNOWN_CAUSES.bit()), 0); + } + + /// The mask fixed point equals steps 2/2b's cell-scan fixed point on + /// created puzzles, and every created puzzle is unique and consistent. + #[test] + fn mask_propagation_equals_the_cell_scan_oracle() { + let (hot, cold) = english(); + let mut rng = Rng(9); + let mut made = 0; + while made < 8 { + let side = if made % 2 == 0 { 5 } else { 7 }; + let Some(c) = create(&hot, side, &mut rng, 200_000) else { + continue; + }; + made += 1; + let st = solve(&hot, &c.puz, &c.givens).unwrap(); + assert_eq!( + cell_scan_solve(&cold, &c.puz, &c.givens).unwrap(), + st.placed + ); + assert!(cell_consistent(&hot, &c.puz, &c.solution)); + let mut b = 1_000_000; + assert_eq!(count_fills(&hot, &c.puz, &c.givens, 3, &mut b), Some(1)); + } + } + + /// The mask grid rules agree with a cell-by-cell check, and refuse. + #[test] + fn mask_grid_rules_equal_a_cell_check() { + fn cell_valid(g: &Grid) -> bool { + let n = g.side; + let white = |r: usize, c: usize| g.rows[r] >> c & 1 == 1; + let sym = (0..n).all(|r| (0..n).all(|c| white(r, c) == white(n - 1 - r, n - 1 - c))); + let mut runs_ok = true; + for across in [true, false] { + for a in 0..n { + let mut run = 0; + for b in 0..=n { + let w = b < n && if across { white(a, b) } else { white(b, a) }; + if w { + run += 1; + } else { + if run > 0 && run < MIN_WORD { + runs_ok = false; + } + run = 0; + } + } + } + } + let cells: Vec<(usize, usize)> = (0..n) + .flat_map(|r| (0..n).map(move |c| (r, c))) + .filter(|&(r, c)| white(r, c)) + .collect(); + let mut seen = vec![cells[0]]; + let mut i = 0; + while i < seen.len() { + let (r, c) = seen[i]; + for (dr, dc) in [(0i32, 1i32), (0, -1), (1, 0), (-1, 0)] { + let (r2, c2) = (r as i32 + dr, c as i32 + dc); + if r2 >= 0 && c2 >= 0 && (r2 as usize) < n && (c2 as usize) < n { + let p = (r2 as usize, c2 as usize); + if white(p.0, p.1) && !seen.contains(&p) { + seen.push(p); + } + } + } + i += 1; + } + let blacks = n * n - cells.len(); + sym && runs_ok && seen.len() == cells.len() && blacks * MAX_BLACK_DIVISOR <= n * n + } + let mut rng = Rng(11); + let (mut valid, mut invalid) = (0, 0); + for _ in 0..3000 { + let side = [5, 7, 9][rng.below(3) as usize]; + let mut g = Grid { + side, + rows: vec![((1u32 << side) - 1) as u16; side], + }; + for i in 0..side * side { + if rng.below(9) == 0 { + g.rows[i / side] &= !(1 << (i % side)); + } + } + if g.rows.iter().all(|&r| r == 0) { + continue; + } + let ok = g.is_nyt_valid(); + assert_eq!(ok, cell_valid(&g), "{:?}", g.rows); + if ok { + valid += 1; + } else { + invalid += 1; + } + } + assert!( + valid > 10 && invalid > 10, + "{valid} valid, {invalid} invalid" + ); + } + + /// Solving runs without the strings: `Cold` is dropped before the solve, + /// and the result is the same. + #[test] + fn the_hot_path_needs_no_strings() { + let (hot, cold) = english(); + let mut rng = Rng(13); + let c = loop { + if let Some(c) = create(&hot, 7, &mut rng, 200_000) { + break c; + } + }; + let expect = cell_scan_solve(&cold, &c.puz, &c.givens).unwrap(); + drop(cold); + assert_eq!(solve(&hot, &c.puz, &c.givens).unwrap().placed, expect); + } + + /// Three ways, partition, declaration gate: the shared fold, unchanged. + #[test] + fn the_shared_fold_counts_the_crossword_lane_three_ways() { + let (hot, _) = english(); + let mut rng = Rng(17); + let (mut lane, mut claims) = (Lane::default(), Claims::default()); + while lane.edges.len() < 20_000 { + if let Some(c) = create(&hot, 5, &mut rng, 200_000) { + let st = solve(&hot, &c.puz, &c.givens).unwrap(); + emit(&c, &st, &mut lane, &mut claims); + } + } + let decl = declarations(CROSSWORD_CLASS); + let counts = check_three_ways(&lane, &decl, CROSSWORD_CLASS); + assert!(counts.iter().all(|&n| n > 0), "a question went unexercised"); + let union = queries()[1..].iter().fold(0, |u, q| u | q.population); + assert_eq!(count_in(&lane.edges, union), lane.edges.len()); + assert!(admit(&decl, CROSSWORD_CLASS).is_some()); + assert!(admit(&decl, 0x0906).is_none()); + } +} From 1e2a104d15255ff1177c5a3d63f9c4c5d8bd58c5 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 17:18:02 +0000 Subject: [PATCH 2/8] D-PUZZLE-0 step 3: MooreSymbol8 codebook and four arms on one fixed point - Letters are a declared MooreSymbol8 reading of one byte: 0 unknown, 1..=26 a-z, 27 ae, 28 oe, 29 ue, 30 sz, 31..=255 reserved. Accented Latin letters fold to their base (cliche); umlauts and sz stay letters. - Four arms on identical puzzles and givens: A token (WordId masks over the cross lane), B Cartesian (MooreSymbol8 board, masks recomputed from cell symbols), C literal (steps 2/2b string re-scan), D hybrid (token masks, letters routed through the board and the occupant lane). The suite and the benchmark assert equal placed words in all four and equal masks in A/B/D. - Boards up to 21x21 (MAX_LEN 21, u32 row masks); creation reports whether a miss was the fill or the uniqueness check. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .../crossword_mask_propagation_probe.rs | 1242 +++++++++++------ 1 file changed, 790 insertions(+), 452 deletions(-) diff --git a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs index 13f179da3..dd9cd29f5 100644 --- a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs +++ b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs @@ -1,63 +1,66 @@ //! D-PUZZLE-0, step 3: is crossword propagation the same substrate operation -//! as Sudoku's? +//! as Sudoku's, and which physical representation of it is best? //! //! The question is not whether a crossword can be solved. It is whether, once a //! puzzle is compiled, propagation is `mask → intersect → popcount → promote → -//! expose → adjacent masks → repeat`, with nothing crossword-shaped left on the -//! hot path. Steps 2 and 2b (`crossword_population_fold_probe`, -//! `crossword_real_words_probe`) re-derive candidates from a cell array on every -//! step; they stay as the oracle here. +//! expose → adjacent masks → repeat`, with nothing crossword-shaped on the hot +//! path, and how four representations of that one logic compare on the SAME +//! puzzles, givens and fixed points. //! -//! # Three coordinate systems, kept apart +//! # Coordinates, kept apart //! -//! | what | encoding | where it comes from | +//! | what | encoding | source | //! |---|---|---| //! | WHERE a letter sits | cell = `Morton8x8::from_xy(col, row)` (u16) | `lance_graph_contract::morton8x8` | -//! | WHICH slot position | `(slot:offset)` = `FacetTier { hi: slot, lo: offset }.as_u16()` | `facet::FacetTier`, the `(group:member)` reading | -//! | WHAT word | `WordId` (u16) from DeepNSM-v2 `PaletteVocab::from_frequency_ranked` | `deepnsm_v2::vocab` | -//! | what is BELIEVED | CE64 bits 59..63, the shared GIVEN 20 / FORCED 21 / ENTAILED 22 / CANDIDATE 4 | `shared/population_fold.rs` | +//! | WHICH slot position | `(slot:offset)` = `FacetTier { hi: slot, lo: offset }.as_u16()` | `facet::FacetTier` | +//! | WHAT letter is in a cell | [`MooreSymbol8`] (u8) | declared here (see below) | +//! | WHAT word | `WordId` (u16), DeepNSM-v2 `PaletteVocab::from_frequency_ranked` | `deepnsm_v2::vocab` | +//! | what is BELIEVED | CE64 bits 59..63: GIVEN 20 / FORCED 21 / ENTAILED 22 / CANDIDATE 4 | `shared/population_fold.rs` | //! //! A crossing is equality of two cell codes. The contract stores no topology //! (`morton8x8.rs`: "no neighbour list and no stored edge"), so the equality is -//! evaluated once, at compile time: positions are sorted by cell code and every -//! equal pair becomes `cross[slot:offset] = (other slot:other offset)`, one u16 -//! per position. The runtime reads that lane; it never sees the grid. +//! evaluated once, at compile time, into two lanes: `cross[slot:offset]` = the +//! other `(slot:offset)` on the same cell, and `occupant[cell]` = the (at most +//! two) positions on a cell. The runtime reads lanes; it never sees the grid. //! -//! # Letters +//! # `MooreSymbol8`: the letter reading of one byte //! -//! No canonical Moore symbol space exists to borrow: every Moore in the repo is -//! a direction table, a grid-edge validity byte or a palette-law operand, and -//! none of them may be read as a letter. So a letter is a probe-local code: the -//! sorted alphabet of the admitted words, code `k + 1` for the `k`-th letter, -//! `0` = no letter. The alphabet is read from the vocabulary, never assumed: -//! English gives 27 (a–z and the é of `sauté`, `cliché`), German its own. +//! No existing Moore byte may be read as a letter: every Moore in the repo is a +//! direction table, a grid-edge validity byte or a palette-law operand. So this +//! probe declares one reading of a Moore-local byte as a symbol codebook: +//! `0` unknown, `1..=26` a–z, `27` ä, `28` ö, `29` ü, `30` ß, `31..=255` +//! reserved. Crossword spelling ignores accents: an accented Latin letter folds +//! to its base (`cliché` is spelled `cliche`); ä ö ü ß are letters of their +//! own and do not fold. A word with any other character is not a crossword +//! word; it keeps its `WordId`. Whether this reading becomes a canonical value +//! tenant is a contract decision, not taken here. //! -//! # Vocabularies (DeepNSM-v2) +//! # Vocabularies //! //! - **English** (committed): `crates/deepnsm/word_frequency/academic_20k.csv`, -//! column `word`, lowercased, into `PaletteVocab::from_frequency_ranked`: -//! exactly the vocabulary DeepNSM-v2 builds (`examples/genre_shapes.rs`), -//! 18,555 distinct ids (`vocab.rs` quotes 18,559 distinct surfaces; four -//! collapse when lowercased). Words with a hyphen or apostrophe keep their id but -//! are not crossword words. +//! column `word`, lowercased, through `PaletteVocab::from_frequency_ranked`, +//! exactly as DeepNSM-v2 builds it (`examples/genre_shapes.rs`): 18,555 ids. +//! (`vocab.rs` quotes 18,559 distinct surfaces; four collapse when +//! lowercased.) //! - **German** (runtime only): DeepNSM-v2 builds no German vocabulary and the //! repository holds none. With `DEREKO_PATH` set, the 20,000 most frequent -//! DeReKo-2014 forms (lowercased, proper nouns and non-alphabetic forms -//! dropped, frequencies summed) go through the same `from_frequency_ranked`. -//! CC BY-NC 3.0; nothing derived is committed. ä ö ü ß stay letters of their -//! own (no `ae`/`ss` folding: the vocabulary has none); text is assumed NFC, -//! and a form with a combining mark is refused because it is not alphabetic. +//! DeReKo-2014 forms (lowercased, `NE` and non-alphabetic forms dropped, +//! frequencies summed) go through the same `from_frequency_ranked`. CC BY-NC +//! 3.0; nothing derived is committed. Text is assumed NFC. //! -//! # The candidate population +//! # The four arms (identical puzzles, givens and fixed points) //! -//! For a language, a word length `L`, an offset `i` and a letter code `x`, -//! `P(L, i, x)` is a bitset over that language's `WordId`s. A slot's -//! candidates start as `P_all(L)` and every exposed crossing letter ANDs one -//! `P` into them (`ndarray::simd::mask_and_assign`); `popcount_batch_u64` -//! decides: 0 = contradiction, 1 = forced, more = still candidates. A forced -//! word exposes its letters to its crossings through the `cross` lane, and the -//! queue runs to a fixed point. Backtracking is used only to CREATE puzzles and -//! to check uniqueness; solving is the fixed point alone. +//! | arm | content | addressing | propagation | +//! |---|---|---|---| +//! | A token | `WordId` masks | `cross[slot:offset]` | a placed word's letter at offset `i` ANDs `P(L, j, letter)` into the crossing slot | +//! | B Cartesian | `MooreSymbol8` board | `occupant[cell]` | a placed word writes its symbols; each affected slot's mask is recomputed from its cells' symbols | +//! | C literal | strings, chars | cell array | every unplaced slot re-scans the vocabulary strings each round (steps 2/2b) | +//! | D hybrid | `WordId` masks + `MooreSymbol8` board | `occupant[cell]` | a placed word writes its symbols; each NEW symbol ANDs one `P` into the cell's other slot | +//! +//! `P(L, i, x)` is a bitset over a language's `WordId`s: words of length `L` +//! with symbol `x` at offset `i`. `popcount_batch_u64` decides: 0 = +//! contradiction, 1 = forced, more = still candidates. Backtracking (arm A's +//! masks) only CREATES puzzles and checks uniqueness; solving is the fixed point. //! //! Run: `cargo run --release -p cognitive-shader-driver --example crossword_mask_propagation_probe` //! German too: `DEREKO_PATH=/path/DeReKo-2014-II-MainArchive-STT.100000.freq cargo run ...` @@ -86,10 +89,10 @@ const CROSSWORD_CLASS: ClassId = 0x0907; /// NYT rules: shortest word; most black squares (a sixth). const MIN_WORD: usize = 3; const MAX_BLACK_DIVISOR: usize = 6; -/// Longest slot (NYT 15×15) and the per-slot stride of every position lane. -const MAX_LEN: usize = 15; -const STRIDE: usize = 16; -/// No crossing at this position. +/// Longest slot (21×21 Sunday board) and the per-slot stride of position lanes. +const MAX_LEN: usize = 21; +const STRIDE: usize = 32; +/// No crossing / no occupant. const NONE: u16 = u16::MAX; /// No word placed in this slot. const UNSET: WordId = WordId::MAX; @@ -98,27 +101,82 @@ const GERMAN_VOCAB: usize = 20_000; const ACADEMIC: &str = include_str!("../../deepnsm/word_frequency/academic_20k.csv"); +// ─────────────────────────────── symbols ─────────────────────────────── + +/// The declared letter reading of one Moore-local byte: `0` unknown, `1..=26` +/// a–z, `27` ä, `28` ö, `29` ü, `30` ß, `31..=255` reserved. +#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash, Default)] +#[repr(transparent)] +struct MooreSymbol8(u8); + +impl MooreSymbol8 { + const UNKNOWN: Self = Self(0); + /// Codes in use, `0..=30`. + const COUNT: usize = 31; + + /// The symbol of a lowercase character after accent folding. + fn of(c: char) -> Option { + match fold(c) { + c @ 'a'..='z' => Some(Self(c as u8 - b'a' + 1)), + 'ä' => Some(Self(27)), + 'ö' => Some(Self(28)), + 'ü' => Some(Self(29)), + 'ß' => Some(Self(30)), + _ => None, + } + } + + /// The letter of a symbol; `None` for unknown and reserved codes. + fn letter(self) -> Option { + match self.0 { + 1..=26 => Some((b'a' + self.0 - 1) as char), + 27 => Some('ä'), + 28 => Some('ö'), + 29 => Some('ü'), + 30 => Some('ß'), + _ => None, + } + } +} + +/// Crossword spelling ignores accents: an accented Latin letter folds to its +/// base. ä ö ü ß are letters of their own and do not fold. +fn fold(c: char) -> char { + match c { + 'à' | 'á' | 'â' | 'ã' | 'å' | 'ā' => 'a', + 'ç' => 'c', + 'è' | 'é' | 'ê' | 'ë' | 'ē' => 'e', + 'ì' | 'í' | 'î' | 'ï' => 'i', + 'ñ' => 'n', + 'ò' | 'ó' | 'ô' | 'õ' => 'o', + 'ù' | 'ú' | 'û' => 'u', + 'ý' | 'ÿ' => 'y', + c => c, + } +} + +/// A word's crossword spelling, or `None` if it is not a crossword word +/// (wrong length, or a character with no symbol). +fn spelling(w: &str) -> Option> { + let s: Option> = w.chars().map(MooreSymbol8::of).collect(); + s.filter(|s| (MIN_WORD..=MAX_LEN).contains(&s.len())) +} + // ─────────────────────────────── vocabulary ─────────────────────────────── -/// The language a lexicon and a puzzle belong to. A puzzle is only ever -/// solved against its own language's populations. +/// The language a lexicon and a puzzle belong to. #[derive(Clone, Copy, Debug, PartialEq, Eq)] enum Lang { En, De, } -/// Cold side: strings and the letter codebook. Construction and checks only. +/// Cold side: the strings. Construction, the literal arm and checks only. struct Cold { vocab: PaletteVocab, - letters: Vec, } impl Cold { - fn code(&self, c: char) -> Option { - self.letters.binary_search(&c).ok().map(|i| i as u8 + 1) - } - fn word(&self, w: WordId) -> &str { self.vocab.word(w).expect("word id in range") } @@ -128,82 +186,66 @@ impl Cold { struct Hot { lang: Lang, blocks: usize, - /// Alphabet size + 1 (code 0 = no letter). - codes: usize, /// Per `WordId`: crossword length, 0 = not a crossword word. len: Vec, - /// `[id * STRIDE + offset]` = letter code. - spell: Vec, + /// `[id * STRIDE + offset]` = the symbol. + spell: Vec, /// By length: every crossword word of that length. all: Vec>, - /// By length: `[(offset * codes + code) * blocks ..]` = `P(L, offset, code)`. + /// By length: `[(offset * COUNT + symbol) * blocks ..]` = `P(L, offset, symbol)`. at: Vec>, } impl Hot { - fn pop(&self, len: usize, offset: usize, code: u8) -> &[u64] { - let i = (offset * self.codes + code as usize) * self.blocks; + fn pop(&self, len: usize, offset: usize, x: MooreSymbol8) -> &[u64] { + let i = (offset * MooreSymbol8::COUNT + x.0 as usize) * self.blocks; &self.at[len][i..i + self.blocks] } - fn letter(&self, w: WordId, offset: usize) -> u8 { + fn letter(&self, w: WordId, offset: usize) -> MooreSymbol8 { self.spell[w as usize * STRIDE + offset] } -} -/// A crossword word: `MIN_WORD..=MAX_LEN` letters, every one alphabetic and -/// lowercase. -fn admissible(w: &str) -> bool { - let n = w.chars().count(); - (MIN_WORD..=MAX_LEN).contains(&n) && w.chars().all(|c| c.is_alphabetic() && !c.is_uppercase()) + /// Bytes held by the populations (the fixed per-language cost). + fn population_bytes(&self) -> usize { + 8 * (self.all.iter().map(Vec::len).sum::() + + self.at.iter().map(Vec::len).sum::()) + } } -/// Build both sides from a frequency-ranked word list (most frequent first). +/// Both sides from a frequency-ranked word list (most frequent first). fn build_lexicon(lang: Lang, ranked: &[String]) -> (Hot, Cold) { let mut vocab = PaletteVocab::new(); vocab.from_frequency_ranked(ranked.iter().map(String::as_str)); let n = vocab.len(); assert!(n < UNSET as usize, "WordId::MAX is the empty marker"); - let mut letters: Vec = (0..n) - .filter_map(|id| vocab.word(id as WordId)) - .filter(|w| admissible(w)) - .flat_map(str::chars) - .collect(); - letters.sort_unstable(); - letters.dedup(); - assert!(letters.len() < 255, "letter codes are one byte"); - let cold = Cold { vocab, letters }; - let codes = cold.letters.len() + 1; let blocks = n.div_ceil(64); let mut hot = Hot { lang, blocks, - codes, len: vec![0; n], - spell: vec![0; n * STRIDE], + spell: vec![MooreSymbol8::UNKNOWN; n * STRIDE], all: vec![Vec::new(); MAX_LEN + 1], at: vec![Vec::new(); MAX_LEN + 1], }; for l in MIN_WORD..=MAX_LEN { hot.all[l] = vec![0; blocks]; - hot.at[l] = vec![0; l * codes * blocks]; + hot.at[l] = vec![0; l * MooreSymbol8::COUNT * blocks]; } for id in 0..n { - let w = cold.word(id as WordId); - if !admissible(w) { + let Some(sp) = spelling(vocab.word(id as WordId).expect("in range")) else { continue; - } - let l = w.chars().count(); + }; + let l = sp.len(); hot.len[id] = l as u8; let bit = 1u64 << (id % 64); hot.all[l][id / 64] |= bit; - for (i, c) in w.chars().enumerate() { - let code = cold.code(c).expect("letter in codebook"); - hot.spell[id * STRIDE + i] = code; - hot.at[l][(i * codes + code as usize) * blocks + id / 64] |= bit; + for (i, &x) in sp.iter().enumerate() { + hot.spell[id * STRIDE + i] = x; + hot.at[l][(i * MooreSymbol8::COUNT + x.0 as usize) * blocks + id / 64] |= bit; } } - (hot, cold) + (hot, Cold { vocab }) } /// English: DeepNSM-v2's academic vocabulary, built as `genre_shapes` builds it. @@ -247,12 +289,12 @@ fn german_ranked(text: &str) -> Vec { #[derive(Clone, Debug, PartialEq)] struct Grid { side: usize, - rows: Vec, + rows: Vec, } -/// Every white bit of a line lies in a run of at least three: OR of the +/// Every white bit of a line lies in a run of at least three: the OR of the /// three-wide windows covers the line. -fn line_runs_ok(w: u16) -> bool { +fn line_runs_ok(w: u32) -> bool { let t = w & (w >> 1) & (w >> 2); (t | (t << 1) | (t << 2)) == w } @@ -260,12 +302,12 @@ fn line_runs_ok(w: u16) -> bool { impl Grid { #[cfg(test)] fn from_rows(rows: &[&str]) -> Self { - let rows: Vec = rows + let rows: Vec = rows .iter() .map(|r| { r.bytes() .enumerate() - .fold(0u16, |m, (c, b)| m | (u16::from(b == b'.') << c)) + .fold(0u32, |m, (c, b)| m | (u32::from(b == b'.') << c)) }) .collect(); Self { @@ -274,20 +316,16 @@ impl Grid { } } - fn full_line(&self) -> u16 { - ((1u32 << self.side) - 1) as u16 + fn full_line(side: usize) -> u32 { + ((1u64 << side) - 1) as u32 } - fn cols(&self) -> Vec { + fn cols(&self) -> Vec { (0..self.side) - .map(|c| (0..self.side).fold(0u16, |m, r| m | (((self.rows[r] >> c) & 1) << r))) + .map(|c| (0..self.side).fold(0u32, |m, r| m | (((self.rows[r] >> c) & 1) << r))) .collect() } - fn reverse(&self, w: u16) -> u16 { - w.reverse_bits() >> (16 - self.side) - } - fn blacks(&self) -> usize { self.side * self.side - self @@ -299,7 +337,8 @@ impl Grid { /// 180° symmetry: row `r` reversed is row `side - 1 - r`. fn symmetric(&self) -> bool { - (0..self.side).all(|r| self.reverse(self.rows[r]) == self.rows[self.side - 1 - r]) + let rev = |w: u32| w.reverse_bits() >> (32 - self.side); + (0..self.side).all(|r| rev(self.rows[r]) == self.rows[self.side - 1 - r]) } /// Every across and down run is at least `MIN_WORD` long. @@ -310,14 +349,13 @@ impl Grid { .all(|&w| line_runs_ok(w)) } - /// One white region: grow from the first white cell by row-mask dilation - /// until nothing changes. + /// One white region: grow from the first white cell by row-mask dilation. fn connected(&self) -> bool { let Some(r0) = self.rows.iter().position(|&w| w != 0) else { return false; }; - let full = self.full_line(); - let mut reach = vec![0u16; self.side]; + let full = Self::full_line(self.side); + let mut reach = vec![0u32; self.side]; reach[r0] = self.rows[r0] & self.rows[r0].wrapping_neg(); loop { let mut changed = false; @@ -352,7 +390,7 @@ impl Grid { loop { let mut g = Self { side, - rows: vec![((1u32 << side) - 1) as u16; side], + rows: vec![Self::full_line(side); side], }; for i in 0..cells.div_ceil(2) { if rng.below(8) == 0 { @@ -376,9 +414,10 @@ struct Puzzle { len: Vec, /// `[slot * STRIDE + offset]`: the crossing `(slot:offset)` tile, or `NONE`. cross: Vec, - /// `[slot * STRIDE + offset]`: the Morton cell code. Construction and the - /// oracle only; propagation never reads it. + /// `[slot * STRIDE + offset]`: the Morton cell code. cell: Vec, + /// `[cell]`: the (at most two) `(slot:offset)` tiles on a cell. + occupant: Vec<[u16; 2]>, } impl Puzzle { @@ -387,6 +426,18 @@ impl Puzzle { } } +fn tile(s: usize, o: usize) -> u16 { + FacetTier { + hi: s as u8, + lo: o as u8, + } + .as_u16() +} + +fn untile(t: u16) -> (usize, usize) { + ((t >> 8) as usize, (t & 0xFF) as usize) +} + /// Compile a grid: slots from the row and column masks, each position's cell /// from `Morton8x8::checked_offset`, crossings from equal cell codes. fn compile(grid: &Grid, lang: Lang) -> Puzzle { @@ -424,19 +475,25 @@ fn compile(grid: &Grid, lang: Lang) -> Puzzle { } } assert!(len.len() < 255, "slot ids are one byte"); - let mut pos: Vec<(u16, u8, u8)> = len + let tiles = cell .iter() - .enumerate() - .flat_map(|(s, &l)| (0..l).map(move |o| (s, o))) - .map(|(s, o)| (cell[s * STRIDE + o as usize], s as u8, o)) - .collect(); - pos.sort_unstable(); + .filter(|&&c| c != NONE) + .max() + .map_or(0, |&m| m as usize + 1); + let mut occupant = vec![[NONE; 2]; tiles]; let mut cross = vec![NONE; len.len() * STRIDE]; - for w in pos.windows(2) { - if w[0].0 == w[1].0 { - let tile = |(_, s, o): (u16, u8, u8)| FacetTier { hi: s, lo: o }.as_u16(); - cross[w[0].1 as usize * STRIDE + w[0].2 as usize] = tile(w[1]); - cross[w[1].1 as usize * STRIDE + w[1].2 as usize] = tile(w[0]); + for (s, &l) in len.iter().enumerate() { + for o in 0..l as usize { + let c = cell[s * STRIDE + o] as usize; + let slot = if occupant[c][0] == NONE { 0 } else { 1 }; + assert_eq!(occupant[c][slot], NONE, "at most two positions per cell"); + occupant[c][slot] = tile(s, o); + if slot == 1 { + let other = occupant[c][0]; + cross[s * STRIDE + o] = other; + let (t, j) = untile(other); + cross[t * STRIDE + j] = tile(s, o); + } } } Puzzle { @@ -444,27 +501,42 @@ fn compile(grid: &Grid, lang: Lang) -> Puzzle { len, cross, cell, + occupant, } } -// ─────────────────────────────── runtime ─────────────────────────────── +// ─────────────────────────────── arms ─────────────────────────────── /// Why a solve did not start or did not finish. #[derive(Debug, Clone, Copy, PartialEq, Eq)] enum Stop { /// The puzzle and the populations are different languages. Language, - /// A slot was left with no candidate, or two placed words disagree. + /// A slot was left with no candidate, or two letters disagree on a cell. Contradiction(u8), } -/// Per-slot candidate masks over `WordId` and the placed word, if any. +/// What a solve leaves, and what it cost. +#[derive(Clone, Default)] +struct Solved { + placed: Vec, + /// `[slot * blocks ..]`: final candidates (a placed slot holds one bit). + cand: Vec, + /// Mask intersections (`P` ANDs) performed. + ands: usize, + /// Symbols written to the board (arms B and D). + writes: usize, + /// Bytes of runtime state the arm holds. + bytes: usize, +} + +/// Per-slot candidate masks plus the placed word, shared by arms A and D. #[derive(Clone)] struct State { blocks: usize, cand: Vec, placed: Vec, - events: usize, + ands: usize, } impl State { @@ -481,7 +553,7 @@ impl State { blocks: b, cand, placed: vec![UNSET; puz.slots()], - events: 0, + ands: 0, }) } @@ -500,8 +572,7 @@ impl State { self.placed[s] = w; } - /// Promote every unplaced slot already down to one candidate; refuse any - /// with none. + /// Promote every unplaced slot already down to one candidate. fn seed(&mut self, queue: &mut Vec) -> Result<(), Stop> { for s in 0..self.placed.len() { if self.placed[s] == UNSET { @@ -518,6 +589,23 @@ impl State { } Ok(()) } + + /// AND one population into slot `t`; promote at popcount 1. + fn narrow(&mut self, t: usize, p: &[u64], queue: &mut Vec) -> Result<(), Stop> { + let b = self.blocks; + let m = &mut self.cand[t * b..(t + 1) * b]; + mask_and_assign(m, p); + self.ands += 1; + match popcount_batch_u64(m) { + 0 => Err(Stop::Contradiction(t as u8)), + 1 => { + self.placed[t] = first_bit(m); + queue.push(t as u8); + Ok(()) + } + _ => Ok(()), + } + } } fn first_bit(m: &[u64]) -> WordId { @@ -541,12 +629,14 @@ fn bits(m: &[u64]) -> Vec { out } -/// The fixed point. Each queued slot exposes its word's letters through the -/// `cross` lane; an unplaced crossing slot ANDs `P(len, offset, letter)` into -/// its mask and is promoted at popcount 1. Reads only `Hot` and the compiled -/// `len` / `cross` lanes. -fn propagate(hot: &Hot, puz: &Puzzle, st: &mut State, queue: &mut Vec) -> Result<(), Stop> { - let b = st.blocks; +/// Arm A's fixed point: each queued slot hands its letter at every crossed +/// offset to the crossing `(slot:offset)` read from `cross`. +fn propagate_token( + hot: &Hot, + puz: &Puzzle, + st: &mut State, + queue: &mut Vec, +) -> Result<(), Stop> { while let Some(s) = queue.pop() { let s = s as usize; let w = st.placed[s]; @@ -555,7 +645,7 @@ fn propagate(hot: &Hot, puz: &Puzzle, st: &mut State, queue: &mut Vec) -> Re if c == NONE { continue; } - let (t, j) = ((c >> 8) as usize, (c & 0xFF) as usize); + let (t, j) = untile(c); let x = hot.letter(w, off); if st.placed[t] != UNSET { if hot.letter(st.placed[t], j) != x { @@ -563,25 +653,13 @@ fn propagate(hot: &Hot, puz: &Puzzle, st: &mut State, queue: &mut Vec) -> Re } continue; } - let m = &mut st.cand[t * b..(t + 1) * b]; - mask_and_assign(m, hot.pop(puz.len[t] as usize, j, x)); - st.events += 1; - match popcount_batch_u64(m) { - 0 => return Err(Stop::Contradiction(t as u8)), - 1 => { - let w2 = first_bit(m); - st.placed[t] = w2; - queue.push(t as u8); - } - _ => {} - } + st.narrow(t, hot.pop(puz.len[t] as usize, j, x), queue)?; } } Ok(()) } -/// Solve from the givens: place them, seed singles, run to the fixed point. -fn solve(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result { +fn start(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result<(State, Vec), Stop> { let mut st = State::new(hot, puz)?; let mut queue = Vec::new(); for &(s, w) in givens { @@ -589,13 +667,235 @@ fn solve(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result Result { + let (mut st, mut queue) = start(hot, puz, givens)?; + propagate_token(hot, puz, &mut st, &mut queue)?; + let bytes = st.cand.len() * 8 + st.placed.len() * 2; + Ok(Solved { + placed: st.placed, + cand: st.cand, + ands: st.ands, + writes: 0, + bytes, + }) +} + +/// Arm D: token masks, with the letters routed through a `MooreSymbol8` board +/// and the `occupant` lane. Each NEW symbol on a cell ANDs one `P` into the +/// cell's other slot. +fn solve_hybrid(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result { + let (mut st, mut queue) = start(hot, puz, givens)?; + let mut board = vec![MooreSymbol8::UNKNOWN; puz.occupant.len()]; + let mut writes = 0; + while let Some(s) = queue.pop() { + let s = s as usize; + let w = st.placed[s]; + for off in 0..puz.len[s] as usize { + let c = puz.cell[s * STRIDE + off] as usize; + let x = hot.letter(w, off); + let cur = board[c]; + if cur != MooreSymbol8::UNKNOWN { + if cur != x { + return Err(Stop::Contradiction(s as u8)); + } + continue; + } + board[c] = x; + writes += 1; + for &occ in &puz.occupant[c] { + if occ == NONE { + continue; + } + let (t, j) = untile(occ); + if t == s || st.placed[t] != UNSET { + continue; + } + st.narrow(t, hot.pop(puz.len[t] as usize, j, x), &mut queue)?; + } + } + } + let bytes = st.cand.len() * 8 + st.placed.len() * 2 + board.len(); + Ok(Solved { + placed: st.placed, + cand: st.cand, + ands: st.ands, + writes, + bytes, + }) +} + +/// Arm B: the board is the state. A placed word writes its symbols; every +/// unplaced slot on a newly written cell recomputes its mask from ALL of its +/// cells' symbols. (Slot-indexed: `placed` and the lanes share the index.) +#[allow(clippy::needless_range_loop)] +fn solve_cartesian(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result { + if hot.lang != puz.lang { + return Err(Stop::Language); + } + let b = hot.blocks; + let mut board = vec![MooreSymbol8::UNKNOWN; puz.occupant.len()]; + let mut placed = vec![UNSET; puz.slots()]; + let mut scratch = vec![0u64; b]; + let (mut ands, mut writes) = (0usize, 0usize); + let recompute = |t: usize, board: &[MooreSymbol8], scratch: &mut [u64], ands: &mut usize| { + let l = puz.len[t] as usize; + scratch.copy_from_slice(&hot.all[l]); + for j in 0..l { + let x = board[puz.cell[t * STRIDE + j] as usize]; + if x != MooreSymbol8::UNKNOWN { + mask_and_assign(scratch, hot.pop(l, j, x)); + *ands += 1; + } + } + popcount_batch_u64(scratch) + }; + let mut queue: Vec = Vec::new(); + for &(s, w) in givens { + placed[s as usize] = w; + queue.push(s); + } + for t in 0..puz.slots() { + if placed[t] == UNSET { + match recompute(t, &board, &mut scratch, &mut ands) { + 0 => return Err(Stop::Contradiction(t as u8)), + 1 => { + placed[t] = first_bit(&scratch); + queue.push(t as u8); + } + _ => {} + } + } + } + while let Some(s) = queue.pop() { + let s = s as usize; + let w = placed[s]; + for off in 0..puz.len[s] as usize { + let c = puz.cell[s * STRIDE + off] as usize; + let x = hot.letter(w, off); + let cur = board[c]; + if cur != MooreSymbol8::UNKNOWN { + if cur != x { + return Err(Stop::Contradiction(s as u8)); + } + continue; + } + board[c] = x; + writes += 1; + for &occ in &puz.occupant[c] { + if occ == NONE { + continue; + } + let (t, _) = untile(occ); + if t == s || placed[t] != UNSET { + continue; + } + match recompute(t, &board, &mut scratch, &mut ands) { + 0 => return Err(Stop::Contradiction(t as u8)), + 1 => { + placed[t] = first_bit(&scratch); + queue.push(t as u8); + } + _ => {} + } + } + } + } + let bytes = board.len() + placed.len() * 2 + scratch.len() * 8; + let mut cand = vec![0u64; puz.slots() * b]; + for t in 0..puz.slots() { + if placed[t] == UNSET { + recompute(t, &board, &mut scratch, &mut 0); + cand[t * b..(t + 1) * b].copy_from_slice(&scratch); + } else { + cand[t * b + placed[t] as usize / 64] = 1 << (placed[t] % 64); + } + } + Ok(Solved { + placed, + cand, + ands, + writes, + bytes, + }) +} + +/// Arm C, steps 2/2b's method: letters on a cell array, every unplaced slot +/// re-scanned from the vocabulary STRINGS each round, singles placed until +/// nothing changes. `None` on contradiction. (Slot-indexed on purpose: it +/// mirrors the loop it stands in for.) +#[allow(clippy::needless_range_loop)] +fn solve_literal(cold: &Cold, puz: &Puzzle, givens: &[(u8, WordId)]) -> Option> { + let words: Vec<(WordId, Vec)> = (0..cold.vocab.len() as WordId) + .filter_map(|id| { + let w = cold.word(id); + spelling(w)?; + Some((id, w.chars().map(fold).collect())) + }) + .collect(); + let mut board: HashMap = HashMap::new(); + let mut placed = vec![UNSET; puz.slots()]; + let put = |s: usize, w: &[char], board: &mut HashMap| -> bool { + (0..w.len()).all(|o| *board.entry(puz.cell[s * STRIDE + o]).or_insert(w[o]) == w[o]) + }; + for &(s, w) in givens { + let chars: Vec = cold.word(w).chars().map(fold).collect(); + if !put(s as usize, &chars, &mut board) { + return None; + } + placed[s as usize] = w; + } + loop { + let mut changed = false; + for s in 0..puz.slots() { + if placed[s] != UNSET { + continue; + } + let l = puz.len[s] as usize; + let fits: Vec<&(WordId, Vec)> = words + .iter() + .filter(|(_, w)| { + w.len() == l + && (0..l).all(|o| { + board + .get(&puz.cell[s * STRIDE + o]) + .is_none_or(|&c| c == w[o]) + }) + }) + .collect(); + match fits.len() { + 0 => return None, + 1 => { + if !put(s, &fits[0].1, &mut board) { + return None; + } + placed[s] = fits[0].0; + changed = true; + } + _ => {} + } + } + if !changed { + return Some(placed); + } + } } // ─────────────────────────────── creation ─────────────────────────────── -/// Depth-first fill over the same masks, most-constrained slot first, random +/// Why creation gave up on one attempt. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Miss { + /// No complete fill found within the budget. + Fill, + /// A fill was found, but uniqueness could not be decided within the budget. + Unique, +} + +/// Depth-first fill over arm A's masks, most-constrained slot first, random /// order within it. `budget` counts tried words. Creation only. fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Option> { fn go(hot: &Hot, puz: &Puzzle, st: State, rng: &mut Rng, budget: &mut usize) -> Option { @@ -615,7 +915,7 @@ fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Op let mut next = st.clone(); next.place(s, w); let mut q = vec![s as u8]; - if propagate(hot, puz, &mut next, &mut q).is_ok() { + if propagate_token(hot, puz, &mut next, &mut q).is_ok() { if let Some(done) = go(hot, puz, next, rng, budget) { return Some(done); } @@ -623,10 +923,8 @@ fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Op } None } - let mut st = State::new(hot, puz).ok()?; - let mut q = Vec::new(); - st.seed(&mut q).ok()?; - propagate(hot, puz, &mut st, &mut q).ok()?; + let (mut st, mut q) = start(hot, puz, &[]).ok()?; + propagate_token(hot, puz, &mut st, &mut q).ok()?; go(hot, puz, st, rng, budget).map(|st| st.placed) } @@ -655,7 +953,7 @@ fn count_fills( let mut next = st.clone(); next.place(s, w); let mut q = vec![s as u8]; - if propagate(hot, puz, &mut next, &mut q).is_ok() { + if propagate_token(hot, puz, &mut next, &mut q).is_ok() { n += go(hot, puz, next, cap - n, budget)?; if n >= cap { break; @@ -664,38 +962,44 @@ fn count_fills( } Some(n) } - match solve(hot, puz, givens) { - Ok(st) => go(hot, puz, st, cap, budget), - Err(_) => Some(0), + let Ok((mut st, mut q)) = start(hot, puz, givens) else { + return Some(0); + }; + if propagate_token(hot, puz, &mut st, &mut q).is_err() { + return Some(0); } + go(hot, puz, st, cap, budget) } -/// A created puzzle: its compiled layout, its solution and the givens that make -/// the solution unique. +/// A created puzzle: its compiled layout, its solution and the givens that +/// make the solution unique. struct Created { puz: Puzzle, solution: Vec, givens: Vec<(u8, WordId)>, + compile_ns: f64, } -/// Grid, random fill, then givens in random order until unique. `None` when -/// the budget runs out; the caller draws again. -fn create(hot: &Hot, side: usize, rng: &mut Rng, budget: usize) -> Option { +/// Grid, compile, random fill, then givens in random order until unique. +fn create(hot: &Hot, side: usize, rng: &mut Rng, budget: usize) -> Result { let grid = Grid::random_nyt(side, rng); + let t = Instant::now(); let puz = compile(&grid, hot.lang); + let compile_ns = t.elapsed().as_nanos() as f64; let mut b = budget; - let solution = random_fill(hot, &puz, rng, &mut b)?; + let solution = random_fill(hot, &puz, rng, &mut b).ok_or(Miss::Fill)?; let mut order: Vec = (0..puz.slots() as u8).collect(); rng.shuffle(&mut order); let mut givens = Vec::new(); for s in order { givens.push((s, solution[s as usize])); let mut b = budget; - if count_fills(hot, &puz, &givens, 2, &mut b)? == 1 { - return Some(Created { + if count_fills(hot, &puz, &givens, 2, &mut b).ok_or(Miss::Unique)? == 1 { + return Ok(Created { puz, solution, givens, + compile_ns, }); } } @@ -714,7 +1018,7 @@ struct Claims { /// One instance's claims at the fixed point: given and forced slots one claim /// each, an unplaced slot its true word as ENTAILED and every other surviving /// word as CANDIDATE. -fn emit(c: &Created, st: &State, lane: &mut Lane, claims: &mut Claims) { +fn emit(c: &Created, sol: &Solved, blocks: usize, lane: &mut Lane, claims: &mut Claims) { for s in 0..c.puz.slots() { let given = c.givens.iter().any(|g| g.0 as usize == s); let truth = c.solution[s]; @@ -724,15 +1028,15 @@ fn emit(c: &Created, st: &State, lane: &mut Lane, claims: &mut Claims) { claims.word.push(w); }; if given { - assert_eq!(st.placed[s], truth); + assert_eq!(sol.placed[s], truth); push(GIVEN, truth, lane); lane.expected[0] += 1; - } else if st.placed[s] != UNSET { - assert_eq!(st.placed[s], truth, "a forced word is the true word"); + } else if sol.placed[s] != UNSET { + assert_eq!(sol.placed[s], truth, "a forced word is the true word"); push(FORCED, truth, lane); lane.expected[1] += 1; } else { - let alive = bits(st.mask(s)); + let alive = bits(&sol.cand[s * blocks..(s + 1) * blocks]); assert!(alive.contains(&truth), "the law never rules out the truth"); push(ENTAILED, truth, lane); lane.expected[2] += 1; @@ -745,12 +1049,10 @@ fn emit(c: &Created, st: &State, lane: &mut Lane, claims: &mut Claims) { lane.close_group(); } -// ─────────────────────────────── oracles ─────────────────────────────── - -/// Every cell shared by two placed words carries the same letter, checked -/// from the Morton cell lanes alone (never from `cross`). +/// Every cell shared by two placed words carries the same symbol, checked from +/// the Morton cell lanes alone (never from `cross` or `occupant`). fn cell_consistent(hot: &Hot, puz: &Puzzle, placed: &[WordId]) -> bool { - let mut at: HashMap = HashMap::new(); + let mut at: HashMap = HashMap::new(); for (s, &w) in placed.iter().enumerate() { if w == UNSET { continue; @@ -765,66 +1067,6 @@ fn cell_consistent(hot: &Hot, puz: &Puzzle, placed: &[WordId]) -> bool { true } -/// Steps 2/2b's method, as the reference: letters on a cell array, every -/// unplaced slot's candidates re-scanned from the vocabulary STRINGS each -/// round, singles placed until nothing changes. Returns the placed word per -/// slot, or `None` on contradiction. (Slot-indexed on purpose: it mirrors the -/// step-2 loop it stands in for.) -#[allow(clippy::needless_range_loop)] -fn cell_scan_solve(cold: &Cold, puz: &Puzzle, givens: &[(u8, WordId)]) -> Option> { - let words: Vec<(WordId, Vec)> = (0..cold.vocab.len() as WordId) - .map(|id| (id, cold.word(id))) - .filter(|(_, w)| admissible(w)) - .map(|(id, w)| (id, w.chars().collect())) - .collect(); - let mut board: HashMap = HashMap::new(); - let mut placed = vec![UNSET; puz.slots()]; - let put = |s: usize, w: &[char], board: &mut HashMap| -> bool { - (0..w.len()).all(|o| *board.entry(puz.cell[s * STRIDE + o]).or_insert(w[o]) == w[o]) - }; - for &(s, w) in givens { - let chars: Vec = cold.word(w).chars().collect(); - if !put(s as usize, &chars, &mut board) { - return None; - } - placed[s as usize] = w; - } - loop { - let mut changed = false; - for s in 0..puz.slots() { - if placed[s] != UNSET { - continue; - } - let l = puz.len[s] as usize; - let fits: Vec<&(WordId, Vec)> = words - .iter() - .filter(|(_, w)| { - w.len() == l - && (0..l).all(|o| { - board - .get(&puz.cell[s * STRIDE + o]) - .is_none_or(|&c| c == w[o]) - }) - }) - .collect(); - match fits.len() { - 0 => return None, - 1 => { - if !put(s, &fits[0].1, &mut board) { - return None; - } - placed[s] = fits[0].0; - changed = true; - } - _ => {} - } - } - if !changed { - return Some(placed); - } - } -} - // ─────────────────────────────── benchmark ─────────────────────────────── fn median(mut v: Vec) -> f64 { @@ -832,86 +1074,20 @@ fn median(mut v: Vec) -> f64 { v[v.len() / 2] } -/// Size sweep: how big can puzzles be created, and how long does solving take. -fn sweep(name: &str, hot: &Hot, cold: &Cold) { - println!("\n{name}: creation and solving by board size (budget 200,000 tried words per attempt, 20 s per size)"); - println!( - " {:>4} {:>7} {:>6} {:>7} {:>9} {:>11} {:>11} {:>9} {:>11}", - "side", - "puzzles", - "slots", - "givens", - "solved", - "create ms", - "solve us", - "events", - "scan us" - ); - let mut rng = Rng(0x5EED); - for side in [5, 7, 9, 11, 13, 15] { - let t0 = Instant::now(); - let (mut made, mut tried) = (Vec::new(), 0usize); - let mut create_ms = Vec::new(); - while t0.elapsed() < Duration::from_secs(20) && made.len() < 40 { - tried += 1; - let t = Instant::now(); - if let Some(c) = create(hot, side, &mut rng, 200_000) { - create_ms.push(t.elapsed().as_secs_f64() * 1e3); - made.push(c); - } - } - if made.is_empty() { - println!( - " {side:>4} {:>7} (none created in {tried} attempts within 20 s)", - 0 - ); - continue; - } - let (mut solve_us, mut scan_us, mut events, mut solved) = (Vec::new(), Vec::new(), 0, 0); - for c in &made { - let t = Instant::now(); - let st = - black_box(solve(hot, &c.puz, &c.givens)).expect("a created puzzle is consistent"); - solve_us.push(t.elapsed().as_secs_f64() * 1e6); - events += st.events; - if st.placed.iter().all(|&w| w != UNSET) { - solved += 1; - } - assert!(cell_consistent(hot, &c.puz, &st.placed)); - if side <= 9 { - let t = Instant::now(); - let scan = cell_scan_solve(cold, &c.puz, &c.givens).expect("consistent"); - scan_us.push(t.elapsed().as_secs_f64() * 1e6); - assert_eq!(scan, st.placed, "mask fixed point == cell-scan fixed point"); - } - } - let n = made.len(); - let slots = made.iter().map(|c| c.puz.slots()).sum::() as f64 / n as f64; - let givens = made.iter().map(|c| c.givens.len()).sum::() as f64 / n as f64; - let scan = if scan_us.is_empty() { - "-".to_string() - } else { - format!("{:.0}", median(scan_us)) - }; - println!( - " {side:>4} {n:>4}/{tried:<3} {slots:>6.1} {givens:>7.1} {:>8.0}% {:>11.1} {:>11.1} {:>9.1} {:>11}", - 100.0 * solved as f64 / n as f64, - median(create_ms), - median(solve_us), - events as f64 / n as f64, - scan - ); - } +fn time_us(f: impl FnOnce() -> T) -> (T, f64) { + let t = Instant::now(); + let r = black_box(f()); + (r, t.elapsed().as_secs_f64() * 1e6) } -/// Filter micro-benchmark: one slot pattern (length + revealed letters) as a -/// mask AND chain vs a direct scan over the vocabulary's spell lane. -fn filter_bench(name: &str, hot: &Hot) { +/// Filter micro-benchmark: one slot pattern (length + revealed symbols) as a +/// mask AND chain vs a direct scan of the spell lane. +fn filter_bench(hot: &Hot) { let mut rng = Rng(0xF17); let words: Vec = (0..hot.len.len() as WordId) .filter(|&w| hot.len[w as usize] as usize >= 5) .collect(); - let patterns: Vec<(usize, Vec<(usize, u8)>)> = (0..2000) + let patterns: Vec<(usize, Vec<(usize, MooreSymbol8)>)> = (0..2000) .map(|_| { let w = words[rng.below(words.len() as u64) as usize]; let l = hot.len[w as usize] as usize; @@ -950,39 +1126,150 @@ fn filter_bench(name: &str, hot: &Hot) { assert_eq!(black_box(claims), scanned, "mask filter == direct scan"); let p = patterns.len() as f64; println!( - " {name}: candidate filter over {} patterns, {:.1} surviving claims each", + " candidate filter, {} patterns, {:.1} surviving claims each:", patterns.len(), claims as f64 / p ); println!( - " mask AND chain + popcount: {:>9.0} ns/pattern {:>7.2} ns/claim", + " mask AND chain + popcount {:>8.0} ns/pattern {:>8.2} ns/claim", mask_ns / p, mask_ns / claims as f64 ); println!( - " direct scan of the spell lane: {:>5.0} ns/pattern {:>7.2} ns/claim", + " direct spell-lane scan {:>8.0} ns/pattern {:>8.2} ns/claim", scan_ns / p, scan_ns / claims as f64 ); } +/// Size sweep: what can be created, and every arm's solve on the same puzzles. +fn sweep(hot: &Hot, cold: &Cold) -> usize { + println!(" creation by board size (budget 100,000 tried words per step, 30 s per size):"); + println!( + " {:>4} {:>8} {:>9} {:>11} {:>6} {:>6} {:>10}", + "side", "made", "miss f/u", "create ms", "slots", "givens", "compile us" + ); + let mut rng = Rng(0x5EED); + let mut largest = 0; + let mut per_size: Vec<(usize, Vec)> = Vec::new(); + for side in [5, 7, 9, 11, 13, 15, 17, 19, 21] { + let t0 = Instant::now(); + let (mut made, mut miss_fill, mut miss_unique, mut ms) = (Vec::new(), 0, 0, Vec::new()); + while t0.elapsed() < Duration::from_secs(30) && made.len() < 30 { + match time_us(|| create(hot, side, &mut rng, 100_000)) { + (Ok(c), us) => { + ms.push(us / 1e3); + made.push(c); + } + (Err(Miss::Fill), _) => miss_fill += 1, + (Err(Miss::Unique), _) => miss_unique += 1, + } + } + if made.is_empty() { + println!(" {side:>4} {:>8} {:>4}/{:<4}", 0, miss_fill, miss_unique); + continue; + } + largest = side; + let n = made.len() as f64; + println!( + " {side:>4} {:>8} {:>4}/{:<4} {:>11.1} {:>6.1} {:>6.1} {:>10.1}", + made.len(), + miss_fill, + miss_unique, + median(ms), + made.iter().map(|c| c.puz.slots()).sum::() as f64 / n, + made.iter().map(|c| c.givens.len()).sum::() as f64 / n, + median(made.iter().map(|c| c.compile_ns / 1e3).collect()) + ); + per_size.push((side, made)); + } + println!( + " solving the same puzzles, four arms (median us; ANDs, symbol writes, bytes per puzzle):" + ); + println!( + " {:>4} {:>7} {:>9} {:>9} {:>9} {:>9} {:>7} {:>7} {:>7} {:>8} {:>8}", + "side", + "solved", + "A token", + "D hybrid", + "B cart", + "C literal", + "ANDs A", + "ANDs B", + "writes", + "bytes A", + "bytes B" + ); + for (side, made) in &per_size { + let (mut ta, mut td, mut tb, mut tc) = (Vec::new(), Vec::new(), Vec::new(), Vec::new()); + let (mut and_a, mut and_b, mut writes, mut bytes_a, mut bytes_b, mut solved) = + (0, 0, 0, 0, 0, 0); + for c in made { + let (a, us) = time_us(|| solve_token(hot, &c.puz, &c.givens).unwrap()); + ta.push(us); + let (d, us) = time_us(|| solve_hybrid(hot, &c.puz, &c.givens).unwrap()); + td.push(us); + let (b, us) = time_us(|| solve_cartesian(hot, &c.puz, &c.givens).unwrap()); + tb.push(us); + assert_eq!(a.placed, d.placed); + assert_eq!(a.placed, b.placed); + assert_eq!(a.cand, d.cand); + assert_eq!(a.cand, b.cand); + assert!(cell_consistent(hot, &c.puz, &a.placed)); + if *side <= 9 { + let (lit, us) = time_us(|| solve_literal(cold, &c.puz, &c.givens).unwrap()); + tc.push(us); + assert_eq!(lit, a.placed, "literal fixed point == mask fixed point"); + } + and_a += a.ands; + and_b += b.ands; + writes += d.writes; + bytes_a += a.bytes; + bytes_b += b.bytes; + if a.placed.iter().all(|&w| w != UNSET) { + solved += 1; + } + } + let n = made.len(); + let lit = if tc.is_empty() { + "-".to_string() + } else { + format!("{:.0}", median(tc)) + }; + println!( + " {side:>4} {:>6.0}% {:>9.1} {:>9.1} {:>9.1} {:>9} {:>7.1} {:>7.1} {:>7.1} {:>8} {:>8}", + 100.0 * solved as f64 / n as f64, + median(ta), + median(td), + median(tb), + lit, + and_a as f64 / n as f64, + and_b as f64 / n as f64, + writes as f64 / n as f64, + bytes_a / n, + bytes_b / n + ); + } + largest +} + /// About a million claims at one size, folded the shared three ways. -fn lane_run(name: &str, hot: &Hot, side: usize) { +fn lane_run(hot: &Hot, side: usize) { let decl = declarations(CROSSWORD_CLASS); admit(&decl, CROSSWORD_CLASS).expect("the crossword class declares the canonical reading"); let mut rng = Rng(0x1A4E); let (mut lane, mut claims, mut slots) = (Lane::default(), Claims::default(), Vec::new()); let t = Instant::now(); while lane.edges.len() < 1_000_000 { - let Some(c) = create(hot, side, &mut rng, 200_000) else { + let Ok(c) = create(hot, side, &mut rng, 100_000) else { continue; }; - let st = solve(hot, &c.puz, &c.givens).expect("consistent"); + let sol = solve_token(hot, &c.puz, &c.givens).expect("consistent"); slots.push(c.puz.slots()); - emit(&c, &st, &mut lane, &mut claims); + emit(&c, &sol, hot.blocks, &mut lane, &mut claims); } println!( - "\n {name}: {} claims from {} {side}x{side} puzzles, built in {:.2?}", + " lane: {} claims from {} {side}x{side} puzzles, built in {:.2?}", lane.edges.len(), lane.groups, t.elapsed() @@ -999,20 +1286,24 @@ fn lane_run(name: &str, hot: &Hot, side: usize) { fn run_language(name: &str, hot: &Hot, cold: &Cold) { let words = hot.len.iter().filter(|&&l| l > 0).count(); + let used: String = (1..MooreSymbol8::COUNT as u8) + .map(MooreSymbol8) + .filter(|&x| (MIN_WORD..=MAX_LEN).any(|l| hot.pop(l, 0, x).iter().any(|&w| w != 0))) + .filter_map(MooreSymbol8::letter) + .collect(); println!( - "\n=== {name}: {} WordIds, {} crossword words, {} letters ({})", + "\n=== {name}: {} WordIds, {} crossword words, first letters in use [{used}], populations {:.1} MB", cold.vocab.len(), words, - cold.letters.len(), - cold.letters.iter().collect::() + hot.population_bytes() as f64 / 1e6 ); - filter_bench(name, hot); - sweep(name, hot, cold); - lane_run(name, hot, 7); + filter_bench(hot); + sweep(hot, cold); + lane_run(hot, 5); } fn main() { - println!("D-PUZZLE-0 step 3: crossword propagation as Cartesian-addressed population masking"); + println!("D-PUZZLE-0 step 3: crossword propagation, four representations of one fold"); let (hot, cold) = build_lexicon(Lang::En, &english_ranked()); run_language("English, DeepNSM-v2 academic_20k", &hot, &cold); match std::env::var("DEREKO_PATH") { @@ -1049,27 +1340,65 @@ mod tests { cold.vocab.id(w).expect("in the fixture") } - /// The English vocabulary is DeepNSM-v2's: 18,555 distinct ids in - /// frequency order, and the hyphen/apostrophe rows keep ids but no length. + type Arm = fn(&Hot, &Puzzle, &[(u8, WordId)]) -> Result; + const ARMS: [(&str, Arm); 3] = [ + ("token", solve_token), + ("hybrid", solve_hybrid), + ("cartesian", solve_cartesian), + ]; + + fn created(hot: &Hot, side: usize, seed: u64) -> Created { + let mut rng = Rng(seed); + loop { + if let Ok(c) = create(hot, side, &mut rng, 100_000) { + return c; + } + } + } + + /// The codebook is the declared one: a–z 1..=26, ä ö ü ß 27..=30, 0 for + /// unknown; accents fold; anything else has no symbol. + #[test] + fn moore_symbol8_is_the_declared_codebook() { + assert_eq!(MooreSymbol8::of('a'), Some(MooreSymbol8(1))); + assert_eq!(MooreSymbol8::of('z'), Some(MooreSymbol8(26))); + for (c, n) in [('ä', 27), ('ö', 28), ('ü', 29), ('ß', 30)] { + assert_eq!(MooreSymbol8::of(c), Some(MooreSymbol8(n))); + } + assert_eq!(MooreSymbol8::of('é'), MooreSymbol8::of('e')); + assert_eq!(MooreSymbol8::of('ç'), MooreSymbol8::of('c')); + for c in ['-', '\'', ' ', 'ø', 'A'] { + assert_eq!(MooreSymbol8::of(c), None, "{c}"); + } + for n in 1..=30u8 { + let x = MooreSymbol8(n); + assert_eq!(MooreSymbol8::of(x.letter().unwrap()), Some(x)); + } + assert_eq!(MooreSymbol8::UNKNOWN.letter(), None); + assert_eq!(MooreSymbol8(31).letter(), None); + } + + /// The English vocabulary is DeepNSM-v2's, and the crossword projection + /// only spells words: `cliché` keeps its id and is spelled `cliche`; + /// hyphenated rows keep their id and have no spelling. #[test] fn english_is_the_deepnsm_v2_academic_vocabulary() { let (hot, cold) = english(); assert_eq!(cold.vocab.len(), 18_555); assert_eq!(cold.word(0), "the"); - let id = word(&cold, "so-called"); - assert_eq!(hot.len[id as usize], 0); - // The letters come from the data: a-z plus the é of `sauté` and - // `cliché` (DeepNSM-v2 lowercases and does not fold accents). - let mut az: Vec = ('a'..='z').collect(); - az.push('é'); - assert_eq!(cold.letters, az); - assert_eq!(hot.len[word(&cold, "cliché") as usize], 6); - } - - /// The letter codes round-trip: every crossword word decodes from its - /// spell lane back to its own string. + assert_eq!(hot.len[word(&cold, "so-called") as usize], 0); + let cliche = word(&cold, "cliché"); + let spelled: String = (0..6) + .map(|o| hot.letter(cliche, o).letter().unwrap()) + .collect(); + assert_eq!(spelled, "cliche"); + // English has no umlaut words, so their populations are empty. + assert!(hot.pop(5, 0, MooreSymbol8(27)).iter().all(|&w| w == 0)); + } + + /// Every crossword word decodes from its symbols to its folded string. #[test] - fn every_word_decodes_from_its_letter_codes() { + fn every_word_decodes_from_its_symbols() { let (hot, cold) = english(); let mut n = 0; for id in 0..cold.vocab.len() as WordId { @@ -1078,40 +1407,36 @@ mod tests { continue; } let back: String = (0..l) - .map(|o| cold.letters[hot.letter(id, o) as usize - 1]) + .map(|o| hot.letter(id, o).letter().unwrap()) .collect(); - assert_eq!(back, cold.word(id)); + assert_eq!(back, cold.word(id).chars().map(fold).collect::()); n += 1; } - assert!(n > 10_000); + assert!(n > 15_000); } - /// Indexed positional populations equal a direct scan over the vocabulary - /// strings for random patterns. + /// Positional populations equal a direct scan over the vocabulary strings. #[test] fn positional_populations_equal_a_string_scan() { let (hot, cold) = english(); let mut rng = Rng(2); for _ in 0..500 { - let l = MIN_WORD + rng.below((MAX_LEN - MIN_WORD + 1) as u64) as usize; + let l = MIN_WORD + rng.below(10) as usize; let k = rng.below(4) as usize; let fixed: Vec<(usize, char)> = (0..k) .map(|_| { let o = rng.below(l as u64) as usize; - ( - o, - cold.letters[rng.below(cold.letters.len() as u64) as usize], - ) + (o, (b'a' + rng.below(26) as u8) as char) }) .collect(); let mut m = hot.all[l].clone(); for &(o, c) in &fixed { - mask_and_assign(&mut m, hot.pop(l, o, cold.code(c).unwrap())); + mask_and_assign(&mut m, hot.pop(l, o, MooreSymbol8::of(c).unwrap())); } let scan: Vec = (0..cold.vocab.len() as WordId) .filter(|&id| { - let w: Vec = cold.word(id).chars().collect(); - admissible(cold.word(id)) + let w: Vec = cold.word(id).chars().map(fold).collect(); + spelling(cold.word(id)).is_some() && w.len() == l && fixed.iter().all(|&(o, c)| w[o] == c) }) @@ -1120,18 +1445,19 @@ mod tests { } } - /// A slot narrowed to one candidate is promoted. + /// A slot narrowed to one candidate is promoted, in every arm. #[test] fn a_single_candidate_is_forced() { let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "cow", "dog", "ant"])); let puz = compile(&plus(), Lang::En); - let (across, down) = (0u8, 1u8); assert_eq!(puz.len, [3, 3]); - let st = solve(&hot, &puz, &[(across, word(&cold, "cat"))]).unwrap(); - assert_eq!(st.placed[down as usize], word(&cold, "ant")); + for (name, arm) in ARMS { + let sol = arm(&hot, &puz, &[(0, word(&cold, "cat"))]).unwrap(); + assert_eq!(sol.placed[1], word(&cold, "ant"), "{name}"); + } } - /// A forced word constrains EVERY crossing slot, not just the first. + /// A placed word constrains EVERY crossing slot, in every arm. #[test] fn a_placed_word_masks_every_crossing() { let (hot, cold) = build_lexicon( @@ -1140,85 +1466,96 @@ mod tests { ); let puz = compile(&Grid::from_rows(&["...", ".#.", ".#."]), Lang::En); assert_eq!(puz.len, [3, 3, 3]); - let st = solve(&hot, &puz, &[(0, word(&cold, "cat"))]).unwrap(); - let first = |s: usize| -> Vec { - bits(st.mask(s)) - .into_iter() - .map(|w| cold.word(w).chars().next().unwrap()) - .collect() - }; - assert!(first(1).iter().all(|&c| c == 'c') && first(1).len() == 3); - assert!(first(2).iter().all(|&c| c == 't') && first(2).len() == 2); + for (name, arm) in ARMS { + let sol = arm(&hot, &puz, &[(0, word(&cold, "cat"))]).unwrap(); + let first = |s: usize| -> Vec { + bits(&sol.cand[s * hot.blocks..(s + 1) * hot.blocks]) + .into_iter() + .map(|w| cold.word(w).chars().next().unwrap()) + .collect() + }; + assert_eq!(first(1), ['c', 'c', 'c'], "{name}"); + assert_eq!(first(2), ['t', 't'], "{name}"); + } } - /// No candidate left is a contradiction, never a silent empty slot. + /// No candidate left is a contradiction, in every arm. #[test] fn an_empty_slot_is_a_contradiction() { let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "cow", "dog"])); let puz = compile(&plus(), Lang::En); - assert_eq!( - solve(&hot, &puz, &[(0, word(&cold, "cat"))]).err(), - Some(Stop::Contradiction(1)) - ); + for (name, arm) in ARMS { + assert_eq!( + arm(&hot, &puz, &[(0, word(&cold, "cat"))]).err(), + Some(Stop::Contradiction(1)), + "{name}" + ); + } } - /// The crossing comes from the compiled lane. Re-pointing one crossing - /// lets an incompatible word survive, and the cell oracle sees it; with - /// the lane emptied nothing propagates at all. + /// Crossings come from the compiled lanes. Re-pointing one crossing in + /// `cross` lets an incompatible word survive in arm A, and the cell oracle + /// sees it. Emptying `cross` (A) or `occupant` (B, D) stops propagation. #[test] - fn crossings_are_read_from_the_compiled_lane() { + fn crossings_are_read_from_the_compiled_lanes() { let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "cow", "dog", "ant"])); let puz = compile(&plus(), Lang::En); let given = [(0u8, word(&cold, "cat"))]; - let ok = solve(&hot, &puz, &given).unwrap(); - assert!(cell_consistent(&hot, &puz, &ok.placed)); + for (_, arm) in ARMS { + assert!(cell_consistent( + &hot, + &puz, + &arm(&hot, &puz, &given).unwrap().placed + )); + } - // across offset 1 really crosses down offset 0; re-point the pair - // (both directions) at down offset 1. let mut wrong = puz.clone(); - assert_eq!(wrong.cross[1], FacetTier { hi: 1, lo: 0 }.as_u16()); - wrong.cross[1] = FacetTier { hi: 1, lo: 1 }.as_u16(); + assert_eq!(wrong.cross[1], tile(1, 0)); + wrong.cross[1] = tile(1, 1); wrong.cross[STRIDE] = NONE; - wrong.cross[STRIDE + 1] = FacetTier { hi: 0, lo: 1 }.as_u16(); - let bad = solve(&hot, &wrong, &given).unwrap(); + wrong.cross[STRIDE + 1] = tile(0, 1); + let bad = solve_token(&hot, &wrong, &given).unwrap(); assert_eq!(bad.placed[1], word(&cold, "cat")); assert!(!cell_consistent(&hot, &wrong, &bad.placed)); let mut cut = puz.clone(); cut.cross.fill(NONE); - let none = solve(&hot, &cut, &given).unwrap(); - assert_eq!(none.placed[1], UNSET); - assert_eq!(none.events, 0); + let a = solve_token(&hot, &cut, &given).unwrap(); + assert_eq!((a.placed[1], a.ands), (UNSET, 0)); + + let mut cut = puz.clone(); + cut.occupant.iter_mut().for_each(|o| *o = [NONE; 2]); + for arm in [solve_hybrid as Arm, solve_cartesian] { + let s = arm(&hot, &cut, &given).unwrap(); + assert_eq!(s.placed[1], UNSET); + } } - /// Crossing tiles are symmetric and land on the same Morton cell. + /// Crossing tiles are symmetric, land on one Morton cell, and match the + /// occupant lane, up to 21x21. #[test] fn every_crossing_names_the_same_cell_both_ways() { let mut rng = Rng(4); - for side in [5, 7, 9, 15] { + for side in [5, 7, 15, 21] { let puz = compile(&Grid::random_nyt(side, &mut rng), Lang::En); for s in 0..puz.slots() { for o in 0..puz.len[s] as usize { let c = puz.cross[s * STRIDE + o]; assert_ne!(c, NONE, "an NYT grid checks every square"); - let (t, j) = ((c >> 8) as usize, (c & 0xFF) as usize); - assert_eq!(puz.cell[t * STRIDE + j], puz.cell[s * STRIDE + o]); - assert_eq!( - puz.cross[t * STRIDE + j], - FacetTier { - hi: s as u8, - lo: o as u8 - } - .as_u16() - ); + let (t, j) = untile(c); + let cell = puz.cell[s * STRIDE + o]; + assert_eq!(puz.cell[t * STRIDE + j], cell); + assert_eq!(puz.cross[t * STRIDE + j], tile(s, o)); + let occ = puz.occupant[cell as usize]; + assert!(occ.contains(&tile(s, o)) && occ.contains(&c)); } } } } - /// Languages are separate populations: German keeps ä ö ü ß as letters, - /// the same WordId names different words, and a puzzle is refused against - /// the other language's populations. + /// Languages are separate populations: WordId 0 names different words, + /// only German has umlaut populations, and every arm refuses a puzzle + /// against the other language's populations. #[test] fn languages_do_not_share_ordinals_or_populations() { let (en, en_cold) = build_lexicon(Lang::En, &words(&["the", "and", "house", "street"])); @@ -1227,27 +1564,27 @@ mod tests { &words(&["der", "und", "haus", "straße", "über", "größe", "mädchen"]), ); assert_ne!(en_cold.word(0), de_cold.word(0)); - for c in ['ä', 'ö', 'ü', 'ß'] { - assert!(de_cold.code(c).is_some() && en_cold.code(c).is_none()); - } + let umlaut_u = MooreSymbol8::of('ü').unwrap(); + assert!(en.pop(4, 0, umlaut_u).iter().all(|&w| w == 0)); + assert!(de.pop(4, 0, umlaut_u).iter().any(|&w| w != 0)); let de_puz = compile(&plus(), Lang::De); - assert_eq!(State::new(&en, &de_puz).err(), Some(Stop::Language)); - assert!(State::new(&de, &de_puz).is_ok()); + for (name, arm) in ARMS { + assert_eq!(arm(&en, &de_puz, &[]).err(), Some(Stop::Language), "{name}"); + assert!(arm(&de, &de_puz, &[]).is_ok(), "{name}"); + } } /// Bits 59..63 carry state only: every claim edge is zero outside them, - /// its code is one of the four shared states, and two different words in - /// the same state have the identical edge. + /// its code is one of the four shared states, two different words in one + /// state have the identical edge, and reserved codes never appear. #[test] fn ce64_carries_state_never_content() { let (hot, _) = english(); - let mut rng = Rng(7); let (mut lane, mut claims) = (Lane::default(), Claims::default()); - while lane.groups < 5 { - if let Some(c) = create(&hot, 5, &mut rng, 200_000) { - let st = solve(&hot, &c.puz, &c.givens).unwrap(); - emit(&c, &st, &mut lane, &mut claims); - } + for seed in 0..4 { + let c = created(&hot, 5, 70 + seed); + let sol = solve_token(&hot, &c.puz, &c.givens).unwrap(); + emit(&c, &sol, hot.blocks, &mut lane, &mut claims); } let codes: Vec = population_fold::STATES .iter() @@ -1263,36 +1600,36 @@ mod tests { let (a, b) = (cands[0], cands[1]); assert_ne!(claims.word[a], claims.word[b]); assert_eq!(lane.edges[a], lane.edges[b]); - // reserved codes 24..31 never appear assert_eq!(count_in(&lane.edges, !0u32 << 24), 0); assert_eq!(count_in(&lane.edges, UNKNOWN_CAUSES.bit()), 0); } - /// The mask fixed point equals steps 2/2b's cell-scan fixed point on - /// created puzzles, and every created puzzle is unique and consistent. + /// The four arms reach one fixed point on created puzzles: same placed + /// words (A, B, C, D) and same surviving masks (A, B, D); every created + /// puzzle is unique and consistent. #[test] - fn mask_propagation_equals_the_cell_scan_oracle() { + fn the_four_arms_agree_on_created_puzzles() { let (hot, cold) = english(); - let mut rng = Rng(9); - let mut made = 0; - while made < 8 { - let side = if made % 2 == 0 { 5 } else { 7 }; - let Some(c) = create(&hot, side, &mut rng, 200_000) else { - continue; - }; - made += 1; - let st = solve(&hot, &c.puz, &c.givens).unwrap(); + for seed in 0..6 { + let c = created(&hot, 5, 90 + seed); + let a = solve_token(&hot, &c.puz, &c.givens).unwrap(); + let b = solve_cartesian(&hot, &c.puz, &c.givens).unwrap(); + let d = solve_hybrid(&hot, &c.puz, &c.givens).unwrap(); + assert_eq!(a.placed, b.placed); + assert_eq!(a.placed, d.placed); + assert_eq!(a.cand, b.cand); + assert_eq!(a.cand, d.cand); + assert_eq!(solve_literal(&cold, &c.puz, &c.givens).unwrap(), a.placed); + assert!(cell_consistent(&hot, &c.puz, &c.solution)); + let mut budget = 1_000_000; assert_eq!( - cell_scan_solve(&cold, &c.puz, &c.givens).unwrap(), - st.placed + count_fills(&hot, &c.puz, &c.givens, 3, &mut budget), + Some(1) ); - assert!(cell_consistent(&hot, &c.puz, &c.solution)); - let mut b = 1_000_000; - assert_eq!(count_fills(&hot, &c.puz, &c.givens, 3, &mut b), Some(1)); } } - /// The mask grid rules agree with a cell-by-cell check, and refuse. + /// The row-mask grid rules agree with a cell-by-cell check, both ways. #[test] fn mask_grid_rules_equal_a_cell_check() { fn cell_valid(g: &Grid) -> bool { @@ -1341,10 +1678,10 @@ mod tests { let mut rng = Rng(11); let (mut valid, mut invalid) = (0, 0); for _ in 0..3000 { - let side = [5, 7, 9][rng.below(3) as usize]; + let side = [5, 7, 9, 15, 21][rng.below(5) as usize]; let mut g = Grid { side, - rows: vec![((1u32 << side) - 1) as u16; side], + rows: vec![Grid::full_line(side); side], }; for i in 0..side * side { if rng.below(9) == 0 { @@ -1368,20 +1705,21 @@ mod tests { ); } - /// Solving runs without the strings: `Cold` is dropped before the solve, - /// and the result is the same. + /// Solving runs without the strings: `Cold` is dropped first, and every + /// mask arm still reaches the literal arm's fixed point. #[test] fn the_hot_path_needs_no_strings() { let (hot, cold) = english(); - let mut rng = Rng(13); - let c = loop { - if let Some(c) = create(&hot, 7, &mut rng, 200_000) { - break c; - } - }; - let expect = cell_scan_solve(&cold, &c.puz, &c.givens).unwrap(); + let c = created(&hot, 5, 13); + let expect = solve_literal(&cold, &c.puz, &c.givens).unwrap(); drop(cold); - assert_eq!(solve(&hot, &c.puz, &c.givens).unwrap().placed, expect); + for (name, arm) in ARMS { + assert_eq!( + arm(&hot, &c.puz, &c.givens).unwrap().placed, + expect, + "{name}" + ); + } } /// Three ways, partition, declaration gate: the shared fold, unchanged. @@ -1391,9 +1729,9 @@ mod tests { let mut rng = Rng(17); let (mut lane, mut claims) = (Lane::default(), Claims::default()); while lane.edges.len() < 20_000 { - if let Some(c) = create(&hot, 5, &mut rng, 200_000) { - let st = solve(&hot, &c.puz, &c.givens).unwrap(); - emit(&c, &st, &mut lane, &mut claims); + if let Ok(c) = create(&hot, 5, &mut rng, 100_000) { + let sol = solve_token(&hot, &c.puz, &c.givens).unwrap(); + emit(&c, &sol, hot.blocks, &mut lane, &mut claims); } } let decl = declarations(CROSSWORD_CLASS); From 4706c6f004c16244ef8e164dcacf1f679c797349 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 17:44:24 +0000 Subject: [PATCH 3/8] D-PUZZLE-0 step 3: cui-bono ablation of givens; warm, rotated arm timing Ablation: remove each given in turn, re-run arm A's fixed point and the uniqueness count, and classify the reaction from measured readouts only (no new state, CE64 codebook unchanged): Redundant (fixed point and uniqueness unchanged), ShortcutOnly (fixed point weakens, search still unique), SilentlyNecessary (fixed point unchanged, a second fill appears), Necessary (both). The readout nominates; it does not prove cause. Timing: the first benchmark charged whichever arm ran first for the cold populations (arm A 16.6 us vs D 6.9 us at an equal AND count). Each arm now gets a warm-up call and its best of 5 runs in rotated order. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .../crossword_mask_propagation_probe.rs | 222 +++++++++++++++++- 1 file changed, 209 insertions(+), 13 deletions(-) diff --git a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs index dd9cd29f5..9f2e999f1 100644 --- a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs +++ b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs @@ -530,6 +530,9 @@ struct Solved { bytes: usize, } +/// One mask arm: puzzle and givens in, fixed point out. +type Arm = fn(&Hot, &Puzzle, &[(u8, WordId)]) -> Result; + /// Per-slot candidate masks plus the placed word, shared by arms A and D. #[derive(Clone)] struct State { @@ -1006,6 +1009,91 @@ fn create(hot: &Hot, side: usize, rng: &mut Rng, budget: usize) -> Result, +} + +/// The four outcomes of one ablation. The readout nominates; it does not +/// prove cause: a given can be necessary in this set and replaceable by +/// another set. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Reaction { + /// The fixed point is unchanged and the fill stays unique. + Redundant, + /// The fixed point weakens, but search still finds only one fill. + ShortcutOnly, + /// The fixed point is unchanged, yet the fill is no longer unique: + /// propagation never used this given, search did. + SilentlyNecessary, + /// The fixed point weakens and the fill is no longer unique. + Necessary, +} + +impl Ablation { + fn reaction(self) -> Option { + let moved = self.lost_placed > 0 || self.popcount_delta != 0; + let unique = self.fills? == 1; + Some(match (moved, unique) { + (false, true) => Reaction::Redundant, + (true, true) => Reaction::ShortcutOnly, + (false, false) => Reaction::SilentlyNecessary, + (true, false) => Reaction::Necessary, + }) + } +} + +fn total_popcount(sol: &Solved, blocks: usize) -> i64 { + sol.cand + .chunks_exact(blocks) + .map(|m| popcount_batch_u64(m) as i64) + .sum() +} + +/// Remove each given in turn, re-run arm A's fixed point and the uniqueness +/// count, and report the difference against the full set of givens. +fn ablate(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)], budget: usize) -> Vec { + let base = solve_token(hot, puz, givens).expect("the full givens are consistent"); + let base_pop = total_popcount(&base, hot.blocks); + givens + .iter() + .enumerate() + .map(|(i, &(slot, _))| { + let rest: Vec<(u8, WordId)> = givens + .iter() + .enumerate() + .filter(|&(k, _)| k != i) + .map(|(_, &g)| g) + .collect(); + let sol = solve_token(hot, puz, &rest).expect("a subset of consistent givens"); + let lost_placed = (0..puz.slots()) + .filter(|&s| s != slot as usize) + .filter(|&s| base.placed[s] != UNSET && sol.placed[s] == UNSET) + .count(); + let mut b = budget; + Ablation { + slot, + lost_placed, + popcount_delta: total_popcount(&sol, hot.blocks) - base_pop, + fills: count_fills(hot, puz, &rest, 2, &mut b), + } + }) + .collect() +} + // ─────────────────────────────── the lane ─────────────────────────────── /// Content lanes beside the epistemic edges: the claim "slot holds word". @@ -1142,15 +1230,17 @@ fn filter_bench(hot: &Hot) { ); } +/// Timed rounds per puzzle and arm (after one warm-up call each). +const ROUNDS: usize = 5; + /// Size sweep: what can be created, and every arm's solve on the same puzzles. -fn sweep(hot: &Hot, cold: &Cold) -> usize { +fn sweep(hot: &Hot, cold: &Cold) -> Vec<(usize, Vec)> { println!(" creation by board size (budget 100,000 tried words per step, 30 s per size):"); println!( " {:>4} {:>8} {:>9} {:>11} {:>6} {:>6} {:>10}", "side", "made", "miss f/u", "create ms", "slots", "givens", "compile us" ); let mut rng = Rng(0x5EED); - let mut largest = 0; let mut per_size: Vec<(usize, Vec)> = Vec::new(); for side in [5, 7, 9, 11, 13, 15, 17, 19, 21] { let t0 = Instant::now(); @@ -1169,7 +1259,6 @@ fn sweep(hot: &Hot, cold: &Cold) -> usize { println!(" {side:>4} {:>8} {:>4}/{:<4}", 0, miss_fill, miss_unique); continue; } - largest = side; let n = made.len() as f64; println!( " {side:>4} {:>8} {:>4}/{:<4} {:>11.1} {:>6.1} {:>6.1} {:>10.1}", @@ -1184,7 +1273,7 @@ fn sweep(hot: &Hot, cold: &Cold) -> usize { per_size.push((side, made)); } println!( - " solving the same puzzles, four arms (median us; ANDs, symbol writes, bytes per puzzle):" + " solving the same puzzles, four arms (median over puzzles of each arm's best of {ROUNDS} warm runs, us; ANDs, symbol writes, bytes per puzzle):" ); println!( " {:>4} {:>7} {:>9} {:>9} {:>9} {:>9} {:>7} {:>7} {:>7} {:>8} {:>8}", @@ -1205,12 +1294,24 @@ fn sweep(hot: &Hot, cold: &Cold) -> usize { let (mut and_a, mut and_b, mut writes, mut bytes_a, mut bytes_b, mut solved) = (0, 0, 0, 0, 0, 0); for c in made { - let (a, us) = time_us(|| solve_token(hot, &c.puz, &c.givens).unwrap()); - ta.push(us); - let (d, us) = time_us(|| solve_hybrid(hot, &c.puz, &c.givens).unwrap()); - td.push(us); - let (b, us) = time_us(|| solve_cartesian(hot, &c.puz, &c.givens).unwrap()); - tb.push(us); + // One warm-up call per arm, then ROUNDS rounds in rotated order; + // each arm keeps its best time. A single first-call timing would + // charge whichever arm runs first for the cold populations. + let a = solve_token(hot, &c.puz, &c.givens).unwrap(); + let d = solve_hybrid(hot, &c.puz, &c.givens).unwrap(); + let b = solve_cartesian(hot, &c.puz, &c.givens).unwrap(); + let arms: [Arm; 3] = [solve_token, solve_hybrid, solve_cartesian]; + let mut best = [f64::INFINITY; 3]; + for round in 0..ROUNDS { + for k in 0..3 { + let i = (k + round) % 3; + let (_, us) = time_us(|| arms[i](hot, &c.puz, &c.givens).unwrap()); + best[i] = best[i].min(us); + } + } + ta.push(best[0]); + td.push(best[1]); + tb.push(best[2]); assert_eq!(a.placed, d.placed); assert_eq!(a.placed, b.placed); assert_eq!(a.cand, d.cand); @@ -1250,7 +1351,54 @@ fn sweep(hot: &Hot, cold: &Cold) -> usize { bytes_b / n ); } - largest + per_size +} + +/// Ablate every given of every created puzzle and tally the reactions. +fn ablation_report(hot: &Hot, per_size: &[(usize, Vec)]) { + println!(" cui bono: remove each given, re-run the fixed point and the uniqueness count:"); + println!( + " {:>4} {:>7} {:>10} {:>9} {:>9} {:>9} {:>8} {:>13} {:>10}", + "side", + "givens", + "redundant", + "shortcut", + "silent", + "necessary", + "budget", + "lost placed", + "pop delta" + ); + for (side, made) in per_size { + let mut tally = [0usize; 4]; + let (mut out_of_budget, mut n, mut lost, mut pop) = (0usize, 0usize, 0usize, 0i64); + for c in made { + for a in ablate(hot, &c.puz, &c.givens, 100_000) { + n += 1; + lost += a.lost_placed; + pop += a.popcount_delta; + match a.reaction() { + Some(Reaction::Redundant) => tally[0] += 1, + Some(Reaction::ShortcutOnly) => tally[1] += 1, + Some(Reaction::SilentlyNecessary) => tally[2] += 1, + Some(Reaction::Necessary) => tally[3] += 1, + None => out_of_budget += 1, + } + } + } + if n == 0 { + continue; + } + println!( + " {side:>4} {n:>7} {:>10} {:>9} {:>9} {:>9} {out_of_budget:>8} {:>13.2} {:>10.1}", + tally[0], + tally[1], + tally[2], + tally[3], + lost as f64 / n as f64, + pop as f64 / n as f64 + ); + } } /// About a million claims at one size, folded the shared three ways. @@ -1298,7 +1446,8 @@ fn run_language(name: &str, hot: &Hot, cold: &Cold) { hot.population_bytes() as f64 / 1e6 ); filter_bench(hot); - sweep(hot, cold); + let per_size = sweep(hot, cold); + ablation_report(hot, &per_size); lane_run(hot, 5); } @@ -1340,7 +1489,6 @@ mod tests { cold.vocab.id(w).expect("in the fixture") } - type Arm = fn(&Hot, &Puzzle, &[(u8, WordId)]) -> Result; const ARMS: [(&str, Arm); 3] = [ ("token", solve_token), ("hybrid", solve_hybrid), @@ -1722,6 +1870,54 @@ mod tests { } } + /// Cui bono, on fixtures with known answers: a given the fixed point + /// re-derives is redundant; a given whose loss weakens the fixed point + /// but leaves one fill is a shortcut; a given whose loss admits a second + /// fill is necessary. + #[test] + fn ablation_separates_redundant_shortcut_and_necessary_givens() { + // across cat crossing down ant at across offset 1 / down offset 0 + let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "ant", "dog"])); + let puz = compile(&plus(), Lang::En); + let (cat, ant) = (word(&cold, "cat"), word(&cold, "ant")); + let both = [(0u8, cat), (1u8, ant)]; + let a = ablate(&hot, &puz, &both, 1_000); + // dropping ant: cat still forces it back + assert_eq!(a[1].reaction(), Some(Reaction::Redundant)); + assert_eq!((a[1].lost_placed, a[1].popcount_delta), (0, 0)); + // dropping cat from {cat}: nothing propagates, search still unique + let only = ablate(&hot, &puz, &[(0, cat)], 1_000); + assert_eq!(only[0].reaction(), Some(Reaction::ShortcutOnly)); + assert_eq!(only[0].lost_placed, 1); + assert!(only[0].popcount_delta > 0); + + // with art as a second a-word, ant is no longer forced by cat + let (hot, cold) = build_lexicon(Lang::En, &words(&["cat", "ant", "art", "dog"])); + let (cat, ant) = (word(&cold, "cat"), word(&cold, "ant")); + let a = ablate(&hot, &puz, &[(0, cat), (1, ant)], 1_000); + assert_eq!(a[1].fills, Some(2)); + assert_eq!(a[1].reaction(), Some(Reaction::Necessary)); + assert_eq!(a[1].popcount_delta, 1); + } + + /// On created puzzles the last given added is necessary by construction + /// (the generator stops at the first unique set), so ablation must find + /// at least one necessary given per puzzle. + #[test] + fn every_created_puzzle_has_a_necessary_given() { + let (hot, _) = english(); + for seed in 0..4 { + let c = created(&hot, 5, 120 + seed); + let a = ablate(&hot, &c.puz, &c.givens, 1_000_000); + assert_eq!(a.len(), c.givens.len()); + let last = a.last().unwrap(); + assert!(matches!( + last.reaction(), + Some(Reaction::Necessary | Reaction::SilentlyNecessary) + )); + } + } + /// Three ways, partition, declaration gate: the shared fold, unchanged. #[test] fn the_shared_fold_counts_the_crossword_lane_three_ways() { From 32a7dff8a8e8c2d31b76d8359d696f4fcbda89c9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 17:54:39 +0000 Subject: [PATCH 4/8] D-PUZZLE-0 step 3 board: entry, status row, README; #1382/#1384 records Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .claude/board/LATEST_STATE.md | 5 + .claude/board/PR_ARC_INVENTORY.md | 13 ++ .claude/board/STATUS_BOARD.md | 2 +- ...-07-puzzle-0-crossword-mask-propagation.md | 131 ++++++++++++++++++ .claude/board/entries/README.md | 3 +- .../examples/README.md | 16 +++ 6 files changed, 168 insertions(+), 2 deletions(-) create mode 100644 .claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index b1ccc1ff9..eebd0b74f 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,8 @@ +## 2026-10-07 — lance-graph #1382 and #1384 MERGED; D-PUZZLE-0 step 3 on branch + +- #1382 (`bccc46e`): Sudoku population-fold probe. #1384 (`23e8d93`): shared fold module + crossword probes (steps 2/2b). No contract change in either. +- Step 3 (branch `claude/epistemicstate5-ce64-bits-3oe0dq`, unmerged): `crossword_mask_propagation_probe.rs`, four representations of one fold; `deepnsm-v2` is now a dev-dependency of `cognitive-shader-driver`. No contract type added: `MooreSymbol8` is probe-local. + ## 2026-10-07 — lance-graph #1381 MERGED (`7334ae9`) The D-EPI-CANON-0 / D-EPI-MIG-0 section below is now on `main`, together with diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index bdd89528f..9a9ab054e 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -1,3 +1,16 @@ +## 2026-10-07 — lance-graph PR #1384 (merged `23e8d93`, branch `claude/epistemicstate5-ce64-bits-3oe0dq`, head `94499c3`, 9 commits) — D-PUZZLE-0 steps 2/2b: crosswords on the Sudoku population fold + +- **Added:** `examples/shared/population_fold.rs` (the domain-free half: propagation reading, five `facts_population` questions, three folds, declaration gate; a fence test bans domain words); `crossword_population_fold_probe.rs` (synthetic 3-letter alphabet); `crossword_real_words_probe.rs` (DeepNSM v1 4096 + `academic_20k`, DeReKo top 20k from `DEREKO_PATH`; NYT-rule grids; Down-and-Across answers from `XWORD_ANSWERS_DIR`). +- **Locked (test-pinned):** Sudoku counts unchanged after the refactor (1,000,066 edges); every crossword count equal three ways; disable runs red. +- **Deferred / open:** the 2b NYT-grid and crossword-answer modes (`94499c3`) were never run end to end; step 3 supersedes the grid generator. +- **Confidence:** high for the shared-fold claim; the 2b real-word numbers are one run each. + +## 2026-10-07 — lance-graph PR #1382 (merged `bccc46e`, branch `claude/epistemicstate5-ce64-bits-3oe0dq`, head `55f2d0f`, 3 commits) — D-PUZZLE-0 step 1: Sudoku at scale on the EpistemicState5 population fold + +- **Added:** `sudoku_population_fold_probe.rs`: ~1M claim edges in the propagation reading (GIVEN 20 / FORCED 21 / ENTAILED 22 / CANDIDATE 4); five questions as `facts_population` masks, counted by filter, histogram and per-edge decode. +- **Locked (test-pinned):** all counts equal three ways and to the generator's counters; the probe is a registered test target (Codex P2). +- **Confidence:** high; timings one machine. + ## 2026-10-07 — lance-graph PR #1381 (merged `7334ae9`, branch `claude/epistemicstate5-ce64-bits-3oe0dq`, head `0ba5d24`, 17 commits) — CE64 bits 59..63 = `EpistemicState5 = Topology2 × Certification3`; facts populations; legacy deprojection - **Added:** `lance_graph_contract::epistemic_state5` — `Topology2`, `Certification3`, per-factor facts, `EpistemicState5` (`raw5 = topology | certification << 2`, 24 meaningful / 8 reserved), `Epi5Reading`, `Epi5Declarations::project_state5`, `facts_population` / `Population` / `bit()`; `ClassView::epistemic_reading`; `CausalEdge64::{epistemic_raw5, with_epistemic_raw5}`, `EPISTEMIC_MASK`. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 261664c94..83ebe6172 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -48,7 +48,7 @@ Plan: `.claude/plans/2026-10-06-global-sudoku-replayable-orchestration-v1.md`. O | **D-EPI-CANON-0** | Canonical joint `EpistemicState5` reading of bits 59..63 in the contract | Shipped (#1381) | `contract::epistemic_state5`: Cartesian `Topology2 × Certification3`, `raw5 = t \| c << 2`, 24 meaningful / 8 reserved; per-factor facts; declarations + `project_state5`; `CausalEdge64::{epistemic_raw5, with_epistemic_raw5}`; non-declared-reading writers deprecated | | **D-EPI-POP-0** | Facts → 32-state population: the missing operator between one edge's coordinate (u5) and a semantic condition over all coordinates (u32) | Shipped (#1381) | `contract::epistemic_state5::{Population, facts_population, EpistemicState5::bit}`: `const fn`, no DSL / query object / planner. Tests pin exact sets (`CAUSES` = {20..23}, `IND_UNKNOWN \| RELATED` = {10,14,18,22}, `IND_KNOWN \| SUPPORTS` = {13,17,21}, `RELATED` = 8..23), operator == state enumeration over all 2048 requirements with no reserved code, conjunction == intersection; 2 disable runs red (any-of semantics; reserved read as fact-less) | | **D-EPI-LEGACY-DEPROJECT-0** | Deproject the remaining `ReasoningBand` / CE64-`TrustTexture` consumers: each goes to an exact coordinate, a `facts_population(..)`, MUL, or another subsystem (e.g. counterfactual context) — never a 1:1 replacement enum | Shipped (#1381) | `entries/2026-10-07-epi-legacy-deprojection.md`: 6 ReasoningBand caller sites (1 certification question, 1 non-interference witness → both exact coordinates; 2 context/obsolete → compat; 2 refusal witnesses), 0 production CE64 `TrustTexture` consumers, 4 `TrustTexture` types with no bridge; `EdgeRole::band` removed; hinge + V3 lift migrated; 4 code loops → `facts_population` | -| **D-PUZZLE-0** | Sudoku / crossword / chess on the same population algebra under domain laws | In progress | step 1 Sudoku shipped (#1382). Step 2 crossword (`crossword_population_fold_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-same-algebra.md`): domain-free half moved to `examples/shared/population_fold.rs` (fence test, Sudoku counts unchanged); 1,000,052 crossword edges, counts equal 3 ways; filter 0.30–0.43 ns/edge in both domains. Step 2b real words (`crossword_real_words_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-real-words.md`): English = DeepNSM v1 4096 + academic_20k (committed), German = DeReKo top 20k only from `DEREKO_PATH` (CC BY-NC, nothing committed); ~1.0M edges each, counts equal 3 ways. Next: chess in `stockfish-rs` (GPL) — how it reaches the shared half is open | +| **D-PUZZLE-0** | Sudoku / crossword / chess on the same population algebra under domain laws | In progress | step 1 Sudoku shipped (#1382). Step 2 crossword (`crossword_population_fold_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-same-algebra.md`): domain-free half moved to `examples/shared/population_fold.rs` (fence test, Sudoku counts unchanged); 1,000,052 crossword edges, counts equal 3 ways; filter 0.30–0.43 ns/edge in both domains. Step 2b real words (`crossword_real_words_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-real-words.md`): English = DeepNSM v1 4096 + academic_20k (committed), German = DeReKo top 20k only from `DEREKO_PATH` (CC BY-NC, nothing committed); ~1.0M edges each, counts equal 3 ways. Steps 2/2b merged (#1384). Step 3 (`crossword_mask_propagation_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-mask-propagation.md`): Morton cell + `(slot:offset)` lanes, DeepNSM-v2 `WordId` masks, `MooreSymbol8` letters; four arms on identical puzzles (token 2.6 µs = hybrid, Cartesian 4.2, literal ~5,200 at 5×5); creation fails beyond 7×7 with the 18–20k vocabularies; cui-bono ablation of givens. Next: chess in `stockfish-rs` (GPL) — how it reaches the shared half is open | | **D-EPI-MIG-0** | Migrate readers/writers to the canonical reading; #1378 → conformance | Shipped (#1381) | `entries/2026-10-07-epi-mig-0.md`: affordance law (per-factor compile) + P7a (identity packing) + 6 probes + dismech + V3 lift; conformance census 24/0/8 (was 3/5/16); 3 historical-band probes cannot migrate without invention | | **D-EPI-CONFLICT-0** | (superseded by D-EPI-MIG-0: probe converted to conformance) Pin the disagreement between the D-ACR-7-declared P7a band and the affordance law on CE64 bits 59..63 | Superseded | **Current:** the same probe is the conformance test, census 24 agree / 0 disagree / 8 reserved under the Cartesian V1 (D-EPI-MIG-0). **HISTORICAL — the pre-Cartesian pins below no longer describe the code:** `cognitive-shader-driver/examples/epistemic_reading_conflict_probe.rs`: contract-projected `Causes` reads as affordance code 20 `Related` (topology 0) or refuses (1..3), `COUNTERFACTUAL_PROBE` never eligible; `CAUSES` codes 1 and 17 sit on P7a `Open` / `CausalCandidate`; codes 25, 30 on P7a reserved bands; census over 24 cells = 3 agree, 5 disagree, 16 refuse; the affordance law measures where the contract refuses (undeclared / band `Absent`). Both readings run from shared code (`shared/affordance_law.rs`, and P7a's reading moved unchanged into `shared/certification_reading.rs`, used by `relational_certification_probe` too), never copied. Pins of current state, to become agreement tests when the issue closes (`ISS-EPISTEMIC-READINGS-DISAGREE-ON-BITS-61-63`); 5 tests, 6 disable runs red (3 on the affordance codebook, 3 on P7a's: `Contributes` entailment, a reserved code admitted, `Causes` recoded) | | **D-MBX-CUTOVER-0** | ShaderDriver on MailboxSoA without BindSpace | In PR | `entries/2026-10-06-shaderdriver-bindspace-cutover.md`: every ShaderDriver → BindSpace edge classified (production state / shared resource / fallback / test oracle / fossil / dead). In `run()` the only mailbox-mode BindSpace read was the ontology; outside it, metrics and the builder read the singleton and `backing()` silently fell back for any mailbox not under id 0. Cutover: optional BindSpace, explicit mailbox selection with no fallback, ontology on the driver, metrics from the selected substrate; no temporal change. `tests/mailbox_cutover.rs` 8 tests, 8 disable runs red | diff --git a/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md b/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md new file mode 100644 index 000000000..570666508 --- /dev/null +++ b/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md @@ -0,0 +1,131 @@ +# 2026-10-07 — D-PUZZLE-0-XWORD-MASK (D-PUZZLE-0 step 3): crossword propagation as Cartesian-addressed masking + +**Probe:** `cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs`, +over `examples/shared/population_fold.rs` (unchanged). +**Status:** MEASURED (one machine, release, `avx2=true avx512f=false`), +TEST-PINNED (19 tests, 11 disable runs red). + +## Question + +Once a crossword is compiled, is propagation the Sudoku operation +(`mask → intersect → popcount → promote → expose → adjacent masks → repeat`), +with nothing crossword-shaped on the hot path? And which physical +representation of that one logic is best, on identical puzzles? + +## Inventory (read-only, before any code) + +| need | existing facility | verdict | +|---|---|---| +| cell (WHERE) | `Morton8x8::from_xy(col,row)`, u16 | used | +| slot position | `FacetTier{hi: slot, lo: offset}.as_u16()` | used | +| crossing | no stored relation (`morton8x8.rs`: "no stored edge"); equality of cell codes, compiled once into `cross[slot:offset]` and `occupant[cell]` lanes | used | +| word (WHAT) | DeepNSM-v2 `PaletteVocab::from_frequency_ranked`, `WordId = u16` | used | +| masks | `ndarray::simd::{mask_and_assign, popcount_batch_u64}` (already a dependency) | used | +| `MaskOp::Gather` (#1383) | foreign-key semijoin | not needed: elimination is an AND within one WordId space | +| letter | no canonical Moore symbol space: every Moore in the repo is a direction table, a grid-edge validity byte or a palette-law operand | declared probe-local `MooreSymbol8` | + +No new primitive and no contract change. `deepnsm-v2` became a dev-dependency +of the driver (its `guid-v3-tail` is already a default contract feature). + +## Encodings + +- `MooreSymbol8` (u8): `0` unknown, `1..=26` a–z, `27` ä, `28` ö, `29` ü, + `30` ß, `31..=255` reserved. Accented Latin letters fold to the base letter + (`cliché` is spelled `cliche`); ä ö ü ß are letters of their own. A word with + any other character keeps its `WordId` and is not a crossword word. + Promoting this reading to a canonical value tenant is a contract decision, + not taken here. +- English vocabulary: `academic_20k.csv`, lowercased, through + `from_frequency_ranked`, exactly as `deepnsm-v2/examples/genre_shapes.rs` + builds it: **18,555** ids (`vocab.rs` quotes 18,559 distinct surfaces; four + collapse when lowercased), 17,548 crossword words. +- German vocabulary: no German vocabulary is committed and DeepNSM-v2 builds + none. With `DEREKO_PATH`, the 20,000 most frequent DeReKo-2014 forms + (lowercased, `NE` and non-alphabetic forms dropped, frequencies summed), + 19,835 crossword words. CC BY-NC 3.0; counts only here, nothing derived + committed. +- Candidates: `P(L, i, x)` bitsets over a language's `WordId`s; a slot starts + at `P_all(L)`. 16.4 MB (EN) / 17.7 MB (DE) of populations for L = 3..=21. +- State: CE64 bits 59..63 only, the shared GIVEN 20 / FORCED 21 / ENTAILED 22 + / CANDIDATE 4. Slot and word sit in separate content lanes. + +## The four arms (same puzzles, same givens; equal fixed points asserted) + +| arm | content | addressing | propagation | +|---|---|---|---| +| A token | `WordId` masks | `cross` | letter at offset `i` ANDs `P(L,j,x)` into the crossing slot | +| B Cartesian | `MooreSymbol8` board | `occupant` | symbols written; affected slots recompute from all their cells | +| C literal | strings | cell array | every unplaced slot re-scans vocabulary strings each round (steps 2/2b) | +| D hybrid | `WordId` masks + board | `occupant` | each new symbol ANDs one `P` into the cell's other slot | + +## Measured + +Filter, 2000 patterns (length + 1–3 revealed symbols): + +| | EN | DE | +|---|---|---| +| mask AND chain + popcount | 670 ns/pattern, 9.7 ns/claim | 683 ns/pattern, 7.6 ns/claim | +| direct spell-lane scan | 41,769 ns/pattern | 39,585 ns/pattern | + +Solve, median over puzzles of each arm's best of 5 warm runs (µs): + +| | A token | D hybrid | B Cartesian | C literal | ANDs A / B | bytes A / B | +|---|---|---|---|---|---|---| +| EN 5×5 (30) | 2.6 | 2.6 | 4.2 | 5,229 | 10.7 / 19.1 | 23,220 / 2,382 | +| EN 7×7 (5) | 3.0 | 3.1 | 5.0 | 5,753 | 15.6 / 33.6 | 42,724 / 2,417 | +| DE 5×5 (30) | 2.6 | 2.7 | 4.5 | 6,017 | 10.3 / 17.1 | 25,060 / 2,566 | +| DE 7×7 (2) | 5.5 | 5.6 | 9.1 | 8,689 | 14.0 / 20.0 | 42,602 / 2,599 | + +- A and D cost the same: routing letters through the Morton board adds no + measurable time. B does about twice the ANDs (it recomputes) and is ~1.6× + slower, but holds ~10× fewer bytes (one board instead of a full WordId + mask per slot). C, the string method, is ~2,000× slower than A. +- The first run timed each arm once, cold, A first, and showed A at 16.6 µs + against D at 6.9 µs with an equal AND count. That was cache warm-up, not + representation; the table above is the corrected method. +- The shared fold on a 5×5 lane (~1.0M claims, EN and DE): every count equal + three ways; filter 0.28–0.35 ns/edge, histogram ~2.0, decode oracle ~3.9 — + the same costs as Sudoku and steps 2/2b. + +Creation (budget 100,000 tried words per step, 30 s per size), NYT-rule grids: + +| side | EN made / attempts | DE made / attempts | +|---|---|---| +| 5 | 30 / 30 | 30 / 30 | +| 7 | 5 / 175 | 2 / 163 | +| 9–21 | 0 (all misses are fill, none uniqueness) | 0 (same) | + +Solving is not the limit at any size measured; **creation is**. With 18–20k +general vocabularies a random NYT-dense grid of side ≥ 9 is not filled within +the budget. The likely cause is vocabulary density (crosswords are built from +answer lists with many short fill words); not measured here. + +Cui bono, each given removed in turn (fixed point re-run + uniqueness count): + +| | givens | Redundant | ShortcutOnly | SilentlyNecessary | Necessary | out of budget | +|---|---|---|---|---|---|---| +| EN 5×5 | 118 | 32 | 37 | 0 | 49 | 0 | +| EN 7×7 | 51 | 39 | 6 | 0 | 6 | 0 | +| DE 5×5 | 98 | 21 | 29 | 0 | 48 | 0 | +| DE 7×7 | 7 | 0 | 5 | 0 | 1 | 1 | + +`SilentlyNecessary` (a given propagation never used but search needs) did not +occur in the 273 ablations that finished within budget. The readout nominates; it does not prove cause: a +given necessary in one set can be replaced by another set. + +## Falsifiers (disable-verified red) + +Arm A ignores crossings (4 tests); letters a/b swapped (codebook, decode); +index drops offset 0 (5, incl. index-vs-scan); no promotion at popcount 1 (2); +first crossing only (every-crossing test); zero candidates not a contradiction; +language gate removed; word bits written into the edge (CE64 test); crossings +rediscovered from the cell lane (compiled-lanes test); ablation ignores +uniqueness, and ablation removes nothing (both ablation tests). + +## Open + +- Creation beyond 7×7 with these vocabularies. +- Whether `MooreSymbol8` (and a `MooreLearn8` companion) become canonical + value tenants: a contract decision. +- The step-2b NYT-grid + crossword-answer mode (#1384, `94499c3`) was never run + end to end; this probe supersedes its grid generator. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 99adcc008..2bfd335e0 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -249 entries, 2026-08-06 .. 2026-10-07. +252 entries, 2026-08-06 .. 2026-10-07. | date | entry id | finding | file | |---|---|---|---| @@ -34,6 +34,7 @@ exactly one of them, never both. | 2026-10-07 | `D-PUZZLE-0` | | [2026-10-07-puzzle-0-sudoku-population-fold.md](2026-10-07-puzzle-0-sudoku-population-fold.md) | | 2026-10-07 | `D-PUZZLE-0-XWORD` | | [2026-10-07-puzzle-0-crossword-same-algebra.md](2026-10-07-puzzle-0-crossword-same-algebra.md) | | 2026-10-07 | `D-PUZZLE-0-XWORD-REAL` | | [2026-10-07-puzzle-0-crossword-real-words.md](2026-10-07-puzzle-0-crossword-real-words.md) | +| 2026-10-07 | `D-PUZZLE-0-XWORD-MASK` | | [2026-10-07-puzzle-0-crossword-mask-propagation.md](2026-10-07-puzzle-0-crossword-mask-propagation.md) | | 2026-10-07 | `gated-gather-bit-schedule` | Production gather is 3.8× slow from a per-row branch; bit-gated visitation wins or ties at every density; ordinal vectors never win (R1 holds) | [2026-10-07-gated-gather-bit-schedule.md](2026-10-07-gated-gather-bit-schedule.md) | | 2026-10-07 | `D-EPI-MIG-INVENTORY` | | [2026-10-07-epi-mig-inventory.md](2026-10-07-epi-mig-inventory.md) | | 2026-10-07 | `D-EPI-CANON-0` | | [2026-10-07-epi-mig-0.md](2026-10-07-epi-mig-0.md) | diff --git a/crates/cognitive-shader-driver/examples/README.md b/crates/cognitive-shader-driver/examples/README.md index 8caa26ec8..43d961211 100644 --- a/crates/cognitive-shader-driver/examples/README.md +++ b/crates/cognitive-shader-driver/examples/README.md @@ -442,3 +442,19 @@ edges per language; every count agrees three ways. Board: cargo run --release -p cognitive-shader-driver --example crossword_real_words_probe DEREKO_PATH=/path/DeReKo-2014-II-MainArchive-STT.100000.freq cargo run --release -p cognitive-shader-driver --example crossword_real_words_probe ``` + +## crossword_mask_propagation_probe.rs + +D-PUZZLE-0 step 3. Is crossword propagation the Sudoku operation, and which +representation of it is best? Cells are `Morton8x8` codes, slot positions +`(slot:offset)` tiles, crossings two compiled u16 lanes, letters a declared +`MooreSymbol8` byte, candidates DeepNSM-v2 `WordId` bitsets. Four arms (token, +Cartesian board, literal strings, hybrid) solve the same created puzzles to the +same fixed point; the run also ablates each given (cui bono) and folds a ~1M +claim lane the shared three ways. German needs `DEREKO_PATH`. Board: +`.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md`. + +``` +cargo run --release -p cognitive-shader-driver --example crossword_mask_propagation_probe +``` + From 2e54d03f5359e44d2835d8d3bc4e1812c750d0eb Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 18:04:10 +0000 Subject: [PATCH 5/8] D-PUZZLE-0 step 3: time the literal arm under the same protocol Codex (P1) on #1387: arm C was timed once with no warm-up, and its timing included rebuilding every vocabulary entry into chars, while A/B/D got a warm-up and the best of 5 rotated runs with prebuilt populations. The literal word list is now built once per language (Literal::new, outside timing) and arm C joins the warm, rotated rounds for boards up to 9x9. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .../crossword_mask_propagation_probe.rs | 72 +++++++++++++------ 1 file changed, 51 insertions(+), 21 deletions(-) diff --git a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs index 9f2e999f1..37fad73e9 100644 --- a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs +++ b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs @@ -826,27 +826,43 @@ fn solve_cartesian(hot: &Hot, puz: &Puzzle, givens: &[(u8, WordId)]) -> Result)>, + index: Vec>, +} + +impl Literal { + fn new(cold: &Cold) -> Self { + let mut words = Vec::new(); + let mut index = vec![None; cold.vocab.len()]; + for id in 0..cold.vocab.len() as WordId { + let w = cold.word(id); + if spelling(w).is_some() { + index[id as usize] = Some(words.len()); + words.push((id, w.chars().map(fold).collect())); + } + } + Self { words, index } + } +} + /// Arm C, steps 2/2b's method: letters on a cell array, every unplaced slot /// re-scanned from the vocabulary STRINGS each round, singles placed until /// nothing changes. `None` on contradiction. (Slot-indexed on purpose: it /// mirrors the loop it stands in for.) #[allow(clippy::needless_range_loop)] -fn solve_literal(cold: &Cold, puz: &Puzzle, givens: &[(u8, WordId)]) -> Option> { - let words: Vec<(WordId, Vec)> = (0..cold.vocab.len() as WordId) - .filter_map(|id| { - let w = cold.word(id); - spelling(w)?; - Some((id, w.chars().map(fold).collect())) - }) - .collect(); +fn solve_literal(lit: &Literal, puz: &Puzzle, givens: &[(u8, WordId)]) -> Option> { + let words = &lit.words; let mut board: HashMap = HashMap::new(); let mut placed = vec![UNSET; puz.slots()]; let put = |s: usize, w: &[char], board: &mut HashMap| -> bool { (0..w.len()).all(|o| *board.entry(puz.cell[s * STRIDE + o]).or_insert(w[o]) == w[o]) }; for &(s, w) in givens { - let chars: Vec = cold.word(w).chars().map(fold).collect(); - if !put(s as usize, &chars, &mut board) { + let chars = &lit.words[lit.index[w as usize]?].1; + if !put(s as usize, chars, &mut board) { return None; } placed[s as usize] = w; @@ -1235,6 +1251,7 @@ const ROUNDS: usize = 5; /// Size sweep: what can be created, and every arm's solve on the same puzzles. fn sweep(hot: &Hot, cold: &Cold) -> Vec<(usize, Vec)> { + let lit = Literal::new(cold); println!(" creation by board size (budget 100,000 tried words per step, 30 s per size):"); println!( " {:>4} {:>8} {:>9} {:>11} {:>6} {:>6} {:>10}", @@ -1297,31 +1314,41 @@ fn sweep(hot: &Hot, cold: &Cold) -> Vec<(usize, Vec)> { // One warm-up call per arm, then ROUNDS rounds in rotated order; // each arm keeps its best time. A single first-call timing would // charge whichever arm runs first for the cold populations. + // The literal arm (C) joins the same protocol for boards up to 9x9; + // its word list is prebuilt once per language, outside timing. let a = solve_token(hot, &c.puz, &c.givens).unwrap(); let d = solve_hybrid(hot, &c.puz, &c.givens).unwrap(); let b = solve_cartesian(hot, &c.puz, &c.givens).unwrap(); + let with_literal = *side <= 9; + if with_literal { + let l = solve_literal(&lit, &c.puz, &c.givens).unwrap(); + assert_eq!(l, a.placed, "literal fixed point == mask fixed point"); + } let arms: [Arm; 3] = [solve_token, solve_hybrid, solve_cartesian]; - let mut best = [f64::INFINITY; 3]; + let n_arms = if with_literal { 4 } else { 3 }; + let mut best = [f64::INFINITY; 4]; for round in 0..ROUNDS { - for k in 0..3 { - let i = (k + round) % 3; - let (_, us) = time_us(|| arms[i](hot, &c.puz, &c.givens).unwrap()); + for k in 0..n_arms { + let i = (k + round) % n_arms; + let (_, us) = if i < 3 { + time_us(|| arms[i](hot, &c.puz, &c.givens).unwrap().placed) + } else { + time_us(|| solve_literal(&lit, &c.puz, &c.givens).unwrap()) + }; best[i] = best[i].min(us); } } ta.push(best[0]); td.push(best[1]); tb.push(best[2]); + if with_literal { + tc.push(best[3]); + } assert_eq!(a.placed, d.placed); assert_eq!(a.placed, b.placed); assert_eq!(a.cand, d.cand); assert_eq!(a.cand, b.cand); assert!(cell_consistent(hot, &c.puz, &a.placed)); - if *side <= 9 { - let (lit, us) = time_us(|| solve_literal(cold, &c.puz, &c.givens).unwrap()); - tc.push(us); - assert_eq!(lit, a.placed, "literal fixed point == mask fixed point"); - } and_a += a.ands; and_b += b.ands; writes += d.writes; @@ -1767,7 +1794,10 @@ mod tests { assert_eq!(a.placed, d.placed); assert_eq!(a.cand, b.cand); assert_eq!(a.cand, d.cand); - assert_eq!(solve_literal(&cold, &c.puz, &c.givens).unwrap(), a.placed); + assert_eq!( + solve_literal(&Literal::new(&cold), &c.puz, &c.givens).unwrap(), + a.placed + ); assert!(cell_consistent(&hot, &c.puz, &c.solution)); let mut budget = 1_000_000; assert_eq!( @@ -1859,7 +1889,7 @@ mod tests { fn the_hot_path_needs_no_strings() { let (hot, cold) = english(); let c = created(&hot, 5, 13); - let expect = solve_literal(&cold, &c.puz, &c.givens).unwrap(); + let expect = solve_literal(&Literal::new(&cold), &c.puz, &c.givens).unwrap(); drop(cold); for (name, arm) in ARMS { assert_eq!( From 2bdb50b89d6fdd62ccdfa43c6bda74ffb8dffe86 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 18:05:26 +0000 Subject: [PATCH 6/8] D-PUZZLE-0 step 3: fills never repeat a word CodeRabbit on #1387: random_fill and count_fills took every bit of a slot's mask, so a created fill could repeat an entry and the uniqueness count counted such fills. Both searches now skip words already placed, prune a duplicate forced by propagation, and accept a completed fill only without repeats, so the generator and the uniqueness oracle agree. Solving (the fixed point) is unchanged: a forced word is always the solution's word. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .../crossword_mask_propagation_probe.rs | 48 +++++++++++++++++-- 1 file changed, 43 insertions(+), 5 deletions(-) diff --git a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs index 37fad73e9..cae4103d2 100644 --- a/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs +++ b/crates/cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs @@ -914,6 +914,17 @@ enum Miss { Unique, } +/// Crossword fills never repeat an entry: no two placed slots hold the same +/// word. Both creation searches apply this rule, so the generator and the +/// uniqueness count agree on what a fill is. +fn no_repeats(placed: &[WordId]) -> bool { + let mut seen: Vec = placed.iter().copied().filter(|&w| w != UNSET).collect(); + let n = seen.len(); + seen.sort_unstable(); + seen.dedup(); + seen.len() == n +} + /// Depth-first fill over arm A's masks, most-constrained slot first, random /// order within it. `budget` counts tried words. Creation only. fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Option> { @@ -922,9 +933,10 @@ fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Op .filter(|&s| st.placed[s] == UNSET) .min_by_key(|&s| st.count(s)) else { - return Some(st); + return no_repeats(&st.placed).then_some(st); }; let mut cs = bits(st.mask(s)); + cs.retain(|w| !st.placed.contains(w)); rng.shuffle(&mut cs); for w in cs { if *budget == 0 { @@ -934,7 +946,7 @@ fn random_fill(hot: &Hot, puz: &Puzzle, rng: &mut Rng, budget: &mut usize) -> Op let mut next = st.clone(); next.place(s, w); let mut q = vec![s as u8]; - if propagate_token(hot, puz, &mut next, &mut q).is_ok() { + if propagate_token(hot, puz, &mut next, &mut q).is_ok() && no_repeats(&next.placed) { if let Some(done) = go(hot, puz, next, rng, budget) { return Some(done); } @@ -961,10 +973,13 @@ fn count_fills( .filter(|&s| st.placed[s] == UNSET) .min_by_key(|&s| st.count(s)) else { - return Some(1); + return Some(usize::from(no_repeats(&st.placed))); }; let mut n = 0; - for w in bits(st.mask(s)) { + for w in bits(st.mask(s)) + .into_iter() + .filter(|w| !st.placed.contains(w)) + { if *budget == 0 { return None; } @@ -972,7 +987,7 @@ fn count_fills( let mut next = st.clone(); next.place(s, w); let mut q = vec![s as u8]; - if propagate_token(hot, puz, &mut next, &mut q).is_ok() { + if propagate_token(hot, puz, &mut next, &mut q).is_ok() && no_repeats(&next.placed) { n += go(hot, puz, next, cap - n, budget)?; if n >= cap { break; @@ -1930,6 +1945,29 @@ mod tests { assert_eq!(a[1].popcount_delta, 1); } + /// A fill may not repeat a word, in the generator and in the uniqueness + /// count alike: across `bab` with down `aaa` is the one legal fill; + /// `aaa`/`aaa` crosses correctly but repeats the entry. + #[test] + fn fills_never_repeat_a_word() { + let (hot, cold) = build_lexicon(Lang::En, &words(&["aaa", "bab"])); + let puz = compile(&plus(), Lang::En); + let mut budget = 1_000; + assert_eq!(count_fills(&hot, &puz, &[], 3, &mut budget), Some(1)); + let mut rng = Rng(1); + for _ in 0..20 { + let mut b = 1_000; + let fill = random_fill(&hot, &puz, &mut rng, &mut b).unwrap(); + assert_eq!(fill, [word(&cold, "bab"), word(&cold, "aaa")]); + } + // only the repeated fill exists: no fill at all + let (hot, _) = build_lexicon(Lang::En, &words(&["aaa"])); + let mut budget = 1_000; + assert_eq!(count_fills(&hot, &puz, &[], 3, &mut budget), Some(0)); + let mut b = 1_000; + assert!(random_fill(&hot, &puz, &mut Rng(2), &mut b).is_none()); + } + /// On created puzzles the last given added is necessary by construction /// (the generator stops at the first unique set), so ablation must find /// at least one necessary given per puzzle. From 0340797d93512fe57dbec65fb4af7f1118bddc65 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 18:07:13 +0000 Subject: [PATCH 7/8] Board: regenerate the entries index after merging main (#1386 entry) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .claude/board/entries/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 00c553138..1ff7aaeda 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -252 entries, 2026-08-06 .. 2026-10-07. +253 entries, 2026-08-06 .. 2026-10-07. | date | entry id | finding | file | |---|---|---|---| From 9f8b5e4bc4e6a5bae45b07ce8c0f9e70bf83ab1c Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 7 Oct 2026 18:14:49 +0000 Subject: [PATCH 8/8] Board: final step-3 numbers (no-repeat fills stop creation at 5x5; literal arm ~1 ms) The final run, with both review fixes, measured: - the literal arm, timed like the others (prebuilt word list, best of 5), at ~1,000 us at 5x5, not ~5,200 us; - no 7x7 board made once a fill may not repeat a word (before: EN 5, DE 2, all of which reused a word); - the falsifier count is 13 disable runs, now listed in full. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0177nSSyQaXqbNuGR1vMXXN6 --- .claude/board/STATUS_BOARD.md | 2 +- ...-07-puzzle-0-crossword-mask-propagation.md | 69 +++++++++++-------- 2 files changed, 40 insertions(+), 31 deletions(-) diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 83ebe6172..20a58ef56 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -48,7 +48,7 @@ Plan: `.claude/plans/2026-10-06-global-sudoku-replayable-orchestration-v1.md`. O | **D-EPI-CANON-0** | Canonical joint `EpistemicState5` reading of bits 59..63 in the contract | Shipped (#1381) | `contract::epistemic_state5`: Cartesian `Topology2 × Certification3`, `raw5 = t \| c << 2`, 24 meaningful / 8 reserved; per-factor facts; declarations + `project_state5`; `CausalEdge64::{epistemic_raw5, with_epistemic_raw5}`; non-declared-reading writers deprecated | | **D-EPI-POP-0** | Facts → 32-state population: the missing operator between one edge's coordinate (u5) and a semantic condition over all coordinates (u32) | Shipped (#1381) | `contract::epistemic_state5::{Population, facts_population, EpistemicState5::bit}`: `const fn`, no DSL / query object / planner. Tests pin exact sets (`CAUSES` = {20..23}, `IND_UNKNOWN \| RELATED` = {10,14,18,22}, `IND_KNOWN \| SUPPORTS` = {13,17,21}, `RELATED` = 8..23), operator == state enumeration over all 2048 requirements with no reserved code, conjunction == intersection; 2 disable runs red (any-of semantics; reserved read as fact-less) | | **D-EPI-LEGACY-DEPROJECT-0** | Deproject the remaining `ReasoningBand` / CE64-`TrustTexture` consumers: each goes to an exact coordinate, a `facts_population(..)`, MUL, or another subsystem (e.g. counterfactual context) — never a 1:1 replacement enum | Shipped (#1381) | `entries/2026-10-07-epi-legacy-deprojection.md`: 6 ReasoningBand caller sites (1 certification question, 1 non-interference witness → both exact coordinates; 2 context/obsolete → compat; 2 refusal witnesses), 0 production CE64 `TrustTexture` consumers, 4 `TrustTexture` types with no bridge; `EdgeRole::band` removed; hinge + V3 lift migrated; 4 code loops → `facts_population` | -| **D-PUZZLE-0** | Sudoku / crossword / chess on the same population algebra under domain laws | In progress | step 1 Sudoku shipped (#1382). Step 2 crossword (`crossword_population_fold_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-same-algebra.md`): domain-free half moved to `examples/shared/population_fold.rs` (fence test, Sudoku counts unchanged); 1,000,052 crossword edges, counts equal 3 ways; filter 0.30–0.43 ns/edge in both domains. Step 2b real words (`crossword_real_words_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-real-words.md`): English = DeepNSM v1 4096 + academic_20k (committed), German = DeReKo top 20k only from `DEREKO_PATH` (CC BY-NC, nothing committed); ~1.0M edges each, counts equal 3 ways. Steps 2/2b merged (#1384). Step 3 (`crossword_mask_propagation_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-mask-propagation.md`): Morton cell + `(slot:offset)` lanes, DeepNSM-v2 `WordId` masks, `MooreSymbol8` letters; four arms on identical puzzles (token 2.6 µs = hybrid, Cartesian 4.2, literal ~5,200 at 5×5); creation fails beyond 7×7 with the 18–20k vocabularies; cui-bono ablation of givens. Next: chess in `stockfish-rs` (GPL) — how it reaches the shared half is open | +| **D-PUZZLE-0** | Sudoku / crossword / chess on the same population algebra under domain laws | In progress | step 1 Sudoku shipped (#1382). Step 2 crossword (`crossword_population_fold_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-same-algebra.md`): domain-free half moved to `examples/shared/population_fold.rs` (fence test, Sudoku counts unchanged); 1,000,052 crossword edges, counts equal 3 ways; filter 0.30–0.43 ns/edge in both domains. Step 2b real words (`crossword_real_words_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-real-words.md`): English = DeepNSM v1 4096 + academic_20k (committed), German = DeReKo top 20k only from `DEREKO_PATH` (CC BY-NC, nothing committed); ~1.0M edges each, counts equal 3 ways. Steps 2/2b merged (#1384). Step 3 (`crossword_mask_propagation_probe.rs`, `entries/2026-10-07-puzzle-0-crossword-mask-propagation.md`): Morton cell + `(slot:offset)` lanes, DeepNSM-v2 `WordId` masks, `MooreSymbol8` letters; four arms on identical puzzles (EN 5×5: token 3.0 µs ≈ hybrid 2.9, Cartesian 5.1, literal ~1,045); with no repeated words, creation stops at 5×5 with the 18–20k vocabularies; cui-bono ablation of givens. Next: chess in `stockfish-rs` (GPL) — how it reaches the shared half is open | | **D-EPI-MIG-0** | Migrate readers/writers to the canonical reading; #1378 → conformance | Shipped (#1381) | `entries/2026-10-07-epi-mig-0.md`: affordance law (per-factor compile) + P7a (identity packing) + 6 probes + dismech + V3 lift; conformance census 24/0/8 (was 3/5/16); 3 historical-band probes cannot migrate without invention | | **D-EPI-CONFLICT-0** | (superseded by D-EPI-MIG-0: probe converted to conformance) Pin the disagreement between the D-ACR-7-declared P7a band and the affordance law on CE64 bits 59..63 | Superseded | **Current:** the same probe is the conformance test, census 24 agree / 0 disagree / 8 reserved under the Cartesian V1 (D-EPI-MIG-0). **HISTORICAL — the pre-Cartesian pins below no longer describe the code:** `cognitive-shader-driver/examples/epistemic_reading_conflict_probe.rs`: contract-projected `Causes` reads as affordance code 20 `Related` (topology 0) or refuses (1..3), `COUNTERFACTUAL_PROBE` never eligible; `CAUSES` codes 1 and 17 sit on P7a `Open` / `CausalCandidate`; codes 25, 30 on P7a reserved bands; census over 24 cells = 3 agree, 5 disagree, 16 refuse; the affordance law measures where the contract refuses (undeclared / band `Absent`). Both readings run from shared code (`shared/affordance_law.rs`, and P7a's reading moved unchanged into `shared/certification_reading.rs`, used by `relational_certification_probe` too), never copied. Pins of current state, to become agreement tests when the issue closes (`ISS-EPISTEMIC-READINGS-DISAGREE-ON-BITS-61-63`); 5 tests, 6 disable runs red (3 on the affordance codebook, 3 on P7a's: `Contributes` entailment, a reserved code admitted, `Causes` recoded) | | **D-MBX-CUTOVER-0** | ShaderDriver on MailboxSoA without BindSpace | In PR | `entries/2026-10-06-shaderdriver-bindspace-cutover.md`: every ShaderDriver → BindSpace edge classified (production state / shared resource / fallback / test oracle / fossil / dead). In `run()` the only mailbox-mode BindSpace read was the ontology; outside it, metrics and the builder read the singleton and `backing()` silently fell back for any mailbox not under id 0. Cutover: optional BindSpace, explicit mailbox selection with no fallback, ontology on the driver, metrics from the selected substrate; no temporal change. `tests/mailbox_cutover.rs` 8 tests, 8 disable runs red | diff --git a/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md b/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md index 570666508..35aca99c8 100644 --- a/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md +++ b/.claude/board/entries/2026-10-07-puzzle-0-crossword-mask-propagation.md @@ -3,7 +3,7 @@ **Probe:** `cognitive-shader-driver/examples/crossword_mask_propagation_probe.rs`, over `examples/shared/population_fold.rs` (unchanged). **Status:** MEASURED (one machine, release, `avx2=true avx512f=false`), -TEST-PINNED (19 tests, 11 disable runs red). +TEST-PINNED (20 tests; 13 disable runs, listed below, all red). ## Question @@ -60,57 +60,64 @@ of the driver (its `guid-v3-tail` is already a default contract feature). ## Measured +Final run at `2bdb50b` (both review fixes in). One machine, release, +`avx2=true avx512f=false`. + Filter, 2000 patterns (length + 1–3 revealed symbols): | | EN | DE | |---|---|---| -| mask AND chain + popcount | 670 ns/pattern, 9.7 ns/claim | 683 ns/pattern, 7.6 ns/claim | -| direct spell-lane scan | 41,769 ns/pattern | 39,585 ns/pattern | +| mask AND chain + popcount | 775 ns/pattern, 11.2 ns/claim | 543 ns/pattern, 6.0 ns/claim | +| direct spell-lane scan | 35,755 ns/pattern | 42,124 ns/pattern | -Solve, median over puzzles of each arm's best of 5 warm runs (µs): +Solve, median over puzzles of each arm's best of 5 warm runs, rotated order, +one warm-up per arm; the literal arm's word list is built once per language, +outside the timing (µs): | | A token | D hybrid | B Cartesian | C literal | ANDs A / B | bytes A / B | |---|---|---|---|---|---|---| -| EN 5×5 (30) | 2.6 | 2.6 | 4.2 | 5,229 | 10.7 / 19.1 | 23,220 / 2,382 | -| EN 7×7 (5) | 3.0 | 3.1 | 5.0 | 5,753 | 15.6 / 33.6 | 42,724 / 2,417 | -| DE 5×5 (30) | 2.6 | 2.7 | 4.5 | 6,017 | 10.3 / 17.1 | 25,060 / 2,566 | -| DE 7×7 (2) | 5.5 | 5.6 | 9.1 | 8,689 | 14.0 / 20.0 | 42,602 / 2,599 | +| EN 5×5 (30) | 3.0 | 2.9 | 5.1 | 1,045 | 11.3 / 20.4 | 23,220 / 2,381 | +| DE 5×5 (30) | 2.3 | 2.3 | 3.7 | 986 | 7.6 / 11.4 | 25,060 / 2,568 | - A and D cost the same: routing letters through the Morton board adds no measurable time. B does about twice the ANDs (it recomputes) and is ~1.6× slower, but holds ~10× fewer bytes (one board instead of a full WordId - mask per slot). C, the string method, is ~2,000× slower than A. -- The first run timed each arm once, cold, A first, and showed A at 16.6 µs - against D at 6.9 µs with an equal AND count. That was cache warm-up, not - representation; the table above is the corrected method. + mask per slot). C, the string method, is ~350–430× slower than A. +- Two earlier timing methods were wrong and are superseded by the table + above: timing each arm once, cold, A first (A showed 16.6 µs against D's + 6.9 µs at an equal AND count: cache warm-up), and timing the literal arm + once including the rebuild of its word list (~5,200 µs; Codex review). - The shared fold on a 5×5 lane (~1.0M claims, EN and DE): every count equal - three ways; filter 0.28–0.35 ns/edge, histogram ~2.0, decode oracle ~3.9 — + three ways; filter 0.34–0.35 ns/edge, histogram ~2.0, decode oracle ~3.6 — the same costs as Sudoku and steps 2/2b. -Creation (budget 100,000 tried words per step, 30 s per size), NYT-rule grids: +Creation (budget 100,000 tried words per step, 30 s per size), NYT-rule +grids, **no word used twice in one fill**: -| side | EN made / attempts | DE made / attempts | +| side | EN made / misses | DE made / misses | |---|---|---| -| 5 | 30 / 30 | 30 / 30 | -| 7 | 5 / 175 | 2 / 163 | -| 9–21 | 0 (all misses are fill, none uniqueness) | 0 (same) | +| 5 | 30 / 1 | 30 / 4 | +| 7–21 | 0 (all misses are fill, none uniqueness) | 0 (same) | -Solving is not the limit at any size measured; **creation is**. With 18–20k -general vocabularies a random NYT-dense grid of side ≥ 9 is not filled within -the budget. The likely cause is vocabulary density (crosswords are built from -answer lists with many short fill words); not measured here. +The no-repeat rule (CodeRabbit review) changed the 7×7 result. Before it, +EN made 5 / 175 and DE 2 / 163 boards at 7×7; with it, none. Those fills +reused a word across slots, which a crossword does not allow. So with the +18–20k general vocabularies, creation stops at 5×5. Solving is not the limit +at any size measured; creation is. The likely cause is vocabulary density +(crosswords are built from answer lists with many short fill words); not +measured here. -Cui bono, each given removed in turn (fixed point re-run + uniqueness count): +Cui bono, each given removed in turn (fixed point re-run + uniqueness count), +5×5 only (no larger board was created): | | givens | Redundant | ShortcutOnly | SilentlyNecessary | Necessary | out of budget | |---|---|---|---|---|---|---| -| EN 5×5 | 118 | 32 | 37 | 0 | 49 | 0 | -| EN 7×7 | 51 | 39 | 6 | 0 | 6 | 0 | -| DE 5×5 | 98 | 21 | 29 | 0 | 48 | 0 | -| DE 7×7 | 7 | 0 | 5 | 0 | 1 | 1 | +| EN 5×5 | 116 | 28 | 40 | 0 | 48 | 0 | +| DE 5×5 | 74 | 14 | 25 | 0 | 35 | 0 | +Mean slots lost from the fixed point per removed given: EN 1.26, DE 0.20. `SilentlyNecessary` (a given propagation never used but search needs) did not -occur in the 273 ablations that finished within budget. The readout nominates; it does not prove cause: a +occur in the 190 ablations. The readout nominates; it does not prove cause: a given necessary in one set can be replaced by another set. ## Falsifiers (disable-verified red) @@ -120,11 +127,13 @@ index drops offset 0 (5, incl. index-vs-scan); no promotion at popcount 1 (2); first crossing only (every-crossing test); zero candidates not a contradiction; language gate removed; word bits written into the edge (CE64 test); crossings rediscovered from the cell lane (compiled-lanes test); ablation ignores -uniqueness, and ablation removes nothing (both ablation tests). +uniqueness, and ablation removes nothing (both ablation tests); no-repeat +rule removed (fills-never-repeat test). ## Open -- Creation beyond 7×7 with these vocabularies. +- Creation beyond 5×5 with these vocabularies (try a crossword answer list + at runtime). - Whether `MooreSymbol8` (and a `MooreLearn8` companion) become canonical value tenants: a contract decision. - The step-2b NYT-grid + crossword-answer mode (#1384, `94499c3`) was never run