Skip to content

preserve-numbers: dropped year masked by an unrelated same-value digit #876

Description

@s-annam

Problem

checkNumbersPreserved's drop check (src/lib/webllm/preserve-numbers.ts) has a known masking gap for year-claimed atoms specifically. Every 4-digit number in 19002099 auto-classifies as a "year" claim in bareIntegerClaim, with no surrounding-context gate at all (unlike headcount, which needs a management verb or a people noun). That means a coincidental, unrelated 4-digit number in that range — a suite number, a model number, a version — is itself always "claimed," so it can mask a genuinely dropped year.

Repro (from src/lib/webllm/preserve-numbers.test.ts, "does NOT extend to year" test, added in #874's revision round):

checkNumbersPreserved(
  ["Founded the program in 1900.", "Operated out of suite 1900."],
  ["Founded the program.", "Operated out of suite 1900."],
)
// -> { ok: true, dropped: [], added: [] }

The founding year 1900 is genuinely dropped, but checkNumbersPreserved scores it clean because the unrelated, unchanged "suite 1900" shares the same num:1900 key and (per the current classifier) is always claimed.

Why this wasn't fixed in #874's revision round

The equivalent bug for headcount claims was fixed (count-aware presence check — a NEW unclaimed occurrence of a key still counts as a legitimate reword, but a pre-existing, unrelated unclaimed occurrence doesn't mask a drop). That mechanism doesn't transfer to year: because every year-shaped digit is unconditionally claimed, there's no "unclaimed" bucket for the count-aware guard to compare against — the fix is a no-op for this claim kind.

What a real fix looks like

Give year classification a context gate, mirroring headcount's PEOPLE_VERB_PREFIX / PEOPLE_NOUN_FOLLOW: require a temporal cue near the digit (e.g. "in", "since", "during", "by", "from", a preceding/following year-range dash) before classifying a bare 4-digit number as a year, falling back to unclaimed otherwise. This needs its own false-positive audit — a badly-scoped gate would create the missed-legitimate-year-reword failure mode that headcount's design docblock (preserve-numbers.ts module docblock, rule 1) explicitly warns about.

Pointers

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions