🐛 sphinx-needs: under -j N, a need id defined in two workers renders its card once, on the document that kept it

[chrisjsewell]

What

• A Need node now carries the docname of the need it renders (node["docname"], set in _create_need_node). format_need_nodes drops a node whose need is recorded on a different document, before the hide branch, with no new warning (the merge already warned needs.duplicate_id). The node's tag alone decides, so a kept need created with add_need(…, docname=None) still drops the other document's copy. A node without the tag (a doctree pickled before this change) renders as before.
• merge_data no longer lets a worker's copy of a need node replace the node of an id the environment already holds, so needextract copies the content of the need that was kept.
• Changelog entry under Bug fixes.

Why

Serially, a second directive with an existing id is refused (needs.create_need) and returns no node. Under -j N, two documents read by different workers each create the need; the merge keeps the first one merged and warns needs.duplicate_id about the other, but the losing document's doctree still held its node, and format_need_nodes rendered every Need node from needs[id]: the losing page showed a card with the kept need's title and fields over the dropped directive's content, and the page carried a second id="<ID>". Reproduced 3/3 with plain req directives (index and page_b, read in chunks index, pad_0, pad_1 | pad_2, pad_3, page_b). Separately, the merge's node registry was a plain dict.update, so a needextract of the id copied the loser's content (5/5).

The decision reads the node's own tag, not the document name doctree-resolved passes (for singlehtml and latex that is the root document, for the one assembled tree) and not node.source (not a reliable statement of the node's document). The node is replaced before any writer sees it, so the attribute reaches no output; the doctree snapshot of test_basic_doc gains it on its two Need nodes. No environment-version bump: the current ENV_DATA_VERSION is itself unreleased, so every upgrade from a release re-reads everything anyway.

A related pre-existing problem found on the way is filed separately as #2100: deleting the document that kept the need drops the id from needs.json and crashes the next write of the loser's page.

ubCode parity

Nothing to match in this PR: ubCode's needextract already shows the kept need's content (rust/ubc_parser_ctrl/src/extract_cards.rs:1246-1249). For need directives ubCode differs in the other direction: it renders a card at every directive site of a duplicate id (the demoted registration is hydrated by id with the winner's data, rust/ubc_parser_ctrl/src/hydration.rs:2538, and its anchor minted from the node, rust/ubc_ast/src/render_html/special.rs ~1545), recorded there as a "registered residual". Filed as useblocks/ubcode#3893.

Tests

New tests/test_parallel_duplicate_id.py, every case serial and -j 2 (each -j 2 project has six documents, so Sphinx 7 reads it in parallel too; the -j 2 variants skip where parallel_available is False). Each -j 2 project's conf.py holds a small file barrier on source-read and env-merge-info, so its two chunks are always read concurrently and merged in a chosen order: without it, Sphinx can merge the first chunk before forking the second, which then refuses the duplicate as a serial build does (reproduced with a delay between Sphinx's add_task calls; the barrier fails loudly after 60 s rather than hanging). The assertions take the winner from the warning:

• one need, one warning, one card, on the kept need's page (html), with either document's chunk merged first;
• singlehtml and latex: one card in the assembled document, and a need defined only on page_b still renders, the case a rule on the doctree-resolved docname fails; and the root document as the loser (root_doc = "zz_root", so it sorts last);
• a kept need created through add_need with no docname: the other document's copy is still dropped (and, merged in the other order, the known unwarned two-card case of a docname-less loser, pinned as it is);
• needextract shows the kept need's content;
• incremental -j 2 rebuilds, touching the winner (the loser's page written from its pickled doctree) or the loser (refused as in serial): one card after each;
• a need in an included fragment renders on the including page;
• a doctree pickled without the tag still renders.

The parallel cases fail before the fix and pass after it. test_parallel_execution.py, test_needs_builder.py and test_complex_doc.py pass, and none of them changed.

Closes #2087

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.39%. Comparing base (ccf3ff6) to head (2d18926).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2106   +/-   ##
=======================================
  Coverage   92.38%   92.39%           
=======================================
  Files         136      136           
  Lines       19309    19320   +11     
=======================================
+ Hits        17839    17850   +11     
  Misses       1470     1470           

Flag	Coverage Δ
ai-index	100.00% <ø> (ø)
codelinks	95.35% <ø> (ø)
mounts	94.26% <ø> (ø)
pytests	91.75% <100.00%> (+<0.01%)	⬆️
reports	88.01% <ø> (ø)
ub-test-reports	90.16% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.