[WesIngwersen]

Closes #1140. Related: #1138, #836, #815, #615, https://github.com/cornerstone-data/ceda/issues/547

What changed? Why?

The nowcast Margins table splits each transaction's margin across the 24 margin-providing commodities (10 wholesale, 9 retail, 5 transport codes). That split was taken from each buyer's 2017 mix of margin services, so every commodity a buyer purchased got the same split. As a result, A_margin (margin dollars paid to each margin commodity per dollar of a commodity's producer value) reflected who buys a commodity, not what the commodity is. In v0.6, automobiles (336111) put less wholesale margin on motor vehicle wholesale (423100) than on machinery, electrical or professional equipment wholesale.

This PR splits each transaction's margin by the commodity purchased:

• Fit (Step 6, commodity_placement): within each buyer, a biproportional (RAS) fit holds two sets of totals: each commodity's Transportation, Wholesale and Retail margin (the five published Margins columns), and what each margin commodity earns from the buyer (its row of the Use table). The commodity prior decides the split inside those totals. Give-back (negative) cells are fitted on their own sign. A new Step 6 gate fails the build if any margin commodity's earnings from any buyer move by more than $1M.
• Transport prior: the existing 2017 mode x commodity fit (joint_mode_fit), now exposed as transport_commodity_prior.
• Wholesale and retail prior (trade_commodity_prior): 2017 Economic Census product-line sales by kind of business, mapped to trade codes by the trade crosswalk and to BEA commodities by the Step 1C: Inventories FBS - integrate the change in inventories column #529 product concordance. Each trade code's margin is its published 2017 margin output over its sales of all product lines, applied to its sales of each good. Two corrections for this use only:
  • Census suppression recovery shares a withheld product's residual equally across withheld industries. Here it is re-shared by each industry's observed sales of the same commodity family; equal shares had put grain on furniture and machinery wholesalers.
  • Four goods lines the shared concordance misses are mapped locally: light trucks (336112), nonprescription drugs (325412), retail "medicines, vitamins, minerals, and supplements" (325412/325411) and retail pet food (311111). They stay out of the shared concordance because it also drives the inventories attribution.
  • Anchored on 2017 and guarded: 2022 retail is on NAICS 2022 kinds of business (449, 455-459) that the trade crosswalk does not cover, and nonstore retail (454000) is not observable on its own.
• Change in private inventories (F03000) is left out of A_margin. Its 2017 placement nets offsetting signs (pipeline is 117% of its net transport margin), which put pipeline margin on automobile inventory builds. Phi (producer to purchaser price ratio) still includes it.
• New config 2025_usa_cornerstone_v0_6_margin_split: v0.6 with nowcast_mut_vintage: v0.6.0_5d40d42.

Impact

• No change to A, q, x, B or N. The rebuilt 2024 tables (v0.6.0_5d40d42, built from the same balanced 2024 tables as v0.5.0_a166975) match v0.5.0_a166975 on Make and Import exactly and on Use within $0.98. Under the two configs, q is identical and A_dom differs by at most 1.6e-12.
• A_margin changes in its split only. Total margin per dollar of each commodity (the column sums) is unchanged to 2e-16; 4,352 cells move. Selected columns, margin dollars per dollar of producer value:

Commodity	Family	v0.6 (buyer split)	This PR (commodity split)
Automobiles 336111	wholesale	machinery 0.091, electrical 0.082, professional equip. 0.073	motor vehicle wholesale 0.325
Automobiles 336111	transport	truck 0.040, rail 0.008	rail 0.036, truck 0.013
Pharmaceuticals 325412	wholesale	professional equip. 0.105, drugs 0.104	drugs 0.366
Refined petroleum 324110	wholesale	petroleum 0.118, other durables 0.050	petroleum 0.275
Coal 212100	transport	rail 0.136, truck 0.133, pipeline 0.094	rail 0.335, truck 0.026

• Margins table, after redefinitions: goods rows match v0.5.0_a166975 exactly in the five published columns. On margin-commodity rows, Producers' Value matches; Purchasers' Value (the buyer's direct purchase of the margin service, stored as Producers' Value minus margin routed onto it) moves by $14.1bn gross, $0 net, because Step 7 rescales each goods row with its own redefinition. Negative direct-purchase cells fall from 526 (-$8.65bn) to 434 (-$5.78bn).
• Interim A_margin for CEDA under this config: gs://cornerstone-default/snapshots/a_margin/v0.6.0_5d40d42/A_margin.parquet. The feat(snapshots): add A_margin as a snapshot object #1138 sidecar at .../a_margin/e10c75f5.../A_margin.parquet is untouched; @bl-young, feat(snapshots): add A_margin as a snapshot object #1138 can point at the new object.
• v0.6.0_5d40d42 is otherwise the first v0.7 input; the v0.6 release and .SNAPSHOT_KEY are unchanged.

Testing

• Unit tests: commodity_placement (empty prior reproduces the buyer split; both totals held; a commodity lands on its own code; give-backs fitted on their own sign), the 2017 guard, and the suppression re-split.
• 2024 rebuild: Step 6 earnings gate within $1 for all 24 margin commodities; after-redefinitions comparison against v0.5.0_a166975 as above; identity Producers' Value - Purchasers' Value = routed margin exact on every margin-commodity row.
• Local black and ruff; the full test suite is left to CI.

🤖 Generated with Claude Code

[bl-young]

Looks good — approving with a few pre-merge / follow-on notes.

Must before merge

1. Fix typecheck. test_use_price_bridge.py (~402, 419–420): cast .loc[...] results to float (or use pytest.approx) before comparing to numeric literals. Format/lint/tests are otherwise green.

Follow-on

2. Time-series MUTs: I've created #1146 from this PR (rebuild NowcastMUT 2017–2024 with the commodity-split Step 6 and retarget v1.0 year configs from #1145). Only v0.6.0_5d40d42 / v0_6_margin_split has the fix today; a166975 / 011f41f stay buyer-split. Balanced SUT pin is out of scope there.

Merge posture

3. Fine alongside #1138. No file overlap: #1138 registers A_margin for future snapshot cuts; this PR corrects the matrix content / interim sidecar CEDA already loads (…/a_margin/v0.6.0_5d40d42/…). Either merge order works.

Optional validation

4. Running 2025_usa_cornerstone_v0_6_margin_split through CEDA to confirm global EFs are unchanged is available if we want extra comfort, but not necessary for merge given Make/Use/Import parity and CEDA already consuming this sidecar.