Skip to content

feat(ghg): retain ActivityProducedBy and stub source-resolved E snapshots - #1136

Merged
bl-young merged 3 commits into
mainfrom
by-activity-pipeline
Oct 21, 2026
Merged

bl-young merged 3 commits into
mainfrom
by-activity-pipeline

Conversation

@bl-young

@bl-young bl-young commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

cc: @MoLi7
Related: #1134, #1135

What changed? Why?

Cornerstone GHG year methods set retain_activity_columns: true so regenerated FBS keeps ActivityProducedBy (GHGI line names) through generateFlowBySector. Waste Use resplits pool on that column when present so activity labels survive. Gas E mass is unchanged; the pin is not bumped here.

Snapshot generation renames the gas matrix to E_USA_GAS and reserves E_USA_ES for ceda EmissionsSource x industries (AR6 kg CO2e), plus E_USA_ES_LONG for the GWP long table. Until the #1135 crosswalk fills real labels, E_USA_ES is a zero stub (123 placeholder rows) and the long table is schema-only empty.

Impact

Published E and the pinned GHG FBS are unchanged until a pin bump or E_USA_* snapshot regeneration.

Testing

Waste Use activity pooling unit test; FBS regen compare collapses activity until the pin bumps.

uv run pytest bedrock/transform/allocation/__tests__/test_waste_use_emissions_split.py bedrock/transform/__tests__/test_fbs.py -q -m "not fbs_regeneration"

Review focus

  • Confirm gas totals stay invariant with retained activities (waste pool keys include ActivityProducedBy).
  • Stub E_USA_ES intentionally does not map to ceda codes yet.

Heads-up

Do not regenerate or publish E_USA_* snapshots from this branch until the ceda reader cutover: a regen would overwrite published gas E_USA_ES with stub zeros. Follow-ups: FBS pin bump (year stem), #1135 crosswalk fill-in, coordinated snapshot bump with ceda.

Enable source-resolved E export by keeping GHGI activity names through
generateFlowBySector, and pool waste Use resplits by activity so those
labels survive.
…sources

Rename gas E to E_USA_GAS and reserve E_USA_ES for ceda EmissionsSource x
industries. Stub ES as zeros (123 rows) until the #1135 crosswalk fills it;
add empty E_USA_ES_LONG for the GWP long table.
@catherinebirney

Copy link
Copy Markdown
Member

@bl-young We have a potential for issues due to the work incorporating the GHGRP floor selecting the top activity set activity when new rows are inserted. Right now, the mapping created in #1135 would work correctly because mappings are based on all activities (*) - but if we ever change that mapping, we could end up with incorrectly mapped activities to ceda.

The GHGRP floor moves CO2 from industries that are above their own facility reports to industries that are below them, and the national total for that fuel stays the same. When the floor takes CO2 away, it shrinks the rows that are already there. Those rows keep their real line names.

When the floor gives CO2 to another industry, it inserts new rows. Each new row copies the first line name in that group, while the amount is a blend of every line in the group.

Groups with one name are labeled correctly: natural gas, motor gasoline, and distillate. Natural gas is the large move.
Two groups get a bad name on the new rows only: coal (Coal Industrial and Coal Industrial - Manufacturing) and facility petroleum (Petroleum Coke, HGL, and Kerosene). Those new rows are marked GHGRP_sector_floor or GHGRP_sector_ceiling in AttributionSources. Sector totals and the gas E matrix add the amounts and ignore the name, so this run's totals are fine. The name matters later, if those new rows are mapped from line name to an emissions source: the whole blend would be counted as whichever name was copied.

@catherinebirney

Copy link
Copy Markdown
Member

I ran the 2024 FBS method and the parquet output does look good - but it does have 5 empty columns appended for ProducedBySectorType, ConsumedBySectorType, SourceName, Description, Suppressed

we should drop SourceName, Description, Suppressed?

@bl-young

Copy link
Copy Markdown
Member Author

yeah or maybe all 5? is that a function of leaving the activity names in there you think?

@catherinebirney

Copy link
Copy Markdown
Member

yeah or maybe all 5? is that a function of leaving the activity names in there you think?

The regular FBS (without activity names) has columns for Produced/Consumed sector type

but the other 3 are appended by keeping the activity names - and we do not need those 3

@catherinebirney catherinebirney left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed the extra blank columns with f5b8b2a

approved

@bl-young
bl-young enabled auto-merge (squash) October 21, 2026 18:05
@bl-young
bl-young merged commit 635777c into main Oct 21, 2026
5 checks passed
@bl-young
bl-young deleted the by-activity-pipeline branch October 21, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants