Repository navigation
[PROBE] mass/interception-response: the water a larger canopy intercepts has to leave as evaporation #161
Description
Authors
Yuanhang Liu, Independent Researcher. ORCID: https://orcid.org/0009-0005-9633-6004. GitHub: @kawh1111.
Conservation law
Mass. Proposed probe: mass/interception-response.
This is a new probe, not one of ROADMAP's listed items. It asks a question the current suite cannot: when the same weather is run with a larger canopy capacity, does the water that capacity intercepts actually leave as evaporation, rather than disappearing or being turned into extra runoff? A model can pass every existing mass probe and still ignore canopy_capacity_mm entirely; the paired design below catches that.
Residual equation
Run the same seed and the same weather twice, changing only static.canopy_capacity_mm. Write Delta X = X(high) - X(low) for the same window, spinup included; sums are in mm over the window.
Delta pr = 0 by construction, so the paired water budget is
R = Delta evspsbl + Delta mrro + Delta gwex
+ Delta (mrso + snw + canopy + gw + channel) [mm]
and a model that conserves water has R = 0. Four criteria:
- A larger canopy must not increase runoff:
max(0, Delta mrro) / Delta cap <= tau - It must not decrease evaporation:
max(0, -Delta evspsbl) / Delta cap <= tau - The extra intercepted water must be allocated, not erased:
abs(R) / Delta cap <= tau - The response must exist, and it must be an evaporation response:
Delta evspsbl / Delta cap >= phi
with Delta cap = cap_high - cap_low in mm, tau = 0.05 and phi = 0.5.
Criterion 4 is what makes the probe more than a closure check. A model that declares canopy_capacity_mm and ignores it produces bit-identical runs: every Delta is zero, criteria 1-3 pass trivially, and 4 fails. A model that shaves runoff without moving the water into evaporation or storage fails 3; one that lets the larger canopy increase runoff fails 1; one that dries the catchment to fill the canopy fails 2.
I put the floor on the evaporation term rather than on the canopy-store peak on purpose. A model that routes interception loss directly to evaporation is physically sound and should not be failed for not exposing a separate store, while a model that parks the extra water permanently in storage fails 4. The measured models peak at or just below the capacity change anyway, so the two readings agree on them.
Implementation path
This needs one new paired criterion. It cannot reuse counterfactual_response, whose driver is a forcing column it sums over time; canopy_capacity_mm is a scalar in static.json. The criterion would live in src/hydroturing/criteria/interception.py and be registered in criteria/__init__.py, configured as:
- interception_response:
parameter: canopy_capacity_mm
response: evspsbl
tolerance: 0.05
response_floor: 0.5It receives the paired runs as RunResults, reads the parameter from each run's case.static, and computes Delta cap from those two values rather than trusting a number in the probe file. The four tests are then applied to the paired sums. No generator or harness change is needed beyond the criterion: the paired-run machinery and case.static are already there.
No-canopy-store accounting
Criterion 3 builds the budget from evspsbl, mrro and gwex, plus every state the model reports (mrso, snw, canopy, gw, channel — whichever are in emits.states). If canopy is absent, it is not in the sum and the budget is still defined: a model that routes interception loss straight to mrso evaporation shows it as Delta evspsbl and Delta mrso, and passes 3 and 4 if the allocation is right. If a model has a canopy store but omits it from emits.states, the sum misses that store and R opens by exactly its change, which is the existing "report every store you have" rule, not a special case here.
Tolerance and its denominator
tau = 0.05 of Delta cap, the capacity change the case prescribes, not of a background flux or storage. The denominator cannot approach zero because the generator fixes cap_low and cap_high (2 mm and 20 mm in the measurement below, so Delta cap = 18 mm). I add the suite's absolute floor, max(tau * Delta cap, 1e-3 mm).
Measured on current main (seed 186927550, mass/dry-down's 1095-day weather, cap 2 -> 20 mm):
| model | Delta sum mrro | Delta sum evspsbl | R | peak canopy 2 -> 20 |
|---|---|---|---|---|
reference_bucket |
-48.42 mm | +48.42 mm | +0.000001 mm | 1.26 -> 19.14 mm |
flex_lumped |
-73.16 mm | +73.16 mm | +0.006 mm | 2.00 -> 20.00 mm |
flex_topo |
-96.51 mm | +96.53 mm | +0.014 mm | 2.00 -> 20.00 mm |
All three close to better than 0.02 mm on an 18 mm capacity change (0.1 percent of Delta cap), and all three move between 48 and 97 mm of water, well above the criterion-4 floor. Their response magnitudes differ by a factor of two, which is why criterion 4 is a floor and not a magnitude test.
How would an unphysical model fail this?
Declares canopy_capacity_mm and ignores it. The two runs are bit-identical, every Delta is zero, and criterion 4 fails. That is reference_interception_blind, the purpose-built must_fail: it is exactly the "passes while understanding no physics" case this form asks about, and criterion 4 is the one that catches it.
How is the case generated?
A synthetic temperate record of two to three years with several storms large enough to fill a 20 mm canopy, followed by a dry spell long enough that both runs drain their canopy by the end, so the response has to appear in the event peaks rather than in a final store. Both variants are drawn from the same RNG state and see identical weather; only static.canopy_capacity_mm differs (cap_low = 2, cap_high = 20). requires.static: [canopy_capacity_mm] makes the probe an opt-in: a model that does not declare the static is N/A (INCOMPATIBLE) and its standing does not move. The probe does not require canopy as a state; the closure criterion sums every store the model reports, so a model with a canopy store that omits it leaks by exactly its change.
Overlap
mass/dry-downusescanopy_capacity_mmas one of its case attributes, but it varies the weather (a wet year then two dry ones) and scores recession. The capacity is fixed there, so that probe does not test whether a model responds to a change in it.mass/precipitation-counterfactual,mass/human-abstractionandmass/exchange-responseperturb a forcing (rain, withdrawal, prescribed head) and test the direction of a response. This probe perturbs a static parameter, and it scores the allocation of the intercept, not the direction of a forcing response.- No probe in the suite declares
requires.static: [canopy_capacity_mm]today.
Baselines and feasibility
must_pass: reference_bucket, flex_lumped and flex_topo, all measured above. cwatm and summa also declare canopy_capacity_mm and report canopy; they remain candidates until the Docker adapter passes the gate.
@cehw ran native CWatM 1.11 on the same 1,095-day seed at 2 -> 20 mm: Delta ET +63.32 mm, Delta runoff -62.84 mm, paired residual 0.784 mm against the 0.9 mm allowance (5 percent of 18 mm), both canopies empty. That is a fourth native positive reference. Its residual is 87 percent of the allowance, so CWatM enters must_pass only after the gate confirms it on every seed.
lisflood is left out of the 2 -> 20 mm case: its lai_for_canopy_capacity conversion tops out at about 11.718 mm, and 20 mm raises before the native run. Because compatibility is declaration-based, LISFLOOD still declares canopy_capacity_mm and would be scored at 20 and archived as ERROR rather than N/A. If that is not wanted, the alternative is a common representable pair (for example 2 -> 10 mm), with all baselines and CWatM re-measured at that pair. At 2 -> 10 mm I measured reference_bucket -29.95/+29.95 mm, flex_lumped -47.66/+47.66 mm and flex_topo -61.02/+61.03 mm, all well above the criterion-4 floor, but CWatM's 0.784 mm residual would then be close to the 0.4 mm allowance, so the pair and the tolerance need to be settled together.
must_fail: four controls, one per criterion, so each is shown to catch something the others do not:
| control | behaviour | expected |
|---|---|---|
reference_interception_blind |
declares the static, ignores it | fails criterion 4 (zero response) |
reference_interception_wetter |
larger capacity increases runoff | fails criterion 1 |
reference_interception_drier |
larger capacity decreases evaporation | fails criterion 2 |
reference_interception_leaky |
shaves 0.8 Delta cap of runoff, returns 0.6 Delta cap as evaporation, drops the rest | fails criterion 3 (0.2 Delta cap = 20 percent of Delta cap, over the 5 percent limit) and passes 4 |
sacsma_snow17 and dhbv2 emit canopy as an identically zero state and do not declare canopy_capacity_mm, so they are N/A (INCOMPATIBLE) on this probe and their standing does not move.
Will you build it?
Yes — assign it to me.
Activity
github-actions commentedon Oct 18, 2026
Scope
This asks whether a model's response to a declared capacity parameter is physically grounded, not just whether its budget closes — a model can pass every mass probe on main while silently ignoring
canopy_capacity_mm. That is a real gap: no merged probe (probes/mass/*/probe.yaml) perturbs a static parameter and checks for a required flux response;precipitation-counterfactualandresponse-nonnegativityperturb forcing, andexchange-responseperturbs a prescribed forcing driver (gwh), not a static. Paired variants that hold weather fixed and vary only a static are explicitly supported (docs/writing-a-probe.md, "Draw everything that comes from the seed before you branch on the variant"), so the design fits the harness.Overlap
No merged probe, open proposal, or PR touches
canopy_capacity_mmor interception response specifically. #136 and #62 mentioncanopyonly as one of several generic stores a closure residual sums over, not as the subject of a counterfactual. This reads as new, not a duplicate or extension.Soundness
canopy_capacity_mmandcanopyare real, precedented contract variables (AGENTS.mdtable; used instate_boundsinmass/exchange-responseandmass/precipitation-counterfactual), andrequires.staticis a real schema key (froude_subcritical'sbed_elevation_m). The baselines claim checks out:reference_bucketand (by declareduses_static/needs_static)flex_lumped/flex_topoconsume the static;sacsma_snow17anddhbv2both emit acanopystate but report it as identically zero and declare no consumption ofcanopy_capacity_mm, so they are correctlyN/A (INCOMPATIBLE)here even though the proposal's shorthand ("no interception store") slightly overstates it.What a gate would still need, that the four criteria as written do not give: none of the catalogued criteria (
docs/writing-a-probe.md's table) takes a static-parameter difference as its normalizing denominator.counterfactual_responseis the closest shape but its implementation (src/hydroturing/criteria/response.py) requiresdriverto be a forcing column it can sum over time (control.forcing[driver]);canopy_capacity_mmis a scalar instatic.json, not a forcing series, so this would need a new criterion, not a reuse of an existing one. That is a normal cost of a new probe, not a flaw, but the proposal should say so rather than imply the four criteria slot into existing machinery unchanged.Two smaller gaps: the "floor on the evaporation term rather than the canopy-store peak" choice is argued once but the proposal doesn't say what happens to a model that reports no
canopystore at all yet still shows the required evaporation response (e.g. routing interception loss straight throughmrsoevaporation) — the text implies this should pass criterion 4 but the closure math in criterion 3 sums "every store the model reports," which needs to be explicit about what happens whencanopyis absent fromemits.states. Second,reference_interception_blindis named as the solemust_fail_onlycontrol; a model that responds tocanopy_capacity_mmonly partially (e.g. shaves a little runoff without moving water fully into evaporation) is the one criteria 1-3 are meant to catch, and no such intermediate control is described — worth adding one so criteria 1-3 are shown to be doing something beyond criterion 4.Questions for the proposer
canopy_capacity_mmbut reportscanopyas a fixed/absent store (e.g. routes interception straight tomrsoorevspsbl)? Spell out the closure accounting for that case specifically, since it is the one the floor on evaporation (vs. canopy peak) was designed to protect.reference_interception_blind, is there a planned negative control for criteria 1-3 individually (e.g. a model that partially shaves runoff without fully reallocating to evaporation or storage), so each criterion is shown to catch somethingreference_interception_blinddoes not?drivercan be a static value, can you confirm (or propose) howDelta cap-normalized criteria would actually be implemented, so reviewers can judge the gate before code is written?Suggested next step
Revise to spell out the no-canopy-store accounting and an intermediate negative control, and confirm the criteria's implementation path, before building.
Posted by the proposal-review workflow. Run
kawh1111 commentedon Oct 20, 2026
Revised the body to answer the three questions:
interception_response) is required;counterfactual_responsecannot take a static driver. It readsDelta capfrom the two runs'case.staticand applies the four tests; no generator or harness change is needed.emits.states. A model that routes interception loss straight tomrsoevaporation is scored onDelta evspsblandDelta mrsoand can pass 3 and 4. A model with a canopy store that omits it opensRby its change and fails 3, which is the existing rule rather than a special case.reference_interception_blind(4),reference_interception_wetter(1),reference_interception_drier(2), andreference_interception_leaky(3: 0.8 Delta cap of runoff shaved, 0.6 Delta cap returned as evaporation, 0.2 Delta cap dropped, which is 20 percent of Delta cap against the 5 percent limit).cehw commentedon Oct 21, 2026
Thanks, Yuanhang — varying a declared capacity fills a useful gap beyond checking whether a model's water budget closes. I ran native CWatM 1.11 outside Docker on your cited 1,095-day dry-down seed, changing only capacity from 2 to 20 mm. Total ET increased by 63.32 mm and runoff fell by 62.84 mm; the paired residual was 0.784 mm against the 0.9 mm allowance, and both canopies ended empty. This supports the proposed case.
One practical note for the LISFLOOD candidate: its current
lai_for_canopy_capacityconversion has a maximum representable capacity of about 11.718 mm; calling it with 20 mm raises before native simulation. I'd leave it outside the initial baseline set for the fixed 2-to-20 mm case. If including it is a priority, a common representable capacity pair would need to be chosen and the existing baselines rechecked on that same pair.The CWatM result provides another native positive reference; keeping the candidate input range explicit should make the eventual gate easier to interpret.
kawh1111 commentedon Oct 22, 2026
Thanks, cehw - that CWatM run is exactly the native positive reference the proposal needed: +63.32 mm ET, -62.84 mm runoff, R 0.784 mm against the 0.9 mm (5 percent of 18 mm) allowance, canopies empty. I have recorded it in the body.
On LISFLOOD: agreed that it should not be a baseline at 2 -> 20 mm. One consequence to settle before the gate: compatibility is declaration-based, so LISFLOOD still declares
canopy_capacity_mm, gets scored at 20 mm, and its adapter raises -> ERROR rather than N/A. I would rather not turn a representable-range limit into a model failure. The options I see:reference_bucket-29.95/+29.95 mm,flex_lumped-47.66/+47.66 mm andflex_topo-61.02/+61.03 mm, all well above the criterion-4 floor. CWatM's 0.784 mm residual would then be close to the 0.4 mm allowance, so its numbers and the absolute floor need rechecking at that pair;Do you have a preference? If 2 -> 10 mm is acceptable, I will re-run CWatM at that pair and update the proposal, the baselines and the tolerance together.
chrimerss commentedon Oct 22, 2026
It sounds good to me
cehw commentedon Oct 23, 2026
Thanks, Yuanhang. The new reference runs retain a clear response with the smaller capacity change. I'd use 2→10 mm as the common pair and remeasure CWatM before settling the allowance; our 0.784 mm residual belongs to the 2→20 mm experiment. One correction before implementation:
gwexis positive into the catchment, so the paired residual should use−Δgwex, matching the existing closure criterion. Our zero-exchange CWatM runs do not test that term. This gives the first gate a common input range and keeps the water-budget convention consistent.