Skip to content

Rock fragment and texture input causes unpredictable / counter-intuitive score changes (global/HWSD) #377

Description

@DerekCaelin

Entering a rock-fragment value that matches a listed soil can decrease that soil's properties score, while a non-matching soil can jump to the top. Scores appear to shift arbitrarily for soils whose rock fragment value did not change.

Similarly, entering Texture input counter-intuitively impacts scores in some cases. In example below, entering "Loam" texture increases Clay soils' scores and lowers Loam soils' scores

Region: global (HWSD / rank_soils_global, soil_id/global_soil.py)

Steps To Reproduce

Rock Fragment

  1. Create site at 8.48144, 36.35144
  2. Set rock fragment = 0–1% on the top 3 layers. Note "Soil Properties" scores.
  3. Change rock fragment to 15–35% on the top 3 layers (no other changes). Note "Soil Properties" scores.
  4. See issue

Texture
4. Add texture = Loam to the top 3 layers. Note "Soil Properties" scores.
5. See issue

Expected behavior

Rock Fragment
Soils whose listed rock-fragment class is closer to 15–35% should rise relative to those at 1–15%; matching a soil should increase its properties score.

Texture
Loam-textured soils should rise; Clay and Clay Loam soils should fall.

Actual behavior

Rock Fragment
Humic Alisols (listed 15–35%): properties score falls 84% → 55% when the user matches it.
Eutric Fluvisols (listed 1–15%, non-matching): properties score jumps to 95% (highest).

Texture
Eutric Vertisol (Clay, 1–15%): 55% → 85% (rose).
Humic Nitisol (Clay, 1–15%): 55% → 63% (rose).
Dystric Gleysol (Clay Loam, 1–15%): 55% → 18% (fell sharply).
Eutric Fluvisol (Loam, matching): 95% → 79% (fell).

Additional context

DeepSeek feedback:

Cause: The global ranking computes Gower distance per 1-cm depth slice with no stable theoretical range: _slice_gower_distance → gower_distances(slice_mat) (global_soil.py:576, 856). Each slice normalizes numeric features by that slice's own max − min (utils.py:923-935), and the user's own value is part of that min/max. Changing one input therefore rescales every candidate's distance in every slice. The US path avoids this by passing theoretical_ranges (us_soil.py:2307-2310); the global path does not. A secondary factor is mean-imputation of missing values (utils.py:850, SimpleImputer(strategy="mean")), which suppresses the intended "missing data / shallow soil" penalty.

Suggested fix: direction Pass theoretical_ranges (or equivalent fixed 0–80 range for rock fragments and texture ranges) into the global gower_distances call, matching the US behavior; stop mean-imputing missing feature values so the NaN/soil-vs-non-soil infill (global_soil.py:877-895) can operate.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

  • Status
    Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions