Repository navigation
fix(grouping): Classify Unity IL2CPP frames on every platform - #122130
Conversation
5f2fb65 to
645be8e
Compare
32a660e to
11d91c3
Compare
3527a4f to
03c6eab
Compare
11d91c3 to
e92eac2
Compare
03c6eab to
4836d6b
Compare
Sentry Snapshot Testing
|
4836d6b to
7c871a7
Compare
d77201a to
3abe850
Compare
There was a problem hiding this comment.
This looks reasonable overall, but there are three changes I'd make:
Change 1:
Applied to both all-platforms:2023-01-11 and all-platforms:2026-01-20 so the fix lands regardless of which grouping config an organization is on.
We don't actually want this. The point of the old config is to allow us to match to existing hashes, so we can't change how it works. So the fix should be in the newer config file only.
Change 2:
To make this a little drier, and a little more explicit, I think we should use categories where we can, instead of doing -app directly.
I had Claude write it up for me:
Change details
The "exclude everything, then restore the real code" shape is right, but the first step can assign a category and let the restores match on that rather than repeating the three binary names four times.
Some background on why this works, since it's ordering-sensitive: the classifier is one flat ordered list applied top to bottom in a single pass, later actions overwrite earlier ones, and a category: matcher sees categories assigned by any rule above it — that's how category:ui function:handleMessage category=indirection works at line 58 of the config. Category names are free-form, no validation, so it's not a problem to add a new category.
One gotcha that constrains the layout: you can't categorize the output binaries as framework and let the existing category:framework -app do the marking, because that rule lives at line 488 — below the restores — and would clobber them. [Edit from the human: This is arguably a bug, but not one we can fix in this config.] The -app for the IL2CPP category has to stay local to the block. The engine binaries have no restores, so they can ride the line-488 rule.
The full set
Everything below goes in all-platforms@2026-01-20.txt only, at three insertion points.
a. Sentry SDK binaries → telemetry, in the "Categorization of frames" section right after package:"**/libBugly.so" category=telemetry (line 187) — already "third-party crash reporter, matched by binary name", which is the exact precedent:
## Unity ships the Sentry SDK under a different name on each platform, none of them the embedded
## `Sentry.framework` covered by the support-framework rules below
family:native package:**/PlugIns/Sentry.* category=telemetry
family:native package:**/PlugIns/libsentry.* category=telemetry
family:native package:**/SentryObjC.framework/** category=telemetry
family:native package:**/libsentry.so category=telemetry
family:native package:**/libsentry-android.so category=telemetry
family:native package:**/sentry.dll category=telemetry
These replace the six -app rules currently added to the "Support frameworks that are not in-app" section, which then needs no additions at all.
Note this brings a -group along with the -app, since the category rule is category:telemetry -app -group (line 444). I think that's what we want — and it's consistent with how Sentry's own crash-handling frames are already treated: family:native function:crashpad::* category=telemetry sits three lines above the insertion point, and kscm_* / sentrycrashcm_* just below the support-framework block are already -app -group. It's only the four **/Frameworks/*.framework/** path rules that are -app alone. Flagging it explicitly though, since it's a hash change on top of the in_app regroup the description already calls out.
b. The IL2CPP block, after family:native package:/Users/** +app (line 301), where the current block already goes:
# Unity IL2CPP compiles game, engine and BCL code into one binary, so the bundle rules above would
# mark all of it in-app. Categorize the binary, exclude it, then restore only the frames the IL2CPP
# line mapping attributes back to project sources; anything still pointing at generated
# `il2cppOutput/cpp/*.cpp` went unmapped and is engine, BCL or runtime code. The restores match on
# the category rather than the paths alone, since `Assets/` and `Packages/` are not Unity-specific.
family:native package:**/GameAssembly.* category=il2cpp
family:native package:**/UnityFramework category=il2cpp
family:native package:**/libil2cpp.so category=il2cpp
category:il2cpp -app
## Code the line mapping attributed back to project sources and embedded packages...
category:il2cpp path:**/Assets/** +app
category:il2cpp path:**/Packages/** +app
## ...except Unity's own engine C#, which ships sources under an `Assets` directory of its own,
## and registry packages, which are third-party and miss the restore only by UPM convention
## (sources under `Runtime/`), so one shipping an `Assets` directory needs excluding explicitly.
category:il2cpp path:**/unity/Modules/** -app
category:il2cpp path:**/Library/PackageCache/** -app
## The engine, which never contains game code
family:native package:**/UnityPlayer.* category=framework
family:native package:**/libunity.so category=framework
family:native package:**/libiPhone-lib.dylib category=framework
family:native package:**/UnityRuntime.framework/** category=framework
# Unity's Android GameActivity glue, not the developer's `libgame`
family:native package:**/libgame.so category=framework
## Unity's generated iOS launch stubs: template code rather than anything the developer wrote,
## and iOS-only, so leaving them in splits an otherwise identical crash from its counterparts.
family:native path:**/MainApp/main.mm -app
# Unity dispatches coroutines from its own scheduler, so the frames below `InvokeMoveNext` are
# whoever started the coroutine rather than the caller that threw. Which of them get captured
# differs by platform, splitting one crash into several issues.
family:other package:UnityEngine.CoreModule* function:InvokeMoveNext v-group
c. Nothing in the support-frameworks section — see (a).
What it adds up to
28 rules becomes 15 rules. The restores go 12 → 4, and each binary name appears exactly once, so the next name Unity invents is a one-line change instead of a five-line one. Best part: the property the description calls out —
Every restore is qualified by the IL2CPP binary, because these paths are not specific to Unity and would otherwise pull unrelated native apps into the app frame set.
— becomes structural rather than hand-repeated twelve times. Someone adding a fifth restore path can't forget the qualifier.
I spliced these into the real config and ran them through the enhancer. Same in_app outcomes as this PR on all of: project source / unmapped il2cppOutput / engine C# under unity/Modules / embedded package / registry package, across all three binaries; the five engine binaries; the six SDK binaries; and the iOS launch stub. Also checked the leak case both directions — an unrelated native binary with Assets/ or Library/PackageCache/ in its paths keeps whatever in_app it arrived with, since the restores now require the category.
Two minor notes, neither blocking. The new category is persisted to frame.data.category in the event payload, so it's a small quasi-public surface — though that's arguably a feature, since a Unity customer could then write category:il2cpp path:**/ThirdParty/** -app in their own enhancements, which they have no way to express today. And assigning categories inside this block rather than in the top "Categorization of frames" section bends the file's convention a little; I'd take that trade to keep the block readable as one unit, but it's a judgment call.
While he was in there, he also pointed this out:
Bonus thought
While poking at this I noticed ** in these patterns matches whole path segments only, not substrings. So this rule:
family:native package:**.app/Contents/** +app
never matches a real bundle path. /Applications/Game.app/Contents/MacOS/Game comes back in_app=None; **/*.app/Contents/** matches it. The rule appears to be dead in both bases.
Nothing in this PR is affected — every pattern here is **/Name-shaped and correctly segment-aligned. But it does undercut the PR description's "the existing bundle rules (*.app/Contents/**, ...) mark all of it in-app" — on macOS a shipped game only gets in-app frames via /Users/**, so a game installed to /Applications is closer to the broken-Android case (nothing in-app) than the broken-iOS one. Much wider blast radius than Unity, so definitely its own fix.
Change 3:
It'd be great if the tests could be switched to using our grouping snapshot framework. I had Claude help me summarize the advantages:
Advantages of snapshot tests
The case for snapshot tests, beyond "that's how we usually test enhancement rules":
The snapshots record why, not just what. Frame hints name the rule that fired:
frame (marked out of app by built-in stacktrace rule (category:framework -app))
frame (marked in-app by built-in stacktrace rule (family:native package:**/Containers/Bundle/Application/** +app))
assert result["in_app"] is True doesn't distinguish whether a frame came back in-app via the Assets/ restore or incidentally via the /Users/** bundle rule. Right outcome, wrong reason, no test failure to let you know. Since this PR's whole design is which rule should be responsible for each frame, that distinction is the thing most worth testing, and the snapshot is the only form that captures it.
It tests the actual claim. The headline here is a grouping claim — same crash on different platforms, one issue — and per-frame in_app assertions can't demonstrate it, which is why the description has to make the argument with screenshots. The variants snapshots carry the per-variant hash, so three platform inputs converging on one app hash is a checked artifact. (I ran this locally with the rules applied: they do converge, and the system hashes stay distinct.)
It goes through the real save pipeline. run_as_grouping_inputs_snapshot_test runs the default config through EventManager.save, where the explicit tests call apply_category_and_updated_in_app_to_frames directly. For a change whose entire risk surface is "real events get different in_app and regroup," that gap matters.
It covers the frames nobody thought to enumerate. Both styles load the full config, so both catch cross-rule interference for the cases written down. The difference is the other 30 frames in a real Unity crash: a snapshot annotates all of them, so a rule added at line 600 next year that flips frame 27 shows up as a diff.
Config coverage is automatic. The harness pulls configs from the registry (set(GROUPING_CONFIG_CLASSES.keys())), so there's no hardcoded list of enhancement bases to keep updated as configs come and go.
On the loss of comments, which is the obvious objection: it's smaller than it looks here. The hint text names the rule, so the snapshot says what each frame is demonstrating without needing prose, and the "why" for each rule belongs in the config file anyway, where someone reading the rules will actually find it. For the handful of cases that genuinely need explaining — an unrelated native app with an Assets/ directory, a developer's own libgame.so — the function name in the fixture carries it into the snapshot output.
Negative cases work fine in this format, incidentally. Give the frame in_app: true in the fixture and the snapshot shows either marked in-app by the client (nothing touched it, which is the assertion) or the name of whichever rule wrongly claimed it.
I also had him help me write test inputs for this, which you can check out in #124621. The ordering then should be:
- I merge that PR, so we have the "before" state recorded. [UPDATE]: Done.
- In this PR, you replace the commit adding the test file with a commit updating snapshots. Since the rules are only going to go in the newer config, only those snapshots should change. (To do the updates, first set
SENTRY_SNAPSHOTS_WRITEBACK=1in your env, then runpytest tests/sentry/grouping --reuse-db.)
### The Goal Add grouping input fixtures and their snapshots for Unity IL2CPP stack traces. **No rule changes** — this PR is fixtures and generated snapshots only, so the snapshots here record today's behavior. This change is split out from #122130 (which updates IL2CPP in-app rules) deliberately. Landing the inputs first means that PR's diff consists of the rule change plus the snapshot churn it causes, and the churn _is_ the review artifact: every frame whose `in_app` or grouping changes shows up as a line, annotated with the rule responsible. ### What's covered Five inputs, each packing several cases into one stack so the snapshot reads as a table rather than one assertion per file: | Input | Covers | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `unity_il2cpp_macos.json` | `GameAssembly.dylib`, `UnityPlayer.dylib`, `PlugIns/Sentry.*`, `PlugIns/libsentry.*` | | `unity_il2cpp_ios.json` | `UnityFramework`, nested `UnityRuntime.framework`, `SentryObjC.framework`, the generated `MainApp/main.mm` launch stub | | `unity_il2cpp_android.json` | `libil2cpp.so`, `libunity.so`, `libgame.so`, `libsentry.so`, `libsentry-android.so` | | `unity_il2cpp_edge_cases.json` | negative cases, Windows backslash paths, `GameAssembly.so`, `libiPhone-lib.dylib`, `sentry.dll`, a registry package under `Runtime/` | | `unity_coroutine_dispatch.json` | `UnityEngine.CoreModule` / `InvokeMoveNext` coroutine dispatch (`family:other`) | The three platform inputs carry the same logical stack — `EnemySpawner.Update` → `CombatSystem.ApplyDamage` → `TMP_Text.SetText` → `Animation.Play` → null-ref raise — differing only in binary names and bundle paths. That's what makes them comparable: the same crash, reported from three platforms. Within each platform stack: a frame the IL2CPP line mapping resolved to `Assets/`, one resolved to an embedded `Packages/` directory, one resolved into `Library/PackageCache/` (a registry package shipping its own `Assets` directory), one resolved into Unity's engine C# under `unity/Modules/**/Scripting/Assets/`, two unmapped frames still pointing at generated `il2cppOutput/cpp/*.cpp`, the engine binary, and the Sentry SDK binaries. ### What the snapshots currently record Three currently broken states, which is the point of adding the fixtures before the fix: | Input | `app` variant hash today | | ---------------------- | ---------------------------------------------------------------------------------- | | `unity_il2cpp_ios` | `8bbdecb4ad2f92d58133a93ee009a07b` — engine, BCL and runtime frames are all in-app | | `unity_il2cpp_android` | `null` — nothing is in-app at all | | `unity_il2cpp_macos` | `null` — nothing is in-app at all | Verified locally with the follow-up rules applied, all three collapse to a single `app` hash while their `system` hashes stay distinct. That's the cross-platform grouping claim, and once these inputs are on master it becomes a checked artifact rather than a screenshot. ### Authoring notes Worth knowing if you extend these: - **The IL2CPP method hashes differ across the three platform inputs.** `_il2cpp_method_hash` in `sentry/stacktraces/functions.py` strips the `_m<40 hex>` suffix before grouping, so identical hashes would have made the matching app hash ambiguous. Differing ones show the trimming is doing the work, and these inputs will diverge loudly if that regex ever changes. - **Function names in `unity_il2cpp_edge_cases.json` are descriptive rather than realistic** (`unrelated_native_app_source_under_assets_dir`). That input is a synthetic stack whose hash is only a regression detector, so the names carry the intent. The three platform inputs use realistic mangled IL2CPP names instead, since their hashes are meaningful.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a65d604. Configure here.
|
Thanks, all three are in.
I also replayed the That testing led to one more change: the launch-stub rule is now I'll follow up on the bonus .app/Contents/ . |
This modifies and adds to our existing grouping snapshot inputs for Unity, to make them more realistic and cover more platforms. Credit goes to @bitsandfoxes for the work - I'm merely splitting the input changes out from #122130, so that its diff can reflect only the snapshot changes actually caused by its code changes. From his comment on that PR, notes on the new and changed inputs here: - `unity_coroutine_dispatch.json` now uses the frame shape `sentry-unity` actually sends (function: `"InvokeMoveNext"` with the type in module). The fully qualified names never matched the rule, so the snapshot wasn't exercising it. - `unity_il2cpp_ios.json` gains the `-[UnityFramework runUIApplicationMainWithArgc:argv:]` frame that real builds have between `main` and `UIApplicationMain`. - New Windows and Linux inputs.
IL2CPP compiles game, engine and BCL code into one binary whose name differs per platform (GameAssembly.*, UnityFramework, libil2cpp.so). The bundle rules marked all of it in-app on Apple platforms and none of it on Android, Windows and Linux, so the same crash became a separate issue on every platform. Keep only the frames the IL2CPP line mapping resolves to project sources in-app, and mark the engine, Unity's GameActivity glue, the iOS and macOS launch stubs and the Sentry SDK binaries as not in-app. The launch stub is matched by its call into IL2CPP or UnityPlayer, since its source path is lost when the app executable's debug file is not uploaded. Coroutine starters stop contributing, since which of them get captured differs by platform. The rules go into the current enhancements base only, so existing hashes on the legacy base are unaffected, and only the newstyle:2026-01-20 snapshots change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The snapshot helper swaps `-` for `_` to turn input file names and config ids into snapshot names, but it did so on the absolute path, so a checkout whose directory name contains `-` read and wrote snapshots under a directory that doesn't exist. A writeback there leaves a stray copy that later runs compare against, so tests pass on stale snapshots. Apply the swap only below the snapshots directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7e3c70b to
1ddc36a
Compare
lobsterkatie
left a comment
There was a problem hiding this comment.
Thanks for all the work on this!

The Goal
The goal is to have bugs that are the same, but being reported from different platforms, be grouped together as one issue.
Before
Issue grouping

Stack trace quality Either all frames ended up being in-app, or none.
After
Issue grouping

Stack trace quality
All The Context
IL2CPP compiles game, engine and BCL code into a single binary. On iOS the existing bundle rule (
**/Containers/Bundle/Application/**) marks all of it in-app, so engine internals and the IL2CPP runtime end up in the app stack trace and drive grouping. On Android, Windows and Linux the opposite happens: nothing is in-app at all, because no+apprule matches where those binaries live. On macOS only/Users/**applies;**.app/Contents/**never matches a real bundle path, since**only matches whole path segments (a separate fix).Categorize the IL2CPP binary as
il2cppand the engine asframework, take both out of the app, then restore in-app only for frames the IL2CPP line mapping attributes back to project sources,Assets/and embeddedPackages/. A frame still pointing at generatedil2cppOutput/cpp/*.cppstays not-in-app, since an unmapped frame is engine, BCL or runtime code. The restores match on theil2cppcategory, because these paths are not specific to Unity and would otherwise pull unrelated native apps into the app frame set.The binary is named differently on each platform:
GameAssembly.*UnityFrameworklibil2cpp.soMatching only
GameAssembly.*would fix the three desktop platforms and silently do nothing on iOS and Android, leaving both with the broken defaults above.Details to notice:
Assetsdirectory (unity/Modules/**/Scripting/Assets/*.cs), so the restore has to exclude it or the engine frames comes back in-app. Registry packages (Library/PackageCache/**) are third-party and are excluded the same way.Sentry.frameworkthe existing rules covered. They are categorizedtelemetry, which also takes them out of grouping (-group), consistent with how other crash reporters are treated.MainApp/main.mm) into the exported Xcode project, inside the app's own executable. It is template code, and leaving it in-app splits an otherwise identical crash away from its other-platform counterparts. It is matched by its call into the IL2CPP binary rather than by its path, since the path is lost whenever the app executable's debug file is not uploaded. The macOS player'smaingets the same treatment, matched by its call intoUnityPlayer, because a game installed under/Users(a Steam library, a developer build) has its executable marked in-app by the bundle rules.libgame.so. It is matched by Unity's generated sources (unityLibrary/src/main/cpp/GameActivity/**) or its symbols (Unity::*,MainLoop) rather than the file name, sincelibgameis also a natural name for a developer's own NDK library.Separately, Unity dispatches coroutines from its own scheduler, so the frames below
InvokeMoveNextare whoever started the coroutine rather than the caller at the point it threw. Some SDKs capture them and some do not, which splits one crash into several issues, so they no longer contribute to grouping. They remain in-app and visible in the UI.Applied to
all-platforms:2026-01-20only. The legacy base exists to keep matching existing hashes, so it stays unchanged.This changes
in_appfor Unity native events and will regroup them. All rules except the coroutine one arefamily:native. IL2CPP events sent by the C# part of sentry-unity arrive with platformcsharpand theirpackagegets rewritten to the managed assembly name, so only the coroutine rule applies to them.Verification
The grouping snapshot inputs, split out into #129052 so this diff only shows what the rules change, model one crash on Android, iOS, macOS, Windows and Linux. With these rules all five produce the same app hash. Only the
newstyle@2026_01_20snapshots change.The sentry-unity integration-test corpus (Unity 2021.3, 2022.3, 6000.0, 6000.3, 6000.5) was replayed end to end through a local Relay, Symbolicator and ingest pipeline. For every Unity version, the native crash from all five platforms lands in one issue, and so does the C# exception. As a control, the same run on the previous rules splits the native crash into five issues and the C# exception into two. The Apple native crash relies on sentry-cocoa marking its mechanism synthetic, which is a separate SDK change, simulated in these runs.
Snapshot helper fix
A separate commit fixes
get_grouping_input_snapshotter, which swapped-for_on the absolute snapshot path rather than only on the part belowsnapshots/. In a checkout whose directory name contains-, the grouping snapshot tests read and write a directory that doesn't exist, and a writeback there leaves stale copies that later runs pass against. CI's checkout path has no-, so CI was unaffected.test_snapshot_path_keeps_checkout_pathcovers it.🤖 Generated with Claude Code