fix(logging): leave Bugsnag breadcrumbs off the tracing thread - #1663
Merged
Merged
Conversation
Play's ANR list for 2026.9.2 (4581) puts 16 of 21 ANRs in BugsnagBreadcrumbSink.record, tagged lock contention, while handling an FCM push or NotificationService. trace() runs every sink on the caller's thread, so a trace on main waited on whatever Bugsnag was waiting on, for 10s or more. The sink now queues breadcrumbs and one worker on its own thread hands them to Bugsnag in order. The queue holds 100, Bugsnag's own breadcrumb limit, and drops the oldest, so a worker stuck behind the same lock can't grow it. It isn't on Dispatchers.IO because that pool is saturated during cold start, which is when these ANRs fire. Error breadcrumbs still go to Bugsnag on the caller's thread. trace() reports the error right after the sinks run and the report snapshots breadcrumbs then, so a queued one would be missing from its own report. That adds no new way to block: Client.notify leaves its own breadcrumb on the same thread. Bugsnag never reported these ANRs. Its ANR plugin arms by posting to the main looper (AnrPlugin.kt:55, 6.27.0), which can't run while main is blocked in record. Expect Bugsnag's ANR counts to change shape once this ships.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Play's ANR list for 2026.9.2 (4581) puts 16 of 21 ANRs in
BugsnagBreadcrumbSink.record, tagged lock contention, while handling an FCM push (c2dm.intent.RECEIVE) orNotificationService.trace()runs every sink on the caller's thread, so a trace on main waited on whatever Bugsnag was waiting on, for 10s or more.bugsnag-breadcrumbsthread hands them to Bugsnag in order. It isn't onDispatchers.IObecause that pool is saturated during cold start (perf(startup): cut the wait between launch and the wallet tab #1505), which is when these ANRs fire.TraceType.Errorbreadcrumbs still go to Bugsnag on the caller's thread.trace()reports the error right after the sinks run and the report snapshots breadcrumbs then, so a queued one would be missing from its own report.Client.notifyalready leaves its own breadcrumb on that thread (Client.java:978), so this adds no new way to block.Which lock the main thread waited on isn't known yet. The Java side of
leaveBreadcrumbin 6.27.0 takes no monitor, so the NDK state behindNativeBridge.addBreadcrumbis the likely candidate. This change removes main's exposure regardless of the owner.Bugsnag never reported these ANRs: its ANR plugin arms by posting to the main looper (
AnrPlugin.kt:55), which can't run while main is blocked inrecord. Expect Bugsnag's ANR counts to change shape once this ships.Trade-offs: Bugsnag timestamps a queued breadcrumb when the worker delivers it, normally well under a millisecond late, and an error breadcrumb can land ahead of ones still queued.