Fix/cert path validator exception

[panasetskaya]

Background

We were seeing a steady stream of this in Datadog:

Got Error when signing in with BankId: IOError(message=IO Error with message:
java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.)

Three things came out of investigating it.

Our TLS setup is fine. No custom trust config anywhere in the app; authlib uses the default
platform trust manager. auth.prod.hedvigit.com and apollo-router.prod.hedvigit.com serve the same
certificate, rooted at Amazon Root CA 1 cross-signed by Starfield Services Root CA G2, which has
been in the Android trust store since long before our minSdk 23. Not an infra incident, and not a
release regression.

The volume was misleading. 499 of 644 events over a month (77.5%) came from a single device:
one member, 24 sessions, six days, ~499 taps of Retry. Excluding them, the real member-facing rate is
about 4.8/day, flat. Group by @usr.device_id before drawing conclusions from these counts.

We were making it much worse. That member logged in successfully 14 times and was force-logged-out
13 times. Any network error during a background token refresh caused us to delete their tokens, which
dropped them at the BankID login screen and forced a full re-login over the same failing network.

Practical gain of this release

1. Members stop losing their session over a bad moment of network.
The only change with real user impact. Previously one failed background token refresh destroyed the
session; now the session survives and the next request retries. Directly measurable — see the query
below.

2. We stop writing members' access and refresh tokens into Datadog and Crashlytics.
A log line was interpolating the whole AuthStatus.LoggedIn, which carries both tokens. Access tokens
live ~1h, refresh tokens ~24h, and a refresh token can mint new access tokens — so these were live
credentials in our log aggregators. Historical exposure has since expired. Invisible to users.

3. Members' network problems no longer count as our app errors.
Certificate-trust failures now log at WARN instead of ERROR, so the error rate reflects our bugs.

4. We may learn what actually causes the certificate failures. New diagnostic logging, below.

What this release does not do

It does not fix the certificate error. Nobody who currently cannot log in will suddenly be able to.
The gain is that far fewer members get pushed into that state, and that we stop leaking credentials
while it happens.

Queries to run

The lockouts we are now preventing

env:prod service:android "could not reach the backend. Keeping present tokens"

Every hit is a forced logout that would have happened before this release and now doesn't. On the one
device we investigated, this was 13 occurrences in six days.

Legitimate logouts (server actually rejected the token) are unchanged:

env:prod service:android "was rejected by the backend. Invalidating present tokens"

The old message "Refreshing token failed. Invalidating present tokens" no longer exists, so it
should drop to zero.

The new diagnostic

env:prod service:android status:warn "BankId login blocked by certificate trust"

Messages look like:

[TlsDiag] BankId login blocked by certificate trust. failure=javax.net.ssl.SSLHandshakeException
authHost=public transport=wifi userCaCount=1 api=28
msg="java.security.cert.CertPathValidatorException: Trust anchor for certification path not found."

That is a real line captured on device, not an illustration. msg comes last and is quoted because it
is the only field containing spaces, so the others stay cleanly parseable.

The fields are substrings in the message, so full-text search works but faceting does not. Count each:

"userCaCount=0"    "userCaCount=1"    "userCaCount=-1"
"transport=vpn"    "transport=wifi"   "transport=cellular"
"authHost=public"  "authHost=loopback"  "authHost=unspecified"  "authHost=private"

A Datadog grok parser on this message would give proper facets — ops change, no code needed.

What should mostly disappear

env:prod service:android status:error "Got Error when signing in with BankId"

What remains here is non-TLS failure (the Unable to resolve host DNS cases), which still log at
ERROR deliberately.

How to interpret the diagnostic

What dominates	Conclusion	Next step
userCaCount 1 or higher	A CA is installed on those devices — ad-blocker, antivirus, or work profile	Write the user-facing message; naming the cause is justified
userCaCount=0 + authHost=loopback / unspecified / private	A local DNS filter is answering with a sinkhole	Message justified, worded about the network rather than an app
userCaCount=0 + authHost=public + transport=wifi	Interception upstream — router, ISP, corporate wifi — invisible to the app	Inconclusive from our side; accept as environmental or take to infra
userCaCount=-1	The keystore probe threw	Fix the probe, not the app
Few TlsDiag lines while errors persist	Classification isn't matching in the field	Revisit the message-based fallback

Field reliability, established by testing

• userCaCount — trustworthy. Verified on an emulator: goes 0 → 1 when a CA is installed as a
  user certificate. This is the field that matters.
• transport — trustworthy. Returns real values. Note that on-device ad-blockers use Android's
  VPN API, so vpn here alongside a trust failure is a strong signal.
• authHost — narrow. Only catches DNS-level filtering (a local filter answering 0.0.0.0 or
  127.0.0.1). Our own lookup bypasses an HTTP proxy, so proxy-based interception reads public.
• failure — expect javax.net.ssl.SSLHandshakeException, always. Verified on device. The real
  CertPathValidatorException is never a link in the cause chain; Android only names it inside the
  handshake exception's message. Classification therefore matches on the message text ("trust anchor" /
  "certification path"), not on the exception type.
• msg — the wording the classification depends on. Captured verbatim (truncated at 200 chars) so
  we can spot the provider changing it, or it differing across OEMs and API levels. If that wording
  ever changes, these events stop being recognised and revert to the pre-existing status:error line
  we already monitor — so the regression is visible rather than silent.
• No certificate chain is available. We cannot name the intercepting CA. Android does not hand us
  the served chain on a trust failure, so userCaCount and transport are the only interception
  signals we get.

Cautions when reading the results

1. Expect roughly 65 events over two weeks at the current baseline, minus staged-rollout lag — likely
   fewer in practice.
2. Group by @usr.device_id first. One stuck member produced 77% of the previous batch.
3. One screen entry produces exactly one login attempt. Verified on device: a single tap to reach
   the login screen, no further input, one event. The app never retries on its own, so every event
   beyond the first is a real Retry tap — event counts are a fair proxy for how hard somebody was
   trying, and sub-second gaps mean frantic tapping rather than a loop in our code.

Known follow-ups, not in this release

• observeLoginStatus gives up on the first IOException mid-poll, so a brief network blip ends an
  in-progress BankID login permanently (26 occurrences on the investigated device).
• The token-refresh path (AuthTokenResult.Error.IOError) carries no diagnostics; only the login path
  is instrumented.
• No user-facing error message yet — deliberately withheld until the data says what to blame.