Skip to content

SCAL-333657: report how a host event resolved and how long it took (PR 2) - #636

Open
sastaachar wants to merge 9 commits into
SCAL-333657from
SCAL-333657-outcome
Open

SCAL-333657: report how a host event resolved and how long it took (PR 2)#636
sastaachar wants to merge 9 commits into
SCAL-333657from
SCAL-333657-outcome

Conversation

@sastaachar

@sastaachar sastaachar commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

SCAL-333657 · epic SCAL-325536

PR 2, stacked on #635. Base is SCAL-333657, not main — review #635 first, and this diff shows only what PR 2 adds.

Did the host event work, and how long did it take.

PR 1 answers which host event, with which parameters. It says nothing about whether the trigger succeeded, so a host event that times out for 30 seconds looks exactly like one that works.

What this adds

Two properties on the same upload:

status success · error · timed-out · render-not-called · host-event-undefined · no-iframe
durationMs how long the trigger took to settle

The three non-dispatch statuses matter on their own: render-not-called and no-iframe are integration mistakes in the host application that are currently invisible, and no-iframe in particular happens after an auth failure.

timed-out needs saying out loud

processTrigger resolves — it does not reject — with Error(TRIGGER_TIMED_OUT) after 30 seconds. So without a check for that sentinel, a trigger the embedded app never answered reports as a success. That is the single most misleading thing telemetry could say here, which is why the check is in this PR rather than a later one.

isTriggerTimeout is a new export in processTrigger; it changes no behaviour.

An error's message is never uploaded — only the status — because it can echo customer data back. There is a test asserting a GUID from an error message does not appear in the properties.

One behaviour change, on purpose

The upload moves from call time to settle time, because neither the outcome nor the duration exists until the trigger finishes. Same event name, same properties as PR 1, just later by the round trip.

That is the trade this PR asks for: outcome data in exchange for a delayed event. If you would rather keep the call-time event and add a second one at settle, say so — it is a small change, but it doubles the upload count per trigger.

Testing

Four new cases in src/embed/host-event-telemetry.spec.ts: a successful trigger carrying status and durationMs, a trigger the app never answered reported as timed-out, a failed trigger with its error message asserted absent, and a trigger before render().

Full suite 1707 passed (47 files), tsc clean, lint 0 errors.

Still to come

  • PR 3 — the response story, both directions: whether the trigger was answered and with what shape, and the same for an embed event handed a responder via on(ApiIntercept, responder).
  • PR 4 — embed event tracking: which embed events arrive, their payload shape, how many handlers each dispatch ran.
  • PR 5 — the bug fixes: the same 30s timeout is lost entirely for Pin/SaveAnswer/UpdateFilters/DrillDown, where a custom handler turns it into a generic error; and the pre-init Mixpanel queue is unbounded when disableSDKTracking is set.

🤖 Generated with Claude Code

…ng it took

SCAL-333657

PR 2, on top of the parameter tracking in PR 1.

Adds `status` and `durationMs` to the host event upload. Status covers every
exit: `success`, `error`, `timed-out`, and the three that never reach the
embedded app at all — `render-not-called`, `host-event-undefined`, `no-iframe`.

`timed-out` needs saying out loud: processTrigger resolves, rather than
rejects, with an Error when the app never answers, so without a check for
that sentinel a 30-second timeout reports as a success. `isTriggerTimeout`
in processTrigger is a new export and changes no behaviour.

An error's message is never uploaded — only the status — because it can echo
customer data back.

One behaviour change worth calling out on review: the upload moves from call
time to settle time, because the outcome and the duration do not exist until
the trigger finishes. Same event name, same properties as PR 1, later by the
round trip.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sastaachar
sastaachar requested a review from a team as a code owner August 21, 2026 08:59

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances host event telemetry by tracking the status (such as success, error, timeout, and validation failures) and duration of host event triggers, supported by new unit tests. The review feedback suggests wrapping the telemetry upload in a try-catch block to prevent telemetry failures from disrupting the main promise chain, and notes a design improvement opportunity regarding promise rejection on timeouts for future versions.

Comment thread src/embed/ts-embed.ts Outdated
Comment on lines +1685 to +1697
const reportHostEvent = (status: HostEventStatus) => {
uploadMixpanelEvent(
`${MIXPANEL_EVENT.VISUAL_SDK_TRIGGER}-${messageType}`,
getHostEventTelemetryProps({
hostEvent: messageType,
payload: data,
context,
embedComponentType: this.viewConfig?.embedComponentType,
status,
durationMs: Date.now() - triggerStartedAt,
}),
);
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Wrapping the telemetry upload in a try-catch block is highly recommended here. Since reportHostEvent is executed inside the .then handler of triggerHostEvent, any unexpected synchronous error thrown during telemetry generation (e.g., if describeParams or uploadMixpanelEvent fails) will cause the entire promise chain to reject. This would incorrectly report a successful host event trigger as a failure to the host application, disrupting core functionality due to a non-critical telemetry side-effect.

        const reportHostEvent = (status: HostEventStatus) => {
            try {
                uploadMixpanelEvent(
                    MIXPANEL_EVENT.VISUAL_SDK_TRIGGER + '-' + messageType,
                    getHostEventTelemetryProps({
                        hostEvent: messageType,
                        payload: data,
                        context,
                        embedComponentType: this.viewConfig?.embedComponentType,
                        status,
                        durationMs: Date.now() - triggerStartedAt,
                    }),
                );
            } catch (e) {
                logger.debug('Failed to upload host event telemetry', e);
            }
        };

Comment on lines +40 to +41
export const isTriggerTimeout = (value: unknown): boolean => value instanceof Error
&& value.message === ERROR_MESSAGE.TRIGGER_TIMED_OUT;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Resolving a promise with an Error object on timeout instead of rejecting it is generally considered an anti-pattern in Promise design. It forces callers to perform manual type checks (like instanceof Error or using isTriggerTimeout) and can easily lead to unhandled runtime exceptions (e.g., TypeError when destructuring properties like session or currentContext from the resolved value). While this behavior might need to be preserved for backward compatibility in processTrigger, consider rejecting the promise on timeout in the next major version of the SDK to align with standard asynchronous error-handling practices.

sastaachar and others added 8 commits August 21, 2026 15:12
…outcome

SCAL-333657

PR 2, on top of the parameter tracking in PR 1.

A trigger now gets an id and its parameters go into a pending queue at call
time. When the trigger settles, the outcome joins the entry and the whole
thing is pushed as one upload carrying `hostEventId`, `status` and
`durationMs`. If nothing ever settles, a 35-second timer pushes the entry as
it stands with `status: 'no-outcome'`, so a trigger is recorded either way and
the queue cannot grow.

Status covers every exit: `success`, `error`, `timed-out`, and the three that
never reach the embedded app — `render-not-called`, `host-event-undefined`,
`no-iframe`.

`timed-out` needs saying out loud: processTrigger resolves, rather than
rejects, with an Error when the app never answers, so without a check for that
sentinel a 30-second timeout reports as a success. `isTriggerTimeout` is a new
export and changes no behaviour.

The upload itself is deferred to a microtask, so a Mixpanel call never sits on
the caller's synchronous path. Building the properties stays synchronous, on
purpose: the payload belongs to the caller, and describing it later would
report whatever it had been mutated into.

An error's message is never uploaded — only the status — because it can echo
customer data back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…onto it

SCAL-333657

The outcome was reported from inside the promise the caller gets back, so a
throw in telemetry rejected that promise: a working host event surfaced to the
host application as a failure, and was reported as an error at the same time.

The promise is now observed on the side. The trigger's own chain is exactly
what it is on main, telemetry attaches a second handler to it, and the
original is returned. Nothing telemetry does can change what the host
application receives.

The observer chain needs its own terminal catch, which is not obvious until
you try it: a throwing observer produces an unhandled rejection on the derived
promise, and the test for this killed the Node process before that catch went
in. It is a debug log now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sastaachar

Copy link
Copy Markdown
Contributor Author

Applied in 9cdfc435. The promise is observed on the side rather than chained onto:

const triggered = this.hostEventClient.triggerHostEvent(messageType, data, context)
    .catch(/* unchanged from main */);

// Observed, not chained.
triggered.then(
    (response) => reportHostEventOutcome(id, isTriggerTimeout(response) ? 'timed-out' : 'success'),
    () => reportHostEventOutcome(id, 'error'),
).catch((e) => logger.debug('Could not report host event outcome', e));

return triggered;

The trigger's own chain is now exactly what it is on main — the outcome reporting came out of the .catch entirely — and the promise handed back is the one the host application would have received without telemetry. That is a structural guarantee rather than a try/catch one: whatever telemetry does, it is on a derived promise nobody returns.

One non-obvious part, which a test caught rather than review: the observer needs its own terminal .catch. Without it a throwing observer produces an unhandled rejection on the derived promise — and the test written to prove telemetry cannot reach the caller killed the Node process until that catch went in:

[Error: telemetry exploded]
Node.js v22.13.1

So the pattern is right, but incomplete without the trailing catch. That test now passes and pins it.

Full suite 1716 passed (47 files), lint 0 errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants