Repository navigation
Conversation
Multi::OnSocket and Multi::OnTimeout enter JS from a libuv callback, that is, from outside any JS context. Multi::CallOnMessageCallback settles the promise perform() returns from there, and Node.js only performs a microtask checkpoint when a callback scope is closed. Only a HandleScope was being opened, so the continuation that emits `end` was not scheduled until something else drained the microtask queue - and for a caller awaiting one request at a time, that is the next tick of libcurl's own multi timer. On Node.js 26 / Linux this made every serialized request take ~1000ms even though the transfer itself completed in under 3ms: headers and body arrived at ~2ms and CURLINFO_TOTAL_TIME_T reported 0.5-2.8ms. Issuing requests concurrently masked it, because the other in-flight transfers kept waking the loop. Node.js 22, 24 and 25 were unaffected, as was macOS. Open a Napi::CallbackScope in both libuv callbacks, backed by a single long-lived AsyncContext per Multi so that no napi_async_init happens per socket event. Easy::CallSocketEvent, the only other libuv -> JS entry point, already used MakeCallback with an async context and needed no change. Serialized throughput on node:26-alpine goes from 1.3 to 3046 req/s (p50 995.7ms -> 0.29ms). Node.js 24 is unchanged within noise.
The default Multi drains completions from Multi::NotifyCallback when built
against libcurl >= 8.17, and directly from OnSocket / OnTimeout otherwise.
Both sit inside the callback scopes the previous commit added, but only the
notification-enabled path was exercised here, leaving the polling fallback
covered incidentally by whichever CI leg happens to build against an older
libcurl.
Parameterize the helper with an optional Multi and add a case pinned to
`new Multi({ shouldUseNotificationsApi: false })`, closing it afterwards.
Both cases fail on the unfixed addon (8.4s and 7.0s for 11 requests) and
pass with it.
|
hey @aborovsky thanks for the contribution! I am generally aligned with the fix |
JCMais
left a comment
There was a problem hiding this comment.
@aborovsky I pulled it locally and the fix works, but there is one issue with the async context lifetime that needs to be fixed before we can merge this.
Looks like there is a concrete issue, so thanks for looking into it, on Node.js 26.12.1 (against libcurl 7.81), the repro from #454 never emits end on develop, and emits it in ~4ms with this branch. The new test also fails on develop (~1000ms median) and passes on 24 and 26 with the PR.
The issue tho: Dispose() resets asyncContext, and that can happen while the CallbackScope opened by OnSocket / OnTimeout is still open. That happens if someone calls multi.close() from inside onMessage, which is exactly what we do in examples/04-multi.js. napi_open_callback_scope pushes a pointer to the async context resource into the async resource stack (src/node_api.cc), so after close() anything that reads executionAsyncResource() before the scope closes is reading freed memory. AsyncLocalStorage does that on every getStore() / promise init when using the async_hooks implementation (the default on Node.js 22).
This segfaults every time for me on Node.js 26 (it does not crash on develop):
multi.onMessage((error, handle) => {
multi.removeHandle(handle)
handle.close()
multi.close()
executionAsyncResource() // segfault
})On 22 and 24 it does not crash, but it is still a read of freed memory, so pure luck.
The fix is small, keep a reference for the lifetime of the scope, same thing Easy already does with cbOnSocketEventAsyncContext:
- std::unique_ptr<Napi::AsyncContext> asyncContext;
+ std::shared_ptr<Napi::AsyncContext> asyncContext;- if (!obj->asyncContext) return;
+ std::shared_ptr<Napi::AsyncContext> asyncContext = obj->asyncContext;
+ if (!asyncContext) return;
Napi::Env env = obj->Env();
Napi::HandleScope handleScope(env);
- Napi::CallbackScope callbackScope(env, *obj->asyncContext);
+ Napi::CallbackScope callbackScope(env, *asyncContext);(same thing on OnSocket, and make_unique -> make_shared on the constructor)
And a test for it, this one kills the vitest worker with the current branch and passes with the change above:
test
// examples/04-multi.js closes the Multi from inside onMessage, which runs
// while the callback scope opened by OnSocket / OnTimeout is still open.
// Node.js keeps a pointer into the scope's async context on the async
// resource stack, so that context must outlive the scope even when
// close() resets it. Reading the stack (as AsyncLocalStorage does) used to
// crash the process on Node.js 26.
it('should not crash when the Multi is closed from inside onMessage', async () => {
const resourceAfterClose = await new Promise<unknown>(
(resolve, reject) => {
const multi = new Multi()
const easy = new Easy()
withCommonTestOptions(easy)
easy.setOpt('URL', serverInstance.url)
easy.setOpt('WRITEFUNCTION', (_buf, size, nmemb) => size * nmemb)
multi.onMessage((error, handle) => {
multi.removeHandle(handle)
handle.close()
multi.close()
if (error) {
reject(error)
return
}
resolve(executionAsyncResource())
})
multi.addHandle(easy)
},
)
expect(typeof resourceAfterClose).toBe('object')
})(needs import { executionAsyncResource } from 'async_hooks' and Easy added to the ../../lib import)
With both changes the full suite passes locally on 24 and 26
Two smaller things:
- can you add an entry under
[Unreleased]->### Fixedon the CHANGELOG? - this also changes when microtasks /
nextTickcallbacks queued from inside the data/header/progress callbacks run. Before, they ran after each callback, now they run once when theOnSocket/OnTimeoutscope closes. I think that is the correct behavior (it is how Node.js itself does it), but imo worth mentioning in the CHANGELOG entry too.
Can you pull these into the PR?
Calling multi.close() from inside onMessage, as examples/04-multi.js does, runs Dispose() while the callback scope opened by OnSocket / OnTimeout is still open. Dispose() reset the only reference to the async context, so napi_async_destroy freed it under the open scope. Since Node.js 25, napi_open_callback_scope pushes a pointer to the context's resource onto the async resource stack, and anything that read it before the scope closed (executionAsyncResource(), AsyncLocalStorage) read freed memory: it returned undefined, or crashed under Guard Malloc. Hold the context in a shared_ptr and take a local copy for the lifetime of each scope, so it is destroyed only after the scope closes. The regression test runs on the polling path. With the notifications API, onMessage runs inside libcurl's notify callback, and curl_multi_cleanup fails there with CURLM_RECURSIVE_API_CALL.
|
Thanks @JCMais, great catch. I reproduced it locally (macOS arm64, libcurl 8.21) and pushed the fix in 3106150, plus the CHANGELOG entry in 7579a55. Repro: on Node.js 26.9.0, closing the One correction on scope: Node.js 25 is affected too, since Test: I added your test in The pinned test fails on the previous commit on 25 and 26 ( CHANGELOG: added under The full suite passes locally on 24 and 26, as does the #439 stress test. The only local failure is CI note: |
Problem
On Node.js 26, a finished transfer does not emit
enduntil some other JavaScript callback happens to run, even though libcurl has already received the full response. For a caller that awaits one request at a time this pins every request at libcurl's multi timer period (~1000ms on Linux), while the transfer itself completes in a few milliseconds. Node.js 22/24/25 and macOS were unaffected, which is what made it look intermittent.Reported in #454.
Root cause
Multi::OnSocketandMulti::OnTimeoutenter JS from a libuv callback — that is, from outside any JS context.Multi::CallOnMessageCallbacksettles the promise thatperform()returns from there, and Node.js only performs a microtask checkpoint when a callback scope is closed. Only aHandleScopewas being opened, so the continuation that emitsendwas not scheduled until something else drained the microtask queue — and for a serialized caller, that is the next tick of libcurl's own multi timer.Before Node.js 26.4.0, an empty callback scope in
CheckImmediateran the microtask queue on every loop iteration, which masked the missing scope. That fast path was removed in nodejs/node#62969 ("src: fast path empty native immediate drain"), surfacing the latent bug.Multi.addHandle()withMulti.onMessage()is not affected: it invokes the callback directly instead of resolving a promise.Fix
Open a
Napi::CallbackScopein both libuv callbacks (OnSocket/OnTimeout), backed by a single long-livedNapi::AsyncContextperMulti(created in the constructor, reset inDisposewhile the environment is still valid). One async context perMultiavoids anapi_async_initon every socket event.Easy::CallSocketEvent, the only other libuv→JS entry point, already goes throughMakeCallbackwith an async context and needs no change.Test
test/curl/serializedRequests.spec.tsserializes requests against a local server and assertsendfires without waiting for the multi timer. It covers both drain paths:new Multi({ shouldUseNotificationsApi: false }).Both legs fail on the unfixed addon (~1s per request on Node.js 26 / Linux) and pass with the fix.
Notes
This is a port of a fix that has been running in the
@brightsec/node-libcurlfork since 5.1.3 (BrightSec#41). Opened as a draft so CI can exercise it across the Node.js 24/25/26 matrix before review.