Skip to content

refactor(core): move heartbeat/liveness logic into core behind remote-adapter feature - #768

Open
LiamCarPer wants to merge 1 commit into
Totodore:mainfrom
LiamCarPer:refactor/move-heartbeat-to-core
Open

refactor(core): move heartbeat/liveness logic into core behind remote-adapter feature#768
LiamCarPer wants to merge 1 commit into
Totodore:mainfrom
LiamCarPer:refactor/move-heartbeat-to-core

Conversation

@LiamCarPer

Copy link
Copy Markdown
Contributor

refactor(core): move heartbeat/liveness logic into core behind remote-adapter feature

Motivation

The postgres and mongodb adapters independently implement the same
heartbeat/liveness logic: the same nodes_liveness: Mutex<Vec<(Uid, Instant)>>
field, the same interval-based heartbeat job, the same
emit_heartbeat/emit_init_heartbeat helpers, the same recv_heartbeat
handler (~130 LOC duplicated verbatim, only the error type differs) and the
same server_count dead-node pruning. This duplication makes maintenance
harder and is contrary to the goal of #727 (share more code between adapters).

Solution

Move the heartbeat/liveness logic into socketioxide-core behind the existing
remote-adapter feature flag:

  • HeartbeatTracker: wraps nodes_liveness + hb_timeout; on_heartbeat
    (update-or-push, returns whether an InitHeartbeat reply is due),
    server_count (prune dead nodes + self), is_alive.
  • HeartbeatSender trait: uid, send_req, with default
    emit_heartbeat, emit_init_heartbeat and recv_heartbeat implementations.
  • heartbeat_loop helper.

Postgres and mongodb now keep only a thin HeartbeatSender impl (their
adapter-specific send_req + uid) plus a HeartbeatTracker field.
No behavior or wire-format change. Redis is unaffected: it has no heartbeat
(liveness is pub/sub subscriber counting via num_serv).

Closes #767

…-adapter feature

Moves the duplicated heartbeat/liveness logic from the postgres and
mongodb adapters into socketioxide-core behind the existing
remote-adapter feature flag.

- HeartbeatTracker: wraps nodes_liveness + hb_timeout; on_heartbeat
  (update-or-push, returns whether an InitHeartbeat reply is due),
  server_count (prune dead nodes + self), is_alive
- HeartbeatSender trait: uid, send_req, with default emit_heartbeat,
  emit_init_heartbeat and recv_heartbeat implementations
- heartbeat_loop helper

Postgres and mongodb now keep only a thin HeartbeatSender impl.
No behavior or wire-format change.

Closes Totodore#767
@codspeed-hq

codspeed-hq Bot commented Jul 31, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 87 untouched benchmarks


Comparing LiamCarPer:refactor/move-heartbeat-to-core (c0b003a) with main (144497d)

Open in CodSpeed

@LiamCarPer

Copy link
Copy Markdown
Contributor Author

Totodore I wanted to say Thanks for the patience, really appreciate you letting me take on this refactor. Happy to iterate on any feedback.

@Totodore Totodore left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for this initiative. With this work, there is one emerging issue that needs to be adressed before continuing.
Adapters are created per-namespace from an adapter constructor/adapter state.
Currently the heartbeat mechanism is bound to an adapter (per-namespace) wheras it does not rely on any per-namespace feature, it only uses the uid which is a global server id.

For this purpose it would be better to bind the heartbeat mechanism to the constructor/adapter state to avoid duplicating all this logic between every namespace.
Each adapter could then take an instance of HeartbeatTracker to access specific shared infos (like server count).

@Totodore Totodore changed the title refactor(core): move heartbeat/liveness logic into core behind remote… refactor(core): move heartbeat/liveness logic into core behind remote-adapter feature Aug 2, 2026
@LiamCarPer

Copy link
Copy Markdown
Contributor Author

Hey thanks for the review, i am going to tell you what i was thinking on doing and you can give me your nod or tell me what to change.

  • The adapter state/constructor (PostgresAdapterCtr / MongoDbAdapterCtr) owns the hearbeat, a single Arc plus one heartbeat task per server.
  • Each namespace adapter register its path with the state on init and deregisters on close, the single loop emits on every registered namespace channel each interval (and one InitHeartbeat on startup), so the wire format stays unchanged.
  • Each adapter keeps a handle to the shared tracker and uses it for server_count and recv_heartbeat, all namespaces see a consisten server count

Also, two questions:

  1. Okay to keep the per-namespace wire emits (one shared loop + namespace registry) rather than introducing a global per-node channel?
  2. Should the shared hearbeat runtime live as a reusable helper in socketioxide-core (a shared registry/heartbeat type that both constructors drive), or directly inside each constructor? I lean towards the core helper.

@Totodore

Totodore commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Okay to keep the per-namespace wire emits (one shared loop + namespace registry) rather than introducing a global per-node channel?

Could you elaborate on this? Currently the wire emits is not per namespace, the following code doesn't send the namespace:

    /// Emit an initial heartbeat to all nodes.
    fn emit_init_heartbeat(&self) -> impl Future<Output = Result<(), Self::Error>> + Send {
        async move {
            self.send_req(
                RequestOut::new_empty(self.uid(), RequestTypeOut::InitHeartbeat),
                None,
            )
            .await
        }
    }

Should the shared hearbeat runtime live as a reusable helper in socketioxide-core (a shared registry/heartbeat type that both constructors drive), or directly inside each constructor? I lean towards the core helper.

Yes, let's share logic as much as we can as long as the complexity doesn't blow up.

@LiamCarPer

Copy link
Copy Markdown
Contributor Author

The paylood is namespace agnostic, but the routing is namespace-scoped in both adapters right now, so each per namespace adapter emits the same uid hearbeat into its own namespace's scope:

  • postgres: channels are derived from the namespace path, get_global_chan() = hash(prefix#path), get_node_chan(uid) = hash(prefix#path#uid). Each namespace adapter listen()s on its own channel set and send_req(target: None) writes to its own namespace's global channel, so a heartbeat from the / adapter only reaches the / adapters of other servers.
  • mongodb: the Item carries an ns field and the driver change-stream $match filters fullDocument.ns == <path>, so each namespace adapter watches the collection filtered to its namespace.

So, we have one hearbeat loop per namespace, each sending a copy of the same uid hearbeat into its own namespace scope. Moving to a single state-bound loop (emitting on every registered namespace scope) + a shared Arc<HeartbeatTracker> removes that duplication entirely, same wire format.

One thing to confirm though, server_count is used by broadcast_with_ack, which expects server_count() - 1 ack responses from servers sharing this namespace's channels. With a uid-only shared tracker, server count becomes global, identical to today when all nodes run the same namespaces, but it would overcount for a namespace in asymmetric deployments. Is a global server count acceptable, or should the shared tracker keep liveness keyed per namespace (server_count(ns)) to stay exactly correct in that edge case? I would say uid-only tracker is good mainly for simplicity, but i wanted to confirm with you and get your feedback.

@Totodore

Totodore commented Aug 7, 2026

Copy link
Copy Markdown
Owner

You're right about the potential namespace asymmetric between each nodes. Let's stay with the current implementation then. I'll do a review as soon as I can.

@LiamCarPer

LiamCarPer commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Got it, don't worry about it, looking forward to the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move heartbeat/liveness logic into core behind the remote-adapter feature flag

2 participants