Skip to content

transceivers: Implement double-polling for transceiver temperatures - #2668

Merged
jamesmunns merged 17 commits into
masterfrom
james/xcvr-doubletap
Sep 17, 2026
Merged

jamesmunns merged 17 commits into
masterfrom
james/xcvr-doubletap

Conversation

@jamesmunns

Copy link
Copy Markdown
Contributor

CC #2664 (though probably not a permanent solution).

This PR:

  1. Makes the transceiver server a static, so we can pull it from dumps
  2. Now polls each transceiver twice, discarding samples that are >= 5.0C apart
  3. Keeps stats (in the server) about each enabled port, including max sample variance and number of discarded samples

The hope for this PR is to get more trackable information and maybe catch a misbehaving I2C poll in action.

We may want to also add an ereport for this, I'd like to leave that for a follow-on PR though.

This is also currently based on #2667, we should merge that first.

This did require some refactoring of the transceiver server code, I tried to keep the diffs pretty stable and the commits pretty useful so far, if that makes things easier to review. I'll flag a couple of "wuts" I saw along the way.

The previous impl was insufficiently broad, and did not allow for types
like `core::cell::Cell` to be placed in the ClaimOnceCell. I believe
the impl was copied from StaticCell, which needs to have a more restrictive
implementation as the user is not given a full `&mut T`.

@jamesmunns jamesmunns left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some commentary.

Comment thread drv/transceivers-server/src/main.rs
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread lib/static-cell/src/lib.rs
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs
Comment thread drv/transceivers-server/src/main.rs Outdated
@jamesmunns
jamesmunns requested review from hawkw and labbott September 3, 2026 13:41
Comment thread drv/front-io-api/src/transceivers.rs Outdated
Comment on lines +142 to +156
impl Count for LogicalPort {
type Counters = [AtomicU32; NUM_PORTS as usize];

#[allow(clippy::declare_interior_mutable_const)]
const NEW_COUNTERS: Self::Counters =
[const { AtomicU32::new(0) }; NUM_PORTS as usize];

fn count(&self, counters: &Self::Counters) {
// This should never happen, but just in case.
let Some(ctr) = counters.get(self.0 as usize) else {
return;
};
ctr.fetch_add(1, Ordering::Relaxed);
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is cute; I wonder if we might want the counters crate to have a macro or something for saying "yes, this looks like a u32, but it will always be in some range, so you can derive counters for it without having to worry"; elsewhere, I've tended to just use repr(N) enums for this sort of thing, like this thing:

/// PSU numbers represented as an enum. This is intended for use with
/// `counted_ringbuf!`, instead of representing PSU numbers as raw u8s, which
/// cannot derive `counters::Count` (and would have to generate a counter table
/// with 256 entries rather than just 6).
#[derive(Copy, Clone, Eq, PartialEq, counters::Count)]
#[repr(u8)]
enum Slot {
Psu0 = 0,
Psu1 = 1,
Psu2 = 2,
Psu3 = 3,
Psu4 = 4,
Psu5 = 5,
}

but that then requires some weirdish boilerplate for converting between the enum and integers if you also want to index arrays or whatever.

very much not a blocker for this PR, but I wonder if we might throw together a little newtype-integer-counter derive or something that works like this.

Comment on lines +161 to +175
/// Keep counters for how often each port has experienced temperature glitches.
///
/// This *does not* reset when ports are disabled or qsfp xcvrs are removed or
/// re-added. We just pass-through to the existing LogicalPort impl of Count.
impl Count for TempGlitch {
type Counters = <LogicalPort as Count>::Counters;

#[allow(clippy::declare_interior_mutable_const)]
const NEW_COUNTERS: Self::Counters = <LogicalPort as Count>::NEW_COUNTERS;

#[inline]
fn count(&self, counters: &Self::Counters) {
self.port.count(counters);
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, huh, I think this is necessary because the counters crate doesn't support deriving Count for a struct, even if it has count(Children) on a field that implements Count? It could be worth adding that to better support cases like this (not in this PR, I'll go make a ticket).

Comment thread drv/transceivers-server/src/main.rs
Comment thread drv/transceivers-server/src/main.rs
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
@jamesmunns

Copy link
Copy Markdown
Contributor Author

Noting that this is still probably a thing we want to land, it's just lower prio than #2678 right now.

@jamesmunns

Copy link
Copy Markdown
Contributor Author

CC @hawkw @labbott, I have addressed all open comments, and this has run successfully overnight on berlin, I'd like to get this merged so it can be included in the next release. If I could get a review, I'd appreciate it!

@hawkw hawkw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me overall! I had some smallish docs and naming nits.

Comment thread drv/transceivers-server/src/main.rs
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment thread drv/transceivers-server/src/main.rs Outdated
Comment on lines +275 to +276
/// Like [`Self::new()`] but with the given model.
fn init(&mut self, model: Option<ThermalModel>) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unimportant nit, feel free to ignore: hm, this isn't really "like Self::new()", because it initializes self in place...

Comment thread drv/transceivers-server/src/main.rs Outdated
}

/// Error while reading temperature from QSFP transceiver
enum TempReadError {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel like some of these, in the fullness of time, should probably become ereports. not necessarily in this PR, though.

jamesmunns and others added 2 commits September 17, 2026 19:00
Co-authored-by: Eliza Weisman <eliza@elizas.website>
@jamesmunns
jamesmunns enabled auto-merge (squash) September 17, 2026 17:05
@jamesmunns
jamesmunns merged commit b6ba19b into master Sep 17, 2026
196 checks passed
@jamesmunns
jamesmunns deleted the james/xcvr-doubletap branch September 17, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants