[Bug]: WiFi saturation on wireless host node / WLAN driver deadlock

[pm974]

PAIR version or commit

0.1.1 (0.1.1.463)

Affected component

Desktop application

Environment

OS: macOS 26.6.2 (25G83) on affected node (wireless host); peers run macOS (current)
Architecture: arm64 (Apple Silicon, Mac16,8) on all nodes
GPU and driver: Affected node — Apple Silicon integrated GPU (no NVIDIA GPU). WLAN driver: com.apple.DriverKit-AppleBCMWLAN (system DriverKit extension). Peers — Apple Silicon integrated GPUs.
Engine and version: PAIR 0.1.1 (0.1.1.463, com.nvidia.nvpair); engines: LM Studio 0.4.24 (headless llmster daemon) + Ollama on all nodes
Model: N/A on affected node (no model loaded); issue occurs with cluster idle — no inference running
Cluster size: 5 nodes (4 on wired Ethernet, 1 on WiFi — the affected host is the only wireless node)

Steps to reproduce

1. Set up a 5-node PAIR cluster: 4 nodes on Ethernet, 1 node on WiFi. Pair and trust all nodes.
2. Leave the cluster idle (no model loads, no jobs).
3. On the WiFi node, launch PAIR and leave it running for 8–10 minutes.
4. From the WiFi node, ping the local gateway continuously (1/s) while PAIR runs.
5. Observe latency for ~10 minutes, then quit PAIR and keep pinging.

Expected behavior

The wireless host's network stays healthy while PAIR runs: gateway latency stays in the single-digit
milliseconds with 0% loss, on any SSID. Cluster background traffic (discovery, sync, health polls)
should not meaningfully degrade the host's connectivity.

Actual behavior

After ~8 minutes of PAIR running on the wireless host, gateway latency explodes in continuous bursts
regardless of which SSID is selected:

Healthy (PAIR just launched): 3–12 ms, 0% loss
Degraded (after ~8 min): bursts up to 1,441 ms (also 1,169 / 1,118 / 1,085 / 1,048 ms...),
internet pings degrade in mirror -> bottleneck is the local link
Quit PAIR: instant recovery to 3–7 ms, 0% loss (reproduced twice)

In a worst case the same day, the WLAN driver itself wedged and was killed by macOS:
com.apple.DriverKit-AppleBCMWLAN crashed (SIGABRT, thread "WLANDextWatchdogDQ") with stack
IO80211DeadlockPolice::handleDextWatchdogTimer() -> DriverKit panic -> abort() —
a hard WiFi outage until the dext restarted. This was the only WLAN crash in 72h and never
reproduced outside PAIR execution.

Sanitized logs or screenshots

1) Ping timeline (gateway, 1/s, local times)
   11:48–11:55  PAIR running, healthy: 3–12 ms
   11:56–11:58  PAIR running, degraded: sustained bursts 300–1,441 ms; 0 timeouts
   11:58        PAIR quit -> recovery within one ping sample

2) PAIR cluster traffic during the degraded window (from nvpair.jsonl, 5 minutes):
   ~750 "cluster:trust-changed" events
   ~165 remote engine-status polls
   ~38 mDNS discovery cycles
   i.e. cluster writes fan-out to 4 peers every ~500 ms + multicast discovery, with
   zero PAIR-side errors (peers kept responding — the radio link itself saturated).

3) WLAN driver crash report (worst case, same day):
   Process:      com.apple.DriverKit-AppleBCMWLAN (system DriverKit extension)
   Termination:  SIGABRT (abort() called), thread "WLANDextWatchdogDQ"
   Crash stack (top frames):
     libsystem_kernel.dylib __pthread_kill
     libsystem_pthread.dylib pthread_kill
     libsystem_c.dylib abort
     DriverKit panic
     IO80211DriverKit IO80211DeadlockPolice::handleDextWatchdogTimer() (.cold.1)
     IO80211DriverKit IO80211DeadlockPolice::handleDextWatchdogTimer()
     IO80211DriverKit IO80211DQTimer::TimeOutOccurred_Impl(OSAction*, unsigned long long)
     DriverKit IOTimerDispatchSource::TimerOccurred_Invoke(...)
   Last DISASSOC logged ~35 s after crash capture; cascade of re-associations followed.

4) Ruled out:
   - PAIR installs no kernel/network system extension (pure userspace: app + launchd helper
     listening on no port + userspace proxies). Helper CPU ~0% at rest.
   - No interface errors on the wireless interface all day (0 Ierrs/Oerrs).
   - Issue does not occur on the 4 wired nodes (identical PAIR versions).

Confirmations

• I searched existing issues for duplicates.
• This is not a security vulnerability.
• I agree to follow the Code of Conduct.

[sherief-nv]

Thanks for reporting this - are there any other interesting logs from Console that you can share? Please be sure to scrub any PII beforehand.

[stevbev]

I have the almost exact same issue on one MacBook Pro and no issue on two other MacBook Pro. All of my MacBooks are connected via WiFi, no ethernet. The MBP encountering the issue results in severe network stack issues within minutes of opening PAIR, and has 3 times now resulted in an OS crash and restart.

I don't know how pertinent this detail, but every time I launch PAIR, I receive this message in a pop-up message:

Personal AI Router needs administrator permission to complete setup. macOS will ask you to approve a background helper that configures the firewall for local AI traffic. You only need to do this once.

The problem is that no dialog window ever shows up to approve like I experience with other apps. Perhaps this provides a hint towards the source of the problem. Is there something in PAIR that is running in a loop that is trying to change network and/or firewall settings?

I wonder if there is also an Apple driver bug that PAIR is triggering because I think a user land application should not be able to make the OS perform like this. In that spirit, here are some additional details from my setup taken from System Information app. I will work more on detailed performance metrics and logging to see if I can contribute data that can be evaluated.

Device exhibiting extreme network stack problems:

System Version:	macOS 26.6.2 (25G83)
Model Identifier:	Mac14,6
Model Number:	Z176000HCLL/A
Chip:	Apple M2 Max
Total Number of Cores:	12 (8 Performance and 4 Efficiency)
Memory:	64 GB
Card Type:	Wi-Fi  (0x14E4, 0x4388)
Firmware Version:	wl0: Jul 10 2026 02:04:28 version 23.50.20.2.41.51.209 FWID 01-b35f0cf5 IO80211_driverkit-1566.5 "IO80211_driverkit-1566.5" Jul 31 2026 19:08:25

Devices with no apparent issues:

System Version:	macOS 26.6.2 (25G83)
Model Identifier:	MacBookPro18,1
Model Number:	Z14W00105LL/A
Chip:	Apple M1 Pro
Total Number of Cores:	10 (8 Performance and 2 Efficiency)
Memory:	32 GB
Card Type:	Wi-Fi  (0x14E4, 0x4387)
Firmware Version:	wl0: Jan 20 2026 17:44:15 version 20.140.13.0.8.7.224 FWID 01-b2a3fd0d IO80211_driverkit-1566.5 "IO80211_driverkit-1566.5" Jul 31 2026 19:08:25

System Version:	macOS 26.6.2 (25G83)
Model Identifier:	MacBookPro17,1
Model Number:	Z11F000E4LL/A
Chip:	Apple M1
Total Number of Cores:	8 (4 Performance and 4 Efficiency)
Memory:	16 GB
Card Type:	Wi-Fi  (0x14E4, 0x4378)
Firmware Version:	wl0: May  9 2026 17:21:43 version 18.110.6.1.7.8.190 FWID 01-3e596a59 IO80211_driverkit-1566.5 "IO80211_driverkit-1566.5" Jul 31 2026 19:08:25

[sherief-nv]

Thank you so much - agree with you that no user mode app should be able to cause this driver behavior, but pragmatically we should investigate how we can improve the experience for end users. Please share as much information as you can, I'll work on trying to prevent what's causing this and communicating the issue to the vendor once I have a solid repro.

[stevbev]

Through a lot of log diving, environment testing, AI assistant, and luck, I believe I have found the core contributing factors and a few items that I consider bugs that led to the discovery. The hints arrived on the computer having the issue:

• the firewall was turned on
• Chrome was actively used
• the critical bit is that Chrome is allowed to search the local network for nearby devices (using mDNS).

The systems where the problem did not manifest do not have the firewall turned on, and perhaps more importantly, do not normally have Chrome running and were not running Chrome during the investigation.

The root causes

Primary issue 1

The first major issue appears to be caused in socketreuse_unix.go where the mDNS service is binding to UDP:5353 without using SO_REUSEPORT. On macOS there are multiple apps and services that need to use port UDP:5353, including the system mDNSResponder and Chrome's mDNS service. The port must bind reusable in macOS.

The hint came from the firewall logs showing thousands of log entries that indicated the socket was being repeated opened every second. As I observed the socket use over longer periods of time (30-90 minutes), I discovered a race condition between Chrome and PAIR, with Chrome trying to open a shared socket for UDP:5353 and PAIR trying to open an exclusive socket for UDP:5353. When Chrome is open along with PAIR, it is just a matter of time waiting for the race condition to occur and then the entire system starts misbehaving very rapidly where network latency spikes heavily and Chrome becomes severely degraded in responsiveness and ability to load webpages.

What I believe is the conclusive evidence that this is a major root issue is that using the lsof command or the Sloth app, there is a single socket listing for port 5353 by nvpair-node-scanner when the issue is not severe. However during the instance that I observed in detail, there were ~50 entries under Chrome and ~5 entries for nvpair-node-scanner for port 5353.

Primary issue 2

The mDNS implementation opens a new socket on every scan, which occurs every 5 seconds, but does not seem to close the socket. This causes the kernel to maintain a lengthly list of sockets for nvpair-node-scanner. The macOS kernel maintaining that many open sockets manifests as network latency trying to process each packet through the network stack until the degradation is so severe that userland and system services are unable to complete network transactions within the deadlock period, resulting in application crashes, and if left running long enough, system crashes. A secondary observation method is to allow PAIR to run for a while (at least 15 minutes), run a ping to a local network target (like the gateway/router), terminate/quit PAIR, and watch all network traffic fail/timeout while the kernel is unwinding all the sockets opened by PAIR.

Secondary issue 1

Unix based systems suffer from a double broadcast bug caused by the Windows workaround to guarantee a broadcast running on non-Windows devices. Feature gating the Windows workaround to Windows systems resolves the double broadcast problem.

The fix

#72

• Replace the mDNS implementation using grandcat/zeroconf with a fully local mDNS solution that opens a single, reusable socket for the lifespan of the application execution instead of a new socket every 5 seconds.
• Feature gate the Windows broadcast workaround to just Windows systems.

Supplemental details and logs

Firewall logs

The firewall logs (obtained from macOS Console app) show ~4,500 of these log entries over a 10 minute period.

socketfilterfw | Handle flow with processPID: 95625 and bundleid: nvpair-node-scanner
socketfilterfw | KNOWN APP FLOW: file:///Applications/PAIR.app/Contents/Resources/cli-bin/nvpair-node-scanner, return known verdict: 1

These messages occur because nvidia-node-scanner appears to open a new socket for every mDNS broadcast, instead of opening one socket and reusing it for the life of the execution.

The thousands of entries in the logs over 10 minutes appears to be a broadcast storm at least partially caused by the ungated Windows workaround is running on non-Windows systems.

Here is a spreadsheet containing ping output and macOS Console logs to help describe/demonstrate the conditions when the issue is in effect.
Network_latency_PAIR.xlsx

[stevbev]

@pm974 I would be very interested if you are able to build/run my proposed fix to see if this also resolves the network stack latency problem for you. I installed this on my 4-node PAIR cluster and everything seems to be working fine now.

https://github.com/stevbev/Personal-AI-Router/tree/native-mdns

The command to build the app packages for the different OS are:

cd desktop
npm run build:mac:arm64
npm run build:win:x64
npm run build:linux:x64

Then find the installers in desktop/release/0.1.2/ to install on your devices. If you cannot build these and are willing to extend me some trust, I can build and publish these somewhere so you can test to see if this resolves your issue like it has on my system.