Mlx - #78
Open
dennis-akimov wants to merge 12 commits into
Open
Mlx#78dennis-akimov wants to merge 12 commits into
dennis-akimov wants to merge 12 commits into
Conversation
Stdin EOF is each worker's usual "my parent is gone" signal, but it only arrives once every holder of the pipe closes it, and an Electron helper that outlives the app inherits that descriptor. A first PAIR instance survived its own Electron, held the worker ports, and crash-looped the next instance's workers to "restart budget exhausted", which left the UI with an empty node list. Watching the original parent pid directly is what actually guarantees no orphan is left holding a port. Signed-off-by: Denis Akimov <pnmice@gmail.com>
mlx_lm.server holds exactly one model and offers no unload, so a pool of single-model child processes is what makes eviction possible at all: ending a child is the unload. A model serving a request is refcounted and never evicted. Signed-off-by: Denis Akimov <pnmice@gmail.com>
Because a node holds one model at a time, sending a request to a node that must reload first is the expensive mistake -- a measured 7.2s. Routing to an owner that already has the model resident, and falling back to on-disk owners only when nobody does, moves resident hits from 31.8% to 100.0% against the load-only arm. go test -run TestRoutingPolicyAB runs both arms back to back. Signed-off-by: Denis Akimov <pnmice@gmail.com>
mlx-proxy and mlx-pool join the supervised set, so the broker starts and stops them like every other worker, the build and installer scripts produce them, and uninstall removes them. Signed-off-by: Denis Akimov <pnmice@gmail.com>
The engine manager is manifest-driven, so MLX arrives as a manifest rather than as code: fetch uv, build a virtualenv, install mlx-lm from PyPI at a pinned version. The install is retry-safe (uv venv --clear) and writes nothing into the app bundle, and a user manifest can point the engine at a different mlx-lm without editing the bundled one. Signed-off-by: Denis Akimov <pnmice@gmail.com>
A model you quantized yourself never enters the Hugging Face cache: it has no repo id and is addressed by absolute path, which the cache transfer could not carry. Directory models now transfer over the cluster's mTLS link with every file verified against a digest before it lands, resumable, over several connections (4 by measurement: 1 stream 24 MiB/s, 4 and 8 both 40, 16 37). Signed-off-by: Denis Akimov <pnmice@gmail.com>
Node info describes the machine, so it should not answer any LAN device that asks. Readers are restricted to the addresses of nodes already in the cluster. Signed-off-by: Denis Akimov <pnmice@gmail.com>
Each invitation carries its own PIN and an inviter that cancels and retries mints a fresh one, so rebinding the modal mid-flow means the PIN being read off the other machine belongs to a session that is no longer the one about to consume it. The completion exchange then fails as an EAP-NOOB Noob mismatch, reported as "Incorrect PIN", and the user hunts for a typo that never happened. A second invite no longer repoints a modal already being answered, and both screens now show a short label derived from the invite id. Signed-off-by: Denis Akimov <pnmice@gmail.com>
PROXY_ENGINES and the ProxyEngine type both listed mlx, but isProxyEngine compared against ollama and lm-studio by hand -- a type guard's body is not checked against its own predicate, so TypeScript could not catch it. The node card sat at "Initializing..." forever. Model delete also had three dispatch paths, one of which hardcoded delete_model and could not remove a model held by path. Signed-off-by: Denis Akimov <pnmice@gmail.com>
macOS 15 gates unicast to your own subnet and all multicast behind a per-app grant, and only offers that grant to a bundle carrying a usage string. Without one there is no prompt to show, so discovery fails as EHOSTUNREACH with nothing in the UI to explain it. The grant also follows the responsible app, which is why a dev run launched from an editor is credited to the editor. Signed-off-by: Denis Akimov <pnmice@gmail.com>
Build metadata rather than a version bump, so the fork stays comparable to the upstream release it tracks. Files inherited from upstream keep NVIDIA's copyright notice as Apache-2.0 section 4 requires; files added by the fork carry their own, and the header checker accepts both. Signed-off-by: Denis Akimov <pnmice@gmail.com>
README covers only what the fork changes and points at upstream for the rest. tests/ab_bench.py measures one node, the other, then both, and attributes every response to a node by its system_fingerprint rather than assuming the routing worked. Signed-off-by: Denis Akimov <pnmice@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Scope
Validation
Risk
Checklist
git commit -s), certifying the Developer Certificate of Origin.services/versions.json, and described user-visible changes above so they reach the release notes.