Repository navigation
Move onde-mistralrs to 0.9.5 and release 1.3.4 - #32
Conversation
Point every target at the published onde-mistralrs{,-core} 0.9.5 crates
instead of 0.9.3. Cargo.lock now resolves them from crates.io, with a
single onde-candle-core 0.11.0 and mio 1.2.3.
Upstream #2493 made Metal paged attention the default. Its KV block pool
does not fit in memory on iOS, tvOS, visionOS or watchOS, so both load
paths in engine.rs and the validation load in hf_cache.rs now call
without_paged_attn() on those targets. On tvOS, the auto device map
override is replaced by with_max_model_len(512).
Update the forked-dependencies and sdk-regressions skills with the
publish order and the patch-by-package-name trick for testing unpublished
fork changes.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
This change updates all targets to use onde-mistralrs 0.9.5 (forked from upstream mistral.rs 0.9.5), adds visionOS and watchOS Metal support, and disables paged attention on iOS, tvOS, visionOS, and watchOS to avoid memory issues. On tvOS, the device mapping override is replaced with with_max_model_len(512) due to an upstream API change. The version is bumped to 1.3.4 across the root crate, Kotlin, Dart, React Native, and changelogs. Documentation is updated to reflect the new dependency versions and behavioral changes. No public API changes are introduced. The changes are well-documented and the migration to the new paged attention default is handled with target-specific cfg guards. No findings from previous reviews remain unresolved, and the update appears to be robust and well-tested as described.
Automated review by siGit Code Review · commit 77523af · see the review dashboard
The release preflight runs make verify, which checks that sdk/react-native/package-lock.json agrees with Cargo.toml. The 1.3.4 bump edited the other manifests by hand and missed it, so preflight failed on both version fields. Running scripts/sync-release-version.py updates the lock and nothing else. Co-Authored-By: OndeCode v1.0.3-acp <noreply@ondeinference.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
This change bumps the onde SDK and all wrappers to 1.3.4 and moves all targets from onde-mistralrs 0.9.3 to 0.9.5, syncing with upstream and adding visionOS/watchOS Metal support. It disables paged attention on iOS, tvOS, visionOS, and watchOS to avoid memory issues, and replaces the tvOS device mapping override with with_max_model_len(512) per the new API. The changelogs and all version sources are updated. No public API changes are introduced. The change is internally consistent and matches the described migration and upstream changes. No critical or warning-level issues are introduced by the diff.
Automated review by siGit Code Review · commit 3f2bc56 · see the review dashboard
Moves every target from
onde-mistralrs{,-core}0.9.3 to 0.9.5 and bumps the SDK to 1.3.4. No public API changes.The 0.9.5 crates are published on crates.io. They merge upstream mistral.rs 0.9.5 into the fork and add visionOS and watchOS arms to the Metal
embedded_library!macro, so--features metalbuilds for tvOS, visionOS and watchOS again.Upstream made Metal paged attention the default, and its KV block pool does not fit in memory on iOS, tvOS, visionOS or watchOS. Model loading in
engine.rsand the validation load inhf_cache.rsnow callwithout_paged_attn()on those targets. On tvOS the auto device map override is replaced bywith_max_model_len(512). Desktop and Android are unchanged.The branch has two commits:
Checked against the published crates, with no "patch not used" warnings:
cargo check,cargo clippy -- -D warnings,cargo test inference::(46 passed),cargo check --workspacecargo checkfor iOS, iOS-sim and Androidcargo +nightly check -Z build-stdfor tvOS, visionOS and watchOScargo test hf_cache::runs no tests because that module has none.sdk/dart/rust/Cargo.lockandsdk/react-native/rust/Cargo.lockstill pinonde-mistralrsat 0.9.3. Cargo ignores them in workspace mode and they were already stale, so I left them alone.Not included: the 1.3.4 tag, which triggers the five publish workflows.