Skip to content

Move onde-mistralrs to 0.9.5 and release 1.3.4 - #32

Merged
setoelkahfi merged 3 commits into
mainfrom
feature/sync-mistralrs-0.9.5
Oct 30, 2026
Merged

setoelkahfi merged 3 commits into
mainfrom
feature/sync-mistralrs-0.9.5

Conversation

@setoelkahfi

Copy link
Copy Markdown
Collaborator

Moves every target from onde-mistralrs{,-core} 0.9.3 to 0.9.5 and bumps the SDK to 1.3.4. No public API changes.

The 0.9.5 crates are published on crates.io. They merge upstream mistral.rs 0.9.5 into the fork and add visionOS and watchOS arms to the Metal embedded_library! macro, so --features metal builds for tvOS, visionOS and watchOS again.

Upstream made Metal paged attention the default, and its KV block pool does not fit in memory on iOS, tvOS, visionOS or watchOS. Model loading in engine.rs and the validation load in hf_cache.rs now call without_paged_attn() on those targets. On tvOS the auto device map override is replaced by with_max_model_len(512). Desktop and Android are unchanged.

The branch has two commits:

  • the dependency move and the mobile paged-attention change
  • the 1.3.4 version bump across the root crate, Kotlin, Dart, React Native, their lockfiles and the changelogs

Checked against the published crates, with no "patch not used" warnings:

  • host: cargo check, cargo clippy -- -D warnings, cargo test inference:: (46 passed), cargo check --workspace
  • cargo check for iOS, iOS-sim and Android
  • cargo +nightly check -Z build-std for tvOS, visionOS and watchOS

cargo test hf_cache:: runs no tests because that module has none.

sdk/dart/rust/Cargo.lock and sdk/react-native/rust/Cargo.lock still pin onde-mistralrs at 0.9.3. Cargo ignores them in workspace mode and they were already stale, so I left them alone.

Not included: the 1.3.4 tag, which triggers the five publish workflows.

setoelkahfi and others added 2 commits October 30, 2026 11:45
Point every target at the published onde-mistralrs{,-core} 0.9.5 crates
instead of 0.9.3. Cargo.lock now resolves them from crates.io, with a
single onde-candle-core 0.11.0 and mio 1.2.3.

Upstream #2493 made Metal paged attention the default. Its KV block pool
does not fit in memory on iOS, tvOS, visionOS or watchOS, so both load
paths in engine.rs and the validation load in hf_cache.rs now call
without_paged_attn() on those targets. On tvOS, the auto device map
override is replaced by with_max_model_len(512).

Update the forked-dependencies and sdk-regressions skills with the
publish order and the patch-by-package-name trick for testing unpublished
fork changes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Bump the root crate, the Kotlin, Dart and React Native packages, and the
root and per-SDK lockfiles from 1.3.3 to 1.3.4. Add changelog entries for
the onde-mistralrs 0.9.5 move and the mobile paged-attention change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@setoelkahfi setoelkahfi self-assigned this Oct 30, 2026

@sigit-code-review sigit-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change updates all targets to use onde-mistralrs 0.9.5 (forked from upstream mistral.rs 0.9.5), adds visionOS and watchOS Metal support, and disables paged attention on iOS, tvOS, visionOS, and watchOS to avoid memory issues. On tvOS, the device mapping override is replaced with with_max_model_len(512) due to an upstream API change. The version is bumped to 1.3.4 across the root crate, Kotlin, Dart, React Native, and changelogs. Documentation is updated to reflect the new dependency versions and behavioral changes. No public API changes are introduced. The changes are well-documented and the migration to the new paged attention default is handled with target-specific cfg guards. No findings from previous reviews remain unresolved, and the update appears to be robust and well-tested as described.


Automated review by siGit Code Review · commit 77523af · see the review dashboard

The release preflight runs make verify, which checks that
sdk/react-native/package-lock.json agrees with Cargo.toml. The 1.3.4
bump edited the other manifests by hand and missed it, so preflight
failed on both version fields. Running scripts/sync-release-version.py
updates the lock and nothing else.

Co-Authored-By: OndeCode v1.0.3-acp <noreply@ondeinference.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@sigit-code-review sigit-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change bumps the onde SDK and all wrappers to 1.3.4 and moves all targets from onde-mistralrs 0.9.3 to 0.9.5, syncing with upstream and adding visionOS/watchOS Metal support. It disables paged attention on iOS, tvOS, visionOS, and watchOS to avoid memory issues, and replaces the tvOS device mapping override with with_max_model_len(512) per the new API. The changelogs and all version sources are updated. No public API changes are introduced. The change is internally consistent and matches the described migration and upstream changes. No critical or warning-level issues are introduced by the diff.


Automated review by siGit Code Review · commit 3f2bc56 · see the review dashboard

@setoelkahfi
setoelkahfi merged commit 778c5a5 into main Oct 30, 2026
38 checks passed
@setoelkahfi
setoelkahfi deleted the feature/sync-mistralrs-0.9.5 branch October 31, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants