Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/compass-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ jobs:
"dist/verify/$name/${{ matrix.binary }}" query --help
test -f "dist/verify/$name/LICENSE-MIT"
test -f "dist/verify/$name/LICENSE-APACHE"
test -f "dist/verify/$name/LICENSE-BOOST"
test -f "dist/verify/$name/THIRD_PARTY_NOTICES.md"
(
cd dist
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
cached and OCR-derived artifact coherence, bound aggregate raster work and
per-tile deadlines, and serialize model installation while rejecting symlinked
model artifacts and markers.
Preserve Intel macOS builds by excluding the unavailable ONNX runtime on that
target; native document processing remains available and managed OCR reports
an explicit unsupported-platform error before downloading model weights.

- Hard-cut Swift, Dart, Scala, and Groovy/Gradle onto version-1 qualifying
universal evidence pipelines. The bounded AST-first producer publishes
Expand Down
6 changes: 6 additions & 0 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@ by source digest, schema, normalizer, rasterizer, OCR policy, preprocessing,
profile manifest/model digests, and languages. An incompatible cache entry is
a miss or explicit corruption error, never a fallback to flattened text.

Managed OCR is unavailable on Intel (`x86_64`) macOS because the pinned ONNX
Runtime distribution has no self-contained build for that target. Compass
therefore omits the OCR runtime dependency on Intel macOS instead of requiring
a system ONNX installation. Native document processing and `--ocr off` remain
fully available; `models install` and OCR-enabled processing fail explicitly.

The selected OCR identity is included in graph build and immutable history
profiles. Native text remains authoritative; OCR is additive derived evidence
with exact source owner, geometry, confidence, and model provenance. Partial
Expand Down
23 changes: 23 additions & 0 deletions LICENSE-BOOST
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Boost Software License - Version 1.0 - August 17th, 2003

Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
5 changes: 3 additions & 2 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
Compass links OAR-OCR 0.9.2 and its `ort`/ONNX Runtime integration for optional
local OCR. OAR-OCR is Apache-2.0; the Rust `ort` crates are MIT or Apache-2.0.
Compass also links Hayro 0.7.1 for pure-Rust PDF rendering under MIT or
Apache-2.0. The corresponding license texts are covered by `LICENSE-MIT` and
`LICENSE-APACHE` in release bundles.
Apache-2.0. OAR-OCR's text-region geometry stack links clipper2-rust 1.1.0
under BSL-1.0. The corresponding license texts are covered by `LICENSE-MIT`,
`LICENSE-APACHE`, and `LICENSE-BOOST` in release bundles.

The separately installed `pp-ocrv6-small` and `pp-ocrv6-medium` model files
come from the immutable GreatV/OAR-OCR `v0.7.0` release and originate from the
Expand Down
47 changes: 37 additions & 10 deletions crates/compass-cli/tests/document_cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,14 @@ fn explicit_ocr_missing_profile_has_one_actionable_command() -> Result<(), Box<d
assert!(!output.status.success());
assert!(output.stdout.is_empty());
let stderr = String::from_utf8(output.stderr)?;
let command = "compass models install pp-ocrv6-small";
assert_eq!(stderr.matches(command).count(), 1, "{stderr}");
assert!(stderr.contains("no system OCR package is required"));
if cfg!(all(target_os = "macos", target_arch = "x86_64")) {
assert!(stderr.contains("managed local OCR is unavailable on Intel macOS"));
assert!(stderr.contains("native PDF, DOCX, PPTX, and XLSX processing remains available"));
} else {
let command = "compass models install pp-ocrv6-small";
assert_eq!(stderr.matches(command).count(), 1, "{stderr}");
assert!(stderr.contains("no system OCR package is required"));
}
Ok(())
}

Expand Down Expand Up @@ -136,13 +141,35 @@ fn extract_ocr_uses_the_same_managed_profile_contract() -> Result<(), Box<dyn Er
assert!(!output.status.success());
assert!(output.stdout.is_empty());
let stderr = String::from_utf8(output.stderr)?;
assert_eq!(
stderr
.matches("compass models install pp-ocrv6-small")
.count(),
1
);
assert!(stderr.contains("no system OCR package is required"));
if cfg!(all(target_os = "macos", target_arch = "x86_64")) {
assert!(stderr.contains("managed local OCR is unavailable on Intel macOS"));
assert!(stderr.contains("native PDF, DOCX, PPTX, and XLSX processing remains available"));
} else {
assert_eq!(
stderr
.matches("compass models install pp-ocrv6-small")
.count(),
1
);
assert!(stderr.contains("no system OCR package is required"));
}
Ok(())
}

#[cfg(all(target_os = "macos", target_arch = "x86_64"))]
#[test]
fn model_install_rejects_unsupported_runtime_before_download() -> Result<(), Box<dyn Error>> {
let directory = tempfile::tempdir()?;
let cache = directory.path().join("models");
let output = Command::new(env!("CARGO_BIN_EXE_compass"))
.args(["models", "install", "pp-ocrv6-small"])
.env("COMPASS_CACHE_DIR", &cache)
.output()?;
assert!(!output.status.success());
assert!(output.stdout.is_empty());
let stderr = String::from_utf8(output.stderr)?;
assert!(stderr.contains("managed local OCR is unavailable on Intel macOS"));
assert!(!cache.exists(), "unsupported model install created a cache");
Ok(())
}

Expand Down
15 changes: 7 additions & 8 deletions crates/compass-core/src/task_context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -230,19 +230,18 @@ impl TaskContext {
self.work.response_bytes
)));
}
if let Some(agent) = &self.agent_knowledge {
if agent.schema != "compass.agent-knowledge/1"
if let Some(agent) = &self.agent_knowledge
&& (agent.schema != "compass.agent-knowledge/1"
|| agent.effective_identity.as_str() != self.graph_identity
|| agent
.assertions
.len()
.saturating_add(agent.challenges.len())
> MAX_KNOWLEDGE_ITEMS as usize
{
return Err(TaskContextError::InvalidResult(
"Agent knowledge identity, schema, or record bound is invalid".to_owned(),
));
}
> MAX_KNOWLEDGE_ITEMS as usize)
{
return Err(TaskContextError::InvalidResult(
"Agent knowledge identity, schema, or record bound is invalid".to_owned(),
));
}
Ok(())
}
Expand Down
4 changes: 3 additions & 1 deletion crates/compass-ocr/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,15 @@ categories.workspace = true

[dependencies]
image.workspace = true
oar-ocr.workspace = true
serde.workspace = true
serde_json.workspace = true
sha2.workspace = true
tempfile.workspace = true
thiserror.workspace = true
ureq.workspace = true

[target.'cfg(not(all(target_os = "macos", target_arch = "x86_64")))'.dependencies]
oar-ocr.workspace = true

[lints]
workspace = true
90 changes: 65 additions & 25 deletions crates/compass-ocr/src/engine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,22 @@ use std::io::Cursor;
use std::sync::atomic::{AtomicBool, Ordering};

use image::{DynamicImage, ImageDecoder, ImageReader, RgbImage, imageops::FilterType};
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
use oar_ocr::core::config::OrtSessionConfig;
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
use oar_ocr::oarocr::OAROCRBuilder;
use sha2::{Digest, Sha256};

use crate::models::{ModelProfile, verify_profile};
use crate::models::ModelProfile;
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
use crate::models::verify_profile;
use crate::{
OCR_ENGINE_MAX_SIDE, OCR_ENGINE_THREADS, OCR_MAX_OBSERVATIONS_PER_RASTER,
OCR_MAX_RASTER_LONG_EDGE, OCR_MAX_RASTER_PIXELS, OCR_SCHEMA, OCR_TILE_OVERLAP, OcrError,
OcrObservation, OcrPoint, OcrRequest, OcrResponse,
OCR_ENGINE_MAX_SIDE, OCR_MAX_RASTER_LONG_EDGE, OCR_MAX_RASTER_PIXELS, OCR_TILE_OVERLAP,
OcrError, OcrRequest, OcrResponse,
};
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
use crate::{
OCR_ENGINE_THREADS, OCR_MAX_OBSERVATIONS_PER_RASTER, OCR_SCHEMA, OcrObservation, OcrPoint,
};

#[derive(Clone, Debug)]
Expand Down Expand Up @@ -53,35 +60,40 @@ pub trait OcrEngine {
}

pub struct ManagedOarEngine {
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
runtime: oar_ocr::oarocr::OAROCR,
profile: crate::OcrProfileIdentity,
}

impl ManagedOarEngine {
pub fn load(profile: ModelProfile) -> Result<Self, OcrError> {
let files = verify_profile(profile)?;
let session = OrtSessionConfig::default()
.with_intra_threads(OCR_ENGINE_THREADS)
.with_inter_threads(OCR_ENGINE_THREADS);
let runtime = OAROCRBuilder::new(&files.detector, &files.recognizer, &files.dictionary)
.ort_session(session)
.image_batch_size(1)
.region_batch_size(4)
.build()
.map_err(|error| OcrError::EngineUnavailable(error.to_string()))?;
Ok(Self {
runtime,
profile: files.identity,
})
}
}

impl OcrEngine for ManagedOarEngine {
fn identity(&self) -> &crate::OcrProfileIdentity {
&self.profile
crate::ensure_managed_runtime_available()?;
#[cfg(all(target_os = "macos", target_arch = "x86_64"))]
{
let _ = profile;
Err(crate::managed_runtime_unavailable_error())
}
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
{
let files = verify_profile(profile)?;
let session = OrtSessionConfig::default()
.with_intra_threads(OCR_ENGINE_THREADS)
.with_inter_threads(OCR_ENGINE_THREADS);
let runtime = OAROCRBuilder::new(&files.detector, &files.recognizer, &files.dictionary)
.ort_session(session)
.image_batch_size(1)
.region_batch_size(4)
.build()
.map_err(|error| OcrError::EngineUnavailable(error.to_string()))?;
Ok(Self {
runtime,
profile: files.identity,
})
}
}

fn recognize(
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
fn recognize_with_runtime(
&self,
request: &OcrRequest,
raster: &PreparedRaster,
Expand Down Expand Up @@ -166,6 +178,28 @@ impl OcrEngine for ManagedOarEngine {
response.validate_for(request)?;
Ok(response)
}
}

impl OcrEngine for ManagedOarEngine {
fn identity(&self) -> &crate::OcrProfileIdentity {
&self.profile
}

fn recognize(
&self,
request: &OcrRequest,
raster: &PreparedRaster,
) -> Result<OcrResponse, OcrError> {
#[cfg(all(target_os = "macos", target_arch = "x86_64"))]
{
let _ = (request, raster);
Err(crate::managed_runtime_unavailable_error())
}
#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
{
self.recognize_with_runtime(request, raster)
}
}

fn recognize_cancellable(
&self,
Expand Down Expand Up @@ -327,6 +361,7 @@ pub fn prepare_raster_cancellable(
})
}

#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
fn geometry_key(points: &[oar_ocr::processors::Point]) -> (u32, u32) {
let min_y = points
.iter()
Expand All @@ -341,6 +376,7 @@ fn geometry_key(points: &[oar_ocr::processors::Point]) -> (u32, u32) {
(sortable_float(min_y), sortable_float(min_x))
}

#[cfg(not(all(target_os = "macos", target_arch = "x86_64")))]
fn sortable_float(value: f32) -> u32 {
if !value.is_finite() || value <= 0.0 {
0
Expand All @@ -349,6 +385,10 @@ fn sortable_float(value: f32) -> u32 {
}
}

#[cfg(any(
not(all(target_os = "macos", target_arch = "x86_64")),
test
))]
fn quantize_coordinate(value: f32, bound: u32) -> Result<u32, OcrError> {
if !value.is_finite() || value < 0.0 || value > bound as f32 || bound == 0 {
return Err(OcrError::InvalidOutput(
Expand Down
32 changes: 32 additions & 0 deletions crates/compass-ocr/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,30 @@ pub const OCR_MAX_TEXT_CHARS_PER_DOCUMENT: usize = 5_000_000;
pub const OCR_MAX_LANGUAGE_HINTS: usize = 32;
pub const OCR_MAX_PROFILE_FIELD_BYTES: usize = 256;

fn managed_runtime_supported_for(target_os: &str, target_arch: &str) -> bool {
target_os != "macos" || target_arch != "x86_64"
}

#[must_use]
pub fn managed_runtime_available() -> bool {
managed_runtime_supported_for(std::env::consts::OS, std::env::consts::ARCH)
}

pub(crate) fn ensure_managed_runtime_available() -> Result<(), OcrError> {
if managed_runtime_available() {
Ok(())
} else {
Err(managed_runtime_unavailable_error())
}
}

pub(crate) fn managed_runtime_unavailable_error() -> OcrError {
OcrError::EngineUnavailable(
"managed local OCR is unavailable on Intel macOS because the pinned ONNX Runtime does not provide a self-contained x86_64 macOS build; native PDF, DOCX, PPTX, and XLSX processing remains available with OCR off"
.to_owned(),
)
}

pub fn normalize_language_hints(hints: &[String]) -> Result<Vec<String>, OcrError> {
if hints.len() > OCR_MAX_LANGUAGE_HINTS {
return Err(OcrError::InvalidRequest(
Expand Down Expand Up @@ -538,4 +562,12 @@ mod tests {
assert!(validate_dimensions(6_000, 4_001).is_err());
assert!(validate_dimensions(6_001, 1).is_err());
}

#[test]
fn managed_runtime_support_matrix_excludes_intel_macos() {
assert!(!managed_runtime_supported_for("macos", "x86_64"));
assert!(managed_runtime_supported_for("macos", "aarch64"));
assert!(managed_runtime_supported_for("linux", "x86_64"));
assert!(managed_runtime_supported_for("windows", "x86_64"));
}
}
1 change: 1 addition & 0 deletions crates/compass-ocr/src/models.rs
Original file line number Diff line number Diff line change
Expand Up @@ -527,6 +527,7 @@ fn expected_verified_marker(profile: ModelProfile) -> VerifiedProfileMarker {
}

pub fn install_profile(profile: ModelProfile) -> Result<ModelFiles, OcrError> {
crate::ensure_managed_runtime_available()?;
ModelCache::from_environment()?.install(profile, &HttpsArtifactFetcher::default())
}

Expand Down
5 changes: 5 additions & 0 deletions crates/compass-output/src/workbench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,10 @@ pub struct WorkbenchView {
rename_all = "snake_case",
rename_all_fields = "camelCase"
)]
// Keep the established public model types and serialized contract intact. The
// effective-graph context can make history the largest variant, but boxing it
// here would be a source-breaking public API change.
#[allow(clippy::large_enum_variant)]
pub enum WorkbenchViewContent {
Code {
model: GraphViewModel,
Expand Down Expand Up @@ -380,6 +384,7 @@ mod tests {
edges: Vec::new(),
communities: Vec::new(),
hyperedges: Vec::new(),
effective_graph: None,
},
community_details: BTreeMap::new(),
},
Expand Down
Loading
Loading