fix(safetensors): preserve native FDICT-shaped headers

[mldangelo-oai]

Valid SafeTensors files with header lengths such as 8,312 bytes start with bytes that also resemble a preset-dictionary zlib stream. Keep native SafeTensors routing when the bounded header parses successfully, so these files receive normal metadata and tensor validation. Unknown extensions, inconclusive headers, and genuine compressed files retain compression routing.

Supersedes #1859, preserving the contributor's commits, synchronizing with main, removing unrelated dependency changes, and expanding regression coverage. Fixes #1854.

Validation:

• Ruff formatting/lint and CI-matched mypy pass.
• 19 focused routing cases pass, including all five reported header sizes, malicious metadata, benign metadata, genuine dictionary-compressed input, and oversized/deep headers.
• The installed SafeTensors reader accepts all five synthetic format fixtures.
• The fast suite reached 14,847 passed and 13 skipped before the existing macOS shard-pinning failure. That failure also reproduces on main and outside the filesystem sandbox; it occurs before metadata analysis. Native pickle scanning was rebuilt from the current source before this run.

Current-base verification (2026-09-30): head 25a8996066150e23c17ff63a375ba1d0823f9c48 merges cleanly with main e32b430ed2e72c0b0942246634f750af65c818e0. The temporary merged tree passed all 18 FDICT/zlib routing tests, repository-wide Ruff 0.16.9 checks, and mypy 2.3.1 (480 files). The temporary merge was removed after testing; the published head is unchanged and has 23 successful checks plus eight intentional skips. No unresolved review threads were found; required reviewer approval is still pending.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-03T19:06:00.928371Z	6c4eccd	New commits
🔒 Security Review	✅ Completed 2026-10-03T19:06:55.387248Z	6c4eccd	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

Workflow run and artifacts

Performance Benchmarks

Compared 13 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 13 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 2.370s -> 2.397s (+1.1%).

Workload	Benchmark	Target	Size	Files	Baseline	Current	Change	Status
clean-training-checkpoint	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint	safe_large	278.2 KiB	1	71.70ms	80.06ms	+11.7%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64]	nested_base64	98 B	1	128.0us	141.8us	+10.8%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw]	nested_raw	78 B	1	115.3us	126.3us	+9.6%	stable
direct-malicious-upload	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload	malicious_reduce	52 B	1	92.3us	100.7us	+9.1%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex]	nested_hex	130 B	1	134.4us	145.3us	+8.1%	stable
padded-multi-stream-upload	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload	multi_stream_padded	4.1 KiB	1	144.5us	154.9us	+7.2%	stable
chunked-upload-stream	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream	chunked_stream	278.2 KiB	1	73.41ms	77.15ms	+5.1%	stable
mixed-model-repository	tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository	release-candidate	547.3 KiB	32	356.87ms	371.13ms	+4.0%	stable
single-checkpoint-preflight	tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load	single_checkpoint.pkl	183.0 KiB	1	58.13ms	55.93ms	-3.8%	stable
duplicate-heavy-registry	tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot	registry-snapshot	915.2 KiB	13	329.53ms	333.97ms	+1.3%	stable
suspicious-pickle-intake	tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake	suspicious-intake	183.8 KiB	4	74.45ms	75.31ms	+1.2%	stable
warm-cache-rescan	tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan	release-candidate	547.3 KiB	32	86.77ms	87.67ms	+1.0%	stable
rejected-basic-auth-candidates	tests/benchmarks/test_scan_benchmarks.py::test_rejected_basic_auth_candidates_scan_linearly	-	371.1 KiB	1	1.319s	1.315s	-0.3%	stable