Bound stdlib file filters and define a safe symlink/file-type policy (#648)

[leynos]

Summary

Bounds the stdlib file-reading filters (contents, linecount, hash, and digest) and defines one coherent symlink/file-type policy for them.

• StdlibConfig gains with_file_max_read_bytes (default 8 MiB, mirroring fetch_max_response_bytes), threaded into the path filters through a FileConfig carrier.
• All four filters open the final path component without following symlinks (O_NOFOLLOW on Unix, a pre-open symlink check on Windows), open non-blocking so a FIFO or device cannot wedge a worker, verify the opened handle is a regular file, and stream against a running byte total.
• linecount counts line terminators incrementally instead of materialising the file in a String.
• hash and digest stop digesting once the budget is exceeded; within-budget results are unchanged.
• Per-call max_bytes narrows the operator ceiling (never raises it) and a named follow_symlinks=true opt-in permits reading through a final symlink.
• Rejections surface localized InvalidOperation diagnostics naming the path and the applicable limit, never file contents; new keys ship in all 35 catalogues with RTL-catalogue translations.
• Tests cover the exact-limit boundary, one byte over, per-call narrowing and clamping, symlink rejection, the opt-in, and a FIFO fixture; docs cover defaults, limits, symlink handling, and the trust model in the users' guide, the Jinja guide, and the security audit.

Closes #648

References

• Issue: Bound stdlib file filters and define a safe symlink/file-type policy #648
• Codex finding: https://chatgpt.com/codex/cloud/security/findings/5f7edd87824c81918fa2e7dffe2957f7
• Lody session: https://lody.ai/leynos/sessions/73de8032-6dc1-4987-b24d-fc7a7d8fb0d0

Summary by Sourcery

Bound standard-library file reads and enforce a consistent safe policy for symlinks and file types.

New Features:

• Add a configurable 8 MiB default read budget for the contents, linecount, hash, and digest filters, with per-call narrowing and explicit symlink-following opt-in.

Bug Fixes:

• Prevent file-reading filters from consuming unbounded data, following final-component symlinks by default, or blocking on FIFOs and other non-regular files.

Enhancements:

• Stream file reads and line counting with consistent regular-file and byte-limit enforcement across platforms.
• Return localized diagnostics that identify rejected paths and limits without exposing file contents.

Build:

• Promote the filesystem flag dependency to support safe file opens across platforms.

Documentation:

• Document file-read limits, symlink behavior, supported overrides, and the associated trust model in the users' and Jinja guides and security audit.

Tests:

• Add coverage for read-limit boundaries, per-call clamping, symlink handling, and FIFO rejection.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Warning

Your free Security trial is over. An organization admin can upgrade to Advanced for continuous pull request security review or dismiss this notice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

The PR adds a configurable, streaming byte budget and safe final-entry policy to the contents, linecount, hash, and digest filters, propagates that policy through stdlib configuration, validates symlink and file types during opens, and backs the behavior with localized diagnostics, documentation, and cross-platform/end-to-end tests.

Sequence diagram for bounded safe file filter reads

sequenceDiagram
    participant Template
    participant Filter as PathFilter
    participant FS as fs_utils
    participant File as FileHandle

    Template->>Filter: contents(raw, encoding, kwargs)
    Filter->>Filter: path_call_limits(kwargs, configured_max_read_bytes)
    Filter->>FS: read_utf8(path, limits)
    FS->>FS: open_file_checked(path, limits)
    FS->>File: open_with(path, O_NOFOLLOW)
    File-->>FS: opened handle
    FS->>File: metadata()
    File-->>FS: regular-file metadata
    loop bounded chunks
        FS->>File: read(buffer)
        File-->>FS: chunk
        FS->>FS: read_bounded_chunk(total, max_bytes)
    end
    FS-->>Filter: contents or localized limit/type error
    Filter-->>Template: rendered value or diagnostic

Loading

File-Level Changes

Change	Details	Files
Introduce a shared bounded file-reading policy for standard-library path filters.	Add an 8 MiB configurable operator ceiling and per-call narrowing via max_bytes. Stream contents, line counts, hashes, and digests while enforcing the running byte budget. Add localized diagnostics for limit violations and non-regular files. Propagate file-read configuration through stdlib registration.	Cargo.toml src/stdlib/config/mod.rs src/stdlib/config_types.rs src/stdlib/register.rs src/stdlib/path/filters.rs src/stdlib/path/fs_utils.rs src/stdlib/path/hash_utils.rs src/localization/keys.rs src/stdlib/config_tests.rs
Enforce an explicit final-path file-type and symlink policy during reads.	Open final entries without following symlinks by default using platform-specific handling. Verify the opened handle is a regular file and avoid blocking on Unix FIFOs or device nodes. Support an explicit follow_symlinks=true opt-in.	src/stdlib/path/fs_utils.rs src/stdlib/path/filters.rs tests/std_filter_tests/read_policy_filters.rs
Expand end-to-end coverage for read limits and unsafe file types.	Test exact-boundary and over-budget behavior across all reading filters. Test per-call clamping, symlink rejection and opt-in following, and Unix FIFO rejection. Add the new documentation examples to the example registry.	tests/std_filter_tests/read_policy_filters.rs tests/std_filter_tests.rs tests/std_filter_tests/support.rs tests/documentation_examples_tests.rs
Document the new file-reading limits, trust model, and configuration API.	Describe defaults, configuration, keyword arguments, and rejected file types. Record the security audit remediation and link the policy to user-facing guidance.	docs/security-network-command-audit.md docs/stdlib-yaml-and-jinja-guide.md docs/users-guide.md
Add localization entries for configuration validation and file-read failures.	Register new localization keys for positive limits, oversized files, and non-regular files. Add translations across supported locale catalogs.	locales/*/messages.ftl src/localization/keys.rs
Move the rustix filesystem dependency to the main dependency set for production Unix file-opening policy support.	Expose filesystem flags and descriptor operations to non-test code. Remove the target-specific development-only declaration.	Cargo.toml

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#493	Ensure tests/env_path_tests.rs uses the injected environment seam and no longer acquires EnvLock or mutates the process environment.	✅
#493	Migrate src/manifest/tests/workspace.rs away from CurrentDirGuard, EnvLock, and in-process CWD mutation by injecting a workspace base or using absolute paths.	✅
#493	Complete the remaining guard-retirement cleanup and satisfy the issue's environment-isolation requirements, including removal of obsolete guard users and preservation of production behavior.	✅

Possibly linked issues

• Migrate the last EnvLock users onto injected seams (env_path_tests + manifest workspace tests) #493: The PR directly completes issue Migrate the last EnvLock users onto injected seams (env_path_tests + manifest workspace tests) #493 by removing the remaining workspace EnvLock user and validating env_path_tests migration.
• Migrate the last EnvLock users onto injected seams (env_path_tests + manifest workspace tests) #493: The PR explicitly closes Migrate the last EnvLock users onto injected seams (env_path_tests + manifest workspace tests) #493 and removes EnvLock/CwdGuard usage through injected workspace-base test seams.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codescene-access]

No quality gates enabled for this code.

[leynos]

@coderabbitai Please investigate the cause of the following issue using codegraph exploration and research, identify a fix and provide an AI coding agent prompt for the fix:

                | error: unused import: `OpenOptionsExt`
                |  --> src\stdlib\path\fs_utils.rs:8:39
                |   |
                | 8 |     fs_utf8::{Dir, File, OpenOptions,
  OpenOptionsExt},
                |   |                                       
  ^^^^^^^^^^^^^^
                |   |
                |   = note: `-D unused-imports` implied by
  `-D warnings`
                |   = help: to override `-D warnings` add
  `#[allow(unused_imports)]`
                | 
                | error: could not compile `netsuke-build` (lib) due to
  1 previous error
                | warning: build failed, waiting for other jobs to 
  finish...

https://github.com/leynos/netsuke/actions/runs/33702312492/job/100484346530?pr=669

[coderabbitai]

Oops, something went wrong! Please try again later. 🐰 💔