Skip to content

fix(index): skip dot-files, not only dot-directories (closes #25) - #36

Open
tonydzi wants to merge 3 commits into
mainfrom
fix-25-dotfiles
Open

tonydzi wants to merge 3 commits into
mainfrom
fix-25-dotfiles

Conversation

@tonydzi

@tonydzi tonydzi commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Closes #25.

iter_notes filtered hidden directories via rel.parts[:-1] — every path component except the filename — so .draft.md in the vault root, or in any visible folder, was indexed while the same file inside .drafts/ was skipped. Reproduced 2026-10-13 (.secret.md next to ok.md: both returned).

The filter now checks every component. BRAIN_INDEX_HIDDEN=1 still admits both kinds of hidden path.

Red first: tests/test_index_notes_dotfiles.py pins four cases; on the old line exactly the two file cases fail (2/4), the directory case and the opt-in flag pass. Full suite after: 102 passed.

@Kaap10 — this is your repro from the 13th, six days late; you hold write access, so if you would rather land it yourself, say so and I will close this one.

🤖 Generated with Claude Code


Note

Low Risk
Indexing scope narrows for dot-prefixed .md files until re-index; behaviour is intentional and reversible via BRAIN_INDEX_HIDDEN=1, with no auth or data-migration risk.

Overview
Dot-prefixed note filenames are now excluded from indexing the same way dot-directories already were. iter_notes used to inspect only parent path segments (rel.parts[:-1]), so files like .draft.md in the vault root or a normal folder were indexed while the same name under .drafts/ was not. The filter now checks every relative path component, so hidden filenames are skipped by default; BRAIN_INDEX_HIDDEN=1 still opts in to both hidden files and folders.

Eval gold generation shares that single rule. eval/build_gold.py calls iter_notes instead of duplicating its own rglob and hidden-folder logic, so evaluation questions are not built from notes the indexer would skip.

Docs (CHANGELOG.md, examples/README.md) describe the behaviour. New tests in tests/test_index_notes_dotfiles.py and tests/test_eval.py lock in skip/opt-in cases and a vault rooted at a dot directory.

Reviewed by Cursor Bugbot for commit caaa2e6. Bugbot is set up for automated code reviews on this repo. Configure here.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T19:28:31.780912Z 5aa298e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5aa298e. Configure here.

Comment thread tests/test_index_notes_dotfiles.py Outdated
@tonydzi

tonydzi commented Oct 19, 2026

Copy link
Copy Markdown
Owner Author

Mycroft speaking, the synthetic AI co-founder at Palo Alto AI Research Lab. No pulse, but a working diff reader.

Logged your pull request ("fix(index): skip dot-files, not only dot-directories (closes #25)") after 3 h of silence, which is on us. What a machine can already tell: 2 file(s) touched, +66/-3, CI green. A human from the lab reads it within 48 hours, as CONTRIBUTING.md promises; if that clock runs out, ping the thread, it is not rude.

— TonyDzi, Palo Alto AI Research Lab · more of the machine this came from (second brain + fleet coordination) lives at github.com/tonydzi, DMs open.

@tonydzi
tonydzi requested a review from Kaap10 October 20, 2026 07:24
tonydzi and others added 2 commits October 20, 2026 15:57
iter_notes filtered hidden directories by checking rel.parts[:-1] — every
path component except the last, which is the filename. So .draft.md or
.trash.md in the vault root, or in any visible folder, was indexed as a real
note while the same file one level down inside .drafts/ was skipped.
Reproduced 2026-10-13 with .secret.md next to ok.md: both came back.

The filter now checks every component. BRAIN_INDEX_HIDDEN=1 still admits
both kinds of hidden path, since someone legitimately keeps notes in one.

tests/test_index_notes_dotfiles.py pins all four cases; shown red on the
old line (2 of 4 fail, exactly the two file cases) before the change.
Full suite: 102 passed.

Assisted-by: Claude Code / claude-fable-5-1
Machine: MacBook-Anton
Account: dzyatkovskiy.a2@gmail.com
Operator: anton

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>


- eval/build_gold.py read_notes() walked the vault with its own copy of the
  filter (rel.parts[:-1]) and ignored BRAIN_INDEX_HIDDEN, so a root-level
  .draft.md could become a gold question while the indexer skipped it. It now
  calls sqlite_graph_memory.index_notes.iter_notes: one rule, one place.
- tests: _names() uses as_posix() so the forward-slash assertions hold on
  Windows; new test pins a vault whose root is itself a dot-directory
  (guards against filtering p.parts instead of rel.parts); new test_eval
  case proves build_gold drops .draft.md and honours BRAIN_INDEX_HIDDEN=1.
- docs: index_notes docstring, examples/README env table and CHANGELOG say
  dot-files and dot-directories.

Assisted-by: Claude Code / claude-opus-5-5
Machine: MacBook-Anton
Account: dzyatkovskiy.a2@gmail.com
Operator: anton

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	CHANGELOG.md
Kaap10
Kaap10 previously approved these changes Oct 20, 2026

@Kaap10 Kaap10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased onto latest main to resolve the CHANGELOG conflict. Fix and tests for #25 look solid!

…LOG only)

Assisted-by: Claude Code/Sonnet 5.5
Machine: MacBook-Anton
Account: tonydzi
Operator: robot:contrib-watch-daily

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tonydzi

tonydzi commented Oct 20, 2026

Copy link
Copy Markdown
Owner Author

@Kaap10 thanks for the rebase and the approval. Another push landed after it: main moved again (the PageRank lane, #52), and the only conflict was CHANGELOG.md, where I kept both Unreleased entries. No code changed. Full suite on the merge head: 170 passed.

Because main dismisses stale approvals, the one you gave is reset; a second approve on the new head is all it needs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

index_notes: dot-files (e.g. .draft.md, .trash.md) are indexed despite BRAIN_INDEX_HIDDEN default

2 participants