Skip to content

fix: leave out every gitignored file when apify push collects files - #1537

Draft
DaveHanns wants to merge 57 commits into
masterfrom
fix/push-ignore-rules
Draft

DaveHanns wants to merge 57 commits into
masterfrom
fix/push-ignore-rules

Conversation

@DaveHanns

@DaveHanns DaveHanns commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

apify push picked the wrong files in a few gitignore cases. The worst case: an Actor in an ignored directory uploaded everything, including files its own .gitignore lists, such as a .env. This fixes the five cases from #1536. Actor discovery for apify push --all uses the same helper, so it marks ignored Actors the same way.

Changes

  • With git, the helper reads git ls-files -z and matches the listed paths exactly. A path is ignored when git lists it or it lies under a listed entry. That also covers an ignored symlink to a directory, which git lists without a trailing slash. Names are compared in NFC, so names with non-ASCII characters (in either Unicode form) and names like #hash.txt or [ab].js now match.
  • When the rules ignore the pushed directory itself, or git fails, the helper reads the rules itself. git check-ignore --no-index answers whether the directory is ignored, whatever is tracked in it. The helper then applies every .gitignore inside the directory, every one above it up to and including the repository root, and .git/info/exclude and the global excludes file when git names them. In the files above, the lines that ignore the pushed directory are dropped, since pushing it is explicit. Their other lines still apply.
  • These rules rank as they do in git. A deeper .gitignore wins over a shallower one, so its ! lines re-include files. Every .gitignore wins over .git/info/exclude, which wins over the global excludes file. A file below an ignored directory stays ignored. All lines go into one ignore instance, rewritten to match paths from the repository root.
  • Without git, the walk now reads the repository root .gitignore too. It still stops there, so an outer repository's rules don't apply. When the pushed directory is the repository root, the walk no longer reads the .gitignore files above it.
  • Actor discovery passes the Actor directories it found to the helper, which asks git check-ignore about them. An ignored Actor directory that holds a tracked file is now marked ignored.
  • .actorignore works as before, including its ! lines that force-include gitignored files.

Tests

  • One test per case, each failing on the old code: the repository root .gitignore without git, non-ASCII names in both forms, names that look like patterns, an ignored symlink to a directory, a pushed directory that is itself ignored (with and without git, and with a tracked file in it), and a deeper ! line (with and without git).
  • .actorignore force-includes with directories and files git lists, and in an ignored directory.
  • The no-git tests create a fake .git, so the apify-cli .gitignore no longer applies to them.

Notes for reviewers

  • Stacked on feat: let apify push select which Actors to push in a repository with several #1504. It'll be retargeted to master after feat: let apify push select which Actors to push in a repository with several #1504 merges.
  • .git/info/exclude and the global excludes file are applied in the ignored-directory path, so the only rules left out are the lines that ignore the pushed directory. Without git, they're skipped as before.
  • Lines are dropped only from a file whose rules ignore the pushed directory. A whitelist like * with !*/ doesn't ignore directories, so it stays whole.
  • A subdirectory of an ignored directory counts as ignored too, so it takes the same path. git ls-files fails there anyway (git 2.39 reports an internal error).
  • The rules path doesn't know which files are tracked. In an ignored directory, a tracked file that a rule matches is left out. Outside one, git mode keeps tracked files as before.
  • Exact matching is a bit stricter than the old pattern matching: a name git lists in one place no longer excludes a tracked file with the same name elsewhere.
  • Each push and each discovery runs one more git command, git check-ignore.

Closes #1536

🤖 Generated with Claude Code

…r push

The loop-level listener kept a signal from terminating the CLI anywhere in the selection, so a Ctrl+C during an upload let the current push run on. Outside a build wait the listener now names the Actors not pushed (and, with --json, prints the results so far), removes itself and raises the signal again, so Node's default action ends the process as it does for a single Actor. During a build wait the build is still aborted and the loop stops after it.
Ctrl+C at the Git-source question rejects the prompt, which ends a single push. In the loop the error was caught and the next Actor was pushed; the loop now stops and names the Actors not pushed.
DaveHanns and others added 27 commits October 27, 2026 15:08
…tors

# Conflicts:
#	src/commands/actors/push.ts
#	src/lib/hooks/telemetry/trackEvent.ts
Co-authored-by: Edyta <142720610+szaganek@users.noreply.github.com>
Co-authored-by: Edyta <142720610+szaganek@users.noreply.github.com>
One Actor could be pushed three ways (--dir, --actor-dir, or both),
which confuses users and agents. --all and the interactive prompt stay.
Without a terminal and without --all, the listing now points to --dir
and shows paths that work as its value from where the command ran.
Restores the command framework to its master state, including the
inline invalid-choice error and the short-form = handling, which were
reshaped only for repeated values.
The exported ignore list had been inserted between the comment and the
function it describes.
…cher

getActorLocalFilePaths now asks the shared helper whether a path is
ignored. No behavior change: the same git ls-files call, and the same
fallback when git fails.
findActorDirs marks each Actor whose directory .gitignore ignores, using
the same check as the upload. --all skips those Actors and names each one
on stderr with the apify push --dir command that pushes it. When every
Actor is ignored, --all exits 4. The picker and the non-interactive list
still show them, labeled [ignored by .gitignore], and a picked one is
pushed. Under --json, skipped Actors aren't entries in the results.
…n .gitignore

Without git, the fallback matcher turned an Actor directory into an empty
path for the Actor's own .gitignore, and the ignore package throws on it.
A .gitignore can't ignore its own directory, so an empty path now isn't
ignored. The discovery tests also cover a rule only git reads, and the
doc comments no longer claim that discovery and the upload always agree.
The skip note for an ignored Actor prints a command to copy. A path with a
space or another character a shell splits or expands is now quoted.
With git, read git ls-files -z and match the listed paths exactly, so
names with non-ASCII characters or pattern characters match.

When git lists the pushed directory itself as ignored, or fails, read
the rules: the .gitignore files inside the directory and above it up to
and including the repository root, plus .git/info/exclude and the
global excludes file when git names them. Lines above the directory
that ignore it are dropped, so the other rules still apply.

Without git, the repository root .gitignore now applies too.

Closes #1536
git lists an ignored symlink to a directory without a trailing slash, so
the files the glob found through the link were uploaded. Every entry git
lists now covers the paths below it.

git on macOS prints names in NFC, while the glob returns them in the form
they have on disk. An ignored file with a decomposed name never matched.
Both sides are now compared in NFC.
When apify push reads the rules itself, it left out a path as soon as
any one ignore file ignored it. A `!` line in a deeper .gitignore never
re-included a file a shallower one ignores. Reading the repository root
.gitignore without git made this common: a root `dist/` hid the `dist/`
an Actor's own .gitignore brings back.

Every line now goes into one ignore instance, rewritten to match paths
from the repository root, from the lowest rank to the highest: the
global excludes file, .git/info/exclude, then the .gitignore files from
the root down. As in git, the last matching line wins, and a file below
an ignored directory stays ignored. Names are compared in NFC here too.
When an ignored directory holds a tracked file, git lists its untracked
files one by one instead of `./`. Pushing that directory still left out
its own files, such as main.js, and Actor discovery marked it as not
ignored.

The matcher now asks `git check-ignore --no-index` whether the rules
ignore the pushed directory itself, whatever is tracked in it, and then
reads the rules itself. Discovery passes the Actor directories it found,
and the rules decide about them in one more `git check-ignore` call.
Without git, the walk now reads up to the repository root. The no-git
tests run in test/tmp, so it reached the apify-cli .gitignore, whose
`dist` and `build` lines hid whether a test's own .gitignore was read.
Each test that builds its own tree now has a fake .git, so the walk
stops there.
Base automatically changed from feat/push-select-actors to master October 31, 2026 00:22

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant