missing log in `only_generate` mode

[weixlu]

Hi maintainer,

I think fuzz test for CPython is a great idea! For example, multi-threading related bugs are quite hard to catch with regular tests (although we have buildbot to track this), and fuzz testing or stress testing can be very effective approaches.

I'm learning how Fusil generates code (it’s amazing to see test code be automatically generated!), so I run it in --only-generate mode. During this I noticed an issue: even though the console prints:

[0][session 1] Fusil log written into python-1/project.log

the file does not actually exist. I need to manually enable --keep-sessions to keep this file.

I have two suggestions:

1. If --keep-sessions is not enabled, we probably should not print Fusil log written into xxx, which seems confusing.
2. More generally, I suggest that --only-generate mode should enable --keep-sessions by default.

The fix I propose is to return True in both ProjectDirectory.keepDirectory() and SessionDirectory.checkKeepDirectory():

if application.options.only_generate:
    return True

What do you think about this? I’d be happy to open a PR.

[weixlu]

Also some investigation: Why does it print Fusil log written into xxx even when --keep-sessions is not set?

I found that the normal exit path is: first Project.destroy(), then Application.exit(). In Project.destroy(), the logger is cleaned up, so Application.exit() does not print the log message anymore.

However, the --only-generate mode follows a different exit path through PythonProcess.on_python_source(), where it directly calls Application.exit(). As a result, it prints the log message first (making it look like the file exists), and afterwards cleans up the directory.

[devdanzin]

Thanks for the thorough investigation, @weixlu!

One thing to add from reproducing it: by default --only-generate doesn't just print a misleading line, but also deletes the whole run directory, source.py included, so the mode currently produces no inspectable output at all. So it's worth fixing properly rather than just quieting the message.

Your suggestion 2 is correct, and you already spotted the subtle part: the guard is needed in both SessionDirectory.checkKeepDirectory() (keeps the session dir with source.py) and ProjectDirectory.keepDirectory() (the project dir is removed independently, because session_executed is still 0 at that point). This single change closes the issue on its own and the confusing message goes away for free.

Would you like to take that as the PR? Scope-wise it's the two return True guards, a small test that the run directory survives under --only-generate, and a one-line note in the --only-generate help text so the "keeps its output" behavior is discoverable.

Two enhancements I'll layer on afterward (no need for you to touch them unless you'd like the stretch goal):

1. Make Application.exit()'s log line existence-checked, so it's truthful on every exit path instead of depending on teardown ordering.
2. Print Generated source written into <path> in only-generate mode. That's the file you're actually after here, and right now the mode only ever mentions project.log.

(Meanwhile, if you just want the generated file at a known location today, --source-output-path FILE writes it straight there and it survives regardless, which is handy for grabbing a single script.)

Thanks again! This is a great first issue to pick up, and your diagnosis did the hard part.

[weixlu]

I’d be happy to open a PR! Also, the two enhancements you mentioned look quite straightforward. Would you mind if I take them as well?

(Meanwhile, if you just want the generated file at a known location today, --source-output-path FILE writes it straight there and it survives regardless, which is handy for grabbing a single script.)

thanks for the detailed reminder!

[devdanzin]

Great, thank you! Feel free to take them as well, much appreciated.