[high] Re-apply #98: exit non-zero and keep converting when a rule fails with --output-dir (reverted with #105)

[elhoim]

BLUF

• What: re-applies [high] Exit non-zero and keep converting when a rule fails with --output-dir #98 ("Exit non-zero and keep converting when a rule fails with --output-dir"). It was merged, but the revert cdd5436 that came in with [high] Write finalized, postprocessed queries with --output-dir #105 (merge 8effb4c) silently removed it from main.
• Problem it fixes (unchanged): with --output-dir, the first rule that fails to convert aborts the conversion of every later rule, only a warning is printed, and the command exits 0 - a CI job reports success with an incomplete or empty output directory.
• Change, adapted to [high] Write finalized, postprocessed queries with --output-dir #105: [high] Write finalized, postprocessed queries with --output-dir #105 rewrote write_separate_files() to write finalized, postprocessed queries. This keeps that design and only replaces its single try: backend.convert(...) except Exception: warn with a per-rule loop (as Backend.convert does, without a callback). Each successful conversion stores its finalized result on the rule, which [high] Write finalized, postprocessed queries with --output-dir #105's get_output_rules() / get_conversion_result() collection then writes. Failing rules are reported and the command exits 1 when any failed.
• Tests: [high] Exit non-zero and keep converting when a rule fails with --output-dir #98's two tests (as merged, with the stderr assertions) are back; the main one fails on current main and passes here. All of [high] Write finalized, postprocessed queries with --output-dir #105's tests still pass, including test_convert_output_dir_basic, the test the reverted resolution broke. Full suite with the poetry.lock pins (pySigma 1.4.0, click 8.4.2, pyparsing 3.3.2, Python 3.12): 123 passed, 1 skipped.

Priority: high

What happened

main no longer contains #98, although it was merged:

• [high] Report collection errors and broken filters in sigma check #99 (47712d0) and [high] Exit non-zero and keep converting when a rule fails with --output-dir #98 (c8348d7) were merged on 2026-09-27 before [high] Write finalized, postprocessed queries with --output-dir #105.
• [high] Write finalized, postprocessed queries with --output-dir #105 then had conflicts with main. The resolution 86d3041 ("Merge github/main: resolve conflicts in convert.py and test_convert.py") brought [high] Exit non-zero and keep converting when a rule fails with --output-dir #98 and [high] Report collection errors and broken filters in sigma check #99 into the [high] Write finalized, postprocessed queries with --output-dir #105 branch, but it fails tests/test_convert.py::test_convert_output_dir_basic (reproduced locally).
• It was reverted six minutes later by cdd5436 ("Revert "Merge github/main: ...""), and [high] Write finalized, postprocessed queries with --output-dir #105 was merged with that revert (8effb4c). The revert removed the whole merged main side from the branch, so merging [high] Write finalized, postprocessed queries with --output-dir #105 took [high] Exit non-zero and keep converting when a rule fails with --output-dir #98's and [high] Report collection errors and broken filters in sigma check #99's changes back out of main: 8effb4c deletes the 22 lines [high] Report collection errors and broken filters in sigma check #99 added to sigma/cli/check.py and its 51 lines of tests, and restores the old --output-dir error handling.

Probably nobody intended to revert #98 and #99; they were collateral of undoing a broken conflict resolution.

Original description (#98)

BLUF

• Problem: with --output-dir, any conversion error is downgraded to a warning and the command exits 0. Every rule after the failing one is silently dropped.
• Impact: sigma convert -t <backend> -od out/ rules/ in a CI job reports success while out/ is incomplete, and can even be empty. The same input without -od exits 1.
• Fix: convert rule by rule in write_separate_files(), report each failing rule, keep writing the others, and exit 1 when any rule failed.
• Tests: two new CliRunner tests. The first (fail rule + good rule → exit 1, good rule still written) fails on main and passes with the fix. The second guards --skip-unsupported (still exit 0, reported under "Ignored errors").

Priority: high

Details

Root cause: sigma/cli/convert.py:192-195

try:
    backend.convert(rule_collection, format, correlation_method, callback=write_callback)
except Exception as e:
    click.echo(f"Warning: Failed to convert rules: {e}", err=True)

Backend.convert() is a single list comprehension over all rules. The first exception aborts it, so the callback never runs for later rules. The exception is then swallowed. The single-output path (convert.py:590-601) turns the same SigmaError / NotImplementedError into Error while converting: ... with exit status 1.

The fix does what Backend.convert() does: init_processing_pipeline() and resolve_rule_references(), then convert_rule() / convert_correlation_rule() in collection order with the existing callback. The difference is that SigmaError / NotImplementedError is caught per rule. finalize() is not called, but the old code discarded its return value anyway. After the files are written, a ClickException reports the number of failed rules.

-s/--skip-unsupported still works as before. In that mode the backend collects SigmaErrors itself (collect_errors=True), and they are listed under "Ignored errors" with exit 0.

Before (rule a.yml uses an unresolvable |expand placeholder, b.yml is fine):

$ sigma convert -t text_query_test -od out rules/
Warning: Failed to convert rules: Attempt to convert unhandled placeholder 'var' into query.
Wrote 0 file(s) to out
$ echo $?
0

After:

Error: Failed to convert rule /tmp/demo/rules/a.yml: Attempt to convert unhandled placeholder 'var' into query.
Wrote 1 file(s) to out
Error: 1 rule(s) failed to convert, see errors above.
$ echo $?
1

This PR is independent of the other --output-dir PR, which changes the write loop of the same function. Whichever lands second, I'll rebase it.

Testing

• New tests: test_convert_output_dir_conversion_error_fails_and_continues (fails on main, passes here) and test_convert_output_dir_conversion_error_skip_unsupported.
• Full suite, run the way CI runs it (Python 3.12, dependencies pinned to poetry.lock: pySigma 1.4.0, click 8.4.2, pyparsing 3.3.2): pytest --cov=sigma → 120 passed, 1 skipped.

🤖 Generated with Claude Code