Refuse to record an empty snapshot over one that has rows - #439
Merged
Conversation
An artifact that needs an optional dependency disables itself and returns nothing when that dependency is absent. Recording on a machine without it therefore replaces real rows with zero, and the snapshot then asserts the absence as the correct answer. Nothing errors, the recorder prints its usual saved-to line, and the gate goes green on the wrong data. The recorder now compares against the newest existing snapshot for the same unit and refuses the write when this run produced 0 rows and that one has rows, naming the counts and pointing at the run log. --allow-empty records anyway, for when an artifact genuinely stopped finding anything. Measured on ALEAPP without sqlcipher3, which has no wheel on every platform: threema_messages 33 rows and threema_contacts 3 both refused, threema_account unaffected because it does not need the dependency. A first-ever recording is never blocked, since there is no prior snapshot to compare against. This is the recorder only. run_test_cases.py calls process_artifact directly, so the gate still fails on a genuine drop to zero rather than skipping it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An artifact that needs an optional dependency disables itself and returns nothing when that dependency is absent. Recording on a machine without it therefore replaces real rows with zero, and the snapshot then asserts the absence as the correct answer. Nothing errors and the gate goes green on the wrong data.
The recorder now compares against the newest existing snapshot for the same unit and refuses the write when this run produced 0 rows and that one has rows, naming both counts.
--allow-emptyrecords anyway, for when an artifact genuinely stopped finding anything.Measured on ALEAPP without sqlcipher3: threema_messages (33 rows) and threema_contacts (3) both refused, threema_account unaffected because it does not need the dependency. A first-ever recording is never blocked.
Recorder only. run_test_cases.py calls process_artifact directly, so the gate still fails on a genuine drop to zero rather than skipping it.