Add JSON edge-case coverage; clear parse-error messages on both engines - #175
Merged
Merged
Conversation
parseJSON() in assets/profiler-engine.js and read_table() in faircode/loaders_extra.py had no test coverage for malformed or unusual JSON input, so it wasn't clear what a user actually sees when they drop a bad file (yakew7#169). - Truncated/invalid JSON syntax previously leaked a raw, engine-specific parser error (a browser SyntaxError in JS, an internal pandas ValueError in Python - and in Python's case, retried under orient="split" only to fail with an equally confusing second error). Both now fail fast with the same clear "Unsupported JSON format" wording the existing tabular-shape checks already use. - Found in the process: the JS engine's "columns orientation" check only verified each top-level value was a plain object, not that its entries were scalars, so a deeply-nested non-tabular structure like {"a": {"b": {"c": 1}}} was silently misread as one column "a" with a row "b" whose cell was the object {"c": 1}. It now throws the same clear error instead of producing a garbled table. - Added scripts/parse-json-js.js (mirrors the existing scripts/profile-json-js.js pattern) so tests can assert on parseJSON()'s error message via subprocess instead of parsing a Node stack trace. - Added tests/test_json_edge_cases.py covering all four cases named in the issue (truncated JSON, array of primitives, empty object, deeply nested structure) on both engines. Where Python's pandas-backed reader is intentionally more lenient than the JS engine (array-of-primitives, empty-object) and already returns a well-defined result rather than erroring, the test pins that existing behavior instead of changing it - loosening/tightening pandas' own JSON-orientation handling felt out of scope for a parse-error-message fix. All existing tests (including tests/test_js_parity.py) still pass. Closes yakew7#169
Contributor
|
@ImMortaL0P is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
Hey @ImMortaL0P, your first PR on Fair Code, that's awesome, thank you. This project is about making AI more accountable, and contributions like yours are what keep that work going. We'll review your changes shortly. If you haven't already, give the contributing guide a quick read: it covers how audits are structured and what we look for in a review. |
yakew7
reviewed
Aug 6, 2026
yakew7
left a comment
Owner
There was a problem hiding this comment.
Valid edits will be merging the pr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes #169.
assets/profiler-engine.js'sparseJSON()andfaircode/loaders_extra.py'sread_table()had no test coverage for malformed/unusual JSON input, so it wasn't clear what a user actually sees when they drop a bad file.Findings & fixes
Truncated/invalid JSON syntax used to leak a raw, engine-specific parser error — a browser
SyntaxErrorin JS, an internal pandasValueErrorin Python (which, worse, retried underorient="split"only to fail again with an equally confusing second error). Both now fail fast with the same clear"Unsupported JSON format"wording the existing tabular-shape checks already use.Found while writing the tests: the JS engine's "columns orientation" check only verified each top-level value was a plain object, not that its entries were scalars — so a deeply-nested, non-tabular structure like
{"a": {"b": {"c": 1}}}was silently misread as one column"a"with a row"b"whose cell was the object{"c": 1}, instead of raising an error. It now throws the same clear message as the other unsupported-shape cases.Scope note: Python's pandas-backed reader is intentionally more lenient than the JS engine for two of the four cases named in the issue — a bare array of primitives (
[1,2,3]) becomes a one-column DataFrame, and{}becomes an empty DataFrame — rather than erroring. I left that behavior alone and the new tests just pin what it currently does; changing pandas' own JSON-orientation handling felt like a bigger, riskier change than what a parse-error-message fix calls for. Happy to revisit if you'd rather tighten it for parity with the JS side.Changes
assets/profiler-engine.js: wrapJSON.parsesyntax errors; tighten the columns-orientation check to reject nested (non-scalar) values.faircode/loaders_extra.py: pre-validate JSON syntax withjson.loadsbefore handing off topd.read_json, so a genuine syntax error gets one clear message instead of two confusing pandas ones.scripts/parse-json-js.js(new): mirrors the existingscripts/profile-json-js.jspattern — reportsparseJSON()'s result or error message as JSON on stdout so tests can assert on it via subprocess.tests/test_json_edge_cases.py(new): covers all four cases named in the issue (truncated JSON, array of primitives, empty object, deeply-nested structure) on both engines.Testing
pytest tests/→ 80 passed, 9 skipped (skips are pre-existing, for the optional excel/parquet extras I didn't install locally).tests/test_js_parity.py(the Python/JS profiler-output parity suite) passes unchanged — confirms the columns-orientation tightening doesn't affect any valid input already covered there.parseJSON()in Node directly.Disclosure: I used an AI coding assistant (Claude) to investigate this issue and write the fix/tests. I read and verified CLAUDE.md's paper-freeze scope first — this only touches the actively-developed profiler tool (
profiler.py/profiler-engine.js), not the frozen audit/benchmark pipeline. Happy to iterate on review feedback.