Skip to content

Add JSON edge-case coverage; clear parse-error messages on both engines - #175

Merged
yakew7 merged 1 commit into
yakew7:mainfrom
ImMortaL0P:test/parseJSON-edge-case-coverage
Aug 6, 2026
Merged

Add JSON edge-case coverage; clear parse-error messages on both engines#175
yakew7 merged 1 commit into
yakew7:mainfrom
ImMortaL0P:test/parseJSON-edge-case-coverage

Conversation

@ImMortaL0P

Copy link
Copy Markdown
Contributor

What

Closes #169. assets/profiler-engine.js's parseJSON() and faircode/loaders_extra.py's read_table() had no test coverage for malformed/unusual JSON input, so it wasn't clear what a user actually sees when they drop a bad file.

Findings & fixes

Truncated/invalid JSON syntax used to leak a raw, engine-specific parser error — a browser SyntaxError in JS, an internal pandas ValueError in Python (which, worse, retried under orient="split" only to fail again with an equally confusing second error). Both now fail fast with the same clear "Unsupported JSON format" wording the existing tabular-shape checks already use.

Found while writing the tests: the JS engine's "columns orientation" check only verified each top-level value was a plain object, not that its entries were scalars — so a deeply-nested, non-tabular structure like {"a": {"b": {"c": 1}}} was silently misread as one column "a" with a row "b" whose cell was the object {"c": 1}, instead of raising an error. It now throws the same clear message as the other unsupported-shape cases.

Scope note: Python's pandas-backed reader is intentionally more lenient than the JS engine for two of the four cases named in the issue — a bare array of primitives ([1,2,3]) becomes a one-column DataFrame, and {} becomes an empty DataFrame — rather than erroring. I left that behavior alone and the new tests just pin what it currently does; changing pandas' own JSON-orientation handling felt like a bigger, riskier change than what a parse-error-message fix calls for. Happy to revisit if you'd rather tighten it for parity with the JS side.

Changes

  • assets/profiler-engine.js: wrap JSON.parse syntax errors; tighten the columns-orientation check to reject nested (non-scalar) values.
  • faircode/loaders_extra.py: pre-validate JSON syntax with json.loads before handing off to pd.read_json, so a genuine syntax error gets one clear message instead of two confusing pandas ones.
  • scripts/parse-json-js.js (new): mirrors the existing scripts/profile-json-js.js pattern — reports parseJSON()'s result or error message as JSON on stdout so tests can assert on it via subprocess.
  • tests/test_json_edge_cases.py (new): covers all four cases named in the issue (truncated JSON, array of primitives, empty object, deeply-nested structure) on both engines.

Testing

  • pytest tests/ → 80 passed, 9 skipped (skips are pre-existing, for the optional excel/parquet extras I didn't install locally).
  • tests/test_js_parity.py (the Python/JS profiler-output parity suite) passes unchanged — confirms the columns-orientation tightening doesn't affect any valid input already covered there.
  • Manually re-verified all edge cases from the issue against the fixed parseJSON() in Node directly.

Disclosure: I used an AI coding assistant (Claude) to investigate this issue and write the fix/tests. I read and verified CLAUDE.md's paper-freeze scope first — this only touches the actively-developed profiler tool (profiler.py/profiler-engine.js), not the frozen audit/benchmark pipeline. Happy to iterate on review feedback.

parseJSON() in assets/profiler-engine.js and read_table() in
faircode/loaders_extra.py had no test coverage for malformed or
unusual JSON input, so it wasn't clear what a user actually sees when
they drop a bad file (yakew7#169).

- Truncated/invalid JSON syntax previously leaked a raw, engine-specific
  parser error (a browser SyntaxError in JS, an internal pandas ValueError
  in Python - and in Python's case, retried under orient="split" only to
  fail with an equally confusing second error). Both now fail fast with
  the same clear "Unsupported JSON format" wording the existing
  tabular-shape checks already use.
- Found in the process: the JS engine's "columns orientation" check
  only verified each top-level value was a plain object, not that its
  entries were scalars, so a deeply-nested non-tabular structure like
  {"a": {"b": {"c": 1}}} was silently misread as one column "a" with a
  row "b" whose cell was the object {"c": 1}. It now throws the same
  clear error instead of producing a garbled table.
- Added scripts/parse-json-js.js (mirrors the existing
  scripts/profile-json-js.js pattern) so tests can assert on parseJSON()'s
  error message via subprocess instead of parsing a Node stack trace.
- Added tests/test_json_edge_cases.py covering all four cases named in
  the issue (truncated JSON, array of primitives, empty object, deeply
  nested structure) on both engines. Where Python's pandas-backed reader
  is intentionally more lenient than the JS engine (array-of-primitives,
  empty-object) and already returns a well-defined result rather than
  erroring, the test pins that existing behavior instead of changing it -
  loosening/tightening pandas' own JSON-orientation handling felt out of
  scope for a parse-error-message fix.

All existing tests (including tests/test_js_parity.py) still pass.

Closes yakew7#169
@ImMortaL0P
ImMortaL0P requested a review from yakew7 as a code owner August 6, 2026 15:40
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@ImMortaL0P is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

Hey @ImMortaL0P, your first PR on Fair Code, that's awesome, thank you.

This project is about making AI more accountable, and contributions like yours are what keep that work going. We'll review your changes shortly.

If you haven't already, give the contributing guide a quick read: it covers how audits are structured and what we look for in a review.

@yakew7 yakew7 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid edits will be merging the pr

@yakew7
yakew7 merged commit 50991ea into yakew7:main Aug 6, 2026
10 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parseJSON() has no coverage for malformed/edge-case JSON input

2 participants