Skip to content

test: add regression test for malformed AST import in AsmJsonImporter - #16874

Open
cavdarahmet wants to merge 1 commit into
argotorg:developfrom
cavdarahmet:add-asmjsonimporter-malformed-ast-test
Open

test: add regression test for malformed AST import in AsmJsonImporter#16874
cavdarahmet wants to merge 1 commit into
argotorg:developfrom
cavdarahmet:add-asmjsonimporter-malformed-ast-test

Conversation

@cavdarahmet

Copy link
Copy Markdown

Description

While doing a coverage-gap analysis (clang source-based coverage over the soltest suite,
including SMT/Z3) to find invariant-checking code paths that no existing test reaches, I
found that AsmJsonImporter's validation of the inline-assembly/Yul subtree of an imported
"SolidityAST" input was never exercised by any test.

This PR adds a regression test that feeds a hand-crafted, corrupted AST (via the
"SolidityAST" standard-json import mode) through StandardCompiler and confirms it's
rejected with a clean error instead of crashing. It covers three previously 0-coverage
yulAssert() lines in AsmJsonImporter.cpp (bad nodeType prefix, non-string "src", unmatched
nodeType), plus one additional gap the assert search didn't catch (a missing required
field that surfaces as a raw nlohmann::json type_error instead of a yulAssert).

Verified against both v0.8.36 and current develop.

Checklist

AI Disclosure

  • No AI tools were used

@cavdarahmet

Copy link
Copy Markdown
Author

While working on this, I noticed the generic catch (...) fallback in StandardCompiler::compileSolidity() uses boost::current_exception_diagnostic_information(), which leaks internal details (file/line, C++ type names) into the error message shown to users, e.g.:

Before:

Unknown exception during compilation: /solidity/libsolidity/interface/StandardCompiler.cpp(1487): Throw in function ... Dynamic exception type: boost::wrapexcept<solidity::util::Exception> std::exception::what: Failed to import AST: Invalid nodeType prefix

After (using .what() when it's a util::Exception):

Unknown exception during compilation: Failed to import AST: Invalid nodeType prefix

Happy to open a small separate PR for this if useful, once this one is merged.

@cameel

cameel commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

which leaks internal details (file/line, C++ type names) into the error message shown to users, e.g.:

This is intentional and should stay as is. What the handler prints is not a typical error message meant to be shown in the course of normal usage of the compiler. It's more of a crash dump that helps us track down the bug that caused it. If anything, I'd make it more, not less detailed.

BOOST_CHECK(!containsError(result, "FatalError", ""));
}

BOOST_AUTO_TEST_CASE(import_ast_malformed_inline_assembly)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can these tests be replaced with ASTJSON or astPropertyTests tests? We're trying to reduce the number of Boost-based tests in favor of custom isoltest test cases, which are easier to maintain. I would not add more tests here if we can avoided and even the existing tests should be rewritten with isoltest whenever possible (though this is very low priority).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though they do not accommodate error cases currently. We should extend them though so that they can show errors.

Comment on lines +2557 to +2572
// AsmJsonImporter::createStatement/createExpression: yulAssert(nodeType prefix == "Yul")
{
Json result = compileWithMutation(
baseAstJson,
[](Json& _ast)
{
Json* node = findFirstNodeByType(_ast, "YulIdentifier");
BOOST_REQUIRE(node);
(*node)["nodeType"] = "XulIdentifier";
});
BOOST_REQUIRE(result["errors"][0]["message"].is_string());
BOOST_CHECK(
result["errors"][0]["message"].get<std::string>().find("Failed to import AST: Invalid nodeType prefix")
!= std::string::npos);
BOOST_CHECK(result["errors"][0]["type"] == "Exception");
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't do it like this. This basically amounts to testing a failing assertion, i.e. compiler's undefined behavior resulting from a bug:

yulAssert(nodeType.substr(0, 3) == "Yul", "Invalid nodeType prefix");

Before this code can be tested, it needs to have proper validations. The fact is that AST import is still an experimental feature and is unfinished. It wrongly uses asserts for things that should be proper validations.

If you want to add a test for this, we need to change that. We need to define a proper exception type to represent these failure conditions (they should not come out as just type: Exception). Then change the exporter to throw that exception (or use solRequire() when the validation is a simple one-liner).

@cavdarahmet

Copy link
Copy Markdown
Author

Thanks for the review and the explanation.

I see the issue now. It makes sense that these conditions should be handled through proper validation rather than testing failing assertions. I'll look into the suggested approaches and rework the implementation accordingly before updating the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants