frontend: eliminate hs as an error handling primitive. - #474
Conversation
This commit is a work in progress. Goal ==== Simplify frontend representation by eliminating `hs` (handler scope) which were previously inserted to make it clear which instructions could fail so that control-flow sensitive analyses were precise. Unfotunately, many instructions were inserted pre-typing leading to imprecise assumptions that many non-failing operations could fail, and requiring expensive *CleanupTemporaries* passes to reduce the IR to something small enough to debug. Now, we can eliminate it because failure is based on whether a callee has a declared *Result* return type. Changes done ============ Passes that unify and manipulate control flow were largely rewritten: - *MakeResultsExplicit* now works based on the nesting structure of control flow block elements to capture function body and module results in `return__123` variables. - *Weaver* now uses the same block structure instead of working on a CFG it builds using the block structure. Both of these have been tested thoroughly. Our *MaximalPaths* representation of a basic block control flow graph has been adapted to work with *bubbly* branches since we can no longer rely on `fail#123` expressions as conditions for transitions to error handling blocks, but it needs more debugging, and passes that rely on it are probably buggy. The staging of some of these μpasses has changed: 1. *MakeResultsExplicit* now runs first. It allocates return variables early. 2. *Weaver* now calls *MagicSecurityDust.sprinkle* internally, and has been rewritten and is probably quite a bit faster and simpler. It allocates far fewer temporaries. *Typer* can no longer rely on `hs(...)` calls at places where a failing result can reach a call as an argument or be assigned to a variable. It used to serve as a typing helper by converting `Foo | Bubble` to `Foo`. Now the *Typer* explicitly excludes bubbles coming out of calls, and when propagating type information from a typed, bubbly `return__123` variable to a temporary, for example. Still to do =========== Most of the staging tests pass, but some that interpret `or...else`s or that deal with RTTI checks fail. Need to debug these. *MaximalPaths* is probably buggy. Need more testing. Some static error checks like "function body bubbles but does not declare `throws`" might depend on information that now needs to be found by analysis of bubbly calles and CF block structure. Also ==== I used the *AssertModuleAtStage* regenerate test outputs trick. I tweaked some of it to preserve comments inside test golden files. Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
| * | ||
| * See also [lang.temper.frontend.MagicSecurityDust]. | ||
| */ | ||
| private object HandlerScopeFn : SpecialFunction, NamedBuiltinFun { |
There was a problem hiding this comment.
I'm not out of scope; it's the scopes that are out.
| .also { | ||
| helpSnippet(it, "Boolean inverse", "builtin/!") | ||
| } | ||
| val notFn: CallableValue = NotFn |
There was a problem hiding this comment.
control flow stuff becomes easier if we can refer to ! from fundamentals.
| { Rn(name) }, | ||
| ) | ||
| Block { | ||
| If( |
There was a problem hiding this comment.
This commit adds machinery to TreeFarm to allow building control-flow block structures directly.
That's really helpful for testing, but one nice side effect is you can just use it in frontend code too.
| internal.insert(internal.size) { | ||
| V(vInitSymbol) | ||
| // TODO Could implement a coalesce macro with this content. | ||
| Call { |
There was a problem hiding this comment.
This was kind of gnarly: building a call to the if macro. Now it just uses the tree farm.
| for (typeFormal in typeFormals) { | ||
| if (typeFormal.needsAdapting) { | ||
| val newName = p.nameMaker.unusedSourceNameWithPrefix(prefix = "adapterFor", typeFormal.name) | ||
| val newName = p.resolvedNameMaker.unusedSourceNameWithPrefix(prefix = "adapterFor", typeFormal.name) |
There was a problem hiding this comment.
nameMaker is now available from Planting, so I just renamed nameMaker to avoid collisions.
| pos: Position, | ||
| treeInnards: List<TreeInnard>, | ||
| flowMaker: FlowMaker?, | ||
| ): Pair<List<Tree>, FlowMaker?> { |
There was a problem hiding this comment.
Reusing the same infer-position from minimal position info tricks for control flow boundaries.
| flowStack.last().add(BlockChildReference(children.lastIndex, innardPos)) | ||
| } | ||
| is FlowMark -> when (innard.flowMarkKind) { | ||
| FlowMarkKind.StartBlock -> flowStack.add(BlockInProgress(innardPos)) |
There was a problem hiding this comment.
The idea is similar to AST lifting.
We build a list of marks: start-block a-tree end-block
Then we walk it matching starts and ends, and it all works out because the only way to edit the list is via the capitalized methods that pair marks up.
| } | ||
|
|
||
| private class UnpositionedBlockTemplate( | ||
| val flowMaker: FlowMaker = linearFlowMaker, |
There was a problem hiding this comment.
TODO: maybe get rid of this flowMaker. I think I might've already whacked them all, possibly outside UnsetTerminalExpressionsTest.
There was a problem hiding this comment.
This one still needs debugging.
| val oldContent = (oldValue as? JsonString)?.s | ||
| var adjustedString: String? = null | ||
|
|
||
| // If we have any `##` lines, then try to reinsert them in sensible places by diffing. |
There was a problem hiding this comment.
The above part of this file was extracted with some refactoring from AssertModuleAtStage but all the content adjustment tricks are new.
Tests for it in next added file.
mikesamuel
left a comment
There was a problem hiding this comment.
GH is demanding:
You need to leave a comment indicating the requested changes.
But I am me.
I also rewrote some test harness code to use the new way of specifying control flow, and MaximalPaths does a better job eliminating useless transitions. Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
| Stmt("foo"), | ||
| Stmt("bar"), | ||
| ) | ||
| Do { |
There was a problem hiding this comment.
Converted all of these to use the new treeFarm equivalents.
| @@ -23,9 +25,8 @@ class MaximalPathTest { | |||
| | | |||
| |Path#0 | |||
| """.trimMargin(), | |||
| expectedTerminalExpressions = "Sometimes", | |||
There was a problem hiding this comment.
No longer testing the terminal expressions stuff since it's now based on the CF block structure.
| @@ -330,40 +297,36 @@ class MaximalPathTest { | |||
| assertMaximalPaths( | |||
| want = """ | |||
| |Entry Path#0 | |||
| |Exits Path#5 | |||
| |Exits Path#3 | |||
There was a problem hiding this comment.
Checked these loop test changes by dumping to mermaid.
| |-> Path#1 | ||
| | | ||
| |Path#1 | ||
| |- ref#1: `body()` |
There was a problem hiding this comment.
Quite a few changes like this since I fixed some bugs in eliminateEmptyTransitions.
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…ing or clauses Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…Test to relate types to control flow elements Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…rust the fixup that pulls them into the loop body Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
… of using a temporary Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
The coroutine converter is driven by backend code because the decision whether to convert is made per-backend. The original coroutine converter straddles the frontend IR and an IR used during TmpL translation, Pretranslated. This makes it complex and hard to debug. It is a recurring maintenance headache. This commit introduces a new coroutine converter as a frontend IR -> frontend IR transform, with its own test suite. Hopefully this will prove easier to maintain. The new converter also properly handles failures during coroutine stepping whereas that was deferred in the old converter. Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…erter This integrates the new coroutine converter into TmpL translation and removes the old code from the branch. It also fixes bugs that diffing the new converter output against the old exposed and does some improved simplification to improve the output. The old converter directly generated `TmpL.ReturnStatement`s but the new one, which generates frontend IR trees, cannot do that, so this improves *simplifyControlFlow* to recognize when a labeled break inside a loop that is the last statement in a function body is really a break to the end. It also improves *simplifyFunctionBody* to better identify assignments and breaks that are effectively `return`s and to use `TmpL.ReturnStatement`. Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
… backend conventions Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
This commit fixes up backends and gets existing backends working with recent changes to TmpL translation. Since TmpL translation now produces ComputedJumpStatements but not all backends have an analogous statement type, SupportNetwork now has a strategy enum that explains when to use computed jumps instead of `if` chains and when not to. This commit also fixes a CoroutineConverter bug where the function type for the inner function was missing the generator callback type parameter. JS backend mostly works. Python requires a bunch of name mangling adjustments (le sigh) and Java mostly works. Next commits are grinding away on backend tests trying to separate the superfluous & cosmetic changes from ones that might indicate bugs. Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
This commit is a work in progress.
Goal
Simplify frontend representation by eliminating
hs(handler scope) which were previously inserted to make it clear which instructions could fail so that control-flow sensitive analyses were precise.Unfotunately, many instructions were inserted pre-typing leading to imprecise assumptions that many non-failing operations could fail, and requiring expensive CleanupTemporaries passes to reduce the IR to something small enough to debug.
Now, we can eliminate it because failure is based on whether a callee has a declared Result return type.
Changes done
Passes that unify and manipulate control flow were largely rewritten:
return__123variables.Both of these have been tested thoroughly.
Our MaximalPaths representation of a basic block control flow graph has been adapted to work with bubbly branches since we can no longer rely on
fail#123expressions as conditions for transitions to error handling blocks, but it needs more debugging, and passes that rely on it are probably buggy.The staging of some of these μpasses has changed:
Typer can no longer rely on
hs(...)calls at places where a failing result can reach a call as an argument or be assigned to a variable. It used to serve as a typing helper by convertingFoo | BubbletoFoo. Now the Typer explicitly excludes bubbles coming out of calls, and when propagating type information from a typed, bubblyreturn__123variable to a temporary, for example.Still to do
Most of the staging tests pass, but some that interpret
or...elses or that deal with RTTI checks fail.Need to debug these.
MaximalPaths is probably buggy. Need more testing.
Some static error checks like "function body bubbles but does not declare
throws" might depend on information that now needs to be found by analysis of bubbly calles and CF block structure.Also
I used the AssertModuleAtStage regenerate test outputs trick. I tweaked some of it to preserve comments inside test golden files.