Skip to content

frontend: eliminate hs as an error handling primitive. - #474

Draft
mikesamuel wants to merge 24 commits into
mainfrom
no-handler-scopery
Draft

frontend: eliminate hs as an error handling primitive.#474
mikesamuel wants to merge 24 commits into
mainfrom
no-handler-scopery

Conversation

@mikesamuel

Copy link
Copy Markdown
Contributor

This commit is a work in progress.

Goal

Simplify frontend representation by eliminating hs (handler scope) which were previously inserted to make it clear which instructions could fail so that control-flow sensitive analyses were precise.

Unfotunately, many instructions were inserted pre-typing leading to imprecise assumptions that many non-failing operations could fail, and requiring expensive CleanupTemporaries passes to reduce the IR to something small enough to debug.

Now, we can eliminate it because failure is based on whether a callee has a declared Result return type.

Changes done

Passes that unify and manipulate control flow were largely rewritten:

  • MakeResultsExplicit now works based on the nesting structure of control flow block elements to capture function body and module results in return__123 variables.
  • Weaver now uses the same block structure instead of working on a CFG it builds using the block structure.

Both of these have been tested thoroughly.

Our MaximalPaths representation of a basic block control flow graph has been adapted to work with bubbly branches since we can no longer rely on fail#123 expressions as conditions for transitions to error handling blocks, but it needs more debugging, and passes that rely on it are probably buggy.

The staging of some of these μpasses has changed:

  1. MakeResultsExplicit now runs first. It allocates return variables early.
  2. Weaver now calls MagicSecurityDust.sprinkle internally, and has been rewritten and is probably quite a bit faster and simpler. It allocates far fewer temporaries.

Typer can no longer rely on hs(...) calls at places where a failing result can reach a call as an argument or be assigned to a variable. It used to serve as a typing helper by converting Foo | Bubble to Foo. Now the Typer explicitly excludes bubbles coming out of calls, and when propagating type information from a typed, bubbly return__123 variable to a temporary, for example.

Still to do

Most of the staging tests pass, but some that interpret or...elses or that deal with RTTI checks fail.
Need to debug these.

MaximalPaths is probably buggy. Need more testing.

Some static error checks like "function body bubbles but does not declare throws" might depend on information that now needs to be found by analysis of bubbly calles and CF block structure.

Also

I used the AssertModuleAtStage regenerate test outputs trick. I tweaked some of it to preserve comments inside test golden files.

This commit is a work in progress.

Goal
====
Simplify frontend representation by eliminating `hs` (handler scope)
which were previously inserted to make it clear which instructions
could fail so that control-flow sensitive analyses were precise.

Unfotunately, many instructions were inserted pre-typing leading
to imprecise assumptions that many non-failing operations could fail,
and requiring expensive *CleanupTemporaries* passes to reduce the IR
to something small enough to debug.

Now, we can eliminate it because failure is based on whether a callee
has a declared *Result* return type.

Changes done
============
Passes that unify and manipulate control flow were largely rewritten:

- *MakeResultsExplicit* now works based on the nesting structure of
  control flow block elements to capture function body and module
  results in `return__123` variables.
- *Weaver* now uses the same block structure instead of working on
  a CFG it builds using the block structure.

Both of these have been tested thoroughly.

Our *MaximalPaths* representation of a basic block control flow graph
has been adapted to work with *bubbly* branches since we can no longer
rely on `fail#123` expressions as conditions for transitions to error
handling blocks, but it needs more debugging, and passes that rely on
it are probably buggy.

The staging of some of these μpasses has changed:

1. *MakeResultsExplicit* now runs first.
   It allocates return variables early.
2. *Weaver* now calls *MagicSecurityDust.sprinkle* internally, and
   has been rewritten and is probably quite a bit faster and simpler.
   It allocates far fewer temporaries.

*Typer* can no longer rely on `hs(...)` calls at places where a
failing result can reach a call as an argument or be assigned to
a variable.  It used to serve as a typing helper by converting
`Foo | Bubble` to `Foo`.  Now the *Typer* explicitly excludes bubbles
coming out of calls, and when propagating type information from
a typed, bubbly `return__123` variable to a temporary, for example.

Still to do
===========
Most of the staging tests pass, but some that interpret `or...else`s or
that deal with RTTI checks fail.
Need to debug these.

*MaximalPaths* is probably buggy.  Need more testing.

Some static error checks like "function body bubbles but does not declare
`throws`" might depend on information that now needs to be found by analysis
of bubbly calles and CF block structure.

Also
====
I used the *AssertModuleAtStage* regenerate test outputs trick.
I tweaked some of it to preserve comments inside test golden files.

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
*
* See also [lang.temper.frontend.MagicSecurityDust].
*/
private object HandlerScopeFn : SpecialFunction, NamedBuiltinFun {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not out of scope; it's the scopes that are out.

.also {
helpSnippet(it, "Boolean inverse", "builtin/!")
}
val notFn: CallableValue = NotFn

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

control flow stuff becomes easier if we can refer to ! from fundamentals.

{ Rn(name) },
)
Block {
If(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commit adds machinery to TreeFarm to allow building control-flow block structures directly.

That's really helpful for testing, but one nice side effect is you can just use it in frontend code too.

internal.insert(internal.size) {
V(vInitSymbol)
// TODO Could implement a coalesce macro with this content.
Call {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was kind of gnarly: building a call to the if macro. Now it just uses the tree farm.

for (typeFormal in typeFormals) {
if (typeFormal.needsAdapting) {
val newName = p.nameMaker.unusedSourceNameWithPrefix(prefix = "adapterFor", typeFormal.name)
val newName = p.resolvedNameMaker.unusedSourceNameWithPrefix(prefix = "adapterFor", typeFormal.name)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nameMaker is now available from Planting, so I just renamed nameMaker to avoid collisions.

pos: Position,
treeInnards: List<TreeInnard>,
flowMaker: FlowMaker?,
): Pair<List<Tree>, FlowMaker?> {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reusing the same infer-position from minimal position info tricks for control flow boundaries.

flowStack.last().add(BlockChildReference(children.lastIndex, innardPos))
}
is FlowMark -> when (innard.flowMarkKind) {
FlowMarkKind.StartBlock -> flowStack.add(BlockInProgress(innardPos))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is similar to AST lifting.

We build a list of marks: start-block a-tree end-block

Then we walk it matching starts and ends, and it all works out because the only way to edit the list is via the capitalized methods that pair marks up.

}

private class UnpositionedBlockTemplate(
val flowMaker: FlowMaker = linearFlowMaker,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: maybe get rid of this flowMaker. I think I might've already whacked them all, possibly outside UnsetTerminalExpressionsTest.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one still needs debugging.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

val oldContent = (oldValue as? JsonString)?.s
var adjustedString: String? = null

// If we have any `##` lines, then try to reinsert them in sensible places by diffing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above part of this file was extracted with some refactoring from AssertModuleAtStage but all the content adjustment tricks are new.

Tests for it in next added file.

@mikesamuel mikesamuel left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GH is demanding:

You need to leave a comment indicating the requested changes.

But I am me.

I also rewrote some test harness code to use the new way of specifying
control flow, and MaximalPaths does a better job eliminating useless
transitions.

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Stmt("foo"),
Stmt("bar"),
)
Do {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted all of these to use the new treeFarm equivalents.

@@ -23,9 +25,8 @@ class MaximalPathTest {
|
|Path#0
""".trimMargin(),
expectedTerminalExpressions = "Sometimes",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer testing the terminal expressions stuff since it's now based on the CF block structure.

@@ -330,40 +297,36 @@ class MaximalPathTest {
assertMaximalPaths(
want = """
|Entry Path#0
|Exits Path#5
|Exits Path#3

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked these loop test changes by dumping to mermaid.

|-> Path#1
|
|Path#1
|- ref#1: `body()`

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite a few changes like this since I fixed some bugs in eliminateEmptyTransitions.

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…ing or clauses

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…Test to relate types to control flow elements

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…rust the fixup that pulls them into the loop body

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
… of using a temporary

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
The coroutine converter is driven by backend code because the decision
whether to convert is made per-backend.

The original coroutine converter straddles the frontend IR and an
IR used during TmpL translation, Pretranslated.

This makes it complex and hard to debug.
It is a recurring maintenance headache.

This commit introduces a new coroutine converter as a frontend IR
-> frontend IR transform, with its own test suite.

Hopefully this will prove easier to maintain.

The new converter also properly handles failures during coroutine
stepping whereas that was deferred in the old converter.

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
…erter

This integrates the new coroutine converter into TmpL translation and
removes the old code from the branch.

It also fixes bugs that diffing the new converter output against the
old exposed and does some improved simplification to improve the output.
The old converter directly generated `TmpL.ReturnStatement`s but the
new one, which generates frontend IR trees, cannot do that, so this
improves *simplifyControlFlow* to recognize when a labeled break
inside a loop that is the last statement in a function body is really
a break to the end.  It also improves *simplifyFunctionBody* to better
identify assignments and breaks that are effectively `return`s and to
use `TmpL.ReturnStatement`.

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
… backend conventions

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
This commit fixes up backends and gets existing backends
working with recent changes to TmpL translation.

Since TmpL translation now produces ComputedJumpStatements
but not all backends have an analogous statement type,
SupportNetwork now has a strategy enum that explains when
to use computed jumps instead of `if` chains and when not to.

This commit also fixes a CoroutineConverter bug where the
function type for the inner function was missing the generator
callback type parameter.

JS backend mostly works.  Python requires a bunch of name mangling
adjustments (le sigh) and Java mostly works.  Next commits are
grinding away on backend tests trying to separate the superfluous &
cosmetic changes from ones that might indicate bugs.

Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Signed-off-by: Mike Samuel <mikesamuel@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant