Slot-based C# syntax tree via a Roslyn source generator

[siegfriedpammer]

Overview

This reworks the C# AST in ICSharpCode.Decompiler/CSharp/Syntax/ from the NRefactory
role + doubly-linked-list child model to a slot-based model (in the spirit of the ILAst),
with the per-node boilerplate emitted by a Roslyn source generator, and sheds the NRefactory
machinery the decompiler does not need. The end state also turns on nullable reference types across
the AST and its consumers.

Each child of a node is declared once as a [Slot] partial property; the generator owns everything
mechanical derived from those declarations.

What changes

• Source generator for the AST: from [DecompilerAstNode]/[Slot] declarations it emits the
  visitor interface, AcceptVisitor, DoMatch, the slot accessors and flattened child-index API
  (GetChildCount/GetChild/SetChild/GetChildSlotInfo/GetCollectionByKind/CloneChildrenInto),
  a single-pass child enumerator, the per-node constructors (parameters in source order), and
  node.Slot, the per-node CSharpSlotInfo<T>, and a generated Slots holder of the canonical kinds. A [Slot] on an AstNode-typed property
  is a child slot; a [Slot] on a string property is a name -- a convenience string over a
  backing Identifier token slot the generator owns (the property type disambiguates, so no separate
  attribute is needed).
• Storage flip: children move from a per-node linked list to real slot backing fields, with
  AstNodeCollection<T> backed by a List<T>.
• CheckInvariant (DEBUG only, the analog of ILInstruction.CheckInvariant): runs before the
  transform pipeline and after every transform. It recursively verifies the slot structure (each
  child's parent back-pointer, its flattened index, and its slot type) and that every required
  (non-optional) slot is filled, and node types override it to assert their own scalar constraints --
  the invariants formerly enforced by throwing setters (e.g. ComposedType.PointerRank >= 0,
  ArraySpecifier.Dimensions >= 1) now read as auto-properties checked there.
• Drop token nodes: CSharpTokenNode/CSharpModifierToken and InsertSpecialsDecorator are
  gone. Punctuation/keyword/operator text lives on the nodes; modifiers, specifiers, the await flag,
  etc. become scalar fields. Source locations are stored at print time instead of recomputed from
  child token nodes.
• Comments and directives become trivia on a leading/trailing side-channel, off the child-index
  space (like annotations), rather than positional children.
• Null-objects to nullable reference types: optional single-child slots become T? (with the C#
  grammar as the oracle for optionality), the null-object pattern is deleted, and #nullable enable
  is turned on across the transforms, the AST consumer layer (output visitor, the IL-to-C# builders,
  TypeSystemAstBuilder, the translation-result wrappers, sequence-point/namespace collectors, the
  annotation helpers), and the syntax node files themselves. Optional names follow suit: a name that
  may be absent is typed string? and reads as null when absent (rather than a non-null empty-string
  sentinel); required names stay non-null string.
• Delete Role/Role<T> and TokenRole: child positions are keyed on typed slot objects --
  node.GetChild(Slots.Left) infers the child type and node.Slot.Kind == Slots.Initializer is matched
  by object identity (there is no kind enum); token text is carried as plain constants.
• Pattern matcher ported to list-indexed slot children (slim INode, list-by-index collection
  matching, with a nullable other so a pattern can match against an absent child); generated
  DoMatch matches every structural member, which fixed a few latent under-matching bugs.
• Misc: AST node classes are annotated with their C# grammar productions; trivial scalar
  properties become auto-properties; leaf node classes are sealed; dead parsing-only state and the
  NodeType enum are removed. The now-unused per-node flags word was dropped (its only user,
  Identifier.IsVerbatim, is a plain bool), which the CLR's field-padding makes free on the node
  carrying it while shrinking every other node by 8 bytes -- a measured win that a planned bit-packing
  scheme did not deliver (the CLR already packs scalars into padding).

Behavior and testing

This is a representation change, not an output change: the Pretty test suite stays byte-identical
throughout, and the full decompiler suite is green on top of current master. Each commit builds the
decompiler libraries, ilspycmd, and the ILSpy UI, and the suite is green per logical step. Generated
DoMatch made several matchers stricter where the hand-written ones under-matched; those are captured
by the pattern-matching tests.

Notes

• The history is organised into logical, self-contained commits; the early commits are the original
  generator bring-up, the later ones the migration on top.
• Nullable reference types are enabled across the AST and its CSharp/ consumers; the semantic
  resolver layer (CSharp/Resolver/) is intentionally out of scope here.
• CSharp/Syntax/README.md documents the generator,
  its attributes, the slot/kind model, CheckInvariant, and how to add a node.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

[siegfriedpammer]

Posted by an AI agent (Claude) on Siegfried's behalf.

---

Reviewed the full diff with focused passes over the source generator, the AST runtime, the pattern matcher, output/locations, the null-object→nullable migration, and the node declarations.

Built clean in Release and Debug (0 warnings / 0 errors, no packages.lock.json churn). Ran the in-memory AST tests locally: PatternMatching and LocationsInAst are green (35/36; the one failure is a missing legacy reference-assembly in my environment, not the change). The matcher's index-alignment, checkpoint save/restore, and null handling also pass an out-of-process probe harness.

No correctness blockers. The inline notes are minor: one stale comment, two DoMatch tightenings that have no live consumer, an enumerator-semantics narrowing, a narrowed backreference contract, and latent (currently-unread) metadata in the generator.

Non-blocking coverage gaps (all pass when probed, just not pinned): the matcher's absent-single-optional-slot, null-capture, Repeat+OptionalNode, and backreference-across-Repeat-backtrack paths in PatternMatchingTests; and the bare catch { (type-less) sequence-point span in LocationsInAstSamples.