[rjb32]

Implement edge scans with both ends bound to a label.

reactome, in-memory, median of 7 runs, speedup = 7c03a3426 (main) time / 92faa8ad2 time
A: dump as loaded, 1 datapart
B: A + 102 commits + 5,000 nodes created then deleted

query                                              A         B
(:Pathway)-->(:Species)                            2.72x     1.83x
(:EntityWithAccessionedSequence)-->(:Compartment)  2.49x     2.62x
(:Reaction)-->(:Complex)                           2.80x     2.86x
(:Complex)<--(:Reaction)                           1.53x     1.46x
(a)-->(:Species) WHERE a:Pathway                   8.12x     8.17x
(:Person)-->(:InstanceEdit)                        1.60x     1.63x
(:Pathway)-->(b) WHERE b:Species OR b:Compartment  1.18x     0.99x
count(n) PhysicalEntity                            0.59x     2.44x
count(n) DatabaseIdentifier                        0.94x     5.02x
count(n.displayName) PhysicalEntity                0.88x   805.56x

[rjb32]

Note

AI-generated summary.

Summary

A hop whose two ends both carry labels now runs as one edge scan. The scan walks one end's by-label edge index and checks the other end's labels per edge, so the label filter and its rows are gone. Codegen is unchanged; FuseScanEdgesByEndpointLabel builds the new ops.

 MATCH (a)-->(s:Species) WHERE a:Pathway RETURN count(*)
-%0, %1, %2, %3 = db.scan_out_edges_by_label_tgt(["Species"])
-%4 = db.get_node_label_set(%0)
-%5 = db.check_label_constraint(%4, ["Pathway"])
-%6 = db.filter(%5, {%0})
-%7 = db.count(%6) rows
+%0, %1, %2, %3 = db.scan_out_edges_by_label_src_tgt(["Pathway"], ["Species"])
+%4 = db.count(%0) rows

The pass picks which end's index to walk from the edges each index holds under its labels. The count is the size of the walk, read from the existing EdgeIndexer spans:

indexesByTarget(scan)
  no graph view            -> keep the side the scan already walked
  targetInEdges < sourceOutEdges -> target index (db.scan_in_edges_by_label_src_tgt)
  otherwise                -> source index (db.scan_out_edges_by_label_src_tgt)

Two storage counts that count_from_metadata answers from were also reworked. Deleted nodes, and property entries a newer datapart overwrites, are now found from the smaller side, instead of checking every entry against every later datapart.

The branch has a binary search for the datapart owning a node (06f3347ed) and its revert (92faa8ad2). It slowed down label lookups on reactome, where nearly every node is in the first datapart.

Evidence

• Before: MATCH (a)-->(s:Species) WHERE a:Pathway RETURN count(*) walks the Species in-index (742,452 edges) and filters each row on a:Pathway: 11.11 ms.
  After: one scan of the Pathway out-index (269,508 edges): 1.37 ms. Same count, 23,620.

Reactome, median of 7 runs of the server's own time, in-memory servers. Speedup is the 7c03a3426 time divided by the 92faa8ad2 time. A is the dump as loaded (1 datapart). B adds 102 commits and 5,000 deleted nodes, the same writes on both. Every query returns the same result on both binaries.

query                                              A         B
(:Pathway)-->(:Species)                            2.72x     1.83x
(:EntityWithAccessionedSequence)-->(:Compartment)  2.49x     2.62x
(:Reaction)-->(:Complex)                           2.80x     2.86x
(:Complex)<--(:Reaction)                           1.53x     1.46x
(a)-->(:Species) WHERE a:Pathway                   8.12x     8.17x
(:Person)-->(:InstanceEdit)                        1.60x     1.63x
(:Pathway)-->(b) WHERE b:Species OR b:Compartment  1.18x     0.99x
count(n) PhysicalEntity                            0.59x     2.44x
count(n) DatabaseIdentifier                        0.94x     5.02x
count(n.displayName) PhysicalEntity                0.88x   805.56x

Tests, all passing; ctest runs 864 of 864 at 92faa8ad2:

ScanEdgesByEndpointLabelsTest              13  both-ends scans end to end on simpledb, index side and rows
FuseScanEdgesByEndpointLabelTest           23  the pass on hand-written db programs, rows before and after fusion
EdgeScanIndexCostTest                       4  hub graph: 2 Hub nodes with 105 out-edges vs 5 Target nodes with 5 in-edges
ScanEdgesByEndpointLabelsChunkWriterTest    4  fills of 1-4 rows, deleted edges, both index sides
EdgeCountMatchingLabelsetTest               2  edge counts equal the rows the by-label writer walks
NodeCountMatchingLabelsetTest               2  7 deleted nodes, both ways of finding them
NodeCountWithPropertyExclusionsTest         2  overwritten, nulled and deleted entries across 4 commits

Before the pass chose the index side, 4 ScanEdgesByEndpointLabelsTest cases failed. Under a node-count rule, the 2 hub cases of EdgeScanIndexCostTest failed. The chunk writer test and the two node-count tests each fail when the path they cover is broken on purpose.

Merge Danger

Door: two-way

No dump, wire or ValueType format changes. The new db and nl ops exist only in the IR, so a revert restores the old plans.

Blast Radius: planner-wide

Every MATCH with labels on both ends of a hop now takes the new scan, and may emit rows in a different order. Two V3QueryTestSuite fixtures with no ORDER BY were reordered for this (join-comma-shared-variable, match-call-wildcard-3); the rows are unchanged. count(n) and count(n.prop) answered from metadata go through the reworked counts.