Remap syntax contexts and expansion ids for det. metadata encoding by aerooneqq · Pull Request #163668 · rust-lang/rust

aerooneqq · 2026-10-02T15:49:21Z

#163495 which parallelizes AST -> HIR lowering breaks det. encoding of syntax contexts and expansions, however this PR with #162809 should fix the problem.

Now draft for perf and CI (dont look at the code). Remapping of those entities can cost us much, because unlike def ids there may be many expansions and contexts after we committed end of determinism.

r? @petrochenkov

petrochenkov · 2026-10-02T15:50:24Z

@bors try @rust-timer queue

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

petrochenkov · 2026-10-02T15:59:46Z

@bors try cancel

rust-bors · 2026-10-02T15:59:49Z

Try build cancelled. Cancelled workflows:

https://github.com/rust-lang/rust/actions/runs/37029803876

Hint: if you want to run another try build, you do not need to manually cancel the previous one. Just run @bors try and bors will cancel the previous build automatically.

petrochenkov · 2026-10-02T15:59:56Z

@bors try @rust-timer queue

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

aerooneqq · 2026-10-05T14:37:35Z

@bors try @rust-timer queue

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

rust-bors · 2026-10-05T15:37:35Z

☀️ Try build successful (CI)
Build commit: a7c6732 (a7c67326654d086b506b6f55d82af110f520b88b)
Base parent: 602727f (602727f26878dcf9eb2999e2f58e5df1269ff47e)

rust-timer · 2026-10-05T16:22:44Z

Finished benchmarking commit (a7c6732): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.3%, 1.2%]	7
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.2%	[-1.2%, -0.1%]	57
Improvements ✅ (secondary)	-0.3%	[-0.7%, -0.1%]	28
All ❌✅ (primary)	-0.1%	[-1.2%, 1.2%]	64

Max RSS (memory usage)

Results (primary -2.9%, secondary -1.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.1%	[3.1%, 3.1%]	1
Regressions ❌ (secondary)	2.6%	[1.6%, 6.6%]	9
Improvements ✅ (primary)	-3.0%	[-8.3%, -0.9%]	112
Improvements ✅ (secondary)	-3.1%	[-5.4%, -1.2%]	29
All ❌✅ (primary)	-2.9%	[-8.3%, 3.1%]	113

Cycles

Results (primary 11.8%, secondary 2.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	11.8%	[2.3%, 27.3%]	15
Regressions ❌ (secondary)	3.8%	[3.0%, 4.5%]	5
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.4%	[-4.4%, -4.4%]	1
All ❌✅ (primary)	11.8%	[2.3%, 27.3%]	15

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 490.152s -> 495.841s (1.16%)
Artifact size: 408.63 MiB -> 408.61 MiB (-0.00%)

rust-timer · 2026-10-13T17:56:34Z

Finished benchmarking commit (1e274b8): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.1%, 0.3%]	2
Improvements ✅ (primary)	-0.2%	[-0.4%, -0.1%]	32
Improvements ✅ (secondary)	-0.4%	[-0.8%, -0.1%]	14
All ❌✅ (primary)	-0.2%	[-0.4%, -0.1%]	32

Max RSS (memory usage)

Results (secondary 1.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	6.1%	[2.6%, 9.9%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.6%	[-4.8%, -4.5%]	2
All ❌✅ (primary)	-	-	0

Cycles

Results (primary 0.8%, secondary 1.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	5.1%	[5.1%, 5.1%]	1
Regressions ❌ (secondary)	3.9%	[2.3%, 5.5%]	2
Improvements ✅ (primary)	-3.5%	[-3.5%, -3.5%]	1
Improvements ✅ (secondary)	-3.0%	[-3.0%, -3.0%]	1
All ❌✅ (primary)	0.8%	[-3.5%, 5.1%]	2

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 490.893s -> 487.782s (-0.63%)
Artifact size: 406.79 MiB -> 406.98 MiB (0.05%)

aerooneqq · 2026-10-15T10:52:48Z

@rustbot ready

petrochenkov · 2026-10-15T11:32:21Z

-        let idx = self.hygiene_ctxt.borrow_mut().get_syntax_ctxt_encoding_index(syntax_context);
-        idx.encode(self);
+        self.hygiene_ctxt.borrow_mut().enqueue_syntax_context_for_encoding(syntax_context);
+        self.emit_u32(self.syntax_context_remapping.map_id(syntax_context).into());


Suggested change

self.emit_u32(self.syntax_context_remapping.map_id(syntax_context).into());

self.emit_u32(self.syntax_context_remapping.map_id(syntax_context).as_u32());

It's usually better not to use into outside of generic contexts.
(Same in a few other places.)

View changes since the review

Turns out SyntaxContext has (almost) all methods of rustc_index::newtype_index! but it is not an rustc_index::newtype_index!, maybe we should rework it through rustc_index::newtype_index!?

Turns out SyntaxContext has (almost) all methods of rustc_index::newtype_index! but it is not an rustc_index::newtype_index!, maybe we should rework it through rustc_index::newtype_index!?

Yes, it would be good to try (not in this PR), maybe it was just missed when similar newtypes were migrated to newtype_index!.

petrochenkov · 2026-10-15T11:58:59Z

Some style nits and one remaining bigger problem (#163668 (comment)).
@rustbot author

rustbot · 2026-10-15T11:59:04Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

aerooneqq · 2026-10-15T13:28:34Z

@bors try @rust-timer queue

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

rust-bors · 2026-10-15T14:33:00Z

☀️ Try build successful (CI)
Build commit: 16bad69 (16bad6970e588aaf1a6a1209c1dbc71db8f90238)
Base parent: 8a9dd7a (8a9dd7abdf013ec10cfeaaa3685feba18ef1c90c)

rust-timer · 2026-10-15T15:34:50Z

Finished benchmarking commit (16bad69): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.1%, 0.4%]	3
Improvements ✅ (primary)	-0.3%	[-0.8%, -0.1%]	50
Improvements ✅ (secondary)	-0.4%	[-0.9%, -0.1%]	24
All ❌✅ (primary)	-0.3%	[-0.8%, -0.1%]	50

Max RSS (memory usage)

Results (primary 0.4%, secondary 2.5%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.7%	[0.5%, 2.4%]	5
Regressions ❌ (secondary)	3.6%	[2.1%, 5.3%]	11
Improvements ✅ (primary)	-3.0%	[-4.9%, -1.0%]	2
Improvements ✅ (secondary)	-3.6%	[-4.9%, -2.2%]	2
All ❌✅ (primary)	0.4%	[-4.9%, 2.4%]	7

Cycles

Results (primary 2.5%, secondary -3.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.5%	[2.5%, 2.5%]	1
Regressions ❌ (secondary)	3.3%	[3.3%, 3.3%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.1%	[-12.0%, -2.1%]	4
All ❌✅ (primary)	2.5%	[2.5%, 2.5%]	1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 490.736s -> 490.318s (-0.09%)
Artifact size: 407.05 MiB -> 407.18 MiB (0.03%)

aerooneqq · 2026-10-15T15:37:13Z

@rustbot ready
Perf is ok, but code is a bit ugly.

rustbot added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Oct 2, 2026

rustbot assigned petrochenkov Oct 2, 2026

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Oct 2, 2026

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 2, 2026

rust-bors Bot pushed a commit that referenced this pull request Oct 2, 2026

Auto merge of #163668 - aerooneqq:syntax-contexts-and-expansions-ids-…

1fbc989

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

This comment has been minimized.

Sign in to view

aerooneqq force-pushed the syntax-contexts-and-expansions-ids-remapping branch from 4195a79 to 46ee492 Compare October 2, 2026 15:57

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 2, 2026

Auto merge of #163668 - aerooneqq:syntax-contexts-and-expansions-ids-…

19b2816

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

aerooneqq force-pushed the syntax-contexts-and-expansions-ids-remapping branch 2 times, most recently from 07a0f64 to 43e622c Compare October 5, 2026 14:34

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 5, 2026

Auto merge of #163668 - aerooneqq:syntax-contexts-and-expansions-ids-…

a7c6732

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Oct 5, 2026

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Oct 13, 2026

aerooneqq added 4 commits October 15, 2026 13:18

Address some review comments

309f5ae

Address review comments

17dc5b6

Small cleanups

1d7ab78

Connect different hashings of syntax context

90dbc88

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 15, 2026

petrochenkov reviewed Oct 15, 2026

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 15, 2026

aerooneqq added 2 commits October 15, 2026 15:44

Address review comments

e0cfd89

Try stable hash

1498020

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 15, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 15, 2026

Auto merge of #163668 - aerooneqq:syntax-contexts-and-expansions-ids-…

16bad69

…remapping, r=<try> Remap syntax contexts and expansion ids for det. metadata encoding

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 15, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 15, 2026

	self.emit_u32(self.syntax_context_remapping.map_id(syntax_context).into());
	self.emit_u32(self.syntax_context_remapping.map_id(syntax_context).as_u32());

Conversation

aerooneqq commented Oct 2, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented Oct 2, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

petrochenkov commented Oct 2, 2026

Uh oh!

rust-bors Bot commented Oct 2, 2026

Uh oh!

petrochenkov commented Oct 2, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

aerooneqq commented Oct 5, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rust-bors Bot commented Oct 5, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Oct 5, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

This comment has been minimized.

rust-timer commented Oct 13, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

aerooneqq commented Oct 15, 2026

Uh oh!

petrochenkov Oct 15, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aerooneqq Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

petrochenkov commented Oct 15, 2026

Uh oh!

rustbot commented Oct 15, 2026

Uh oh!

aerooneqq commented Oct 15, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors Bot commented Oct 15, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Oct 15, 2026

Overall result: ❌✅ regressions and improvements - please read:

Uh oh!

aerooneqq commented Oct 15, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

aerooneqq commented Oct 2, 2026 •

edited by rustbot

Loading

petrochenkov Oct 15, 2026 •

edited by rustbot

Loading

aerooneqq Oct 15, 2026 •

edited

Loading