Repository navigation
Rust 1.59 rustc greatly increased compile time with include_str! #94390
Description
Activity
- changed the title
[-]Rust 1.59 rustc ~infinite loop with include_str![/-][+]Rust 1.59 rustc greatly increased compile time with include_str![/+]on Feb 26, 2022 From poking at the codebase a bit I have determined that
include_str!()itself is not the direct cause of the issue, rather it is the optimizer atopt-level = 2trying to optimize theserde_json::from_str::<Translations>()function given compile time knowledge of what the json string looks like. The issue also occurs if you manually inline the stringI have created a much smaller code sample that recreates this bug https://github.com/MarkDDR/long_compile_rustc_1.59.0
Reacted by Sean Stangl and Carson McManusThanks for putting together the reproductions! This seems to have started with #92419. cc @erikdesjardins
Presumably there are more issues with runaway inlining (possibly the same thing as #89524 / https://reviews.llvm.org/D98481 - serde almost certainly has mutual recursion somewhere).
It is unlikely that there will be a trivial fix, and I don't have time to investigate it deeply right now. I'll open a revert in a bit.
- added a commit that references this issue
on Mar 1, 2022 - added a commit that references this issue
on Mar 3, 2022 I looked into this a bit, and I'm not sure this qualifies as a catastrophic inlining issue. The input IR for
"_ZN189_$LT$long_compile_time..big_struct.._..$LT$impl$u20$serde..de..Deserialize$u20$for$u20$long_compile_time..big_struct..BigStruct$GT$..deserialize..__Visitor$u20$as$u20$serde..de..Visitor$GT$9visit_map17h6f5c5ed07f452137E"is already more than 50k lines to start with and "only" grows to 140k lines during optimization. The result is huge because the input is huge.Huge functions have a high chance of hitting one or the other super-linear optimization behavior in LLVM. I fixed one issue this hits in llvm/llvm-project@1dbeb64, which drops compile-time from 3:55 to 2:45.
I personally wouldn't block reapplication of the patch on this issue, as the interaction here is somewhat incidental -- large machine-generated code will always run into these sorts of issues, and minor perturbations in the generated code, rustc or LLVM can cause major variations in compile-time. Serde should avoid generating a large amount of code into a single function.
5 remaining items
On my mac, with
-> rustc --version --verbose rustc 1.75.0-nightly (2e5a9dd6c 2023-10-02) binary: rustc commit-hash: 2e5a9dd6c9eaa42f0684b4b760bd68fc27cbe51b commit-date: 2023-10-02 host: aarch64-apple-darwin release: 1.75.0-nightly LLVM version: 17.0.2cargo build, following the reproduction steps listed in the first comment of this issue takes 5 minutes and 18 seconds.- added a commit that references this issue
on Oct 8, 2023
The following repo/hash compiles with rustc v1.58, but loops forever in v1.59 and Nightly (1.61): sstangl/openpowerlifting@ac057e3
The problem appears to be with the compilation of a very small crate (
modules/langpack) that usesinclude_str!to include some JSON files. The problematic method is the implementation ofdefault():When this function is removed, compilation succeeds.
GDB shows a thread stuck within LLVM, doing various optimizations, in a seemingly-endless loop (I waited several minutes. Previously this compilation with 1.58.1 took ~32s):
Steps to Reproduce
git clone https://github.com/sstangl/openpowerlifting.gitgit checkout ac057e388ac911fda2e93acd8689d13f6df23034cd modules/langpackcargo buildMeta
rustc --version --verbose:The problem also exists with Nightly: