fix(derive): avoid higher-ranked bounds on plain-field WriteToAsyncRef - #1
Conversation
For plain (non-newtype) fields the derive emitted higher-ranked bounds
of the form
for<'trivial> #field_ty: WriteToAsyncRef<'vfs, Vfs>
for<'trivial> <#field_ty as WriteToAsyncRef<'vfs, Vfs>>::Future<'fut>: Future + Send + Unpin + 'fut
where the bound variable 'trivial is never actually used. The
higher-ranked quantification forces rustc to perform uncached
higher-ranked normalization over deeply nested derived structures,
which can blow up compile times exponentially (blocking `cargo check`
for minutes or even hitting an internal compiler error).
Keep only a plain (non-higher-ranked) `WriteToAsyncRef<'vfs, Vfs>`
bound on the impl header, and constrain the associated `Future` via a
plain, non-higher-ranked `'fut` bound on the future enum's where
clause. The redundant `Send` bound is dropped while the `Unpin` bound
required by `poll` is kept. The newtype branch is unaffected.
dnbln
left a comment
There was a problem hiding this comment.
Hi, and thank you for your interest in contributing! It seems like trivial bounds improved quite a bit in rustc since I initially wrote this code, this trick with the HRTB used to be required to get the code to even compile (rust-lang/rust#48214), but it seems like they're no longer necessary now, so it's probably time to get rid of those 'trivial lifetimes. Just a small comment.
| async_write_ref_future.clauses_ref_vfs = true; | ||
| let bound = vec![ | ||
| parse_quote! { | ||
| for<'trivial> #actual_field_ty_perform: ::dir_structure::traits::asy::WriteToAsyncRef<'vfs, Vfs> |
There was a problem hiding this comment.
This bound is still necessary when #actual_field_ty_perform isn't a trivial type, as its part of the bounds that make it on the impl WriteToAsyncRef for YourStructure where ..., and if YourStructure has generics itself that might become a problem (edit: this comment applies to the other bound with the future, I see this one is still there).
I'd do this a bit differently:
diff --git a/dir-structure-macros/src/dir_structure_async/write_to_async_ref.rs b/dir-structure-macros/src/dir_structure_async/write_to_async_ref.rs
index 27754ac..fb397ef 100644
--- a/dir-structure-macros/src/dir_structure_async/write_to_async_ref.rs
+++ b/dir-structure-macros/src/dir_structure_async/write_to_async_ref.rs
@@ -91,18 +91,27 @@ pub(super) fn expand_dir_structure_for_field(
}
None => {
async_write_ref_future.clauses_ref_vfs = true;
+ let write_to_async_ref_bound: WherePredicate = parse_quote! {
+ #actual_field_ty_perform: ::dir_structure::traits::asy::WriteToAsyncRef<'vfs, Vfs>
+ };
+ let fut_clause = quote! {
+ <#actual_field_ty_perform as ::dir_structure::traits::asy::WriteToAsyncRef<'vfs, Vfs>>::Future<'fut>: ::std::future::Future<Output = ::dir_structure::error::VfsResult<(), Vfs>> + ::std::marker::Unpin + 'fut
+ };
let bound = vec![
+ write_to_async_ref_bound.clone(),
parse_quote! {
- for<'trivial> #actual_field_ty_perform: ::dir_structure::traits::asy::WriteToAsyncRef<'vfs, Vfs>
+ for<'fut> #fut_clause
},
+ ];
+ async_write_ref_future.clauses.extend([
+ write_to_async_ref_bound,
parse_quote! {
- for<'trivial> <#actual_field_ty_perform as ::dir_structure::traits::asy::WriteToAsyncRef<'vfs, Vfs>>::Future<'fut>: ::std::future::Future<Output = ::dir_structure::error::VfsResult<(), Vfs>> + ::std::marker::Send + ::std::marker::Unpin + 'fut
+ #fut_clause
},
- ];
- async_write_ref_future.clauses.extend(bound.clone());
- async_write_ref_future.clauses.push(parse_quote! {
- 'vfs: 'fut
- });
+ parse_quote! {
+ 'vfs: 'fut
+ },
+ ]);
(
quote! {
There was a problem hiding this comment.
Thanks for the review and the suggestions! This is my first PR, so I truly appreciate your guidance and apologize for any non-standard practices.
Also, I have to say this library is fantastic, it's been a huge help for my project's protocol design.
I've pushed the requested changes here.
One thing to note: in my real-world use case, this specific implementation causes compilation times to spike to ~20 minutes.
For comparison, the version in this branch finishes in under 1 minute.
I’ve included a test script to help you reproduce/verify this compilation bottleneck.
dir-structure.rs.sh
Problem
When a struct that derives
DirStructurecontains plain (non-newtype) fieldsand is nested several levels deep,
cargo check/cargo buildcan stall forminutes or even hit an
internal compiler error. The trigger is thehigher-ranked (forall-quantified) bounds that the derive currently emits for the
plain-field branch of the generated
WriteToAsyncReffuture:The bound variable
'trivialis never used, and the higher-rankedquantification forces rustc into uncached higher-ranked normalization over the
recursively nested derived structures, which blows up exponentially.
Reproduction
We reproduced this against this crate at
trunk(ca091dd) with a real-worldconsumer: a project whose storage layer derives
DirStructureon a deeplynested set of
#[dir_structure(path = ...)]structs/enums that use plainfields through
WriteToAsyncRef.not finish
cargo checkafter 10+ minutes (rustc pegged at 100% CPU, noprogress), where the same crate compiles in ~1–2 min on the same machine with
the fix applied.
plain-field (
None) branch; the newtype (Some) branch and thedir-structure-toolsbounds are untouched (verified by ablation).Fix
for<'trivial>binder.WriteToAsyncRef<'vfs, Vfs>bound on the impl header.Futurevia a plain, non-higher-ranked'futbound on the future enum's where clause (keeps the
Unpinbound needed bypoll, drops the redundantSend).Validation
cargo test -p dir-structure --all-features --lib --testspasses.cargo test -p dir-structure-tools --all-features --testspasses(30 + 16 + 8 tests).