msvc: small float aggregates are returned incorrectly with `extern "Rust"`

[folkertdev]

Consider

https://godbolt.org/z/4dYTvseGK

// --target=x86_64-pc-windows-msvc -Copt-level=3
#[repr(C)]
#[derive(Clone, Copy)]
struct Foo {
    a: f32, 
    b: f32,
}

#[unsafe(no_mangle)]
fn foo(foo: &Foo) -> Foo {
    *foo
}

The LLVM backend emits

foo:
        movss   xmm0, dword ptr [rcx]
        movss   xmm1, dword ptr [rcx + 4]
        ret

while cg_clif emits

foo:
 push	rbp
 mov	rbp, rsp
 sub	rsp, 0x30
 movdqu	xmmword ptr [rsp + 0x10], xmm6
 movdqu	xmmword ptr [rsp + 0x20], xmm7
 movss	xmm7, dword ptr [rdx]
 movss	xmm6, dword ptr [rdx + 0x4]
 movss	dword ptr [rsp], xmm7
 movss	dword ptr [rsp + 0x4], xmm6
 movss	xmm0, dword ptr [rsp]
 movss	dword ptr [rcx], xmm6
 movdqu	xmm6, xmmword ptr [rsp + 0x10]
 movdqu	xmm7, xmmword ptr [rsp + 0x20]
 add	rsp, 0x30
 mov	rsp, rbp
 pop	rbp
 ret

A bit of a mess, but note that xmm1 is not written to, and it looks like there is an outpointer which the LLVM also does not have.

This issue showed up in rust-lang/rust#162832

[bjorn3]

The Windows x86_64 ABI only reserves rax or xmm0 as return register. LLVM happens to also allow rdx and xmm1, but Cranelift doesn't. https://github.com/rust-lang/rust/blob/420ed2a0c3d7225b1744266fd884d431b4d8cfe0/compiler/rustc_target/src/callconv/mod.rs#L796-L847 may need to be updated to also exclude return values larger than a single register on x86_64 Windows.

[folkertdev]

Will you be looking into that? I'm not sure what is reasonable to change in rustc versus cg_clif versus cranelift itself.

[RalfJung]

Are you suggesting we make the "Rust" ABI less performant on Windows, costing performance for everyone on stable, in order to fix a cranelift issue? That does not seem great.

[folkertdev]

I think this is as simple as bytecodealliance/wasmtime#14361, there is already an exception for GPRs specifically for extern "Rust" so this seems consistent.