[ASDAlexander77]

Summary

A @cppvtable method returning a struct by value now also works under the Itanium C++ ABI (g++, clang) on x86-64 and AArch64, which covers Linux and Android. Until now it was an error on any non-MSVC target.

The Itanium ABI decides by the struct:

C++ return type	How it's returned	How to declare it
has a destructor or copy constructor, like nvrhi::RefCountPtr<T>	through a hidden pointer passed before this, as the call's sret argument (RDI on x86-64, X8 on AArch64)	@cppnontrivial createBuffer(desc: Ref<BufferDesc>): RefCountPtr;
trivial, one or two 8-byte integer or pointer fields, like nvrhi::Object	in RAX:RDX / X0:X1, the same way tslang returns that tuple	getNativeObject(type: u32): NativeObject;, with no mark
any other trivial struct up to 16 bytes	packed into registers by the SysV / AAPCS64 rules, which tslang doesn't do	compile error

A tuple can't express whether the C++ type has a destructor, so @cppnontrivial says so. MSVC ignores it, since MSVC always passes the pointer after this. Methods on any other target (for example 32-bit x86) are still an error. DeclarationPrinter keeps @cppnontrivial, so modules that import the class see it too.

Changes

• TargetInfo::hiddenReturnPointerBeforeThis: set for non-MSVC x86-64 and AArch64 targets.
• mlirGenClassCppVTableHiddenReturn: lowers @cppnontrivial methods to (this, Ref<T>, args) -> Ref<T>, the same as MSVC, and records that the pointer goes before this. Unmarked methods returning one or two 8-byte fields keep their declared type. It reports errors for other structs and for @cppnontrivial on a method that doesn't return a struct.
• Call site (mlirGenCall): calls the slot as (Ref<T>, this, args...) and tags the call with ts.sret_arg.
• Lowering: the tag is carried from CallIndirect to CallInternal or Invoke, and LowerToLLVM turns it into llvm.sret = T on the argument. That is what moves it to X8 on AArch64. MSVC's pointer after this stays an ordinary argument; Windows ARM64 passes it in X1.
• Narrow arguments (ts.cpp_call): every call of a @cppvtable method is tagged, and LowerToLLVM marks its arguments narrower than 32 bits zeroext (or signext if signed), booleans included. C++ callers extend them this way and clang-compiled callees rely on it; before this, tslang's calls left the register's upper bits undefined. MSVC callees don't rely on it, so this is harmless there, and a C++ bool can now be declared boolean rather than u8. Both tags reach the call through a new callAttrs parameter of mlirGenCallFunction.

Tests

• test-jit-cppvtable-abi (new): checks against code from a real C++ compiler, not a vtable built by hand in tslang. cppvtable/abi_host.cpp is built as a shared library by the host's C++ compiler (MSVC on Windows, g++ or clang on x86-64 or AArch64 Linux). The JIT then calls the object's methods through a @cppvtable class, abi_msvc.ts or abi_itanium.ts, which differ in their destructor slots. It covers a struct with a destructor, trivial 8- and 16-byte structs, a plain int, and narrow(bool, uint8_t, int8_t) called with values truncated from a wider one. If the slots are shifted by one, it fails.
• test-compile-cppvtable-itanium-call-<triple> (new): for x86-64 Linux and AArch64 Android, checks that the IR has zeroext/signext on the narrow arguments and sret on the hidden pointer. It runs on any host.
• Error tests:
  • cppvtable-itanium-record: an unmarked [s32, s32] on x86-64 Linux;
  • cppvtable-nontrivial-no-struct: @cppnontrivial on a method returning a number;
  • cppvtable-hidden-return-target: now uses i686 Linux.
• Full suite on Windows: 4006/4006 before the rebase onto main. After rebasing onto main (with @cppvtable: methods returning a record through MSVC's hidden pointer #551 and import: a module's file for the target, <name>.<os>.ts, before <name>.ts #552 squash-merged), the 322 @cppvtable, import/export, <os>.ts import and ownership tests pass.
• x86-64 Itanium, run for real: I compiled a probe for x86_64-unknown-linux-gnu, linked it with a g++-built C++ object in WSL Ubuntu, and ran it. All four kinds of method returned the right values, both before and after the sret change. cppvtable/abi_itanium.ts itself, linked with a g++-built abi_host.cpp, also prints done. there.
• AArch64 (not run, no arm64 runtime here): I compared the code tslang generates for aarch64-linux-android29 with NDK r30 clang's code for the same C++ calls. Both put the result pointer in X8 and this in X0, and use the same vtable offsets (Integer types #16/not implemented forEach #24/check if the following code is working #32). The 16-byte result comes back in X0:X1. Calls inside try (invoke) carry sret too.

Docs

docs/cppvtable-classes.md: the Itanium rules, @cppnontrivial, and how to get Itanium slot numbers with clang (-fdump-vtable-layouts; slot N is entry N + 2).

🤖 Generated with Claude Code