@cppvtable: classes implementing C++ methods (C++ calls them through their vtable)

docs/cppvtable-classes.md

@@ -40,7 +40,8 @@ fields (`.rtti`, `.size`). A `@cppvtable` class has none of these. Its methods a
 slots from 0, in declaration order, with a base class's slots first.
 
 The class gets no vtable, constructor, RTTI or allocation of its own. It is a declaration: its
-methods have no bodies, and an `abstract` class with `abstract` methods reads the same.
+methods have no bodies, and an `abstract` class with `abstract` methods reads the same. (A class
+whose methods have bodies is the other way round: see "Implementing C++ methods".)
 
 Fields declared on the class are the C++ object's fields after the vtable pointer, laid out like
 a tuple's.
@@ -53,7 +54,7 @@ these are errors and not warnings.
 
 | Not allowed | Why |
 | --- | --- |
-| a method with a body, a constructor, a field initializer | the objects and their code are C++'s |
+| a constructor or a field initializer, in a class none of whose methods has a body | the objects and their code are C++'s (a method with a body makes it a class implementing C++ methods, see below) |
 | static members, `get`/`set` accessors, `accessor` fields | they would take vtable slots, and accessor slots don't follow declaration order |
 | `implements` | an implemented interface takes a slot |
 | `extends` an ordinary class, or an ordinary class extending a `@cppvtable` one | the ordinary class's slots (`.instanceOf`, `.rtti`, ...) would shift the numbering |
@@ -122,8 +123,57 @@ A generic or static class can't be `@cppvtable` either.
 - **Casting.** `<NvDevice>opaqueValue` is a plain pointer cast with no check: the object must
   really be one.
 
+## Implementing C++ methods
+
+A `@cppvtable` class whose methods have bodies implements C++ methods: C++ calls them through the
+object's vtable. An example is a callback interface such as `nvrhi::IMessageCallback`. Declare the C++ interface
+as above, and extend it:
+
+```ts
+@cppvtable
+class IMessageCallback {               // nvrhi::IMessageCallback (MSVC)
+    _destructor(): void;               // slot 0
+    message(severity: u8, text: string): void;
+}
+
+@cppvtable
+class MessageLog extends IMessageCallback {
+    errors = 0;
+
+    message(severity: u8, text: string): void {
+        if (severity >= 2) {
+            this.errors++;
+        }
+    }
+}
+
+const log = new MessageLog();          // pass <Opaque>log to C++
+```
+
+- **The vtable** is the C++ layout and nothing else: the slots in order, with no `.instanceOf`,
+  interface or static slots. `new` points the object's first word at it. A method with a body
+  overrides the base's slot of the same name. A slot left to a method of the C++ interface (a
+  method with no body, such as the destructor) is null, as a pure virtual's would be: C++ calling
+  it crashes. A method that matches no base slot appends a slot, as in C++.
+- **The object** is allocated by tslang: the vtable pointer, the C++ interface's fields, then the
+  class's own fields, laid out like a tuple's. It may have a constructor and field initializers.
+- **Calls from C++** come in as C++ calls them: `this` first, then the arguments. A narrow result
+  (`boolean`, `u8`, ...) is extended to 32 bits, as C++ callers expect. Declare by-value struct
+  parameters the way the ABI passes them (see "Writing a binding").
+- **Lifetime.** C++ keeps a raw pointer that the collector can't see. Hold the object in tslang,
+  for example in a global, for as long as C++ may call it. A `string` argument is C++'s buffer:
+  copy it to keep it past the call.
+- **Not yet supported:**
+  - returning a struct (an error);
+  - exporting the class (an error: an importer would get the methods but not the vtable);
+  - type information: C++ must not `dynamic_cast` or `typeid` the object;
+  - throwing from a method C++ calls.
+
 ## Tests
 
 - `tslang/test/tester/tests/00cppvtable_class.ts` checks the slots, fields, `extends` and
   `abstract` against a vtable made by hand. It is registered for AOT and JIT.
 - `tslang/test/tester/cppvtable/*.ts` holds one compile error per rule above.
+- `test-jit-cppvtable-abi` checks against code from a real C++ compiler. `cppvtable/abi_host.cpp` is built as a shared library by the host's compiler (MSVC, or g++/clang). `abi_msvc.ts` or `abi_itanium.ts` then:
+  - calls its object's methods, including struct returns and narrow arguments;
+  - implements its `Callback` interface (`Counter`), which C++ calls back.

tslang/include/TypeScript/Defines.h

@@ -204,6 +204,9 @@
 // On a call: it calls a C++ method (of a @cppvtable class), whose arguments narrower than 32 bits
 // LowerToLLVM marks zeroext / signext, as a C++ caller extends them (clang relies on it)
 #define CPP_CALL_ATTR_NAME "ts.cpp_call"
+// On a function: a C++ method implemented in tslang (a method with a body of a @cppvtable class),
+// called by C++, whose results narrower than 32 bits LowerToLLVM extends as C++ callers expect
+#define CPP_METHOD_ATTR_NAME "ts.cpp_method"
 
 #if __LP64__
 #define TRAMPOLINE_SIZE 48

tslang/include/TypeScript/MLIRLogic/MLIRGenStore.h

@@ -790,6 +790,11 @@ struct ClassInfo
     // vtable's slots from 0 in declaration order, a base class's first: no `.instanceOf`, `.new`,
     // interface or static-field slots, no RTTI, and no vtable or constructor of its own.
     bool isCppVTable;
+    // a @cppvtable class with methods that have bodies: C++ methods implemented here, so C++ can
+    // call them through its objects. It has a vtable (the C++ one's layout: the slots and nothing
+    // else, a slot left to a declared C++ method null), may have a constructor and field
+    // initializers, and `.new`; still no RTTI. tslang allocates its objects.
+    bool isCppVTableImplementation = false;
     ProcessingStages processingAtEvaluation;
     ProcessingStages processing;
     // the module `processing` was reached in: a discovery module is thrown away with what it holds

tslang/lib/TypeScript/LowerToLLVM.cpp

@@ -1928,7 +1928,10 @@ struct FuncOpLowering : public TsLlvmPattern<mlir_ts::FuncOp>
         // relies on it (x86-64 SysV). A declaration - implemented elsewhere, usually in C - says
         // so. A definition does not: a `signext` parameter makes the callee trust its caller, and
         // tslang's indirect calls carry no call-site attributes.
-        if (funcOp.getBody().empty())
+        // A C++ method implemented here (CPP_METHOD_ATTR_NAME) is called by C++ code, which takes
+        // a narrow result as extended by the callee: so it extends its results.
+        auto isDeclaration = funcOp.getBody().empty();
+        if (isDeclaration)
         {
             for (auto [index, type] : llvm::enumerate(fnType.getInputs()))
             {
@@ -1937,7 +1940,10 @@ struct FuncOpLowering : public TsLlvmPattern<mlir_ts::FuncOp>
                     newFuncOp.setArgAttr(index, extension, rewriter.getUnitAttr());
                 }
             }
+        }
 
+        if (isDeclaration || funcOp->hasAttr(CPP_METHOD_ATTR_NAME))
+        {
             for (auto [index, type] : llvm::enumerate(fnType.getResults()))
             {
                 if (auto extension = getIntegerExtensionAttrName(type); !extension.empty())
@@ -1949,7 +1955,7 @@ struct FuncOpLowering : public TsLlvmPattern<mlir_ts::FuncOp>
 
         for (const auto &namedAttr : funcOp->getAttrs())
         {
-            if (namedAttr.getName() == funcOp.getFunctionTypeAttrName())
+            if (namedAttr.getName() == funcOp.getFunctionTypeAttrName() || namedAttr.getName() == CPP_METHOD_ATTR_NAME)
             {
                 continue;
             }

tslang/lib/TypeScript/MLIRGenClasses.cpp

@@ -206,16 +206,18 @@ namespace mlirgen
         llvm::SmallVector<VirtualMethodOrInterfaceVTableInfo> virtualTable;
         newClassPtr->getVirtualTable(virtualTable);
 
-        // a @cppvtable class is never constructed, typed or allocated here: its objects come from C++
-        auto hasOwnRuntimeMembers = !newClassPtr->isStatic && !newClassPtr->isCppVTable;
+        // a @cppvtable class is never constructed, typed or allocated here: its objects come from C++;
+        // unless it implements C++ methods, which it does in an object of its own
+        auto hasOwnRuntimeMembers = !newClassPtr->isStatic && (!newClassPtr->isCppVTable || newClassPtr->isCppVTableImplementation);
 
         if (hasOwnRuntimeMembers)
         {
             mlirGenClassDefaultConstructor(classDeclarationAST, newClassPtr, classGenContext);
         }
 
 #ifdef ENABLE_RTTI
-        if (hasOwnRuntimeMembers)
+        // no `.instanceOf`: a C++ vtable has only the C++ methods
+        if (hasOwnRuntimeMembers && !newClassPtr->isCppVTable)
         {
             // INFO: .instanceOf must be first element in VTable for Cast Any
             mlirGenClassInstanceOfMethod(classDeclarationAST, newClassPtr, classGenContext);
@@ -353,12 +355,20 @@ namespace mlirgen
                 }
 
                 // a binding to C++ objects: declared (its methods have no bodies), with a vtable
-                // field even with no `abstract` method (see ClassInfo::isCppVTable)
+                // field even with no `abstract` method (see ClassInfo::isCppVTable); or, with
+                // methods that have bodies, C++ methods implemented here
+                // (ClassInfo::isCppVTableImplementation)
                 if (name == CPP_VTABLE)
                 {
                     newClassPtr->isCppVTable = true;
-                    newClassPtr->isDeclaration = true;
                     newClassPtr->hasVirtualTable = true;
+                    newClassPtr->isCppVTableImplementation = llvm::any_of(classDeclarationAST->members, [](ClassElement member) {
+                        return member == SyntaxKind::MethodDeclaration && member.as<MethodDeclaration>()->body;
+                    });
+                    if (!newClassPtr->isCppVTableImplementation)
+                    {
+                        newClassPtr->isDeclaration = true;
+                    }
                 }
             });
 
@@ -384,9 +394,14 @@ namespace mlirgen
     // record of one or two 8-byte integer or pointer fields comes back in RAX:RDX / X0:X1, as tslang
     // returns such a tuple, so the method keeps its declared type; other trivial records are packed
     // into registers or returned in memory, which isn't done here.
+    //
+    // A method with a body (of a class implementing C++ methods) is called by C++: it is marked
+    // CPP_METHOD_ATTR_NAME, and can't return a record yet (its body would have to write it through
+    // the hidden pointer).
     mlir::LogicalResult MLIRGenImpl::mlirGenClassCppVTableHiddenReturn(mlir::Location location, ClassInfo::TypePtr newClassPtr,
-                                                          StringRef funcName, bool nonTrivial)
+                                                          mlir_ts::FuncOp funcOp, bool nonTrivial, bool hasBody)
     {
+        auto funcName = funcOp.getName();
         if (!newClassPtr->isCppVTable)
         {
             if (nonTrivial)
@@ -399,6 +414,10 @@ namespace mlirgen
         }
 
         cppVTableMethods.insert(funcName);
+        if (hasBody)
+        {
+            funcOp->setAttr(CPP_METHOD_ATTR_NAME, builder.getUnitAttr());
+        }
 
         auto method = llvm::find_if(newClassPtr->methods, [&](auto &method) { return method.funcName == funcName; });
         if (method == newClassPtr->methods.end() || method->declaredFuncType)
@@ -418,6 +437,12 @@ namespace mlirgen
             return mlir::success();
         }
 
+        if (hasBody)
+        {
+            emitError(location) << "a method implementing a C++ method (a @" CPP_VTABLE " class's, with a body) can't return a struct yet";
+            return mlir::failure();
+        }
+
         auto resultType = funcType.getResult(0);
         auto &targetInfo = compileOptions.targetInfo;
         auto beforeThis = false;
@@ -564,6 +589,15 @@ namespace mlirgen
             }
         }
 
+        // implementing C++ methods (isCppVTableImplementation): bodies, a constructor and field
+        // initializers are its own; it is not exported, as an importer would get its declaration
+        // only, with the methods' bodies but not the vtable they are in
+        auto implementation = newClassPtr->isCppVTableImplementation;
+        if (implementation && getExportModifier(classDeclarationAST))
+        {
+            error(location) << "a @" CPP_VTABLE " class implementing C++ methods can't be exported";
+        }
+
         for (auto &classMember : classDeclarationAST->members)
         {
             auto memberLocation = loc(classMember);
@@ -573,21 +607,17 @@ namespace mlirgen
                 continue;
             }
 
+            // a method is a slot; with a body, it implements a C++ method (isCppVTableImplementation)
             if (classMember == SyntaxKind::MethodDeclaration)
             {
-                if (classMember.as<MethodDeclaration>()->body)
-                {
-                    error(memberLocation) << "a method of a @" CPP_VTABLE " class is a C++ vtable slot: it can't have a body";
-                }
-
                 continue;
             }
 
             if (classMember == SyntaxKind::PropertyDeclaration)
             {
                 // a field of the C++ object, after its vtable pointer
                 auto propertyDeclaration = classMember.as<PropertyDeclaration>();
-                if (propertyDeclaration->initializer)
+                if (propertyDeclaration->initializer && !implementation)
                 {
                     error(memberLocation) << "a field of a @" CPP_VTABLE " class can't have an initializer";
                 }
@@ -602,12 +632,17 @@ namespace mlirgen
 
             if (classMember == SyntaxKind::Constructor)
             {
-                error(memberLocation) << "a @" CPP_VTABLE " class can't have a constructor: its objects are made by C++";
+                if (!implementation)
+                {
+                    error(memberLocation) << "a @" CPP_VTABLE " class can't have a constructor: its objects are made by C++, "
+                                          << "unless its methods have bodies";
+                }
+
                 continue;
             }
 
             // accessors are generated methods: their slots wouldn't follow the declaration order
-            error(memberLocation) << "a @" CPP_VTABLE " class can only have methods without bodies and fields";
+            error(memberLocation) << "a @" CPP_VTABLE " class can only have methods and fields";
         }
 
         return failed ? mlir::failure() : mlir::success();
@@ -703,7 +738,7 @@ namespace mlirgen
         }
 #endif
 
-        if (!newClassPtr->isStatic && !newClassPtr->isCppVTable)
+        if (!newClassPtr->isStatic && (!newClassPtr->isCppVTable || newClassPtr->isCppVTableImplementation))
         {
             mlirGenClassSizeStaticField(location, classDeclarationAST, newClassPtr, genContext);
         }
@@ -2351,7 +2386,50 @@ genContext);
                             slotType = mlir_ts::RefType::get(vtRecord.staticFieldInfo.type);
                         }
 
-                        if (isOwnedByDynamicImport(symbolName, vtRecord.isStaticField))
+                        // a class implementing C++ methods: a slot it leaves to a method of the
+                        // C++ class it extends, which has no body, is null, as a pure virtual's
+                        auto isUnimplementedCppMethod = [&]() {
+                            if (!newClassPtr->isCppVTable || vtRecord.isStaticField)
+                            {
+                                return false;
+                            }
+
+                            std::function<ClassInfo::TypePtr(ClassInfo::TypePtr)> findOwner =
+                                [&](ClassInfo::TypePtr cls) -> ClassInfo::TypePtr {
+                                if (llvm::any_of(cls->methods, [&](auto &m) { return m.funcName == symbolName; }))
+                                {
+                                    return cls;
+                                }
+
+                                for (auto &base : cls->baseClasses)
+                                {
+                                    if (auto owner = findOwner(base))
+                                    {
+                                        return owner;
+                                    }
+                                }
+
+                                return ClassInfo::TypePtr();
+                            };
+
+                            auto owner = findOwner(newClassPtr);
+                            return owner && owner->isCppVTable && !owner->isCppVTableImplementation;
+                        };
+
+                        if (isUnimplementedCppMethod())
+                        {
+                            // through Opaque: a null function is no TypeScript value
+                            auto nullValue = builder.create<mlir_ts::NullOp>(location, getNullType());
+                            auto nullOpaque = builder.create<mlir_ts::CastOp>(location, getOpaqueType(), nullValue);
+                            auto castResult = cast(location, slotType, nullOpaque, genContext);
+                            if (castResult.failed_or_no_value())
+                            {
+                                return TypeValueInitType{mlir::Type(), mlir::Value(), TypeProvided::No};
+                            }
+
+                            methodOrFieldNameRef = V(castResult);
+                        }
+                        else if (isOwnedByDynamicImport(symbolName, vtRecord.isStaticField))
                         {
                             auto dllSymbolName = !vtRecord.isStaticField && !vtRecord.methodInfo.dllName.empty()
                                 ? StringRef(vtRecord.methodInfo.dllName)
@@ -2530,8 +2608,11 @@ genContext);
             classMethodMemberInfo.setFuncOp(funcOp);
             if (classMethodMemberInfo.registerClassMethodMember(loc(funcLikeDeclaration), orderWeight, accessLevel))
             {
-                if (mlir::failed(mlirGenClassCppVTableHiddenReturn(loc(funcLikeDeclaration), newClassPtr, funcOp.getName(),
-                                                                   MLIRHelper::hasDecorator(funcLikeDeclaration, CPP_NON_TRIVIAL))))
+                // C++ methods: not a constructor, nor `.new` (static)
+                if (!classMethodMemberInfo.isConstructor && !classMethodMemberInfo.isStatic
+                    && mlir::failed(mlirGenClassCppVTableHiddenReturn(loc(funcLikeDeclaration), newClassPtr, funcOp,
+                                                                      MLIRHelper::hasDecorator(funcLikeDeclaration, CPP_NON_TRIVIAL),
+                                                                      static_cast<bool>(funcLikeDeclaration->body))))
                 {
                     return mlir::failure();
                 }

tslang/lib/TypeScript/MLIRGenImpl.h

@@ -11247,7 +11247,7 @@ class MLIRGenImpl
     }
 
     mlir::LogicalResult mlirGenClassCppVTableHiddenReturn(mlir::Location location, ClassInfo::TypePtr newClassPtr,
-                                                          StringRef funcName, bool nonTrivial);
+                                                          mlir_ts::FuncOp funcOp, bool nonTrivial, bool hasBody);
 
     bool isCppRecordReturnedInIntegerRegisters(mlir_ts::TupleType recordType);