Introduce module-kind convention plugins

[bric3]

What Does This Do

Introduces module-kind convention plugins for Java-backed Gradle modules and applies them from plugins {} while keeping the existing script plugins as the behavior source.

The basic change is usually removing the apply from of the java and publish script plugins, and use the modern recommended plugins block with the identified module kind :

- apply from: "$rootDir/gradle/java.gradle"
+ plugins {
+   id 'dd-trace-java.module.instrumentation'
+ }

In particular these are the identified module kind, in this PR, it's likely some kind are not represented in this PR, and that's not the goal to be exhaustive at this stage, it can be refined later (for example for the shared modules in the agent jar).

• dd-trace-java.module.product-subsystem

• dd-trace-java.module.annotation-processor

• dd-trace-java.module.bootstrap-component

  It is for code whose artifact is intentionally part of the agent's bootstrap surface. It is expected to be included at the root of the final agent jar, currently directly through shadowInclude.

  dd-trace-java/dd-java-agent/build.gradle

  Lines 395 to 402 in 0a0226c

  	// Includes for the top level shadow jar
  	shadowInclude project(path: ':components:environment', configuration: 'shadow')
  	shadowInclude project(path: ':dd-java-agent:agent-bootstrap')
  	shadowInclude project(path: ':dd-java-agent:agent-debugger:debugger-bootstrap')
  	shadowInclude project(path: ':dd-java-agent:agent-otel:otel-bootstrap', configuration: 'shadow')
  	// embed an extra copy of our otel-shim for drop-in/extension support
  	shadowInclude project(path: ':dd-java-agent:agent-otel:otel-shim')
  	shadowInclude project(path: ':products:feature-flagging:feature-flagging-bootstrap')

  The classes in these modules (and their dependencies) are visible through the JVM bootstrap classloader once the agent jar is appended to the bootstrap search path. This the strictest category because the code can run very early and can be referenced from instrumented application classes.

  Note, the dependencies of these modules do not use this convention.

  The bootstrap modules and their dependencies happens to be excluded from the shared/ prefix

  dd-trace-java/dd-java-agent/build.gradle

  Lines 267 to 282 in 0a0226c

  	it.dependencies {
  	exclude(project(':dd-java-agent:agent-bootstrap'))
  	exclude(project(':dd-java-agent:agent-logging'))
  	exclude(project(':dd-trace-api'))
  	exclude(project(':internal-api'))
  	exclude(project(':components:context'))
  	exclude(project(':utils:config-utils'))
  	exclude(project(':utils:logging-utils'))
  	exclude(project(':utils:time-utils'))
  	exclude(project(':products:metrics:metrics-api'))
  	exclude(project(':products:metrics:metrics-agent'))
  	exclude(dependency('org.slf4j::'))
  	// use dd-instrument-java's embedded copy of asm
  	exclude(dependency('org.ow2.asm:asm:'))
  	}
  	}

• dd-trace-java.module.distributable.api (here the dot is intended, as the idea is to have a distributable.agent)

  Applies the publish.gradle plugin script.

• dd-trace-java.module.instrumentation

  Instrumentation modules are named according to their minimal supported version, however for "common" instrumented parts, modules like <lib>-common exists, as they are instrumentation per se, they do not require a different more specialized convention. In other words they also have to apply dd-trace-java.module.instrumentation.

• dd-trace-java.module.internal-api

• dd-trace-java.module.product-library

• dd-trace-java.module.internal-library

  This is for internal implementation libraries. They are regular shared implementation code, generally intended to be consumed by other agent modules and often packaged inside shared/, trace/, or a product-specific directory in the final jar. Some of them can still land at the jar root (bootstrap) transitively when a bootstrap component depends on them, for example metrics-agent; that does not necessarily make them bootstrap components. It just means a bootstrap component needs them.

• dd-trace-java.module.platform-component

  It is for low-level platform building blocks shared by several parts of the agent: context, json, annotations, HTTP primitives, etc. These modules are not agent
  products and are not feature implementations. They are reusable infrastructure. They may end up bootstrap-visible when a bootstrap module depends on them, but that is a consequence of who consumes them.

• dd-trace-java.module.smoke-test

• dd-trace-java.module.testing-support

I chose module prefix as a way to convey the "higher level" module kind. While there might be other lower level technical conventions.

Small point of attention to the :dd-java-agent:instrumentation module, this PR replaces this check

if (subProj.path != ':dd-java-agent:instrumentation:vertx:vertx-redis-client:vertx-redis-client-stubs') {
  // don't include the redis RequestImpl stubs
  parent_project.dependencies {
    addProvider("implementation", providers.provider { project(subProj.path) })
  }
}

By checking if the sub project has the dd-trace-java.module.instrumentation convention, if it does it applies the necessary conventions for the instrumentation, so it just wraps existing code within the following closure:

subProj.pluginManager.withPlugin("dd-trace-java.module.instrumentation") {
  // ...
  parent_project.dependencies {
     addProvider("implementation", providers.provider { project(subProj.path) })
  }
}

Motivation

Prepare a gradual migration from lower-level script plugin application to higher-level conventions that describe what each module is.

Gradle references used for this migration direction:

• Sharing build logic using convention plugins
• Script plugins

Additional Details

The current forbidden-apis wiring for instrumentation modules is preserved: instrumentation tasks read both main.txt and instrumentation.txt.

:dd-java-agent:instrumentation itself is intentionally not migrated further here. Its aggregator-specific wiring, especially around muzzle reports, needs preparatory work before moving more of that project into convention-plugin logic.

jardiff was used to compare the agent jar from master and this branch. The agent jar contains both .class and .classdata entries, so the stricter check includes both and coalesces .classdata as class-like bytecode:

$ mise exec java@corretto-25.0.3.9.1 -- jardiff --stat -c classdata -i '**/*.class,**/*.classdata' /private/tmp/dd-trace-java-jardiff-master-640b1156e1/dd-java-agent/build/libs/dd-java-agent-1.64.0-SNAPSHOT.jar ./build/libs/dd-java-agent-1.64.0-SNAPSHOT.jar | tail -n 1

0 files changed, 0 insertions(+), 0 deletions(-)

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Add your completed PR to the merge queue by commenting /merge. You can also:
  • Customize the commit message associated with the merge with /merge --commit-message "..."
  • Remove your PR from the merge queue with /merge -c
  • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
  • Get more information in this doc

Jira ticket: [N/A]

[datadog-datadog-prod-us1]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 57.78% (-0.01%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c9629e4 | Docs | Datadog PR Page | Give us feedback!}

[mcculls]

I think moving to conventions is definitely the right direction - but the categories need some work, especially around internals. For example, when is something a component rather than a util? And should we be putting utils in the same convention as product libraries.

[bric3]

Thanks for the thoughtful comments

[jandro996]

Reviewed overall, and more specifically the files under my CODEOWNERS scope (asm-java: appsec/iast/aiguard build.gradle files and related instrumentation modules)

[mcculls]

dd-trace-java.module.agent-product.gradle.kts needs updating to dd-trace-java.module.product-subsystem.gradle.kts as discussed - other than that LGTM

( also this is not to be merged until after the 1.65.0 release :)

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.