Collect Gradle daemon diagnostics on failures

[daniel-mohedano]

What Does This Do

• Routes nested Gradle daemon crash reports into the collected build/reports directory.
• Preserves the newest daemon log when a TestKit build fails, including PID-named and UUID-named logs.
• Records the exception chain, daemon identifier, PID presence, and available /proc information.
• Captures cgroup v2 memory and PID counters at failure time.
• Keeps diagnostic collection best-effort so it does not replace the original test failure.

Additional context gathering, such as heap and thread dumps, or global debug logging, is deferred for a future PR if needed.

Motivation

We have seen some Gradle smoke test scenarios failing after the nested build reported BUILD SUCCESSFUL. TestKit then reported that the daemon had disappeared.

The existing artifacts contained the daemon log but did not show whether the daemon had crashed, been killed, or remained alive after a connection failure. The TestKit directory was also deleted during cleanup.

These diagnostics provide additional evidence to investigate these failure modes if the issue happens again.

Additional Notes

test-environment-trigger: skip

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1630

[datadog-datadog-us1-prod]

Bits found no code fix to apply

Warning

This comment is related to an earlier commit.

🟢 Investigated · ⚪ No code fix needed

build failed because verifyAgentJarIntegrations treated the CI-injected JAVA_TOOL_OPTIONS banner as unexpected stderr. The customer change only modifies Gradle smoke-test failure diagnostics; an existing upstream change already removes that environment variable for this verification task, so no scoped code change was made.

---

View in Datadog | Reviewed commit 27ec1ac · Any feedback? Reach out in #deveng-pr-agent

[daniel-mohedano]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 27ec1ac927

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[daniel-mohedano]

/datadog autotest review

[datadog-datadog-us1-prod]

More details

The added failure path preserves the original Gradle failure while best-effort diagnostics remain isolated behind their own exception handling. No diff-only behavioral regression was identified; the smoke test itself could not be executed because the repository requires Java 25 and this sandbox provides Java 21.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 3d7789b · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.94 s	13.87 s	[-0.1%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent	12.89 s	13.02 s	[-1.7%; -0.3%] (maybe better)
startup:petclinic:appsec:Agent	16.98 s	16.78 s	[+0.1%; +2.3%] (maybe worse)
startup:petclinic:iast:Agent	16.93 s	16.97 s	[-1.1%; +0.7%] (no difference)
startup:petclinic:profiling:Agent	16.94 s	16.96 s	[-1.3%; +1.1%] (no difference)
startup:petclinic:sca:Agent	16.89 s	16.77 s	[-0.2%; +1.7%] (no difference)
startup:petclinic:tracing:Agent	16.01 s	16.11 s	[-1.7%; +0.4%] (no difference)

Commit: 7196560a · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[datadog-datadog-us1-prod]

The new diagnostics directory is initialized during class loading from datadog.smoketest.builddir. Running the same Paths.get construction without that Gradle-injected property throws NullPointerException, so direct IDE/JUnit execution now fails before any test runs; the existing Gradle task supplies the property, but the new static initialization makes that invocation dependency unconditional.

_{📊 Validated against 5 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 3d7789b · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d7789baaf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[bric3]

LGTM

I wonder as an improvement if it's useful to capture / interpret cgroup v1 (in the eventuality smoke test are moved outside the microVM)

Alos, I believe we could improve this in a follow-up to capture cgroup-v2 snapshots before/after each nested build? In our microVM/Kubernetes CI, these counters cover the build container hierarchy and accumulate across many Gradle processes.

Capturing the delta in general, e.g. with memory.max and memory.high, would help found some evidence about what's happening.

I believe there's another thing to handle in this improvement, the oom_kill counter in the memory.events file is cumulative, so capturing its value before the nested build starts, could also help pin point the issue. Likewise with the max value (count of rejected process creation attempts) in pid.events file.