Skip to content

feat(tracing): add COMPOSE_OTEL_DEBUG to surface OTel internals - #14152

Open
htoyoda18 wants to merge 1 commit into
docker:mainfrom
htoyoda18:feat/otel-debug-env-var
Open

feat(tracing): add COMPOSE_OTEL_DEBUG to surface OTel internals#14152
htoyoda18 wants to merge 1 commit into
docker:mainfrom
htoyoda18:feat/otel-debug-env-var

Conversation

@htoyoda18

@htoyoda18 htoyoda18 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

What I did

OTel SDK/exporter internal errors (otel.ErrorHandler) and the tracer shutdown/flush error in wrapRunE were both always discarded, making it impossible to diagnose tracing issues (e.g. a misconfigured or unreachable OTLP endpoint) without instrumenting the code.

Added tracing.DebugEnabled(), gated on a new COMPOSE_OTEL_DEBUG environment variable, and used it in both places:

  • internal/tracing's otel.ErrorHandler now prints to stderr when enabled instead of always being a no-op
  • the previously-discarded tracingShutdown(ctx) error in cmd/cmdtrace/cmd_span.go is now printed to the command's stderr (dockerCli.Err()) when enabled

Default behavior (COMPOSE_OTEL_DEBUG unset) is unchanged — tracing plumbing still never leaks into ordinary CLI output.

Related issue
N/A

(not mandatory) A picture of a cute animal, if possible in relation to what you did
image

Signed-off-by: hiroto.toyoda <hiroto.toyoda@dena.com>
@htoyoda18
htoyoda18 requested review from a team as code owners August 28, 2026 19:23
@htoyoda18
htoyoda18 requested review from glours and ndeloof August 28, 2026 19:23

@glours glours left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, the diagnosability goal makes sense, but we don't think a new environment variable is needed here: Compose already has a debug mechanism with --debug that surfaces this exact kind of tracing diagnostic elsewhere in the codebase.

Could you route the OTel error printing through that instead of introducing COMPOSE_OTEL_DEBUG? Should end up simpler, with a single debug switch instead of two overlapping ones.

Comment thread cmd/cmdtrace/cmd_span.go

cmd.SetContext(ctx)
wrapRunE(cmd, cmdSpan, tracingShutdown)
wrapRunE(cmd, cmdSpan, tracingShutdown, dockerCli.Err())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No needed , we should use logrus to write the error which is already configured to write in the right IO

Suggested change
wrapRunE(cmd, cmdSpan, tracingShutdown, dockerCli.Err())
wrapRunE(cmd, cmdSpan, tracingShutdown)

Comment thread cmd/cmdtrace/cmd_span.go
// Unfortunately, PersistentPostRun(E) can't be used for this purpose because it
// only runs if RunE does _not_ return an error, but this should run unconditionally.
func wrapRunE(c *cobra.Command, cmdSpan trace.Span, tracingShutdown tracing.ShutdownFunc) {
func wrapRunE(c *cobra.Command, cmdSpan trace.Span, tracingShutdown tracing.ShutdownFunc, errOut io.Writer) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func wrapRunE(c *cobra.Command, cmdSpan trace.Span, tracingShutdown tracing.ShutdownFunc, errOut io.Writer) {
func wrapRunE(c *cobra.Command, cmdSpan trace.Span, tracingShutdown tracing.ShutdownFunc) {

Comment thread cmd/cmdtrace/cmd_span.go
// OTel components for debugging purposes
_ = tracingShutdown(ctx)
if err := tracingShutdown(ctx); err != nil && tracing.DebugEnabled() {
fmt.Fprintln(errOut, "otel: shutdown:", err)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use logrus

Suggested change
fmt.Fprintln(errOut, "otel: shutdown:", err)
logrus.Debugf("otel: shutdown: %v", err)

Comment on lines +27 to +34
// DebugEnabled reports whether OTel SDK/exporter internals should print
// their diagnostics, controlled by the COMPOSE_OTEL_DEBUG environment
// variable. It defaults to false so tracing plumbing never leaks into
// ordinary CLI output.
func DebugEnabled() bool {
enabled, _ := strconv.ParseBool(os.Getenv("COMPOSE_OTEL_DEBUG"))
return enabled
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for a dedicated env var; logrus.Debugf is already a no-op unless --debug is set, so the check is redundant.

Comment on lines +42 to +44
if DebugEnabled() {
fmt.Fprintln(os.Stderr, "otel:", err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reuses the same debug mechanism cmd/main.go:55 already uses for tracing setup failures, instead of a second, parallel gate for the same diagnostic category.

Suggested change
if DebugEnabled() {
fmt.Fprintln(os.Stderr, "otel:", err)
}
logrus.Debugf("otel: %v", err)

detect.ServiceName = "compose"
// do not log tracing errors to stdio
otel.SetErrorHandler(skipErrors{})
// do not log tracing errors to stdio, unless COMPOSE_OTEL_DEBUG is set

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// do not log tracing errors to stdio, unless COMPOSE_OTEL_DEBUG is set
// do not log tracing errors to stdio, unless `--debug` is set

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants