Allow specific exception types to bypass the dead letter queue

[nabisobhi]

Problem

When a dead letter queue is configured, every exception from a message handler is dead-lettered once retries are exhausted. That's the wrong behavior for fatal or systemic failures.

If a shared dependency goes down (database unreachable, bad configuration, a bug that fails on every message), every message fails, every message gets dead-lettered, and offsets keep committing. The result is that an entire topic quietly drains into the dead letter queue in seconds, and the natural backpressure you'd get from a failing consumer is lost.

Change

Adds an opt-in bypass so selected exception types skip the dead letter queue entirely. A matching exception is not retried and not dead-lettered — it propagates out of message dispatch, and Dafda does not commit the offset for the message.

services.AddConsumer(options =>
{
    options.WithDeadLetterQueue("orders.dead-letter")
           .WithMaxRetries(3)
           .BypassFor<MyFatalException>()         // by exception type (incl. derived)
           .BypassWhen(ex => ex is DbException);  // or by predicate
});

Multiple bypass registrations are OR'd together. When none are registered, behavior is unchanged.

What happens after the bypass

The exception escapes Consumer.ConsumeAll and is handled by ConsumerHostedService, which routes it through the configured IConsumerErrorHandler:

• Default handler — ConsumerFailureStrategy.Default stops the application, and the orchestrator restarts it. This is the intended failure mode.
• RestartConsumer — the consumer is restarted and the redelivered message fails again, in a tight loop. If you configure a restart strategy via WithConsumerErrorHandler, it needs to back off.

The bypass deliberately does not override the configured error handler, since that's an explicit user choice.

Commit semantics

Redelivery of a bypassed message requires manual commits (enable.auto.commit=false). Dafda only commits the offset itself in that mode. With automatic commits — the default — the Kafka client stores offsets as messages are consumed and commits them on its interval and on close, so a bypassed message may still be marked as consumed.

This is a property of Dafda's existing auto-commit mode rather than something the bypass introduces: on master today, an exception from a handler in a consumer with no dead letter queue configured already propagates past the explicit commit in the same way. Closing that gap properly (e.g. enable.auto.offset.store=false plus an explicit store after successful handling) is worth its own change and is not attempted here. The behavior is documented on the public API instead.

Implementation

The bypass is evaluated in the existing exception filter in Consumer.Dispatch, so a matching exception never enters the retry/dead-letter path at all:

catch (Exception exception) when (deadLetterQueueEnabled
    && !cancellationToken.IsCancellationRequested
    && !ShouldBypassDeadLetterQueue(exception))

The predicate is threaded from DeadLetterQueueOptions through ConsumerConfigurationBuilder → ConsumerConfiguration → Consumer, following the same path as the existing MaxRetries setting. Both AddConsumer overloads are wired. The registered predicates are snapshotted when the combined delegate is composed, so configuration is fixed at build time.

API surface

Additive only. Two new public methods on DeadLetterQueueOptions (itself new in the unreleased 2.1.0-beta1); everything else touched is internal. No new dependencies, no default behavior change. Minor version bump.

Prior art

This mirrors how other Kafka frameworks separate poison messages from systemic failures — Spring Kafka's DefaultErrorHandler with addNotRetryableExceptions(...) plus ContainerStoppingErrorHandler, and Kafka Connect's errors.tolerance = none. Fail fast on fatal errors; dead-letter only genuine poison messages.

Tests

Coverage for a bypassed exception propagating without being retried, dead-lettered, or having its offset committed by Dafda; for non-matching exceptions still being dead-lettered as before; and for predicate composition and snapshotting. Full suite: 160 passing, build clean with 0 warnings.

Notes for reviewers

This is the first of three related PRs. The other two stack on this one and should be reviewed after it:

• Log an error when an exception bypasses the dead letter queue #354 bypass logging — logs when an exception bypasses the DLQ
• Add configurable retry backoff before dead-lettering #356 retry backoff — adds a delay between retries, which is currently a tight loop

Follow-ups worth discussing separately, none of them changing default behavior in this PR: at-least-once commit semantics under auto-commit (above), a "stop this consumer only" option like Spring's ContainerStoppingErrorHandler, and a hard stop that a RestartConsumer strategy can't override.

[Copilot]

Pull request overview

This PR adds an opt-in “dead letter queue bypass” so selected exception types (or predicates) skip the retry/dead-letter path and instead propagate out of message dispatch, enabling fail-fast behavior for systemic/fatal failures.

Changes:

• Added a DLQ bypass hook to Consumer.Dispatch so matching exceptions do not retry and do not dead-letter.
• Introduced fluent configuration (BypassFor<TException>(), BypassWhen(...)) on DeadLetterQueueOptions and threaded the resulting predicate through configuration → DI wiring → Consumer.
• Added/updated tests and test builders to cover bypass vs non-bypass behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/Dafda/Consuming/Consumer.cs	Adds bypass predicate plumbing and uses it in the DLQ exception filter.
src/Dafda/Configuration/DeadLetterQueueOptions.cs	Adds fluent API to register bypass exception types/predicates and composes them into a single predicate.
src/Dafda/Configuration/ConsumerServiceCollectionExtensions.cs	Wires the bypass predicate into Consumer construction for both AddConsumer overloads.
src/Dafda/Configuration/ConsumerConfigurationBuilder.cs	Extracts bypass predicate from DLQ options into built configuration.
src/Dafda/Configuration/ConsumerConfiguration.cs	Stores the bypass predicate on the built configuration object.
src/Dafda.Tests/Consuming/TestConsumer.cs	Adds tests for bypassed exceptions propagating and non-matching exceptions still dead-lettering.
src/Dafda.Tests/Builders/ConsumerBuilder.cs	Adds builder support for passing a DLQ bypass predicate into Consumer.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

[joseban-iii]

Looks Legit! 👍