Skip to content

Document Kafka release pairings and actual validation coverage #289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jordepic merged 1 commit into from
Sep 29, 2026
Merged

Document Kafka release pairings and actual validation coverage #289

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/connectors/kafka.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,30 @@ The `streamfusion-kafka` connector extension and the matching `streamfusion-*` f
must both be installed. A missing extension is a planner fallback, never a linkage failure. See
[Deployment](../deployment.md).

## Flink release lines

Each build line pins the released Kafka connector and the client used with it:

| | Default build | `flink-1.18` build |
| --- | --- | --- |
| Flink | 2.2.1 | 1.18.1 |
| Kafka connector | `flink-connector-kafka:5.0.0-2.2` | `flink-connector-kafka:3.2.0-1.18` |
| Kafka client | 4.2.0 | 3.4.0 |

The direct Kafka client dependency is needed for source compilation. Keep it aligned with
its connector: overriding the transitive client also changes the broker-facing implementation.
StreamFusion retains Flink's connector for broker interactions, so its native format paths do
not supply connector or client fixes absent from the selected releases.

The `streamfusion-kafka` module tests exercise the 1.18 pairing under `-Pflink-1.18`.
The unchanged upstream Kafka 3.2 suite exercises `KafkaChangelogTableITCase`,
`KafkaTableITCase` and `UpsertKafkaTableITCase` against real brokers on that line.
`DynamicKafkaTableITCase` belongs to the newer connector suite. Upstream Kafka invocations do
not yet have individual native-route contracts: passing these integration tests establishes
compatibility, not native execution for every invocation. See the
[upstream suite's validation evidence](../upstream-flink-suite.md) for the distinction between
suite results and native coverage.

## Source: Flink consumption, native decode

Flink continues to own topic enumeration, split assignment, offsets, checkpoints, authentication,
Expand Down
3 changes: 3 additions & 0 deletions docs/flink-compatibility.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ Install one complete line; a mixed install is an error.
Both payload lines use the host's SLF4J 1.7 API and binding; it does not bundle Arrow's transitive
SLF4J 2 API into Flink's global classpath.

Each line also pins its Kafka connector and client. [Kafka](connectors/kafka.md#flink-release-lines)
records these pairings and the integration evidence available for each line.

The Kernel-based Delta implementation belongs only to the 2.2 source root and does not enter
the 1.18 compilation, Javadoc or source artifacts. There is no admitted native Delta connector on 1.18 yet. Do not build or install a 1.18 Delta payload;
this is unavailable functionality, not a verified host fallback.
Expand Down
Loading