From 4f37014a6555fc2501ad6f98c5d0df905852529d Mon Sep 17 00:00:00 2001 From: root Date: Tue, 29 Sep 2026 08:06:53 -0400 Subject: [PATCH] Document Kafka release pairings with accurate validation scope List the connector and client pinned by each Flink line, and distinguish upstream compatibility tests from per-invocation native coverage. The older Kafka suite contains changelog, table and upsert tests; dynamic-table tests belong to the newer release. --- docs/connectors/kafka.md | 24 ++++++++++++++++++++++++ docs/flink-compatibility.md | 3 +++ 2 files changed, 27 insertions(+) diff --git a/docs/connectors/kafka.md b/docs/connectors/kafka.md index 415b8520..b7a41abc 100644 --- a/docs/connectors/kafka.md +++ b/docs/connectors/kafka.md @@ -13,6 +13,30 @@ The `streamfusion-kafka` connector extension and the matching `streamfusion-*` f must both be installed. A missing extension is a planner fallback, never a linkage failure. See [Deployment](../deployment.md). +## Flink release lines + +Each build line pins the released Kafka connector and the client used with it: + +| | Default build | `flink-1.18` build | +| --- | --- | --- | +| Flink | 2.2.1 | 1.18.1 | +| Kafka connector | `flink-connector-kafka:5.0.0-2.2` | `flink-connector-kafka:3.2.0-1.18` | +| Kafka client | 4.2.0 | 3.4.0 | + +The direct Kafka client dependency is needed for source compilation. Keep it aligned with +its connector: overriding the transitive client also changes the broker-facing implementation. +StreamFusion retains Flink's connector for broker interactions, so its native format paths do +not supply connector or client fixes absent from the selected releases. + +The `streamfusion-kafka` module tests exercise the 1.18 pairing under `-Pflink-1.18`. +The unchanged upstream Kafka 3.2 suite exercises `KafkaChangelogTableITCase`, +`KafkaTableITCase` and `UpsertKafkaTableITCase` against real brokers on that line. +`DynamicKafkaTableITCase` belongs to the newer connector suite. Upstream Kafka invocations do +not yet have individual native-route contracts: passing these integration tests establishes +compatibility, not native execution for every invocation. See the +[upstream suite's validation evidence](../upstream-flink-suite.md) for the distinction between +suite results and native coverage. + ## Source: Flink consumption, native decode Flink continues to own topic enumeration, split assignment, offsets, checkpoints, authentication, diff --git a/docs/flink-compatibility.md b/docs/flink-compatibility.md index 11190f37..effbb8d9 100644 --- a/docs/flink-compatibility.md +++ b/docs/flink-compatibility.md @@ -46,6 +46,9 @@ Install one complete line; a mixed install is an error. Both payload lines use the host's SLF4J 1.7 API and binding; it does not bundle Arrow's transitive SLF4J 2 API into Flink's global classpath. +Each line also pins its Kafka connector and client. [Kafka](connectors/kafka.md#flink-release-lines) +records these pairings and the integration evidence available for each line. + The Kernel-based Delta implementation belongs only to the 2.2 source root and does not enter the 1.18 compilation, Javadoc or source artifacts. There is no admitted native Delta connector on 1.18 yet. Do not build or install a 1.18 Delta payload; this is unavailable functionality, not a verified host fallback.