fix: Backport FLINK-40093 patch so paused idle source splits resume

[velo]

Problem

FLINK-40093 is a race between watermark alignment and source idleness detection. When a split is paused by alignment, pauseOrResumeSplits also suspends its idleness timer — but with a low idle timeout and low allowed watermark drift, the split can be marked idle before that happens. From then on:

• alignment skips idle splits entirely, so the split is never resumed;
• because it stays paused at the connector level, no records flow, so the watermark generator never sees anything that would clear the idle flag.

The split has records and never emits them again.

The fix is released in Flink 2.2.2 / 2.3.1 / 2.4.0. We pin flink.version to 2.3.0 and 2.3.1 is not out yet.

Change

Vendor the two patched flink-runtime classes into flink-sql-runner, following the existing AvroDeserializationSchema precedent:

• org/apache/flink/streaming/api/operators/SourceOperator.java — the currentlyIdleSplits check moves inside the "watermark too far ahead" branch, so idle splits are still skipped when pausing (preserving their idle status) but can be resumed. They stay idle until they emit their next record.
• org/apache/flink/runtime/metrics/groups/InternalSourceSplitMetricGroup.java — the paused/idle race is benign now, so the WARN becomes INFO.

Both files are byte-identical to upstream commit 2a2d3590 on release-2.3 (the backport of apache/flink#28689). The 2.3.0 versions they replace are in turn byte-identical to that commit's parent, so there is no local merge to carry — the git blob hashes match upstream exactly, and spotless leaves them untouched (**/org/apache/flink/** is already routed to the AOSP + Flink-import-order config and excluded from the license plugin).

flink-runtime is added as a provided dependency to compile them. provided also wins nearest-definition over any transitive compile scope, so upstream flink-runtime classes are not bundled into the uber jar — verified that org/apache/flink/streaming/api/operators/ and org/apache/flink/runtime/metrics/groups/ contain only our classes.

Why this takes effect

Flink's constructFlinkClassPath forces flink-dist*.jar to the end of the classpath; everything else in /opt/flink/lib comes first. sql-runner.uber.jar is already there, so these classes shadow the unpatched originals. Confirmed on the built image:

$ java -cp <lib classpath> Probe
org.apache.flink.streaming.api.operators.SourceOperator <- file:/opt/flink/lib/sql-runner.uber.jar
org.apache.flink.runtime.metrics.groups.InternalSourceSplitMetricGroup <- file:/opt/flink/lib/sql-runner.uber.jar

Guard test

SourceOperatorPatchTest asserts two things:

1. both patched classes resolve to the same code source as our own CliRunner, i.e. shadowing is intact — this is what breaks if the dependency scope or shade config regresses;
2. EnvironmentInformation.getVersion() is still 2.3.0, so the build fails on the version bump with instructions to delete the patch.

Behavioural correctness of the race fix rides on upstream's SourceOperatorSplitWatermarkAlignmentTest#testPausedIdleSplitsCanBeResumedByAlignmentCheck, which is not vendored here.

Removal

On the bump to Flink 2.3.1+: delete the two vendored sources, their package-info.java files, SourceOperatorPatchTest, and the flink-runtime provided dependency. The version assertion fails the build on that bump, so it cannot be silently forgotten.

Verification

• mvn test — full reactor green.
• mvn install -Pfast — uber jar and image build; patched classes present, no upstream collisions in those packages.
• Classpath ordering and actual loaded-class origin confirmed inside the built image (above).

Not covered: an end-to-end reproduction of the original hang against the real workload.