feat(java): open a dataset at a branch or tag

[zy-kkk]

Closes #9702

Adds ReadOptions.Builder.setRef(Ref) so Java callers can open a dataset directly at a branch, a version on a branch, or a tag, matching the Rust DatasetBuilder::with_branch / with_tag (and the tag form of Python's lance.dataset(version=...)). It reuses the Ref type Dataset.checkout(Ref) already takes.

Motivation: Java callers that read a table at a tag or a branch, such as query engines, currently open the latest version and check out the ref. With setRef they can open the ref directly.

• The JNI layer converts the Java Ref with the existing transform_jref_to_ref and forwards it to with_version / with_branch / with_tag.
• OpenDatasetBuilder carries the ref over when opening through a namespace client; it rebuilds ReadOptions field by field there, so without this the ref would be silently dropped.
• setRef is mutually exclusive with setVersion and with setSerializedManifest, and build() rejects both.

Rust fix included

DatasetBuilder::from_uri(root).with_branch(branch, Some(version)) failed when the root chain had no manifest for version, for example opening a branch at a version created after it forked while main is behind (Dataset at path .../_versions/3.manifest was not found), while checkout_version((branch, version)) worked. The builder defers the branch checkout but still loaded the uri chain at the requested version first, and the fallback in load_by_uri only triggers on Error::VersionNotFound, which the storage handler (default_resolve_version returns an unchecked V1 path) and the external manifest store do not return. When the checkout is deferred, the builder now loads the uri chain's latest version and leaves (branch, version) to the checkout, as a tag that points into a branch already does. Only with_branch(branch, Some(version)) without a namespace-managed store changes; its cost matches the other deferred opens (the uri chain's latest manifest is resolved instead of probing version). The new assertion in test_branch fails without the fix, and the test also pins the error for a version the branch does not have and an older version opened from the branch's own directory. This is in its own commit.

Worth a look

• Main references opened from a branch directory. Ref documents "main" as an alias for the default branch, and checkout follows that. The builder instead resolves a branch-less ref on the chain the uri points at, and with_branch("main", ..) standardizes to a branch-less ref. So with the uri at tree/dev/, setRef(Ref.ofMain(2)), Ref.ofBranch("main", 2) and Ref.ofBranch("main") read dev, while checkout with the same refs reads the default branch. with_version and Python's lance.dataset(uri, version=...) resolve on the uri's chain too. The Javadoc states this and asks callers to open the dataset root; happy to follow up in Rust if the builder should match checkout.
• The Java namespace open passes its own ExternalManifestCommitHandler, so the builder takes the unmanaged path and does not qualify the branch up front as DatasetBuilder::from_namespace does. Results are correct; moving Java onto the managed flow would be a separate change.
• The manifest/ref check lives in ReadOptions.build(); it can move into DatasetBuilder if preferred.

Validation

• cargo fmt --all -- --check, cargo fmt --manifest-path java/lance-jni/Cargo.toml --all -- --check, ./mvnw spotless:check
• cargo clippy --all --tests --benches -- -D warnings, cargo clippy --tests --manifest-path java/lance-jni/Cargo.toml -- -D warnings
• cargo test -p lance --lib dataset_versioning, cargo test -p lance-namespace-impls branch
• cd java && ./mvnw test (578 run, 0 failures; the skipped ones are the LANCE_INTEGRATION_TEST classes)

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[lance-gatekeeper]

⚠️ Gate recommendation: approve with a non-blocking risk.

The documented dataset-root requirement mitigates the difference from checkout for default-branch refs. Callers adopting setRef should retain the root URI.

[lance-gatekeeper]

Opening a default-branch Ref from a branch URI reads that branch's data, unlike Dataset.checkout(Ref). A caller replacing an open/checkout sequence with setRef can therefore select different data when it reuses a branch dataset's uri().

I verified the three default-branch forms below against diverged main/dev snapshots: opening from the branch URI returned dev's 5 rows, while checkout and opening from the root returned main's 8. The documented dataset-root requirement avoids the mismatch; no change is required for accepting this PR.

Reproducer

Add this as java/src/test/java/org/lance/Gate9703ReferenceContractTest.java and run it with ./mvnw test -Dtest=Gate9703ReferenceContractTest from java/. These assertions capture the observed behavior.

/*
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.lance;

import org.apache.arrow.memory.RootAllocator;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;

import java.nio.file.Path;
import java.util.Collections;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

class Gate9703ReferenceContractTest {
  @Test
  void defaultBranchReferenceUsesTheUriChain(@TempDir Path tempDir) {
    String root = tempDir.resolve("table").toString();
    try (RootAllocator allocator = new RootAllocator()) {
      TestUtils.SimpleTestDataset fixture = new TestUtils.SimpleTestDataset(allocator, root);
      try (Dataset initial = fixture.createEmptyDataset();
          Dataset main = fixture.write(1, 5);
          Dataset dev = main.createBranch("dev", Ref.ofMain(2))) {
        dev.updateConfig(Collections.singletonMap("branch", "dev"));
        try (Dataset main3 = fixture.write(2, 3)) {
          assertEquals(8, main3.countRows());
          assertEquals(5, dev.countRows());
          assertEquals(3, dev.version());
          Ref[] refs = {Ref.ofMain(3), Ref.ofBranch("main", 3), Ref.ofBranch("main")};
          for (Ref ref : refs) {
            ReadOptions options = new ReadOptions.Builder().setRef(ref).build();
            try (Dataset fromBranchUri = Dataset.open(allocator, dev.uri(), options);
                Dataset fromRootUri = Dataset.open(allocator, root, options);
                Dataset checkedOut = dev.checkout(ref)) {
              assertEquals(5, fromBranchUri.countRows(), ref.toString());
              assertTrue(fromBranchUri.uri().contains("tree/dev"));
              assertEquals(8, fromRootUri.countRows(), ref.toString());
              assertEquals(8, checkedOut.countRows(), ref.toString());
            }
          }
        }
      }
    }
  }
}

[yanghua]

LGTM