feat(namespace): add in-memory directory manifest cache mode

[ChilePiquin]

Directory namespace manifest reads currently re-check the manifest dataset and run filtered scans for repeated lookup/list operations. That is the right default for long-lived readers, but it is expensive for callers that intentionally operate on a fixed namespace snapshot and issue many reads over the same manifest. This PR adds an opt-in manifest_cache_mode=in_memory setting for directory namespaces. In this mode, a namespace instance loads the __manifest rows once and serves supported manifest reads from that pinned in-memory snapshot; a new namespace instance observes newer commits. Because cached snapshots can become stale, in_memory is a read-only mode: namespace/table mutations are rejected before they can validate write preconditions against old manifest state or mutate a table resolved from a stale location.

This was accomplished with the following changes:

• DirectoryNamespaceBuilder now carries manifest_cache_mode, parses it from properties, and exposes a manifest_cache_mode(...) builder method.
• ManifestCacheMode in dir/manifest.rs defines the supported values, none and in_memory, with validation for invalid property values.
• DatasetConsistencyWrapper now stores ManifestDatasetState, including the manifest Dataset, optional ManifestSnapshot, and cache mode.
• ManifestSnapshot loads the manifest columns needed for namespace reads into sorted in-memory rows for binary lookup and direct-child iteration.
• ManifestNamespace uses the snapshot fast path for manifest containment checks, table/namespace lookup, root table locations, table listing, namespace listing, and namespace-empty checks.
• DirectoryNamespace rejects in_memory writes before initializing the write-manifest cell, preserving the pinned read snapshot even after failed writes.
• Manifest rewrites, staged manifest writes, table schema mutations, data writes, version/index/tag/branch writes, restore, and transaction sidecar updates now reject in_memory namespace instances before mutating state.
• Manifest writes still clear/update the cached snapshot through set_latest, preserving the existing writer behavior for uncached namespace instances that mutate the manifest.

Testing

Added coverage for parsing manifest_cache_mode, rejecting invalid values, preserving the default none behavior, verifying that an in_memory namespace instance pins its manifest snapshot, and confirming cached namespace instances cannot orphan children, mutate data after deregistration, change their pinned view after a failed write, or perform writes.

Verified with:

• cargo test -p lance-namespace-impls manifest_cache_mode --no-default-features --quiet
• cargo test -p lance-namespace-impls in_memory_manifest_cache --no-default-features --quiet
• cargo check -p lance-namespace-impls --no-default-features --quiet

[lance-gatekeeper]

✅ Gate recommendation: approve.

The opt-in snapshot mode now has a consistent read-only boundary. Catalog writes fail before the write-manifest cell is initialized, and direct table mutations fail before using cached locations. The new regressions cover both previously reported failures.

Please mark this PR with the breaking-change label.