Skip to content

feat(variant): allow creating VariantArray from Option items convertible to Variant - #11053

Open
jabrailkhalil wants to merge 1 commit into
apache:mainfrom
jabrailkhalil:variant-fromiter-into
Open

feat(variant): allow creating VariantArray from Option items convertible to Variant#11053
jabrailkhalil wants to merge 1 commit into
apache:mainfrom
jabrailkhalil:variant-fromiter-into

Conversation

@jabrailkhalil

Copy link
Copy Markdown

Rationale

Closes #8628.

Previously VariantArray could be collected only from Option<Variant> items:

let arr: VariantArray = values.iter().map(|v| Some(Variant::from(*v))).collect();

With this change, any type that converts cleanly into Variant can be collected directly:

let arr: VariantArray = vec![Some(1_i64), None, Some(42_i64)].into_iter().collect();

let arr: VariantArray = vec![Some("hello"), Some("a very long string exceeding the short string limit")].into_iter().collect();

This follows the suggestion made in #8625 (comment).

Changes

  • parquet-variant-compute/src/variant_array.rs: generalize FromIterator<Option<Variant<'m, 'v>>> to impl<'m, 'v, V: Into<Variant<'m, 'v>>> FromIterator<Option<V>>, converting each item through Into::into. Existing code that collects Option<Variant> items keeps working (identity Into).
  • Added regression tests: i64 items with nulls, &str items covering both short-string (ShortString) and long-string (String) storage.

Verification

  • cargo test -p parquet-variant-compute --lib — 366 tests passed (364 existing + 2 new), including the new test_from_option_into_variants_into_variant_array and test_from_option_str_into_variant_array.
  • cargo clippy -p parquet-variant-compute --all-targets -- -D warnings — clean.
  • cargo fmt --all -- --check — clean.

Note: local verification used the x86_64-pc-windows-gnu target; upstream CI covers the full matrix.

AI disclosure

Per the Arrow project's AI-generated submissions guidance: this change was implemented with the assistance of an AI coding agent. The author reviewed the final diff, verified it compiles and passes tests/clippy/fmt locally, and takes full ownership of the change.

The ASF ICLA may need to be on file before merge (asf-clabot will report the status).

…ble to Variant

Generalize FromIterator<Option<Variant>> for VariantArray to FromIterator<Option<V>> where V: Into<Variant>, so values that convert cleanly to Variant (e.g. i64, &str) can be collected directly. Adds regression tests for i64 and str items, including null rows and short vs long string storage.
@github-actions github-actions Bot added the parquet-variant parquet-variant* crates label Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet-variant parquet-variant* crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Variant] Improve ergonomic for creating VariantArray

1 participant