Skip to content

Migrate Expression and ExpressionExecutor in ClusterStatsGenerator #7579

Description

@Xuanwo

Discussed in #7570

Originally posted by Xuanwo September 11, 2022
Hi, I'm working on removing old planners. More specifically, I'm try to removing sqlparser.

However, there are some code in storage still depends on sqlparser's Expression:

#[derive(Clone, Default)]
pub struct ClusterStatsGenerator {
    cluster_key_id: u32,
    cluster_key_index: Vec<usize>,
    expression_executor: Option<ExpressionExecutor>,
    level: i32,
    block_compactor: BlockCompactor,
}

// This can be used in deletion, for an existing block.
pub fn gen_with_origin_stats(
    &self,
    data_block: &DataBlock,
    origin_stats: Option<ClusterStatistics>,
) -> Result<Option<ClusterStatistics>> {
    if origin_stats.is_none() {
        return Ok(None);
    }

    let origin_stats = origin_stats.unwrap();
    if origin_stats.cluster_key_id != self.cluster_key_id {
        return Ok(None);
    }

    let block = if let Some(executor) = &self.expression_executor {
        // For a clustered table, data_block has been sorted, but may not contain cluster key.
        // So only need to get the first and the last row for execute.
        let indices = vec![0u32, data_block.num_rows() as u32 - 1];
        let input = DataBlock::block_take_by_indices(data_block, &indices)?;
        executor.execute(&input)?
    } else {
        data_block.clone()
    };

    self.clusters_statistics(&block, origin_stats.level)
}

The cluster_key here is table's column name parsed by:

async fn analyze_cluster_keys(
    &mut self,
    cluster_by: &[Expr<'a>],
    schema: DataSchemaRef,
) -> Result<Vec<String>> { .. }

ClusterStatsGenerator will use expression_executor to remove unused columns.


How can I remove the Expression here?

Activity

  1. Xuanwo commented on Sep 13, 2022

    @Xuanwo
    CollaboratorAuthor
  2. self-assigned this
    on Sep 13, 2022
  3. Xuanwo commented on Sep 14, 2022

    @Xuanwo
    CollaboratorAuthor

    To address #7593, I plan to:

  4. assigned and unassigned on Sep 14, 2022
  5. moved this to 📋 Backlog in Xuanwo's Workon Sep 15, 2022
  6. Xuanwo commented on Sep 22, 2022

    @Xuanwo
    CollaboratorAuthor

    Current progress: #7806

  7. Xuanwo commented on Nov 14, 2022

    @Xuanwo
    CollaboratorAuthor

    Already implemented

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions