Skip to content

预处理代码存在一处不严谨 #17

Description

@Paulisca8

感谢您的优秀项目!

我在进行数据预处理时发现了一个小bug,在代码中

test_item_names = [x for x in item_names if any([ts in x for ts in hparams['test_prefixes']])]
valid_item_names = [x for x in item_names if any([ts in x for ts in hparams['valid_prefixes']])]

建议改为

test_item_names = [x for x in item_names if any(x.startswith(ts) for ts in hparams['test_prefixes'])]
valid_item_names = [x for x in item_names if any(x.startswith(ts) for ts in hparams['valid_prefixes'])]

😂这个"in"让我的数据分布发生了改变,因为我的prefix在部分文件名的后续字段中也有出现😭😭😭

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions