Skip to content

Incorrect edit_functions values for some instances #24

Description

@jatinganhotra

Hi Loc-Bench creators,

We have our own implementation of extracting the modified nodes from a given gold patch and we were converting our format to map to Loc-Bench edit_functions format.

We have hit an issue for these 3 instances, where the edit_functions values are incorrect for the Loc-Bench czlll/Loc-Bench_V1 dataset on Huggingface. For these 3 instances, these files and functions/modules are missing from edit_functions

Mismatch instance_id: AzureAD__microsoft-authentication-library-for-python-454
  missing_from_ours (0): []
  extra_in_ours   (3): ['msal/application.py', 'msal/oauth2cli/assertion.py', 'msal/oauth2cli/oauth2.py']
================================================================================
Mismatch instance_id: NVIDIA__NeMo-9100
  missing_from_ours (0): []
  extra_in_ours   (1): ['nemo/collections/asr/parts/submodules/ctc_greedy_decoding.py']
================================================================================
Mismatch instance_id: matrix-org__synapse-8744
  missing_from_ours (0): []
  extra_in_ours   (1): ['synapse/handlers/receipts.py']

I noticed another open issue on GH here: #20 which points out the same issue.

Could you please take a look? Please let me know if you need additional information.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions