Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug in _get_max_mapped_special_id #133

Open
floccinauc opened this issue Jul 24, 2024 · 0 comments
Open

Bug in _get_max_mapped_special_id #133

floccinauc opened this issue Jul 24, 2024 · 0 comments
Assignees

Comments

@floccinauc
Copy link
Collaborator

The function _get_max_mapped_special_id is just a copy of _get_max_mapped_id, so it doesn't do what it needs to. Instead, line https://github.com/BiomedSciAI/fuse-drug/blob/818cb8c3bb79065ce6539b9523cb6a1e594091cf/fusedrug/data/tokenizer/modulartokenizer/modular_tokenizer.py#L1389C13-L1389C56 implements a similar concept with max(special_vocab.values()).

Suggestion: Replace line 1389 with a call to _get_max_mapped_special_id and use max(special_vocab.values()) to implement _get_max_mapped_special_id.

@floccinauc floccinauc self-assigned this Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant