Could multitoken sets be matched using quantifier operators?
Good day team,
When matching multiple tokens using a Matcher, it seems advantageous to use quantifier operators, in case the multiple tokens are repeated a variable number of times.
A basic example would be a list of things, where the length of the list can vary. For example "Text with list: red cat, green bat, blue cow.", in this case, multiple tokens are repeated, an adjective, followed by a noun, followed by a punctuation mark, multiple times. Each set of multiple tokens could be matched using {'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}, but this only matches one single set, and not the full list.
Could these multiple token sets be matched using quantifier operators? Here is a possible way to write a pattern to match the list:
{'REPEATED_TOKENS': [{'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}], 'OP': '{2, 3}'}
This would match the repeated tokens from 2 to 3 times as given by 'OP'. Here is a complete code example:
import spacy
from spacy.matcher import Matcher
nlp = spacy.load('en_core_web_sm')
doc = nlp("Text with list: red cat, green bat, blue cow.")
matcher = Matcher(nlp.vocab)
matcher.add('rule1', [[{'REPEATED_TOKENS': [{'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}], 'OP': '{2, 3}'}]])
coincidences = matcher(doc)
This would match the full list "red cat, green bat, blue cow." in the coincidences.
Could this be possible in the future? I am aware of the workaround to achieve this using extensions, but ideally this could be done using native syntax to create patterns to match.
Thank you very much for your attention 🙏
Your Environment
- Operating System: macOS-26.6.2-arm64-arm-64bit
- Python Version Used: 3.12.12
- spaCy Version Used: 3.8.16
- Environment Information:
Could multitoken sets be matched using quantifier operators?
Good day team,
When matching multiple tokens using a Matcher, it seems advantageous to use quantifier operators, in case the multiple tokens are repeated a variable number of times.
A basic example would be a list of things, where the length of the list can vary. For example "Text with list: red cat, green bat, blue cow.", in this case, multiple tokens are repeated, an adjective, followed by a noun, followed by a punctuation mark, multiple times. Each set of multiple tokens could be matched using
{'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}, but this only matches one single set, and not the full list.Could these multiple token sets be matched using quantifier operators? Here is a possible way to write a pattern to match the list:
{'REPEATED_TOKENS': [{'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}], 'OP': '{2, 3}'}This would match the repeated tokens from 2 to 3 times as given by 'OP'. Here is a complete code example:
This would match the full list "red cat, green bat, blue cow." in the
coincidences.Could this be possible in the future? I am aware of the workaround to achieve this using extensions, but ideally this could be done using native syntax to create patterns to match.
Thank you very much for your attention 🙏
Your Environment