Skip to content

Multitoken sets cannot be matched using quantifier operators #14032

Description

@Polirecyliente

Could multitoken sets be matched using quantifier operators?

Good day team,

When matching multiple tokens using a Matcher, it seems advantageous to use quantifier operators, in case the multiple tokens are repeated a variable number of times.

A basic example would be a list of things, where the length of the list can vary. For example "Text with list: red cat, green bat, blue cow.", in this case, multiple tokens are repeated, an adjective, followed by a noun, followed by a punctuation mark, multiple times. Each set of multiple tokens could be matched using {'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}, but this only matches one single set, and not the full list.

Could these multiple token sets be matched using quantifier operators? Here is a possible way to write a pattern to match the list:

{'REPEATED_TOKENS': [{'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}], 'OP': '{2, 3}'}

This would match the repeated tokens from 2 to 3 times as given by 'OP'. Here is a complete code example:

import spacy
from spacy.matcher import Matcher
nlp = spacy.load('en_core_web_sm')
doc = nlp("Text with list: red cat, green bat, blue cow.")
matcher = Matcher(nlp.vocab)
matcher.add('rule1', [[{'REPEATED_TOKENS': [{'POS': 'ADJ'}, {'POS': 'NOUN'}, {'POS': 'PUNCT'}], 'OP': '{2, 3}'}]])
coincidences = matcher(doc)

This would match the full list "red cat, green bat, blue cow." in the coincidences.

Could this be possible in the future? I am aware of the workaround to achieve this using extensions, but ideally this could be done using native syntax to create patterns to match.

Thank you very much for your attention 🙏

Your Environment

  • Operating System: macOS-26.6.2-arm64-arm-64bit
  • Python Version Used: 3.12.12
  • spaCy Version Used: 3.8.16
  • Environment Information:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions