Removing url matching from tokenizer

[chelle-rr]

Hi!

I'm trying to use spaCy to work with a set of filepaths and names, so I've needed to set some specific rules for tokenization. However, I'm getting some unexpected results, and via nlp.tokenizer.explain I can see that the issue is that it's occasionally matching parts of the filename as a URL. Is there a way I can disable this?

nlp.tokenizer.explain("(!)Object Files_/pull out acc#s_Site Visit Report_APS_2012.pdf")

Output:

[('TOKEN', '(!)Object'), ('TOKEN', 'Files'), ('INFIX', '_'), ('INFIX', '/'), ('TOKEN', 'pull'), ('TOKEN', 'out'), ('TOKEN', 'acc#s'), ('INFIX', '_'), ('TOKEN', 'Site'), ('TOKEN', 'Visit'), ('URL_MATCH', 'Report_APS_2012.pdf')]

Desired output:

[('TOKEN', '(!)Object'), ('TOKEN', 'Files'), ('INFIX', '_'), ('INFIX', '/'), ('TOKEN', 'pull'), ('TOKEN', 'out'), ('TOKEN', 'acc#s'), ('INFIX', '_'), ('TOKEN', 'Site'), ('TOKEN', 'Visit'), ('TOKEN', 'Report'), ('INFIX', '_'), ('TOKEN', 'APS'), ('INFIX', '_'), ('TOKEN', '2012'), ('INFIX', '.'), ('TOKEN', 'pdf')]

I'm very new to this so please excuse me if I'm missing something obvious. Thank you!

[chelle-rr]

If anyone needs it later, here's the answer:

nlp.tokenizer.url_match = None