Support for constituency parsing

[elyase]

It would be great if spacy offered some sort of constituency parsing information. I think the API can look similar to the one used for NER.

[honnibal]

This is part of the current road-map, using something like this approach:

https://www.cs.cmu.edu/~nasmith/papers/kong+rush+smith.naacl15.pdf

In the meantime I'd like to provide just base noun phrases and prepositional phrases using heuristics. This might be ready within the next month or so.

[aeneaswiener]

Has there been any progress on noun phrase extraction mentioned above?

[geovedi]

I've seen some progress on NP extraction. Pretty smart, I'd say :-)
https://github.com/honnibal/spaCy/blob/gaz/spacy/tokens/doc.pyx#L185

[elyase]

is constituency parsing still on the roadmap?

[ohenepee]

It's 2017, is constituency parsing still in the works, outta the works, or almost done?

[f11r]

@honnibal Would be great to know if/where this is on the current roadmap.

[ines]

Quick update: This might be a nice use case for the new custom processing pipeline components and extension attributes introduced in v2.0!

[maxhawkins]

The author of the "Transforming Dependencies" paper has an open source implementation: https://github.com/ikekonglp/PAD

[iloveluce]

You might also want to take a look at AllenNLP, they're open source, built on top of SpaCy and just released Constituency Parsing

[nikitakit]

For anyone interested in English constituency parsing I now have a release version out for the paper I'll be presenting at ACL this year ("Constituency Parsing with a Self-Attentive Encoder"). The package ships with a pre-trained English model (95 F1 on the Penn Treebank WSJ test set) and spaCy integration via extension attributes.

[afparsons]

Edit: no longer true

---

FYI to anyone who comes across this: the Berkeley Neural Parser works quite well (thank you, @nikitakit!) but seems to no longer be maintained. For example, users now have to fork or monkey-patch the package in order to get Tensorflow to cooperate.

[nikitakit]

It's correct to say that I haven't updated benepar in a long time, but I actually just wrapped up and published a major new version of the parser. It also has updates to ensure that it still works with spaCy 3.0

[afparsons]

Congrats! Thank you for your hard work.

[ines]

@nikitakit Ah, this is cool and a great use of the extension attributes. Looking forward to playing with it! Would you mind if we added the project to the spaCy Universe?

[nikitakit]

@ines Sure, happy to have a mention included in the spaCy Universe!

[honnibal]

@nikitakit Wow, great paper! I need to try your attention component in the parser and NER! Seems simple enough...

I'm intrigued that the results keep going up even at a window of 20. Not by much, but it's still helping. It surprises me -- I would've thought the long-range effects would be very weak.

There are a few other things that are interesting here. Thanks for carefully exploring this question of the effects of factoring the different sources of information. Together with Dozat and Manning's results, I think we can say we now have two systems indicating this can be important. I wonder whether it's just a dataset size issue (it might be too easy to overfit on these small parsing problems), or whether we'll find more problems where controlling the flow of information in the network proves useful. If so we'll end up back towards the factor graphs I think, and back towards something that looks like feature engineering.

I haven't read all of the paper yet, and I haven't gone through the attention mechanism carefully. So perhaps this doesn't make sense, but: if you restrict the window of the attention component, and then stack those layers, wouldn't you get an increasing "receptive field", just like in CNNs? So, imagine you encode with a window size of 5. Then, you encode with another window size of 5. Aren't you drawing information from up to 10 words away?

[nikitakit]

@honnibal Glad you liked the paper! There's definitely a lot more to explore in terms of trying self-attention for different tasks, and also figuring out if factoring information helps across tasks or could be generalized in some way.

For the attention windowing, you're absolutely right that stacking two layers with window size 5 gives an effective receptive field of size 10. That's why it's surprising that 8 layers, with window size 10 each, still don't quite match the accuracy of the un-windowed model. I don't have any convincing explanation for why this is the case.

[thesofakillers]

Hi Guys, besides the great plugin provided by Nikita (thank you so much btw), are there any plans to bring constituency parsing into the core API still? Or have we settled on using the plugin? Thanks!

[randomgambit]

interested in this as well, thanks!

[nikitakit]

@honnibal @ines I see that there are new Span/SpanGroup APIs under development for spaCy 3 (and maybe even released, as of a few days ago). Would you say that these are appropriate for use with constituency parsing? I haven't familiarized myself with these APIs, so I'm not sure if there may be an gotchas that when using these for deeply nested tree structures, or if using these for constituency parsing will somehow conflict with other use cases like NER.

[afparsons]

I'm not familiar with the publication history behind constituency parsing (although names like Michael Collins and Fei Xia seem to appear often), so please forgive any naivety.

Can we use the DependencyMatcher to form a constituency tree? Or is there some flaw in relying on a rules-based conversion?

[adrianeboyd]

I think you could potentially develop good rule-based conversions for individual treebanks, but I don't think the dependency matcher on its own is going to be the right tool to do this kind of tree transformation. In a quick search, these papers looked relevant and fairly recent in the scheme of things, but I haven't done a thorough search or review: https://aclanthology.org/N15-1080, https://aclanthology.org/C16-1041

The spacy Doc object has been focused on dependency trees and doesn't really include a good way to store a constituency tree, so you'd probably want to look at other libraries or tools to work on this.