Extending Sentencizer to allow for "start of sentence chars" instead of just "punct chars"

[NixBiks]

Currently you can only add punct chars as input to the sentencizer.

It would be great to be able to also add characters that denotes the start of a sentence, e.g. a bullet point. I can make a PR if you think it's a good idea.

Let me know.

[adrianeboyd]

We appreciate the idea and can definitely see how it would be useful, but since it's relatively easy to write a custom rule-based component for sentence boundary detection, I think we'd prefer not to expand sentencizer with too many different possible settings.

[NixBiks]

I understand but adding another custom component for just that would require another iteration over all the tokens. Not very performant.

I agree it's good to keep the component really simple. However this feature doesn't really increase the complexity IMO (very minimal).

Are you certain you don't want to implement this?

[adrianeboyd]

To clarify a bit, I meant that it would be possible to write a custom component that handles both sentence-initial and sentence-final punctuation in the same component.

I will grant you that writing custom components in cython is a hassle and you would see speed differences in cython vs. python versions of the sentencizer (but I haven't done any detailed profiling recently).

[NixBiks]

Thanks for clarifying but yeah ;-)