improve section detection

[keewis]

Potentially fixes #316.

This tries to allow parsing sections which are not separated by blank lines (there should probably be a warning in that case, I'll add that once the general idea has been approved). In order to get that to work used a few tricks (e.g. add a optional doc parameter to _is_at_section to allow calling it on a different reader) so it might need some refactoring before being truly ready.

cc @Carreau, my main motivation was trying to get velin to auto-fix this

[keewis]

yielding StopIteration seems like a bug:

numpydoc/numpydoc/docscrape.py

Lines 214 to 219 in 265ab91

	if name.startswith('..'): # index section
	yield name, data[1:]
	elif len(data) < 2:
	yield StopIteration
	else:
	yield name, self._strip(data[2:])

because it will cause this to fail with a obscure error:

numpydoc/numpydoc/docscrape.py

Lines 380 to 381 in 265ab91

	sections = list(self._read_sections())
	section_names = set([section for section, content in sections])

I think this should be fixed, either by changing the yield value to yield StopIteration, StopIteration or yield "", [], or by raising an error instead (if that's what that was supposed to indicate).

[keewis]

Ping. Does anyone have any comments on this?

[rgommers]

Ping. Does anyone have any comments on this?

I think the discussion in gh-316 shows this is not desirable. Detecting in order to raise a better warning or even an exception makes sense though.

[keewis]

apologies for pinging and then forgetting about it for a year.

I implemented the requested change, such that it will now warn on every missing empty line (between summary and the first section, or between two sections).

The current implementation comes with a (slight?) performance regression because NumpyDocString._is_at_section is now called for every line, and I had to do a slightly ugly trick to make sure _is_at_section doesn't swallow empty lines.

Instead, I could also imagine doing a two-pass implementation: find all separators (multiple - or = per line) and check if those belong to a section which is not preceded by an empty line in the first pass, then run the actual extraction in the second pass. That might make the code a bit simpler and potentially faster, but every line would be visited twice (I'm not sure how much of an issue that would be).

Edit: I think the CI failures are unrelated (not sure, though)