Inserting a new line in a large script might be slow

[icersong]

vim 8 insert new line, python filesize 1K

FUNCTION  <SNR>138_find_opening_paren()
    Defined: ~/.cache/vimfiles/plugins/vim-python-pep8-indent/indent/python.vim line 96
Called 4 times
Total time:   2.822975
 Self time:   0.728050

count  total (s)   self (s)
                                " optional arguments: line and column (defaults to 1) to search around
    4              0.000005     if a:0 > 0
    2              0.000009         let view = winsaveview()
    2              0.000009         call cursor(a:1, a:0 > 1 ? a:2 : 1)
    2              0.000007         let ret = s:find_opening_paren()
    2              0.000071         call winrestview(view)
    2              0.000002         return ret
    2              0.000001     endif
                            
                                " Return if cursor is in a comment.
    2              0.000036     exe 'if' s:skip_search '| return [0, 0] | endif'
                            
    2              0.000004     let nearest = [0, 0]
    8              0.000028     for [p, maxoff] in items(s:paren_pairs)
    6              0.000046         let stopline = max([0, line('.') - maxoff, nearest[0]])
    6   1.411188   0.727807         let next = searchpairpos( '\V'.p[0], '', '\V'.p[1], 'bnW', s:skip_special_chars, stopline)
    6              0.000036         if next[0] && (next[0] > nearest[0] || (next[0] == nearest[0] && next[1] > nearest[1]))
                                        let nearest = next
    6              0.000007         endif
    8              0.000008     endfor
    2              0.000003     return nearest

[toejough]

mine is even slower, and I've got a decent macbook:

FUNCTION  <SNR>102_find_opening_paren()
    Defined: ~/.vim/plugged/vim-python-pep8-indent/indent/python.vim line 96
Called 44 times
Total time:  22.053020
 Self time:   5.826575

count  total (s)   self (s)
                                " optional arguments: line and column (defaults to 1) to search around
   44              0.000057     if a:0 > 0
   22              0.000093         let view = winsaveview()
   22              0.000077         call cursor(a:1, a:0 > 1 ? a:2 : 1)
   22              0.000091         let ret = s:find_opening_paren()
   22              0.001036         call winrestview(view)
   22              0.000024         return ret
   22              0.000010     endif
                            
                                " Return if cursor is in a comment.
   22              0.016860     exe 'if' s:skip_search '| return [0, 0] | endif'
                            
   22              0.000053     let nearest = [0, 0]
   88              0.000312     for [p, maxoff] in items(s:paren_pairs)
   66              0.000462         let stopline = max([0, line('.') - maxoff, nearest[0]])
   66  11.006864   5.807683         let next = searchpairpos( '\V'.p[0], '', '\V'.p[1], 'bnW', s:skip_special_chars, stopline)
   66              0.000322         if next[0] && (next[0] > nearest[0] || (next[0] == nearest[0] && next[1] > nearest[1]))
                                        let nearest = next
   66              0.000078         endif
   88              0.000092     endfor
   22              0.000030     return nearest

[blueyed]

Thanks for the report/info in general.

I've looked into optimizing this a while back, but it is not trivial.

You can try changing

vim-python-pep8-indent/indent/python.vim

Line 51 in 3e61be5

let s:paren_pairs = {'()': 50, '[]': 100, '{}': 1000}

, which defines how far it looks for matching/opening parenthesis.

There are many tests at least, so you would be covered when trying to optimize it yourself.
Unfortunately the logic / code is also not easy to follow (e.g. with :debug echo GetPythonPEPIndent(line(".")+1), but you might want to try it anyway).

[blueyed]

Also an example script would be useful in general (anonymized if needed, or just something that triggers this) - IIRC it might be especially slow with unmatched opening {.

I guess that it is (especially) slow for your with unclosed opening parenthesis as well?

[blueyed]

One idea might be to use a timeout with searchpairpos, but that should be done with vim/vim@9d32276 (8.0.1483) only.

Related issue for Vim's own python indent script: vim/vim#1098 (comment)

[blueyed]

@toejough
It appears to be similar slow for you btw, just that you have used it more (called 44 times instead of 4).

[blueyed]

To mitigate this you might want to try #118 (or just add the timeout yourself to the call).

I've found some older stashes in this regard, where I've experimented e.g. with doing a quick round first (only looking up e.g. 5 lines), before looking potentially 1000 lines up for {}.

Also caching might help in general.

Anyway, it appears to be related to your specific use case (the lines before o) - I usually have no problem with this.

See #99 (comment) for discussion about using the Python AST actually.

[blueyed]

Came across some bummer in Vim itself: it appears to evaluate the skip expression always.. :/
(note that this could maybe get optimized anyway, and likely should be handled manually for lower Vims anyway given this issue?!) (vim/vim#3613)

[toejough]

wow, thank you for the prompt and informative responses!! 🥇

The updates in 118 (at least as of the last passing check I saw - efa7e6b) cut the delay from this plugin in half. That's some significant improvement - thank you!

Happy to try out other updates if you'd like but what's in 118 is already significantly better (at least for me!)

[blueyed]

Great to hear, pushed some more.
You can also try the red ones - tests are passing but the patch is not fully covered (i.e. the conceiled part).

[blueyed]

Merged #118.

Please open new issues with remaining things (and examples) for improvements.