TextBuffer makes vim word motion painfully slow in large file

[jedwards1211]

See t9md/atom-vim-mode-plus#1113.

Basically, it's TextBuffer's fault. TextBuffer.findAllInRangeSync is going super slow at the beginning of the file.
Considering this and issues with find in replace in large files being really slow, it's obvious that TextBuffer needs some serious improvement.

---

Edit by @rsese to copy the detailed issue description from t9md/atom-vim-mode-plus#1113

Execute Vim Mode Plus: Clip Debug Info(which write info to clipboard) then paste here.

debug info
Read and check all "Checklist" below.

Checklist

You have to check all before open issue.

• Provide your environment info clipped by Vim Mode Plus: Clip Debug Info command.
• Try with latest Atom and latest vim-mode-plus.
• Pick a descriptive and non-ambiguous subject
• Express "what" you want(feature? config option?, behavior change?) in short sentence(not long!).
• Contrast current behavior if you want to change current behavior, with sample text, operation(keystroke) and result.
• Include real use case so that maintainer can understand "why" you need help.
• Include Atom(atom --version), vim-mode-plus version, and OS version(e.g. macOS Sierra 10.12.3).
• If keybinding issue, Read this.

Here is an imitation of the data file I'm working with (60k lines):
sample-cave-data.txt

Moving forward by a word (w command) at the beginning of the file takes approx 240 ms per jump. Near the end of the file it only takes 1 ms per jump (see profiler output below).

240 ms is painfully slow. I'd like w to take 1 ms everywhere within a file of any size.

Slowness comes from TextBuffer.findAllInRangeSync. (TextBuffer also makes search and replace performance utterly pathetic in large files, so I don't blame vim-mode-plus for this really, thanks for all of your hard work!)

Versions

OS: macOS Mojave 10.14.2
Atom : 1.34.0
Electron: 2.0.16
Chrome : 61.0.3163.100
Node : 8.9.3
vim-mode-plus: 1.36.0

Saved profiler output

vim-mode-plus-slow-word-motion.zip

Profiler screenshot -- start of file

Profiler screenshot -- end of file

[rsese]

Thanks @jedwards1211 - @t9md if you find something specific in your investigation that you think the team should take a look at and it's not something you can resolve in vim-mode-plus, please let us know.

[t9md]

Investigated quickly.
vim-mode-plus depends on editor.scanInBufferRange and editor.backwardsScanInBufferRange heavily.
These function call TextBuffer's scanInRange internally.
The problem is although scanInBufferRange accept iterator which stops scanning at the given condition it collects all matching entries unnecessarily by calling findAllInRangeSync.

https://github.com/atom/text-buffer/blob/master/src/text-buffer.js#L1536

Maybe I believe the original assumption is findAllInRangeSync is fast enough for us to ignore this inefficiency.
But here, when scanRange become very big like 60K lines of buffer.
This inefficiency becomes significant.
I want text-buffer provide checking iterator line by line instead of preliminarily collect all entries then checking iterator.

What I can come up with to overcome current limitation is to implement own version of editor.scanInRange , but I don't think it is an ideal solution.