add tool to "enforce" semantic line breaks

[tshepang]

See #1132

Run it so:

cd ci/sembr
cargo run -- ../../src [--overwrite]

Diff is large, so maybe should be automated.

[JohnTitor]

I don't have an objection to enforcing sembr but the implementation does look fragile, e.g. there's no tokenizer. Have you considered using an existing tool like https://github.com/bobheadxi/readable?

[tshepang]

I have not... thanks for the suggestion, will have a look.

[m-spitfire]

I have tried using readable to have a GitHub action that runs check, and format all the existing code, but the tool is immature, it gave errors for some files saying Cannot take length of multi-line node '[object Object]' (bobheadxi/readable#18) and for others the aforementioned issue is there, it does the wrong thing when it encounters e.g.(bobheadxi/readable#17). I am not sure what should be done here, make readable better or create another tool?

[tshepang]

This tool of mine handles both issues without choking. I intend to get back to making it more fancy... I keep postponing.

[Kobzol]

Sorry, due to me messing up a git operation, we sadly had to force-push the whole commit history of rustc-dev-guide :( If you'd like to update this pull request, you will have to rebase it in a special way onto the new commit history (the new master):

git fetch origin --all
git checkout <pr-branch>
git rebase --onto origin/master origin/master-old
git push --force-with-lease

More context can be found here.

[rustbot]

Thanks for the PR. If you have write access, feel free to merge this PR if it does not need reviews. You can request a review using r? rustc-dev-guide or r? <username>.

[tshepang]

I don't know how to do tokenization, but I attached a sample chapter to show output from the tool

[rustbot]

Thanks for the PR. If you have write access, feel free to merge this PR if it does not need reviews. You can request a review using r? rustc-dev-guide or r? <username>.