Enable using dash or underscore in CLI

[muellerzr]

What does this PR do?

This PR adds a pinch of magic for users that are used to doing --a-b with arguments that pass-through to things such as torchrun, accelerate can process those if we have them defined ourselves already.

E.g. a user can do:

accelerate launch --no_python --multi_gpu --num_processes 8 --monitor_interval=1 ./trampoline.sh python3 -c "print('hello')"

OR:

accelerate launch --no-python --multi-gpu --num-processes 8 --monitor-interval=1 ./trampoline.sh python3 -c "print('hello')"

Fixes #2241

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@BenjaminBossan, @stas00 (for your future sanity 😄 )

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[stas00]

Zach, in the OP, the 2 lines of code are identical - a copy-n-paste mistake?

[muellerzr]

OP was posted before I did this 😅 I'll go update it!

[muellerzr]

Actually, was weird phrasing on my end. Edited it to be more succinct :)

[stas00]

I meant this:

accelerate launch --no_python --multi_gpu --num_processes 8 --monitor_interval=1 ./trampoline.sh python3 -c "print('hello')"
accelerate launch --no-python --multi-gpu --num-processes 8 --monitor-interval=1 ./trampoline.sh python3 -c "print('hello')"

the 2 are identical. but it has OR in between.

[stas00]

oh, wait, they aren't identical ;) it's the multi_gpu vs multi-gpu

[BenjaminBossan]

I'm +/- in this PR. I understand the goal to enable both ways of writing this, but I'm uncertain if the extra complexity is worth it.

We override two private methods, which in turn use more private methods. Are we sure that they work in all edge cases with all Python versions? Is the new code 100% covered by tests? If there is an issue there, it has the potential to break our users' scripts.

I went looking for alternatives. argparse has the concept of custom Actions, which could be a solution, but the documentation wasn't very helpful IMO:

https://docs.python.org/3/library/argparse.html#action-classes

Also, at a different place, it says "New in version 3.9", so not sure if this works with 3.8.

Maybe another way would be to detect if there are missing arguments with _ in the name that, if replaced by -, would match an existing argument, and then add a useful warning, but leave the arguments untouched otherwise? This could be less disruptive.

Also note:

• click made the change to only use -
• typer automatically uses - instead of _

[BenjaminBossan]

Suggested change

	# If not found, but looks like a negative number, probably posisional
	# If not found, but looks like a negative number, probably positional

[BenjaminBossan]

Suggested change

	def test_hiphen(self):
	def test_hyphen(self):

[BenjaminBossan]

I think the doc would be clearer with a real example, for num-processes instead of a-b.

[akx]

Wouldn't it be easier to just override add_argument() add dash-y aliases for option strings that have underscores?

[muellerzr]

The aim is to not bloat --help, these are passthrough arguments to downstream API hook-ins such as torchrun which uses - rather than _.