Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix system command processing during logging in user tools #633

Merged
merged 3 commits into from
Oct 27, 2023

Conversation

parthosa
Copy link
Collaborator

Fixes #632. The cmd argument in run_sys_cmd() in user tools can be of either list or string type. The inner method process_credentials_option() expects cmd to be of list type and processes each item. However, if cmd is of string type, then process_credentials_option() will process each character instead.

Changes:

  1. Check the type of the cmd argument and split it into a list if it is of string type.
  2. Add type constraints for the cmd argument in run_sys_cmd().

Testing:

The cmd described in the issue has been tested.

Cmd: spark_rapids_user_tools dataproc diagnostic psarthi-dataproc-test -v

Previous output:

2023-10-24 16:19:46,498 DEBUG rapids.tools.cmd_driver: executing CMD:
 <CMD: g c l o u d   c o m p u t e   s s h   p s a r t h i - d a t a p r o c - t e s t - m   - - z o n e   u s - c e n t r a l 1 - a   - - c o m m a n d = " P R E F I X = d i a g _ 2 0 2 3 1 0 2 4 2 3 1 9 0 2 _ E b 9 E E 5 f d   P L A T F O R M _ T Y P E = d a t a p r o c   / t m p / c o l l e c t . s h "   - - f o r m a t   j s o n>[
        <STDOUT>
        | /usr/lib/spark/bin/pyspark
        | Archive '/tmp/diag_20231024231902_Eb9EE5fd_info.tgz' is successfully created!
        | Archive '/tmp/diag_20231024231902_Eb9EE5fd_log.tgz' is successfully created!
        | []]; [
        <STDERR>

Current output:

 <CMD: gcloud compute ssh psarthi-dataproc-test-m --zone us-central1-a --command="PREFIX=diag_20231024231902_Eb9EE5fdPLATFORM_TYPE=dataproc/tmp/collect.sh"--format json>[

@parthosa parthosa added the user_tools Scope the wrapper module running CSP, QualX, and reports (python) label Oct 24, 2023
@parthosa parthosa self-assigned this Oct 24, 2023
Signed-off-by: Partho Sarthi <[email protected]>
nartal1
nartal1 previously approved these changes Oct 25, 2023
Copy link
Collaborator

@nartal1 nartal1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @parthosa !

if isinstance(cmd, list):
cmd_list = cmd
else:
cmd_list = cmd.split(' ')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this is more concise, nit: cmd_list = cmd if isinstance(cmd, list) else cmd.split(' ')

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored to make it more concise.

cindyyuanjiang
cindyyuanjiang previously approved these changes Oct 27, 2023
Signed-off-by: Partho Sarthi <[email protected]>
@parthosa parthosa dismissed stale reviews from cindyyuanjiang and nartal1 via 585d326 October 27, 2023 15:40
Copy link
Collaborator

@cindyyuanjiang cindyyuanjiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @parthosa!

@parthosa parthosa merged commit 6f1a573 into NVIDIA:dev Oct 27, 2023
8 checks passed
@parthosa parthosa deleted the spark-rapids-tools-632 branch October 27, 2023 21:00
@parthosa parthosa added the bug Something isn't working label Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working user_tools Scope the wrapper module running CSP, QualX, and reports (python)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Log statement in user tools has unnecessary spaces
3 participants