Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix mem_per_cpu in SlurmIO not being passed as snake_case #44

Merged
merged 1 commit into from
Aug 7, 2024

Conversation

janosh
Copy link
Collaborator

@janosh janosh commented Aug 6, 2024

passing memory_per_thread currently crashes job submissions with ValueError

resources = QResources(
   # ...
   memory_per_thread=10_000,
)

reason being that it's mapped to mem-per-gpu by _qresources_mapping

"memory_per_thread": "mem-per-cpu",

but

#SBATCH --mem-per-cpu=$${mem_per_cpu}

expects it as snake case

  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/jobflow_remote/joline 3334, in lock_job_for_update                                            
    yield lock                                                               
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/jobflow_remote/joline 538, in advance_state                                                   
    states_methods[state](lock)                                              
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/jobflow_remote/joline 685, in submit                                                          
    submit_result = queue_manager.submit(                                    
                    ^^^^^^^^^^^^^^^^^^^^^                                    
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/jobflow_remote/reline 155, in submit                                                          
    script_fpath = self.write_submission_script(                             
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                             
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/jobflow_remote/reline 186, in write_submission_script                                         
    script_str = self.get_submission_script(                                 
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                 
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/jobflow_remote/reline 104, in get_submission_script                                           
    return self.scheduler_io.get_submission_script(commands_list, options)   
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/qtoolkit/io/base.… 
line 57, in get_submission_script                                            
    if header := self.generate_header(options):                              
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                               
  File                                                                       
"/lambdafs/janosh/.venv/p312/lib/python3.12/site-packages/qtoolkit/io/base.… 
line 88, in generate_header                                                  
    raise ValueError(msg)                                                    
ValueError: The following keys are not present in the template: mem-per-cpu  
                                                                             
}

@janosh janosh changed the title snake_case mem_per_cpu in SlurmIO Fix mem_per_cpu in SlurmIO not being passed as snake_case Aug 6, 2024
@ml-evs ml-evs self-requested a review August 6, 2024 21:45
@janosh
Copy link
Collaborator Author

janosh commented Aug 7, 2024

a thing to note here is that this wasn't caught by tests because it seems the template insertion isn't run by any test but probably should be?

Copy link
Member

@ml-evs ml-evs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a thing to note here is that this wasn't caught by tests because it seems the template insertion isn't run by any test but probably should be?

Yeah, I was looking into writing some tests for this as part of my review, but let's not hold this up. Thanks for reporting and fixing @janosh!

@ml-evs ml-evs merged commit 833f1f0 into Matgenix:develop Aug 7, 2024
5 checks passed
@janosh janosh deleted the fix-mem-per-thread branch August 7, 2024 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants