Documentation: Additional clarity about pre-specified resource info should be added to the getting started docs

[Andrew-S-Rosen]

Now that I have the appropriate hardware set up over here, I am finally trying out jobflow-remote!

I have a very basic question regarding setup. I am using the all-in-one approach where I am running everything from a dedicated login node that can directly jobs to the Slurm queue. However, it is not obvious to me from the setup instructions how to do things like set the default Slurm account name. How do I go about modifying these kinds of default parameters? Apologies if I'm missing this in the docs. If it's not there in a prominent place, this issue can be considered a docs update request :)

workers:
  basic:
    type: local
    scheduler_type: slurm
    work_dir: /scratch/gpfs/ROSENGROUP/asrosen/jobflow
    pre_run: conda activate quacc
    timeout_execute: 60

[gpetretto]

Hi @Andrew-S-Rosen,
good to hear that you are all set now and you are willing to try jobflow-remote.

I think you are right that this is probably not properly covered in the documentation, as information are scattered in different places. I will try to improve it on this point.
Now, to answer your question, everything that is used to generate the submission script should be set in the resources. For each Job, the system will look for the resources details in two places:

1. first it will check if the resources are set in the specific Job. This can be done at submission time by passing the resources to submit_flow or customizing the values per job with set_run_config
2. if a value is not defined for a specific Job, it will use the resources specified in the worker configuration. In your case an example could be:

workers:
 basic:
   type: local
   scheduler_type: slurm
   work_dir: /scratch/gpfs/ROSENGROUP/asrosen/jobflow
   pre_run: conda activate quacc
   timeout_execute: 60
   resources: 
       account: xxx
       partition: yyy
       tasks: 128
       nodes: 1

Note that the system will only use one of the two. It will not attempt to make a mix. If this is a request, we can consider adding an option to allow that.

Does this answer the question?

[Andrew-S-Rosen]

@gpetretto: Thank you very much for the helpful response! Yes, this makes perfect sense! I figured it must have been something like this but wasn't sure the details. This is great.

To explain my thought process, I was going through the docs in order. I did the setup and installation + the project configuration. This then lead me to the quickstart, so I wanted to test out my configuration with the minimal example. However, the minimal example kept failing because (of course) I did not have any of the Slurm resources specified, and the lack of an account in this case was causing an error. This would have been clarified if I had gone to the tuning job execution section, but since I was not done with the minimal quickstart, I posted here instead.

I did manage to stumble upon the resources section you shared, but it also wasn't immediately obvious to me what my options were because it linked to the full QToolKit documentation rather than an example YAML with resources and a link to all the available arguments. The docs also have a SLURM hyperlink, but it wasn't obvious to me how header_template relates to the resources keyword argument.

In any case, this is looking great! And now I'm ready to go :) The above comment is mainly just to highlight my own progression of things. When I start using jobflow-remote regularly next month, I can also contribute some bits to the docs.