sr3 3.0.53 mkdir crash when configurations have statehost set.

[petersilva]

[me@host ~]$ sr3 status
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
missing state for winnow/winnow-ss5-04
Traceback (most recent call last):
  File "/home/pas037/.local/bin/sr3", line 11, in <module>
    load_entry_point('metpx-sr3==3.0.53', 'console_scripts', 'sr3')()
  File "/usr/lib/python3.6/site-packages/sarracenia/sr.py", line 2963, in main
    gs = sr_GlobalState(cfg, cfg.configurations)
  File "/usr/lib/python3.6/site-packages/sarracenia/sr.py", line 1212, in __init__
    self._resolve()
  File "/usr/lib/python3.6/site-packages/sarracenia/sr.py", line 909, in _resolve
    os.mkdir(self.user_cache_dir + os.sep + c + os.sep + cfg)
FileExistsError: [Errno 17] File exists: '/home/pas037/.cache/sr3/winnow/winnow-ss5-04'
[me@host ~]$

[petersilva]

crash addressed by #1076 but not root cause.
Root cause seems to be not looking at statehost in that part of the code. more work needed.

[petersilva]

diagnosis:

• the actual state directory (with statehost) is missing and needs to be created.
• code wrongly tries to make the directory (without statehost) and that exists.
• will actually work once... (creating the wrong directory) but on second invocation will crash with EEXIST.

Actual fix is to create the correct state directory, rather than the wrong one.

work-around: create the correct state directory manually/outside sr3. If it exists, it's lack won't be detected, and it won't mkdir it.

[me@host sr3]$ sr3 features
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
statehost
missing state for winnow/winnow-ss5-03
Traceback (most recent call last):
  File "/home/pas037/.local/bin/sr3", line 11, in <module>
    load_entry_point('metpx-sr3', 'console_scripts', 'sr3')()
  File "/fs/homeu2/ssc/di/pas037/Sarracenia/sr3/sarracenia/sr.py", line 2963, in main
    gs = sr_GlobalState(cfg, cfg.configurations)
  File "/fs/homeu2/ssc/di/pas037/Sarracenia/sr3/sarracenia/sr.py", line 1212, in __init__
    self._resolve()
  File "/fs/homeu2/ssc/di/pas037/Sarracenia/sr3/sarracenia/sr.py", line 909, in _resolve
    os.mkdir(self.user_cache_dir + os.sep + c + os.sep + cfg)
FileExistsError: [Errno 17] File exists: '/home/pas037/.cache/sr3/winnow/winnow-ss5-03'
[me@host sr3]$

repair by building the missing state directory:

[me@host sr3]$ mkdir ~/.cache/sr3/host/winnow/winnow-ss5-03

on each host where the command is to be invoked... this may be quite tedious on a large cluster where hundreds of hosts are involved.

[petersilva]

A second work-around, in the case of missing directories, is to start them up on the machine.
This will create the missing directories for the local host.

[petersilva]

There is an additional weirdness where two host directories are created, one with fqdn, the other with just the hostname. the fqdn one gets the an empty log file created, and the the other gets the rest of the state files (including a functional log file.)

fix is here: dfe67d8

[petersilva]

Alternative approach to fixing this problem: why are there mkdirs in that part of the code at all... it means that if we run "sr3 status" it creates a whole bunch of state directories... feels wrong and odd. It should be just reading the state that is there. another branch is taking that approach.

[petersilva]

remove unneded statehost print 4ac239a