OutputPathGenerator is still flakey.

[charleskawczynski]

ERROR: Path output\default_config\output_active not properly synced
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:35
 [2] maybe_wait_filesystem(context::ClimaComms.SingletonCommsContext{…}, path::String; check_func::ClimaUtilities.OutputPathGenerator.var"#6#8", sleep_time::Float64, max_attempts::Int64)
   @ ClimaUtilities.OutputPathGenerator C:\Users\kawcz\.julia\packages\ClimaUtilities\KKORG\src\OutputPathGenerator.jl:47
 [3] maybe_wait_filesystem
   @ C:\Users\kawcz\.julia\packages\ClimaUtilities\KKORG\src\OutputPathGenerator.jl:33 [inlined]
 [4] generate_output_path(::ClimaUtilities.OutputPathGenerator.ActiveLinkStyle, output_path::String; context::ClimaComms.SingletonCommsContext{…})
   @ ClimaUtilities.OutputPathGenerator C:\Users\kawcz\.julia\packages\ClimaUtilities\KKORG\src\OutputPathGenerator.jl:222
 [5] generate_output_path
   @ C:\Users\kawcz\.julia\packages\ClimaUtilities\KKORG\src\OutputPathGenerator.jl:164 [inlined]
 [6] #generate_output_path#2
   @ C:\Users\kawcz\.julia\packages\ClimaUtilities\KKORG\src\OutputPathGenerator.jl:134 [inlined]
 [7] get_sim_info(config::ClimaAtmos.AtmosConfig{…})
   @ ClimaAtmos C:\Users\kawcz\Dropbox\Caltech\work\dev\CliMA\ClimaAtmos.jl\src\solver\type_getters.jl:495
 [8] get_simulation(config::ClimaAtmos.AtmosConfig{…})
   @ ClimaAtmos C:\Users\kawcz\Dropbox\Caltech\work\dev\CliMA\ClimaAtmos.jl\src\solver\type_getters.jl:604
 [9] top-level scope
   @ REPL[30]:1
Some type information was truncated. Use `show(err)` to see complete types.

[Sbozzolo]

Can you provide steps to reproduce?

I need more information to understand what has happened here. The error is produced whan the state on the filesystem is inconsistent with expecatations (which might happen on distributed filesystems).

In your case, this was triggered by line 222, which means that a link was found when it should have not been there.

[charleskawczynski]

Another failure example:

C:\Users\kawcz\dev\ClimaAtmos.jl>julia .dev\climaformat.jl .
  Activating project at `C:\Users\kawcz\dev\ClimaAtmos.jl\.dev`
ERROR: LoadError: TaskFailedException

    nested task error: TaskFailedException

        nested task error: TaskFailedException

            nested task error: IOError: realpath("C:\\Users\\kawcz\\Dropbox\\Caltech\\work\\dev\\CliMA\\ClimaAtmos.jl\\output\\default_config\\output_active"): no such file or directory (ENOENT)
            Stacktrace:
             [1] uv_error
               @ .\libuv.jl:100 [inlined]
             [2] realpath(path::String)
               @ Base.Filesystem .\path.jl:487
             [3] format(path::String, options::JuliaFormatter.Configuration)
               @ JuliaFormatter C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:428
             [4] macro expansion
               @ C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:444 [inlined]
             [5] (::JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}})(tid::Int64; onethread::Bool)
               @ JuliaFormatter .\threadingconstructs.jl:215
             [6] #187#threadsfor_fun
               @ .\threadingconstructs.jl:182 [inlined]
             [7] (::Base.Threads.var"#1#2"{JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}}, Int64})()
               @ Base.Threads .\threadingconstructs.jl:154
        Stacktrace:
         [1] threading_run(fun::JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}}, static::Bool)
           @ Base.Threads .\threadingconstructs.jl:172
         [2] macro expansion
           @ .\threadingconstructs.jl:220 [inlined]
         [3] format(path::String, options::JuliaFormatter.Configuration)
           @ JuliaFormatter C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:442
         [4] macro expansion
           @ C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:444 [inlined]
         [5] (::JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}})(tid::Int64; onethread::Bool)
           @ JuliaFormatter .\threadingconstructs.jl:215
         [6] #187#threadsfor_fun
           @ .\threadingconstructs.jl:182 [inlined]
         [7] (::Base.Threads.var"#1#2"{JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}}, Int64})()
           @ Base.Threads .\threadingconstructs.jl:154
    Stacktrace:
     [1] threading_run(fun::JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}}, static::Bool)
       @ Base.Threads .\threadingconstructs.jl:172
     [2] macro expansion
       @ .\threadingconstructs.jl:220 [inlined]
     [3] format(path::String, options::JuliaFormatter.Configuration)
       @ JuliaFormatter C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:442
     [4] macro expansion
       @ C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:444 [inlined]
     [5] (::JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}})(tid::Int64; onethread::Bool)
       @ JuliaFormatter .\threadingconstructs.jl:215
     [6] #187#threadsfor_fun
       @ .\threadingconstructs.jl:182 [inlined]
     [7] (::Base.Threads.var"#1#2"{JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}}, Int64})()
       @ Base.Threads .\threadingconstructs.jl:154
Stacktrace:
 [1] threading_run(fun::JuliaFormatter.var"#187#threadsfor_fun#295"{JuliaFormatter.var"#187#threadsfor_fun#292#296"{String, Base.Threads.Atomic{Bool}, Vector{String}}}, static::Bool)
   @ Base.Threads .\threadingconstructs.jl:172
 [2] macro expansion
   @ .\threadingconstructs.jl:220 [inlined]
 [3] format(path::String, options::JuliaFormatter.Configuration)
   @ JuliaFormatter C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:442
 [4] format
   @ C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:420 [inlined]
 [5] format(paths::Vector{String}; options::@Kwargs{indent::Int64, margin::Int64, always_for_in::Bool, whitespace_typedefs::Bool, whitespace_ops_in_indices::Bool, remove_extra_newlines::Bool})
   @ JuliaFormatter C:\Users\kawcz\.julia\packages\JuliaFormatter\7y9aN\src\JuliaFormatter.jl:413
 [6] top-level scope
   @ C:\Users\kawcz\dev\ClimaAtmos.jl\.dev\climaformat.jl:85
in expression starting at C:\Users\kawcz\dev\ClimaAtmos.jl\.dev\climaformat.jl:85