Better restarts

[mjberger]

added capability to restart the gauge output instead of clobbering the existing file
(did same in geoclaw)

also fixed a few output field widths.

[mandli]

This failing because gaugetools is importing matplotlib which we try not to do on Travis. I am not sure what is going wrong here though since this PR does not change any of the Python.

[rjleveque]

We need to update the documentation if we make this change, with some warnings that this isn't always the right thing to do. E.g. if the user restarts from a checkpoint file that wasn't at the final time from the previous run, or if several restarts are done from the same checkpoint file.

We started some discussion of this in #39.

Current documentation at http://www.clawpack.org/restart.html#output-files-after-a-restart

[mandli]

@mjberger can you merge in the current master branch so that the tests can be run?

[rjleveque]

Thinking more about this, I think appending to fort.gauge is not the best approach in general. If a restart is done from an earlier time than when the code died, then there will be a repeated set of lines in the file that should be edited out, or else missing lines, depending on whether all lines were printed or whether they were lost due to buffering. I'd suggest something more like what I've done in #144, where a restart causes the old fort.gauge to be moved to fort.gauge_<datetime> and the user can then easily examine the files to see what's redundant or missing. This would be hard to do in a single appended fort.gauge if it has hundreds of thousands of lines (often true in GeoClaw runs).

The way it's done in #144, the latest fort.gauge is still just called fort.gauge, which would also be useful if doing multiple restarts from the same checkpoint file in order to debug something going on later (whereas appending would not be what's wanted in this case).

[mjberger]

Your alternative approach is ok too although I think it requires more manual intervention. Inf the ordinary restart is bcause you want to run something longer, then appending is the right thing to do, instead of manually concatenating files. In the repeated case, you have to edit out the last lines from where you restart.

My main concern is not to lose data, so either is ok, but simply overwriting isn't.

[rjleveque]

Closing this since we decided not to do it this way.
I opened an issue clawpack/clawutil#93 as a reminder that this has not been fixed.