JIRA [IMG-1024] BatchProfiler improvements

[0x00b1]

From @thouis on November 23, 2011 20:50

1. automatic resub attempts
   2. dynamic job status reporting
   3. automatic testing
   4. generalization and documentation to allow others to utilize

Original reporter: klmadden
Jira link: https://jira.broadinstitute.org/browse/IMG-1024
Fix version: future
Assignee: leek

Comment from anne on Mon, 25 Apr 2011 17:50:40 -0400:
A more complete description of these, from the CP SW grant:

* Automatic resubmission attempts. Currently, researchers monitor the web page to detect failed or stalled jobs, diagnose the cause, and restart them. We will enable BatchProfiler to resubmit jobs that have transient failures.
* Dynamic job status reporting. Currently, the researcher must monitor the web page until all jobs are done. We will enable BatchProfiler to notify the user of job progress.
* Automatic testing. Currently, researchers discover failures in the BatchProfiler/CellProfiler tool chain only when jobs fail. We will set up automatic daily testing of the tool chain, with results reported to a local administrator. We will also package BatchProfiler for automated installation and testing.
* Generalization and documentation. Currently, BatchProfiler is tailored to the computing environment at the Broad Institute; administrators in other computing environments would need to alter the code accordingly. We will separate the web-server logic from the configuration data to enable adaptation to other computing environments, and we will create documentation to facilitate the process.

Copied from original issue: CellProfiler/CellProfiler#154