Test for duplicate data sets

[AndyClifton]

Pull the tests for duplicate data sets into the main code (not just rattling round on my laptop) to ensure that we are not double counting.

Add some kind of tolerance (e.g. 1% variability) to detect cases where slightly different preanalyses might have been applied to the data set (e.g. trimming one day off)