filemanager: add crawl function

[mmalenic]

I think the filemanager API should have a crawl function. This could be based on the existing inventory function, but I think it might be more convenient to just use list operations directly. Often records become out of date due to missing features or bugs, so along with ingesting new objects, the crawl function could be used to correct records if they are inconsistent. By placing this inside the filemanager directly, S3 API code can be re-used, rather than tying the logic to update scripts and individual API calls.

Related to #844.

[alexiswl]

@mmalenic could this be a workaround in the interim? https://stackoverflow.com/a/37475908/6946787

[mmalenic]

Possibly.. All the filemanager needs to link the objects is the umccr-org:OrcaBusFileManagerIngestId tag which is present on objects, which should have the value of the ingest_id. So if copying from source to destination also copies the tags and that triggers a new Object Created event, then the filemanager should pick up the existing tag.

[mmalenic]

Sorry to clarify, the tag with the ingest id is already present, so I think this should work if it creates a new Object Created event.

[alexiswl]

No stress, for these runs we're compressing them so they will generate new objects anyway