Advancedharvestmodes

Record providers will have the ability to select from three Advanced Harvest Modes to change the behaviour of the ANDS Harvester. These harvest modes include:

• Standard - will harvest all records from the data source feed (this will become the default mode)
• Incremental - will use OAI-PMH to support harvesting on a "from and until" basis. This means that the Harvester will only add new records that have been created or modified "from" the last harvest date, "until" the date of the new harvest.
• Refresh - will consider the latest Harvest to be the "complete set of records" and remove all previously harvested records that are no longer in the feed. Records created manually using the Add Registry Object screens will not be removed.

The "Incremental" harvest mode will only function correctly with OAI-PMH providers that correctly support the "from and until" parameters as per the OAI-PMH Specification.

Notes:

For existing providers, the mode will be set to "Incremental" (which has identical behaviour to the current harvester).

Any new providers will have the mode default to "Standard". ANDS strongly recommends that providers who support OAI-PMH Incremental Harvesting should update the Advanced Harvest Mode to "Incremental" to ensure that their harvests perform optimally.

Harvester will ignore multiple copies of the same record (i.e. multiple records with identical keys) when they are received in the same harvest. The first record will be harvested and any duplicates found in that same harvest will be ignored.