Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to export dataset not detected, dataset marked as exported #8484

Closed
landreev opened this issue Mar 14, 2022 · 4 comments · Fixed by #8782
Closed

Failure to export dataset not detected, dataset marked as exported #8484

landreev opened this issue Mar 14, 2022 · 4 comments · Fixed by #8782
Assignees
Labels
Feature: Harvesting Feature: Metadata NIH OTA DC Grant: The Harvard Dataverse repository: A generalist repository integrated with a Data Commons NIH OTA: 1.4.1 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 prdOwnThis is an item synched from the product ... pm.epic.nih_harvesting pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues pm.GREI-d-1.4.2 NIH, yr1, aim4, task2: Create working group on packaging standards
Milestone

Comments

@landreev
Copy link
Contributor

This is a followup issue to #8452. The user's/installation's issue was addressed by modifying the export code to properly handle a specific metadata field becoming multiple (#8473). But there was one underlying bug that I would still like to address: despite the failure to export, the dataset was marked as exported; and then added to the OAI set. (The fact that the partial, incomplete export was left written on disk made it even worse).

Must be a straightforward enough issue of properly intercepting exceptions.

@landreev
Copy link
Contributor Author

Added the "feature: harvesting" label, to add it to the pile of Harvesting issues that we want to fix as part of the GREI effort. (#8574)

@landreev
Copy link
Contributor Author

This is not a specifically harvesting issue of course; but it does affect harvesting (a record like this breaks the ListRecords output; that's how it was discovered/reported).
Thinking about it, this should be addressed in both places - the export code needs to do a better job detecting errors; and the Harvesting server code needs to be able to detect the condition of the exported metadata fragment missing, without generating invalid output.

@mreekie
Copy link

mreekie commented Apr 27, 2022

sprint:

  • this one is a side effect from an fix for an installation in France. That issue was addressed, and this came up during the investigation
  • straightforward. Testing is time consuming.
  • small

@mreekie mreekie added pm.epic.nih_harvesting NIH OTA DC Grant: The Harvard Dataverse repository: A generalist repository integrated with a Data Commons labels May 9, 2022
@landreev landreev self-assigned this May 18, 2022
@mreekie
Copy link

mreekie commented May 25, 2022

landreev added a commit that referenced this issue Jun 8, 2022
landreev added a commit that referenced this issue Jul 19, 2022
@pdurbin pdurbin added this to the 5.12 milestone Jul 25, 2022
@mreekie mreekie added the NIH OTA: 1.4.1 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 prdOwnThis is an item synched from the product ... label Nov 4, 2022
@mreekie mreekie added pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues pm.GREI-d-1.4.2 NIH, yr1, aim4, task2: Create working group on packaging standards labels Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature: Harvesting Feature: Metadata NIH OTA DC Grant: The Harvard Dataverse repository: A generalist repository integrated with a Data Commons NIH OTA: 1.4.1 4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 prdOwnThis is an item synched from the product ... pm.epic.nih_harvesting pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues pm.GREI-d-1.4.2 NIH, yr1, aim4, task2: Create working group on packaging standards
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants