1.11 YAML Preingest Ingest

Overview of the Actor/Job stack in Hyrax, as related to ingest:

Early in the DefaultMiddleWareStack, there are two actors pertinent to attaching FileSets to a Work:

CreateWithRemoteFilesActor (for files as HashWithIndifferentAccess, from BrowseEverything)

• #create, calls:
  • #attach_files, loops calls to: (1)
    • #create_from_file_url
      • FileSetActor:
        • #create_metadata(visibility) --> InterpretVisibilityActor
        • #attach_to_work (2)
      • IngestLocalFileJob or ImportUrlJob (depending on uri type): #peform
        • FileSetActor: #create_content (3)

CreateWithFilesActor (for files as Hyrax::UploadedFile, from local file upload action)

• #create, calls:
  • #attach_files, calls:
    • AttachFilesToWorkJob: #perform, loops call to: (1)
      • FileSetActor:
        • #create_metadata(visibility) --> InterpretVisibilityActor
        • #create_content (3)
        • #attach_to_work (2)

Both call stacks end up calling FileSetActor#create_content, which conditionally makes direct calls to FileActor#ingest_file or builds IngestJob to do so: FileSetActor: #create_content

• IngestJob: #perform
  • FileActor: #ingest_file (3)

There are 3 considerations pertinent to ingest:

• (1) Where we need to capture file_set.id values to update the proxy values in the :structure map, if such is provided
• (2) Where we have an opportunity to improve performance by attaching the FileSets to the Work in batch, instead of incrementally
• (3) Where we need to tap the low-level file ingest to conditionally store files

We are currently using a custom actor to replace CreateWithRemoteFilesActor, which covers our needs for ingest.

We are not currently using a custom actor to replace CreateWithFilesActor, although we potentially could for the performance improvement of uploading multiple files at once, with regards to (2).