Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
terrywbrady authored May 24, 2024
1 parent 68635d5 commit 9efc26f
Showing 1 changed file with 24 additions and 3 deletions.
27 changes: 24 additions & 3 deletions design/content-ingest-2024/README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,45 @@
# Content Ingest Workspace Design

## Workspace Domain
- Workspace should be considered a DEV environment.
- Content may be ingested into either Stage or Prod.

## File System

ZFS/EFS file systems should be mounted and unmounted as projects are started and completed.
All content should be accessible to CDL servers by URL.

Moderately performant to ingest.

- Merritt Box process should write to the workspace
- Any hard drives that are sent to Merritt should be copied to this workspace

If no active content projects are in process, the environment should scale down to minimal costs.

Consider a 2TB workspace, separate from the shared ZFS to upload incoming content to.

File system should be mounted to the ingest servers.
File system should be mounted to the PROD and STAGE ingest servers as well as DEV servers for manifest creation.

Merritt manifests will pull content with `file://` URLs.

Possibly want to back up until content goes away.


## Compute Environment

Activities
- Run zips
- Run checksums
- Generate manfiests

Merritt manifests will pull content with file:// URLs.
Options
- re-use the batch servers (may need to evaluate instance type)
- provision a special batch server on demand
- get IAS' opinion
- ideally Eric could run a command to provision on-demand
- IAS requests would be needed to re-size
- Example: up for 6 weeks, then down for 6 weeks
- minimal scratch space
- scripts and tools should persist in source code manifest

## Cloud Storage

Expand Down

0 comments on commit 9efc26f

Please sign in to comment.