(Archiving) Refactor DDN flow

[islean]

Description

This PR splits the ddn_dataflow.py module up into three modules all in the DDN package. We have

• ddn_data_flow_client.py which houses mainly the class DDNDataFlowClient
• models.py which houses the many base models used in the DDN flow
• constants.py which houses the constants used in the DDN flow.

Changed

• Split the ddn_data_flow.py module into three and encapsulated them in a DDN package.

How to prepare for test

• Ssh to relevant server (depending on type of change)
• Use stage: us
• Paxa the environment: paxa
• Install on stage (example for Hasta):

  bash /home/proj/production/servers/resources/hasta.scilifelab.se/update-tool-stage.sh -e S_cg -t cg -b archiving-refactor-ddn -a

How to test

• Do ...

Expected test outcome

• Check that ...
• Take a screenshot and attach or copy/paste the output.

Review

• Tests executed by
• "Merge and deploy" approved by
  Thanks for filling in who performed the code review and the test!

This version is a

• MAJOR - when you make incompatible API changes
• MINOR - when you add functionality in a backwards compatible manner
• PATCH - when you make backwards compatible bug fixes or documentation/instructions

Implementation Plan

• Document in ...
• Deploy this branch on ...
• Inform to ...

[henrikstranneheim]

Nice 👍

[sonarqubecloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[islean]

Tested archiving files:

Files submitted to karolinska_bucket with archival task id 194.
Adding an archive to file 5057623 with archiving task id 194
Adding an archive to file 5057624 with archiving task id 194
1d [isak.ohlsson@hasta:~] [S_base] 28s $ 

update-job-statuses:

Miria returned status Completed for job 194
Job with id 194 has finished, updating Archive entries.

Reflected in the db:

Delete one file:

Deleting file /home/proj/stage/housekeeper-bundles/isaks_test_bundle/2023-12-06/isaks_test_spring_file.spring from disc
Deleting file 5057623 from housekeeper

[islean]

Deployed to production:

Logging deploy ...
Getting deployer... done.
Getting last commit message and SHA... done.
Getting version of deploy scripts... /home/isak.ohlsson
done.
Log deploy... done.
cg, version 54.4.6
1d [isak.ohlsson@hasta:~] [P_base] 2m28s $