Move and adjust the s3 inventory

[stijnvanhoey]

• create new bucket "aloft-inventory" and move inventory their
• use the CSV inventory (not the parquet one)
• add the functionality to remove old inventory files (from manifest -> remove files)
• inventory need to be applied for all 'sources' (baltrad, ecol,...)
• the file name is used as single source of truth and the file name <-> s3-path logic is a function that can be passed (injected) to the routine.
• assign ec2 engine rights to access aloft-inventory bucket

[stijnvanhoey]

New bucket is created and inventory configuration switched.

inventory need to be applied for all 'sources' (baltrad, ecol,...)

The setup is done that it looks for */hdf5 as prefix of the path; need to verify that this effectively works.

[peterdesmet]

@stijnvanhoey there isn't necessarily an issue to include daily and monthly in that inventory. Those could also be included in the coverage file. It's fairly easy for the user to filter them out

[stijnvanhoey]

As the inventory is not working with the *-wildcard, w"ll manage without the daily/monthly and filter them on the data routine.

[stijnvanhoey]

add the functionality to remove old inventory files (from manifest -> remove files)

Will be tackled by @TheJenne18 as part of the updated deployment