SDAP-481: Enable internal compression for netcdf match up outputs

[skorper]

SDAP-481

(NOTE: Will change PR target once 455 is merged)

• Enabled internal compression for netcdf match up outputs
• Upgraded to netcdf4==1.6.4. This is required as the compression params in createVariable are only available in newer versions of this library.

---

Tested this locally. Compared the output with the following command output:

nccopy -4 -d 5 ...

Confirmed variables all contain _DeflateLevel = 5 in the following output:

ncdump -h -s ...

Vardis also confirmed the outputs look as expected.

---

Idea for future improvement -- allow user to specify compression level? This is currently set to 5.

[RKuttruff]

Trying to verify, but the netcdf4 dependency bump may be causing the image build to fail due to mamba being unable to solve the environment.

[skorper]

Hmm thanks @RKuttruff. I don't have mamba installed so my local build used Conda. I'll take a look and see if I can figure out which dependency is causing the mamba solve issue

[RKuttruff]

What I did to test this is that I commented out the analysis install and everything following it from the dockerfile then ran a shell in a container from that image so that I can run the mamba install myself to see what's going on.
I compared with the previous version, and mamba could solve it really fast (like on the order of single minutes).

When trying to solve with the bumped dependency version, it fails after about 2+ hours with the message:

WARNING solver.cpp:308 Problem count: 1

Encountered problems while solving.
Problem: package gdal-3.2.1-py38hc0b2d6b_4 requires libgdal 3.2.1 h0a4f56a_4, but none of the providers can be installed

[RKuttruff]

Also confirmed that the environment solves successfully without the gdal requirement so that may need to be bumped.
From what it looks like, both netcdf4 and gdal have libnetcdf as a dependency, but differing versions: 4.9.2 vs 4.7.4

[skorper]

I'm trying out different versions of Gdal now. thanks for bringing this to my attention