v3.2.0

What's Changed

• Add gradiometry job
• Use access-protected SDC APIs
• Database now stores creation, modification and deletion dates for files
• Fix bug with nanosecond precision in science CDFs for calibration
• Fix CI/CD failures due to Windows jobs (temporarely disabled)
• Dependency updates

PRs

• task: Add support for IMAP access key by @mfacchinelli in #113
• task: Add creation, modified, and deletion dates for database files by @mfacchinelli in #115
• Fix/timestamp precision bug by @mhairifin in #114
• fix: Use quality_flags (plural) for science CDFs by @mfacchinelli in #116
• build: Update version to 3.2.0 by @mfacchinelli in #122
• ci(fix): Pre-commit autoupdate permissions by @mfacchinelli in #117
• feat: Gradiometry Calibration by @mhairifin in #107
• build(deps-dev): bump pytest from 8.3.3 to 8.4.1 by @dependabot[bot] in #118
• Chore: Update Pre-Commit Hooks by @github-actions[bot] in #124
• build(deps-dev): bump pytest-cov from 5.0.0 to 6.2.1 by @dependabot[bot] in #121
• build(deps-dev): bump pytest-asyncio from 0.26.0 to 1.1.0 by @dependabot[bot] in #120
• build(deps): bump psycopg from 3.2.5 to 3.2.9 by @dependabot[bot] in #119
• build: Dependency updates by @mfacchinelli in #125
• fix: Merge issue in gradiometry flow definition by @mfacchinelli in #126
• Move off of wiremock fork by @mhairifin in #127
• fix: Use correct pattern to query database by @mfacchinelli in #128
• task: (temp) remove win tests & pyinstaller, build only on PRs by @alastairtree in #130
• fix: MATLAB tests and SQL injection by @mfacchinelli in #129
• build(fix): libgdk-pixbuf package name has changed by @mfacchinelli in #133
• fix: add additional test assertions for correct metadata by @mhairifin in #131

New Contributors

• @github-actions[bot] made their first contribution in #124

Full Changelog: v3.1.0...v3.2.0

Testing

Tested in the test environment on 14/08/2025 and confirmed working correctly.

Deployed to prod: https://github.com/ImperialCollegeLondon/mag-lab-data-platform/actions/runs/16967856194

v3.1.0

• Download all MAG HK packets (including error packets)
• Refactor downloading of HK from WebPODA to make sure new decoded (L1) HK file versions include existing data, not just the newly downloaded data

What's Changed

• task: Add all MAG HK for polling by @mfacchinelli in #105
• refactor: HK download and file management by @mfacchinelli in #106

Full Changelog: v3.0.0...v3.1.0

Tested in the test environment on 24/07/2025 and confirmed working correctly.

Deployed to prod: https://github.com/ImperialCollegeLondon/mag-lab-data-platform/actions/runs/16499268711

v3.0.0

What's Changed

• Better layout of files in the datastore so hk, science and calibration are clearly defined separate.
• Many Fixes and Improvements by @mfacchinelli in #96
  Only download MAGo for L2 data (downloading MAGi results in duplicates, as sensor name ignored for L2)
  Wrong polling schedule caused by CRON env var names for polling science
  Downloads binary packets in a big chunck instead of by day (consistent with SDC APIs)
• fixed: correct naming of offset files by @mhairifin in #87
• fixed CDF L2 Pre has magnitudes by @mhairifin in #100

Updated dependencies

• build(deps): bump typer from 0.15.2 to 0.15.4 by @dependabot in #93
• build(deps): bump requests from 2.32.3 to 2.32.4 by @dependabot in #89
• build(deps): bump numpy from 2.1.3 to 2.2.6 by @dependabot in #90
• build(deps): bump typing-extensions from 4.12.2 to 4.14.0 by @dependabot in #91

Full Changelog: v2.2.0...v3.0.0

Deployed https://github.com/ImperialCollegeLondon/mag-lab-data-platform/actions/runs/16138866294/job/45541407648

v2.2.0

What's Changed

• feat: Publish files to SDC by @mfacchinelli in #85
• Download L2 files (also in #85)

This release enables publishing of offset files to SDC (post-calibration) and enables automatic daily polling of L2 files.

Tested in the test environment on 01/07/2025 and confirmed working correctly.

Deployed to PROD https://github.com/ImperialCollegeLondon/mag-lab-data-platform/actions/runs/15998292134/job/45126699267

Full Changelog: v2.1.0...v2.2.0

v2.1.0

What's Changed

• feat: Calibration Layers by @mhairifin in #65
• Feat/adding cal code by @mhairifin in #80

This release adds the first working version of calibration layers and the pipeline now does and "empty" calibration of all zeros, and then applies that to an L1C file to generate the L2 Prelim and L2 offset files for the SDC.

Tested in the test environment on 19/06/2025 and confirmed working correctly.

Deployed to PROD https://github.com/ImperialCollegeLondon/mag-lab-data-platform/actions/runs/15775982986/job/44470425255

Full Changelog: v2.0.1...v2.1.0

v2.0.1

What's Changed

• task: IMAP User by @mfacchinelli in #75
• build: Increase version to v2.0.1 by @mfacchinelli in #77

Full Changelog: v2.0.0...v2.0.1

Release checklist: #76
Deployment to prod: 15684509750

v2.0.0

What's Changed

• feat: Poll SDC by @mfacchinelli in #60
• Refactor config in polling HK for CLI and prefect by @alastairtree in #67
• feat: Add support for SDC ingestion date and WebPODA ERT by @mfacchinelli in #68
• feat: Poll WebPODA and SDC by @mfacchinelli in #57
• build(deps): bump dorny/test-reporter from 1 to 2 by @dependabot in #64
• build(deps): bump cdflib from 1.3.1 to 1.3.3 by @dependabot in #48
• build(deps-dev): bump testcontainers from 4.8.1 to 4.9.1 by @dependabot in #50
• build(deps): bump sqlalchemy from 2.0.35 to 2.0.38 by @dependabot in #54
• chore: Dependencies by @mfacchinelli in #71
• build(deps): bump h11 from 0.14.0 to 0.16.0 by @dependabot in #72
• test: Windows fix by @mfacchinelli in #73

Full Changelog: v1.0.0...v2.0.0

Release checklist: #70
Deployment to prod: N/A

Initial release

v1.0.0

Merge remote-tracking branch 'origin/refactor/split-main' into rel/1.0