I-ALiRT Data Access Package

This lightweight Python package allows users to query the I-ALiRT database and list/download files from S3.

Command Line Utility

To install

pip install ialirt-data-access
ialirt-data-access -h

Query / Search for logs

Find all files from a given year, day of year, and instance

$ ialirt-data-access --url <url> ialirt-log-query --year <year> --doy <doy> --instance <instance>

Query / Search for packets

Find all files from a given year, day of year, hour, minute, and second.

$ ialirt-data-access --url <url> ialirt-packet-query --year <year> --doy <doy> [--hh <hour>] [--mm <minute>] [--ss <second>]

Download from S3

Download a file and place it in the Downloads/ directory by default, or optionally specify another location using --downloads_dir. Valid filetype options include: logs, packets, archive.

$ ialirt-data-access --url <url> ialirt-download --filetype <filetype> --filename <filename>

Query the database

Query the database for a given time. Examples shown below.

$ ialirt-data-access --url <url> ialirt-db-query --met_in_utc_start <met_in_utc_start> --met_in_utc_end <met_in_utc_end>

or

$ ialirt-data-access --url <url> ialirt-db-query --met_start <met_start> --met_end <met_end>

or

$ ialirt-data-access --url <url> ialirt-db-query --last_modified_start <last_modified_start> --last_modified_end <last_modified_end>

or to return all data from met_start onward

$ ialirt-data-access --url <url> ialirt-db-query --met_start <met_start>

Importing as a package

import ialirt_data_access

# Search for files
results = ialirt_data_access.log_query(year="2024", doy="045", instance="1")

Configuration

Data Access URL

To change the default URL that the package accesses, you can set the environment variable IALIRT_DATA_ACCESS_URL or within the package ialirt_data_access.config["DATA_ACCESS_URL"]. The default is the production server https://ialirt.imap-mission.com.

Automated use with API Keys

The default for the CLI is to use the public endpoints. To access some unreleased data products and quicklooks, you may need elevated permissions. To programmatically get that, you need an API Key, which can be requested from the SDC team.

To use the API Key you can set environment variables and then use the tool as usual. Note that the api endpoints are prefixed with /api-key to request unreleased data. This will also require an update to the data access url. So the following should be used when programatically accessing the data.

IMAP_API_KEY=<your-api-key> IALIRT_DATA_ACCESS_URL=https://ialirt.imap-mission.com/api-key ialirt-data-access ...

or with CLI flags

ialirt-data-access --api-key <your-api-key> --url https://ialirt.imap-mission.com/api-key ...

Example:

ialirt-data-access --api-key <api_key> --url https://ialirt.imap-mission.com/api-key ialirt-db-query --met_start 100

Troubleshooting

Network issues

SSL

If you encounter SSL errors similar to the following:

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>

That generally means the Python environment you're using is not finding your system's root certificates properly. This means you need to tell Python how to find those certificates with the following potential solutions.

1. Upgrade the certifi package

   pip install --upgrade certifi

2. Install system certificates Depending on the Python version you installed the program with the command will look something like this:

   /Applications/Python\ 3.10/Install\ Certificates.command

HTTP Error 502: Bad Gateway

This could mean that the service is temporarily down. If you continue to encounter this, reach out to the IMAP SDC at [email protected].