PlanetaryPy

PlanetaryPy

Beta release: This will become (part of) the core package of the PlanetaryPy organisation.

Potential logo:

Install

pip install planetarypy

This will pull in these other dependencies and their dependencies:

tomlkit pandas pvl numpy python-dateutil tqdm lxml yarl hirise-tools kalasiris

Suggested standard abbreviations:

• Inside these docs the package will be called PLPY for brevity.
• A standard Python import could be: plp or plpy
  • because the last p in plp can be pronounced out, we consider these equivalent for human conversation and pronounce these “plippy”.

General scope

First and foremost this package should provide support in working with planetary science data.

With working we mean:

• locating
• retrieving
• reading
• further processing

of data.

Locating

This library manages, via its PDS tools, multiple PDS3 index files per instrument that can be used for identifying data of interest. These index files are automatically downloaded and converted to the very performant (and cloud-ready) parquet file format. > Parquet is able to store advanced datatypes like nan-capable integer and full datetime objects, as opposed to HDF5.

Retrieving

The interface to getting data is via a path-retrieving function based on a PDS product-id. If that product-id is available locally, the path will be returned. If it is not, it will previously be downloaded, stored in a systematic fashion organized by mission and instrument, and then the local path will be returned.

Reading

For now, the library only returns the path to the object and the user needs to sort out the reading process. A recently funded NASA project Planetary Data Reader will be integrated here, so that basic reading into memory can be provided.

As such, we anticipate two classes of reading support: 1. basic reading into numpy and/or xarray 1. added reader functionality like basic plots and basic geospatial processing, as supported by interested parties

There will exist larger other packages that focus on working with a given instrument’s data, in which case that package could become an affiliated package with the planetarypy GitHub organization, if so desired.

Further processing

In the future, additional frequently used procedures will be added to this library, e.g. * frequently used GDAL/rasterio procedures * frequently used SPICE operations * like surface illumination on a given body

PDS tools

Look at the Apps docs to see what pds.apps exist for easily getting PDS indexes. The find_index app is specifically useful when you don’t know what index files exist.

So far, the following indexes are supported (but not necessarily all tested within PLPY):

• Cassini
  • ISS (all)
  • UVIS (all)
• MRO
  • CTX EDR
  • HiRISE
    • EDR, RDR, DTM
      • EDR index has a bug (as delivered by the team, reported), where I need to activate an existing fix for it.
• LRO
  • Diviner (DLRE)
    • EDR, RDR
  • LOLA
    • EDR, RDR

More indexes

More indexes of other instruments can be easily added by following the existing structure of what has been copied into your config at ~/.planetarypy_config.toml.

Please consider submitting a pull request for adding further PDS index files into the config file at its source: https://github.com/michaelaye/nbplanetary/blob/master/planetarypy/data/planetarypy_config.toml

Utils

Find something in Utils for working with NASA timestamps and a well working URL download function url_retrieve, among other stuff.

Experiment/Instrument Specific

So far, planetarypy supports CTX EDR and HiRISE RGB.NOMAP data. Look at the CTX and HiRISE pages for descriptions of classes for working with these data.

Bug reports

Please submit bug reports at https://github.com/michaelaye/nbplanetary/issues

How to use

Indexes

See PDS apps for more details.

from planetarypy.pds.apps import get_index

ctrindex = get_index("mro.ctx", "edr", refresh=False)
ctrindex.sample(5, random_state=42)  # setting random_state to always get same files for docs

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	VOLUME_ID	FILE_SPECIFICATION_NAME	ORIGINAL_PRODUCT_ID	PRODUCT_ID	IMAGE_TIME	INSTRUMENT_ID	INSTRUMENT_MODE_ID	LINE_SAMPLES	LINES	SPATIAL_SUMMING	...	SUB_SOLAR_LATITUDE	SUB_SPACECRAFT_LONGITUDE	SUB_SPACECRAFT_LATITUDE	SOLAR_DISTANCE	SOLAR_LONGITUDE	LOCAL_TIME	IMAGE_SKEW_ANGLE	RATIONALE_DESC	DATA_QUALITY_DESC	ORBIT_NUMBER
86101	MROX_2757	DATA/J07_047671_1256_XN_54S338W.IMG	4A_04_10F0036A00	J07_047671_1256_XN_54S338W	2016-09-27 00:24:54.682	CTX	NIFL	2528	8192	1	...	-19.41	338.09	-54.43	207762612.4	230.58	15.48	90.1	Crater in Noachis Terra	OK	47671
54420	MROX_1819	DATA/D10_031170_1808_XN_00N221W.IMG	4A_04_109C017D00	D10_031170_1808_XN_00N221W	2013-03-21 06:58:09.892	CTX	NIFL	5056	7168	1	...	-24.39	221.52	0.88	209925149.5	286.15	14.49	90.1	Ride-along with HiRISE	OK	31170
71053	MROX_2331	DATA/F10_039530_1470_XI_33S203W.IMG	4A_04_10C9000F00	F10_039530_1470_XI_33S203W	2015-01-01 16:20:01.840	CTX	ITL	5056	52224	1	...	-25.3	203.96	-33.12	207070298.2	263.93	15.2	90.4	Terrain in Terra Cimmeria	OK	39530
51385	MROX_1742	DATA/D04_028919_2026_XN_22N327W.IMG	4A_04_109002A800	D04_028919_2026_XN_22N327W	2012-09-26 21:17:00.483	CTX	NIFL	5056	6144	1	...	0.67	327.06	22.78	219814412.3	178.44	15.53	90.1	Ride-along with HiRISE	OK	28919
51923	MROX_1757	DATA/D05_029283_1524_XN_27S180W.IMG	4A_04_1092020B00	D05_029283_1524_XN_27S180W	2012-10-25 05:44:21.786	CTX	NIFL	5056	9216	1	...	-6.28	178.36	-27.56	214830233.7	194.72	15.57	90.1	Ride-along with HiRISE	OK	29283

5 rows × 51 columns

hirise_rdr = get_index("mro.hirise", "rdr")
hirise_rdr.sample(5, random_state=42)

<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

</style>

	VOLUME_ID	FILE_NAME_SPECIFICATION	INSTRUMENT_HOST_ID	INSTRUMENT_ID	OBSERVATION_ID	PRODUCT_ID	PRODUCT_VERSION_ID	TARGET_NAME	ORBIT_NUMBER	MISSION_PHASE_NAME	...	LINE_PROJECTION_OFFSET	SAMPLE_PROJECTION_OFFSET	CORNER1_LATITUDE	CORNER1_LONGITUDE	CORNER2_LATITUDE	CORNER2_LONGITUDE	CORNER3_LATITUDE	CORNER3_LONGITUDE	CORNER4_LATITUDE	CORNER4_LONGITUDE
144822	MROHR_0001	RDR/ESP/ORB_074100_074199/ESP_074107_1410/ESP_...	MRO	HIRISE	ESP_074107_1410	ESP_074107_1410_COLOR	1	MARS	74107	Extended Science Phase	...	-4568562.5	2009279.5	-38.7446	159.311	-38.7465	159.287	-38.614	159.269	-38.6121	159.293
22518	MROHR_0001	RDR/ESP/ORB_014000_014099/ESP_014080_1040/ESP_...	MRO	HIRISE	ESP_014080_1040	ESP_014080_1040_RED	1	MARS	14080	Extended Science Phase	...	-3193960.0	-845180.0	-76.0185	165.215	-76.0315	164.858	-75.6958	164.658	-75.6831	165.005
86634	MROHR_0001	RDR/ESP/ORB_046300_046399/ESP_046395_1730/ESP_...	MRO	HIRISE	ESP_046395_1730	ESP_046395_1730_COLOR	1	MARS	46395	Extended Science Phase	...	-1584330.0	-26573400.0	-6.9957	292.569	-6.9979	292.551	-6.6838	292.51	-6.6815	292.529
138163	MROHR_0001	RDR/ESP/ORB_069100_069199/ESP_069178_2345/ESP_...	MRO	HIRISE	ESP_069178_2345	ESP_069178_2345_RED	1	MARS	69178	Extended Science Phase	...	12762196.0	15708938.0	53.8598	76.7591	53.8474	76.602	54.0013	76.5672	54.0137	76.7249
125490	MROHR_0001	RDR/ESP/ORB_063500_063599/ESP_063504_1995/ESP_...	MRO	HIRISE	ESP_063504_1995	ESP_063504_1995_COLOR	1	MARS	63504	Extended Science Phase	...	4638178.5	12794202.0	19.4523	124.149	19.45	124.128	19.5686	124.113	19.5709	124.133

5 rows × 54 columns

Instrument tools

CTX

from planetarypy.ctx import CTX

pid = ctrindex.sample(random_state=42).squeeze().PRODUCT_ID
pid

'J07_047671_1256_XN_54S338W'

ctx = CTX(pid)
ctx

PRODUCT_ID: J07_047671_1256_XN_54S338W
URL: https://pds-imaging.jpl.nasa.gov/data/mro/mars_reconnaissance_orbiter/ctx/mrox_2757/data/J07_047671_1256_XN_54S338W.IMG
source_path: /remote/trove/geo/planet/Mars/CTX/pds/mrox_2757/J07_047671_1256_XN_54S338W.IMG
Shape: (8192, 2528)

ctx.source_path

Path('/remote/trove/geo/planet/Mars/CTX/pds/mrox_2757/J07_047671_1256_XN_54S338W.IMG')

ctx.download()

File exists. Use `overwrite=True` to download fresh.

Acknowledgements

The creation of this library was in part supported by:

• NASA Cassini UVIS mission
• NASA SSW grant NNX15AH36G
• NASA PDART grant 80NSSC20K0875
• NASA SSW grant 80NSSC20K0748
• German Space Agency (DLR Bonn), grant 50 OO 2204, on behalf of the German Federal Ministry for Economic Affairs and Climate Action.
• This research has made use of the USGS Integrated Software for Imagers and Spectrometers (ISIS)(Laura et al. 2022).

Laura, Jason, Alex Acosta, Travis Addair, Lauren Adoram-Kershner, James Alexander, Oleg Alexandrov, Stacey Alley, et al. 2022. Integrated Software for Imagers and Spectrometers (version 7.2.0_RC1). Zenodo. https://doi.org/10.5281/zenodo.7443567.