CCSDS OEM conversion

[ChristopherRabotin]

High level description

CCSDS OEM is a standard for transferring orbital element data. As such, ANISE shall support reading these files and producing them from an ephemeris.

The API should also allow for converting an ephemeris into another frame prior to the serialization of the data in OEM format.

Requirements

1. Convert the OEM state data into an SPK with relevant metadata where possible.
2. Copy would-be discarded data into DAF comment section

Test plans

1. Generate a CCSDS OEM file from a public tool, or find one online, and convert that into an SPK file. Then make sure that when queried at the epochs present in the OEM file, the state data from the SPK matches that of the OEM file.
2. Test that optional fields do not cause the parser to fail.
3. Test that incorrectly formatted fields do not cause the parser to fail.

Design

Algorithm demonstration

N/A

API definition

1. A new module called ccsds will be added along with an oem submodule. All of the OEM parsing logic will be there.
2. The ccsds module will only be available with the ccsds crate feature, enabled by default, but not supported in no_std environments.

High level architecture

The text version of the OEMs will likely be parsed using an abstract syntax tree. Although, it is to be see whether that is an overkill since the text file is relatively simple. The XML version, if it is to be supported at all, will use an standard XML parser.

Ref. 1: https://createlang.rs/01_calculator/ast.html
Ref. 2: https://doc.rust-lang.org/stable/nightly-rustc/rustc_ast/ast/index.html

Detailed design

The detailed design *will be used in the documentation of how ANISE works.

Feel free to fill out additional QA sections here, but these will typically be determined during the development, including the release in which this issue will be tackled.

[ascended121]

I'm happy to take a look at this issue, and have been playing around and have some code to parse/write OEM files. I located a version 1.0 OEM file (from the Herschel mission) and a version 2.0 OEM file (from the ISS) to use as test data and have both of those parsing. Still need to find an example of a version 3.0 file to test with.

Also wrote some code to write those structs back into an OEM file and an able to round-trip those files (save for some blank lines which didn't seem important to preserve).

I'm finding it a little trickier to convert the data to an SPK. I've implemented a function to convert the OEM into an SPK (excerpt below):

impl OEM {
    pub fn to_spk(&self) -> Result<MutSPK, OemError>{
        let mut spk = MutSPK {
            bytes: BytesMut::with_capacity(RCRD_LEN),
            crc32_checksum: 0,
            _daf_type: PhantomData::<SPKSummaryRecord>,
        };
       ...
       spk.set_nth_data(
                idx,
                LagrangeSetType9::from_f64_slice(&segment_bytes))
                ...);
   }
}

however this produces the error Failed to set segment data: DAF/SPKSummaryRecord: data index 0: bytes between indexes 0 and 1024 could not be read, array contains 0 bytes (data malformed?).

It appears that set_nth_data() attempts to read the file record, which doesn't exist for this fictitious in-memory file that I'm trying to create. So I need to figure out how to create that file record. Found some docs on it and started playing with sticking some values in, but getting some out of bounds subtractions so I'm obviously doing something wrong there.

Any suggestions for proper values to put in that record for a fresh/empty file (which set_nth_data() will then update), or existing code which might help write a new SPK file (is MutDAF/set_nth_data() the right way to approach this??) would be appreciated.

[ChristopherRabotin]

Hi there! Thanks for your help on this feature.

Although ANISE does not have an OEM parser/write, Nyx currently provides both: https://github.com/nyx-space/nyx/blob/343a2424f167ac4ffa3b61574b86212e20e3f972/src/md/trajectory/sc_traj.rs#L159 ; https://github.com/nyx-space/nyx/blob/343a2424f167ac4ffa3b61574b86212e20e3f972/src/md/trajectory/sc_traj.rs#L277 . You can also find some example public OEM files here: https://github.com/nyx-space/nyx/tree/343a2424f167ac4ffa3b61574b86212e20e3f972/data/tests/ccsds/oem. I'd be interested in comparing your implementation with that in Nyx.

Both functions could be in theory copied here directly nearly as-is for parsing and writing CCSDS OEM files in version 1. To my knowledge, the only additional feature from version 2 compared to version 1 is the support for covariance information: Nyx does not support reading that covariance information. ANISE does not currently support any covariance on any type (and I think that feature should be reserved for Nyx as it's pertinent to orbit determination, one of the hallmarks of Nyx).

Since I wrote this issue over two years ago, I think the better approach to support CCSDS OEM in ANISE is to provide two helper functions:

1. Read an OEM file and convert it to BSP (in Lagrange or Hermite type): this would be a stand-alone tool similar to NAIF's oem2spk: https://naif.jpl.nasa.gov/naif/utilities_PC_Linux_64bit.html . But, as someone who automated using that tool in operations, the NAIF version is far from user-friendly, so a better tool would be great. For example, that tool could be configured either using typical run-time arguments, or by passing a config file as an input.
2. From a list of Orbit structures, build a BSP file (as you've started working on). Then, separately, build a stand-alone tool that reads an OEM file and returns a list of Orbit data structures. This would allow converting between BSP formats trivially as well since one could just pass a list of Orbit structs from a BSP query to the function that builds the BSP file.

Diving into the details ... writing DAF files is not trivial and the current support is super limited. I tried to implement some version of it in the persist function:

anise/anise/src/naif/daf/daf.rs

Line 351 in 4f56a4a

pub fn persist<P: AsRef<Path>>(&self, path: P) -> IoResult<()> {

. That works for setting and renaming the name of segments, but that's about it if I recall correctly. However, that persist function and the parse function (

anise/anise/src/naif/daf/mut_daf.rs

Lines 40 to 41 in 4f56a4a

	me.file_record()?;
	me.name_record()?;

) gives some hints as to what's needed: each DAF file needs a file record (

anise/anise/src/naif/daf/file_record.rs

Line 47 in 4f56a4a

pub struct FileRecord {

) and a name record (or a slice of zero bytes of that exact length) (

anise/anise/src/naif/daf/name_record.rs

Line 21 in 4f56a4a

raw_names: [u8; RCRD_LEN],

). These two will specify the flavor of DAF: BSP, BPC, etc. Then, the format expects a sequence of summary records and then the segment data itself.

You've found the best and only resource of the DAF format. For the SPK format, you'll also want to read through this section of the SPK specific paragraph: https://naif.jpl.nasa.gov/pub/naif/toolkit_docs/C/req/spk.html#SPK%20File%20Structure .

To address the specific error you're seeing when calling set_nth_data, I think the problem you're seeing is because the set_nth_data function assumes you're overwriting existing data, and tries to find the byte start and end to overwrite:

anise/anise/src/naif/daf/mut_daf.rs

Lines 90 to 93 in 4f56a4a

	let orig_index_start = this_summary.start_index() - 1;
	let orig_index_end = this_summary.end_index();
	let orig_data_start = orig_index_start * DBL_SIZE;
	let orig_data_end = orig_index_end * DBL_SIZE;

. But in the case of a new SPK structure, the data summaries are non existent, so the call to data_summaries (

anise/anise/src/naif/daf/mut_daf.rs

Line 78 in 4f56a4a

let summaries = self.data_summaries()?;

) fails.

ANISE has really good support for reading SPK files. So the best way to test if you're making progress is what I think you've done: write a test file and load it in ANISE.

I hope this helps, I'm happy to work on a solution together directly in this issue.