- 
                Notifications
    You must be signed in to change notification settings 
- Fork 29
Open
Description
In #237 we updated to DataFusion 50, which includes some new features for caching metadata. In our GeoParquet reader we do one or two extra metadata fetches that would benefit greatly from using this! Some of the code we had to copy over doesn't pipe through the global cache.
Upstream:
DFParquetMetadata::new(store.as_ref(), object)
                    .with_metadata_size_hint(self.metadata_size_hint())
                    .with_decryption_properties(file_decryption_properties.as_ref())
                    .with_file_metadata_cache(Some(Arc::clone(&file_metadata_cache)))
                    .with_coerce_int96(coerce_int96)
                    .fetch_schema_with_location()
                    .await?;Our code:
sedona-db/rust/sedona-geoparquet/src/format.rs
Lines 187 to 190 in c67484c
| DFParquetMetadata::new(store.as_ref(), object) | |
| .with_metadata_size_hint(self.inner().metadata_size_hint()) | |
| .fetch_metadata() | |
| .await | 
Metadata
Metadata
Assignees
Labels
No labels