You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am working on POC and try to process ROS generated data which is converted then to parquet in GCS, this .NET package is really cool, amazing performance. While it works for some topics (without nested repeated structs), it is failing on large complec data structures with multi level lists/structs.
I found multiple error types on deserializing to C# classes, one is the same that was closed last year - destination is too short. Another one is ArgumentOutOfRange.
I prepared a small test on which can be reproduced the second error type, another one is more difficult to reproduce, will try to prepare another test dataset and submit another ticket.
Here is the example file with truncated schema to just a few fields but the ArgumentOutOfRange can be reproduced: 000000000000.parquet.zip
Error:
Failing test
Classto deserialize to:publicclassHeaderStamp{publicInt64?secs{get;set;}publicInt64?nsecs{get;set;}}publicclassHeader{publicInt64?seq{get;set;}publicHeaderStampstamp{get;set;}publicStringframe_id{get;set;}}publicclassTrackedObjectsTestListElement{publiclong?track_id{get;set;}publicDouble?existence_probability{get;set;}publicBoolean?moving{get;set;}}publicclassTrackedObjectsTest{publicHeaderheader{get;set;}publicList<TrackedObjectsTestListElement> tracked_objects {get;set;}publicString_launch_id{get;set;}}
And deserialize command:varr=await ParquetSerializer.DeserializeAsync<TrackedObjectsTest>("000000000000.parquet");
The text was updated successfully, but these errors were encountered:
Library Version
4.23.5
OS
MacOS
OS Architecture
64 bit
How to reproduce?
Hello,
I am working on POC and try to process ROS generated data which is converted then to parquet in GCS, this .NET package is really cool, amazing performance. While it works for some topics (without nested repeated structs), it is failing on large complec data structures with multi level lists/structs.
I found multiple error types on deserializing to C# classes, one is the same that was closed last year - destination is too short. Another one is ArgumentOutOfRange.
I prepared a small test on which can be reproduced the second error type, another one is more difficult to reproduce, will try to prepare another test dataset and submit another ticket.
Here is the example file with truncated schema to just a few fields but the ArgumentOutOfRange can be reproduced:
000000000000.parquet.zip
Error:
Failing test
The text was updated successfully, but these errors were encountered: