Panic on small *.parquet file

[krumpira]

Hello kind people of Parseable :)
I have encountered an error on version 1.3.0 of Parseable, similar to #830 which you already fixed in v1.3.0.
This time it was a small (<10B) parquet file.

Error and stack backtrace:

thread '<unnamed>' panicked at server/src/storage/object_storage.rs:490:85:
called `Result::unwrap()` on an `Err` value: Parquet error: Invalid Parquet file. Size is smaller than footer

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
   1: parseable::catalog::manifest::create_from_parquet_file
   2: parseable::storage::object_storage::ObjectStorage::sync::{{closure}}
   3: parseable::sync::object_store_sync::{{closure}}::{{closure}}::{{closure}}::{{closure}}::{{closure}}
   4: <clokwerk::async_scheduler::AsyncSchedulerFuture as core::future::future::Future>::poll
   5: <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll
   6: tokio::runtime::runtime::Runtime::block_on
   7: std::panicking::try
   8: std::sys_common::backtrace::__rust_begin_short_backtrace
   9: core::ops::function::FnOnce::call_once{{vtable.shim}}
  10: std::sys::pal::unix::thread::Thread::new::thread_start
  11: <unknown>
  12: clone
stack backtrace:
   0:     0x563989d72fd6 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h9cca0343d66d16a8
   1:     0x563989da0c40 - core::fmt::write::h4311bce0ee536615
   2:     0x563989d6fa7f - std::io::Write::write_fmt::h0685c51539d0a0cd
   3:     0x563989d72db4 - std::sys_common::backtrace::print::h2fb8f70628a241ed
   4:     0x563989d74637 - std::panicking::default_hook::{{closure}}::h05093fe2e3ef454d
   5:     0x563989d74399 - std::panicking::default_hook::h5ac38aa38e0086d2
   6:     0x563989d74ac8 - std::panicking::rust_panic_with_hook::hed79743dc8b4b969
   7:     0x563989d749a2 - std::panicking::begin_panic_handler::{{closure}}::ha437b5d58f431abf
   8:     0x563989d734d6 - std::sys_common::backtrace::__rust_end_short_backtrace::hd98e82d5b39ec859
   9:     0x563989d746f4 - rust_begin_unwind
  10:     0x563986fd3945 - core::panicking::panic_fmt::hc69c4d258fe11477
  11:     0x563986fd3e93 - core::result::unwrap_failed::hff299ec748d62aab
  12:     0x563987819a8c - parseable::storage::object_storage::ObjectStorage::sync::{{closure}}::hc3970bafa8610334
  13:     0x56398738d577 - parseable::sync::object_store_sync::{{closure}}::{{closure}}::{{closure}}::{{closure}}::{{closure}}::hb173344c807ee3d2
  14:     0x563987a9bcc5 - <clokwerk::async_scheduler::AsyncSchedulerFuture as core::future::future::Future>::poll::h93925fcc6b55d1ab
  15:     0x563987539cd3 - <tokio::task::local::RunUntil<T> as core::future::future::Future>::poll::h058dcc70a0d46f4b
  16:     0x563987356ba3 - tokio::runtime::runtime::Runtime::block_on::hc5935087007a9cff
  17:     0x563987789c75 - std::panicking::try::h41cef33c8949afdf
  18:     0x56398760ba24 - std::sys_common::backtrace::__rust_begin_short_backtrace::h7a3436071f169cf5
  19:     0x56398761020c - core::ops::function::FnOnce::call_once{{vtable.shim}}::h9ec81a60cd0ed021
  20:     0x563989d788d5 - std::sys::pal::unix::thread::Thread::new::thread_start::h40e6fd3f8ce15a14
  21:     0x7ff4d6514134 - <unknown>
  22:     0x7ff4d6593a40 - clone
  23:                0x0 - <unknown>

Issue was worked around by finding the small file and removing it:

$ find /path/to/staging/ -type f -size -10c
/path/to/staging/streamname/date=2024-07-15.hour=12.minute=40.f.q.d.n.data.bIPhuanTft8RVUa.parquet
$ rm -f /path/to/staging/streamname/date=2024-07-15.hour=12.minute=40.f.q.d.n.data.bIPhuanTft8RVUa.parquet

[nitisht]

Thanks for reporting @krumpira - we'll take a look shortly