Failing test on Arch Linux

[lahwaacz]

The following test is failing when I build orc 2.1.2 on Arch Linux

[----------] 5 tests from TestFileMetadata
[ RUN      ] TestFileMetadata.testRaw
/usr/src/debug/apache-orc/apache-orc/tools/test/TestFileMetadata.cc:153: Failure
Expected equality of these values:
  expected
    Which is: "Raw file tail: /build/apache-orc/src/apache-orc/examples/orc_split_elim.orc\npostscript {\n  footer_length: 288\n  compression: NONE\n  version: 0\n  version: 12\n  metadata_length: 526\n  magic: \"ORC\"\n}\nfooter {\n  header_length: 3\n  content_length: 245568\n  stripes {\n    offset: 3\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 45571\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 91139\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 136707\n    index_length: 138\n    data_length: 45283\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 200000\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  types {\n    kind: STRUCT\n    subtypes: 1\n    subtypes: 2\n    subtypes: 3\n    subtypes: 4\n    subtypes: 5\n    field_names: \"userid\"\n    field_names: \"string1\"\n    field_names: \"subtype\"\n    field_names: \"decimal1\"\n    field_names: \"ts\"\n  }\n  types {\n    kind: LONG\n  }\n  types {\n    kind: STRING\n  }\n  types {\n    kind: DOUBLE\n  }\n  types {\n    kind: DECIMAL\n  }\n  types {\n    kind: TIMESTAMP\n  }\n  number_of_rows: 25000\n  statistics {\n    number_of_values: 25000\n  }\n  statistics {\n    number_of_values: 25000\n    int_statistics {\n      minimum: 2\n      maximum: 100\n      sum: 2499619\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    string_statistics {\n      minimum: \"bar\"\n      maximum: \"zebra\"\n      sum: 124990\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    double_statistics {\n      minimum: 0.8\n      maximum: 80\n      sum: 200051.40000000002\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    decimal_statistics {\n      minimum: \"0\"\n      maximum: \"5.5\"\n      sum: \"16.6\"\n    }\n  }\n  statistics {\n    number_of_values: 25000\n  }\n  row_index_stride: 10000\n}\nfile_length: 246402\npostscript_length: 19\n"
  output
    Which is: "Raw file tail: /build/apache-orc/src/apache-orc/examples/orc_split_elim.orc\ngoo.gle/debugstr    \npostscript {\n  footer_length: 288\n  compression: NONE\n  version: 0\n  version: 12\n  metadata_length: 526\n  magic: \"ORC\"\n}\nfooter {\n  header_length: 3\n  content_length: 245568\n  stripes {\n    offset: 3\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 45571\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 91139\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 136707\n    index_length: 138\n    data_length: 45283\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 200000\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  types {\n    kind: STRUCT\n    subtypes: 1\n    subtypes: 2\n    subtypes: 3\n    subtypes: 4\n    subtypes: 5\n    field_names: \"userid\"\n    field_names: \"string1\"\n    field_names: \"subtype\"\n    field_names: \"decimal1\"\n    field_names: \"ts\"\n  }\n  types {\n    kind: LONG\n  }\n  types {\n    kind: STRING\n  }\n  types {\n    kind: DOUBLE\n  }\n  types {\n    kind: DECIMAL\n  }\n  types {\n    kind: TIMESTAMP\n  }\n  number_of_rows: 25000\n  statistics {\n    number_of_values: 25000\n  }\n  statistics {\n    number_of_values: 25000\n    int_statistics {\n      minimum: 2\n      maximum: 100\n      sum: 2499619\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    string_statistics {\n      minimum: \"bar\"\n      maximum: \"zebra\"\n      sum: 124990\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    double_statistics {\n      minimum: 0.8\n      maximum: 80\n      sum: 200051.40000000002\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    decimal_statistics {\n      minimum: \"0\"\n      maximum: \"5.5\"\n      sum: \"16.6\"\n    }\n  }\n  statistics {\n    number_of_values: 25000\n  }\n  row_index_stride: 10000\n}\nfile_length: 246402\npostscript_length: 19\n"
With diff:
@@ +1,4 @@
 Raw file tail: /build/apache-orc/src/apache-orc/examples/orc_split_elim.orc
+goo.gle/debugstr
 postscript {
   footer_length: 288


/usr/src/debug/apache-orc/apache-orc/tools/test/TestFileMetadata.cc:157: Failure
Expected equality of these values:
  expected
    Which is: "Raw file tail: /build/apache-orc/src/apache-orc/examples/orc_split_elim.orc\npostscript {\n  footer_length: 288\n  compression: NONE\n  version: 0\n  version: 12\n  metadata_length: 526\n  magic: \"ORC\"\n}\nfooter {\n  header_length: 3\n  content_length: 245568\n  stripes {\n    offset: 3\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 45571\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 91139\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 136707\n    index_length: 138\n    data_length: 45283\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 200000\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  types {\n    kind: STRUCT\n    subtypes: 1\n    subtypes: 2\n    subtypes: 3\n    subtypes: 4\n    subtypes: 5\n    field_names: \"userid\"\n    field_names: \"string1\"\n    field_names: \"subtype\"\n    field_names: \"decimal1\"\n    field_names: \"ts\"\n  }\n  types {\n    kind: LONG\n  }\n  types {\n    kind: STRING\n  }\n  types {\n    kind: DOUBLE\n  }\n  types {\n    kind: DECIMAL\n  }\n  types {\n    kind: TIMESTAMP\n  }\n  number_of_rows: 25000\n  statistics {\n    number_of_values: 25000\n  }\n  statistics {\n    number_of_values: 25000\n    int_statistics {\n      minimum: 2\n      maximum: 100\n      sum: 2499619\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    string_statistics {\n      minimum: \"bar\"\n      maximum: \"zebra\"\n      sum: 124990\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    double_statistics {\n      minimum: 0.8\n      maximum: 80\n      sum: 200051.40000000002\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    decimal_statistics {\n      minimum: \"0\"\n      maximum: \"5.5\"\n      sum: \"16.6\"\n    }\n  }\n  statistics {\n    number_of_values: 25000\n  }\n  row_index_stride: 10000\n}\nfile_length: 246402\npostscript_length: 19\n"
  output
    Which is: "Raw file tail: /build/apache-orc/src/apache-orc/examples/orc_split_elim.orc\ngoo.gle/debugonly    \npostscript {\n  footer_length: 288\n  compression: NONE\n  version: 0\n  version: 12\n  metadata_length: 526\n  magic: \"ORC\"\n}\nfooter {\n  header_length: 3\n  content_length: 245568\n  stripes {\n    offset: 3\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 45571\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 91139\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 136707\n    index_length: 138\n    data_length: 45283\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  stripes {\n    offset: 200000\n    index_length: 137\n    data_length: 45282\n    footer_length: 149\n    number_of_rows: 5000\n  }\n  types {\n    kind: STRUCT\n    subtypes: 1\n    subtypes: 2\n    subtypes: 3\n    subtypes: 4\n    subtypes: 5\n    field_names: \"userid\"\n    field_names: \"string1\"\n    field_names: \"subtype\"\n    field_names: \"decimal1\"\n    field_names: \"ts\"\n  }\n  types {\n    kind: LONG\n  }\n  types {\n    kind: STRING\n  }\n  types {\n    kind: DOUBLE\n  }\n  types {\n    kind: DECIMAL\n  }\n  types {\n    kind: TIMESTAMP\n  }\n  number_of_rows: 25000\n  statistics {\n    number_of_values: 25000\n  }\n  statistics {\n    number_of_values: 25000\n    int_statistics {\n      minimum: 2\n      maximum: 100\n      sum: 2499619\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    string_statistics {\n      minimum: \"bar\"\n      maximum: \"zebra\"\n      sum: 124990\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    double_statistics {\n      minimum: 0.8\n      maximum: 80\n      sum: 200051.40000000002\n    }\n  }\n  statistics {\n    number_of_values: 25000\n    decimal_statistics {\n      minimum: \"0\"\n      maximum: \"5.5\"\n      sum: \"16.6\"\n    }\n  }\n  statistics {\n    number_of_values: 25000\n  }\n  row_index_stride: 10000\n}\nfile_length: 246402\npostscript_length: 19\n"
With diff:
@@ +1,4 @@
 Raw file tail: /build/apache-orc/src/apache-orc/examples/orc_split_elim.orc
+goo.gle/debugonly
 postscript {
   footer_length: 288


[  FAILED  ] TestFileMetadata.testRaw (13 ms)

Any idea what is wrong?

[dongjoon-hyun]

Thank you for reporting. At the first glance, goo.gle/debugstr looks like the root cause but it's not injected by Apache ORC source code. Maybe, please check your local ly installed debug library.

[ffacs]

https://protobuf.dev/programming-guides/deserialize-debug/ There seems to be a breaking change in protobuf’s debug string output in version 30.x.

[dongjoon-hyun]

Thank you, @ffacs .

If the error message comes from non-vanilla Apache ORC source code, there is not much thing we can discuss in Apache ORC repo. Let me close for now.

To @lahwaacz , feel free to re-open with the reproducible procedure. For example, what you did after git clone.

[lahwaacz]

@dongjoon-hyun I don't know what you mean by "non-vanilla Apache ORC source code" since the issue, as @ffacs pointed out, is due to breaking changes in protobuf.

Anyway, to reproduce this, feel free to run the following reproducer in an Arch Linux Docker/Podman container, which currently has protobuf 31.

Save the following script as repro.sh (the procedure is based on the apache-orc PKGBUILD):

#!/bin/bash

# Exit as soon as some command fails
set -e

# Install Arch Linux dependencies
pacman -Suy --noconfirm abseil-cpp lz4 protobuf snappy zlib zstd cmake git

# Clone apache-orc
git clone --revision v2.1.2 https://github.com/apache/orc.git
cd orc

# Fix build with protobuf 31 (the google::protobuf::int64 type is not found without this)
echo '#include <google/protobuf/stubs/common.h>' >> c++/src/wrap/coded-stream-wrapper.h
echo '#include <google/protobuf/stubs/common.h>' >> c++/src/wrap/zero-copy-stream-wrapper.h

# Fix for missing /etc/localtime https://github.com/apache/orc/issues/2225
cat > fix-timezone.patch << EOF
--- a/c++/src/Timezone.cc             2025-03-12 12:09:01.267309399 +0000
+++ b/c++/src/Timezone.cc             2025-03-12 12:12:12.303295722 +0000
@@ -28,6 +28,12 @@
 #include <map>
 #include <sstream>
 
+#ifndef _MSC_VER
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+#endif
+
 namespace orc {
 
   // default location of the timezone files
@@ -694,6 +700,14 @@
 #ifdef _MSC_VER
     return getTimezoneByName("UTC");
 #else
+    // The absence of LOCAL_TIMEZONE means UTC conventionally
+    {
+      struct stat _ignored;
+      if (stat(LOCAL_TIMEZONE, &_ignored) == -1) {
+        return getTimezoneByName("UTC");
+      }
+    }
+
     return getTimezoneByFilename(LOCAL_TIMEZONE);
 #endif
   }
EOF
patch -p1 -i fix-timezone.patch

# Fix the cmake modules https://github.com/apache/orc/pull/2240
cat > fix-cmake-modules.patch << EOF
From 3c99a2e44e022ab171d258f9cb6876302fd6bb7f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jakub=20Klinkovsk=C3=BD?=
 <1289205+lahwaacz@users.noreply.github.com>
Date: Sat, 24 May 2025 16:02:15 +0200
Subject: [PATCH] Fix Find* CMake modules

- allow finding header files in default paths - this is consistent with
  the logic for finding the library files
- replace undefined variables \${SNAPPY_LIB_NAME} and \${LZ4_LIB_NAME}
  with the actual names
---
 cmake_modules/FindGTest.cmake    | 1 -
 cmake_modules/FindLZ4.cmake      | 3 +--
 cmake_modules/FindProtobuf.cmake | 3 ---
 cmake_modules/FindSnappy.cmake   | 3 +--
 cmake_modules/FindZLIB.cmake     | 1 -
 cmake_modules/FindZSTD.cmake     | 1 -
 6 files changed, 2 insertions(+), 10 deletions(-)

diff --git a/cmake_modules/FindGTest.cmake b/cmake_modules/FindGTest.cmake
index ee40378a..7398ed45 100644
--- a/cmake_modules/FindGTest.cmake
+++ b/cmake_modules/FindGTest.cmake
@@ -32,7 +32,6 @@ file (TO_CMAKE_PATH "\${GTEST_HOME}" _gtest_path )
 
 find_path (GTEST_INCLUDE_DIR gmock/gmock.h HINTS
   \${_gtest_path}
-  NO_DEFAULT_PATH
   PATH_SUFFIXES "include")
 
 find_library (GMOCK_LIBRARY NAMES gmock HINTS
diff --git a/cmake_modules/FindLZ4.cmake b/cmake_modules/FindLZ4.cmake
index 3b9cc7fb..09db140b 100644
--- a/cmake_modules/FindLZ4.cmake
+++ b/cmake_modules/FindLZ4.cmake
@@ -40,14 +40,13 @@ message (STATUS "LZ4_HOME: \${LZ4_HOME}")
 
 find_path (LZ4_INCLUDE_DIR lz4.h HINTS
   \${_lz4_path}
-  NO_DEFAULT_PATH
   PATH_SUFFIXES "include")
 
 find_library (LZ4_LIBRARY NAMES lz4 liblz4 HINTS
   \${_lz4_path}
   PATH_SUFFIXES "lib" "lib64")
 
-find_library (LZ4_STATIC_LIB NAMES \${CMAKE_STATIC_LIBRARY_PREFIX}\${LZ4_LIB_NAME}\${CMAKE_STATIC_LIBRARY_SUFFIX} HINTS
+find_library (LZ4_STATIC_LIB NAMES \${CMAKE_STATIC_LIBRARY_PREFIX}lz4\${CMAKE_STATIC_LIBRARY_SUFFIX} HINTS
   \${_lz4_path}
   PATH_SUFFIXES "lib" "lib64")
 
diff --git a/cmake_modules/FindProtobuf.cmake b/cmake_modules/FindProtobuf.cmake
index ca91fb5a..2df328ec 100644
--- a/cmake_modules/FindProtobuf.cmake
+++ b/cmake_modules/FindProtobuf.cmake
@@ -96,12 +96,10 @@ if (Protobuf_FOUND)
 else()
     find_path (PROTOBUF_INCLUDE_DIR google/protobuf/io/zero_copy_stream.h HINTS
       \${_protobuf_path}
-      NO_DEFAULT_PATH
       PATH_SUFFIXES "include")
 
     find_path (PROTOBUF_INCLUDE_DIR google/protobuf/io/coded_stream.h HINTS
       \${_protobuf_path}
-      NO_DEFAULT_PATH
       PATH_SUFFIXES "include")
 
     find_library (PROTOBUF_LIBRARY NAMES protobuf libprotobuf HINTS
@@ -122,7 +120,6 @@ else()
 
     find_program(PROTOBUF_EXECUTABLE protoc HINTS
       \${_protobuf_path}
-      NO_DEFAULT_PATH
       PATH_SUFFIXES "bin")
 endif ()
 
diff --git a/cmake_modules/FindSnappy.cmake b/cmake_modules/FindSnappy.cmake
index 1ad99145..db7b06d7 100644
--- a/cmake_modules/FindSnappy.cmake
+++ b/cmake_modules/FindSnappy.cmake
@@ -44,14 +44,13 @@ message (STATUS "SNAPPY_HOME: \${SNAPPY_HOME}")
 
 find_path (SNAPPY_INCLUDE_DIR snappy.h HINTS
   \${_snappy_path}
-  NO_DEFAULT_PATH
   PATH_SUFFIXES "include")
 
 find_library (SNAPPY_LIBRARY NAMES snappy HINTS
   \${_snappy_path}
   PATH_SUFFIXES "lib" "lib64")
 
-find_library (SNAPPY_STATIC_LIB NAMES \${CMAKE_STATIC_LIBRARY_PREFIX}\${SNAPPY_LIB_NAME}\${CMAKE_STATIC_LIBRARY_SUFFIX} HINTS
+find_library (SNAPPY_STATIC_LIB NAMES \${CMAKE_STATIC_LIBRARY_PREFIX}snappy\${CMAKE_STATIC_LIBRARY_SUFFIX} HINTS
   \${_snappy_path}
   PATH_SUFFIXES "lib" "lib64")
 
diff --git a/cmake_modules/FindZLIB.cmake b/cmake_modules/FindZLIB.cmake
index 374814a7..f0654ffa 100644
--- a/cmake_modules/FindZLIB.cmake
+++ b/cmake_modules/FindZLIB.cmake
@@ -40,7 +40,6 @@ message (STATUS "ZLIB_HOME: \${ZLIB_HOME}")
 
 find_path (ZLIB_INCLUDE_DIR zlib.h HINTS
   \${_zlib_path}
-  NO_DEFAULT_PATH
   PATH_SUFFIXES "include")
 
 if (NOT ZLIB_STATIC_LIB_NAME)
diff --git a/cmake_modules/FindZSTD.cmake b/cmake_modules/FindZSTD.cmake
index 58171945..0af87e19 100644
--- a/cmake_modules/FindZSTD.cmake
+++ b/cmake_modules/FindZSTD.cmake
@@ -40,7 +40,6 @@ message (STATUS "ZSTD_HOME: \${ZSTD_HOME}")
 
 find_path (ZSTD_INCLUDE_DIR zstd.h HINTS
         \${_zstd_path}
-        NO_DEFAULT_PATH
         PATH_SUFFIXES "include")
 
 find_library (ZSTD_LIBRARY NAMES zstd HINTS
-- 
2.49.0
EOF
patch -p1 -i fix-cmake-modules.patch

# Build the orc
cmake_options=(
  -B build
  -S .
  -DCMAKE_BUILD_TYPE=None
  -DCMAKE_INSTALL_PREFIX=/usr
  -DSTOP_BUILD_ON_WARNING=OFF
  -DLZ4_HOME=/usr
  -DPROTOBUF_HOME=/usr
  -DSNAPPY_HOME=/usr
  -DZLIB_HOME=/usr
  -DZSTD_HOME=/usr
  -DGTEST_HOME=/usr
  -DORC_PREFER_STATIC_PROTOBUF=OFF
  -DORC_PREFER_STATIC_SNAPPY=OFF
  -DORC_PREFER_STATIC_LZ4=OFF
  -DORC_PREFER_STATIC_ZSTD=OFF
  -DORC_PREFER_STATIC_ZLIB=OFF
  -DBUILD_LIBHDFSPP=OFF
  -DBUILD_JAVA=OFF
  -DINSTALL_VENDORED_LIBS=OFF
  -Wno-dev
)
cmake "${cmake_options[@]}"
cmake --build build --parallel

# Run the test
ctest --test-dir build --output-on-failure

Run it as podman run -it --rm --userns nomap -v ./repro.sh:/repro.sh quay.io/archlinux/archlinux:base-devel /repro.sh

We need a few patches to successfully build apache-orc on Arch Linux, but those are separate issues: #2225 and #2240

[dongjoon-hyun]

To @lahwaacz , what I meant is any changes after cloning the git repo. It's not Apache ORC code any longer.

In the community, a test failure is used as a completely different term from a compilation failure .

In this issue, you reported with

• Title: Failing test on Arch Linux
• Description: The following test is failing when I build orc 2.1.2 on Arch Linux

Technically, it's an invalid claim when a compilation fails from the beginning.

[dongjoon-hyun]

To @lahwaacz , I got a hint from your comment.

Anyway, to reproduce this, feel free to run the following reproducer in an Arch Linux Docker/Podman container, which currently has protobuf 31.

Recently, I revisit this issue and it turns out that the root cause is the breaking change of ProtocolBuffer at v30.0. They injected that debug markers intentionally.

• https://github.com/protocolbuffers/protobuf/releases/tag/v30.0

Make DebugString print debug output, enable debug markers for debug output

Anyway, I want to inform you that I made a PR for your issue. Hopefully, it can help you with Apache ORC 2.2.1.

• ORC-1974: [C++] Use google::protobuf::TextFormat instead of DebugString for Protobuf v30+ #2358