Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DocDB] Remove unused headers in catalog_manager.h and client.h #26589

Closed
1 task done
SrivastavaAnubhav opened this issue Mar 28, 2025 · 1 comment
Closed
1 task done
Assignees
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/low Low priority

Comments

@SrivastavaAnubhav
Copy link
Contributor

SrivastavaAnubhav commented Mar 28, 2025

Jira Link: DB-15960

Description

These are some of the most heavily included files and each header within them that must be parsed greatly increases build time.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@SrivastavaAnubhav SrivastavaAnubhav added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Mar 28, 2025
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Mar 28, 2025
@SrivastavaAnubhav SrivastavaAnubhav self-assigned this Mar 28, 2025
@SrivastavaAnubhav SrivastavaAnubhav added priority/low Low priority and removed priority/medium Medium priority issue status/awaiting-triage Issue awaiting triage labels Mar 28, 2025
@SrivastavaAnubhav SrivastavaAnubhav changed the title [DocDB] Remove unused headers in catalog_manager.h, client.h, and status.h [DocDB] Remove unused headers in catalog_manager.h and client.h Mar 28, 2025
SrivastavaAnubhav added a commit that referenced this issue Mar 30, 2025
…ent.h

Summary:
These are some of the most heavily included files and each header within them that they also include must be parsed in every downstream use, which greatly increases build time.

This diff removes some unused headers. The majority of the savings comes from cleaning up dockv_fwd.h to not include dockv/value_type.h, and keeping metrics.h from being included in so many files.

This speeds up the build by 178 CPU seconds, or 17.8s on my M1 mac.
Jira: DB-15960

Test Plan:
```
./yb_build.sh daemons --no-ccache --skip-tests --clean
ClangBuildAnalyzer --all build/latest capture.out
ClangBuildAnalyzer --analyze capture.out > results.out
```

Results before: {F344021}
Results after: {F344022}

Reviewers: hsunder

Reviewed By: hsunder

Subscribers: yql, ybase

Differential Revision: https://phorge.dev.yugabyte.com/D42837
@andrei-mart
Copy link
Contributor

Build fails on Ubuntu 24.04 with GCC13 after 83dc8ed

FAILED: src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o 
/home/amartsin/code/yugabyte-db/build-support/compiler-wrappers/c++ -Dtserver_mock_EXPORTS -I/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/src -I/home/amartsin/code/yugabyte-db/src -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/usearch -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/fp16 -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/hnswlib -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/simsimd -I/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/postgres/include -I/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/postgres/include/server -isystem /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/common/include -isystem /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include -Werror -fno-strict-aliasing -Wall -msse4.2 -Winvalid-pch -pthread -DBOOST_BIND_NO_PLACEHOLDERS -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DROCKSDB_PLATFORM_POSIX -DBOOST_ERROR_CODE_HEADER_ONLY -DCDS_USE_BOOST_ATOMIC -march=ivybridge -mcx16 -DYB_COMPILER_TYPE=gcc13 -DYB_COMPILER_VERSION=13.3.0 -DSNAPPY -DLZ4 -DYB_ZLIB -mno-avx -mno-bmi -mno-bmi2 -mno-fma -D__STDC_FORMAT_MACROS -Wno-deprecated-declarations -DGFLAGS=gflags -Wnon-virtual-dtor -Werror=enum-compare -Werror=reorder -Werror=switch -Werror=return-type -Werror=non-virtual-dtor -Wno-array-bounds -Wno-stringop-overflow -Wno-stringop-overread -Wno-misleading-indentation -ggdb -mno-abm -mno-movbe -fPIC -DYB_ABSL_ENABLED -DYB_GOOGLE_TCMALLOC -DYB_TCMALLOC_ENABLED -Werror=missing-field-initializers -Wno-dangling-reference -faligned-new -std=gnu++20 -fPIC -MD -MT src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o -MF src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o.d -o src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o -c /home/amartsin/code/yugabyte-db/src/yb/tserver/tserver_xcluster_context_mock.cc

( cd "/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja" && YB_THIRDPARTY_DIR=/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13 /usr/bin/g++-13 -Dtserver_mock_EXPORTS -I/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/src -I/home/amartsin/code/yugabyte-db/src -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/usearch -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/fp16 -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/hnswlib -I/home/amartsin/code/yugabyte-db/src/inline-thirdparty/simsimd -I/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/postgres/include -I/home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/postgres/include/server -isystem /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/common/include -isystem /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include -Werror -fno-strict-aliasing -Wall -msse4.2 -Winvalid-pch -pthread -DBOOST_BIND_NO_PLACEHOLDERS -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DROCKSDB_PLATFORM_POSIX -DBOOST_ERROR_CODE_HEADER_ONLY -DCDS_USE_BOOST_ATOMIC -march=ivybridge -mcx16 -DYB_COMPILER_TYPE=gcc13 -DYB_COMPILER_VERSION=13.3.0 -DSNAPPY -DLZ4 -DYB_ZLIB -mno-avx -mno-bmi -mno-bmi2 -mno-fma -D__STDC_FORMAT_MACROS -Wno-deprecated-declarations -DGFLAGS=gflags -Wnon-virtual-dtor -Werror=enum-compare -Werror=reorder -Werror=switch -Werror=return-type -Werror=non-virtual-dtor -Wno-array-bounds -Wno-stringop-overflow -Wno-stringop-overread -Wno-misleading-indentation -ggdb -mno-abm -mno-movbe -fPIC -DYB_ABSL_ENABLED -DYB_GOOGLE_TCMALLOC -DYB_TCMALLOC_ENABLED -Werror=missing-field-initializers -Wno-dangling-reference -faligned-new -std=gnu++20 -fPIC -MD -MT src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o -MF src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o.d -o src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o -c /home/amartsin/code/yugabyte-db/src/yb/tserver/tserver_xcluster_context_mock.cc )  # Compiler exit code: 1.

/-------------------------------------------------------------------------------
| COMPILATION FAILED
|-------------------------------------------------------------------------------
In file included from /usr/include/c++/13/bits/stl_pair.h:60,
                 from /usr/include/c++/13/bits/stl_algobase.h:64,
                 from /usr/include/c++/13/algorithm:60,
                 from /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-actions.h:137,
                 from /opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock.h:56,
                 from ../../src/yb/tserver/tserver_xcluster_context_mock.h:16,
                 from ../../src/yb/tserver/tserver_xcluster_context_mock.cc:14:
/usr/include/c++/13/type_traits: In instantiation of ‘constexpr const bool std::is_convertible_v<yb::YsqlFullTableName, std::optional<yb::HybridTime> >’:
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:212:33:   required by substitution of ‘template<class T, class, class> static void testing::internal::internal_stream_operator_without_lexical_name_lookup::StreamPrinter::PrintValue(const T&, std::ostream*) [with T = yb::YsqlFullTableName; <template-parameter-1-2> = void; <template-parameter-1-3> = <missing>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:290:36:   required by substitution of ‘template<class T, class Printer, class ... Printers> struct testing::internal::FindFirstPrinter<T, decltype (Printer::PrintValue(declval<const T&>(), nullptr)), Printer, Printers ...> [with T = yb::YsqlFullTableName; Printer = testing::internal::internal_stream_operator_without_lexical_name_lookup::StreamPrinter; Printers = {testing::internal::ProtobufPrinter, testing::internal::ConvertibleToIntegerPrinter, testing::internal::ConvertibleToStringViewPrinter, testing::internal::RawBytesPrinter, testing::internal::FallbackPrinter}]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:286:8:   [ skipping 5 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:870:19:   required from ‘static void testing::internal::UniversalPrinter<T&>::Print(const T&, std::ostream*) [with T = const yb::YsqlFullTableName; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:658:71:   required from ‘void testing::internal::PrintTupleTo(const T&, std::integral_constant<long unsigned int, _Size>, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&>; long unsigned int I = 1; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:665:15:   required from ‘void testing::internal::PrintTo(const std::tuple<_UTypes ...>&, std::ostream*) [with Types = {const yb::YsqlFullTableName&}; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:702:12:   required from ‘static void testing::internal::UniversalPrinter<T>::Print(const T&, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&>; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:999:30:   required from ‘void testing::internal::UniversalPrint(const T&, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&>; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-spec-builders.h:1558:19:   required from ‘void testing::internal::FunctionMocker<R(Args ...)>::UntypedDescribeUninterestingCall(const void*, std::ostream*) const [with R = void; Args = {const yb::YsqlFullTableName&}; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-spec-builders.h:1762:11:   required from ‘R testing::internal::FunctionMocker<R(Args ...)>::InvokeWith(ArgumentTuple&&) [with R = void; Args = {const yb::YsqlFullTableName&}; ArgumentTuple = std::tuple<const yb::YsqlFullTableName&>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-spec-builders.h:1482:12:   required from ‘testing::internal::FunctionMocker<R(Args ...)>::Result testing::internal::FunctionMocker<R(Args ...)>::Invoke(Args ...) [with R = void; Args = {const yb::YsqlFullTableName&}; Result = void]’
../../src/yb/tserver/tserver_xcluster_context_mock.h:55:3:   required from here
/usr/include/c++/13/type_traits:3364:44: error: invalid use of incomplete type ‘struct yb::YsqlFullTableName’ [-fpermissive]
 3364 |   inline constexpr bool is_convertible_v = __is_convertible(_From, _To);
      |                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../src/yb/tserver/tserver_xcluster_context_mock.h:22:
../../src/yb/tserver/tserver_xcluster_context_if.h:25:8: note: forward declaration of ‘struct yb::YsqlFullTableName’
   25 | struct YsqlFullTableName;
      |        ^~~~~~~~~~~~~~~~~
/usr/include/c++/13/type_traits: In instantiation of ‘constexpr const bool std::is_convertible_v<yb::PgObjectId, std::optional<yb::HybridTime> >’:
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:212:33:   required by substitution of ‘template<class T, class, class> static void testing::internal::internal_stream_operator_without_lexical_name_lookup::StreamPrinter::PrintValue(const T&, std::ostream*) [with T = yb::PgObjectId; <template-parameter-1-2> = void; <template-parameter-1-3> = <missing>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:290:36:   required by substitution of ‘template<class T, class Printer, class ... Printers> struct testing::internal::FindFirstPrinter<T, decltype (Printer::PrintValue(declval<const T&>(), nullptr)), Printer, Printers ...> [with T = yb::PgObjectId; Printer = testing::internal::internal_stream_operator_without_lexical_name_lookup::StreamPrinter; Printers = {testing::internal::ProtobufPrinter, testing::internal::ConvertibleToIntegerPrinter, testing::internal::ConvertibleToStringViewPrinter, testing::internal::RawBytesPrinter, testing::internal::FallbackPrinter}]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:286:8:   [ skipping 6 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:652:15:   recursively required from ‘void testing::internal::PrintTupleTo(const T&, std::integral_constant<long unsigned int, _Size>, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&>; long unsigned int I = 3; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:652:15:   required from ‘void testing::internal::PrintTupleTo(const T&, std::integral_constant<long unsigned int, _Size>, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&>; long unsigned int I = 4; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:665:15:   required from ‘void testing::internal::PrintTo(const std::tuple<_UTypes ...>&, std::ostream*) [with Types = {const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&}; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:702:12:   required from ‘static void testing::internal::UniversalPrinter<T>::Print(const T&, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&>; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gtest/gtest-printers.h:999:30:   required from ‘void testing::internal::UniversalPrint(const T&, std::ostream*) [with T = std::tuple<const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&>; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-spec-builders.h:1558:19:   required from ‘void testing::internal::FunctionMocker<R(Args ...)>::UntypedDescribeUninterestingCall(const void*, std::ostream*) const [with R = yb::Status; Args = {const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&}; std::ostream = std::basic_ostream<char>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-spec-builders.h:1762:11:   required from ‘R testing::internal::FunctionMocker<R(Args ...)>::InvokeWith(ArgumentTuple&&) [with R = yb::Status; Args = {const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&}; ArgumentTuple = std::tuple<const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&>]’
/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20250220003605-3c4d4d8481-ubuntu2404-x86_64-gcc13/installed/uninstrumented/include/gmock/gmock-spec-builders.h:1482:12:   required from ‘testing::internal::FunctionMocker<R(Args ...)>::Result testing::internal::FunctionMocker<R(Args ...)>::Invoke(Args ...) [with R = yb::Status; Args = {const yb::YsqlFullTableName&, const yb::PgObjectId&, unsigned int, const yb::HybridTime&}; Result = yb::Status]’
../../src/yb/tserver/tserver_xcluster_context_mock.h:50:3:   required from here
/usr/include/c++/13/type_traits:3364:44: error: invalid use of incomplete type ‘struct yb::PgObjectId’ [-fpermissive]
 3364 |   inline constexpr bool is_convertible_v = __is_convertible(_From, _To);
      |                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../src/yb/tserver/tserver_xcluster_context_if.h:18:
../../src/yb/common/common_fwd.h:51:8: note: forward declaration of ‘struct yb::PgObjectId’
   51 | struct PgObjectId;
      |        ^~~~~~~~~~

Input files:
  /home/amartsin/code/yugabyte-db/build/debug-gcc13-dynamic-ninja/src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o
  /home/amartsin/code/yugabyte-db/src/yb/tserver/tserver_xcluster_context_mock.cc
Output file (from -o): src/yb/tserver/CMakeFiles/tserver_mock.dir/tserver_xcluster_context_mock.cc.o
\-------------------------------------------------------------------------------

Builds OK after git revert 83dc8ed67d8880e0b6f570bd4f185d40b29638cd.
It complains about incomplete type. My guess it can't find the type definion, after some "unused" headers were removed.

@andrei-mart andrei-mart reopened this Mar 31, 2025
SrivastavaAnubhav added a commit that referenced this issue Apr 1, 2025
Summary:
Commit 83dc8ed removes some unused headers. `tserver_xcluster_context_mock.h` actually used one of these headers on gcc13 (even though it probably shouldn't and a forward declaration should be enough, but we're using an extremely old gmock framework). This diff adds the header to that file to fix the build.
Jira: DB-15960

Test Plan:
./yb_build.sh --gcc13
Jenkins: compile only

Reviewers: amartsinchyk

Reviewed By: amartsinchyk

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D42894
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/low Low priority
Projects
None yet
Development

No branches or pull requests

3 participants