Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libc] delete hdrgen #117220

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

nickdesaulniers
Copy link
Member

@nickdesaulniers nickdesaulniers commented Nov 21, 2024

Thanks to the effort of @RoseZhang03 and @aaryanshukla under the guidance of
@michaelrj-google and @amykhuang, we now have newhdrgen and no longer have a
dependency on TableGen and thus LLVM in order to start bootstrapping a full
build.

This PR removes:

  • LIBC_HDRGEN_EXE; the in tree newhdrgen is the only hdrgen that can be used.
  • LIBC_USE_NEW_HEADER_GEN; newhdrgen is the default and only option.
  • LIBC_HDRGEN_ONLY; there is no need to have a distinct build step for old
    hdrgen.
  • libc-api-test and libc-api-test-tidy build targets.
  • Deletes all .td files.

It does not rename newhdrgen to just hdrgen. Will follow up with a distinct PR
for that.

Link: #117209
Link: #117254
Fixes: #117208

@llvmbot
Copy link
Member

llvmbot commented Nov 21, 2024

@llvm/pr-subscribers-libc

Author: Nick Desaulniers (nickdesaulniers)

Changes

Thanks to the effort of @RoseZhang03 and @aaryanshukla under the guidance of
@michaelrj-google and @amykhuang, we now have newhdrgen and no longer have a
dependency on TableGen and thus LLVM in order to start bootstrapping a full
build.

This PR:

  • Removes being able to set LIBC_HDRGEN_EXE.
  • LIBC_HDRGEN_EXE; the in tree newhdrgen is the only hdrgen that can be used.
  • LIBC_USE_NEW_HEADER_GEN; newhdrgen is the default and only option.
  • LIBC_HDRGEN_ONLY; there is no need to have a distinct build step for old
    hdrgen.
  • libc-api-test and libc-api-test-tidy build targets.
  • Deletes all .td files.

It does not rename newhdrgen to just hdrgen. Will follow up with a distinct PR
for that.

Link: #117209
Fixes: #117208


Patch is 243.67 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/117220.diff

36 Files Affected:

  • (modified) libc/CMakeLists.txt (+1-17)
  • (modified) libc/cmake/modules/LLVMLibCHeaderRules.cmake (-33)
  • (removed) libc/config/baremetal/api.td (-59)
  • (removed) libc/config/gpu/api.td (-50)
  • (removed) libc/config/linux/api.td (-276)
  • (removed) libc/config/public_api.td (-26)
  • (modified) libc/docs/dev/header_generation.rst (-3)
  • (modified) libc/docs/gpu/building.rst (+4-9)
  • (modified) libc/include/CMakeLists.txt (+8-18)
  • (removed) libc/spec/bsd_ext.td (-87)
  • (removed) libc/spec/gnu_ext.td (-316)
  • (removed) libc/spec/gpu_ext.td (-18)
  • (removed) libc/spec/linux.td (-328)
  • (removed) libc/spec/llvm_libc_ext.td (-116)
  • (removed) libc/spec/llvm_libc_stdfix_ext.td (-27)
  • (removed) libc/spec/posix.td (-1867)
  • (removed) libc/spec/spec.td (-253)
  • (removed) libc/spec/stdc.td (-1819)
  • (removed) libc/spec/stdc_ext.td (-82)
  • (modified) libc/test/src/CMakeLists.txt (-78)
  • (removed) libc/utils/HdrGen/CMakeLists.txt (-22)
  • (removed) libc/utils/HdrGen/Command.cpp (-15)
  • (removed) libc/utils/HdrGen/Command.h (-54)
  • (removed) libc/utils/HdrGen/Generator.cpp (-203)
  • (removed) libc/utils/HdrGen/Generator.h (-60)
  • (removed) libc/utils/HdrGen/IncludeFileCommand.cpp (-50)
  • (removed) libc/utils/HdrGen/IncludeFileCommand.h (-32)
  • (removed) libc/utils/HdrGen/Main.cpp (-62)
  • (removed) libc/utils/HdrGen/PrototypeTestGen/CMakeLists.txt (-5)
  • (removed) libc/utils/HdrGen/PrototypeTestGen/PrototypeTestGen.cpp (-106)
  • (removed) libc/utils/HdrGen/PublicAPICommand.cpp (-331)
  • (removed) libc/utils/HdrGen/PublicAPICommand.h (-48)
  • (removed) libc/utils/HdrGen/README.md (-5)
  • (removed) libc/utils/LibcTableGenUtil/APIIndexer.cpp (-173)
  • (removed) libc/utils/LibcTableGenUtil/APIIndexer.h (-86)
  • (removed) libc/utils/LibcTableGenUtil/CMakeLists.txt (-13)
diff --git a/libc/CMakeLists.txt b/libc/CMakeLists.txt
index 77b659b2ef2322..d00888b8801dc5 100644
--- a/libc/CMakeLists.txt
+++ b/libc/CMakeLists.txt
@@ -54,27 +54,12 @@ set(LIBC_NAMESPACE ${default_namespace}
 add_subdirectory(newhdrgen)
 
 
-if(LLVM_LIBC_FULL_BUILD OR LLVM_LIBC_GPU_BUILD)
-  if(NOT LIBC_HDRGEN_EXE)
-    # We need to set up hdrgen first since other targets depend on it.
-    add_subdirectory(utils/LibcTableGenUtil)
-    add_subdirectory(utils/HdrGen)
-    # Calling add_tablegen sets variables like LIBC_TABLEGEN_EXE in
-    # PARENT_SCOPE which get lost until saved in the cache.
-    set(LIBC_TABLEGEN_EXE "${LIBC_TABLEGEN_EXE}" CACHE INTERNAL "")
-    set(LIBC_TABLEGEN_TARGET "${LIBC_TABLEGEN_TARGET}" CACHE INTERNAL "")
-  else()
-    message(STATUS "Will use ${LIBC_HDRGEN_EXE} for libc header generation.")
-  endif()
-endif()
 # We will build the GPU utilities if we are not doing a runtimes build.
 option(LIBC_BUILD_GPU_LOADER "Always build the GPU loader utilities" OFF)
 if(LIBC_BUILD_GPU_LOADER OR (LLVM_LIBC_GPU_BUILD AND NOT LLVM_RUNTIMES_BUILD))
   add_subdirectory(utils/gpu)
 endif()
 
-option(LIBC_USE_NEW_HEADER_GEN "Generate header files using new headergen instead of the old one" ON)
-
 set(NEED_LIBC_HDRGEN FALSE)
 if(NOT LLVM_RUNTIMES_BUILD)
   if("libc" IN_LIST LLVM_ENABLE_RUNTIMES)
@@ -88,8 +73,7 @@ if(NOT LLVM_RUNTIMES_BUILD)
     endforeach()
   endif()
 endif()
-option(LIBC_HDRGEN_ONLY "Only build the 'libc-hdrgen' executable" OFF)
-if(LIBC_HDRGEN_ONLY OR NEED_LIBC_HDRGEN)
+if(NEED_LIBC_HDRGEN)
   # When libc is build as part of the runtimes/bootstrap build's CMake run, we
   # only need to build the host tools to build the libc. So, we just do enough
   # to build libc-hdrgen and return.
diff --git a/libc/cmake/modules/LLVMLibCHeaderRules.cmake b/libc/cmake/modules/LLVMLibCHeaderRules.cmake
index 76c4e1f2d3244c..766225cbeab404 100644
--- a/libc/cmake/modules/LLVMLibCHeaderRules.cmake
+++ b/libc/cmake/modules/LLVMLibCHeaderRules.cmake
@@ -220,46 +220,13 @@ function(add_gen_header target_name)
 
   set(gen_hdr_script "${LIBC_BUILD_SCRIPTS_DIR}/gen_hdr.py")
 
-  file(GLOB td_includes ${LIBC_SOURCE_DIR}/spec/*.td)
-
   set(ENTRYPOINT_NAME_LIST_ARG ${TARGET_ENTRYPOINT_NAME_LIST})
   list(TRANSFORM ENTRYPOINT_NAME_LIST_ARG PREPEND "--e=")
 
-  if(LIBC_HDRGEN_EXE)
-    set(hdrgen_exe ${LIBC_HDRGEN_EXE})
-  else()
-    set(hdrgen_exe ${LIBC_TABLEGEN_EXE})
-    set(hdrgen_deps "${LIBC_TABLEGEN_EXE};${LIBC_TABLEGEN_TARGET}")
-  endif()
-  add_custom_command(
-    OUTPUT ${out_file}
-    COMMAND ${hdrgen_exe} -o ${out_file} --header ${ADD_GEN_HDR_GEN_HDR}
-            --def ${in_file} ${replacement_params} -I ${LIBC_SOURCE_DIR}
-           ${ENTRYPOINT_NAME_LIST_ARG}
-           ${LIBC_SOURCE_DIR}/config/${LIBC_TARGET_OS}/api.td
-
-    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
-    DEPENDS ${in_file} ${fq_data_files} ${td_includes}
-            ${LIBC_SOURCE_DIR}/config/${LIBC_TARGET_OS}/api.td
-            ${hdrgen_deps}
-  )
-
   if(LIBC_TARGET_OS_IS_GPU)
     file(MAKE_DIRECTORY ${LIBC_INCLUDE_DIR}/llvm-libc-decls)
     file(MAKE_DIRECTORY ${LIBC_INCLUDE_DIR}/llvm-libc-decls/gpu)
     set(decl_out_file ${LIBC_INCLUDE_DIR}/llvm-libc-decls/${relative_path})
-    add_custom_command(
-      OUTPUT ${decl_out_file}
-      COMMAND ${hdrgen_exe} -o ${decl_out_file}
-              --header ${ADD_GEN_HDR_GEN_HDR} --def ${in_file} --export-decls
-              ${replacement_params} -I ${LIBC_SOURCE_DIR} ${ENTRYPOINT_NAME_LIST_ARG}
-              ${LIBC_SOURCE_DIR}/config/${LIBC_TARGET_OS}/api.td
-
-      WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
-      DEPENDS ${in_file} ${fq_data_files} ${td_includes}
-              ${LIBC_SOURCE_DIR}/config/${LIBC_TARGET_OS}/api.td
-              ${hdrgen_deps}
-    )
   endif()
 
   if(ADD_GEN_HDR_DEPENDS)
diff --git a/libc/config/baremetal/api.td b/libc/config/baremetal/api.td
deleted file mode 100644
index 7421d86fabeb08..00000000000000
--- a/libc/config/baremetal/api.td
+++ /dev/null
@@ -1,59 +0,0 @@
-include "config/public_api.td"
-
-include "spec/stdc.td"
-include "spec/stdc_ext.td"
-include "spec/bsd_ext.td"
-include "spec/llvm_libc_stdfix_ext.td"
-
-
-def CTypeAPI : PublicAPI<"ctype.h"> {
-}
-
-def FEnvAPI : PublicAPI<"fenv.h"> {
-  let Types = ["fenv_t", "fexcept_t"];
-}
-
-def IntTypesAPI : PublicAPI<"inttypes.h"> {
-  let Types = ["imaxdiv_t"];
-}
-
-def MathAPI : PublicAPI<"math.h"> {
-  let Types = ["double_t", "float_t"];
-}
-
-def StdIOAPI : PublicAPI<"stdio.h"> {
-  let Types = ["size_t"];
-}
-
-def StdlibAPI : PublicAPI<"stdlib.h"> {
-  let Types = [
-    "div_t",
-    "ldiv_t",
-    "lldiv_t",
-    "size_t",
-    "__bsearchcompare_t",
-    "__qsortcompare_t",
-  ];
-}
-
-def StringAPI : PublicAPI<"string.h"> {
-  let Types = ["size_t"];
-}
-
-def TimeAPI : PublicAPI<"time.h"> {
-  let Types = [
-    "clock_t",
-    "time_t",
-    "struct tm",
-    "struct timespec",
-  ];
-}
-
-def UCharAPI : PublicAPI<"uchar.h"> {
-  let Types = [
-    "mbstate_t",
-    "char8_t",
-    "char16_t",
-    "char32_t",
-  ];
-}
diff --git a/libc/config/gpu/api.td b/libc/config/gpu/api.td
deleted file mode 100644
index 995ff31c4ac9e9..00000000000000
--- a/libc/config/gpu/api.td
+++ /dev/null
@@ -1,50 +0,0 @@
-include "config/public_api.td"
-
-include "spec/stdc.td"
-include "spec/posix.td"
-include "spec/gpu_ext.td"
-include "spec/gnu_ext.td"
-include "spec/stdc_ext.td"
-include "spec/llvm_libc_ext.td"
-
-
-def StringAPI : PublicAPI<"string.h"> {
-  let Types = ["size_t"];
-}
-
-def StdlibAPI : PublicAPI<"stdlib.h"> {
-  let Types = [
-    "div_t",
-    "ldiv_t",
-    "lldiv_t",
-    "size_t",
-    "__bsearchcompare_t",
-    "__qsortcompare_t",
-    "__qsortrcompare_t",
-    "__atexithandler_t",
-  ];
-}
-
-def FenvAPI: PublicAPI<"fenv.h"> {
-  let Types = ["fenv_t"];
-}
-
-def StdIOAPI : PublicAPI<"stdio.h"> {
-  let Types = [
-    "FILE",
-    "off_t",
-    "size_t",
-  ];
-}
-
-def IntTypesAPI : PublicAPI<"inttypes.h"> {
-  let Types = ["imaxdiv_t"];
-}
-
-def TimeAPI : PublicAPI<"time.h"> {
-  let Types = [
-    "clock_t",
-    "time_t",
-    "struct timespec",
-  ];
-}
diff --git a/libc/config/linux/api.td b/libc/config/linux/api.td
deleted file mode 100644
index a00e0f61b90df7..00000000000000
--- a/libc/config/linux/api.td
+++ /dev/null
@@ -1,276 +0,0 @@
-include "config/public_api.td"
-
-include "spec/stdc.td"
-include "spec/posix.td"
-include "spec/linux.td"
-include "spec/gnu_ext.td"
-include "spec/bsd_ext.td"
-include "spec/stdc_ext.td"
-include "spec/llvm_libc_ext.td"
-include "spec/llvm_libc_stdfix_ext.td"
-
-def CTypeAPI : PublicAPI<"ctype.h"> {
-}
-
-def FCntlAPI : PublicAPI<"fcntl.h"> {
-  let Types = [
-    "mode_t",
-    "off_t",
-  ];
-}
-
-def IntTypesAPI : PublicAPI<"inttypes.h"> {
-  let Types = ["imaxdiv_t"];
-}
-
-def MathAPI : PublicAPI<"math.h"> {
-  let Types = ["double_t", "float_t", "float128"];
-}
-
-def FenvAPI: PublicAPI<"fenv.h"> {
-  let Types = ["fenv_t", "fexcept_t"];
-}
-
-def StringAPI : PublicAPI<"string.h"> {
-  let Types = ["size_t"];
-}
-
-def StdIOAPI : PublicAPI<"stdio.h"> {
-  let Macros = [
-    SimpleMacroDef<"stderr", "stderr">,
-    SimpleMacroDef<"stdin", "stdin">,
-    SimpleMacroDef<"stdout", "stdout">,
-  ];
-  let Types = [
-    "FILE",
-    "cookie_io_functions_t",
-    "off_t",
-    "size_t",
-  ];
-}
-
-def StdlibAPI : PublicAPI<"stdlib.h"> {
-  let Types = [
-    "div_t",
-    "ldiv_t",
-    "lldiv_t",
-    "size_t",
-    "__bsearchcompare_t",
-    "__qsortcompare_t",
-    "__qsortrcompare_t",
-    "__atexithandler_t",
-  ];
-}
-
-def TimeAPI : PublicAPI<"time.h"> {
-  let Types = [
-    "clock_t",
-    "time_t",
-    "struct tm",
-    "struct timespec",
-    "struct timeval",
-    "clockid_t",
-  ];
-}
-
-def SchedAPI : PublicAPI<"sched.h"> {
-  let Types = [
-    "pid_t",
-    "size_t",
-    "cpu_set_t",
-    "struct sched_param",
-    // Needed according to posix standard
-    "time_t",
-    "struct timespec",
-  ];
-}
-
-def SysMManAPI : PublicAPI<"sys/mman.h"> {
-  let Types = ["off_t", "size_t", "mode_t"];
-}
-
-def SignalAPI : PublicAPI<"signal.h"> {
-  let Types = [
-    "sig_atomic_t",
-    "sigset_t",
-    "struct sigaction",
-    "union sigval",
-    "siginfo_t",
-    "stack_t",
-    "pid_t",
-  ];
-}
-
-def ThreadsAPI : PublicAPI<"threads.h"> {
-  let Macros = [
-    SimpleMacroDef<"ONCE_FLAG_INIT", "{0}">,
-  ];
-
-  let Types = [
-    "__call_once_func_t",
-    "once_flag",
-    "cnd_t",
-    "mtx_t",
-    "thrd_t",
-    "thrd_start_t",
-    "tss_t",
-    "tss_dtor_t",
-  ];
-
-  let Enumerations = [
-    "mtx_plain",
-    "mtx_recursive",
-    "mtx_timed",
-    "thrd_timedout",
-    "thrd_success",
-    "thrd_busy",
-    "thrd_error",
-    "thrd_nomem",
-  ];
-}
-
-def PThreadAPI : PublicAPI<"pthread.h"> {
-  let Types = [
-      "__atfork_callback_t",
-      "__pthread_once_func_t",
-      "__pthread_start_t",
-      "__pthread_tss_dtor_t",
-      "pthread_attr_t",
-      "pthread_condattr_t",
-      "pthread_key_t",
-      "pthread_mutex_t",
-      "pthread_mutexattr_t",
-      "pthread_once_t",
-      "pthread_rwlockattr_t",
-      "pthread_rwlock_t",
-      "pthread_spinlock_t",
-      "pthread_t",
-  ];
-}
-
-def DirentAPI : PublicAPI<"dirent.h"> {
-  let Types = [
-    "ino_t",
-    "DIR",
-    "struct dirent",
-  ];
-}
-
-def UniStdAPI : PublicAPI<"unistd.h"> {
-  let Types = ["__exec_argv_t", "__exec_envp_t", "off_t", "pid_t", "size_t",
-               "ssize_t", "uid_t", "__getoptargv_t"];
-}
-
-def WCharAPI : PublicAPI<"wchar.h"> {
-  let Types = [
-    "wchar_t",
-    "wint_t",
-    "size_t",
-  ];
-}
-
-def UCharAPI : PublicAPI<"uchar.h"> {
-  let Types = [
-    "mbstate_t",
-    "char8_t",
-    "char16_t",
-    "char32_t",
-  ];
-}
-
-def SysRandomAPI : PublicAPI<"sys/random.h"> {
-  let Types = ["size_t", "ssize_t"];
-}
-
-def SysSelectAPI : PublicAPI<"sys/select.h"> {
-  let Types = ["fd_set", "sigset_t", "suseconds_t", "time_t", "struct timespec",
-               "struct timeval"];
-}
-
-def SysSocketAPI : PublicAPI<"sys/socket.h"> {
-  let Types = [
-    "sa_family_t",
-    "socklen_t",
-    "struct sockaddr",
-    "struct sockaddr_un",
-    "struct msghdr",
-    "struct iovec",
-    "size_t",
-    "ssize_t",
-  ];
-}
-
-def SysResourceAPI : PublicAPI<"sys/resource.h"> {
-  let Types = ["rlim_t", "struct rlimit"];
-}
-
-def SysStatAPI : PublicAPI<"sys/stat.h"> {
-  let Types = ["mode_t", "dev_t", "ino_t", "nlink_t", "uid_t", "gid_t", "off_t",
-               "struct timespec", "struct timeval", "blksize_t", "blkcnt_t",
-               "struct stat"];
-}
-
-def SysWaitAPI : PublicAPI<"sys/wait.h"> {
-  let Types = ["pid_t", "struct rusage", "siginfo_t"];
-}
-
-def SysSendfileAPI : PublicAPI<"sys/sendfile.h"> {
-  let Types = ["off_t", "size_t", "ssize_t"];
-}
-
-def SysTypesAPI : PublicAPI<"sys/types.h"> {
-  let Types = [
-    "blkcnt_t",
-    "blksize_t",
-    "clockid_t",
-    "dev_t",
-    "gid_t",
-    "ino_t",
-    "mode_t",
-    "nlink_t",
-    "off_t",
-    "pid_t",
-    "pthread_attr_t",
-    "pthread_condattr_t",
-    "pthread_key_t",
-    "pthread_mutex_t",
-    "pthread_mutexattr_t",
-    "pthread_once_t",
-    "pthread_rwlockattr_t",
-    "pthread_rwlock_t",
-    "pthread_t",
-    "size_t",
-    "ssize_t",
-    "suseconds_t",
-    "time_t",
-    "uid_t"
-  ];
-}
-
-def SysUtsNameAPI : PublicAPI<"sys/utsname.h"> {
-  let Types = ["struct utsname"];
-}
-
-def SysEpollAPI : PublicAPI<"sys/epoll.h"> {
-  let Types = ["struct epoll_event", "struct epoll_data", "sigset_t", "struct timespec"];
-}
-
-def SpawnAPI : PublicAPI<"spawn.h"> {
-  let Types = ["mode_t", "pid_t", "posix_spawnattr_t", "posix_spawn_file_actions_t"];
-}
-
-def TermiosAPI : PublicAPI<"termios.h"> {
-  let Types = ["cc_t", "pid_t", "speed_t", "struct termios", "tcflag_t"];
-}
-
-def SetJmpAPI : PublicAPI<"setjmp.h"> {
-  let Types = ["jmp_buf"];
-}
-
-def SearchAPI : PublicAPI<"search.h"> {
-  let Types = ["ACTION", "ENTRY", "struct hsearch_data", "__lsearchcompare_t"];
-}
-
-def SysStatvfsAPI : PublicAPI<"sys/statvfs.h"> {
-  let Types = ["struct statvfs"];
-}
diff --git a/libc/config/public_api.td b/libc/config/public_api.td
deleted file mode 100644
index 1b34506c643c32..00000000000000
--- a/libc/config/public_api.td
+++ /dev/null
@@ -1,26 +0,0 @@
-include "spec/spec.td"
-
-class MacroDef<string name> {
-  string Name = name;
-  string Defn = "";
-}
-
-class SimpleMacroDef<string name, string value> : MacroDef<name> {
-  let Defn = !strconcat("#define ", name, " ", value);
-}
-
-class MacroDefineIfNot<string name, string value> : MacroDef<name> {
-  let Defn = !strconcat("#ifndef ", name, "\n",
-                        "#define " , name, " ", value, "\n",
-                        "#endif // ", name);
-}
-
-class PublicAPI<string name> {
-  string HeaderName = name;
-  list<MacroDef> Macros = [];
-  list<string> Types = [];
-  list<string> Enumerations = [];
-  list<string> Structs = [];
-  list<string> Functions = [];
-  list<string> Objects = [];
-}
diff --git a/libc/docs/dev/header_generation.rst b/libc/docs/dev/header_generation.rst
index ec4206217ca777..0730b9a40c26a1 100644
--- a/libc/docs/dev/header_generation.rst
+++ b/libc/docs/dev/header_generation.rst
@@ -38,9 +38,6 @@ Required Versions:
    ``build/projects/libc/include/sys``.
 
 
-New Headergen is turned on by default, but if you want to use old Headergen,
-you can include this statement when building: ``-DLIBC_USE_NEW_HEADER_GEN=OFF``
-
 To add a function to the YAML files, you can either manually enter it in the
 YAML file corresponding to the header it belongs to or add it through the
 command line.
diff --git a/libc/docs/gpu/building.rst b/libc/docs/gpu/building.rst
index 37dccdab6dc340..6a2d1f23c0bcdf 100644
--- a/libc/docs/gpu/building.rst
+++ b/libc/docs/gpu/building.rst
@@ -86,16 +86,13 @@ compiler. These tools must all be up-to-date with the libc source.
      -DCMAKE_C_COMPILER=$HOST_C_COMPILER      \
      -DCMAKE_CXX_COMPILER=$HOST_CXX_COMPILER  \
      -DLLVM_LIBC_FULL_BUILD=ON                \
-     -DLIBC_HDRGEN_ONLY=ON    \ # Only build the 'libc-hdrgen' tool
      -DCMAKE_BUILD_TYPE=Release # Release suggested to make "clang" fast
   $> ninja # Build the 'clang' compiler
-  $> ninja libc-hdrgen # Build the 'libc-hdrgen' tool
 
-Once this has finished the build directory should contain the ``clang`` compiler
-and the ``libc-hdrgen`` executable. We will use the ``clang`` compiler to build
-the GPU code and the ``libc-hdrgen`` tool to create the necessary headers. We
-use these tools to bootstrap the build out of the runtimes directory targeting a
-GPU architecture.
+Once this has finished the build directory should contain the ``clang``
+compiler executable. We will use the ``clang`` compiler to build the GPU code.
+We use these tools to bootstrap the build out of the runtimes directory
+targeting a GPU architecture.
 
 .. code-block:: sh
 
@@ -105,7 +102,6 @@ GPU architecture.
   $> TARGET_TRIPLE=<amdgcn-amd-amdhsa or nvptx64-nvidia-cuda>
   $> TARGET_C_COMPILER=</path/to/clang>
   $> TARGET_CXX_COMPILER=</path/to/clang++>
-  $> HDRGEN=</path/to/libc-hdrgen>
   $> cmake ../runtimes \ # Point to the runtimes build
      -G Ninja                                  \
      -DLLVM_ENABLE_RUNTIMES=libc               \
@@ -113,7 +109,6 @@ GPU architecture.
      -DCMAKE_CXX_COMPILER=$TARGET_CXX_COMPILER \
      -DLLVM_LIBC_FULL_BUILD=ON                 \
      -DLLVM_RUNTIMES_TARGET=$TARGET_TRIPLE     \
-     -DLIBC_HDRGEN_EXE=$HDRGEN                 \
      -DCMAKE_BUILD_TYPE=Release
   $> ninja install
 
diff --git a/libc/include/CMakeLists.txt b/libc/include/CMakeLists.txt
index 91611026df105c..483e63da48deae 100644
--- a/libc/include/CMakeLists.txt
+++ b/libc/include/CMakeLists.txt
@@ -18,24 +18,14 @@ add_header(
 )
 
 macro(add_header_macro TARGET_NAME YAML_FILE DEF_FILE GEN_HDR DEPENDS)
-  if (LIBC_USE_NEW_HEADER_GEN)
-    add_gen_header2(
-      ${TARGET_NAME}
-      YAML_FILE ${YAML_FILE}
-      DEF_FILE ${DEF_FILE}
-      GEN_HDR ${GEN_HDR}
-      ${DEPENDS}
-      ${ARGN}
-    )
-  else()
-    add_gen_header(
-      ${TARGET_NAME}
-      DEF_FILE ${DEF_FILE}
-      GEN_HDR ${GEN_HDR}
-      ${DEPENDS}
-      ${ARGN}
-    )
-  endif()
+  add_gen_header2(
+    ${TARGET_NAME}
+    YAML_FILE ${YAML_FILE}
+    DEF_FILE ${DEF_FILE}
+    GEN_HDR ${GEN_HDR}
+    ${DEPENDS}
+    ${ARGN}
+  )
 endmacro()
 
 add_header_macro(
diff --git a/libc/spec/bsd_ext.td b/libc/spec/bsd_ext.td
deleted file mode 100644
index 2b91324e36db9a..00000000000000
--- a/libc/spec/bsd_ext.td
+++ /dev/null
@@ -1,87 +0,0 @@
-def BsdExtensions : StandardSpec<"BSDExtensions"> {
-  HeaderSpec Math = HeaderSpec<
-      "math.h",
-      [], // Macros
-      [], // Types
-      [], // Enumerations
-      [
-          FunctionSpec<"isnan", RetValSpec<IntType>, [ArgSpec<DoubleType>]>,
-          FunctionSpec<"isnanf", RetValSpec<IntType>, [ArgSpec<FloatType>]>,
-          FunctionSpec<"isnanl", RetValSpec<IntType>, [ArgSpec<LongDoubleType>]>,
-      ]
-  >;
-
-  HeaderSpec String = HeaderSpec<
-      "string.h",
-      [], // Macros
-      [], // Types
-      [], // Enumerations
-      [
-        FunctionSpec<
-            "strlcat",
-            RetValSpec<SizeTType>,
-            [ArgSpec<CharRestrictedPtr>, ArgSpec<ConstCharRestrictedPtr>, ArgSpec<SizeTType>]
-        >,
-        FunctionSpec<
-            "strlcpy",
-            RetValSpec<SizeTType>,
-            [ArgSpec<CharRestrictedPtr>, ArgSpec<ConstCharRestrictedPtr>, ArgSpec<SizeTType>]
-        >,
-        FunctionSpec<
-            "strsep",
-            RetValSpec<CharPtr>,
-            [ArgSpec<CharRestrictedPtrPtr>, ArgSpec<ConstCharRestrictedPtr>]
-        >,
-      ]
-  >;
-
-  HeaderSpec Strings = HeaderSpec<
-      "strings.h",
-      [], // Macros
-      [], // Types
-      [], // Enumerations
-      [
-        FunctionSpec<
-            "strcasecmp",
-            RetValSpec<IntType>,
-            [ArgSpec<ConstCharPtr>, ArgSpec<ConstCharPtr>]
-        >,
-        FunctionSpec<
-            "strncasecmp",
-            RetValSpec<IntType>,
-            [ArgSpec<ConstCharPtr>, ArgSpec<ConstCharPtr>, ArgSpec<SizeTType>]
-        >,
-        FunctionSpec<
-            "index",
-            RetValSpec<CharPtr>,
-            [ArgSpec<ConstCharPtr>, ArgSpec<IntType>]
-        >,
-        FunctionSpec<
-            "rindex",
-            RetValSpec<CharPtr>,
-            [ArgSpec<ConstCharPtr>, ArgSpec<IntType>]
-        >,
-      ]
-  >;
-
-  HeaderSpec SysWait = HeaderSpec<
-      "sys/wait.h",
-      [], // Macros
-      [StructRUsage], // Types
-      [], // Enumerations
-      [
-        FunctionSpec<
-            "wait4",
-            RetValSpec<PidT>,
-            [ArgSpec<PidT>, ArgSpec<IntPtr>, ArgSpec<IntType>, ArgSpec<StructRUsagePtr>]
-        >
-      ]
-  >;
-
-  let Headers = [
-    Math,
-    String,
-    Strings,
-    SysWait,
-  ];
-}
diff --git a/libc/spec/gnu_ext.td b/libc/spec/gnu_ext.td
deleted file mode 100644
index 64121aed9574f2..00000000000000
--- a/libc/spec/gnu_ext.td
+++ /dev/null
@@ -1,316 +0,0 @@
-def CpuSetT : NamedType<"cpu_set_t">;
-def CpuSetPtr : PtrType<CpuSetT>;
-def ConstCpuSetPtr : ConstType<CpuSetPtr>;
-
-def QSortRCompareT : NamedType<"__qsortrcompare_t">;
-def StructHsearchData : NamedType<"struct hsearch_data">;
-def StructHsearchDataPtr : PtrType<StructHsearchData>;
-
-def GnuExtensions : StandardSpec<"GNUExtensions"> {
-  NamedType CookieIOFunctionsT = NamedType<"cookie_io_functions_t">;
-  HeaderSpec CType = HeaderSpec<
-    "ctype.h",
-    [], // Macros
-    [], // Types
-    [], // Enumerations
-    [
-        FunctionSpec<
-            "toascii",
-            RetValSpec<IntType>,
-            [ArgSpec<IntType>]
-        >,
-    ]
-  >;
-
-  HeaderSpec Malloc = HeaderSpec<
-      "malloc.h",
-      [], // Macros
-      [], // Types
-      [], // Enumerations
-      [
-          FunctionSpec<"mallopt", RetValSpec<IntType>, [ArgSpec<IntType>, ArgSpec<IntType>]>,
-      ]
-  >;
-
-  HeaderSpec Math = HeaderSpec<
-      "math.h",
-      [], // Macros
-      [], // Types
-      [], // Enumerations
-      [
-        FunctionSpec<
-            "sincosf",
-            RetValSpec<VoidType>,
-            [ArgSpec<FloatType>, ArgSpec<FloatPtr>, ArgSpec<FloatPtr>]
-        >,
-        FunctionSpec<
-            "lgamma_r",
-            RetValSpec<DoubleType>,
-            [ArgSpec<DoubleType>, ArgSpec<IntPtr>]
-        >,
-     ...
[truncated]

Thanks to the effort of @RoseZhang03 and @aaryanshukla under the guidance of
@michaelrj-google and @amykhuang, we now have newhdrgen and no longer have a
dependency on TableGen and thus LLVM in order to start bootstrapping a full
build.

This PR removes:
- LIBC_HDRGEN_EXE; the in tree newhdrgen is the only hdrgen that can be used.
- LIBC_USE_NEW_HEADER_GEN; newhdrgen is the default and only option.
- LIBC_HDRGEN_ONLY; there is no need to have a distinct build step for old
  hdrgen.
- libc-api-test and libc-api-test-tidy build targets.
- Deletes all .td files.

It does not rename newhdrgen to just hdrgen. Will follow up with a distinct PR
for that.

Link: llvm#117209
Fixes: llvm#117208
# We will build the GPU utilities if we are not doing a runtimes build.
option(LIBC_BUILD_GPU_LOADER "Always build the GPU loader utilities" OFF)
if(LIBC_BUILD_GPU_LOADER OR (LLVM_LIBC_GPU_BUILD AND NOT LLVM_RUNTIMES_BUILD))
add_subdirectory(utils/gpu)
endif()

option(LIBC_USE_NEW_HEADER_GEN "Generate header files using new headergen instead of the old one" ON)

set(NEED_LIBC_HDRGEN FALSE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be deleted?

Suggested change
set(NEED_LIBC_HDRGEN FALSE)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure. Let me check the original context under which it was added.
9dbedca

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also related c4f236e

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI the GPU build still uses that logic to make the amdhsa-loader and nvptx-loader utilities.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, I think I left the important bits intact in 46251c6, PTAL.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed everything, in 1c11706. But looking at 5274a46, I'm not sure if we should be returning early if(LLVM_ENABLE_RUNTIMES AND NOT LLVM_RUNTIMES_BUILD). PTAL

Comment on lines 76 to 79
if(NEED_LIBC_HDRGEN)
# When libc is build as part of the runtimes/bootstrap build's CMake run, we
# only need to build the host tools to build the libc. So, we just do enough
# to build libc-hdrgen and return.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't NEED_LIBC_HDRGEN be renamed to something else and the comment below be updated? Might want to add a TODO comment here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@michaelrj-google michaelrj-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from what I can see. I'd like to run the tests locally to double check nothing broke, but I'm very excited to see this land.

@@ -18,24 +18,14 @@ add_header(
)

macro(add_header_macro TARGET_NAME YAML_FILE DEF_FILE GEN_HDR DEPENDS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Add a TODO to clean this up

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ae75b6b

I tried replacing invocations of add_header_macro with just add_gen_header, but this seemed to run into issues. Not sure if every function invocation of add_gen_header would need explicit DEF_FILE and GEN_HDR spelled out?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe they will need explicit DEF_FILE and GEN_HDR. The macro for add_gen_header2 (lines22-29) should be used, but can be renamed to add_gen_header

@michaelrj-google
Copy link
Contributor

Tests passed, LGTM once the buildbots are ready

@nickdesaulniers
Copy link
Member Author

nickdesaulniers commented Nov 21, 2024

The post submit buildbots with the libc tag are all moved to newhdrgen now. I'll wait for explicit acks from @petrhosek and @jhuber6 before attempting to [crash] land this.

Copy link
Member

@petrhosek petrhosek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We no longer depend on old hdrgen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[libc] delete old hdrgen
7 participants