data.table installation from source fails on Mac, `symbol not found in flat namespace '___kmpc_barrier'` #6622

dvg-p4 · 2024-11-20T20:33:51Z

This issue may be a result of #6418 causing openmp support to be detected, and data.table to be compiled, but not linked with -fopenmp.

Minimal reproducible example

dgealow@Dans-P4-MacBook-Pro ~ % R          

R version 4.2.3 (2023-03-15) -- "Shortstop Beagle"
[...]

> install.packages('data.table')
Installing package into ‘/Users/dgealow/Library/R/arm64/4.2/library’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
[selected 0-cloud]

  There is a binary version available but the source version is later:
           binary source needs_compilation
data.table 1.15.4 1.16.2              TRUE

Do you want to install from sources the package which needs compilation? (Yes/no/cancel) yes
installing the source package ‘data.table’

Full output

trying URL 'https://cloud.r-project.org/src/contrib/data.table_1.16.2.tar.gz'
Content type 'application/x-gzip' length 5490076 bytes (5.2 MB)
==================================================
downloaded 5.2 MB

* installing *source* package ‘data.table’ ...
** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
zlib 1.2.11 is available ok
* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... no
* checking if R installation supports OpenMP with "-fopenmp" ... yes
** libs
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c assign.c -o assign.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c bmerge.c -o bmerge.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c chmatch.c -o chmatch.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c cj.c -o cj.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c coalesce.c -o coalesce.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c dogroups.c -o dogroups.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fastmean.c -o fastmean.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fcast.c -o fcast.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fifelse.c -o fifelse.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fmelt.c -o fmelt.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c frank.c -o frank.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fread.c -o fread.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c freadR.c -o freadR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c froll.c -o froll.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c frolladaptive.c -o frolladaptive.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fsort.c -o fsort.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fwrite.c -o fwrite.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fwriteR.c -o fwriteR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c gsumm.c -o gsumm.o
gsumm.c:1173:38: warning: implicit conversion from 'long long' to 'long double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
 1173 |       ansd[i] = (ISNAN(s[i]) || s[i]>INT64_MAX || s[i]<=INT64_MIN) ? NA_INTEGER64 : (int64_t)s[i];
      |                                     ~^~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/stdint.h:94:26: note: expanded from macro 'INT64_MAX'
   94 | #define INT64_MAX        9223372036854775807LL
      |                          ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c idatetime.c -o idatetime.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c ijoin.c -o ijoin.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c inrange.c -o inrange.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c nafill.c -o nafill.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c negate.c -o negate.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c nqrecreateindices.c -o nqrecreateindices.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c openmp-utils.c -o openmp-utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c programming.c -o programming.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c quickselect.c -o quickselect.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c reorder.c -o reorder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c shift.c -o shift.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c snprintf.c -o snprintf.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c subset.c -o subset.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c transpose.c -o transpose.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c types.c -o types.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c uniqlist.c -o uniqlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c vecseq.c -o vecseq.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include  -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c wrappers.c -o wrappers.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
ld: warning: -single_module is obsolete
ld: warning: -multiply_defined is obsolete
PKG_CFLAGS = -fopenmp
PKG_LIBS = -lz
if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi
installing to /Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs/data_table.so':
  dlopen(/Users/dgealow/Library/R/arm64/4.2/library/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_barrier'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/dgealow/Library/R/arm64/4.2/library/data.table’
* restoring previous ‘/Users/dgealow/Library/R/arm64/4.2/library/data.table’

The downloaded source packages are in
	‘/private/var/folders/sr/lw2ng95s3mj1qsqtdxywzjvc0000gn/T/Rtmp0Drt3Z/downloaded_packages’
Warning message:
In install.packages("data.table") :
  installation of package ‘data.table’ had non-zero exit status
>

Workaround

Sys.setenv(PKG_LIBS='-fopenmp') before installing the package, as suggested by aitap below along with a possible package-level solution.

Output of sessionInfo()

R version 4.2.3 (2023-03-15)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 15.1.1

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.2.3

The text was updated successfully, but these errors were encountered:

aitap · 2024-11-29T11:41:39Z

Does it work if you Sys.setenv(PKG_LIBS='-fopenmp') first? If yes, we might need an export PKG_LIBS="${PKG_LIBS} -fopenmp" here too:

data.table/configure

Line 128 in 546259d

export PKG_CFLAGS="${PKG_CFLAGS} -fopenmp"

(Interesting how it previously worked when compiling with -fopenmp but linking without it. This is probably better handled by someone with a Mac.)

evg-trushkin · 2024-12-02T08:13:25Z

Does it work if you Sys.setenv(PKG_LIBS='-fopenmp') first?

Works in my case (macOS 14.6.1 M1). Thanks!

dvg-p4 · 2024-12-02T16:41:01Z

Does it work if you Sys.setenv(PKG_LIBS='-fopenmp') first?

Yes, it does!

dvg-p4 · 2024-12-02T16:45:58Z

Interesting how it previously worked when compiling with -fopenmp but linking without it.

I'm not sure that it did--previous installations from source of data table on my mac have always started up with a message along the lines of "this installation was compiled without openmp support and is running in single-threaded mode, go complain to Apple"

Here it is in full:

data.table 1.15.4 using 1 threads (see ?getDTthreads). Latest news: r-datatable.com

This installation of data.table has not detected OpenMP support. It should still work but in single-threaded mode.
This is a Mac. Please read https://mac.r-project.org/openmp/. Please engage with Apple and ask them for support. Check r-datatable.com for updates, and our Mac instructions here: https://github.com/Rdatatable/data.table/wiki/Installation. After several years of many reports of installation problems on Mac, it's time to gingerly point out that there have been no similar problems on Windows or Linux.

dvg-p4 · 2024-12-02T21:42:35Z

Ok so on my R4.4 installation, we take a different branch and hit a similar error that seems resistant to the workaround:

% R

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
[...]
> install.packages('data.table', type = 'source')
--- Please select a CRAN mirror for use in this session ---
[selected 0-cloud]
trying URL 'https://cloud.r-project.org/src/contrib/data.table_1.16.2.tar.gz'
Content type 'application/x-gzip' length 5490076 bytes (5.2 MB)
==================================================
downloaded 5.2 MB

* installing *source* package ‘data.table’ ...

Full output

** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
zlib 1.2.11 is available ok
* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... yes
** libs
using C compiler: ‘Homebrew clang version 19.1.3’
using SDK: ‘’
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c assign.c -o assign.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c between.c -o between.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c bmerge.c -o bmerge.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c chmatch.c -o chmatch.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c cj.c -o cj.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c coalesce.c -o coalesce.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c dogroups.c -o dogroups.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fastmean.c -o fastmean.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fcast.c -o fcast.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fifelse.c -o fifelse.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fmelt.c -o fmelt.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c forder.c -o forder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c frank.c -o frank.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fread.c -o fread.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c freadR.c -o freadR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c froll.c -o froll.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c frollR.c -o frollR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c frolladaptive.c -o frolladaptive.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fsort.c -o fsort.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fwrite.c -o fwrite.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c fwriteR.c -o fwriteR.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c gsumm.c -o gsumm.o
gsumm.c:1173:38: warning: implicit conversion from 'long long' to 'long double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
 1173 |       ansd[i] = (ISNAN(s[i]) || s[i]>INT64_MAX || s[i]<=INT64_MIN) ? NA_INTEGER64 : (int64_t)s[i];
      |                                     ~^~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX15.sdk/usr/include/stdint.h:94:26: note: expanded from macro 'INT64_MAX'
   94 | #define INT64_MAX        9223372036854775807LL
      |                          ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c idatetime.c -o idatetime.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c ijoin.c -o ijoin.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c init.c -o init.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c inrange.c -o inrange.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c nafill.c -o nafill.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c negate.c -o negate.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c nqrecreateindices.c -o nqrecreateindices.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c openmp-utils.c -o openmp-utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c programming.c -o programming.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c quickselect.c -o quickselect.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c rbindlist.c -o rbindlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c reorder.c -o reorder.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c shift.c -o shift.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c snprintf.c -o snprintf.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c subset.c -o subset.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c transpose.c -o transpose.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c types.c -o types.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c uniqlist.c -o uniqlist.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c utils.c -o utils.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c vecseq.c -o vecseq.o
clang -arch arm64 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include   -Xclang -fopenmp   -fPIC  -falign-functions=64 -Wall -g -O2  -c wrappers.c -o wrappers.o
clang -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o data.table.so assign.o between.o bmerge.o chmatch.o cj.o coalesce.o dogroups.o fastmean.o fcast.o fifelse.o fmelt.o forder.o frank.o fread.o freadR.o froll.o frollR.o frolladaptive.o fsort.o fwrite.o fwriteR.o gsumm.o idatetime.o ijoin.o init.o inrange.o nafill.o negate.o nqrecreateindices.o openmp-utils.o programming.o quickselect.o rbindlist.o reorder.o shift.o snprintf.o subset.o transpose.o types.o uniqlist.o utils.o vecseq.o wrappers.o -lomp -lz -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
PKG_CFLAGS = -Xclang -fopenmp
PKG_LIBS = -lomp -lz
if [ "data.table.so" != "data_table.so" ]; then mv data.table.so data_table.so; fi
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id data_table.so data_table.so; fi
installing to /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_dispatch_deinit'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/data.table’

The downloaded source packages are in
	‘/private/var/folders/sr/lw2ng95s3mj1qsqtdxywzjvc0000gn/T/RtmpHgVbPN/downloaded_packages’
Warning message:
In install.packages("data.table", type = "source") :
  installation of package ‘data.table’ had non-zero exit status

Though this time the particular symbol we fail to find is ___kmpc_dispatch_deinit.

aitap · 2024-12-03T18:36:00Z

Interesting. The flags seem correct according to the OpenMP on Mac page, but the workaround is apparently not needed and doesn't work with the Homebrew toolchain. Is this R build also from Homebrew?

I suppose we'll have to update the configure test to not only compile a shared library with R CMD SHLIB but also try to load it before declaring the test successful.

It looks like the same workaround should work because the configure script only appends to the user-provided PKG_CFLAGS and PKG_LIBS environment variables, and yet it doesn't. Hard to figure out without running sh -x configure on a real Mac.

dvg-p4 · 2024-12-03T20:01:41Z

It looks like the same workaround should work because the configure script only appends to the user-provided PKG_CFLAGS and PKG_LIBS environment variables, and yet it doesn't. Hard to figure out without running sh -x configure on a real Mac.

Well, doing this produces a third scenario for me, in which all of the checks fail.

* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... + CPPFLAGS='-I/opt/homebrew/opt/llvm/include -Xclang -fopenmp'
+ PKG_LIBS=-lomp
+ /bin/R CMD SHLIB test-omp.c
+ echo no
no
+ printf %s '* checking if R installation supports OpenMP with "-fopenmp" ... '
* checking if R installation supports OpenMP with "-fopenmp" ... + CPPFLAGS='-I/opt/homebrew/opt/llvm/include -fopenmp'
+ /bin/R CMD SHLIB test-omp.c
+ echo no
no
++ uname -m
+ '[' arm64 = arm64 ']'
+ HOMEBREW_PREFIX=/opt/homebrew
+ '[' -e /opt/homebrew/opt/libomp ']'
+ printf %s '* checking if libomp installation at /opt/homebrew/opt/libomp can be used... '
* checking if libomp installation at /opt/homebrew/opt/libomp can be used... + LIBOMP_INCLUDE='-I/opt/homebrew/opt/libomp/include -Xclang -fopenmp'
+ LIBOMP_LINK='-L/opt/homebrew/opt/libomp/lib -lomp'
+ -I/opt/homebrew/opt/libomp/include -Xclang -fopenmp -L/opt/homebrew/opt/libomp/lib -lomp test-omp.c
+ echo no
no
+ export R_OPENMP_ENABLED=0

dvg-p4 · 2024-12-03T20:02:48Z

--though it occurs to me that I do have some R makevars set, which could be affecting things

% cat ~/.R/Makevars
FC = /opt/homebrew/bin/gfortran
FLIBS = -L/opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/ -lgfortran -lquadmath -lm

Although I don't see how they'd account for the difference between R4.2 and R4.4

dvg-p4 · 2024-12-03T20:09:19Z

Ah, it's probably the fact that R4.3 changed the default C(++) standard to C(++)17. R4.3 behaves the same as R4.4 for me.

aitap · 2024-12-03T20:15:18Z

Sorry, forgot to mention that configure relies on R_HOME=`R RHOME` being set.

dvg-p4 · 2024-12-03T20:16:40Z

dgealow@Dans-P4-MacBook-Pro data.table % export R_HOME=$(R RHOME) 
dgealow@Dans-P4-MacBook-Pro data.table % echo $R_HOME            
/Library/Frameworks/R.framework/Resources
dgealow@Dans-P4-MacBook-Pro data.table % zsh -x configure                                                                                                                       
+configure:4> CC=+configure:4> /Library/Frameworks/R.framework/Resources/bin/R CMD config CC
+configure:4> CC='clang -arch arm64' 
+configure:5> CFLAGS=+configure:5> /Library/Frameworks/R.framework/Resources/bin/R CMD config CFLAGS
+configure:5> CFLAGS='-falign-functions=64 -Wall -g -O2' 
+configure:7> echo 'CC=clang -arch arm64'
+configure:8> echo 'CFLAGS=-falign-functions=64 -Wall -g -O2'
+configure:11> case clang -arch arm64 (gcc*)
+configure:22> msg=0 
+configure:23> NOZLIB=1 
+configure:24> pkg-config --version
+configure:25> [ 0 -ne 0 ']'
+configure:29> pkg-config --exists zlib
+configure:30> [ 0 -ne 0 ']'
+configure:34> NOZLIB=0 
+configure:35> lib=+configure:35> pkg-config --libs zlib
+configure:35> lib=-lz 
+configure:36> cflag=+configure:36> pkg-config --cflags zlib
+configure:36> cflag='' 
+configure:37> expr -- -lz : '.*-lz$'
+configure:38> [ 0 -ne 0 ']'
+configure:50> [ 0 -ne 0 ']'
+configure:67> version=+configure:67> pkg-config --modversion zlib
+configure:67> version=1.2.11 
+configure:68> echo 'zlib 1.2.11 is available ok'
zlib 1.2.11 is available ok
+configure:78> cat
+configure:163> detect_openmp
+detect_openmp:2> uname
+detect_openmp:2> [ Darwin '=' Linux ']'
+detect_openmp:25> uname
+detect_openmp:25> [ Darwin '=' Darwin ']'
+detect_openmp:28> printf %s '* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... '
* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... +detect_openmp:29> CPPFLAGS='-I/opt/homebrew/opt/llvm/include -Xclang -fopenmp' PKG_LIBS=-lomp /Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB test-omp.c
+detect_openmp:30> echo yes
yes
+detect_openmp:31> export PKG_CFLAGS=' -Xclang -fopenmp'
+detect_openmp:32> export PKG_LIBS=' -lomp'
+detect_openmp:33> export R_OPENMP_ENABLED=1
+detect_openmp:34> return
+configure:165> rm -f test-omp.c test-omp.o test-omp.so a.out
+configure:167> [ 1 '=' 0 ']'
+configure:177> sed -e 's|@openmp_cflags@|$(SHLIB_OPENMP_CFLAGS)|' src/Makevars.in
+configure:181> sed -e 's|@PKG_CFLAGS@| -Xclang -fopenmp|' src/Makevars
+configure:181> mv src/Makevars.tmp src/Makevars
+configure:182> sed -e 's|@PKG_LIBS@| -lomp|' src/Makevars
+configure:182> mv src/Makevars.tmp src/Makevars
+configure:185> [ 0 '=' 1 ']'
+configure:190> sed -e 's|@zlib_cflags@||' src/Makevars
+configure:190> mv src/Makevars.tmp src/Makevars
+configure:191> sed -e 's|@zlib_libs@|-lz|' src/Makevars
+configure:191> mv src/Makevars.tmp src/Makevars
+configure:194> exit 0

(/Library/Frameworks/R.framework/Resources symlinks into whichever R version I have selected as current with the handy Rswitch utility -- 4.3 for the above, though 4.4 yields similar results)

aitap · 2024-12-03T20:25:47Z

Thanks! What's the difference if you additionally set PKG_CFLAGS=-fopenmp PKG_LIBS=-fopenmp before running sh -x configure?

dvg-p4 · 2024-12-03T20:36:05Z

dgealow@Dans-P4-MacBook-Pro data.table % R --version
R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

dgealow@Dans-P4-MacBook-Pro data.table % echo $R_HOME
/Library/Frameworks/R.framework/Resources
dgealow@Dans-P4-MacBook-Pro data.table % echo $PKG_CFLAGS
-fopenmp
dgealow@Dans-P4-MacBook-Pro data.table % echo $PKG_LIBS
-fopenmp
dgealow@Dans-P4-MacBook-Pro data.table % zsh -x configure
+configure:4> CC=+configure:4> /Library/Frameworks/R.framework/Resources/bin/R CMD config CC
+configure:4> CC='clang -arch arm64' 
+configure:5> CFLAGS=+configure:5> /Library/Frameworks/R.framework/Resources/bin/R CMD config CFLAGS
+configure:5> CFLAGS='-falign-functions=64 -Wall -g -O2' 
+configure:7> echo 'CC=clang -arch arm64'
+configure:8> echo 'CFLAGS=-falign-functions=64 -Wall -g -O2'
+configure:11> case clang -arch arm64 (gcc*)
+configure:22> msg=0 
+configure:23> NOZLIB=1 
+configure:24> pkg-config --version
+configure:25> [ 0 -ne 0 ']'
+configure:29> pkg-config --exists zlib
+configure:30> [ 0 -ne 0 ']'
+configure:34> NOZLIB=0 
+configure:35> lib=+configure:35> pkg-config --libs zlib
+configure:35> lib=-lz 
+configure:36> cflag=+configure:36> pkg-config --cflags zlib
+configure:36> cflag='' 
+configure:37> expr -- -lz : '.*-lz$'
+configure:38> [ 0 -ne 0 ']'
+configure:50> [ 0 -ne 0 ']'
+configure:67> version=+configure:67> pkg-config --modversion zlib
+configure:67> version=1.2.11 
+configure:68> echo 'zlib 1.2.11 is available ok'
zlib 1.2.11 is available ok
+configure:78> cat
+configure:163> detect_openmp
+detect_openmp:2> uname
+detect_openmp:2> [ Darwin '=' Linux ']'
+detect_openmp:25> uname
+detect_openmp:25> [ Darwin '=' Darwin ']'
+detect_openmp:28> printf %s '* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... '
* checking if R installation supports OpenMP with "-Xclang -fopenmp" ... +detect_openmp:29> CPPFLAGS='-I/opt/homebrew/opt/llvm/include -Xclang -fopenmp' PKG_LIBS=-lomp /Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB test-omp.c
+detect_openmp:30> echo yes
yes
+detect_openmp:31> export PKG_CFLAGS='-fopenmp -Xclang -fopenmp'
+detect_openmp:32> export PKG_LIBS='-fopenmp -lomp'
+detect_openmp:33> export R_OPENMP_ENABLED=1
+detect_openmp:34> return
+configure:165> rm -f test-omp.c test-omp.o test-omp.so a.out
+configure:167> [ 1 '=' 0 ']'
+configure:177> sed -e 's|@openmp_cflags@|$(SHLIB_OPENMP_CFLAGS)|' src/Makevars.in
+configure:181> sed -e 's|@PKG_CFLAGS@|-fopenmp -Xclang -fopenmp|' src/Makevars
+configure:181> mv src/Makevars.tmp src/Makevars
+configure:182> sed -e 's|@PKG_LIBS@|-fopenmp -lomp|' src/Makevars
+configure:182> mv src/Makevars.tmp src/Makevars
+configure:185> [ 0 '=' 1 ']'
+configure:190> sed -e 's|@zlib_cflags@||' src/Makevars
+configure:190> mv src/Makevars.tmp src/Makevars
+configure:191> sed -e 's|@zlib_libs@|-lz|' src/Makevars
+configure:191> mv src/Makevars.tmp src/Makevars
+configure:194> exit 0

dvg-p4 · 2024-12-03T20:38:46Z

(and attempting to actually install with those envars set yields the same error as before, symbol not found in flat namespace '___kmpc_dispatch_deinit')

aitap · 2024-12-04T10:49:59Z

Thank you for the information! If you need a workaround right now, try hard-coding the following instead of the call to detect_openmp in the configure script:

export PKG_CFLAGS="${PKG_CFLAGS} -fopenmp"
export PKG_LIBS="${PKG_LIBS} -fopenmp"
export R_OPENMP_ENABLED=1

Hopefully we'll make a proper fix soon.

dvg-p4 · 2024-12-04T19:34:05Z

If you need a workaround right now, try hard-coding the following instead of the call to detect_openmp in the configure script:

Doesn't actually work for me. When I try to install it with R 4.4.1 or 4.4.3, it still gives the same error (can't find ___kmpc_dispatch_deinit when the installation tests run).

Fortunately I'm not actually blocked since the prebuilt binary installs fine on 4.4.

MichaelChirico added bug platform-specific install labels Nov 21, 2024

dvg-p4 mentioned this issue Dec 4, 2024

Make sure data.table is linked (not just compiled) with -fopenmp in this particular branch of configure for Darwin #6633

Open

aitap linked a pull request Dec 9, 2024 that will close this issue

configure: test linking with -fopenmp, test the resulting shared object #6642

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

data.table installation from source fails on Mac, `symbol not found in flat namespace '___kmpc_barrier'` #6622

data.table installation from source fails on Mac, `symbol not found in flat namespace '___kmpc_barrier'` #6622

dvg-p4 commented Nov 20, 2024 •

edited

Loading

aitap commented Nov 29, 2024

evg-trushkin commented Dec 2, 2024

dvg-p4 commented Dec 2, 2024 •

edited

Loading

dvg-p4 commented Dec 2, 2024 •

edited

Loading

dvg-p4 commented Dec 2, 2024

aitap commented Dec 3, 2024

dvg-p4 commented Dec 3, 2024

dvg-p4 commented Dec 3, 2024 •

edited

Loading

dvg-p4 commented Dec 3, 2024

aitap commented Dec 3, 2024 via email

dvg-p4 commented Dec 3, 2024 •

edited

Loading

aitap commented Dec 3, 2024 via email

dvg-p4 commented Dec 3, 2024

dvg-p4 commented Dec 3, 2024

aitap commented Dec 4, 2024

dvg-p4 commented Dec 4, 2024

data.table installation from source fails on Mac, symbol not found in flat namespace '___kmpc_barrier' #6622

data.table installation from source fails on Mac, symbol not found in flat namespace '___kmpc_barrier' #6622

Comments

dvg-p4 commented Nov 20, 2024 • edited Loading

Minimal reproducible example

Workaround

Output of sessionInfo()

aitap commented Nov 29, 2024

evg-trushkin commented Dec 2, 2024

dvg-p4 commented Dec 2, 2024 • edited Loading

dvg-p4 commented Dec 2, 2024 • edited Loading

dvg-p4 commented Dec 2, 2024

aitap commented Dec 3, 2024

dvg-p4 commented Dec 3, 2024

dvg-p4 commented Dec 3, 2024 • edited Loading

dvg-p4 commented Dec 3, 2024

aitap commented Dec 3, 2024 via email

dvg-p4 commented Dec 3, 2024 • edited Loading

aitap commented Dec 3, 2024 via email

dvg-p4 commented Dec 3, 2024

dvg-p4 commented Dec 3, 2024

aitap commented Dec 4, 2024

dvg-p4 commented Dec 4, 2024

data.table installation from source fails on Mac, `symbol not found in flat namespace '___kmpc_barrier'` #6622

data.table installation from source fails on Mac, `symbol not found in flat namespace '___kmpc_barrier'` #6622

dvg-p4 commented Nov 20, 2024 •

edited

Loading

dvg-p4 commented Dec 2, 2024 •

edited

Loading

dvg-p4 commented Dec 2, 2024 •

edited

Loading

dvg-p4 commented Dec 3, 2024 •

edited

Loading

dvg-p4 commented Dec 3, 2024 •

edited

Loading