Include envoy in (SmartOS) pkgsrc #34
Comments
|
Well, bazel is the main problem really. It only works on sanctioned and explicitly listed platforms, and - not surprisingly - SunOS is not one of them. There's probably no significant reason why it's so, but upstream is pretty clear on not wanting the job, and I'm not sure spending much time on fixing yet another crazy build system is really worth it. Remember, Bazel source tarball comes with close to 10.000 files. |
|
I should add I'm interested in hearing how you ever got close to running bazel on SmartOS. |
|
I think I'm actually pretty close to getting Bazel to work. I've looked at FreeBSD and Darwin support (which were both shoe-horned in later on). For some reason Bazel still complains about not detecting the correct OS but I suspect thats just a minor error I made somewhere. Since I have reviewed FreeBSD and Darwin support and I don't see any additional changes they made that I haven't. Got sidetracked by a bunch of other work but I hope to get back to it in a couple of days. |
|
Cool, let me know if you need a second pair of eyes for anything. |
|
I definitely underestimated the complexity of Bazel. It seems a lot of functionality was never part of the original design and were more or less duct-tapped and shoehorned in later. For example 90% the time Linux is called K8 (probably from Bazel's Kubernetes build tool roots) and other OS'es, like FreeBSD, are regarded as a "CPU". Oddly enough "arm" is the only real cpu regarded as a "CPU" by Bazel. I'm guessing thats actually Linux on ARM. Oddly enough for Windows there is an explicit x64 prefix in it's CPU name ("x64_windows") while FreeBSD x64 is just called "freebsd" (so what about FreeBSD arm support? Or ARM64?). Also Linux (K8) is kind of the default which is implied everywhere and all other things are then explicitly mentioned (Darwin, Windows, FreeBSD, etc.). I got as far as this diff, unfortunely Bazel doesn't compile with it (by running --- bazel-orig/scripts/bootstrap/buildenv.sh 2018-04-01 14:52:36.150686900 +0000
+++ bazel/scripts/bootstrap/buildenv.sh 2018-04-01 14:53:02.524832025 +0000
@@ -83,6 +83,10 @@
JAVA_HOME="${JAVA_HOME:-/usr/local/openjdk8}"
;;
+solaris)
+ JAVA_HOME="${JAVA_HOME:-/opt/local/openjdk8}"
+ ;;
+
darwin)
if [[ -z "$JAVA_HOME" ]]; then
JAVA_HOME="$(/usr/libexec/java_home -v ${JAVA_VERSION}+ 2> /dev/null)" \
diff -uar bazel-orig/scripts/packages/BUILD bazel/scripts/packages/BUILD
--- bazel-orig/scripts/packages/BUILD 2018-04-01 14:52:36.152851396 +0000
+++ bazel/scripts/packages/BUILD 2018-04-01 14:53:02.525363493 +0000
@@ -20,6 +20,7 @@
":zip-bazel-exe_with_jdk",
],
"//src/conditions:freebsd": [],
+ "//src/conditions:solaris": [],
"//src/conditions:darwin": [
":with-jdk/install.sh",
":without-jdk/install.sh",
diff -uar bazel-orig/src/conditions/BUILD bazel/src/conditions/BUILD
--- bazel-orig/src/conditions/BUILD 2018-04-01 14:52:36.170223015 +0000
+++ bazel/src/conditions/BUILD 2018-04-01 14:53:02.525807667 +0000
@@ -41,6 +41,12 @@
)
config_setting(
+ name = "solaris",
+ values = {"cpu": "solaris"},
+ visibility = ["//visibility:public"],
+)
+
+config_setting(
name = "windows",
values = {"cpu": "x64_windows"},
visibility = ["//visibility:public"],
diff -uar bazel-orig/src/conditions/BUILD.tools bazel/src/conditions/BUILD.tools
--- bazel-orig/src/conditions/BUILD.tools 2018-04-01 14:52:36.170282808 +0000
+++ bazel/src/conditions/BUILD.tools 2018-04-01 14:53:02.526128210 +0000
@@ -5,6 +5,12 @@
)
config_setting(
+ name = "solaris",
+ values = {"cpu": "solaris"},
+ visibility = ["//visibility:public"],
+)
+
+config_setting(
name = "darwin",
values = {"cpu": "darwin"},
visibility = ["//visibility:public"],
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/analysis/config/AutoCpuConverter.java bazel/src/main/java/com/google/devtools/build/lib/analysis/config/AutoCpuConverter.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/analysis/config/AutoCpuConverter.java 2018-04-01 14:52:36.216707875 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/analysis/config/AutoCpuConverter.java 2018-04-01 14:53:02.526694805 +0000
@@ -37,6 +37,8 @@
return "darwin";
case FREEBSD:
return "freebsd";
+ case SOLARIS:
+ return "solaris";
case WINDOWS:
switch (CPU.getCurrent()) {
case X86_64:
@@ -79,6 +81,8 @@
return Pair.of(CPU.getCurrent(), OS.DARWIN);
} else if (input.startsWith("freebsd")) {
return Pair.of(CPU.getCurrent(), OS.FREEBSD);
+ } else if (input.startsWith("solaris")) {
+ return Pair.of(CPU.getCurrent(), OS.SOLARIS);
} else if (input.startsWith("x64_windows")) {
return Pair.of(CPU.getCurrent(), OS.WINDOWS);
}
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java 2018-04-01 14:52:36.280856404 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java 2018-04-01 14:53:02.527808835 +0000
@@ -1064,7 +1064,7 @@
}
private static String ifLinux(CppPlatform platform, String... lines) {
- // Platform `LINUX` also includes FreeBSD.
+ // Platform `LINUX` also includes FreeBSD and Solaris.
return ifTrue(platform == CppPlatform.LINUX, lines);
}
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfiguration.java bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfiguration.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfiguration.java 2018-04-01 14:52:36.281580812 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfiguration.java 2018-04-01 14:53:02.528615420 +0000
@@ -214,6 +214,7 @@
static CppConfiguration create(CppConfigurationParameters params)
throws InvalidConfigurationException {
+ System.out.println("Target CPU:" + params.toolchain.getTargetCpu());
CrosstoolConfig.CToolchain toolchain = params.toolchain;
CppOptions cppOptions = params.cppOptions;
PathFragment crosstoolTopPathFragment =
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfigurationLoader.java bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfigurationLoader.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfigurationLoader.java 2018-04-01 14:52:36.281668789 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppConfigurationLoader.java 2018-04-01 15:04:50.380820354 +0000
@@ -143,8 +143,10 @@
CrosstoolConfigurationLoader.CrosstoolFile file =
CrosstoolConfigurationLoader.readCrosstool(env, crosstoolTopLabel);
if (file == null) {
+ System.out.println("No Crosstool file");
return null;
}
+ System.out.println("Crosstool file: '" + file.getProto().toString() + "'.");
CrosstoolConfig.CToolchain toolchain =
CrosstoolConfigurationLoader.selectToolchain(
file.getProto(), options, cpuTransformer.getTransformer());
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationIdentifier.java bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationIdentifier.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationIdentifier.java 2018-04-01 14:52:36.283483130 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationIdentifier.java 2018-04-01 14:53:02.529455917 +0000
@@ -40,6 +40,12 @@
/** Creates a new {@link CrosstoolConfigurationIdentifier} with the given parameters. */
CrosstoolConfigurationIdentifier(String cpu, String compiler, String libc) {
+ System.out.println("cpu: '" + cpu + "' compiler: '" + compiler + "' libc: '" + libc + "'");
+
+ for (StackTraceElement ste : Thread.currentThread().getStackTrace()) {
+ System.out.println(ste);
+ }
+
this.cpu = Preconditions.checkNotNull(cpu);
this.compiler = compiler;
this.libc = libc;
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationLoader.java bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationLoader.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationLoader.java 2018-04-01 14:52:36.283562611 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CrosstoolConfigurationLoader.java 2018-04-01 14:53:02.530014889 +0000
@@ -324,6 +324,7 @@
boolean convertLipoToThinLto,
Function<String, String> cpuTransformer)
throws InvalidConfigurationException {
+ System.out.println("config compiler:" + config.getCompiler() + "config libc: " + config.getLibc());
if ((config.getCompiler() != null) || (config.getLibc() != null)) {
ArrayList<CrosstoolConfig.CToolchain> candidateToolchains = new ArrayList<>();
for (CrosstoolConfig.CToolchain toolchain : release.getToolchainList()) {
@@ -356,6 +357,7 @@
// We use fake CPU values to allow cross-platform builds for other languages that use the
// C++ toolchain. Translate to the actual target architecture.
String desiredCpu = cpuTransformer.apply(config.getCpu());
+ System.out.println("CPU: '" + config.getCpu() + "' desired CPU: '" + desiredCpu + "'");
boolean needsLipo = lipoMode != LipoMode.OFF && !convertLipoToThinLto;
for (CrosstoolConfig.DefaultCpuToolchain selector : release.getDefaultToolchainList()) {
if (needsLipo && !selector.getSupportsLipo()) {
@@ -367,6 +369,7 @@
}
}
+ System.out.println("selectedIdentifier: '" + selectedIdentifier + "'");
if (selectedIdentifier == null) {
StringBuilder cpuBuilder = new StringBuilder();
for (CrosstoolConfig.DefaultCpuToolchain selector : release.getDefaultToolchainList()) {
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/util/OS.java bazel/src/main/java/com/google/devtools/build/lib/util/OS.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/util/OS.java 2018-04-01 14:52:36.327916523 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/util/OS.java 2018-04-01 14:53:02.530468640 +0000
@@ -21,11 +21,12 @@
public enum OS {
DARWIN("osx", "Mac OS X"),
FREEBSD("freebsd", "FreeBSD"),
+ SOLARIS("solaris","SunOS"),
LINUX("linux", "Linux"),
WINDOWS("windows", "Windows"),
UNKNOWN("unknown", "");
- private static final EnumSet<OS> POSIX_COMPATIBLE = EnumSet.of(DARWIN, FREEBSD, LINUX);
+ private static final EnumSet<OS> POSIX_COMPATIBLE = EnumSet.of(DARWIN, FREEBSD, SOLARIS, LINUX);
private final String canonicalName;
private final String detectionName;
diff -uar bazel-orig/src/main/java/com/google/devtools/build/lib/vfs/LocalPath.java bazel/src/main/java/com/google/devtools/build/lib/vfs/LocalPath.java
--- bazel-orig/src/main/java/com/google/devtools/build/lib/vfs/LocalPath.java 2018-04-01 14:52:36.363892385 +0000
+++ bazel/src/main/java/com/google/devtools/build/lib/vfs/LocalPath.java 2018-04-01 14:53:02.531108306 +0000
@@ -667,6 +667,7 @@
switch (OS.getCurrent()) {
case LINUX:
case FREEBSD:
+ case SOLARIS:
case UNKNOWN:
return new UnixOsPathPolicy();
case DARWIN:
diff -uar bazel-orig/src/main/tools/jdk.BUILD bazel/src/main/tools/jdk.BUILD
--- bazel-orig/src/main/tools/jdk.BUILD 2018-04-01 14:52:36.380117028 +0000
+++ bazel/src/main/tools/jdk.BUILD 2018-04-01 14:53:02.531508432 +0000
@@ -21,6 +21,11 @@
)
filegroup(
+ name = "jni_md_header-solaris",
+ srcs = ["include/solaris/jni_md.h"],
+)
+
+filegroup(
name = "jni_md_header-windows",
srcs = ["include/win32/jni_md.h"],
)
diff -uar bazel-orig/third_party/BUILD bazel/third_party/BUILD
--- bazel-orig/third_party/BUILD 2018-04-01 14:52:36.555267787 +0000
+++ bazel/third_party/BUILD 2018-04-01 14:53:02.532119695 +0000
@@ -662,6 +662,11 @@
)
config_setting(
+ name = "solaris",
+ values = {"host_cpu": "solaris"},
+)
+
+config_setting(
name = "s390x",
values = {"host_cpu": "s390x"},
)
diff -uar bazel-orig/tools/cpp/BUILD bazel/tools/cpp/BUILD
--- bazel-orig/tools/cpp/BUILD 2018-04-01 14:52:36.549549685 +0000
+++ bazel/tools/cpp/BUILD 2018-04-01 14:53:02.532556059 +0000
@@ -40,6 +40,7 @@
"armeabi-v7a|compiler": ":cc-compiler-armeabi-v7a",
"darwin|compiler": ":cc-compiler-darwin",
"freebsd|compiler": ":cc-compiler-freebsd",
+ "solaris|compiler": ":cc-compiler-solaris",
"local|compiler": ":cc-compiler-local",
"x64_windows|compiler": ":cc-compiler-x64_windows",
"x64_windows_msvc|compiler": ":cc-compiler-x64_windows_msvc",
@@ -123,6 +124,20 @@
all_files = ":empty",
compiler_files = ":empty",
cpu = "local",
+ dwp_files = ":empty",
+ dynamic_runtime_libs = [":empty"],
+ linker_files = ":empty",
+ objcopy_files = ":empty",
+ static_runtime_libs = [":empty"],
+ strip_files = ":empty",
+ supports_param_files = 0,
+)
+
+cc_toolchain(
+ name = "cc-compiler-solaris",
+ all_files = ":empty",
+ compiler_files = ":empty",
+ cpu = "local",
dwp_files = ":empty",
dynamic_runtime_libs = [":empty"],
linker_files = ":empty",
diff -uar bazel-orig/tools/cpp/BUILD.static bazel/tools/cpp/BUILD.static
--- bazel-orig/tools/cpp/BUILD.static 2018-04-01 14:52:36.549606668 +0000
+++ bazel/tools/cpp/BUILD.static 2018-04-01 14:53:02.532895942 +0000
@@ -19,6 +19,7 @@
toolchains = {
"armeabi-v7a|compiler": ":cc-compiler-armeabi-v7a",
"freebsd|compiler": ":cc-compiler-freebsd",
+ "solaris|compiler": ":cc-compiler-solaris",
"x64_windows|msvc-cl": ":cc-compiler-x64_windows",
"x64_windows|msys-gcc": ":cc-compiler-x64_windows_msys",
"x64_windows|mingw-gcc": ":cc-compiler-x64_windows_mingw",
@@ -45,6 +46,20 @@
all_files = ":empty",
compiler_files = ":empty",
cpu = "local",
+ dwp_files = ":empty",
+ dynamic_runtime_libs = [":empty"],
+ linker_files = ":empty",
+ objcopy_files = ":empty",
+ static_runtime_libs = [":empty"],
+ strip_files = ":empty",
+ supports_param_files = 0,
+)
+
+cc_toolchain(
+ name = "cc-compiler-solaris",
+ all_files = ":empty",
+ compiler_files = ":empty",
+ cpu = "local",
dwp_files = ":empty",
dynamic_runtime_libs = [":empty"],
linker_files = ":empty",
diff -uar bazel-orig/tools/cpp/CROSSTOOL bazel/tools/cpp/CROSSTOOL
--- bazel-orig/tools/cpp/CROSSTOOL 2018-04-01 14:52:36.549774320 +0000
+++ bazel/tools/cpp/CROSSTOOL 2018-04-01 14:53:02.533700016 +0000
@@ -19,6 +19,10 @@
toolchain_identifier: "local_freebsd"
}
default_toolchain {
+ cpu: "solaris"
+ toolchain_identifier: "local_solaris"
+}
+default_toolchain {
cpu: "armeabi-v7a"
toolchain_identifier: "stub_armeabi-v7a"
}
@@ -388,6 +392,126 @@
# Gold linker only? Can we enable this by default?
# linker_flag: "-Wl,--warn-execstack"
# linker_flag: "-Wl,--detect-odr-violations"
+
+ compilation_mode_flags {
+ mode: DBG
+ # Enable debug symbols.
+ compiler_flag: "-g"
+ }
+ compilation_mode_flags {
+ mode: OPT
+
+ # No debug symbols.
+ # Maybe we should enable https://gcc.gnu.org/wiki/DebugFission for opt or
+ # even generally? However, that can't happen here, as it requires special
+ # handling in Bazel.
+ compiler_flag: "-g0"
+
+ # Conservative choice for -O
+ # -O3 can increase binary size and even slow down the resulting binaries.
+ # Profile first and / or use FDO if you need better performance than this.
+ compiler_flag: "-O2"
+
+ # Disable assertions
+ compiler_flag: "-DNDEBUG"
+
+ # Removal of unused code and data at link time (can this increase binary size in some cases?).
+ compiler_flag: "-ffunction-sections"
+ compiler_flag: "-fdata-sections"
+ linker_flag: "-Wl,--gc-sections"
+ }
+ linking_mode_flags { mode: DYNAMIC }
+}
+
+toolchain {
+ abi_version: "local"
+ abi_libc_version: "local"
+ builtin_sysroot: ""
+ compiler: "compiler"
+ host_system_name: "local"
+ needsPic: true
+ supports_gold_linker: false
+ supports_incremental_linker: false
+ supports_fission: false
+ supports_interface_shared_objects: false
+ supports_normalizing_ar: false
+ supports_start_end_lib: false
+ target_libc: "local"
+ target_cpu: "solaris"
+ target_system_name: "local"
+ toolchain_identifier: "local_solaris"
+
+ tool_path { name: "ar" path: "/opt/local/bin/ar" }
+ tool_path { name: "compat-ld" path: "/usr/bin/ld" }
+ tool_path { name: "cpp" path: "/opt/local/gcc7/bin/cpp" }
+ tool_path { name: "dwp" path: "/usr/bin/dwp" }
+ tool_path { name: "gcc" path: "/opt/local/gcc7/bin/gcc" }
+ cxx_flag: "-std=c++0x"
+ linker_flag: "-lstdc++"
+ linker_flag: "-B/usr/bin/"
+
+ # TODO(bazel-team): In theory, the path here ought to exactly match the path
+ # used by gcc. That works because bazel currently doesn't track files at
+ # absolute locations and has no remote execution, yet. However, this will need
+ # to be fixed, maybe with auto-detection?
+ cxx_builtin_include_directory: "/opt/local/gcc7/lib"
+ cxx_builtin_include_directory: "/opt/local/include"
+ cxx_builtin_include_directory: "/usr/include"
+ tool_path { name: "gcov" path: "/opt/local/gcc7/bin/gcov" }
+
+ # C(++) compiles invoke the compiler (as that is the one knowing where
+ # to find libraries), but we provide LD so other rules can invoke the linker.
+ tool_path { name: "ld" path: "/usr/bin/ld" }
+
+ tool_path { name: "nm" path: "/opt/local/bin/nm" }
+ tool_path { name: "objcopy" path: "/usr/bin/objcopy" }
+ objcopy_embed_flag: "-I"
+ objcopy_embed_flag: "binary"
+ tool_path { name: "objdump" path: "/usr/bin/objdump" }
+ tool_path { name: "strip" path: "/usr/bin/strip" }
+
+ # Anticipated future default.
+ unfiltered_cxx_flag: "-no-canonical-prefixes"
+ unfiltered_cxx_flag: "-fno-canonical-system-headers"
+
+ # Make C++ compilation deterministic. Use linkstamping instead of these
+ # compiler symbols.
+ unfiltered_cxx_flag: "-Wno-builtin-macro-redefined"
+ unfiltered_cxx_flag: "-D__DATE__=\"redacted\""
+ unfiltered_cxx_flag: "-D__TIMESTAMP__=\"redacted\""
+ unfiltered_cxx_flag: "-D__TIME__=\"redacted\""
+
+ # Security hardening on by default.
+ # Conservative choice; -D_FORTIFY_SOURCE=2 may be unsafe in some cases.
+ # We need to undef it before redefining it as some distributions now have
+ # it enabled by default.
+ compiler_flag: "-U_FORTIFY_SOURCE"
+ compiler_flag: "-D_FORTIFY_SOURCE=1"
+ compiler_flag: "-fstack-protector"
+ linker_flag: "-Wl,-z,relro,-z,now"
+
+ # Enable coloring even if there's no attached terminal. Bazel removes the
+ # escape sequences if --nocolor is specified. This isn't supported by gcc
+ # on Ubuntu 14.04.
+ # compiler_flag: "-fcolor-diagnostics"
+
+ # All warnings are enabled. Maybe enable -Werror as well?
+ compiler_flag: "-Wall"
+ # Enable a few more warnings that aren't part of -Wall.
+ compiler_flag: "-Wunused-but-set-parameter"
+ # But disable some that are problematic.
+ compiler_flag: "-Wno-free-nonheap-object" # has false positives
+
+ # Keep stack frames for debugging, even in opt mode.
+ compiler_flag: "-fno-omit-frame-pointer"
+
+ # Anticipated future default.
+ linker_flag: "-no-canonical-prefixes"
+ # Have gcc return the exit code from ld.
+ linker_flag: "-pass-exit-codes"
+ # Gold linker only? Can we enable this by default?
+ # linker_flag: "-Wl,--warn-execstack"
+ # linker_flag: "-Wl,--detect-odr-violations"
compilation_mode_flags {
mode: DBG
diff -uar bazel-orig/tools/cpp/cc_configure.bzl bazel/tools/cpp/cc_configure.bzl
--- bazel-orig/tools/cpp/cc_configure.bzl 2018-04-01 14:52:36.550063824 +0000
+++ bazel/tools/cpp/cc_configure.bzl 2018-04-01 14:53:02.534036692 +0000
@@ -24,7 +24,7 @@
Label("@bazel_tools//tools/cpp:dummy_toolchain.bzl"), "dummy_toolchain.bzl")
env = repository_ctx.os.environ
cpu_value = get_cpu_value(repository_ctx)
- if cpu_value == "freebsd":
+ if (cpu_value == "freebsd" or cpu_value == "solaris"):
# This is defaulting to the static crosstool, we should eventually
# autoconfigure this platform too. Theorically, FreeBSD should be
# straightforward to add but we cannot run it in a docker container so
diff -uar bazel-orig/tools/jdk/BUILD bazel/tools/jdk/BUILD
--- bazel-orig/tools/jdk/BUILD 2018-04-01 14:52:36.548335741 +0000
+++ bazel/tools/jdk/BUILD 2018-04-01 14:53:02.534423986 +0000
@@ -77,6 +77,11 @@
)
alias(
+ name = "jni_md_header-solaris",
+ actual = "@local_jdk//:jni_md_header-solaris",
+)
+
+alias(
name = "java",
actual = "@local_jdk//:java",
)
diff -uar bazel-orig/tools/platforms/BUILD bazel/tools/platforms/BUILD
--- bazel-orig/tools/platforms/BUILD 2018-04-01 14:52:36.549193491 +0000
+++ bazel/tools/platforms/BUILD 2018-04-01 14:53:02.534848733 +0000
@@ -56,6 +56,11 @@
)
constraint_value(
+ name = "solaris",
+ constraint_setting = ":os",
+)
+
+constraint_value(
name = "linux",
constraint_setting = ":os",
)
@@ -83,6 +88,7 @@
os_constraints = [
":osx",
":freebsd",
+ ":solaris",
":linux",
":windows",
],
@@ -100,6 +106,7 @@
os_constraints = [
":osx",
":freebsd",
+ ":solaris",
":linux",
":windows",
],
diff -uar bazel-orig/tools/platforms/platforms.BUILD bazel/tools/platforms/platforms.BUILD
--- bazel-orig/tools/platforms/platforms.BUILD 2018-04-01 14:52:36.549245186 +0000
+++ bazel/tools/platforms/platforms.BUILD 2018-04-01 14:53:02.535174740 +0000
@@ -46,6 +46,11 @@
)
constraint_value(
+ name = "solaris",
+ constraint_setting = ":os",
+)
+
+constraint_value(
name = "linux",
constraint_setting = ":os",
)
@@ -73,6 +78,7 @@
os_constraints = [
":osx",
":freebsd",
+ ":solaris",
":linux",
":windows",
],
@@ -90,6 +96,7 @@
os_constraints = [
":osx",
":freebsd",
+ ":solaris",
":linux",
":windows",
],
|
|
So turns out I couldn't let this rest ;-) I've gotten past the point I was stuck and I've created bazelbuild/bazel#4969 and bazelbuild/bazel#4967 to fix some POSIX incompatibilities in Bazel to test the waters to see if they're open to such PR's. Other communities like NetBSD might also be interested in those changes. |
|
You truly are determined! :) |
|
I came across this gem in the Bazel Wiki:
But after a whole bunch of changes I'm getting somewhere! However I could use your help with the following error I'm bumping in to (see below). |
|
You shouldn't need to, GCC figures out by itself where to find libgcc, unless it is told not to look in the default locations. All of the |
|
@jperkin Thanks for the assist! I'll investigate where those flags are coming from. I think the issue with Bazel is that it is (unfortunately for us) not just another build system. It is different from other build systems in the regard that Bazel defines the entire toolchain (compiler flags, linker flags, etc.). That's a fundamentally different view of things then for example autoconf which will try and detect the environment it is in and adapt to that. Bazel's root's are with Google which uses it to build all projects in their giant GIT mono-repo. I have currently defined a SmartOS toolchain with GCC 4.9 in Bazel but for clang support a new toolchain will have to be defined inside the Bazel config. So getting it to work is one thing but getting it into pkgsrc in such a way that it will be maintainable is going to be another. |
|
The paths seem to be correct (see https://gist.github.com/siepkes/956b3bbd17072e8553b4e8f2104025f7). One of the paths is BTW I tried cheating by creating a symlink called |
|
Yes, it's libgcc_s, but one thing is that ideally the linker should not be called directly but rather through GCC. That's where GCC automatically links in any required objects with proper paths. If Bazel tries to call |
|
As far as I can tell the linker is called via gcc (see: https://gist.github.com/siepkes/956b3bbd17072e8553b4e8f2104025f7 ) or am I missing something? |
|
In that case I'd just patch out any attempts to link in -lgcc and the like. Unless Bazel also "helpfully" uses something like -nostdlib or -nodefaultlibs, the standard GCC libs like libgcc_s or libstdc++ should be linked in automatically. The entire "-lgcc -lgcc_eh -lc -lgcc -lgcc_eh" part sounds fishy. |
|
I think I figured out where those flags are coming from. Because gcc in pkgsrc is configured with
|
|
That's not about how GCC was configured, but yes, if something specifies -static or -static-libgcc, it gets expanded that way. Where's this particular protobuf coming from? The one from pkgsrc doesn't call for static linking - and nothing really should ideally. |
|
I read here that if GCC itself was configured with Building Bazel starts with Bazel bootstrapping a (smaller?) Bazel binary. With this bootstrapped Bazel binary it will build the actual Bazel project (the |
|
It's OK to mix shared and static linking when building a binary, so it should be possible to let Bazel link a static protobuf archive into its binary, but link libgcc_s dynamically. You can use the -Bstatic and -Bdynamic linker args to switch behavior (e.g. -Bstatic turns on static linking for all subsequent -l args until -Bdynamic is encountered). Or - have Bazel use 'our' system protobuf lib (from pkgsrc) and link to it dynamically. (In the pkgsrc world, dynamic linking is should be used when possible.) Back to the previous problem - it should be possible to build an entirely static binary that has the libgcc and libgcc_eh objects linked in. The compiler does have those libs - just look at |
|
Woohoo! I've got a working Bazel binary on SmartOS! I've created a gist with the patch (which applies to |
|
How rude of me! In my excitement I forget to thank you guys for the help in getting this to work!...So thanks! ;-) |
|
You don't really have to worry about multiple compiler versions. The only sane pkgsrc scenario makes you have a single compiler version installed and used for everything, otherwise you're facing hell with libs linking against each other and against different compiler libs. So if somebody needs to a build a (future) pkgsrc Envoy package that requires GCC 5, they need to start with a GCC 5+ toolchain in the beginning. |
|
Oh and that's a ugly big patch. Looks good though (the /opt/local paths should not be hardcoded but that's a different thing of course). Let's hope it works and upstream will accept most of it. Good job! |
|
Yeah the patch will definitely be chopped up in to some smaller pieces. I already have 2 POSIX compatibility PR's which got merged separately in to Bazel. These are also still in this patch. There is one other POSIX compatibility change which I need to create a PR for. There are also fixes in it to get I will also look in to rebasing to the latest Bazel version. |
|
Bit of radio silence but I'm still working on this. I'm currently trying to build Envoy with my Solaris Bazel. Good way to test if I can actually build a reallife application with my Bazel port and didn't miss anything. After which I can try to package Bazel in to pkgsrc. I need the help of you guys on what I suspect is an incompatibility between Solaris and Illumos (goes a bit beyond my current knowledge). One of the dependencies Envoy uses is gperftools. Specifically it requires the archive: |
|
Yep, see this Illumos ticket. |
|
Thanks! That explains whats happening. From what I understand in the ticket if I submit a PR to upstream gperftools to get Illumos working properly (ie. by explicitly including |
|
See this comment in ucontext.h. It'd be better if the configure script didn't prefer sys/ucontext.h (which it does for the sake of Darwin systems, if I understand correctly), but rather through the ucontext.h wrapper, then no futher include should be needed: https://github.com/illumos/illumos-gate/blob/master/usr/src/head/ucontext.h#L36 |
|
I agree that it's nicer (as far as I can tell) but I think that is going to be a hard sell to upstream. For one I can't oversee what changing the order might break on other platforms? I think the best we can get into upstream if we want to go the Sorry for all the questions but this part is definitely something I still need to gain some experience. |
|
I believe the standard practice on SunOS should be to include <ucontext.h> rather than the other one. There should be no harm in doing it this way on legacy SunOS platforms. |
|
Sorry for the silence (life got in the way of my Envoy ambitions ;-) but I finally got Envoy working on SmartOS! This is the output for the Envoy getting started tutorial: Had to disable hot restart for now since there is apparently some issue how Envoy tries to bind the socket: But thats just some minor fixing and improving that needs to be done. Overall the thing seems to work! I had to fork libeevent, boringssl, gperftools, grpc and (obviously) envoy itself to make changes to make them Solaris compatible. So now that the fun is over I'll need to polish these changes and get them upstream. @mamash @jperkin Bazel for Envoy is configured to create a statically linked Envoy binary. While I guess it is possible to create a Bazel config which creates a shared linked Envoy binary it is going to be hard and a lot of work to maintain. The first hurdle being that Bazel is basically designed like Go; It wants to create statically linked binaries. Just like one might be able to use a tennis racket to drive a nail into a wall it's still going to be a painful experience. Envoy itself cares even less about shared libraries then Bazel. Which from their perspective kinda makes sense because Envoy is often deployed inside a docker container as a sidecar; You don't want to run in to dependency hell with your sidecar. We can probably debate if this is a good idea or not but it doesn't really matter what our opinion on it is since thats just the way Envoy (and the container world for a large part, certainly the parties backing Envoy like Google) currently work. So aside from getting stuff upstream, etc. how do you guys want to approach this from a pkgsrc perspective? I'm no pkgsrc expert but I would advise to just let Bazel and Envoy generate their statically linked things and package that. |
|
Yes, I don't think it's worth the effort to insist on dynamic linking if it involves fighting a principle design of a tool. So right now how do your bazel and envoy binaries look like linking wise - no dynamic libs linked beyond libc/libm and maybe libgcc_s? It's not entirely unheard of in pkgsrc to have a package that builds some dependecies embedded from source bundled with the distribution (e.g. lately I had to switch lang/nodejs to use OpenSSL embedded, because it requires 1.1.x). It may get tricky if a package requires dependencies to exist, but insists on linking them statically. |
|
Static linking of dependencies should not be a problem in building. The problem is that when there's an issue in a dependency, and it's updated, it requires rebuilding the depending package to get the fix. This is why statically linking -- whether of an actual dependency package or an included copy -- is discouraged in pkgsrc. Stepping back, if I read correctly (too quickly), it seems like this package is almost turning into a mild fork, with changes that upstream is unwilling to take. If that's true, it may be better to actually do that (a following fork) and then package the fork, rather than trying to maintain a fork in a package. (I don't speak for joyent's pkgsrc, but that's what I'd say about pkgsrc proper.) |
|
@mamash Below is how the binary currently looks. It's currently in the im-still-surprised-I-even-got-it-to-work-and-still-needs-polishing state ;-). Very open to feedback though! I'm currently a bit swamped at work but I'm going to plan a couple of days of from work in 1 or 2 weeks so I can get this all properly wrapped up. I'm 80% done, now I just need to finish the other 80% ;-). $ ldd ./bazel-bin/source/exe/envoy-static
librt.so.1 => /lib/64/librt.so.1
libdl.so.1 => /lib/64/libdl.so.1
libpthread.so.1 => /lib/64/libpthread.so.1
libm.so.2 => /lib/64/libm.so.2
libstdc++.so.6 => /opt/local/gcc7//lib/amd64/libstdc++.so.6
libxnet.so.1 => /lib/64/libxnet.so.1
libsocket.so.1 => /lib/64/libsocket.so.1
libnsl.so.1 => /lib/64/libnsl.so.1
libc.so.1 => /lib/64/libc.so.1
libgcc_s.so.1 => /opt/local/gcc7//lib/amd64/libgcc_s.so.1
libmp.so.2 => /lib/64/libmp.so.2
libmd.so.1 => /lib/64/libmd.so.1 |
|
Just to let you guys know I'm still on this. I'm actually trying to get the SmartOS / Illumos / Solaris patches for gRPC upstreamed which is proving somewhat harder then I had hoped. I might just try and get the Bazel things upstreamed or if they don't want them at least the stuff that fixes a bunch of POSIX compliance. I've rebased my patch to Bazel 0.16 in case your interested: https://gist.github.com/siepkes/0d144125974b2a84a92433e0282804b2 |
|
For reference here are the repositories with (working) ports which I keep updated with releases of upstream:
Solaris gRPC support was actually upstreamed. |
|
@siepkes Now that Hashicorp Consul is using Envoy as a Service Mesh Proxy I could see this being used more by the "DevOpers" of the world. Any word on getting packages added to pkgsrc? Great Job doing this. Bazel is brutal. |
|
@Smithx10 I share the feeling that Envoy will only become more relevant in the future. It's also slowly eating away edge proxy market share of NGINX, haproxy and the likes. I've dropped the ball a bit (ok a lot) on getting Envoy included in pkgsrc so there isn't much progress there. I am however keeping Envoy up to date with upstream and Bazel as required (ie. I bump Bazel when Envoy needs it). Bazel is mostly hard to keep up because upstream Bazel is well...let's say....not afraid to refactor their code base 😉. I'm hoping that since they are now in the 1.X version range the internals will stabilize a bit, making porting easier. Ideally I would like to upstream it to Bazel as Illumos support however there are a bunch of upstream dependencies that will first require accepting certain patches (for example madler/zlib#395). I'm not really a pkgsrc packaging pro and both Bazel and Envoy aren't the easiest thing to package. So if anyone wants to take a crack at putting Bazel and Envoy in pkgsrc I'd be happy to assist! |
|
@siepkes @danmcd noted to me in IRC that the following bug will really make using Envoy SmartOS problematic. envoyproxy/envoy#1446 Did you resolve this in your fork? |
NOTE: There's #ifdef APPLE in the relevant code that just needs to include illumos as well. |
|
@Smithx10 Yes, that particular issue is fixed in the fork. The fix which @danmcd mentions is located here: siepkes/envoy-smartos@a70767a#diff-3315c7b291f78a482ebd4cbfc0545362 |
Envoy describes itself as "an opensource edge and service proxy designed for cloud native applications". Envoy usage is becoming quite wide spread (some go as far as to say the defacto standard). It was originally developed at Lyft but is now part of the Cloud Native Computing Foundation.
Long story short; It's a cloud native haproxy and NGINX alternative which can be used with for example HTTP/2 and is also popular for use with gRPC (Google's OSS Protobuf + HTTP/2 RPC implementation).
Aside from Lyft's own success story examples of it's popularity are Google and IBM creating Istio which uses Envoy to create a service mesh, a bunch of other success stories and online tutorials being produced by organisations like RedHat.
I've attempted to get Envoy to build on Illumos / SmartOS but ran in to issues. It was quite some time ago so I don't remember the exact issue. I'll give it a shot again and report my findings here. Envoy uses Bezel for it's build. Bezel is Java based, being primarily a Java dev Bezel shouldn't be too much of an issue for me. However for some of the subtleties of creating patches for Envoy I will require assistance.
The text was updated successfully, but these errors were encountered: