Skip to content

Sync protobuf version with TF for the OSX build. #168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
copybara-service wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Sync protobuf version with TF for the OSX build. #168

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,25 @@ module(
bazel_dep(name = "rules_proto", version = "7.0.2")
bazel_dep(name = "rules_python", version = "0.37.0")
bazel_dep(name = "platforms", version = "0.0.10")
bazel_dep(name = "protobuf", version = "28.3")
# TF proto version is protobuf-3.21.9, and per proto engdoc
# https://protobuf.dev/support/version-support/
# The major version saparation happens at 21.x == 3.21
# so we pick 21.7, which is the same generation as 3.21.9
bazel_dep(name = "protobuf", version = "21.7")
Comment on lines +27 to +31
Copy link
Contributor

@mtsokol mtsokol Aug 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dryman!

With @iindyk we purposefully upgraded protobuf version (we need it for conda-forge feedstocks). The good news is that a few days ago tensorflow 2.20.0rc0 was released with protobuf-6.31.1. We now use this TF version for testing.

tensorflow/tensorflow@50bd166

bazel_dep(name = "googletest", version = "1.15.2")
bazel_dep(name = "abseil-cpp", version = "20240722.0")
bazel_dep(name = "abseil-py", version = "2.1.0")
bazel_dep(name = "eigen", version = "3.4.0.bcr.3")
bazel_dep(name = "riegeli", version = "0.0.0-20241218-3385e3c")
bazel_dep(name = "pybind11_bazel", version = "2.12.0")

single_version_override(
module_name = "riegeli",
patch_strip = 1,
patches = ["//patches:riegeli.patch"],
version = "0.0.0-20241218-3385e3c",
)

SUPPORTED_PYTHON_VERSIONS = [
"3.10",
"3.11",
Expand Down
7 changes: 4 additions & 3 deletions oss/build_whl.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ function main() {
write_to_bazelrc "build --cxxopt=-std=c++17"
write_to_bazelrc "build --host_cxxopt=-std=c++17"
write_to_bazelrc "build --experimental_repo_remote_exec"
write_to_bazelrc "common --check_direct_dependencies=error"
# write_to_bazelrc "common --check_direct_dependencies=error"
Copy link
Contributor

@mtsokol mtsokol Aug 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks downstream, especially Grain, which installs a protobuf version that is incompatible with 21.7.

PLATFORM="$(uname)"

if [ -n "${CROSSTOOL_TOP}" ]; then
Expand Down Expand Up @@ -91,11 +91,12 @@ function main() {
$PYTHON_BIN -c 'import array_record'
$PYTHON_BIN -c 'from array_record.python import array_record_data_source'
# TF is not available on Python 3.13 and above.
if [ "$(uname)" != "Darwin" ] && (( "${PYTHON_MINOR_VERSION}" < 13 )); then
# if [ "$(uname)" != "Darwin" ] && (( "${PYTHON_MINOR_VERSION}" < 13 )); then
if (( "${PYTHON_MINOR_VERSION}" < 13 )); then
Comment on lines +94 to +95
Copy link
Contributor

@mtsokol mtsokol Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once google/grain#954 is merged and Grain gets a pre-release we can actually have this change.

$PYTHON_BIN -m pip install jax tensorflow grain
$PYTHON_BIN oss/test_import_grain.py
$PYTHON_BIN oss/test_import_tensorflow.py
fi
}

main
main
1 change: 1 addition & 0 deletions patches/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package(licenses = ["notice"])
13 changes: 13 additions & 0 deletions patches/riegeli.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff --git a/MODULE.bazel b/MODULE.bazel
index 2c1f801c..5d864a1c 100644
--- a/MODULE.bazel
+++ b/MODULE.bazel
@@ -44,7 +44,7 @@ bazel_dep(
)
bazel_dep(
name = "protobuf",
- version = "28.3",
+ version = "21.7",
repo_name = "com_google_protobuf",
)
bazel_dep(
Loading