setup failure on macOS 13.5.2, commit d7c81a2

[sayantanghosh]

Hello,

I know that building from source has has failed on macOS and been addressed for issue #92 and #7, but I have a setup failure on the latest commit #d7c81a2.

My system specifications are the following:

conda --version: conda 23.7.3
bazel --version: bazel 6.3.2-homebrew
python3 --version: Python 3.11.4
tf.__version__: 2.13.0 

The output of python3 -m pip install . --log install.log is attached here.

I will be grateful for any help.

Thanks in advance!!

install.log

[sayantanghosh]

I dug a further into the origin of the failed installation and found that the issue is with apache-beam which during the tensorflow_gnn installation depends on dill=0.3.1.1. But according to an open issue for apache-beam, there have been a lot of breaking changes between dill=0.3.1.1 and the current dill=0.3.7, which causes a failed installation of apache-beam, which in turn is the bottleneck in the failed installation of tensorflow_gnn.

On a side note, I noticed that tensorflow_gnn depends on torch<2. But I need both torch=2.0.1. and tensorflow_gnn in a single environment. Is there any planned upgrade of tensorflow_gnn to depend on torch>=2.

[phanein]

Hi @sayantanghosh,

Thanks for reporting this -- these are some unfortunate dependency problems for sure. I suspect the root cause is depending on an older version of beam because of protobuf dependencies.

Only a small part of the library actually uses apache-beam. If you're not using distributed sampling, the first thing I would try would be a "beam-less" install. Just edit setup.py and remove the line with 'apache-beam<2.47.0',. I tried this out with a fresh source install and it didn't break too much (beware though -- there may be bonus problems with the protobuf library (I had to pin to 3.20.*)).

W.r.t. the torch dependency, that's another case where things are not too tightly coupled. (Its referenced in a dataset converter for PyG datasets.) There isn't much tying us to torch<2, but its not really an active area of development. I do suspect this could be excised fairly easily too.

I'll talk to the team about a minimalist package for the library.

[mihirparadkar]

Hi @sayantanghosh

The torch<2 in requirements-dev.txt is just the known configuration in which our tests work (not an installation requirement). There might be some way to use torch 2.0.1 with tfgnn, but there are some known issues with importing both torch and tf (one of which is described in https://keras.io/keras_core/announcement/ ).

[sayantanghosh]

Thanks @phanein: I will give it a shot tomorrow and post an update. Just checked that my protobuf=4.24.*. I wonder what else will break by restricting to protobuf=3.20.*!
Thanks @mihirparadkar: Ah thanks! I have encountered the issues, and don't import torch and tensorflow in the same code. I just install both of them in the same conda environment because I frequently have to convert pytorch implementations to tf since I am more comfortable with tf subclassing.

[phanein]

I wasn't starting with a completely fresh environment so I'm not sure if you'll have the same problem. I suspect there is a working combination out there possible. Just make sure to use whatever protobuf works with your version of TF.

(TF-GNN doesn't have any specific requirements to the best of my knowledge, but other packages like beam, vizier, etc might have conflicting dependencies.)

[arnoegw]

It's been a year... is this still current?

[sayantanghosh]

Thanks @arnoegw for your query. It seems fine when I install it with pip on MacOS 13.7 (22H123) on intel MacBook Pro (2019), with the following configuration:

conda --version: conda 24.5.0
conda --version: Python 3.11.10
tf.__version__: TensorFlow 2.16.2

I did not try installing it from source this time, so I can try it on the coming weekend.

It's been a year... is this still current?

[alemkakti]

I am struggling to understand why do you need torch in requirements-dev.txt in the first place.

[sayantanghosh]

Hi @sayantanghosh,

Thanks for reporting this -- these are some unfortunate dependency problems for sure. I suspect the root cause is depending on an older version of beam because of protobuf dependencies.

Only a small part of the library actually uses apache-beam. If you're not using distributed sampling, the first thing I would try would be a "beam-less" install. Just edit setup.py and remove the line with 'apache-beam<2.47.0',. I tried this out with a fresh source install and it didn't break too much (beware though -- there may be bonus problems with the protobuf library (I had to pin to 3.20.*)).

W.r.t. the torch dependency, that's another case where things are not too tightly coupled. (Its referenced in a dataset converter for PyG datasets.) There isn't much tying us to torch<2, but its not really an active area of development. I do suspect this could be excised fairly easily too.

I'll talk to the team about a minimalist package for the library.

@alemkakti See the response from the team earlier.