installing tfx 1.13.0 by pip takes so much time

[anwaralzalek]

System information

Have I specified the code to reproduce the issue (Yes, No): Yes
Environment in which the code is executed: linux Ubuntu
TFX Version: 1.13
Python version: 3.8.16
pip version: 23.3.1
Python dependencies (from pip freeze output):
pip freeze output:
asttokens==2.4.1
backcall==0.2.0
comm==0.2.0
debugpy==1.8.0
decorator==5.1.1
executing==2.0.1
importlib-metadata==6.8.0
ipykernel==6.27.1
ipython==8.12.3
jedi==0.19.1
jupyter_client==8.6.0
jupyter_core==5.5.0
matplotlib-inline==0.1.6
nest-asyncio==1.5.8
packaging==23.2
parso==0.8.3
pexpect==4.9.0
pickleshare==0.7.5
platformdirs==4.0.0
prompt-toolkit==3.0.41
psutil==5.9.6
ptyprocess==0.7.0
pure-eval==0.2.2
Pygments==2.17.2
python-dateutil==2.8.2
pyzmq==25.1.1
six==1.16.0
stack-data==0.6.3
tornado==6.4
traitlets==5.14.0
typing_extensions==4.8.0
wcwidth==0.2.12
zipp==3.17.0

Describe the current behavior
pip install tfx==1.13.0
taking hours to install tfx and solve dependencies on a new clean virtualenv

Describe the expected behavior
a month ago, the same line of pip installation was taking few minutes.
this has been tested on different machines.

Standalone code to reproduce the issue

virtualenv venv_test -p 3.8
source venv_test/bin/activate
pip install tfx==1.13.0

[zamanimd]

I have this problem too, and I tried to install TFX on the docker container base on Python 3.8, 3.9 with any version of TFX 1.14.0, 1.13.0 , ...

[singhniraj08]

@anwaralzalek, @zamanimd,

Similar issue #6468 is already created to address this issue. We are aware on working on fixing this issue. Meanwhile the workaround would be to use Colab environment for local development.

Requesting you to close this issue and follow similar issue for updates. Thank you!

[CRSilkworth]

@singhniraj08 Are there any recommended workarounds to non local dev environments (e.g. github actions, building images, etc.)

Even with colab, I think 1.13.0 is not available for python3.10 (the colab python version). Only 1.14.0, but I can't even install that it seems. I don't really use notebooks so I could be doing something wrong.

This seems like a slightly different issue to 6468.

[singhniraj08]

@CRSilkworth, I tried running the notebook you shared, You can ignore the pip dependency errors and TFX will work fine if you install it as given below. And yes it will install TFX 1.14.0. TFX 1.13 release won't work in colab because of Python dependency issues. I will keep this issue open as well and update it once we have some workaround or fix. Thank you!

!pip install tfx==1.14.0
!pip install -U google-cloud-aiplatform "shapely<2" -q
# restart runtime
from tfx import v1 as tfx
print(tfx.__version__)

[AniketARS]

conda create -n tfx_pipeline python=3.8
conda activate tfx_pipeline
pip --default-timeout=5000 --use-deprecated=legacy-resolver install tfx

worked for me in local environment.

(tfx_pipeline) aniket@ars:~$ pip freeze
absl-py==1.4.0
anyio==4.1.0
apache-beam==2.52.0
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
astunparse==1.6.3
async-lru==2.0.4
attrs==21.4.0
Babel==2.13.1
backcall==0.2.0
backports.zoneinfo==0.2.1
beautifulsoup4==4.12.2
bleach==6.1.0
cachetools==5.3.2
certifi==2023.11.17
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
cloudpickle==2.2.1
comm==0.2.0
crcmod==1.7
debugpy==1.8.0
decorator==5.1.1
defusedxml==0.7.1
dill==0.3.1.1
dnspython==2.4.2
docker==4.4.4
docopt==0.6.2
exceptiongroup==1.2.0
fastavro==1.9.1
fasteners==0.19
fastjsonschema==2.19.0
flatbuffers==23.5.26
gast==0.4.0
google-api-core==2.15.0
google-api-python-client==1.12.11
google-apitools==0.5.32
google-auth==2.25.1
google-auth-httplib2==0.1.1
google-auth-oauthlib==1.0.0
google-cloud-aiplatform==1.37.0
google-cloud-bigquery==2.34.4
google-cloud-bigquery-storage==2.23.0
google-cloud-bigtable==2.21.0
google-cloud-core==2.4.1
google-cloud-datastore==2.18.0
google-cloud-dlp==3.14.0
google-cloud-language==2.12.0
google-cloud-pubsub==2.18.4
google-cloud-pubsublite==1.8.3
google-cloud-recommendations-ai==0.10.6
google-cloud-resource-manager==1.11.0
google-cloud-spanner==3.40.1
google-cloud-storage==2.13.0
google-cloud-videointelligence==2.12.0
google-cloud-vision==3.5.0
google-crc32c==1.5.0
google-pasta==0.2.0
google-resumable-media==2.6.0
googleapis-common-protos==1.62.0
grpc-google-iam-v1==0.13.0
grpcio==1.60.0
grpcio-status==1.60.0
h5py==3.10.0
hdfs==2.7.3
httplib2==0.22.0
idna==3.6
importlib-metadata==7.0.0
importlib-resources==6.1.1
ipykernel==6.27.1
ipython==7.34.0
ipython-genutils==0.2.0
ipywidgets==7.8.1
jedi==0.19.1
Jinja2==3.1.2
joblib==1.3.2
Js2Py==0.74
json5==0.9.14
jsonschema==4.20.0
jsonschema-specifications==2023.11.2
jupyter-events==0.9.0
jupyter-lsp==2.2.1
jupyter_client==8.6.0
jupyter_core==5.5.0
jupyter_server==2.12.1
jupyter_server_terminals==0.4.4
jupyterlab==4.0.9
jupyterlab-widgets==1.1.7
jupyterlab_pygments==0.3.0
jupyterlab_server==2.25.2
keras==2.15.0
keras-tuner==1.4.6
kt-legacy==1.0.5
kubernetes==12.0.1
libclang==16.0.6
Markdown==3.5.1
MarkupSafe==2.1.3
matplotlib-inline==0.1.6
mistune==3.0.2
ml-metadata==1.14.0
ml-pipelines-sdk==1.14.0
nbclient==0.9.0
nbconvert==7.12.0
nbformat==5.9.2
nest-asyncio==1.5.8
notebook==7.0.6
notebook_shim==0.2.3
numpy==1.24.4
oauth2client==4.1.3
oauthlib==3.2.2
objsize==0.6.1
opt-einsum==3.3.0
orjson==3.9.10
overrides==6.5.0
packaging==20.9
pandas==1.5.3
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.9.0
pickleshare==0.7.5
Pillow==10.1.0
pkgutil_resolve_name==1.3.10
platformdirs==4.1.0
portpicker==1.6.0
prometheus-client==0.19.0
prompt-toolkit==3.0.41
proto-plus==1.23.0
protobuf==4.25.1
psutil==5.9.6
ptyprocess==0.7.0
pyarrow==10.0.1
pyarrow-hotfix==0.6
pyasn1==0.5.1
pyasn1-modules==0.3.0
pycparser==2.21
pydot==1.4.2
pyfarmhash==0.3.2
Pygments==2.17.2
pyjsparser==2.7.1
pymongo==4.6.1
pyparsing==3.1.1
python-dateutil==2.8.2
python-json-logger==2.0.7
pytz==2023.3.post1
PyYAML==6.0.1
pyzmq==25.1.2
referencing==0.32.0
regex==2023.10.3
requests==2.31.0
requests-oauthlib==1.3.1
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rpds-py==0.13.2
rsa==4.9
scipy==1.10.1
Send2Trash==1.8.2
shapely==2.0.2
six==1.16.0
sniffio==1.3.0
soupsieve==2.5
sqlparse==0.4.4
tensorboard==2.13.0
tensorboard-data-server==0.7.2
tensorflow==2.13.1
tensorflow-data-validation==1.14.0
tensorflow-estimator==2.13.0
tensorflow-hub==0.13.0
tensorflow-io-gcs-filesystem==0.34.0
tensorflow-metadata==1.14.0
tensorflow-model-analysis==0.45.0
tensorflow-serving-api==2.13.1
tensorflow-transform==1.14.0
termcolor==2.4.0
terminado==0.18.0
tfx==1.14.0
tfx-bsl==1.14.0
tinycss2==1.2.1
tomli==2.0.1
tornado==6.4
traitlets==5.14.0
typing_extensions==4.8.0
tzlocal==5.2
uritemplate==3.0.1
urllib3==2.1.0
wcwidth==0.2.12
webencodings==0.5.1
websocket-client==1.7.0
Werkzeug==3.0.1
widgetsnbextension==3.6.6
wrapt==1.16.0
zipp==3.17.0
zstandard==0.22.0