Upgrade redis.submodule to 6.2.6 #153

tvd0x2a · 2021-10-16T09:42:20Z

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

Upgrade redis to support installation on devices with Apple M1 chips. Fixes #152.

I ran python build_scripts/update_redis_server.py and then changed the version in setup.cfg.

I was not able to test python versions 3.6 or 3.7 on an Apple M1 device. I was able to test with python 3.8 and 3.9, and tests passed.

tox output

$ tox -e py38,py39

GLOB sdist-make: /Users/auser/src/redislite/setup.py
py38 inst-nodeps: /Users/auser/src/redislite/.tox/.tmp/package/1/redislite-6.2.6.zip
py38 installed: certifi==2021.10.8,charset-normalizer==2.0.7,cov-core==1.15.0,coverage==5.5,coveralls==3.2.0,docopt==0.6.2,idna==3.3,nose==1.3.7,nose-cov==1.6,psutil==5.8.0,redis==3.5.3,redislite @ file:///Users/auser/src/redislite/.tox/.tmp/package/1/redislite-6.2.6.zip,requests==2.26.0,urllib3==1.26.7
py38 run-test-pre: PYTHONHASHSEED='3826347625'
py38 run-test: commands[0] | nosetests --exe --with-xunit --xunit-file=nosetests.xml --with-coverage --cover-xml --cover-erase --cover-package=redislite --cover-xml-file=cobertura.xml tests
..........................................
Name                             Stmts   Miss Branch BrPart  Cover
------------------------------------------------------------------
redislite/__init__.py               22      2      0      0    91%
redislite/__init__/__init__.py      22      2      0      0    91%
redislite/client.py                198      0     52      2    99%
redislite/configuration.py          29      0     10      0   100%
redislite/debug.py                  25      0      4      0   100%
redislite/patch.py                  45      0     16      0   100%
------------------------------------------------------------------
TOTAL                              341      4     82      2    99%
----------------------------------------------------------------------
Ran 42 tests in 7.708s

OK
py39 inst-nodeps: /Users/auser/src/redislite/.tox/.tmp/package/1/redislite-6.2.6.zip
py39 installed: certifi==2021.10.8,charset-normalizer==2.0.7,cov-core==1.15.0,coverage==5.5,coveralls==3.2.0,docopt==0.6.2,idna==3.3,nose==1.3.7,nose-cov==1.6,psutil==5.8.0,redis==3.5.3,redislite @ file:///Users/auser/src/redislite/.tox/.tmp/package/1/redislite-6.2.6.zip,requests==2.26.0,urllib3==1.26.7
py39 run-test-pre: PYTHONHASHSEED='3826347625'
py39 run-test: commands[0] | nosetests --exe --with-xunit --xunit-file=nosetests.xml --with-coverage --cover-xml --cover-erase --cover-package=redislite --cover-xml-file=cobertura.xml tests
..........................................
Name                             Stmts   Miss Branch BrPart  Cover
------------------------------------------------------------------
redislite/__init__.py               22      2      0      0    91%
redislite/__init__/__init__.py      22      2      0      0    91%
redislite/client.py                198      0     52      2    99%
redislite/configuration.py          29      0     10      0   100%
redislite/debug.py                  25      0      4      0   100%
redislite/patch.py                  45      0     16      0   100%
------------------------------------------------------------------
TOTAL                              341      4     82      2    99%
----------------------------------------------------------------------
Ran 42 tests in 7.780s

OK
___________________________________ summary ____________________________________
  py38: commands succeeded
  py39: commands succeeded
  congratulations :)

tvd0x2a · 2021-10-16T09:52:08Z

build_scripts/update_redis_server.py

@@ -7,7 +7,7 @@
 import tempfile


-url = 'http://download.redis.io/releases/redis-stable.tar.gz'
+url = 'http://download.redis.io/releases/redis-6.2.6.tar.gz'


Note that this URL is changed.

It looks like redis-stable.tar.gz has not been updated in over a year. The redis.io site offers 6.2.6 as the stable version for download (as of 2021-10-16).

https://web.archive.org/web/20211010161415/https://redis.io/download

curl -sIX GET http://download.redis.io/releases/redis-stable.tar.gz HTTP/1.1 200 OK Etag: "5ef76ae8-228044" Last-Modified: Sat, 27 Jun 2020 15:51:04 GMT Content-Type: application/octet-stream Content-Length: 2261060 Cache-Control: max-age=60, public Expires: Sat, 16 Oct 2021 09:47:37 GMT Date: Sat, 16 Oct 2021 09:46:37 GMT ...

tvd0x2a · 2021-10-18T23:43:33Z

The broken Screwdriver/2880/PR:pkg_ml1_64 screwdriver build fails in the install_dependencies step with this error:

02:44:24 $ $BASE_PYTHON -m screwdrivercd.installdeps
02:44:24 /opt/python/cp37-cp37m/bin/python3: Error while finding module specification for 'screwdrivercd.installdeps' (ModuleNotFoundError: No module named 'screwdrivercd')

This could be related to errors finding libffi in the init_os step:

02:43:40   Building wheels for collected packages: cffi
02:43:40     Building wheel for cffi (setup.py): started
02:43:40     Building wheel for cffi (setup.py): finished with status 'error'
02:43:40     ERROR: Command errored out with exit status 1:
...
02:44:23     Package libffi was not found in the pkg-config search path.
02:44:23     Perhaps you should add the directory containing `libffi.pc'
02:44:23     to the PKG_CONFIG_PATH environment variable
02:44:23     No package 'libffi' found
...
02:44:23     gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/opt/python/cp37-cp37m/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
02:44:23     c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
02:44:23      #include <ffi.h>
02:44:23                      ^
02:44:23     compilation terminated.
02:44:23     error: command 'gcc' failed with exit status 1
...

I don't think that error is related to this PR. I'll try re-running the build, and see if it clears.

tvd0x2a · 2021-10-19T00:43:20Z

The build failure doesn't seem to be transient. Digging in a bit, this seems to be a transitive dependency of screwdrivercd now requiring additional libraries to install.

The transitive dependencies are (on linux quay.io/pypa/manylinux1_x86_64 where this build fails):

screwdrivercd
twine
keyring>=15.1
SecretStorage>=3.2
cryptography>=2.0
cffi>=1.12

Today, that resolves to installing cffi version 1.15, which fails to install due to missing libffi.

One work around is to manually install cffi==1.14 before installing screwdrivercd. Note that this ends up install cryptography 3.3.2. This probably isn't a desirable work around, since it misses any bug fixes and security updates that may have been introduced.

I suspect this is rooted in the cryptography library's rust dependency, which was introduced in version 3.4.

All that said, I'm not sure if it's worth fixing this. The quay.io/pypa/manylinux1_x86_64 image is only support for another 2.5 months and is scheduled to be EOL'd on 2022-01-01. See: https://web.archive.org/web/20210918193655/https://github.com/pypa/manylinux#manylinux1-centos-5-based---eol

Perhaps the pkg_ml1_64 configuration should be removed? I'll submit a PR for that.

The manylinux1 jobs fail due to a transitive dependency on cryptography and cffi libraries, which no longer install on the quay.io/pypa/manylinux1_x86_64 image. The quay.io/pypa/manylinux1_x86_64 image is scheduled to be EOL'd on 2022-01-01 (see: https://web.archive.org/web/20210918193655/https://github.com/pypa/manylinux#manylinux1-centos-5-based---eol). Remove the manylinux1 jobs in anticipation of their EOL.

tvd0x2a · 2021-10-19T00:50:00Z

PR to address the build failure: #155

…redislite into m1-support

carlosguso · 2022-01-06T03:46:41Z

What is the current status of this PR?

tvd0x2a · 2022-01-06T04:12:57Z

What is the current status of this PR?

It’s ready to merge, I believe.

I think this project may be effectively unmaintained at this point though, so that may not happen. See #156 and #157.

rafa-munoz · 2022-01-20T13:33:52Z

It works like a charm with a Python:3.10 Docker image. Thank you.

Uznick · 2022-02-22T17:22:20Z

Hi, @dwighthubbard! Is there a way to move this forward?

dwighthubbard

This looks good to me

Upgrade redis.submodule to 6.2.6

ad0411a

tvd0x2a commented Oct 16, 2021

View reviewed changes

rerun screwdriver build for PR.

9ced746

tvd0x2a added 2 commits October 18, 2021 17:53

rerun screwdriver build for PR.

c745ca3

Merge branch 'remove-manylinux1-build' of https://github.com/tvd0x2a/…

f3d00b3

…redislite into m1-support

tvd0x2a mentioned this pull request Oct 19, 2021

Remove screwdriver manylinux1 jobs. #155

Closed

mostafa mentioned this pull request Jan 20, 2022

Update to redis version 6.0.5 #162

Closed

rafa-munoz mentioned this pull request Jan 20, 2022

Can't build in Python:3.10 #161

Closed

lazToum mentioned this pull request Mar 13, 2022

allow unix-socket connection in RedisSettings (socket_address) python-arq/arq#271

Closed

dwighthubbard approved these changes Jun 28, 2022

View reviewed changes

dwighthubbard merged commit d830ef6 into yahoo:master Jun 28, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade redis.submodule to 6.2.6 #153

Upgrade redis.submodule to 6.2.6 #153

tvd0x2a commented Oct 16, 2021

tvd0x2a Oct 16, 2021

tvd0x2a commented Oct 18, 2021

tvd0x2a commented Oct 19, 2021

tvd0x2a commented Oct 19, 2021

carlosguso commented Jan 6, 2022

tvd0x2a commented Jan 6, 2022

rafa-munoz commented Jan 20, 2022

Uznick commented Feb 22, 2022

dwighthubbard left a comment

Upgrade redis.submodule to 6.2.6 #153

Upgrade redis.submodule to 6.2.6 #153

Conversation

tvd0x2a commented Oct 16, 2021

tvd0x2a Oct 16, 2021

Choose a reason for hiding this comment

tvd0x2a commented Oct 18, 2021

tvd0x2a commented Oct 19, 2021

tvd0x2a commented Oct 19, 2021

carlosguso commented Jan 6, 2022

tvd0x2a commented Jan 6, 2022

rafa-munoz commented Jan 20, 2022

Uznick commented Feb 22, 2022

dwighthubbard left a comment

Choose a reason for hiding this comment