Issue1157 add stat so subscribe/citypage downloads just work. #1163
Conversation
Add fileSizeMax implementation, with documentation in both languages. change bufsize to bufSize (with a synonym so old seting still works) change blocksize to blockSize (with a synonym so old seting still works)
petersilva
requested review from
reidsunderland,
MagikEh,
tysonkaufmann,
junhu3,
andreleblanc11 and
mshak2
|
Also added fileSizeMax option to be able to avoid downloading files that are too much bigger than accepted. |
Test Results1 tests 0 ✅ 14s ⏱️ For more details on these errors, see this check. Results for commit 8ba10cd. ♻️ This comment has been updated with latest results. |
|
something weird has happenned to ubuntu 20.04... all tests on that OS are failing with some kind of SSL error. |
|
otoh, the problem seems to occur when running sr3 entry point and it does an import requests (which is added by this PR) it seems to fail with an inscrutable SSL something or other not found... so it's like the python packages being installed in 20.04 are somehow incompatible with eachother. |
|
kind of frustrating... we have test machines, and one runs 18.04... and all the tests pass when I run them on a vm... tested so far on 24.04 and 18.04... hmm... |
|
weirdnesses so far:
|
|
We could change the code to use urllib instead. Requests is just a little bit nicer to work with. https://docs.python.org/3/library/urllib.request.html#urllib.request.Request --> can set |
|
I'm going to close this PR, merge from development to the branch, and try opening the PR again. |
See https://docs.python.org/3/library/socket.html#socket.timeout socket.timeout has been replaced by TimeoutError. It's only been converted to an alias of TimeoutError since python3.10 Co-authored-by: Reid Sunderland <[email protected]>
* implement messageAgeMax and post_ properly * messageAgeMax documentation English * French documentation of messageAgeMax settings * clarifying message propoerties set by post_messageAgeMax for each message protocol
|
re-submission behaves the same... it's something to do with the change... but no idea what. |
|
The failure on gitlab of images on 20.04 where the SSL crashes prevent running any tests... we are considering re-writing the https stat() call to use urllib.request() |
|
This branch will require work and cleaning... will open a different PR at some point. |
progress on #1157 ...
so before this change:
in the case of citypages, the file has been changed before the subscriber attempted the first download:
with the change:
after a length mismatch on download, stat() the file on the server. to ask it what the size is on the server.
if the date of the file is newer than on the message, discard the message rather than requeing...
If the size matches on the server... then downloading again, won't change anything. so better to discard.
in both cases, subsequent retries will never succeed.,,, so just throw it out.
So testing was done downloading subscribe/hpfx_citypage ... and with these changes, it is clean now, with no retries.