-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gg force --engine lambda failing after emptying bucket #47
Comments
Removing If you look at the error message, it says that the blobs This reminds me we were meant to create a |
that fixed the issue! Now I'm able to see the thunks in the bucket. thanks! I'm running another script and this time it is giving me a different error:
Can I see the execution logs somewhere? Or is there some other way to debug this? |
This is probably because the total size of the input files to that thunk exceeds 230 MiB. That's a hard-coded limit in the lambda engine, since Lambdas come with 500 MiB of disk space ( You have a couple options: either change your thunk to use smaller files as input, or, change the code and increase the maximum limit. Just be careful of the total 500 MiB disk space and the fact that your output needs to fit on disk as well. |
When running a
If none of the engines listed by |
Oh, my script + files were 266MB XD. Changing the limit fixed the issue :). The fallback option will be quite useful later i think! Looks like the lambda workers don't seem to have libpython3.7m.so.1.0 for nuitka to work with:
Will see if they have some something else I can use. I'll take a look at this tomorrow. |
@sadjad can gg be told to upload the libpython.so file as well? |
Oh, that's the issue with Nuitka: it relies on the availability of Python libraries on the host. Let me check something first. |
I think for now the quickest solution might be going back to PyInstaller. There's a way to have |
Cool, I'll try the pyinstaller route tomorrow. Thanks for looking at Nuitka! |
Looks like running a pyinstaller binary on lambda also has a similar problem:
I'm starting a new thread [#48] for this since this thread refers to the solved empty bucket issue. |
Something seems to have broken after I emptied the s3 bucket that was being used by gg. I deleted the .gg directory and re-initialized gg. Still it seems to be wanting to download some old thunks. Running
gg force --engine lambda
isn't adding any thunks to the bucket now. Is there any other cache that I should empty? How do I go about debugging this?The text was updated successfully, but these errors were encountered: