-
Notifications
You must be signed in to change notification settings - Fork 5.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix constant error observations appearing in the logs #5352
Conversation
For some reason, one of the image builds got stuck for over 4h. I restarted them. |
async def filter_for_gitignore(file_list, base_path): | ||
gitignore_path = os.path.join(base_path, '.gitignore') | ||
async def has_gitignore() -> bool: | ||
file_list = await call_sync_from_async(runtime.list_files, '') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if this is a bit heavy, to do twice, for every click. (first at line 68)
Could we avoid it somehow? Or it doesn't matter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I agree we shouldn't do a list_files
first--we should just try to access the file, and use error handling for FileNotFoundError
If that's causing logspam let's remove the logs upstream
try: | ||
read_action = FileReadAction(gitignore_path) | ||
read_action = FileReadAction('.gitignore') | ||
observation = await call_sync_from_async(runtime.run_action, read_action) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a thought, if there is no addition to the stream
on this execution path (and it shouldn't be), could we just get/read the ErrorObs and treat it here directly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this approach, the logs would still contain:
|**ErrorObservation**
|File not found: /workspace/.gitignore. Your current working directory is /workspace
The only ways around this that I can see are:
- Live with this oddity.
- Do not filter files using
.gitignore
- Check that the
.gitignore
exists. (The approach I took) - Update the
FileReadAction
with an optional flag to permit missing files and not yield an error observation. - Update the runtime to allow access to the FileStore and use this directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct me if wrong, but isn't there also
6. Do not log stuff in the FileReadAction, but in its client code?
Or that sounds really bad? I don't think they are many... but I could be wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, this one isn't doing it, it's the run_action
, and that is really useful...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about we use runtime.read
directly instead of run_action
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like it also does run_action
? We could write one that doesn't, I guess, unless I'm missing something.
OK, but it's late and sorry, it seems I no longer understand this problem in the first place: weren't these all in the sandbox now, both the results of list_files
and .gitignore
, so why do we have runtime.list_files
return all files?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for doing this, the log spam is annoying and everyone will be better off when it's gone.
We have a good discussion in the comments, warts and confusions and all. I think hashing out the potential issues in comments makes a PR better, and I would love to see the final form here.
So I'll put a Request changes just to make sure... that it's not merged too fast. 😅
(I noticed you merge PRs a little too fast quite often, before comments are addressed or even acknowledged. Sorry, but I think we can do this better, because sometimes that can break things and some of us really don't like to break things in main
here.)
I'm gonna punt on this one now as there does seem to be concerns about the additional overhead of a file existence check. Perhaps a more thorough refactor will allow a better solution |
Now checking that .gitignore exists. before trying to filter based on its contents.
I see the following constantly in the logs:
The reason is that every time we call
/list-files
we try to filter based on the contents of.gitignore
- so we get an error message every time we try to list files if there is no.gitignore
.This change checks that the
.gitignore
exists before trying to read it.To run this PR locally, use the following command: