occlusion filtering and no-information-label filtering

[Ol1ver0413]

Description

We need to add a filtering mechanism to the snapshot provided by _snapshotForAI, including: occlusion filtering and no-information-label filtering

Checklist

Go over all the following points, and put an x in all the boxes that apply.

• I have read the CONTRIBUTION guide (required)
• I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
• I have checked if any dependencies need to be added or updated in pyproject.toml and uv lock
• I have updated the tests accordingly (required for a bug fix or a new feature)
• I have updated the documentation if needed:
• I have added examples if this is a new feature

If you are unsure about any of these, don't hesitate to ask. We are here to help!

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Ol1ver0413]

Hey, @nitpicker55555! I have completed the Python version of filtering the no-information label and occluded label.
I noticed that the occluded filtering can be a bit aggressive. We could discuss possible optimizations, like further filtering based on the shape or opacity of graphical elements.
Be free to let me know to do some further tests.

[nitpicker55555]

Thanks for your contribution! We need to modify Hybrid_browser_toolkit type script version but not python version. And we can reuse the occlusion mechanism from current browser_get_som_screenshot like we dicussed last time

[Ol1ver0413]

Thanks for your contribution! We need to modify Hybrid_browser_toolkit type script version but not python version. And we can reuse the occlusion mechanism from current browser_get_som_screenshot like we dicussed last time

Thanks @nitpicker55555 a lot! I might not fully understand the hybrid_browser_toolkit yet,, and I’ll take a closer look at the relevant source code and make the changes. I’ll let you know if I run into any issues.

[Ol1ver0413]

Hey, @nitpicker55555 ! I really appreciate the opportunity to work on the browser-use project, but due to my current workload, I won’t be able to continue contributing at this time. Thanks and sorry