feat(agent): add BrowseURLAction to CodeAct (produce markdown from URL)

[xingyaoww]

End-user friendly description of the problem this fixes or functionality that this introduces

• Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

This PR introduce two browsing mode for default CodeAct agent AND improves the agent's ability to read arbitrary documentation/web page on the internet.

• web_read: simply perform information gathering by browsing URL & convert HTML pages into a markdown file for LLM to comprehend, most useful to read documentation, github issues, etc. It also allows the agent to perform google search by directly visit https://www.google.com/search?q=YOUR_QUERY.
• browser: more complex browsing that allows the agent to interact with the web page (clicking, input things into form, scroll up and down, etc).

---

Give a summary of what the PR does, explaining any non-trivial design decisions

A new web_read tool that directly reads off a given URL
Screenshot of it working:

---

Link of any specific issues this addresses

---

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:97df362-nikolaik   --name openhands-app-97df362   docker.all-hands.dev/all-hands-ai/openhands:97df362

[neubig]

Can we evaluate this in some way? WebArena would be ideal, but barring that miniwob

[xingyaoww]

@neubig this will ONLY be triggered when "http://" or "https://" keywords are detected from user messages -- which a lot of instances (if not all) in miniwob/webarena do not have, means these benchmark likely won't be able to evaluate this changes?

[neubig]

I think miniwob and webarena tell you the URL to go to? (but if not, maybe they should...)

[xingyaoww]

@neubig no.. BrowserGym will automatically switch the given page before the eval kicks off.

But anyway -- i made some modification and directly made an additional web_read function that allows the agent to read the website directly - Here's the Miniwob result: 84/125=67.2%, which is very comparable to the 86/125 we got in #4667.

[neubig]

OK, this LGTM, thank you!

And just to clarify, I wanted to make sure that this didn't break our existing browsing functionality. I think the new solution definitely does not. And actually we might even want to turn the standard browsing functionality off in safety-critical settings, allowing only the web-page read tool. This seems like a great solution!

[xingyaoww]

And I evaluated on a subset of SWE-Bench tasks (30) with browsing enabled - doesn't seem to confuse the agent as well. Will go ahead and merge this.

15:53:17 - openhands:INFO: eval_infer.py:418 - # resolved: 14 / 30. (46.67%)
15:53:17 - openhands:INFO: eval_infer.py:418 - # failed_apply_patch: 0 / 30. (0.00%)
15:53:17 - openhands:INFO: eval_infer.py:418 - # error_eval: 0 / 30. (0.00%)
15:53:17 - openhands:INFO: eval_infer.py:418 - # empty_generation: 0 / 30. (0.00%)