Skip to content

ts-cua sample added #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

ts-cua sample added #33

wants to merge 7 commits into from
+1,047 −8

Conversation

raiden-staging
Copy link

  • added a typescript port of the openai CUA sample
  • only uses Kernel , Playwright , OpenAI SDK
  • example runs
$ kernel invoke ts-cua agent-run -p "{\"query\":\"current market price range for a used dreamcast\"}"
 INFO  Invoking "ts-cua" (action: agent-run, version: latest) ...
 INFO  Invocation queued, polling for result...
 SUCCESS  Result:
          {
            "answer": "A used Sega Dreamcast console currently ranges in price depending on condition:\n- **Loose Price (without box)**: Approximately $122.47\n- **CIB Price (Complete in Box)**: Approximately $140.00\n\nThese prices are based on listings from eBay and other marketplace data.",
            "elapsed": 51.47
          }
$ kernel invoke ts-cua agent-run -p "{\"query\":\"list anticipated albums upcoming this week\"}"
 INFO  Invoking "ts-cua" (action: agent-run, version: latest) ...
 INFO  Invocation queued, polling for result...
 SUCCESS  Result:
          {
            "answer": "Here are the anticipated album releases for June 20, 2025:\n\n- \"4\" by Alitch\n- \"American Heart\" by Benson Boone\n- \"I quit\" by HAIM\n- \"Raspberry Moon\" by Hotline TNT\n- \"The Black Dog And The Wandering Boy\" by James McMurtry\n- \"Tropicoqueta\" by Karol G\n- \"hopefully!\" by Loyle Carner\n- \"American Romance\" by Lukas Nelson\n- \"Metallic Life Review\" by Matmos\n- \"Planting By The Signs\" by S.G. Goodman\n- \"Fairyland Codex\" by Tropical Fuck Storm",
            "elapsed": 62.45
          }

juecd
juecd reviewed Jun 19, 2025
View reviewed changes
Copy link
Contributor

@juecd juecd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments!

templates/typescript/cua-sample/.prettierrc Outdated
@@ -0,0 +1,4 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we update this (this is what we're using in other repos)?

{
"semi": true,
"trailingComma": "all",
"singleQuote": true,
"printWidth": 100,
"tabWidth": 2
}

templates/typescript/cua-sample/index.ts Outdated
@@ -0,0 +1,110 @@
// @ts-nocheck
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove and update type errors as needed

templates/typescript/cua-sample/lib/agent.ts Outdated
@@ -0,0 +1,223 @@
// @ts-nocheck
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove and update

templates/typescript/cua-sample/lib/playwright/base.ts Outdated
@@ -0,0 +1,199 @@
// @ts-nocheck
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remote and update

templates/typescript/cua-sample/lib/agent.ts Outdated
/**Handle each item; may cause a computer action + screenshot.*/
if (item.type === "message") {
if (this.print_steps) {
console.log(item.content![0].text);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

console.log(item.content?.[0]?.text); to prevent a runtime error

templates/typescript/cua-sample/lib/playwright/base.ts Outdated
}
};

await this._page!.route("**/*", handleRoute);
Copy link
Contributor

@juecd juecd Jun 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use a type guard method to assert the presence of _page instead of this._page! everywhere

templates/typescript/cua-sample/index.ts Outdated

// start agent run
const response = await agent.runFullTurn(
[
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer runFullTurn to accept an obj with named vars

templates/typescript/cua-sample/index.ts Outdated
}

app.action<CuaInput, CuaOutput>(
"agent-run",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to "cua-task" to conform to other examples

templates/typescript/cua-sample/.gitignore Outdated
@@ -0,0 +1,2 @@
node_modules
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename the folder /cua-sample/ to just /cua/ so it matches other examples

@juecd
Copy link
Contributor

juecd commented Jun 19, 2025

@raiden-staging could you also add a gitignore file that is named _gitignore? When we publish the package, npm excludes actual gitignores. When our CLI copies over _gitignore, it renames it to .gitignore on the user's copy

juecd
juecd reviewed Jun 19, 2025
View reviewed changes
templates/python/browser-use/main.py Outdated


# Define a subclass of BrowserSession that overrides _setup_viewports (which mishandles resizeing on connecting via cdp)
class BrowserSessionCustomResize(BrowserSession):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Can you move it to a separate file so we can keep main.py nice and clean?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juecd juecd juecd left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@raiden-staging @juecd