Skip to content

feat(desktop): add embedded browser panel with annotation support - #7017

Open
myipanta wants to merge 1 commit into
esengine:main-v2from
myipanta:feature/browser-panel
Open

feat(desktop): add embedded browser panel with annotation support#7017
myipanta wants to merge 1 commit into
esengine:main-v2from
myipanta:feature/browser-panel

Conversation

@myipanta

@myipanta myipanta commented Jul 28, 2026

Copy link
Copy Markdown
Contributor
  • Add BrowserPanel component with cross-platform embed_browser backend
  • Support browser annotation with selection-based context transfer
  • Add browser URL validation and security checks
  • Wire up Composer to receive browser-annotation image data
  • Add i18n strings for browser panel (en, zh, zh-TW)
  • Fix z-index token compliance in browser-panel CSS
  • Fix sha256 type mismatch (Blob vs string) in Composer

New files:

  • desktop/embed_browser*.go — cross-platform embedded browser backend
  • desktop/frontend/src/components/BrowserPanel.tsx
  • desktop/frontend/src/lib/browserAnnotation.ts
  • desktop/frontend/src/lib/browserUrl.ts
  • desktop/frontend/src/tests/browser-annotation.test.ts

Summary

Add an embedded browser panel to the desktop client with annotation support.

Changes

  • BrowserPanel component — interactive embedded browser with cross-platform backend (Go + WKWebView/WebView2/WebKitGTK)
  • Browser annotation — select elements on any page, extract text, and send context to the Composer
  • Security — URL validation, protocol whitelist, hostname verification
  • UI — highlight overlay, describe bubble, status bar with navigation
  • i18n — English, Simplified Chinese, Traditional Chinese strings

Build fixes included

  • Fix z-index token compliance in styles.css

  • Fix sha256 type mismatch in Composer.tsx

Verification

Cache impact

Cache-impact: none — desktop frontend changes only
Cache-guard: cd desktop && wails build passes full compilation
System-prompt-review: N/A

For cache-sensitive changes, fill these lines before requesting review:

  • Cache-impact: none, low, medium, or high, plus the reason.
  • Cache-guard: the focused guard test/command added or run, or why an existing guard covers the change.
  • System-prompt-review: required reviewer/approval note when provider-visible system prompt, memory prefix, output style, or skill index behavior changes.

@github-actions github-actions Bot added desktop Wails desktop app (desktop/**) v2 Go rewrite (1.x) — main-v2 branch, active development labels Jul 28, 2026
@myipanta
myipanta force-pushed the feature/browser-panel branch from a7f25a2 to e86321e Compare July 28, 2026 13:12
@myipanta

Copy link
Copy Markdown
Contributor Author
image image 后续多模态可以使用 或者其他支持图片识别的模型

@myipanta

Copy link
Copy Markdown
Contributor Author
image

@myipanta
myipanta force-pushed the feature/browser-panel branch 2 times, most recently from f18d15b to 61a4052 Compare July 28, 2026 13:48
@myipanta

Copy link
Copy Markdown
Contributor Author
image

@myipanta
myipanta force-pushed the feature/browser-panel branch 3 times, most recently from daa7d55 to 14a1c63 Compare July 28, 2026 14:54
- Add BrowserPanel component with cross-platform embed_browser backend
- Support browser annotation with selection-based context transfer
- Add browser URL validation and security checks
- Wire up Composer to receive browser-annotation image data
- Add i18n strings for browser panel (en, zh, zh-TW)
- Fix z-index token compliance in browser-panel CSS
- Fix sha256 type mismatch (Blob vs string) in Composer
- Update platformEmbedSetPickMode signature to accept accent colors
- Move emitEmbedBrowserState/Pick to darwin-only file (fix lint)
- Remove unused browser.emptyHint i18n string

New files:
- desktop/embed_browser*.go — cross-platform embedded browser backend
- desktop/frontend/src/components/BrowserPanel.tsx
- desktop/frontend/src/lib/browserAnnotation.ts
- desktop/frontend/src/lib/browserUrl.ts
- desktop/frontend/src/__tests__/browser-annotation.test.ts

Co-authored-by: Cursor <cursoragent@cursor.com>
@myipanta
myipanta force-pushed the feature/browser-panel branch from 243b008 to c40aeef Compare July 29, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

desktop Wails desktop app (desktop/**) v2 Go rewrite (1.x) — main-v2 branch, active development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants