Skip to content

Commit 4e580db

Browse files
synacktraamlejvajamesmurdzamishushakov
authored
feat: add support for VNC using x11vnc and novnc (#15)
* feat: add ubuntu-desktop template with custom wallpaper and vnc support * Cleanup dockerfile; create dev template * feat: restore original template directory * feat: update env vars in desktop-dev template * feat: add wait_for_port function to verify vnc and novnc status * feat: full control over startup and vnc server * feat: refresh desktop and unique password generation * fix: remove vnc and novnc handles on stop * feat: add js sdk * feat: add auto-connect parameter in novnc URL * fix: resolve extended sandboxOpts issue * fix: resolve e2b wallpaper not applied issue * fix: catch command exit error in wait and verify method * Update README * feat: update the original template * Revert changes made to template to add VNC support * Restore necessary changes to template for VNC support * Use E2B fork of noVNC without control bar and branding * Upgrade typedoc package to 0.27.9 * Add type definitions for NodeJS * Remove autoconnect parameter from examples * Add the password to the stream URL when auth is enabled * Don't print the password in the Python example * Enable stream auth in the TypeScript example * Rename methods and documentation to use "stream" instead of "VNC" * Throw an error instead of returning null for getCursorPosition() and getScreenSize() * Remove README for Python SDK * Remove desktop.hotkey() * Rename takeScreenshot() to screenshot() * Make password member private in the TypeScript SDK * Throw an error on stream.start() if the stream is already running * Always use Sandbox class name instead of Desktop * Move stream parameters to stream.start() * Remove desktop.refresh() * added changeset * changed default sandbox template * Add apt-update * Remove old streaming web page * Make sure to always pull the latest git repo in the sandbox template by invalidating Docker cache * Add readmes to each package * Explain better how stream auth works * Remove example of customizing streaming port from readmes * Change positional params to an options object for the `write()` method in JS SDK --------- Co-authored-by: Vasek Mlejnsky <[email protected]> Co-authored-by: James Murdza <[email protected]> Co-authored-by: Mish Ushakov <[email protected]>
1 parent 650efcd commit 4e580db

22 files changed

+5550
-651
lines changed

‎.changeset/lucky-rules-walk.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"@e2b/desktop-python": minor
3+
"@e2b/desktop": minor
4+
---
5+
6+
Added VNC support

‎README.md

+134-54
Original file line numberDiff line numberDiff line change
@@ -7,78 +7,150 @@ Each sandbox is isolated from the others and can be customized with any dependen
77
![Desktop Sandbox](readme-assets/screenshot.png)
88

99
### Example app using Computer Use with Anthropic's Claude
10-
Check out the [example open-source app](https://github.com/e2b-dev/secure-computer-use) in a separate repository.
1110

11+
Check out the [example open-source app](https://github.com/e2b-dev/secure-computer-use) in a separate repository.
1212

1313
## 🚀 Getting started
14+
1415
The E2B Desktop Sandbox is built on top of [E2B Sandbox](https://e2b.dev/docs).
1516

1617
### 1. Get E2B API key
18+
1719
Sign up at [E2B](https://e2b.dev) and get your API key.
1820
Set environment variable `E2B_API_KEY` with your API key.
1921

2022
### 2. Install SDK
23+
2124
**Python**
25+
2226
```bash
2327
pip install e2b-desktop
2428
```
2529

2630
**JavaScript**
31+
2732
```bash
2833
npm install @e2b/desktop
2934
```
3035

3136
### 3. Create Desktop Sandbox
37+
3238
**Python**
39+
3340
```python
3441
from e2b_desktop import Sandbox
3542

43+
# Basic initialization
3644
desktop = Sandbox()
45+
46+
# With custom configuration
47+
desktop = Sandbox(
48+
display=":0", # Custom display (defaults to :0)
49+
resolution=(1920, 1080), # Custom resolution
50+
dpi=96, # Custom DPI
51+
)
3752
```
3853

3954
**JavaScript**
55+
4056
```javascript
4157
import { Sandbox } from '@e2b/desktop'
4258

59+
// Basic initialization
4360
const desktop = await Sandbox.create()
44-
```
4561

46-
## Stream virtual desktop screen
47-
You can enable streaming the desktop screen by passing `videoStream: true` to the `Sandbox.create` function in JavaScript and `video_stream=True` to the `Sandbox` constructor in Python.
62+
// With custom configuration
63+
const desktop = await Sandbox.create({
64+
display: ':0', // Custom display (defaults to :0)
65+
resolution: [1920, 1080], // Custom resolution
66+
dpi: 96, // Custom DPI
67+
})
68+
```
4869

49-
Then call `getVideoStreamUrl` in JS and `get_video_stream_url` method in Python to get the stream URL that will look like this: `https://e2b.dev/stream/sandbox/<sandbox-id>?token=<secret-token>` and open it in your browser.
70+
## Features
5071

51-
You'll need to wait a couple of seconds for the stream to buffer the first frames.
72+
### Streaming desktop's screen
5273

5374
**Python**
75+
5476
```python
5577
from e2b_desktop import Sandbox
78+
desktop = Sandbox()
79+
80+
# Start the stream
81+
desktop.stream.start()
5682

57-
desktop = Sandbox(video_stream=True)
58-
stream_url = desktop.get_video_stream_url()
59-
print(stream_url)
60-
# Open stream_url in your browser
61-
# You'll need to wait a couple of seconds for the stream to buffer the first frames
83+
# Get stream URL
84+
url = desktop.stream.get_url()
85+
print(url)
86+
87+
# Stop the stream
88+
desktop.stream.stop()
6289
```
6390

6491
**JavaScript**
92+
6593
```javascript
6694
import { Sandbox } from '@e2b/desktop'
6795

68-
const desktop = await Sandbox.create({ videoStream: true, onVideoStreamStart: (streamUrl) => {
69-
console.log(streamUrl)
70-
}})
71-
// Open streamUrl in your browser
72-
// You'll need to wait a couple of seconds for the stream to buffer the first frames
96+
const desktop = await Sandbox.create()
97+
98+
// Start the stream
99+
await desktop.stream.start()
100+
101+
// Get stream URL
102+
const url = desktop.stream.getUrl()
103+
console.log(url)
104+
105+
// Stop the stream
106+
await desktop.stream.stop()
73107
```
74108

75-
![Desktop Sandbox](readme-assets/video-stream.png)
109+
### Streaming with password protection
76110

77-
## Features
111+
**Python**
112+
113+
```python
114+
from e2b_desktop import Sandbox
115+
desktop = Sandbox()
116+
117+
# Start the stream
118+
desktop.stream.start(
119+
enable_auth=True # Enable authentication with an auto-generated password that will be injected in the stream URL
120+
)
121+
122+
# Get stream URL
123+
url = desktop.stream.get_url()
124+
print(url)
125+
126+
# Stop the stream
127+
desktop.stream.stop()
128+
```
129+
130+
**JavaScript**
131+
132+
```javascript
133+
import { Sandbox } from '@e2b/desktop'
134+
135+
const desktop = await Sandbox.create()
136+
137+
// Start the stream
138+
await desktop.stream.start({
139+
enableAuth: true, // Enable authentication with an auto-generated password that will be injected in the stream UR
140+
})
141+
142+
// Get stream URL
143+
const url = desktop.stream.getUrl()
144+
console.log(url)
145+
146+
// Stop the stream
147+
await desktop.stream.stop()
148+
```
78149

79150
### Mouse control
80151

81152
**Python**
153+
82154
```python
83155
from e2b_desktop import Sandbox
84156
desktop = Sandbox()
@@ -92,6 +164,7 @@ desktop.mouse_move(100, 200) # Move to x, y coordinates
92164
```
93165

94166
**JavaScript**
167+
95168
```javascript
96169
import { Sandbox } from '@e2b/desktop'
97170

@@ -108,39 +181,70 @@ await desktop.moveMouse(100, 200) // Move to x, y coordinates
108181
### Keyboard control
109182

110183
**Python**
184+
111185
```python
112186
from e2b_desktop import Sandbox
113187
desktop = Sandbox()
114188

115-
desktop.write("Hello, world!") # Write text at the current cursor position
116-
desktop.hotkey("ctrl", "c") # Press ctrl+c
189+
# Write text at the current cursor position with customizable typing speed
190+
desktop.write("Hello, world!") # Default: chunk_size=25, delay_in_ms=75
191+
desktop.write("Fast typing!", chunk_size=50, delay_in_ms=25) # Faster typing
192+
193+
# Press keys
194+
desktop.press("enter")
195+
desktop.press("space")
196+
desktop.press("backspace")
197+
desktop.press("ctrl+c")
198+
```
199+
200+
**JavaScript**
201+
202+
```javascript
203+
import { Sandbox } from '@e2b/desktop'
204+
205+
const desktop = await Sandbox.create()
206+
207+
// Write text at the current cursor position with customizable typing speed
208+
await desktop.write('Hello, world!')
209+
await desktop.write('Fast typing!', { chunkSize: 50, delayInMs: 25 }) // Faster typing
210+
211+
// Press keys
212+
await desktop.press('enter')
213+
await desktop.press('space')
214+
await desktop.press('backspace')
215+
await desktop.press('ctrl+c') // Copy
117216
```
118217

119218
### Screenshot
219+
220+
**Python**
221+
120222
```python
121223
from e2b_desktop import Sandbox
122224
desktop = Sandbox()
123225

124226
# Take a screenshot and save it as "screenshot.png" locally
125-
image = desktop.take_screenshot()
227+
image = desktop.screenshot()
126228
# Save the image to a file
127229
with open("screenshot.png", "wb") as f:
128230
f.write(image)
129231
```
130232

131233
**JavaScript**
234+
132235
```javascript
133236
import { Sandbox } from '@e2b/desktop'
134237

135238
const desktop = await Sandbox.create()
136-
const image = await desktop.takeScreenshot()
239+
const image = await desktop.screenshot()
137240
// Save the image to a file
138-
fs.writeFileSync("screenshot.png", image)
241+
fs.writeFileSync('screenshot.png', image)
139242
```
140243

141244
### Open file
142245

143246
**Python**
247+
144248
```python
145249
from e2b_desktop import Sandbox
146250
desktop = Sandbox()
@@ -151,18 +255,21 @@ desktop.open("/home/user/index.js") # Then open it
151255
```
152256

153257
**JavaScript**
258+
154259
```javascript
155260
import { Sandbox } from '@e2b/desktop'
156261

157262
const desktop = await Sandbox.create()
158263

159264
// Open file with default application
160-
await desktop.files.write("/home/user/index.js", "console.log('hello')") // First create the file
161-
await desktop.open("/home/user/index.js") // Then open it
265+
await desktop.files.write('/home/user/index.js', "console.log('hello')") // First create the file
266+
await desktop.open('/home/user/index.js') // Then open it
162267
```
163268

164269
### Run any bash commands
270+
165271
**Python**
272+
166273
```python
167274
from e2b_desktop import Sandbox
168275
desktop = Sandbox()
@@ -173,45 +280,18 @@ print(out)
173280
```
174281

175282
**JavaScript**
283+
176284
```javascript
177285
import { Sandbox } from '@e2b/desktop'
178286

179287
const desktop = await Sandbox.create()
180288

181289
// Run any bash command
182-
const out = await desktop.commands.run("ls -la /home/user")
290+
const out = await desktop.commands.run('ls -la /home/user')
183291
console.log(out)
184292
```
185293

186-
### Run PyAutoGUI commands
187-
**Python**
188-
```python
189-
from e2b_desktop import Sandbox
190-
desktop = Sandbox()
191-
192-
# Run any PyAutoGUI command
193-
desktop.pyautogui("pyautogui.click()")
194-
```
195-
196-
**JavaScript**
197-
```javascript
198-
import { Sandbox } from '@e2b/desktop'
199-
200-
const desktop = await Sandbox.create()
201-
202-
// Run any PyAutoGUI command
203-
await desktop.runPyautoguiCode("pyautogui.click()")
204-
```
205-
206-
<!-- ### Customization
207-
```python
208-
from e2b_desktop import Sandbox
209-
desktop = Sandbox()
210-
``` -->
211-
212294
## Under the hood
213-
You can use [PyAutoGUI](https://pyautogui.readthedocs.io/en/latest/) to control the whole environment programmatically.
214295

215296
The desktop-like environment is based on Linux and [Xfce](https://www.xfce.org/) at the moment. We chose Xfce because it's a fast and lightweight environment that's also popular and actively supported. However, this Sandbox template is fully customizable and you can create your own desktop environment.
216297
Check out the sandbox template's code [here](./template/).
217-

0 commit comments

Comments
 (0)