-
Notifications
You must be signed in to change notification settings - Fork 8
[WIP] proof of concept: HTMLCanvas bitmap context in pyodide #115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Vipitis
wants to merge
29
commits into
pygfx:main
Choose a base branch
from
Vipitis:browser
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+794
−14
Draft
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
c81ab50
js bitmap context
Vipitis 393b590
might need a loop -.-
Vipitis f0f533d
working loop
Vipitis d62dc39
assing js_array directly
Vipitis 93e3639
add context class
Vipitis bd9b3ab
fix channels
Vipitis 9017288
register auto backend
Vipitis a10970d
working events!
Vipitis e62274a
fix pixel order
Vipitis d79656c
cleanup testing code
Vipitis 8d51188
remove unused context class
Vipitis 1044d0f
add all events
Vipitis d393c52
add basic documentation
Vipitis 8b71ed7
typos pass
Vipitis f197d7d
embed examples into docs
Vipitis 529c3ec
maybe fix wheel location
Vipitis 7025f87
maybe fix files
Vipitis 58bc775
add canvas selector
Vipitis 51459d3
add multicanvas example
Vipitis 6cf0ba5
use asyncio loop
Vipitis d9f8fc0
ruff format
Vipitis fcd2d1c
add canvas element arg
Vipitis 3e13446
simplify selector argument
Vipitis 75e3ddc
icorrect type hints
Vipitis 879399e
enbled wgpu context
Vipitis 7544b77
fix button ids in pointer events
Vipitis 2dd4824
add resize event
Vipitis 053f2db
make the example resize
Vipitis ef8d9d3
fix pointer_move just inside or down
Vipitis File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
<!doctype html> | ||
<html> | ||
<head> | ||
<!-- Interactive HTMLRenderCanvas via Pyodide:<br> --> | ||
<script src="https://cdn.jsdelivr.net/pyodide/v0.28.2/full/pyodide.js"></script> | ||
</head> | ||
<body> | ||
<canvas id="canvas" width="640" height="480" style="background-color: red;"></canvas><br> | ||
<!-- TODO: redirect stdout prints into something visible without console? --> | ||
<script type="text/javascript"> | ||
async function main(){ | ||
let example_name = window.parent.location.href.split("/").slice(-1)[0].split("#").splice(0)[0].replace(".html", ""); | ||
// TODO: get the script from docs dir? docs/gallery/script.py or the .zip? | ||
// for now get the example from main because the local example is hidden behind a hash in _downloads/hash/example_name.py (but it still exists) | ||
pythonCode = await (await fetch(`https://raw.githubusercontent.com/pygfx/rendercanvas/refs/heads/main/examples/${example_name}.py`)).text(); | ||
let pyodide = await loadPyodide(); | ||
await pyodide.loadPackage("micropip"); | ||
const micropip = pyodide.pyimport("micropip"); | ||
await micropip.install('numpy'); // can we figure out if we need it, or do we always get it? | ||
|
||
// TODO: learn js to implement the other options... | ||
// so we should get the rendercanvas wheel from three locations: | ||
// 1) local wheel from ../dist/*whl? (for local development) | ||
// 2) from the CI artefacts (moves into _static in the job) (for PR branch) | ||
// 3) latest from pypi, when the artefacts are gone - for like stable/main? | ||
// await micropip.install('rendercanvas'); | ||
await micropip.install('rendercanvas-2.2.1-py3-none-any.whl'); // from html/_static/ dir like a PR branch doc build... | ||
|
||
// Run the Python code async because some calls are async it seems. | ||
pyodide.runPythonAsync(pythonCode); | ||
} | ||
main(); | ||
</script> | ||
</body> | ||
</html> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,9 @@ | ||
div.sphx-glr-download, | ||
div.sphx-glr-download-link-note { | ||
display: none; | ||
} | ||
div.document iframe { | ||
width: 100%; | ||
height: 500px; | ||
border: none; | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
<!-- adapted from: https://traineq.org/imgui_bundle_online/projects/min_bundle_pyodide_app/demo_heart.source.txt --> | ||
<!doctype html> | ||
<html> | ||
<head> | ||
RenderCanvas HTML canvas via Pyodide:<br> | ||
<script src="https://cdn.jsdelivr.net/pyodide/v0.28.2/full/pyodide.js"></script> | ||
<style> | ||
/* so the demo page can have interactive resizing */ | ||
#canvas { | ||
position: absolute; | ||
width: 90%; | ||
height: 90%; | ||
} | ||
</style> | ||
</head> | ||
<body> | ||
<!-- WIP: make it more of a "Playground"? --> | ||
<select id="example-select"> | ||
<option value="events.py">events.py</option> | ||
<option value="noise.py">noise.py</option> | ||
<option value="snake.py">snake.py</option> | ||
</select> | ||
<!-- maybe something like add example (to test spawning more canvases and also closing them?) --> | ||
<button id="reload-example">Reload example</button> | ||
<button id="reload-page" style="background-color: orange;"onclick="location.reload();">Reload page</button> | ||
<select id="lib-path"> | ||
<option value="../dist/rendercanvas-2.2.1-py3-none-any.whl" selected>local ./dist/</option> | ||
<option value="rendercanvas">latest PyPI</option> | ||
</select> | ||
<input type="checkbox" id="load-wgpu">Load WebGPU (experimental)</input> | ||
<br> | ||
<!-- maybe wrap in a div so content after it still shows? --> | ||
<canvas id="canvas" style="background-color: lightgrey;"></canvas><br> | ||
some text below the canvas! | ||
<!-- TODO: redirect the Python stdout/stderr to some kind of text box at the bottom --> | ||
<script type="text/javascript"> | ||
|
||
async function loadPythonExample(example_path){ | ||
let response = await fetch(example_path); | ||
let pythonCode = await response.text(); | ||
return pythonCode; | ||
} | ||
|
||
async function main(){ | ||
|
||
// fetch the file locally for easier scripting | ||
// start chrome --allow-file-access-from-files %cd%\examples\local_browser.html or local webserver | ||
// TODO: replace the actual code here (unless you have the module) | ||
let example_select = document.getElementById("example-select"); | ||
let reload_example_button = document.getElementById("reload-example"); | ||
pythonCode = await (await fetch(example_select.value)).text(); | ||
|
||
// Load Pyodide | ||
let pyodide = await loadPyodide(); | ||
|
||
await pyodide.loadPackage("micropip"); | ||
const micropip = pyodide.pyimport("micropip"); | ||
await micropip.install('numpy'); | ||
// await micropip.install('rendercanvas'); | ||
await micropip.install('../dist/rendercanvas-2.2.1-py3-none-any.whl'); // local wheel for auto testing | ||
|
||
// Run the Python code async because some calls are async it seems. | ||
pyodide.runPythonAsync(pythonCode); | ||
|
||
// on click, we reload the example (or a difference one) | ||
reload_example_button.onclick = async function() { | ||
// interrupt existing code? | ||
// sounds complex: https://pyodide.org/en/stable/usage/keyboard-interrupts.html | ||
pyodide.setInterruptBuffer([2]); | ||
pyodide.setInterruptBuffer([2]); | ||
|
||
let example_file = example_select.value; | ||
pythonCode = await (await fetch(example_file)).text(); | ||
pyodide.runPythonAsync(pythonCode); | ||
}; | ||
} | ||
main(); | ||
</script> | ||
</body> | ||
</html> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would argue that providing a constructor like this would be more conventional for the web:
Since often there are multiple canvas elements on the page, users should be able to control which is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One of the goal was to keep the python code portable between auto backends. So passing a string to
__init__()
would work even on backends where this kwarg isn't used likeglfw
and the user doesn't need to use any pyodide specific code in python. (Once we have awgpu-py
version for browser, most examples should just work without changes to shadertoy, pygfx or fastplotlib etc).I also losely followed the idea of https://pyodide.org/en/stable/usage/sdl.html#setting-canvas where they provide a specific API to accessing the canvas, although I not using it.
Maybe I can write a little multi canvas example to see if my approach works.
I have zero webdev experience, so my design decisions are directed to the python devs wanting to write their python code (like myself).
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hear you, but just because you can use python the language, doesn't mean you can "ignore" the environment it's running in! I don't mind what kind of API you choose (I value portability as well) as long as the user can control which
<canvas>
is used.I imagine python devs turning to browsers will often do so because they want to use the browser's capabilities to build the UI they have in mind. It's easy to envision applications with multiple canvases embedded in a richer UI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it shouldn't be impossible to support both.
canvas_el: [str|HTMLCanvasElement] = "canvas"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
apparently kwargs don't get ignored when a different auto backend is selected because the base class calls
super.__init__(*args, *kwargs)
. We could use the title arg as I am not sure if that has a use in the browser, but that seems janky.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For resizing, have a look at
window.ResizeObserve
: I use this code in another project to get the physical size (it's PScript, so you need to convert to Python/JS depending on where it runs):There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In general the standard reference material for browser APIs is on MDN, in this case see this page for example usage: https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserver
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
finally found a bit of time to implement this - also tried to make the demo page react and it seems to work. Altough I am not sure if this matches real webframeworks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure we can hook things up in a clean way, but I will need to sit down and play a bit to get it right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
feel free to commit into this branch or similar if you have the capacity. I am very much out of my depth with the web stuff and also don't have too much time currently (settling into new job and new university) to sit down for some long evenings and figure it out.