[WIP] proof of concept: HTMLCanvas bitmap context in pyodide

[Vipitis]

#38

two evenings of tinkering but I can feel a bit of progress. Rendercanvas looks very web inspired, so I am reading a lot of things between the docs, pyodide docs, pyodide source that sound close but are ever so slightly different. Plus I don't have any webdev experience, it's more like a learning opportunity.

for those that want to give it a try - you can essentially just load the .html as a static page with the python script inserted. But you will need to build wheels and load them locally. Installing from pypi via micropip doesn't include these changes.

some todos:

• support all events
  • resize
  • close
  • mouse move doesn't work yet
  • wheel emits warning
  • (should events block navigation?)
• write tests for CI -> docs
• put interactive canvas in docs? (get wheel from CI build artefacts?)
• replace all local links to latest/stable wheel before merge

[Vipitis]

feel like some stuff breaks due to the python name mangling when using double underscores in combination with the subclass... which happens quite a bit - so I am sorta surprised it still works this far.

But I can't find any such references for Pyodide which would be odd if that is a known limitation.

[Vipitis]

feel like some stuff breaks due to the python name mangling

that wasn't the case. I actually fell through the line here

rendercanvas/rendercanvas/base.py

Lines 450 to 451 in fa7defc

	context = self._canvas_context
	if context:

because I hadn't implemented context as a class. I took a lot of inspiration form the bitmap scripts and got it working
I also registered the auto backend successfully - meaning if you build the wheel and then load it statically. The examples noise.py and snake.py work out of the box (although not events yet). But the weekend has a few more days :)

auto_demo.mp4

E: turns out that wasn't true either and I am using the existing bitmaprenderingcontext due to how get_context is implemented.

[Vipitis]

singular keydown event works... so more events and other types shouldn't be impossible. However I will get to that another day.

snake_events.mp4

[Vipitis]

tried all day to make it work for the docs ... but either the .whl don't get included as static files or pyodide has trouble importing the wheel. I also wanted to automate the iframe inclusion with sphinx-gallery but seems like you need to either modify the gen_rst function or write an image scaper to append some custom rst to the examples... so I just added the comment to the relevant examples instead 🤷

classic "works on my machine", so have a video of what could have been instead:

doc_embed.mp4

[Korijn]

Even so it's awesome how far you managed to take this!

[Korijn]

I would argue that providing a constructor like this would be more conventional for the web:

Suggested change

	canvas = RenderCanvas(title="Example")
	canvas_el = document.getElementById("canvas")
	canvas = RenderCanvas(canvas_el, title="Example")

Since often there are multiple canvas elements on the page, users should be able to control which is used.

[Vipitis]

One of the goal was to keep the python code portable between auto backends. So passing a string to __init__() would work even on backends where this kwarg isn't used like glfw and the user doesn't need to use any pyodide specific code in python. (Once we have a wgpu-py version for browser, most examples should just work without changes to shadertoy, pygfx or fastplotlib etc).

I also losely followed the idea of https://pyodide.org/en/stable/usage/sdl.html#setting-canvas where they provide a specific API to accessing the canvas, although I not using it.

Maybe I can write a little multi canvas example to see if my approach works.

I have zero webdev experience, so my design decisions are directed to the python devs wanting to write their python code (like myself).

[Korijn]

I have zero webdev experience, so my design decisions are directed to the python devs wanting to write their python code (like myself).

I hear you, but just because you can use python the language, doesn't mean you can "ignore" the environment it's running in! I don't mind what kind of API you choose (I value portability as well) as long as the user can control which <canvas> is used.

I imagine python devs turning to browsers will often do so because they want to use the browser's capabilities to build the UI they have in mind. It's easy to envision applications with multiple canvases embedded in a richer UI.

[Vipitis]

it shouldn't be impossible to support both.
canvas_el: [str|HTMLCanvasElement] = "canvas"

[Vipitis]

would work even on backends where this kwarg isn't used like glfw

apparently kwargs don't get ignored when a different auto backend is selected because the base class calls super.__init__(*args, *kwargs). We could use the title arg as I am not sure if that has a use in the browser, but that seems janky.

[Vipitis]

We probably don't want to overwrite the existing canvas size with the default resolution... plus resize and close events aren't yet implemented. Do you have any small example where multiple canvas elements are handled by a framework?

[almarklein]

We have multiple.py. And the qt_app.py example can be changed relatively easily to embed to canvases.

[Korijn]

Awesome!

[almarklein]

For resizing, have a look at window.ResizeObserve: I use this code in another project to get the physical size (it's PScript, so you need to convert to Python/JS depending on where it runs):

        # Inside the  ResizeObserve callback ... 
        entry = entries.find(lambda entry: entry.target is self.node)
        if entry.devicePixelContentBoxSize:
            # Best if we have the physical pixels ...
            psize = [
                entry.devicePixelContentBoxSize[0].inlineSize,
                entry.devicePixelContentBoxSize[0].blockSize,
            ]
        else:
            # ... but not all browsers support that (see issue #423) ...
            if entry.contentBoxSize:
                lsize = [
                    entry.contentBoxSize[0].inlineSize,
                    entry.contentBoxSize[0].blockSize,
                ]
            else:  # even more backward compat
                lsize = [entry.contentRect.width, entry.contentRect.height]
            ratio = get_pixel_ratio()
            psize = Math.floor(lsize[0] * ratio), Math.floor(lsize[1] * ratio)

[Korijn]

In general the standard reference material for browser APIs is on MDN, in this case see this page for example usage: https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserver