Pros and Cons

[SammyJoeOsborne]

This is now the third box2d+liquidfun library that I've tried. The original liquidfun asm release is great but lacked many features of the original liquidfun library, such as adding particles to existing groups or joining groups, and it has a glaring bug that doesn't allow you to create a single particle via CreateParticle(); you can only create groups. (It will add it, but the particle just freezes and never updates)
Another port I tried was flyover's box2d js port with liquid fun, which is awesome because it's very easy to use and all code is exposed, but it's not asm or wasm, just pure js, so it was too slow.

This Box2d-wasm port is exciting because it includes soooo many more features of liquidfun previously unavailable, but it still has some major cons.

Pros:

• So many features included! CreateParticle() works, Adding to existing groups works, JoinParticleGroups works, almost everything I tried worked great and I was excited to be able to do these things in liquidfun js for the first time.
• The codehinting via typescript (even if you're not using typescript) is a godsend!
• The original Box2D functionality is very performant!

Cons:

• The liquidfun aspect is not as performant. Where the original library I was able to get about 10000+ particles before performance would slow down, this library I got about ~4200 before frame rate dropped and the simulation became unstable. I think it's because this port mostly deals with pointers, which have to be converted to the desired objects via reifying arrays, wrapping objects, etc. This is fine in limited use but in high-performant particle simulation like liquidfun it slowed things down quite a bit.
• Steep learning curve. This likely just speaks to my abilities as a developer, but understanding how the pointers work, how to reify arrays, etc took some doing. But once you understand it I could see it being fine. It may just warrant more documentation.

Overall I'm very impressed with this fork, and based on the Issue request, you did it quickly!

[Birch-san]

thanks for the feedback 🙂 this is helpful, and it's consistent with what I suspected.

Steep learning curve

yes, agreed. documentation could help, but requires an expert — this is a machine port, so I built this without really understanding how to use it or how it itself works. yes, the reifyArray helper in particular is not mentioned in the docs here (though fortunately has a JSDoc comment to reward anybody who finds it).

I think that completing a project with this library requires persistence — I'm not sure whether there's a cohort of users who "could succeed, with just a bit more investment in docs". I think that to serve those users, we'd need another library that wraps this project with opinionated, JS-idiomatic APIs (maybe like Planck's APIs?) and has a focus on providing documentation and examples. whereas my aim here has been to provide just the axioms, in a sustainable way — this project can react pretty easily to upstream changes in Box2D, which (as you've noticed) enabled it to incorporate liquidfun without major rewrites.

The liquidfun aspect is not as performant

yeah, I think I'm noticing that too. I didn't get performance anything like the asm release, and I'm not sure whether it's because my demo world is different, or if it's something the library needs to approach differently.

certainly, liquidfun's asm release paid special consideration to "how do we create performant JS bindings to the ASM code" and "can we use our knowledge of the memory layout, to lookup values without making another call into ASM":
https://github.com/google/liquidfun/blob/master/liquidfun/Box2D/lfjs/README

I figured I may not need tricks like that, since browser performance considerations of JS->WASM calls are different nowadays. moreover, the demo that I tried makes almost no calls into WASM:

• one-time overhead for setup:
  • 1x new b2ParticleSystemDef
  • 1x b2World#CreateParticleSystem
  • 1x new b2ParticleGroupDef
  • 1x b2ParticleGroupDef#CreateParticleGroup
• no additional calls in the main loop — just the same, single b2World#Step() as always

moreover, I thought the box2d-wasm distribution would have a few advantages because it:

• targets WASM
• compiles using more optimizations (-O3 rather than -O2)
• is compiled using a substantially newer Emscripten

perhaps none of these assumptions are incorrect.

the reason I'm worried that my own demo isn't a fair comparison, is because (unlike most of lfjs's demos), my particles don't stay together (perhaps this prevents optimizations? does lfjs clump nearby particles into bigger objects?) and they frequently stray far away, expire and respawn (would this cause an allocation?). the slowdown I see is a stutter, which almost suggests garbage collection… but that'd surprise me — Box2D/liquidfun's memory allocation is very careful to re-use old memory rather than malloc()/free()ing. maybe the stutter is heap growth? perhaps I just need to ask for a large heap to start with? it's also possible that I've created a b2ParticleSystemDef with really challenging parameters.

if you're able at all to share the code of your project, I'd be interested to see which APIs you're invoking (especially in the main loop), and see whether there's any obvious ways to avoid allocations, re-use memory, or avoid WASM calls.

[SammyJoeOsborne]

Thanks for the reply, all very interesting!

If you're interested, I added my experiment to a private repo and gave you access as a collaborator.
To run, just "npm ci" to download dependencies, then "npm start", and then check localhost:7777
If you want to add more particles, click and hold the mouse.

I wrote a custom fragment shader to make things look liquidy, but if you want that off so you can just see particles you can disable it in the config.json (just set threshhold filter to enabled: false, and same for the blur filter)

Main files to look at are particle-sim.js (which creates the world, particles, etc) and particle-renderer.js. The particle render function is at the bottom of particle-renderer.js

If you want to access the particle system from the console, there's a reference to it at "window.ps"

One thing to note is I'm not finished with the particle-renderer. Right now if you delete particles from the box2d world, I don't believe the particle-renderer deletes the textures from its pool. Still working on it

[Birch-san]

thanks for sharing. I've had only a lightning look, but some ideas…

there's not many allocations or JS->WASM calls in your main loop, so that's good. but there's some inside if (this.mouseDown):

you're allocating three new b2d.b2Vec2s every loop, instead of creating a single b2Vec2 outside the loop, and reusing it via b2Vec2#Set(x, y).
you're allocating a new b2d.b2ParticleDef every loop, instead of creating a single b2ParticleDef outside the loop, and reusing it.
generally I would use destroy(vec) or destroy(pd) on any structures created via new that I've finished using. but in this particular case I reckon you can keep them alive to re-use them on the next frame.

I see some ways to improve particle-renderer:

I think getPointer does a JS->WASM call if the pointer doesn't already exist in its cache.
and reifyArray does a lot of allocations (creates an n-long array, and invokes getPointer n times).
I think posArr[i].x and posArr[i].y are custom getters that do JS->WASM calls as well.

====

I've been trying to figure out what's the fastest way to do a box2d-wasm application. I started making one over the weekend (requires Chrome/Chromium-based browser):

https://github.com/Birch-san/liquidfun-play

The framerate's pretty good (Box2D + WebGL renderer in a web worker). there's still a small number of allocations (unnecessary matrix recreation, and a few new Float32Array allocations), but fixable. for new Float32Array I'm thinking to use object pooling (re-use Float32Arrays I created previously).

main techniques are "in DebugDraw, use vertices_p as a pointer into Emscripten's HEAPF32, instead of using reifyArray":

https://github.com/Birch-san/liquidfun-play/blob/bb3d734a6d76ea6d775e5cc24059580770dd93e8/src/worker/debugDraw.ts#L61-L63

and then I concatenate every quad's Float32Array into one big Float32Array to give to WebGL:

https://github.com/Birch-san/liquidfun-play/blob/bb3d734a6d76ea6d775e5cc24059580770dd93e8/src/worker/index.ts#L168-L176

I was getting very high framerates (probably >60) on a boring "make loads of falling boxes" example. there was only a small amount of stuttering (probably because of the allocations I've mentioned).

I haven't implemented the entirety of DebugDraw, so it's not capable of drawing particles yet. but looking forward to seeing how it'll perform.

[Birch-san]

@SammyJoeOsborne here's my attempt to make box2d-wasm's liquidfun distribution run performantly:

Demo:
https://birchlabs.co.uk/box2d-wasm-liquidfun/

Source:
https://github.com/Birch-san/liquidfun-play

Main techniques used in main loop / render:

• Avoid GC pauses by avoiding heap allocations
  • Don't use new
  • Don't use []
  • Don't use {}
  • Allocate big Float32Arrays beforehand
    • avoid growing them
    • re-use them
• Draw using WebGL (instead of Canvas APIs)
• Box2D computation + rendering logic runs inside a Web Worker

My demo only works in Chrome/Chromium browsers because I wanted to use requestAnimationFrame inside a Web Worker.

The best way I can think of to do it in Firefox would be:

• main thread uses setTimeout(, 1 / 60) to schedule a postMessage() to ask Web Worker to run a Box2D timestep
• when Web Worker receives message, it would:
  • calculate how much time has passed since last render
  • run a timestep of that duration (but Math.min it to be no more than 1 / 60 secs)
  • clear all buffers designated for storage of render data
  • invoke debugDraw, which would write vertices into a SharedArrayBuffer
  • postMessage() to tell the main thread that SharedArrayBuffer is ready
• main thread would receive that message from Web Worker, would use requestAnimationFrame to schedule a task for the next paint
  • this task would draw contents of SharedArrayBuffer via WebGL
• repeat all of the above

I reckon that would be similarly performant, but a little harder to code. the Chrome way I've gone with is easier, and could eventually work in Firefox if I wait long enough 😛 .

[SammyJoeOsborne]

Wow, your command of typed arrays, bit shifting, matrices, etc is very impressive! I really need to get more familiar with working with typed arrays. Also very clean and well structured code all around. I don't know typescript, so I went through some of the files and most of it is completely over my head (especially the growable arrays and all the bit shifting). But all of this is spectacular. Avoiding the reifyArray calls is huge.

I may play with your source a bit to see just how many particles this thing can support at once. I know my use of Pixi.js for drawing is a bottleneck for my setup; Working directly with WebGL would definitely be faster. I should look into box2d's JSDraw more to know how it works. I just rolled my own using pixi, but its obviously not ideal.

One of my next experiments was to get Box2d running in a webworker! Perfect application for webworkers. Excited to see that it works so well!

Thank you for putting all of this together, being so communicative, and the support!