Feature Request: Backface Culled Quadstrip Brushes

[rockround]

Hi, I'd like to request backface culled quadstrip brushes as a feature.

Currently, OpenBrush's quadstrip-based brushes use a double-sided shader with single-sided geometry. While this allows for two-sided rendering at half the geometric cost, it also disables culling.

I would like to request for a new set of brushes for the QuadStripBrushes which have backface culling enabled (render front side).

These brushes would need to be exposed at the editor level (UnitySDK or OpenBrush environment) so that artists can paint cull-enabled strokes knowing that the other side isn't rendered.

Use case:
By having both for instance Ink and InkCulled, an artist could choose to paint a tree trunk with InkCulled so that the inside isn't rendered, while using Ink to paint the leaves so that both sides are visible in rendering.

This reduces the trunk's rendering cost by 50% in any normal rendering system.

The difference in painting medium would need to be reflected in the exports as well.

For the sketch file, the metadata would need to reflect that a different material is used for rendering InkCulled vs Ink.

For exported meshes, the batched groups of brush types would need to have Ink and InkCulled separated. This is because only one material can be applied to a given mesh in a draw call. However, this functionality should follow with the OpenBrush environment batching modification.

Implementation-wise, it should be similar to the process detailed in this link (though no superseding would be happening)
https://github.com/googlevr/tilt-brush/tree/c5fb8671934fe4f164ce315c675367d46267b1b5/Assets/Resources/Brushes

UI-wise, it may be as simple as a checkbox in the pallette menu of a given quadstrip brush. The toggle would make it choose the Culled version of the material, which AFAIK should be reflected in both runtime batching and exporting.

This feature would improve things because it would address one of the major bottlenecks in rendering brush strokes.
While frustum culling leads to some degree of improvement, separating distinct objects has the same effect.
Occlusion culling would likely benefit rendering the most, but it is complicated and would need to take place in the GPU.
However, backface culling is an easy implementation that should drastically improve performance, especially because one of the appeals of TiltBrush/ OpenBrush is 3D painting, which requires "shells" to create volumetric materials, which currently have the back faces rendered unnecessarily.

If it seems like a lot of work, let me know what I can do to make this a reality.

[andybak]

Thanks for this. It's clearer now we've got a writeup that explains in detail all in one place.

I'd like to find a way to do this without adding user-visible variants for every affected brush. Maybe a modal toggle to switch between the two versions of each brush? We'd need to add metadata so that the culled/unculled pairs can be easily identified. As long as this resulted in the correct brush being used then the export should just work. (Maybe we could use a naming convention instead of metadata?)

It's not clear to me how much demand there is for this feature and therefore how much to prioritise it. It would be good to get more feedback from artists to help gauge demand. I know Zandy produces work that would benefit from this. I'll try and get their input.

If there was an approach that would give 80% of the benefit with less effort, I'd be interested. Would it work if this functionality was initially just part of Tilt Brush toolkit? i.e. the user could load his sketch in Unity and selectively reassign parts of it to use the correct culled brush?

[andybak]

In some ways it would be much easier just to add new brushes for each type as that way we don't have to mess with the UI code as much. But I think that's a high price to pay (adding dozens of new brushes to the brush pallette)

[rockround]

With the number of quadstrip brushes, I would think that having a checkbox below the color wheel in advanced mode would be ideal - doesn't add all the different brushes to scroll through. Though I haven't looked at the UI code so I wouldn't know.

Implementation-wise, it would need to live in OpenBrush, since artists will be the ones that choose between culled and non-culled versions of a brush. The use case of painting a tree demonstrates this - non-culled for leaves, since both sides of a single stroke should be visible, and culled for the bark since only the outwards wood is visible.

The above use-case should be a good example of how it could benefit the system - this feature isn't too straightforward because it impacts how people can use the art the most, especially if the viewing platform doesn't have a powerful GPU.

If it were to be on the toolkit side, the meshified strokes that come in would have both culled and non-culled strokes batched in the same mesh, because they would have been exported on the same material. They would also be rendered with the same material in the toolkit, since it doesn't know if a face should or shouldn't be backface culled.
If it lived on the toolkit, and strokes could be selectively modified, then it may be viable to leave it there. Though implementation-wise it looks like it might be more work to add that functionality...

In terms of a minimum effort implementation - it seems that OpenBrush has applied google's double-sided variant change to builds, while the metadata and materials for the single-sided variants remain, just that they aren't visible.

If the BrushDescriptor asset for the SingleSided variants (for instance Brushes/Basic/OilPaintSingleSided.asset) had "Render Backfaces" checked off, with OilPaint.asset's "Supersedes" set to nothing, it should yield the desired results - single-sided geometry, with only the front face being rendered.

A simple test would look like:
Preparing assets:
Do the above process on the OilPaint brush assets

In-game:
If cull checkbox ticked in brush options, change BrushDescriptor to the SingleSided variant.

[andybak]

Just thought about a potential complication - shadows. The tree example would be fine but if someone painted walls with a single-sided brush then shadowing would behave unexpectedly (light would pass through the back face).

Not sure how much of a big deal this is but it's worth noting that there's a potential gotcha here.

I'm tempted to add a few single sided brush variants in the experimental build (as completely separate brushes) and get some feedback from the community about their effectiveness. If everyone loves them then we can do the clever UI stuff.

[rockround]

True, though IIRC the light pass of shaders can be modified to avoid this issue (I think I read something on Unity Forums regarding it at one point).

I would appreciate it if this could be implemented in a build. Could it be done for Paper, Ink, WetPaint, OilPaint, and Flat?

[andybak]

I can do the simple bit this week (just adding some duplicate brushes). Not sure how easy it is to get those supported in all export formats though. Can you make do without exports initially? (or fix the exports after the fact)

[andybak]

Looks like half the work has already been done:

[rockround]

Yep, so it should hopefully save some time on experimentation.

[andybak]

I realise now that's what you meant above - that the variants already exist.

[TimAidley]

FWIW I think it's not worth spending much time on this until you've verified that it makes much of a difference performance-wise.

Backface culling is only really going to help if you find yourself fillrate limited, and when I was working on the Quest version of TB, I found that often for complex scenes most of the memory bandwidth was being spent on vertices.

I concede that there may be artistic reasons for implementing this - walls that are only visible in one direction, for instance - but I don't think you should expect much of a performance gain.

[rockround]

That makes sense, it is what I'm targeting after all haha.
Anything I can do to help make it happen? Aside from optimizing my game, I would like to think that expanding the use-cases of generated art would be beneficial to both artists and consumers - AFAIK artists need to put substantial effort into optimizing their sketches in blender to get it to render well on things like VRChat.

Hmm, what if I also added URP support for brushes (maybe not all of them at first)? I'm using URP for my game anyways, and have already converted around half of the quadstrip brushes.

[andybak]

There's definitely a lot of interest in URP support in Tilt Brush Toolkit and I actually made a start on the shaders myself. I'll need to refresh my memory about how far I got but I basically tackled the easy ones in an afternoon.

The other thing is to provide some evidence that this will have a significant performance improvement that justifies:

1. The dev time
2. The increase in UI complexity
3. The added complexity for the user (having to think about single vs double sided whilst painting)

That does set a fairly high bar. I think a lot of artists are very interested in improving performance but I do think there's probably some low-hanging fruit we could tackle first.

[andybak]

Of course - another thing you could do would be to do the work yourself. That solves a lot of the problems as it could be in an alternate build for now. I'd be happy to help out where I can - but it's a lot easier to help someone else with their changes than it is to find the time to do it myself.

[rockround]

Huh, that option strangely never occurred for me.

I'll clone the repo and see how it turns out.

[andybak]

Cool. I've also made a start on ability to export batches per stroke instead of per brush or per colour. I can show you how far I got with that.