DataTexture: Proposal to support partial update

[agargaro]

Description

Hi!

I am using DataTexture quite a lot to handle data with BatchedMesh and InstancedMesh2 (InstancedMesh + indirection).

In my case, I would like to update the color of only one instance (on the mouse over), but send the gpu the whole texture is expensive because it's very large.

I tried using the WebGLRenderer.copyTextureToTexture method but it doesn't work when src and dest are the same texture (I might open a separate bug for this).

Anyway, this method is useless if BatchedMesh automatically updates .needsUpdate flag which will make the whole texture update anyway.

It would be fantastic to have a partial update system like BufferAttribute.

I know that it's an important change, but if you want I can help.

Thank you for all the work you do. 😄

Solution

Implementing an addUpdateRange method similar to that of BufferAttribute.

.addUpdateRange ( start : Box2, count : Box2 ) : this

Alternatives

Fix and use WebGLRenderer.copyTextureToTexture, but we should remove .needsUpdate = true from BatchedMesh?

Additional context

No response

[RenaudRohlinger]

Nice to keep this idea on the table. So essentially, adding x, y, width, height parameters to [compressed]texSubImage(2D/3D).

By the way for your specific case, you could take this a step further by avoiding CPU-to-GPU stalls entirely through the use of Pixel Buffer Objects (PBOs). Adding a new API in Three.js that would be designed to enable asynchronous data transfers by leveraging gl.PIXEL_UNPACK_BUFFER. Instead of directly interacting with the GPU as we currently do, the CPU would write data to the PBO, which acts as a staging area, and the GPU processes the buffer asynchronously. This would eliminate blocking and probably improves by a lot your advanced BatchedMesh performances.

For example:

// Bind the PBO for writing (created on the first upload with `texImage2D` for example)
gl.bindBuffer(gl.PIXEL_UNPACK_BUFFER, pbo);

// Write pixel data to the PBO
const pixelData = new Uint8Array(bufferSize); // Your pixel data
gl.bufferSubData(gl.PIXEL_UNPACK_BUFFER, 0, pixelData);

// Upload the data from the PBO to the texture
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texSubImage2D(
    gl.TEXTURE_2D,
    0,               // Mipmap level
    0, 0,            // x and y offsets
    textureWidth,     // Width of the sub-image
    textureHeight,    // Height of the sub-image
    gl.RGBA,          // Format
    gl.UNSIGNED_BYTE, // Type
    0                 // Offset in the PBO
);

// Unbind the PBO
gl.bindBuffer(gl.PIXEL_UNPACK_BUFFER, null);

Basically:
No PBO:

CPU --> GPU (Direct transfer)
         [GPU busy → CPU stalls]

PBO:

CPU --> PBO (Asynchronous transfer)
PBO --> GPU (GPU reads later when ready)
         [CPU free → No stall]

Basically, the PBO serves as a staging area in GPU-accessible memory. Pixel data is first copied into the PBO, and the GPU reads from it at a later time. Spamming DataTexture update wouldn't stall the CPU anymore, adding to the command queue of the Pixel Buffer Object instead, and get processed asynchronously.

[gkjohnson]

I think this would be a great addition - even for any texture. Uploading the whole image for just a single pixel is a waist. A couple comments - I think the signature can be just a single box, right? As in .addUpdateRange ( box : Box2 )

I tried using the WebGLRenderer.copyTextureToTexture method but it doesn't work when src and dest are the same texture (I might open a separate bug for this).

I think the copyTextureToTexture mental model is different to what you're interested in. Due to copying operations the contents of a texture can become out-of-sync with the gpu-memory representation and copyTextureToTexture is designed to copy between gpu-memory instances. Adding addUpdateRange seems like a good, API-consistent addition.

I'm very supportive of this addition.

@RenaudRohlinger

Adding a new API in Three.js that would be designed to enable asynchronous data transfers by leveraging gl.PIXEL_UNPACK_BUFFER. Instead of directly interacting with the GPU as we currently do, the CPU would write data to the PBO, which acts as a staging area, and the GPU processes the buffer asynchronously.

This is interesting. Is there a reason to not always do this when uploading DataTextures? It seems like a strict improvement without drawbacks? Does it need a new three.js API?

[RenaudRohlinger]

This implies an additional array buffer (or 2, one write and another one read to prevent block in lockstep) per DataTexture at initialization:

// Create a PBO
const pbo = gl.createBuffer();

// Bind it to PIXEL_UNPACK_BUFFER
gl.bindBuffer(gl.PIXEL_UNPACK_BUFFER, pbo);

// Allocate storage (size in bytes)
const bufferSize = textureWidth * textureHeight * 4; // Assuming RGBA, 4 bytes per pixel
gl.bufferData(gl.PIXEL_UNPACK_BUFFER, bufferSize, gl.STREAM_DRAW);

// Unbind the PBO
gl.bindBuffer(gl.PIXEL_UNPACK_BUFFER, null);

So I guess we would need to consider the performance benefit/cost before thinking about adding this to the core. But I agree that it sounds like a direct improvement.

[gkjohnson]

It's true that this would cause unnecessary resource creation. I think PBO support for faster uploads can be considered in a separate issue - perhaps a flag can be added to enabled / disable this use.

Before any work is done on this I think it would be good to get opinions from @Mugen87 or @mrdoob. But I think this is a good change.