USDZ exporter premultiplies alpha in textures when Texture.premultiplyAlpha == false

[Mirei3D]

Description

When exporting textures the Canvas 2D API is used to convert Three textures into PNGs. But while input textures usually do not use premultiplied alpha, the output bitmap of a CanvasRenderingContext2D must always use premultiplied alpha for transparent colors. This implies a lossy conversion step when drawing a non-premultiplied texture to a Canvas 2D rendering context where invisible colors disappear.

I encountered this bug in the USDZ exporter but I suspect that other exporters (like the GLTF exporter) have the same bug.

Reproduction steps

1. Run the live example
2. Open the automatically downloaded USDZ in macOS Preview or iOS QuickLook
3. Check for this defect on the checkmark.

Code

// code goes here

Live example

https://jsfiddle.net/ymtn56hv/

Screenshots

Version

r171

Device

Desktop

Browser

Chrome

OS

MacOS

[Mirei3D]

I have been able to fix the issue locally by extracting texture data via a WebGL 2 context instead and using the fast-png library to encode to PNG. But it comes at a heavy performance cost, which I imagine would be even worse for non-trivial scenes, especially when converting to USDZ for iOS AR.

Do note though that this is unfinished and disregards flipY and premultipliedAlpha and doesn't implement maxTextureSize properly. As it was so slow already, I didn't bother implementing them properly, but it should be possible.

import {
    encode
} from 'fast-png';

const _gl = new OffscreenCanvas(1, 1).getContext('webgl2');

function imageToPng( image, flipY, maxTextureSize ) {

    if ( ( typeof HTMLImageElement !== 'undefined' && image instanceof HTMLImageElement ) ||
        ( typeof HTMLCanvasElement !== 'undefined' && image instanceof HTMLCanvasElement ) ||
        ( typeof OffscreenCanvas !== 'undefined' && image instanceof OffscreenCanvas ) ||
        ( typeof ImageBitmap !== 'undefined' && image instanceof ImageBitmap ) ) {

        // TODO: implement flipY, premultipliedAlpha, implement maxTextureSize properly

        let data;

        const width = image.width;
        const height = image.height;

        const glTexture = _gl.createTexture();
        const glFramebuffer = _gl.createFramebuffer();
        _gl.bindTexture( _gl.TEXTURE_2D, glTexture );
        _gl.texImage2D( _gl.TEXTURE_2D, 0, _gl.RGBA, _gl.RGBA, _gl.UNSIGNED_BYTE, image );
        _gl.bindFramebuffer( _gl.FRAMEBUFFER, glFramebuffer );
        _gl.framebufferTexture2D( _gl.FRAMEBUFFER, _gl.COLOR_ATTACHMENT0, _gl.TEXTURE_2D, glTexture, 0 );

        if ( width > maxTextureSize || height > maxTextureSize ) {

            data = new Uint8Array( 4 * maxTextureSize * maxTextureSize ); // TODO: fix texture size

            const glTextureDownScale = _gl.createTexture();
            const glFramebufferDownScale = _gl.createFramebuffer();

            _gl.bindTexture( _gl.TEXTURE_2D, glTextureDownScale );
            _gl.texStorage2D( _gl.TEXTURE_2D, 1, _gl.RGBA8, maxTextureSize, maxTextureSize ); // TODO: fix texture size
            _gl.bindFramebuffer( _gl.DRAW_FRAMEBUFFER, glFramebufferDownScale );
            _gl.framebufferTexture2D( _gl.DRAW_FRAMEBUFFER, _gl.COLOR_ATTACHMENT0, _gl.TEXTURE_2D, glTextureDownScale, 0 );
            _gl.bindFramebuffer(_gl.READ_FRAMEBUFFER, glFramebuffer);
            _gl.blitFramebuffer(0, 0, width, height, 0, 0, maxTextureSize, maxTextureSize, _gl.COLOR_BUFFER_BIT, _gl.LINEAR); // TODO: fix texture size

            _gl.bindFramebuffer( _gl.FRAMEBUFFER, glFramebufferDownScale );
            _gl.readPixels( 0, 0, maxTextureSize, maxTextureSize, _gl.RGBA, _gl.UNSIGNED_BYTE, data ); // TODO: fix texture size

            _gl.deleteFramebuffer( glFramebufferDownScale );
            _gl.deleteTexture( glTextureDownScale );

            return encode( { data: data, width: maxTextureSize, height: maxTextureSize } ); // FIXME! this leaks glFramebuffer, glTexture

        } else {

            data = new Uint8Array( 4 * width * height );

            _gl.readPixels( 0, 0, width, height, _gl.RGBA, _gl.UNSIGNED_BYTE, data );

        }

        _gl.deleteFramebuffer( glFramebuffer );
        _gl.deleteTexture( glTexture );

        return encode( { data: data, width: width, height: height } );

    } else {

		throw new Error( 'THREE.USDZExporter: No valid image data found. Unable to process texture.' );

	}

}