Screenshots blank when taken in post-processing example

[jywarren]

First, AMAZING work especially on postprocessing in 1.7.0 - huge fan and so grateful.

Screenshots come out blank when I use this code, and I was able to confirm on the postprocessing example: https://aframe.io/aframe/examples/showcase/post-processing/ that this happens there too.

document.querySelector('a-scene').components.screenshot.capture('equirectangular');

(using code from https://aframe.io/docs/1.7.0/components/screenshot.html#methods)

The following image results (and same for perspective screenshots):

Is there an order of render change or something to do with postprocessing? The other showcase examples (in the 1.7.0 blog post) do not have this issue; they render normally.

Thank you!!

[dmarcos]

This is expected since the bloom effect uses and manages its own render target and doesn't use the one passed by the screenshot component. Post processing is still experimental

[jywarren]

Hi, thank you, is there any way to redirect the screenshot function to the alternative render target?

[jywarren]

Oof, sorry, I would offer to try to help but this code is a little out of my depth:

aframe/src/components/scene/screenshot.js

Lines 143 to 159 in e89db7d

	cubeRenderTarget = new THREE.WebGLCubeRenderTarget(
	Math.min(this.cubeMapSize, 2048),
	{
	format: THREE.RGBFormat,
	generateMipmaps: true,
	minFilter: THREE.LinearMipmapLinearFilter,
	colorSpace: THREE.SRGBColorSpace
	});
	// Create cube camera and copy position from scene camera.
	cubeCamera = new THREE.CubeCamera(el.camera.near, el.camera.far, cubeRenderTarget);
	// Copy camera position into cube camera;
	el.camera.getWorldPosition(cubeCamera.position);
	el.camera.getWorldQuaternion(cubeCamera.quaternion);
	// Render scene with cube camera.
	cubeCamera.update(el.renderer, el.object3D);
	this.quad.material.uniforms.map.value = cubeCamera.renderTarget.texture;
	size = {width: this.data.width, height: this.data.height};

Is the postprocessing code's render target accessible such that this screenshot code could reference it?

aframe/examples/showcase/post-processing/bloom.js

Lines 50 to 54 in 6a497bf

	var renderTarget = new THREE.WebGLRenderTarget(
	resolution.width,
	resolution.height,
	{ type: THREE.HalfFloatType, samples: 8 }
	);

I don't quite know enough about the architecture to know but perhaps screenshot component could have something like setRenderTarget() or a parameter like document.querySelector('a-scene').components.screenshot.setCapture('equirectangular', myRenderTarget); ?

[mrxz]

The problem isn't so much that the EffectComposer renders into its own render target, but more that it renders the final output to the screen (or WebXR swapchain texture). It will ignore the currently set render target, which breaks the screenshot component.

It is possible to instruct the EffectComposer to not output to the "screen" and use the final texture, however it shouldn't be the responsibility of the screenshot component to know about the post-processing. So there isn't really an easy fix.

Either way, an 'equirectangular' screenshot is likely going to have noticeable seams when rendered with bloom. A cleaner way would be to render the equirectangular as normal and then apply the bloom. That's another aspect of this issue, the post-processing now overwrites the renderer.render method. Even if it would render into the current render target, there is no telling if this should be the "plain" output or the post-processed output (or something else entirely). For the screenshot the post-processed output makes sense, but for a RTT-based mirror/reflection that would result in post-processing being applied twice.

The simplest approach might be to ensure that only the main call to renderer.render by <a-scene> is replaced with the call to the EffectComposer. That way any other render calls behave like before. This would mean the screenshot component will show a non post-processed output, but that's better than a blank output, I guess.

[dmarcos]

Post-processing support is still experimental. We don’t have an official API hence I would not modify any of the built-in components to accommodate just yet.

We could though add a “effects-screenshot” component in the example if someone wants to put the time. thoughts?

[mrxz]

Indeed, we shouldn't be modifying built-in components for this. There should be a clean way to replace the main render call of the scene (scene.js#L667), so that this hack in bloom.js is no longer needed:

aframe/examples/showcase/post-processing/bloom.js

Lines 89 to 97 in 6a497bf

	this.el.renderer.render = function () {
	if (isInsideComposerRender) {
	self.originalRender.apply(this, arguments);
	} else {
	isInsideComposerRender = true;
	self.composer.render(self.el.sceneEl.delta / 1000);
	isInsideComposerRender = false;
	}
	};

Currently all render-to-texture logic is broken when using post-processing.

For getting screenshots with post-processing effects enabled, a dedicated effects-screenshot component like you suggest can be considered, but that would be the next step. Though, conceptually I would like to have post-processing output into the current render target, as that would make it more versatile. Coupled with above mentioned solution, any component can choose whether it just wants to render something (renderer.render) or render the scene "as intended" (scene.renderMethod or something along those lines).