Switch from automatic to manual depth testing for billboards #12821

mzschwartz5 · 2025-08-15T19:43:45Z

Description

Previously, when when using terrain, clamping entities to ground, and enabling depthTestingAgainstTerrain, billboards, labels, and points would noisily clip against the terrain at global scales. (See images https://github.com/iTwin/platform-bentley-community/issues/234).

This issue isn't unexpected: billboards and others will face the camera as it moves, causing them to dip into the globe and become occluded. And when clamped to ground, the problem is exacerbated, with noisy terrain sporadically poking through the entities.

Ideally, billboards would not get clipped at this global scale, but would continue to respect terrain-depth-testing at local scales (e.g. a billboard should become invisible when behind a mountain). The solution this PR targets is described in this comment.

NOTE:

Please review commit-by-commit (starting at Moves to manual billboard depth testing in shaders). Specifically, I did some refactoring in Refactors Billboard fragment shader depth testing into functions, which will make it difficult to review all the changes together. Also, I'm going to leave comments on individual commits to help explain changes, and those will only appear in the right places if looking at those commits.

Issue number and link

#12410

Testing plan

Basic billboard clamped to terrain:

Current release vs. localhost

These should behave the same at local scales. At global scales, the PR-version behaves much better - as if terrain is disabled.

Basic billboard clamped to terrain, with `disableDepthTestDistance`:

Current release vs. localhost

Try this with various depth test distances: 0, really large (non-infinite) numbers, positive infinity, etc. Note: in the current release, there's a bug. The special 3-point depth testing of billboards ONLY applies within the disabled depth test distance region. For instance, if you set the disable distance to 1e20, you would expect pretty much no depth testing - but the 3-point testing occurs all the time now. This bug is fixed in this PR.

Another difference: with this PR, the 3-point depth testing only applies within 5000m, whereas before it would apply within whatever the depth testing distance was (or 5000m if none was set).

And, as before, on global scales, the PR-version behaves much better (as if there's no terrain to occlude).

Basic billboard, on terrain but not clamped:

Current release vs. localhost

These should behave the same at a local scale. Try playing around with different disableDepthTestingDistance settings as well; both examples should continue to behave the same, and no 3-point testing will be done.

Again, on global scales, the PR-version behaves much better.

Billboards among 3D models (regression test)

Current release vs. localhost

Should have identical behavior. However, if you turn on CLAMP_TO_3D_TILE, the current build's billboard clips through the models (because, I believe, with clamping disableDepthTestDistance defaults to 5000m, but with no terrain, it doesn't do the 3-point testing). In the PR-version, this is fixed.

Basic billboard on GP3D tiles

Current release vs. localhost

At global scales, PR-version behaves much better. At local scales, current release has the same bug described before: it clips through the globe because of the default 5km depth test distance and no globe). The PR-version behaves better, but doesn't do the 3-point testing because, again, no globe or terrain.

Billboards with translucent background (no terrain)

Current release vs. localhost

The behavior here differs slightly, but I think it's improved. In the current version, if the blue billboard is on top of the red one, you just see blue. If the red one is on top, they blend to make purple (as expected). In the PR-version, you get purple no matter which is on top.

Current billboard translucency:

CurrentTranslucency.mov

PR billboard translucency:

DevTranslucency.mov

Author checklist

I have submitted a Contributor License Agreement
I have added my name to CONTRIBUTORS.md
I have updated CHANGES.md with a short summary of my change
I have added or updated unit tests to ensure consistent code coverage
I have updated the inline documentation, and included code examples where relevant
I have performed a self-review of my code

github-actions · 2025-08-15T19:44:06Z

Thank you for the pull request, @mzschwartz5!

✅ We can confirm we have a CLA on file for you.

mzschwartz5 · 2025-08-15T19:45:10Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

 #endif

 #ifdef VERTEX_DEPTH_CHECK
 if (lengthSq < disableDepthTestDistance) {


I removed the labelTranslate / pass in 0 now. I honestly don't know the original reason for it - the visual results seem much cleaner to me without it. With it, there's like padding around the billboards that causes it to render way after it's occluded by terrain.

jjspace · 2025-08-19T15:53:40Z

This definitely seems improved at the global scale. However I noticed some weird "flickering" when zooming more locally. I assume this is just due to sampling or terrain loading or I just got lucky with the billboard "point" between some rocks that only obstruct it at certain angles. Probably still better than it was but thought it was worth noting.

simplescreenrecorder-2025-08-19_11.45.00.mp4

I also wanted to confirm, is this expected to impact 3dTiles or only Terrain? I'm still seeing it clip with the google 3d tiles

mzschwartz5 · 2025-08-19T16:00:11Z

I'm inclined to think the flickering predates this PR, as my changes should only make things more permissive on a global scale. Unless the labelTranslate field I messed with is responsible. I can look into it. Could also just be terrain loading as you said, it's hard to tell.

Interesting that it's still clipping on 3D tiles... I expected this to work on both terrain and 3D tiles, and I thought it did, in my testing.

edit- nope, this fix definitely only works for terrain. Will continue investigation into whether it can be extended to 3D tiles.

This reverts commit 4c186a2.

mzschwartz5 · 2025-09-03T18:17:09Z

packages/engine/Source/Scene/BillboardCollection.js


  this._highlightColor = Color.clone(Color.WHITE); // Only used by Vector3DTilePoints
+  this._coarseDepthTestDistance = 50000.0;
+  this._threePointDepthTestDistance = 5000.0;


Previously, the distance at which the three-point depth testing technique occurred was set to the disableDepthTestDistance, or 5km by default if not set. Setting it to disableDepthTestDistance was a strange and buggy choice, though, because the original author misinterpreted the setting. They reversed the meaning to only 3-point depth test within that distance.

Since the original intent was to make close up viewing of billboards clamped to terrain better, I think always having it be 5km (while respecting the disableDepthTestDistance value) is okay.

Can you explain this part (3 point depth test and the bug around misuse of disableDepthTestDistance) again more thoroughly, or point to where it is already written up? I remember talking about this in-person, but I want to make sure I (and others) can understand it clearly.

Yeah, I'll try my best - it's a little hard to wrap one's head around. First of all, this blog describes the qualitative behavior of billboards clamped to terrain, before this PR. tl;dr - in order to improve billboard visibility, a special 3-point test is performed. The idea is: if any of 3 key points on a billboard is visible, the whole billboard should be visible. If all are occluded, the whole billboard gets occluded.

This special 3-point test was (before this PR) applied only if the camera was within a certain distance of the billboard. The way this distance was determined is what was buggy. The logic was, "if the camera is closer than disableDepthTestDistance, do this 3-point test." (And if disableDepthTestDistance is undefined, default to 5km). But this actually defeats the point ofdisableDepthTestDistance. disableDepthTestDistance is supposed to disable occlusion within a given distance, but this 3-point test was actually fully occluding the billboard within that distance, if all 3 key points were occluded.

With this PR, I just hardcoded the distance to 5km - but, importantly, it should also respect disableDepthTestDistance now. Also importantly, the 3-point test still does what it was intended to do: preserve visibility of terrain-clamped billboards when viewed close up.

mzschwartz5 · 2025-09-03T18:19:47Z

packages/engine/Source/Scene/BillboardCollection.js

    isHeightReferenceClamp(billboard.heightReference) &&
    frameState.context.depthTexture;
-  if (!defined(disableDepthTestDistance)) {
-    disableDepthTestDistance = clampToGround ? 5000.0 : 0.0;


This is what I mentioned above. Removing in favor of always using a 5km distance.

mzschwartz5 · 2025-09-03T18:21:17Z

packages/engine/Source/Scene/BillboardCollection.js

        depthTest: {
-          enabled: true,
-          func: WebGLConstants.LESS,
+          enabled: false,


The crux of this PR's change. Disable automatic depth testing and do it manually in the shaders so we have more control of how it happens.

Note: Given the comment below about translucency, I'm a little wary I may have introduced regressions, as I don't do a LEQUAL check in the shaders, but it seems to work better this way (see test case in PR description)

Also note- we still leave automatic write to the depth buffer enabled, we just don't test against it automatically.

mzschwartz5 · 2025-09-03T18:22:58Z

packages/engine/Source/Shaders/BillboardCollectionFS.glsl

+in vec4 v_compressed;                               // x: eyeDepth, y: applyTranslate & enableDepthCheck, z: dimensions, w: imageSize
+const float SHIFT_LEFT1 = 2.0;
+const float SHIFT_RIGHT1 = 1.0 / 2.0;
+


These changes, among several others in this PR, are just moving variables outside of macro checks. These used to be only used if #ifdef FRAGMENT_DEPTH_CHECK was defined, but now they're used all the time

mzschwartz5 · 2025-09-03T18:31:41Z

packages/engine/Source/Shaders/BillboardCollectionFS.glsl

+float getGlobeDepthAtCoords(vec2 st)
+{
+    float logDepthOrDepth = czm_unpackDepth(texture(czm_globeDepthTexture, st));
+    if (logDepthOrDepth == 0.0)
+    {
+        return 0.0; // not on the globe
+    }


This isn't a new function, really. It just breaks the old getGlobeDepth function into two pieces, so we can call this piece separately.

mzschwartz5 · 2025-09-03T18:35:32Z

packages/engine/Source/Shaders/BillboardCollectionFS.glsl

-    float temp2 = (temp - floor(temp)) * SHIFT_LEFT1;
-    bool enableDepthTest = temp2 != 0.0;
-    bool applyTranslate = floor(temp) != 0.0;
+if (!enableDepthCheck) return;


More moving of variables outside compiler macros... And now we check if enableDepthCheck is turned on outside of the macro. This used to just mean "is the 3-point depth check enabled," but I repurposed it to mean "is depth testing enabled at all".

If it is, we do all forms of depth testing below (3-point, and regular)

mzschwartz5 · 2025-09-03T18:36:16Z

packages/engine/Source/Shaders/BillboardCollectionFS.glsl

I would recommend reviewing the VS before the FS.

mzschwartz5 · 2025-09-03T18:37:12Z

packages/engine/Source/Shaders/BillboardCollectionFS.glsl

+vec2 fragSt = gl_FragCoord.xy / czm_viewport.zw;
+float globeDepth = getGlobeDepthAtCoords(fragSt);
+if (globeDepth != 0.0) {
+    float distanceToEllipsoidCenter = -length(czm_viewerPositionWC); // depth is negative by convention
+    float testDistance = (eyeDepth > -u_coarseDepthTestDistance) ? globeDepth : distanceToEllipsoidCenter;
+    if (eyeDepth < testDistance) {
+        discard;
+    }
+}


Again, the crux of this PR. This the manual depth testing that replaces automatic depth testing. Except, now, with manual control, we can depth test against a flat plane at global scales.

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

mzschwartz5 · 2025-09-03T18:42:06Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

+    float enableDepthCheck = 1.0;
 #ifdef DISABLE_DEPTH_DISTANCE
    float disableDepthTestDistance = compressedAttribute3.z;
+    if (disableDepthTestDistance == 0.0 && czm_minimumDisableDepthTestDistance != 0.0)
+    {
+        disableDepthTestDistance = czm_minimumDisableDepthTestDistance;
+    }
+
+    if (lengthSq < disableDepthTestDistance || disableDepthTestDistance < 0.0)
+    {
+        enableDepthCheck = 0.0;
+    }


I repurposed enableDepthCheck - it used to mean "is the three-point depth check applied?" But now it just means what it sounds like "do we depth test at all?"

To accommodate this change, I had to move some chunks upwards in the vert shader. But it's actually more appropriate in this location anyway; i.e. all the disableDepthTestDistance logic is together now.

mzschwartz5 · 2025-09-03T18:43:27Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

+if (lengthSq < (u_threePointDepthTestDistance * u_threePointDepthTestDistance) && (enableDepthCheck == 1.0)) {
    float depthsilon = 10.0;

-    vec2 labelTranslate = textureCoordinateBoundsOrLabelTranslate.xy;


I honestly don't know what the purpose of labelTranslate here was, but it had the (bad) effect of making 3-point testing not work until a label was wayyyy occluded by terrain. Separate from all other changes in this PR, removing labelTranslate and using vec2(0.0) looks way better.

I'll need to verify but I'd expect labelTranslate is the value of the pixelOffset, or maybe eyeOffset, label property.

If that is the case, we might need to factor that back in.

Well, I verified that both pixelOffset and eyeOffset are working just fine in this branch with both variants of the depth testing. No idea why labelTranslate was bing used then. 🤷‍♀️

This change should be OK as-is.

Actually, labelTranslate may be handling offsets from label alignments.

For example, this label is right-aligned and doesn't seem to be handled properly n three-point depth testing when goin behind terrain:

Ooh interesting... that's basically the same problem that was occurring with label translate when left-aligned.

So I'm not sure what the solution is here, but maybe I just keep it as it was so at least the behavior is the same? I might spend a little time seeing if there's a complete solution.

mzschwartz5 · 2025-09-03T18:45:14Z

@ggetz @jjspace, ready for re-review. See PR description: I left detailed test cases, and specifically read the note up top about reviewing commit-by-commit. I left a bunch of comments to guide review on one commit in particular.

ggetz · 2025-09-29T13:51:21Z

packages/engine/Source/Scene/BillboardCollection.js

+  this._coarseDepthTestDistance = 50000.0;
+  this._threePointDepthTestDistance = 5000.0;


Would it make sense for these values be either 1) exposed in the API for tweaking, or 2) specified relative to the ellipsoid size?

Yeah, maybe just relative to the ellipsoid for now? If we get requests to expose these values in the API, easy enough to add later.

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

ggetz · 2025-09-29T13:56:35Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

+        disableDepthTestDistance = czm_minimumDisableDepthTestDistance;
+    }
+
+    if (lengthSq < disableDepthTestDistance || disableDepthTestDistance < 0.0)


Pedantic, but should disableDepthTestDistance actually be called disableDepthTestDistanceSquared then?

I wouldn't call that pedantic... in fact, it may actually be a bug. I'd need to go see if compressedAttribute3.z is actually squared before being passed in. Either way, I agree that it should be updated.

It is squared before being sent to the GPU. So it's a misnomer of a variable, I'll change it.

ggetz · 2025-09-29T13:59:49Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

+if (lengthSq < (u_threePointDepthTestDistance * u_threePointDepthTestDistance) && (enableDepthCheck == 1.0)) {
    float depthsilon = 10.0;

-    vec2 labelTranslate = textureCoordinateBoundsOrLabelTranslate.xy;


I'll need to verify but I'd expect labelTranslate is the value of the pixelOffset, or maybe eyeOffset, label property.

If that is the case, we might need to factor that back in.

ggetz · 2025-09-29T14:40:25Z

packages/engine/Source/Shaders/BillboardCollectionFS.glsl

+#ifdef FS_THREE_POINT_DEPTH_CHECK
+void doThreePointDepthTest(float eyeDepth, bool applyTranslate) {
+
+    if (eyeDepth < -u_threePointDepthTestDistance) return;


Code style is to always use {} curly brackets for any if blocks.

The rest of this file has a few other occurrences. Please do a pass.

Will do. Usually prettier catches me on those but I guess not in shader code!
(My personal preference for return-only if statements is to omit braces, so these are bound to continue to leak through in the future...)

Fair! I'd love to eventually get our shader tooling up to par with what we have for JS.

ggetz · 2025-09-29T15:03:12Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

+if (lengthSq < (u_threePointDepthTestDistance * u_threePointDepthTestDistance) && (enableDepthCheck == 1.0)) {
    float depthsilon = 10.0;

-    vec2 labelTranslate = textureCoordinateBoundsOrLabelTranslate.xy;


Well, I verified that both pixelOffset and eyeOffset are working just fine in this branch with both variants of the depth testing. No idea why labelTranslate was bing used then. 🤷‍♀️

This change should be OK as-is.

ggetz · 2025-09-29T15:26:48Z

packages/engine/Source/Shaders/BillboardCollectionVS.glsl

+if (lengthSq < (u_threePointDepthTestDistance * u_threePointDepthTestDistance) && (enableDepthCheck == 1.0)) {
    float depthsilon = 10.0;

-    vec2 labelTranslate = textureCoordinateBoundsOrLabelTranslate.xy;


Actually, labelTranslate may be handling offsets from label alignments.

For example, this label is right-aligned and doesn't seem to be handled properly n three-point depth testing when goin behind terrain:

ggetz · 2025-09-29T15:46:49Z

In addition to the test cases provided here, I tested the following billboard/label properties against both "variants" of the depth testing behavior and against the default ellipsoid, terrain, and Google P3DT (3D Tiles):

pixelOffset
eyeOffset
horizontalOrigin
verticalOrigin
scale
showBackground /backgroundPadding

I noticed the following during my testing:

Is the three-point depth test expected to work for billboard/labels clamped to 3D Tiles? My expectation is no and that's totally fine for that to out of scope for this PR. But I wanted to confirm the intention.

(Also noted in https://github.com/CesiumGS/cesium/pull/12821/files#r2388391537) Horizontal label alignments do not seem to be properly handled for three-point depth tests against terrain, e.g., this label that's right-aligned:

ggetz · 2025-09-29T15:49:08Z

@mzschwartz5 Overall, this appears to be working super well! There are a few details about the behavior that need clarification, but I'm hoping they do not require a change to the overall approach here.

Could you please update CHANGES.md with a summary of the fix?

ggetz · 2025-09-29T15:51:10Z

@lilleyse Would you be able to confirm the background translucency behavior described in the PR description? I'm not quite sure what the expect behavior is by default vs with OIT.

mzschwartz5 · 2025-09-29T16:17:25Z

@ggetz

Is the three-point depth test expected to work for billboard/labels clamped to 3D Tiles? My expectation is no and that's totally fine for that to out of scope for this PR. But I wanted to confirm the intention.

That's correct. The code that determines whether 3-point testing occurs checks for depthTestAgainstTerrain - so it's terrain specific, not for 3D-tiles. I do think it could be easily enabled for 3D tiles, if we wanted to follow up with that.

lilleyse · 2025-09-29T21:47:47Z

@lilleyse Would you be able to confirm the background translucency behavior described in the PR description? I'm not quite sure what the expect behavior is by default vs with OIT.

The new behavior is consistent with how OIT works. Though I'm curious what actually caused the change.

ggetz · 2025-10-07T15:48:17Z

Though I'm curious what actually caused the change.

@lilleyse This PR changes how the depth test for billboards work in order to support manual but more robust conditions for billboard display, such as what is described in this blog post. Much of the functionality was already there, though not working as intended.

@mzschwartz5 Could you please fill in @lilleyse with any more details of the translucency updates?

Clears globe depth even when terrain depth testing on

4c186a2

mzschwartz5 commented Aug 15, 2025

View reviewed changes

mzschwartz5 mentioned this pull request Aug 15, 2025

Disable depth test against a specific asset for Primitives #12410

Open

Revert "Clears globe depth even when terrain depth testing on"

a3de865

This reverts commit 4c186a2.

ggetz assigned mzschwartz5 Aug 28, 2025

mzschwartz5 added 3 commits September 3, 2025 12:25

Moves to manual billboard depth testing in shaders

234a4a3

Renames billboard shader macros

dda4aa8

Refactors Billboard fragment shader depth testing into functions

3d51a6e

mzschwartz5 commented Sep 3, 2025

View reviewed changes

packages/engine/Source/Shaders/BillboardCollectionVS.glsl Show resolved Hide resolved

mzschwartz5 commented Sep 3, 2025

View reviewed changes

mzschwartz5 marked this pull request as ready for review September 3, 2025 18:45

mzschwartz5 requested review from ggetz and jjspace September 3, 2025 18:45

mzschwartz5 changed the title ~~Clears globe depth even when terrain depth testing on~~ Switch from automatic to manual depth testing for billboards Sep 10, 2025

mzschwartz5 requested a review from lukemckinstry September 10, 2025 17:17

mzschwartz5 added 2 commits September 19, 2025 12:25

Merge remote-tracking branch 'origin' into billboard-terrain-clip-fix

7948447

Merge remote-tracking branch 'origin' into billboard-terrain-clip-fix

2ddb704

Merge branch 'main' into billboard-terrain-clip-fix

6a5a0c4

ggetz requested changes Sep 29, 2025

View reviewed changes

Merge branch 'main' into billboard-terrain-clip-fix

9c9705c

		this._coarseDepthTestDistance = 50000.0;
		this._threePointDepthTestDistance = 5000.0;

Switch from automatic to manual depth testing for billboards #12821

Are you sure you want to change the base?

Switch from automatic to manual depth testing for billboards #12821

Conversation

mzschwartz5 commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

NOTE:

Issue number and link

Testing plan

Basic billboard clamped to terrain:

Basic billboard clamped to terrain, with disableDepthTestDistance:

Basic billboard, on terrain but not clamped:

Billboards among 3D models (regression test)

Basic billboard on GP3D tiles

Billboards with translucent background (no terrain)

Author checklist

Uh oh!

github-actions bot commented Aug 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjspace commented Aug 19, 2025

Uh oh!

mzschwartz5 commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lukemckinstry Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mzschwartz5 Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mzschwartz5 Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mzschwartz5 Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mzschwartz5 commented Sep 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mzschwartz5 Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

mzschwartz5 commented Aug 15, 2025 •

edited

Loading

Basic billboard clamped to terrain, with `disableDepthTestDistance`:

mzschwartz5 commented Aug 19, 2025 •

edited

Loading

lukemckinstry Sep 18, 2025 •

edited

Loading

mzschwartz5 Sep 3, 2025 •

edited

Loading

mzschwartz5 Sep 3, 2025 •

edited

Loading

mzschwartz5 Sep 3, 2025 •

edited

Loading

mzschwartz5 Sep 29, 2025 •

edited

Loading