[Bug] Category Filters filtering the wrong or extra datapoints

[Jon-fiddler]

Description

When using the datafilterextension and using category filters, i am experiencing some odd behavior when I go to filter out the datapoints with the first category option, it also filters out the next data point right after.

Example:
data => [{...dataStuff, field: 'a'}, {...dataStuff, field: 'b'}, {...dataStuff, field: 'c'}, {...dataStuff, field: 'd'}...]
if I go to apply a filter to show only 'b' and 'c' by removing the 'a' then both 'a' and 'b' are hidden.
If i replace the 2nd data with the 3rd, then both 'a' and 'c' will be hidden.
If i keep 'a', but filter out just 'b' or 'c', then it works fine.

CodeSandbox: https://codesandbox.io/p/sandbox/trusting-chatterjee-7dxgrv

Also seeing an oddity if I filter our everything except "d", then it shows only 'b' and 'c'? Maybe I am doing it wrong and if so please inform me how.

Flavors

• Script tag
• React
• Python/Jupyter notebook
• MapboxOverlay
• GoogleMapsOverlay
• CartoLayer
• ArcGIS

Expected Behavior

When excluding the first data point in the list of data then the next datapoint with a different value would also not be filtered out.

Steps to Reproduce

CodeSandbox: https://codesandbox.io/p/sandbox/trusting-chatterjee-7dxgrv

Environment

• Framework version:
• Browser:
• OS:

Logs

No response

[felixpalmer]

Working fine here:

Screen.Recording.2024-09-20.at.09.55.12.mov

Can you share more environment info?

[Jon-fiddler]

Thats crazy! Here is what I see in the above codesandbox example:

Screen.Recording.2024-09-20.at.10.44.09.AM.mov

[felixpalmer]

Can you share more environment info?

☝️

[Jon-fiddler]

Macbook Pro
Chip: Apple M2 Max
Memory: 32 GB
MacOS Sonoma 14.5

Graphics Feature Status

• Canvas: Hardware accelerated
• Canvas out-of-process rasterization: Enabled
• Direct Rendering Display Compositor: Disabled
• Compositing: Hardware accelerated
• Multiple Raster Threads: Enabled
• OpenGL: Enabled
• Rasterization: Hardware accelerated
• Raw Draw: Disabled
• Skia Graphite: Disabled
• Video Decode: Hardware accelerated
• Video Encode: Hardware accelerated
• WebGL: Hardware accelerated
• WebGL2: Hardware accelerated
• WebGPU: Hardware accelerated
• WebNN: Disabled

WebGL Renderer: WebKit WebGL
WebGL Vendor: WebKit
WebGL Version: WebGL 1.0 (OpenGL ES 2.0 Chromium)
Shading Language Version: WebGL GLSL ES 1.0 (OpenGL ES GLSL ES 1.0 Chromium)

What other information would you like?

[felixpalmer]

Environment

Framework version:
Browser:
OS:

[finico]

I have the same issue

Framework version: 9.0.38
Browser: Chrome 131.0.6778.265 (Official Build) (arm64)
       & Firefox 134.0 (aarch64)
       & Safari 18.2 (20620.1.16.11.8)
OS: macOS Sequoia 15.2 (24C101)

I initially noticed this when filtering results with different order in the data. If an item with category 'a' is not first in the array, and filterCategories is ['a'] - it will be like this

I looked through the code of data-filter/data-filter-extension and data-filter/shader-module - I found nothing suspicious... but by brute force I realized that the difference is in 2.. and when I put after

deck.gl/modules/extensions/src/data-filter/data-filter-extension.ts

Lines 366 to 377 in 294b6f3

	for (let c = 0; c < categoryFilters.length; c++) {
	const categoryFilter = categoryFilters[c];
	for (const category of categoryFilter) {
	const key = extension._getCategoryKey.call(this, category, c);
	if (key < maxCategories) {
	const channel = c * (maxCategories / 32) + Math.floor(key / 32);
	categoryBitMask[channel] += Math.pow(2, key % 32); // 1 << key fails for key > 30
	} else {
	log.warn(`Exceeded maximum number of categories (${maxCategories})`)();
	}
	}
	}

this ugly af code (with screenshot)

switch (maxCategories) {
    case 128:
        if (categoryBitMask[0] % 2 === 0) {
            categoryBitMask[0] += 2
        }
        break
    case 64:
        if (categoryBitMask[0] % 2 === 0) {
            categoryBitMask[0] += 2
        }
        if (categoryBitMask[2] % 2 === 0) {
            categoryBitMask[2] += 2
        }
        break
    case 32:
        if (categoryBitMask[0] % 2 === 0) {
            categoryBitMask[0] += 2
        }
        if (categoryBitMask[1] % 2 === 0) {
            categoryBitMask[1] += 2
        }
        if (categoryBitMask[2] % 2 === 0) {
            categoryBitMask[2] += 2
        }
        if (categoryBitMask[3] % 2 === 0) {
            categoryBitMask[3] += 2
        }
        break
}

compare with first screenshot in the comment

Then the filter starts working properly.. But I have no idea what the exact cause is... Apple's GPU (I have this issue on iPhone as well) or something else

[felixpalmer]

@finico interesting, could you make a PR with the changes so I can review better? It's not clear to me exactly how you have modified the code. Can you also include the dataset used to create this repro?

[finico]

@felixpalmer I first noticed it on my data in the app, but it reproduces on any data.

could you make a PR with the changes so I can review better?

I think it's too crooked for PR.

https://stackblitz.com/edit/vitejs-vite-xuj2ug7w?file=src%2FPatchedDataFilterExtension.ts
I made a demo based on your app and added a patched version of the extension, which enables by the “patch” checkbox. The patch inside the method is highlighted in //#region.

And here is a video of how it works for me

deckgl-filter.mov

[felixpalmer]

@finico thank you for putting in the effort to share the repro. I've tried you code and it isn't working on my platform - when I enable the patch, then the filter results are incorrect. I think what your code is doing is somehow correcting for a floating-point precision issue that is present on your platform.

@finico @Jon-fiddler I've put together a PR with what I think should be a more robust approach, could you try it on your system and let me know if it fixes the issue?

[finico]

@felixpalmer I can confirm that on the locally built version from the branch the filtering works correctly. Thank you!