Quality facet - Add a "forbidden" words quality rule

[teolemon]

What

• Add a "forbidden" words quality facet that will look for insults, vulgar words in ingredients, product name, brands
• Add an edit rule that will send a notification to Slack as soon as one is added

Sources

• https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words
• https://github.com/marteinn/The-Big-Username-Blacklist/blob/master/list_raw.txt

Part of

• 🧽 Data quality (tracker) #5538

[CharlesNepote]

I disagree to build a new quality facet as there will be many false positives -- e.g. 7 products use "couille" as part of their name; 1689 products use "bites" in their name; etc.

There should be another process. Example:

• the system alerts some people that XXXX product may contains bad words (in real-time?)
• someone controls the product
• he makes a correction OR leave the product as-is
• he tells the system that the alert has been processed

I think ideally:

• it should be in real-time
• the system should keep the information if a product is a false positive
• it should be replayed

[teolemon]

I prefer a quick and dirty facet with false positives to RT notifications with false positives with a feedback loop that probably won't be implemented.

[CharlesNepote]

Me either! But why do you think a better solution won't be implemented? There are many developers that could do that.

And we can start with RT notifications without keeping false positive information. If our needs are clearly explained, it shouldn't be so long and difficult.

[zigouras]

I can work on this if you guys want. I think what @teolemon proposed is a good initial plan. Start with a simple quality facet first and then we can think about adding RT Slack notifications.

[VaiTon]

@zigouras you can start working on it! I'm assigning you.

[teolemon]

https://world.openfoodfacts.org/product/3168930159896/icetea-saveur-peche-lipton?rev=47

[teolemon]

https://fr.openfoodfacts.org/produit/4002566003149/sexe-amour-pipi-caca-bite-cul-gazi?rev=17

[teolemon]

Ok, we had a much more problematic issue which stayed undetected for several months.

[TaciteOFF]

Found the N word tonight, and obviously not on the product itself