Merge branch 'main' into feat/add-missing-intermediate-nutriment

.github/labeler.yml

@@ -684,6 +684,7 @@ Numbers:
   - any-glob-to-any-file: 'templates/web/pages/product/includes/nutriscore.tt.html'
   - any-glob-to-any-file: 'templates/web/pages/product/includes/nutriscore_details.tt.html'
   - any-glob-to-any-file: 'templates/api/knowledge-panels/health/nutriscore/nutriscore.tt.json'
+  - any-glob-to-any-file: 'ttemplates/api/knowledge-panels/health/nutriscore/nutriscore_2023.tt.json'
   - any-glob-to-any-file: 'tests/unit/nutriscore.t'
   - any-glob-to-any-file: 'tests/unit/ingredients_nutriscore.t'
   - any-glob-to-any-file: 'tests/unit/ingredients_nutriscore.t'

.github/workflows/crowdin-per-language.yml

@@ -22,7 +22,7 @@ jobs:
     - name: Checkout
       uses: actions/checkout@v4
     - name: Matrix
-      uses: crowdin/github-action@v2.4.0
+      uses: crowdin/github-action@v2.5.0
       with:
          upload_translations: false # default is false
          download_translations: true

.github/workflows/crowdin.yml

@@ -13,7 +13,7 @@ jobs:
     - name: Checkout
       uses: actions/checkout@v4
     - name: crowdin action
-      uses: crowdin/github-action@v2.4.0
+      uses: crowdin/github-action@v2.5.0
       with:
         upload_translations: false # default is false
         # Use this option to upload translations for a single specified language

.github/workflows/pull_request.yml

@@ -223,7 +223,7 @@ jobs:
         echo "export USER_GID=$(id -g)" >> .envrc
     - name: Test make dev
       run: |
-        make DOCKER_LOCAL_DATA="$(pwd)" SKIP_SAMPLE_IMAGES=1 dev
+        make DOCKER_LOCAL_DATA="$(pwd)" SKIP_SAMPLE_IMAGES=1 dev_no_build
         make status
     - name: Test all is running
       run: make livecheck || ( tail -n 300 logs/apache2/*error*log; docker compose logs; false )

.sonarcloud.properties

@@ -1,4 +1,4 @@
 sonar.sources=cgi,docker,lib,scripts,templates,scss,html/css,html/js,docker-compose.yml,Dockerfile,Dockerfile.frontend,gulpfile.ts,Makefile
 sonar.tests=tests
 
-sonar.exclusions=scripts/obsolete/**/*,html/css/normalize.css
+sonar.exclusions=scripts/obsolete/**/*,html/css/normalize.css,tests/integration/expected_test_results/**/*

CHANGELOG.md

@@ -1,5 +1,32 @@
 # Changelog
 
+## [2.53.0](https://github.com/openfoodfacts/openfoodfacts-server/compare/v2.52.0...v2.53.0) (2025-01-06)
+
+
+### Features
+
+* data-quality - minimum number of ingredients ([#11152](https://github.com/openfoodfacts/openfoodfacts-server/issues/11152)) ([d7881d4](https://github.com/openfoodfacts/openfoodfacts-server/commit/d7881d4b9fc8d2262508913c1ec67e5dd0ce474a)), closes [#9732](https://github.com/openfoodfacts/openfoodfacts-server/issues/9732)
+* data-quality/apply-remove_insignificant_digits-for-nutriments ([#11147](https://github.com/openfoodfacts/openfoodfacts-server/issues/11147)) ([a6df72f](https://github.com/openfoodfacts/openfoodfacts-server/commit/a6df72f381c242c41ad677b7c184df479ff23aad))
+* Top categories for Open Products Facts ([2239473](https://github.com/openfoodfacts/openfoodfacts-server/commit/2239473a993fbfe0b516633c43ea3b810dae9b4a))
+* Top categories for Open Products Facts ([#11171](https://github.com/openfoodfacts/openfoodfacts-server/issues/11171)) ([2239473](https://github.com/openfoodfacts/openfoodfacts-server/commit/2239473a993fbfe0b516633c43ea3b810dae9b4a))
+
+
+### Bug Fixes
+
+* allow serving size to be hyphenated ([#11161](https://github.com/openfoodfacts/openfoodfacts-server/issues/11161)) ([7c0df2d](https://github.com/openfoodfacts/openfoodfacts-server/commit/7c0df2d7b09ee3d3bcb872adfe2ffd37fefd1b59))
+* Correct indentation, so that CodeQL can work with the code ([#11166](https://github.com/openfoodfacts/openfoodfacts-server/issues/11166)) ([0178ac2](https://github.com/openfoodfacts/openfoodfacts-server/commit/0178ac22cd7f2324039b91d688761c61fa19cf86))
+* data quality - increase threshold for comparison between fiber and its subnutriments ([#11145](https://github.com/openfoodfacts/openfoodfacts-server/issues/11145)) ([f0a2682](https://github.com/openfoodfacts/openfoodfacts-server/commit/f0a2682ee8c33114cd9df569c965a94ab79ac11b))
+* Delete html/images/lang/de/labels/halal.90x90.png ([#11183](https://github.com/openfoodfacts/openfoodfacts-server/issues/11183)) ([80cf708](https://github.com/openfoodfacts/openfoodfacts-server/commit/80cf708e9a475589b2f753f4d3055e2aab477a0e))
+* environmental_score ([#11191](https://github.com/openfoodfacts/openfoodfacts-server/issues/11191)) ([cbe221e](https://github.com/openfoodfacts/openfoodfacts-server/commit/cbe221e40432a70a981f2544234cd80c1585763b))
+* fix OPF PR labelling ([e708ae3](https://github.com/openfoodfacts/openfoodfacts-server/commit/e708ae398fc5d3facca6f008e452fe9724d1b093))
+* fix OPF PR labelling ([#11154](https://github.com/openfoodfacts/openfoodfacts-server/issues/11154)) ([e708ae3](https://github.com/openfoodfacts/openfoodfacts-server/commit/e708ae398fc5d3facca6f008e452fe9724d1b093))
+* fixes for Green-Score ([#11155](https://github.com/openfoodfacts/openfoodfacts-server/issues/11155)) ([7287d8b](https://github.com/openfoodfacts/openfoodfacts-server/commit/7287d8b87dedcf6993f8fa96f27cc3d127bc2064))
+* green-score link ([#11146](https://github.com/openfoodfacts/openfoodfacts-server/issues/11146)) ([abf858a](https://github.com/openfoodfacts/openfoodfacts-server/commit/abf858a3f7ce1ccfe573a8146e6f49a6b73a7342))
+* nutriscore grade from category change for extra virgin olive oils ([#11156](https://github.com/openfoodfacts/openfoodfacts-server/issues/11156)) ([32d58e0](https://github.com/openfoodfacts/openfoodfacts-server/commit/32d58e026a46a5864aa9c01e0fe4fe1321c9dfb1))
+* rm nova drilldown field for beauty ([#11193](https://github.com/openfoodfacts/openfoodfacts-server/issues/11193)) ([3f5b654](https://github.com/openfoodfacts/openfoodfacts-server/commit/3f5b65444eb34709ecdb60828e37a8dc788a98e0))
+* SonarCloud issues ([#11165](https://github.com/openfoodfacts/openfoodfacts-server/issues/11165)) ([b84d545](https://github.com/openfoodfacts/openfoodfacts-server/commit/b84d545c3592bee5603dbffa54dda230a62380da))
+* warnings in import_convert_carrefour_france ([#11189](https://github.com/openfoodfacts/openfoodfacts-server/issues/11189)) ([4643e3a](https://github.com/openfoodfacts/openfoodfacts-server/commit/4643e3a79c42120026fdfa7bac936d88899b2185))
+
 ## [2.52.0](https://github.com/openfoodfacts/openfoodfacts-server/compare/v2.51.0...v2.52.0) (2024-12-18)
 
 

Makefile

@@ -10,6 +10,7 @@ SHELL := $(shell which bash)
 # some vars
 ENV_FILE ?= .env
 NAME = "ProductOpener"
+VERSION = $(shell cat version.txt)
 MOUNT_POINT ?= /mnt
 DOCKER_LOCAL_DATA_DEFAULT = /srv/off/docker_data
 DOCKER_LOCAL_DATA ?= $(DOCKER_LOCAL_DATA_DEFAULT)
@@ -85,7 +86,7 @@ _FORCE:
 # Info #
 #------#
 info:
-	@echo "${NAME} version: ${VERSION}"
+	@echo "${NAME} version: v${VERSION}"
 
 usage:
 	@echo "🥫 Welcome to the Open Food Facts project"
@@ -109,6 +110,13 @@ dev: hello build init_backend _up import_sample_data create_mongodb_indexes refr
 	@echo "🥫 You should be able to access your local install of Open Food Facts at http://world.openfoodfacts.localhost/"
 	@echo "🥫 You have around 100 test products. Please run 'make import_prod_data' if you want a full production dump (~2M products)."
 
+#-------#
+# CI    #
+#-------#
+dev_no_build: hello init_backend _up import_sample_data create_mongodb_indexes refresh_product_tags
+	@echo "🥫 You should be able to access your local install of Open Food Facts at http://world.openfoodfacts.localhost/"
+	@echo "🥫 You have around 100 test products. Please run 'make import_prod_data' if you want a full production dump (~2M products)."
+
 edit_etc_hosts:
 	@grep -qxF -- "${HOSTS}" /etc/hosts || echo "${HOSTS}" >> /etc/hosts
 

cgi/display_org_table.pl

@@ -37,7 +37,7 @@
 	display_error_and_exit($request_ref, $Lang{error_no_permission}{$lc}, 401);
 }
 
-if ((not $request_ref->{admin}) or (not $User{pro_moderator})) {
+if ((not $request_ref->{admin}) and (not $User{pro_moderator})) {
 	$log->debug("user does not have permission to view organisation list", {User_id => $User_id}) if $log->is_debug();
 	display_error_and_exit($request_ref, $Lang{error_no_permission}{$lc}, 403);
 }

docker-compose.yml

@@ -1,6 +1,4 @@
-version: "3.7"
-
-x-backend-conf: &backend-conf
+x-backend-conf: &backend-conf
   image: ghcr.io/openfoodfacts/openfoodfacts-server/backend:${TAG}
   environment:
     - PRODUCERS_PLATFORM

docker/admin-uis.yml

@@ -1,5 +1,4 @@
-version: "3.7"
-services:
+services:
   mongoku:
     image: huggingface/mongoku:1.3.0
     depends_on:

docker/dev.yml

@@ -1,6 +1,4 @@
-version: "3.7"
-
-x-backend-conf: &backend-conf
+x-backend-conf: &backend-conf
   image: openfoodfacts-server/backend:dev
   build:
     context: .

docker/devcontainer.yml

@@ -1,4 +1,3 @@
-version: "3.7"
 services:
   backend:
     build:

docker/geolite2.yml

@@ -1,5 +1,4 @@
-version: "3.7"
-services:
+services:
   backend:
     volumes:
       - geolite2:/usr/local/share/GeoLite2-Country

docker/jslint.yml

@@ -1,6 +1,4 @@
-version: "3.7"
-
-# Some tweaks to be able to run npm run lint
+# Some tweaks to be able to run npm run lint
 services:
   dynamicfront:
     command: ["npm", "run", "lint"]

docker/monitor.yml

@@ -1,4 +1,3 @@
-version: "3.7"
 services:
   apache_exporter:
     image: solsson/prometheus-exporter-apache:latest

docker/perldb.yml

@@ -1,4 +1,3 @@
-version: "3.7"
 services:
   backend:
     image: productopener-backend-perldb

docker/prod.yml

@@ -1,4 +1,3 @@
-version: "3.7"
 services:
   memcached:
     restart: always

docs/api/tutorial-dev-journey.md

@@ -1,16 +1,20 @@
 ## Tutorials for Common developer journeys
+
 ### READ only journeys
-- []()Dev Journey 1: <a href="../reference/api-tutorials/comparing-sodas.md">Comparing sodas</a>
-- []()Dev Journey 2: <a href="../reference/api-tutorials/finding-healthy-cereals.md">Finding healthy breakfast cereals</a>
+
+- Dev Journey 1: [Comparing sodas](./tutorials/comparing-sodas.md)
+- Dev Journey 2: [Finding healthy breakfast cereals](./tutorials/finding-healthy-cereals.md)
 
 ### READ and WRITE journeys
-- []()Dev Journey 3: <a href="../reference/api-tutorials/adding-missing-products.md">Adding missing products</a>
-- []()Dev Journey 4: <a href="../reference/api-tutorials/get-the-nutri-score.md">Get the Nutri-Score</a>
-- []()Dev Journey 5: <a href="../reference/api-tutorials/get-the-eco-score.md">Get the Eco-Score</a>
-- []()Dev Journey 6: <a href="../reference/api-tutorials/get-ingredient-related-analysis.md">Get ingredient related analysis on new or existing products (Nova, allergens, additives…)</a>
-- []()Dev Journey 7: <a href="../reference/api-tutorials/folksonomy-engine.md">Adding non-standard fields to a food product using Folksonomy (coming soon for cosmetics, pet food, and other products)</a>
-- []()Dev Journey 8: <a href="../reference/api-tutorials/product-prices.md">Retrieving and adding prices to products using Open Prices</a>
+
+- Dev Journey 3: [Adding missing products](./tutorials/adding-missing-products.md)
+- Dev Journey 4: [Get the Nutri-Score](./tutorials/get-the-nutri-score.md)
+- Dev Journey 5: [Get the Green-Score](./tutorials/get-the-green-score.md)
+- Dev Journey 6: [Get ingredient related analysis on new or existing products (Nova, allergens, additives…)](./tutorials/get-ingredient-related-analysis.md)
+- Dev Journey 7: [Adding non-standard fields to a food product using Folksonomy (coming soon for cosmetics, pet food, and other products)](./tutorials/folksonomy-engine.md)
+- Dev Journey 8: [Retrieving and adding prices to products using Open Prices](./tutorials/product-prices.md)
 
 ## Adding more journeys
 
-* If you feel a developer journey is missing, feel free <a href="https://github.com/openfoodfacts/openfoodfacts-server/tree/main/docs/reference/api-tutorials">to create a Pull Request</a>, or propose it at [email protected]
+* If you feel a developer journey is missing, or want to correct one,
+  feel free <a href="https://github.com/openfoodfacts/openfoodfacts-server/tree/main/docs/api/tutorials">to create a Pull Request</a>, or propose it at [email protected]

docs/api/tutorial-uploading-photo-to-a-product.md

@@ -40,7 +40,8 @@ Multilingual products have several photos based on the languages present on the
 
 The WRITE operations in the Open Food Facts API require authentication. Therefore you need a valid `user_id` and `password` to write the photo to 100% Real Orange Juice.
 
-> Sign up on the [Open Food Facts App](https://world.openfoodfacts.org/) to get your `user_id` and `password` if you dont have one. For more details, visit the [Open Food Facts Authentication](https://openfoodfacts.github.io/openfoodfacts-server/introduction/api/#authentication).
+> Sign up on the [Open Food Facts App](https://world.openfoodfacts.org/) to get your `user_id` and `password` if you dont have one.
+For more details, visit the : [Authentication paragraph in our introduction](../index.md#authentication).
 
 ## Parameters
 

docs/reference/api-tutorials/adding-missing-products.md → docs/api/tutorials/adding-missing-products.md

@@ -9,13 +9,13 @@ If you have an app that makes POST calls and you don't want your users to authen
 - `user_id`: myappname
 - `password`: 123456
 
-* For a complete reference on AUTH, please read: https://openfoodfacts.github.io/openfoodfacts-server/api/#authentication
-    
+* For a complete reference on AUTH, please read: [Authentication in our introduction](../index.md#authentication)
+
 ---
 
 ## Adding images to support your edit, and get machine learning predictions
 If you ask your users just one thing, it would be to send photos of the packaging (front, ingredients, nutrition, recycling, the more the better)<br>
-<a href="https://openfoodfacts.github.io/openfoodfacts-server/api/tutorial-uploading-photo-to-a-product/"> Please refer to our comprehensive tutorial on uploading images</a>
+Please refer to our [comprehensive tutorial on uploading images](../tutorial-uploading-photo-to-a-product.md)
 
 ---
 
@@ -57,7 +57,7 @@ You can define one or more parameters to add, for example, the brand and the Kos
 
 - `brands`: Häagen-Dazs
 - `labels`: kosher
-    
+
 
 The call looks like this:
 
@@ -94,21 +94,22 @@ It should be structured as: user-agent + user-id.
 `comment=Edit by a Healthy Choices 1.2 iOS user - SxGFRZkFwdytsK2NYaDg4MzRVenNvUEI4LzU2a2JWK05LZkFRSWc9PQ`
 
 
---- 
+---
 ## Leveraging Robotoff ML predictions to simplify life for your users
 
-Leveraging Robotoff prediction to simplify life for your users: https://openfoodfacts.github.io/openfoodfacts-server/api/intro-robotoff/
+Leveraging [Robotoff prediction](../intro-robotoff.md) to simplify life for your users.
 You can get a category prediction from images (useful for Nutri-Score and Eco-Score), you can get quality labels prediction, brand predictions, weight predictions from images.
 To achieve this you need the upload image section above. Adding data like product name, ingredients and nutrition can help, but some predictions can be done with just images.
 
 ## Getting your users to check ingredients OCR in an autonomous fashion
 
-Getting your users to check ingredients OCR in an autonomous fashion: https://openfoodfacts.github.io/openfoodfacts-server/reference/api-tutorials/get-ingredient-related-analysis/
+Getting your users to [check ingredients OCR in an autonomous fashion](./get-ingredient-related-analysis.md)
 
 ## Getting your users to input nutrition facts
-Getting your users to input nutrition facts: https://openfoodfacts.github.io/openfoodfacts-server/dev/explain-nutrition-data/
+[Getting your users to input nutrition facts](../../dev/explain-nutrition-data.md)
+
 ## Getting your users to input packaging data
-Getting your users to input packaging data: https://openfoodfacts.github.io/openfoodfacts-server/dev/explain-packaging-data/
+Getting your users to [input packaging data](../../dev/explain-packaging-data.md)
 
 ## Showing the result of their work
 - You can then reload the product, potentially using Knowledge Panels, to show the rewarding result of their work.

docs/reference/api-tutorials/creating-a-local-cache-of-open-food-facts-data.md → docs/api/tutorials/creating-a-local-cache-of-open-food-facts-data.md

@@ -11,7 +11,7 @@ A local cache is a copy of OFF data stored directly on your system or server. Th
 
 -   **FoodVisor Contributed (Python/MongoDB) backend:** The FoodVisor startup contributed a few years ago <a href="https://github.com/openfoodfacts/openfoodfacts-apirestpython">a Python-based backend with a MongoDB export</a>, providing a solid starting point for caching in Python environments.
 -   **Project-Specific Caches:** Several OFF projects like open-prices and robotoff have implemented local caches for their own needs. While not immediately reusable, they can serve as valuable references.
--   **SDKs** We have <a href="https://openfoodfacts.github.io/openfoodfacts-server/api/#sdks">a number of official SDKs</a> that can be leveraged as part of a caching backend. Please leverage and contribute to those 🙏
+-   **SDKs** We have [a number of official SDKs](../api.md#sdks) that can be leveraged as part of a caching backend. Please leverage and contribute to those 🙏
 -   **You can start a project within Open Food Facts to solve this**
 
 ### Need for Diverse Solutions
@@ -25,7 +25,7 @@ For applications primarily focused on user-generated requests, a local cache may
 ### Licensing and Data Sharing
 
 Even when using a local cache, you're still bound by the Open Database License (ODbL). **Do not mix OFF data with external product data**. All additions or modifications made to OFF data must be shared back to OFF, preferably through the WRITE API. Consider incorporating this functionality into your cache implementation.
-For more on legal issues, <a href="https://openfoodfacts.github.io/openfoodfacts-server/reference/api-tutorials/license-be-on-the-legal-side/">please read this page</a>
+For more on legal issues [please read this page](./license-be-on-the-legal-side.md)
 
 ### Challenges of Cache Maintenance