Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OPF data dump (jsonl.gz export) not up to date #11118

Open
raphodn opened this issue Dec 12, 2024 · 3 comments
Open

OPF data dump (jsonl.gz export) not up to date #11118

raphodn opened this issue Dec 12, 2024 · 3 comments
Labels
Data export We export data nightly as CSV, MongoDB… See: https://world.openfoodfacts.org/data 📸 Open Products Facts Our project to increase the lifespan of objects. https://world.openproductsfacts.org 🎯 P0

Comments

@raphodn
Copy link
Member

raphodn commented Dec 12, 2024 •
edited
Loading

What

In openfoodfacts/open-prices we use the data dumps to sync daily with all the products.
The OPF data dump seems to be broken, new products I add don't appear in it.

Steps to reproduce the behavior

// first download the dump

gzip -dk /home/raphael/.cache/openfoodfacts/datasets/openproductsfacts-products.jsonl.gz
grep -r 3830071491110 /home/raphael/.cache/openfoodfacts/datasets/openproductsfacts-products.jsonl

The product was added 10 days ago: https://fr.openproductsfacts.org/produit/3830071491110/the-world-game-sweetips
Display on Open Prices: https://prices.openfoodfacts.org/products/3830071491110

Thanks!
@github-project-automation github-project-automation bot moved this to To discuss and validate in 🍊 Open Food Facts Server issues Dec 12, 2024
@raphodn raphodn changed the title OPF data dump not up to date OPF data dump (jsonl.gz export) not up to date Dec 12, 2024
@alexgarel
Copy link
Member

I re-enabled the daily feeds services (don't know why they where not active on OBF, OPF and OPFF, let see tomorrow where we are !).

@alexgarel
Copy link
Member

This worked @raphodn ! exports are up to date.

@github-project-automation github-project-automation bot moved this from To discuss and validate to Done in 🍊 Open Food Facts Server issues Dec 17, 2024
@alexgarel alexgarel mentioned this issue Dec 17, 2024
Closed
@raphodn raphodn mentioned this issue Dec 21, 2024
Open
@raphodn
Copy link
Member Author

raphodn commented Dec 30, 2024

I tried the above code again, and still can't find 3830071491110 in the export 😬

@raphodn raphodn reopened this Dec 30, 2024
@teolemon teolemon added Data export We export data nightly as CSV, MongoDB… See: https://world.openfoodfacts.org/data 📸 Open Products Facts Our project to increase the lifespan of objects. https://world.openproductsfacts.org 🎯 P0 labels Dec 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Data export We export data nightly as CSV, MongoDB… See: https://world.openfoodfacts.org/data 📸 Open Products Facts Our project to increase the lifespan of objects. https://world.openproductsfacts.org 🎯 P0
Projects
🍊 Open Food Facts Server issues
Status: In Progress
🎯 Very high impact issues
Status: To do
📸 Open Products Facts
Status: To do
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexgarel @teolemon @raphodn