Skip to content

Latest commit

 

History

History
206 lines (146 loc) · 11.1 KB

dev-environment-quick-start-guide.md

File metadata and controls

206 lines (146 loc) · 11.1 KB

Dev environment quick start guide

This guide will allow you to rapidly build a ready-to-use development environment for Product Opener running in Docker. As an alternative to setting up your environment locally, follow the Gitpod how-to guide to instantly provision a ready-to-code development environment in the cloud.

First setup time estimate is ~10min with the following specs:

  • 8 GB of RAM dedicated to Docker client
  • 6 cores dedicated to Docker client
  • 12 MB/s internet speed

1. Prerequisites

Docker is the easiest way to install the Open Food Facts server, play with it, and even modify the code.

Docker provides an isolated environment, very close to a Virtual Machine. This environment contains everything required to launch the Open Food Facts server. There is no need to install Perl, Perl modules, Nginx, nor Apache separately.

Installation steps:

If you run e.g. Debian, don't forget to add your user to the docker group!

2. Clone the repository from GitHub

You must have a GitHub account if you want to contribute to Open Food Facts development, but it’s not required if you just want to see how it works.

Be aware Open Food Facts server takes more than 1.3 GB (2019/11).

Choose your prefered way to clone, either:

On Windows:

If you are running Docker on Windows, please use the following git clone command:

git clone -c core.symlinks=true https://github.com/openfoodfacts/openfoodfacts-server.git

or

git clone -c core.symlinks=true [email protected]:openfoodfacts/openfoodfacts-server.git

On other systems:

git clone [email protected]:openfoodfacts/openfoodfacts-server.git

or

git clone https://github.com/openfoodfacts/openfoodfacts-server.git

Go to the cloned directory:

cd openfoodfacts-server/

3. [Optional] Review Product Opener's environment

Note: you can skip this step for the first setup since the default .env in the repo contains all the default values required to get started.

Before running the docker-compose deployment, you can review and configure Product Opener's environment (.env file).

The .env file contains ProductOpener default settings:

Field Description
PRODUCT_OPENER_DOMAIN Can be set to different values based on which OFF flavor is run.
PRODUCT_OPENER_PORT can be modified to run NGINX on a different port. Useful when running multiple OFF flavors on different ports on the same host.
Default port: 80.
PRODUCT_OPENER_FLAVOR Can be modified to run different flavors of OpenFoodFacts, amongst openfoodfacts (default), openbeautyfacts, openpetfoodfacts, openproductsfacts.
PRODUCT_OPENER_FLAVOR_SHORT can be modified to run different flavors of OpenFoodFacts, amongst off (default), obf, oppf, opf.
PRODUCERS_PLATFORM can be set to 1 to build / run the producer platform.
ROBOTOFF_URL can be set to connect with a Robotoff instance.
GOOGLE_CLOUD_VISION_API_KEY can be set to enable OCR using Google Cloud Vision.
CROWDIN_PROJECT_IDENTIFIER and CROWDIN_PROJECT_KEY can be set to run translations.
GEOLITE2_PATH, GEOLITE2_ACCOUNT_ID and GEOLITE2_LICENSE_KEY can be set to enable Geolite2.
TAG Is set to latest by default, but you can specify any Docker Hub tag for the frontend / backend images.
Note that this is useful only if you use pre-built images from the Docker Hub (docker/prod.yml override); the default dev setup (docker/dev.yml) builds images locally

The .env file also contains some useful Docker Compose variables:

  • COMPOSE_PROJECT_NAME is the compose project name that sets the prefix to every container name. Do not update this unless you know what you're doing.
  • COMPOSE_FILE is the ;-separated list of Docker compose files that are included in the deployment:
    • For a development-like environment, set it to docker-compose.yml;docker/dev.yml (default)
    • For a production-like environment, set it to docker-compose.yml;docker/prod.yml;docker/mongodb.yml
    • For more features, you can add:
      • docker/admin-uis.yml: add the Admin UIS container
      • docker/geolite2.yml: add the Geolite2 container
      • docker/perldb.yml: add the Perl debugger container
  • COMPOSE_SEPARATOR is the separator used for COMPOSE_FILE.

Note: Instead of modifying .env (with the risk commit it inadvertently), You can also set needed variables in your shell, they will override .env values. Consider creating a .envrc file that you source each time you need to work on the project. On linux and macOS, you can automatically do it if you use direnv.

4. Build your dev environment

From the repository root, run:

make dev

Note:

If you are using Windows, you may encounter issues regarding this command. Take a look at the Troubleshooting section further in this tutorial.

Note:

If docker complains about

ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

It can be solved by adding {"base": "172.80.0.0/16","size": 24}, {"base": "172.90.0.0/16","size": 24} to default-address-pools in /etc/docker/daemon.json and then restarting the docker daemon. Credits to https://theorangeone.net/posts/increase-docker-ip-space/ for this solution.

The command will run 2 subcommands:

  • make up: Build and run containers from the local directory and bind local code files, so that you do not have to rebuild everytime.
  • make import_sample_data: Load sample data into mongodb container (~100 products).

Notes:

  • The first build can take between 10 and 30 minutes depending on your machine and internet connection (broadband connection heavily recommended, as this will download Docker base images, install Debian and Perl modules in preparation of the final container image).

  • You might not immediately see the test products: create an account, login, and they should appear.

  • For a full description of available make targets, see docker/README.md

Hosts file:

Since the default PRODUCT_OPENER_DOMAIN in the .env file is set to openfoodfacts.localhost, add the following to your hosts file (Windows: C:\Windows\System32\drivers\etc\hosts; Linux/MacOSX: /etc/hosts):

127.0.0.1 world.openfoodfacts.localhost fr.openfoodfacts.localhost static.openfoodfacts.localhost ssl-api.openfoodfacts.localhost fr-en.openfoodfacts.localhost

You're done ! Check http://openfoodfacts.localhost/ !

Going further

To learn more about developing with Docker, see the Docker developer's guide.

To have all site page on your dev instance, see Using pages from openfoodfacts-web

Using Repl offers you a way to play with perl.

Visual Studio Code

WARNING: for now this is deprecated, some work needs to be done.

This repository comes with a configuration for Visual Studio Code (VS Code) development containers (devcontainer). This enables some Perl support in VS Code without the need to install the correct Perl version and modules on your local machine.

To use the devcontainer, install prerequisites, clone the repository from GitHub, and (optionally) review Product Opener's environment. Additionally, install Visual Studio Code. VS Code will automatically recommend some extensions, but if you don't want to install all of them, please do install Remote - Containers manually. You can then use the extension command Remote-Containers: Reopen Folder in Container, which will automatically build the container and start the services. No need to use make!

Troubleshooting

make dev error: make: command not found

When running "make dev":

bash: make: command not found

Solution: Click the Windows button, then type “environment properties” into the search bar and hit Enter. Click Environment Variables, then under System variables choose Path and click Edit. Click New and insert C:\Program Files (x86)\GnuWin32\bin, then save the changes. Open a new terminal and test that the command works. (see Make Windows for more)

make dev error: [build_lang] Error 2 - Could not load taxonomy: /mnt/podata/taxonomies/traces.result.sto

When running "make dev":

<h1>Software error:</h1>
<pre>Could not load taxonomy: /mnt/podata/taxonomies/traces.result.sto at /opt/product-opener/lib/ProductOpener/Tags.pm line 1976.
Compilation failed in require at /opt/product-opener/scripts/build_lang.pl line 31, &lt;DATA&gt; line 2104.
BEGIN failed--compilation aborted at /opt/product-opener/scripts/build_lang.pl line 31, &lt;DATA&gt; line 2104.
</pre>
<p>
For help, please send mail to this site's webmaster, giving this error message
and the time and date of the error.
</p>
[Tue Apr  5 19:36:40 2022] build_lang.pl: Could not load taxonomy: /mnt/podata/taxonomies/traces.result.sto at /opt/product-opener/lib/ProductOpener/Tags.pm line 1976.
[Tue Apr  5 19:36:40 2022] build_lang.pl: Compilation failed in require at /opt/product-opener/scripts/build_lang.pl line 31, <DATA> line 2104.
[Tue Apr  5 19:36:40 2022] build_lang.pl: BEGIN failed--compilation aborted at /opt/product-opener/scripts/build_lang.pl line 31, <DATA> line 2104.
make: *** [build_lang] Error 2

Solution: Project needs Symlinks to be enabled. traces.result.sto is a symlink to allergens.result.sto

On Windows, You have to enable the 'Developer Mode' in order to use the symlinks. To enable Developer Mode: Go under Settings > Update & Security > 'For developers', and turn on the toggle for Developer Mode. On Windows systems, the git repository needs to be cloned with symlinks enabled.

You need to remove current directory where you clone the project, and clone the project again, using right options:

git clone -c core.symlinks=true [email protected]:openfoodfacts/openfoodfacts-server.git