Skip to content

Development process

Jump to bottom
Yoom Lam edited this page Nov 17, 2022 · 21 revisions

Check out LHDI's "Building and Running" Development Guide and "Running the application" in README.md in addition to this page.

M1 Macs

For M1 Macs, set export DOCKER_DEFAULT_PLATFORM=linux/amd64 to avoid problems downloading Docker images (e.g., ERROR: no match for platform in manifest or no matching manifest for linux/arm64/v8 in the manifest list entries). Details: https://pythonspeed.com/articles/docker-build-problems-mac/

Secrets

Check out the abd-vro-dev-secrets repo and follow the instructions to set local development credentials and enable querying Lighthouse's development environment.

Pre-commit hook

Enable VRO's pre-commit hook by doing the following:

  1. Install pre-commit (Mac: brew install pre-commit)
  2. Run: pre-commit install. This creates a .git/hooks/pre-commit file.
  3. You'll likely have to install go: brew install go

With this enabled, a series of checks and fixes (defined in .pre-commit-config.yaml) will be applied to changed files when you make a git commit.

(Most of the pre-commit alerts have been addressed in PR #666.)

Modifying existing code

Before modifying application code, it's advantageous to determine in which Docker container (and hence Code structure) the target code in being run.

  • The abd_vro-app container results from the following Gradle subprojects/folders:
    • app
    • api
    • controller
    • persistence
    • service
  • while the abd_vro-db-init container results from the db-init folder;
  • and the abd_vro-service-ruby container results from the service-ruby folder.

Also examine the src/docker/docker-compose.yml files in the various subprojects, i.e., app, db-init, and service-ruby, to identify the container names and dependencies.

With that understanding, you can start to modify code.

  1. First start all the containers needed to run VRO: ./gradlew :app:dockerComposeUp
    • To reset and rebuild container images: ./gradlew clean build check docker
  2. Next determine what code needs to be modified and in which container it is run.
  3. Then follow one of the subsections below:

A. Modify Microservice

If modifying code in the service folder, run using Sprint Boot. Alternatively, the slower option is to rebuild container image.

If modifying code in the abd_vro-service-ruby container, stop the container (docker stop docker-service-ruby-1), and run cd service-ruby/src; ./entrypoint.sh (which calls ruby) to quickly run the modified code. Alternatively, the slower option is to rebuild container image but is consistent with running other containers.

B. Modify Camel Route

These currently reside in *Route(s) classes in the gov.va.vro.service.provider.camel package of the service/provider folder. Changes to this code will result in the abd_vro-app container, run using Sprint Boot or rebuild container image.

C. Modify API, Controller, or Model

Currently, the API, controller, and model code result in the abd_vro-app container, so run using Sprint Boot or rebuild container image.

Run using Spring Boot

Close the abd_vro-app container (docker stop docker-abd_vro-1) and run ./gradlew :app:bootRun to quickly run the modified code using Spring Boot.

Rebuild container image

The alternative (and slower option) for running the modified code is to run it in an updated container: ./gradlew :app:dockerComposeUp, which requires rebuilding the container image and deploying the container locally. This command will automatically stop the running container and start an updated container with the same name.

Tip: Portainer UI

For a visual Docker Container management UI, try Portainer -- example UI. Run docker run -d --name portainer -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v /opt/portainer:/data portainer/portainer-ce and go to http://localhost:9000 in a browser.

New API and microservice

When developing a new API endpoint with a new microservice, some folks like to start with the backend service; others prefer to start from the API; and still others like to start with a dead-simple end-to-end implementation and iterate on it until it satisfies the requirements.

The following sections describe the development steps used to add a new API endpoint and associated microservice. Each step can be done independently and in parallel, with some coordination to ensure consistent interfaces between Controller classes and Camel routes, and between route endpoints and microservices. For details, check out Routing API requests.

Build the microservice

To minimize software coupling and maximize single-responsibility, a microservice should be built independent of existing VRO code. It should be idempotent and testable without having to run any other VRO code. It can be placed in its own Docker container or into an existing container. See the service-ruby folder and PR #71 for an example of Ruby microservices that can be run and tested on its own, without Camel or a REST API -- standalone testing Ruby scripts are in the examples folder. The only requirement is interfacing with a message queue, specifically RabbitMQ.

Test Camel routing to microservice

New backend Camel routing and microservices can be tested without having to implement API endpoint, controller, data model, request/response, and mapper classes. CamelRestConfiguration uses Camel to create a REST API for quick development and testing in a local development environment -- it is not for production. CamelRestConfiguration is only enabled when vro.camel_rest_api.enable = true (set in conf-camel.yml). This is achieved by using Camel's REST endpoint to provide quick API for testing via curl or Postman. The sample CamelApp uses this Camel feature to provide the API, whereas VRO uses Spring's Web MVC described in the next section.

Alternatively, Camel messages can be directly injected into Camel routes, which could be useful in unit tests. For examples, inspect the CamelEntrance class.

Build out the API and Controller

Implement API endpoint, controller, data model, request/response, and mapper classes.

  • Get agreement on the API endpoint specification, including examples and error codes. Check http://localhost:8080/swagger in a browser.
  • Implement controller methods using Request and Response classes, and a Mapper class to convert to service.spi.*.model classes.
  • Have the controller initiate a Camel route that eventually leads to a microservice, for example by calling a method in the CamelEntrance class.
Clone this wiki locally