Deploying Branches to Heroku [WIP]

On Heroku

Create a new app

• click new
• select create new app
• add your name of the app (otherwise, they will assign a random name)
• click create app
• upon returning to the main screen, select overview from the menu
• click on Configure add-ons
• on the next screen, click the search bar (it says quickly add add-ons from Elements)
• add Heroku Postgres and Timber.io Logging, both of wdhich should use the free tier plan.
• Back on the Dashboard screen, select Settings
  • Under Config Vars, click on Reveal Config Vars.
    • Add PYTHONPATH as another key and the value should be ./cfgov
  • Under Buildpacks, click Add Buildpacks and add heroku/nodejs and heroku/python in that specific order.
• Back on the Dashboard screen, select Deploy
  • Under Deploy using Heroku Git:
    • follow the instructions Install Heroku CLI only if you don't already have it installed on your computer

On Local

Placeholders and definitions

For the purpose of the examples in this guide, we use the following placeholders that represent values you should customize when running commands. They are not to be used literally.

Placeholder Name	Description
FEATURE_BRANCH_NAME	The name of the feature branch where you make your code changes
STAGING_BRANCH_NAME	The name of the staging branch that you deploy to Heroku. By convention it should be stage/$FEATURE_BRANCH (i.e. "stage/" and then the name of your feature)
HEROKU_APP_NAME	The name of the Heroku app you create
HEROKU_REMOTE_NAME	This is the name you give to the staging app in Git. You push to this to deploy. It can be anything, but it should be short and describe the feature you're deploying there.

Create a new staging branch

Locally, create new branch off of staging, which has some Heroku-specific adaptations that all staging branches need in order to deploy and run properly on Heroku:

git checkout staging
git checkout -b STAGING_BRANCH_NAME

This will checkout a new branch based on staging. Next, merge your feature branch into the new staging branch you created:

git merge FEATURE_BRANCH_NAME

Initial Deploy

In your local repo, use the heroku command line interface to add the app your created earlier as a remote:

heroku git:remote -a HEROKU_APP_NAME --remote HEROKU_REMOTE_NAME

You may have to do git remote add heroku [NAME OF APP]

Edit your .yarnrc file and delete the line that says: --install.pure-lockfile true if it's present. You'll need to re-add it later before you deploy.

Run yarn.

Add the new yarn.lock file and commit.

Edit .yarnrc again and re-add the --install.pure-lockfile true to the end of the file, then save. Also remove the line --install.offline true

Push your branch to Github. This isn't strictly necessary, but it will allow you to pull it down to other machines if you ever need to deploy from a different computer:

git push -u origin STAGING_BRANCH_NAME

Push to Heroku to deploy:

git push HEROKU_REMOTE_NAME STAGING_BRANCH_NAME:master

The :master part is important, as Heroku only builds and deploys changes pushed to the app's master branch. We're taking the staging branch we created and pushing it to Heroku as if it were master.

After First Deploy

After first deploy, you need to setup the initial data in the DB:

Locally, open an interactive shell into your deployed application, then run the scripts to migrate and seed the DB:

heroku run bash -r HEROKU_REMOTE_NAME

# Once connected, run this command:
DJANGO_ADMIN_USERNAME=admin DJANGO_ADMIN_PASSWORD=admin ./initial-data.sh

# Get back to your local shell and disconnect from Heroku:
exit

This will create the database, setup the tables, and seed the necessary content. It'll also create an admin user in Wagtail with the login info "admin/admin".

Once you're done setting up the DB, switch back to your feature branch so that you don't accidentally commit any changes direct to the staging branch.

git checkout FEATURE_BRANCH_NAME

Subsequent deploys

Make changes on FEATURE_BRANCH_NAME (not the staging branch) as usual, commit, etc. Dev work should always be done on a non-staging branch because we don't want to merge any Heroku-specific adaptations into CFPB's main codebase.

Checkout your staging branch:

git checkout STAGING_BRANCH_NAME

Merge your feature branch in:

git merge FEATURE_BRANCH_NAME

Push to Github and heroku:

git push origin STAGING_BRANCH_NAME
git push HEROKU_REMOTE_NAME STAGING_BRANCH_NAME:master

Heroku Specific Adaptations on Staging Branches

All stage branches share a few changes that needed to be made in order to make cfgov-refresh compatible with Heroku's runtime environment. These changes are unnecessary and could be harmful if merged into master or any mainline dev branch that will eventually be deployed to a CFPB dev server, so it's best that they remain on Heroku-only staging branches.

Python Runtime Version

Heroku looks for a file called runtime.txt in the project root when deploying Python apps. The contents of this file determine which Python version is used to run the app.

runtime.txt:

python-3.6.10

Python Dependencies

CFPB's runtime environment uses a regular web server like Nginx or Apache to serve static files, reverse proxying all requests to the Python app server. Heroku works differently - it serves requests directly from the Python server process. Django doesn't natively serve static assets like images, stylesheets, and JS files because it's inefficient, but we've added a module called WhiteNoise that gives Django this capability.

WhiteNoise has been added to the Pip dependencies on staging branches:

requirements/django.txt:

gunicorn
whitenoise==3.3.1
django==2.2.13

It's then configured as a Django middleware, which allows it to intercept and serve static file requests before they hit the app's routing code.

cfgov/cfgov/settings/base.py:

# Line 115:
MIDDLEWARE = (
    "whitenoise.middleware.WhiteNoiseMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.locale.LocaleMiddleware",
    "django.middleware.http.ConditionalGetMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "core.middleware.ParseLinksMiddleware",
    "core.middleware.DownstreamCacheControlMiddleware",
    "flags.middleware.FlagConditionsMiddleware",
    "wagtail.core.middleware.SiteMiddleware",
    "wagtail.contrib.redirects.middleware.RedirectMiddleware",
)

A small adaptation is also necessary in one of CFPB's custom middleware modules in order to prevent server errors when serving static assets. Added a check to make sure content-type is a valid key of response before trying to dereference it.

cfgov/core/middleware.py:

# Line 55
def __call__(self, request):
    response = self.get_response(request)
    if 'content-type' in response and self.should_parse_links(request.path, response['content-type']):
        response.content = parse_links(
            response.content,
            request.path,
            encoding=response.charset
        )
    return response

package.json

Heroku expects to find a build property in the scripts section of package.json. It runs this during deploy when using the Node buildpack. This is how we tell Heroku to build the app's frontend assets.

Excerpt of package.json:

{
  "name": "cfgov-refresh",
  "scripts": {
    "build": "gulp"
  }
}

Procfile

The Procfile tells Heroku how to run the application's server process as well as any other ancillary processes needed to support it.

Procfile contents:

web: gunicorn cfgov.wsgi --log-file -

Staging Maintence

The staging branch is configured to be deployed to Heroku staging environment and needs to be updated with changes from upstream master from time to time.

git checkout staging
git pull
git merge origin my-money-calendar
# fix conflicts as they arise
git push