mma-dexter

Dexter web app for MMA.

The app is a Flask web app that uses SQLAlchemy to talk to a MySQL database.

Please read the wiki for a full overview of how Dexter works.

Development

• clone the repo
• install a virtual env and activate it: virtualenv --no-site-packages env; source env/bin/activate
• install requirements: pip install -r requirements.txt
• setup the MySQL database (minimum version 5.6.21)

mysql -u root
mysql> CREATE DATABASE mma;
mysql> GRANT ALL ON mma.* TO 'mma'@'localhost';
mysql> exit;

from dexter.models import db
from dexter.models.seeds import seed_db
db.create_all()
seed_db(db)

• setup the AlchemyAPI and OpenCalais API keys:

export ALCHEMY_API_KEY=thekey
export CALAIS_API_KEY=anotherkey

• run the server: python app.py
• log in as [email protected] with password admin
• to add a new article to the database, visit [http://localhost:5000/articles/new] and enter a Mail and Guardian URL.

Topic Clustering

Dexter needs NumPy, SciPy and scikit-learn to run clustering.

On Mac OS X, we recommend this:

brew install gcc
pip install scipy

Running Tests

You will need to setup a test database:

mysql -u root
mysql> CREATE DATABASE mma_test;
mysql> GRANT ALL ON mma.* TO 'mma'@'localhost';
mysql> exit;

Then use nose to run tests:

nosetests

Production

Dexter runs using Dokku, a Docker-based container infrastructure very similar to Heroku

To deploy your changes, simply git push dokku to push to your dokku remote.

To setup a new dokku container:

• Create the app: dokku app create mma-dexter

• Configure the app:

  dokku config:set mma-dexter
  SQLALCHEMY_DATABASE_URI="mysql://user:pass@host/database?charset=utf8&use_unicode=0"
  FLASK_ENV=production NEW_RELIC_CONFIG_FILE='/app/dexter/config/newrelic.ini' NEWSTOOLS_FEED_PASSWORD=newstools-password
  SENDGRID_API_KEY=sendgrid-api-key
  ALCHEMY_API_KEY=api-key
  AWS_ACCESS_KEY_ID=aws-access-key
  AWS_SECRET_ACCESS_KEY=aws-secret-access-key
  CALAIS_API_KEY2=calais-api-key-1
  CALAIS_API_KEY=calais-api-key-2

• Deploy your code: git push dokku

Database

• Add the DB user:

CREATE DATABASE mma;
GRANT ALL ON mma.* TO 'mma'@'localhost' identified by 'PASSWORD';

• restore the database from a backup, if available.

Deploying database changes

Dexter uses Flask-Migrate (which uses Alembic) to handle database migrations.

To add a new model or make changes, update the SQLAlchemy definitions in dexter/models/. Then run

python app.py db migrate --message "a description of your change"

This will autogenerate a change. Double check that it make sense. To apply it on your machine, run

python app.py db upgrade head

Database

The server expects MySQL 5.6 because it uses the CURRENT_TIMESTAMP default value on a DATETIME column, as described here. This means on some systems you'll need to upgrade from 5.5 to 5.6.