Tutorial #1 - Simple Network, Single Trip.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fast-Trips Tutorial 1: Run Fast-Trips for a Single Trip on Simple Network\n",
    "## Install required software and run this notebook from a virtual environment\n",
    "\n",
    "The easiest way to get this notebook to work locally is to [download Anaconda, for Python 2.7 ](https://www.continuum.io/downloads)\n",
    "\n",
    "You can install all the dependencies by importing the virtual environment `ft_tutorial.yml` that is included this git repository.\n",
    "\n",
    "To run each cell, highlight the cell and use shift-enter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os,datetime\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Specify Input Networks + Visualize\n",
    "Where are the networks?  Please update `BASE_DIR` to reflect where you installed the tutorial on your machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "BASE_DIR         = r\"PUT YOUR PATH TO THE TUTORIAL DIRECTORY HERE!\"\n",
    "\n",
    "INPUT_NETWORKS   = os.path.join(BASE_DIR, \"tta\",\"input\",\"network-simple\")\n",
    "INPUT_DEMAND     = os.path.join(BASE_DIR, \"tta\",\"input\",\"demand-single\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validate GTFS Feed\n",
    "The input networks are stored in the [GTFS-Plus](https://github.com/osplanning-data-standards/GTFS-PLUS) format.  \n",
    "\n",
    "We use [`transitfeed`](https://github.com/google/transitfeed) for parsing and validating the GTFS component of GTFS plus.  \n",
    "\n",
    "Note that `transitfeed` does not expect all the extra files associated with GTFS-PLUS and it will be noisy about them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import transitfeed\n",
    "loader        = transitfeed.Loader(INPUT_NETWORKS, memory_db=True)\n",
    "schedule      = loader.Load()\n",
    "schedule.Validate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Simple Map of Routes\n",
    "\n",
    "We use [`folium`](https://github.com/python-visualization/folium) to create a simple vizualization of the input network.  \n",
    "\n",
    "The very simple code to create this map is stored in the file `tutorial_map.py` which is included in this repository.  You can make it fancier if you want to see more things!\n",
    "\n",
    "Note that this would be very slow if you wanted to look at a large city's transit network, but it works for our simple example.  \n",
    "\n",
    "Folium is a python wrapper around the popular [`Leaflet.js`](http://leafletjs.com/) mapping package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import folium\n",
    "import tutorial_map\n",
    "\n",
    "mymap = tutorial_map.make_map(schedule)\n",
    "mymap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run Fast-Trips Example: Single Trip\n",
    "Here we run a simple example of a single rider going from approximately Reynolds Coliseium to the Pullen Park train."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from fasttrips import Run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parameters and Run Configurations\n",
    "Let's look at the parameter files and run configurations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "INPUT_WEIGHTS    = os.path.join(BASE_DIR,\"tta\",\"input\",\"demand-single\",\"pathweight_ft.txt\")\n",
    "RUN_CONFIG       = os.path.join(BASE_DIR,\"tta\",\"input\",\"demand-single\",\"config_ft.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Parameters File:**  `pathweights_ft.txt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pathweights_df = pd.read_csv(INPUT_WEIGHTS)\n",
    "pathweights_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Run Config File:**  `config_ft.txt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with open(RUN_CONFIG, 'r') as myfile:\n",
    "    contents=myfile.read()\n",
    "print contents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fast-Trips can be run with the Run.run_fasttrips command which is a wrapper function that runs:\n",
    "  * `ft = Run.run_setup()`\n",
    "  * `ft.run_setup()`\n",
    "  * `ft.runassignment()`\n",
    "\n",
    "There are six required inputs:\n",
    "\n",
    "  * `input_network_dir`\n",
    "  * `input_denand_dir`\n",
    "  * `run_config`\n",
    "  * `input_weights`\n",
    "  * `output_dir`\n",
    "  * `pathfinding_type`\n",
    "  * `iters`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print Run.run_fasttrips.__doc__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print Run.run_setup.__doc__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "OUTPUT_DIR    = os.path.join(BASE_DIR,\"tta\",\"output\")\n",
    "OUTPUT_FOLDER = \"test_simplenet_noOverlap\"\n",
    "Run.run_fasttrips(input_network_dir= INPUT_NETWORKS,\n",
    "                  input_demand_dir = INPUT_DEMAND,\n",
    "                  run_config       = RUN_CONFIG,\n",
    "                  input_weights    = INPUT_WEIGHTS,\n",
    "                  output_dir       = OUTPUT_DIR,\n",
    "                  iters            = 1,\n",
    "                  output_folder    = OUTPUT_FOLDER,\n",
    "                  dispersion       = 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examine Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vehicle Trip Level\n",
    "The `veh_trips.csv` file is a record of every stop on every transit trip in the network.  It contains information about arrivals and departures as well as ridership totals.  It is a useful file for examining capacity and operational issues and can be fed back into a dynamic traffic assignment procedure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "full_output_directory=os.path.join(OUTPUT_DIR,OUTPUT_FOLDER)\n",
    "vehicles_df = pd.read_csv(os.path.join(full_output_directory,\"veh_trips.csv\"), \n",
    "                                       sep=\",\", \n",
    "                                       parse_dates=['arrival_time', 'departure_time'],\n",
    "                                       date_parser=lambda x: datetime.datetime.strptime(x, '%H:%M:%S') )\n",
    "vehicles_with_boardings_df=vehicles_df[vehicles_df[\"onboard\"]>0]\n",
    "vehicles_with_boardings_df[['iteration','pathfinding_iteration','arrival_time','departure_time','route_id','trip_id','stop_id','boards','onboard','standees']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Plot Ridership by Trip\n",
    "As you can see from examining the vehicle trips file, there were two pathfinding iterations, which will be discussed in a later tutorial.  Let's look at the results from the following one, `pathfinding_iteration==1`. \n",
    "\n",
    "The resulting plot is not very interesting, since we just sent one trip through, but you can see that they took Trip B2 followed by G3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "sns.set(style=\"whitegrid\")\n",
    "fig, axs = plt.subplots(nrows=1,figsize=(8,4))\n",
    "sns.barplot(ax=axs, x=\"trip_id\", \n",
    "            order=[\"B1\",\"B2\",\"B3\",\"G1\",\"R1\",\"G2\",\"R2\",\"G3\",\"G4\"],\n",
    "            y=\"boards\", hue=\"route_id\",  \n",
    "            estimator=sum, \n",
    "            ci=None, \n",
    "            data=vehicles_with_boardings_df[vehicles_with_boardings_df[\"pathfinding_iteration\"]==1])\n",
    "axs.set_title('Ridership by Trip', fontsize=24,color=\"Gray\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Person Level\n",
    "\n",
    "The person-level results use the [`dyno-path`](https://github.com/osplanning-data-standards/dyno-path) format.  \n",
    "\n",
    "There are two levels of files: *path* and *link*.  \n",
    "\n",
    "Additionally, we store both the *chosen* path and links as well as all the paths and links that were available choices within Fast-Trips. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Links on Chosen Path\n",
    "Links represent a segment of the path using a single transit `trip_id` or access, egress, or transfer linkmode.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We use [`Bokeh`](http://bokeh.pydata.org/) to examine the results.  Bokeh is a python wrapper on the popular [`D3.js`](https://d3js.org/).  \n",
    "\n",
    "**Tip**: Sometimes Bokeh is ornery and doesn't play nice.  Usually re-importing it using command below will do the trick."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from bokeh.charts import Bar, Histogram, TimeSeries, output_file, Line\n",
    "from bokeh.models import ColumnDataSource\n",
    "from bokeh.io import push_notebook, show, output_notebook\n",
    "from bokeh.plotting import figure\n",
    "output_notebook()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "CHOSEN_LINKS = \"chosenpaths_links.csv\"\n",
    "chosen_link_df = pd.read_csv(os.path.join(full_output_directory,CHOSEN_LINKS), \n",
    "                                       sep=\",\", \n",
    "                                       parse_dates=['new_A_time', 'new_B_time'],\n",
    "                                       date_parser=lambda x: datetime.datetime.strptime(x, '%H:%M:%S') )\n",
    "\n",
    "chosen_link_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "from bokeh.models import ColumnDataSource, DataRange1d, Plot, LinearAxis, Grid,DatetimeAxis, LabelSet\n",
    "from bokeh.models.glyphs import Segment\n",
    "from bokeh.io import curdoc, show\n",
    "\n",
    "mode2color = {'local_bus':\"#b3de69\",\n",
    "             'walk_access':'#8dd3c7',\n",
    "             'walk_egress':'#bebada',\n",
    "             'transfer':'#fb8072',\n",
    "             'other':'#80b1d3'}\n",
    "\n",
    "def assignColorByMode(mode):\n",
    "    if mode in mode2color.keys():\n",
    "        return mode2color[mode]\n",
    "    return mode2color[\"other\"]\n",
    "    \n",
    "chosen_link_df[\"color\"]= map(assignColorByMode, chosen_link_df[\"mode\"])\n",
    "\n",
    "def createAnnotation(mode,sim_cost,route_id,trip_id):\n",
    "    return str(mode)+\" \"+str(sim_cost)+\" \"+str(route_id)+\":\"+str(trip_id)\n",
    "\n",
    "chosen_link_df[\"annotation\"]= map(createAnnotation, chosen_link_df[\"mode\"],chosen_link_df[\"sim_cost\"],chosen_link_df[\"route_id\"],chosen_link_df[\"trip_id\"])\n",
    "\n",
    "source = ColumnDataSource(chosen_link_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "xdr = DataRange1d()\n",
    "ydr = DataRange1d()\n",
    "\n",
    "plot = Plot(\n",
    "    title=None, x_range=xdr, y_range=ydr, plot_width=800, plot_height=300,\n",
    "    h_symmetry=False, v_symmetry=False, min_border=0, toolbar_location=None)\n",
    "\n",
    "glyph = Segment(y0=\"linknum\", x0=\"new_A_time\", y1=\"linknum\", x1=\"new_B_time\", line_color=\"color\", line_width=\"sim_cost\")\n",
    "plot.add_glyph(source, glyph)\n",
    "\n",
    "xaxis = DatetimeAxis()\n",
    "plot.add_layout(xaxis, 'below')\n",
    "\n",
    "yaxis = LinearAxis()\n",
    "plot.add_layout(yaxis, 'left')\n",
    "\n",
    "plot.add_layout(Grid(dimension=0, ticker=xaxis.ticker))\n",
    "plot.add_layout(Grid(dimension=1, ticker=yaxis.ticker))\n",
    "\n",
    "mode_labels = LabelSet(x=\"new_A_time\", y=\"linknum\", text=\"mode\", y_offset=-5,x_offset=10,\n",
    "                  text_font_size=\"8pt\", text_color=\"#555555\",\n",
    "                  source=source, text_align='left')\n",
    "plot.add_layout(mode_labels)\n",
    "\n",
    "curdoc().add_root(plot)\n",
    "\n",
    "show(plot, notebook_handle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot above shows the chosen path over time with the thicknesses representing the perceived cost of each segment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Path Choice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Path Set\n",
    "**Path** files contain information about the entire path as scored by the individual who is taking a particular trip.\n",
    "\n",
    "The fields `sim_cost` and `probability` summarize how the path was scored.    \n",
    "\n",
    "A quick description of the path is available in the `description` field.\n",
    "\n",
    "Fast-Trips outputs two path files: `chosenpaths_paths.csv` and `pathset_paths.csv`.  Since our simple example only has a few paths, let's see how each of them were evaluated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "PATHS  = r\"pathset_paths.csv\"\n",
    "\n",
    "pathset_paths_df = pd.read_csv(os.path.join(full_output_directory,PATHS), sep=\",\")\n",
    "pd.set_option('display.max_colwidth',160) #widen so you can see whole description\n",
    "pathset_paths_df[[\"person_id\",\"person_trip_id\",\"pathnum\",\"description\",\"sim_cost\",\"logsum\",\"probability\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Choice Set Link Files\n",
    "We can look at the components of each of these paths to find out what is driving the total cost by using the **pathset-links** file: `pathset_links.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "LINKS  = r\"pathset_links.csv\"\n",
    "linkfile_df = pd.read_csv(os.path.join(full_output_directory,LINKS),\n",
    "                                       sep=\",\", \n",
    "                                       parse_dates=['new_A_time', 'new_B_time'],\n",
    "                                       date_parser=lambda x: datetime.datetime.strptime(x, '%H:%M:%S') )\n",
    "\n",
    "## Add Fast-Trips path probability information to Fast-Trips link file\n",
    "linkfile_df = pd.merge(left  = linkfile_df,\n",
    "                       right = pathset_paths_df,\n",
    "                       how   = \"left\",\n",
    "                       left_on = [\"person_id\",\"person_trip_id\",\"pathnum\",\"chosen\",\"missed_xfer\"],\n",
    "                       right_on = [\"person_id\",\"person_trip_id\",\"pathnum\",\"chosen\",\"missed_xfer\"])\n",
    "\n",
    "#linkfile_df.head()\n",
    "#linkfile_df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\n",
    "maxlinknum = linkfile_df[\"linknum\"].max()\n",
    "    \n",
    "linkfile_df[\"color\"]= map(assignColorByMode, linkfile_df[\"mode\"])\n",
    "\n",
    "def createAnnotation(mode,sim_cost,route_id,trip_id,probability):\n",
    "    if mode not in ['local_bus']:\n",
    "        return str(mode)+\" \"+str(sim_cost)\n",
    "    else:\n",
    "        return str(mode)+\" \"+str(sim_cost)+\" \"+str(route_id)+\":\"+str(trip_id)\n",
    "\n",
    "linkfile_df[\"annotation\"]= map(createAnnotation, linkfile_df[\"mode\"],linkfile_df[\"sim_cost_x\"],linkfile_df[\"route_id\"],linkfile_df[\"trip_id\"],linkfile_df[\"probability\"])\n",
    "\n",
    "def yloc(pathnum,linknum):\n",
    "    return (pathnum*(maxlinknum+1))+linknum\n",
    "\n",
    "linkfile_df[\"yloc\"]= map(yloc, linkfile_df[\"pathnum\"],linkfile_df[\"linknum\"])\n",
    "\n",
    "linksource = ColumnDataSource(linkfile_df)\n",
    "#linkfile_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "xdr2 = DataRange1d()\n",
    "ydr2 = DataRange1d()\n",
    "\n",
    "choiceplot = Plot(\n",
    "    title=None, x_range=xdr2, y_range=ydr2, plot_width=800, plot_height=300,\n",
    "    h_symmetry=False, v_symmetry=False, min_border=0, toolbar_location=None)\n",
    "\n",
    "glyph = Segment(y0=\"yloc\", x0=\"new_A_time\", y1=\"yloc\", x1=\"new_B_time\", line_color=\"color\", line_width=\"sim_cost_x\")\n",
    "choiceplot.add_glyph(linksource, glyph)\n",
    "\n",
    "choicexaxis = DatetimeAxis()\n",
    "choiceplot.add_layout(choicexaxis, 'below')\n",
    "\n",
    "choiceplot.add_layout(Grid(dimension=0, ticker=xaxis.ticker))\n",
    "choiceplot.add_layout(Grid(dimension=1, ticker=yaxis.ticker))\n",
    "\n",
    "choice_labels = LabelSet(x=\"new_A_time\", y=\"yloc\", text=\"annotation\", y_offset=-5,x_offset=10,\n",
    "                  text_font_size=\"8pt\", text_color=\"#555555\",\n",
    "                  source=linksource, text_align='left')\n",
    "choiceplot.add_layout(choice_labels)\n",
    "\n",
    "curdoc().add_root(choiceplot)\n",
    "\n",
    "##TODO add nodes\n",
    "##TODO standardize colors by mode\n",
    "##TODO add probability\n",
    "##TODO make each path start at a grid line so you can evaluate cost easier\n",
    "\n",
    "show(choiceplot, notebook_handle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## QUESTION\n",
    "**Question 1a:** What is the best route according to Tuffie?\n",
    "\n",
    "**Question 1b:** What drives it being the best?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}