09_Modules_pygrass_advance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# pygrass Modules, advance usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We already pygrass `Modules` module in the unit 02. Now we will see other modules to run `Modules` in different ways. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "# pretty-print\n",
    "from pprint import pprint\n",
    "from subprocess import PIPE\n",
    "from multiprocessing import Process"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MultiModules, run several modules in serial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`MultiModules` is designed to run a list of modules in serial in the provided order within a temporary region environment. `Modules` can be run in serial synchronously or asynchronously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from grass.pygrass.modules import Module\n",
    "from grass.pygrass.modules import MultiModule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create to Modules objects and try to run them with `MultiModules`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we need to remember to use run_=False\n",
    "listvect = Module('g.list', type='vector', mapset='.', stdout_=PIPE, run_=False)\n",
    "listraster = Module('g.list', type='raster', mapset='.', stdout_=PIPE, run_=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can i\n",
    "mm = MultiModule(module_list=[listvect, listraster])\n",
    "mm.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point you can see the output of the modules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(mm.module_list[0].outputs.stdout)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(mm.module_list[1].outputs.stdout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to get the help for `MultiModules`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mm?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to test the asynchronous mode.\n",
    "You can test it running several map algebra with `r.mapcalc` module. First set the right Region and get info about the raster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(Module('g.region', raster='elevation@PERMANENT', flags='p', stdout_=PIPE).outputs.stdout)\n",
    "print(Module('r.info', map='elevation@PERMANENT', flags='r', stdout_=PIPE).outputs.stdout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it is possible to create a list of `r.mapcalc` `Modules` objects. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "listmapcalc = []\n",
    "val = 0\n",
    "while val < 200:\n",
    "    listmapcalc.append(Module(\"r.mapcalc\",\n",
    "                       expression=\"elevation_{va}=if(elevation@PERMANENT>{va} && elevation@PERMANENT<={va2}, elevation@PERMANENT, null())\".format(va=val, va2=val+50),\n",
    "                       overwrite=True, run_=False, finish_=False)\n",
    "    )\n",
    "    val += 50\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "listmapcalc[-1].get_bash()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point create the `MultiModules` object in asynchronous mode and run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mm = MultiModule(module_list=listmapcalc, sync=False)\n",
    "pro = mm.run()\n",
    "isinstance(pro, Process)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mapcalc_list = mm.wait()\n",
    "for mapcalc in mapcalc_list:\n",
    "    print(mapcalc.popen.returncode)\n",
    "print(Module('g.list', type='raster', pattern='elevation*', stdout_=PIPE).outputs.stdout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GridModules, run a single process in multiprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`GridModules` provide a simple Module to run a GRASS process and split analysis between several processes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from grass.pygrass.modules.grid.grid import GridModule"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create GridModule object and get the different works\n",
    "grd = GridModule('r.slope.aspect',\n",
    "                 width=1000, height=1000, overlap=2,\n",
    "                 processes=None, split=False,\n",
    "                 elevation='elevation@PERMANENT',\n",
    "                 slope='slope_grid', aspect='aspect_grid', overwrite=True)\n",
    "grd.get_works()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point run it and check if the maps are created..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grd.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(Module('g.list', type='raster', mapset='.', stdout_=PIPE, pattern=\"*_grid\").outputs.stdout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have seen\n",
    "\n",
    "* how to run several command in serial mode\n",
    "* how to run a command splitting its calculation in several process. \n",
    "  **Pay attention because for several analysis this is no possible**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Computes the viewshed for ten point on the `elevation` raster map inside the computational region"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Exercise 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run `r.geomorphon` as single process with `Module` and using `GridModule` and check if the results are the same"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}