3 - Latest Sentinel-1 scene.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6zVUMYgV537n"
   },
   "source": [
    "<font face=\"Calibri\" size=\"2\"> <i>Open SAR Toolkit - Tutorial 3, version 1.2, July 2020. Andreas Vollrath, ESA/ESRIN phi-lab</i>\n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kSGwC9r8537o"
   },
   "source": [
    "![title](auxiliary/header_image.PNG)\n",
    "\n",
    "--------\n",
    "\n",
    "# OST Tutorial III\n",
    "## Process the latest Sentinel-1 GRD product for a given point\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ESA-PhiLab/OST_Notebooks/blob/master/3%20-%20Latest%20Sentinel-1%20scene.ipynb)\n",
    "\n",
    "--------\n",
    "\n",
    "**Short description**\n",
    "\n",
    "This notebook demonstrates the interaction between the *Sentinel1* class for data inventory and download, and the *Sentinel1Scene* class, together, for the generation of the latest Sentinel-1 product over a given point coordinate. \n",
    "\n",
    "--------\n",
    "\n",
    "**Requirements**\n",
    "\n",
    "- a PC/Mac with at least 16GB of RAM\n",
    "- about 4GB of free disk space\n",
    "- a Copernicus Open Data Hub user account, ideally valid for at least 7 days (https://scihub.copernicus.eu)\n",
    "--------\n",
    "\n",
    "**NOTE:** all cells that have an * after its number can be executed without changing any code. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gLvPAMdI537p"
   },
   "source": [
    "### 0\\* - Install OST and dependencies \n",
    "\n",
    "**NOTE:** Applies only if you haven't fully installed OST and its dependencies yet, e.g. on Google Colab, so uncomment the lines in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Z21YR74D537q"
   },
   "outputs": [],
   "source": [
    "# !apt-get -y install wget\n",
    "# !wget https://raw.githubusercontent.com/ESA-PhiLab/OST_Notebooks/master/install_ost.sh\n",
    "# !bash install_ost.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DpqUWHp0537t"
   },
   "source": [
    "### 1* - Import python libraries necessary for processing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KRplYtUo537u"
   },
   "outputs": [],
   "source": [
    "# this is for the path handling and especially useful if you are on Windows\n",
    "from pathlib import Path\n",
    "from pprint import pprint\n",
    "\n",
    "# we will need this for our time of period definition\n",
    "from datetime import datetime, timedelta\n",
    "\n",
    "# this is the s1Project class, that basically handles all the workflow from beginning to the end\n",
    "from ost import Sentinel1, Sentinel1Scene"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QP9IBFg7537x"
   },
   "source": [
    "### 2 - Data selection parameters\n",
    "\n",
    "**NOTE:** In case you want to process a different area, all you want to change is the lat and lon values in line 6 \n",
    "\n",
    "\n",
    "As already covered in OST Tutorial 2, we need a minimum of 3 basic parameters to initialise the *Sentinel1* class.\n",
    "\n",
    "**1 Area of Interest:** \n",
    "\n",
    "In this case we only search for a *specific spot on earth, i.e. Rome, Italy*, that is defined by the *Latitude* and *Longitude*. We then create a Well-Known-Text formatted string.\n",
    "\n",
    "**2 Time of Interest:**\n",
    "\n",
    "In this example, the datetime class is used to set the start date to 30 days before today to assure we get any scene within our time of interest.\n",
    "\n",
    "**3 Project directory**\n",
    "\n",
    "Set this to anything you like if not happy with the default one.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cB40bzfX537x"
   },
   "outputs": [],
   "source": [
    "#----------------------------\n",
    "# Area of interest\n",
    "#----------------------------\n",
    "\n",
    "# Here we can either point to a shapefile or as well use \n",
    "lat, lon = 41.8919, 12.5113\n",
    "aoi = 'POINT ({} {})'.format(lon, lat)\n",
    "\n",
    "#----------------------------\n",
    "# Time of interest\n",
    "#----------------------------\n",
    "# we set only the start date to today - 30 days\n",
    "start = (datetime.today() - timedelta(days=30)).strftime(\"%Y-%m-%d\")\n",
    "\n",
    "#----------------------------\n",
    "# Processing directory\n",
    "#----------------------------\n",
    "# get home folder\n",
    "home = Path.home()\n",
    "# create a processing directory within the home folder\n",
    "project_dir = home.joinpath('OST_Tutorials', 'Tutorial_3')\n",
    "\n",
    "#------------------------------\n",
    "# Print out AOI and start date\n",
    "#------------------------------\n",
    "print('AOI: ' + aoi, )\n",
    "print('TOI start: ' + start)\n",
    "print('Our project directory is located at: ' + str(project_dir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JNToZ_1u5370"
   },
   "source": [
    "### 3* - Initialize the Sentinel1 project class\n",
    "\n",
    "After initialisation of our class, where we explicitley add the GRD product type argument, we do a rough search over our AOI for the last 30 days. We print the first 5 entries and plot all images for visualization by using the *Sentinel1* class attribute *inventory* and method *plot_inventory*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yRey1rVI5371"
   },
   "outputs": [],
   "source": [
    "#---------------------------------------------------\n",
    "# for plotting purposes we use this iPython magic\n",
    "%matplotlib inline\n",
    "%pylab inline\n",
    "pylab.rcParams['figure.figsize'] = (19, 19)\n",
    "#---------------------------------------------------\n",
    "\n",
    "# create s1Project class instance\n",
    "s1_project = Sentinel1(\n",
    "    project_dir=project_dir, \n",
    "    aoi=aoi, \n",
    "    start=start,\n",
    "    product_type='GRD'\n",
    ")\n",
    "\n",
    "# search command\n",
    "s1_project.search()\n",
    "# uncomment in case you have issues with the registration procedure \n",
    "#ost_s1.search(base_url='https://scihub.copernicus.eu/dhus')\n",
    "print('We found {} products.'.format(len(s1_project.inventory.identifier.unique())))\n",
    "# combine OST class attribute with pandas head command to print out the first 5 rows of the \n",
    "print(s1_project.inventory.head(5))\n",
    "\n",
    "# we plot the full Inventory on a map\n",
    "s1_project.plot_inventory(transparency=.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2MWfH4ma5374"
   },
   "source": [
    "### 4* - Select the latest scene found during the search\n",
    "\n",
    "Here we use some python-panda syntax on our rough data inventory to filter out the latest scene and create store it in a new dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UiUl2GKY5375"
   },
   "outputs": [],
   "source": [
    "pylab.rcParams['figure.figsize'] = (13, 13)\n",
    "\n",
    "# we give our inventory a shorter name iDf (for inventory Dataframe)\n",
    "iDf = s1_project.inventory.copy()\n",
    "\n",
    "# we select the latest scene based on the metadata entry endposition\n",
    "latest_df = iDf[iDf.endposition == iDf.endposition.max()]\n",
    "\n",
    "# we print out a little info on the date of the  \n",
    "print(' INFO: Latest scene found for {}'.format(latest_df['acquisitiondate'].values[0]))\n",
    "\n",
    "# we use the plotInventory method in combination with the newly\n",
    "# created Geodataframe to see our scene footprint\n",
    "s1_project.plot_inventory(latest_df, transparency=.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "z0J2MNlq5378"
   },
   "source": [
    "### 7* Download scene\n",
    "\n",
    "We use the build-in download method from the *Sentinel1* class. Note that you can pass any Geodataframe generated by OST, and filtered by you (e.g. sort out rows that you do not need). In our case we are only interested in the latest scene, so we pass the newly generated *latest_df* Geodataframe object.\n",
    "\n",
    "**NOTE** that you should use ESA's scihub server in this case, since it is the place where the images arrive first. Other data mirrors might have slight delays, so that the scene found by the inventory might not be available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EVCw8pHg5378"
   },
   "outputs": [],
   "source": [
    "s1_project.download(latest_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1TI2VeaD538A"
   },
   "source": [
    "### 8* - Display some metadata of the latest scene\n",
    "\n",
    "After use of the *Sentinel1* class for finding and downloading the latest scene, we hand the scene identifier over to the *Sentinel1Scene* class for further processing as already demonstrated in OST Tutorial 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kqqZdzKC538A"
   },
   "outputs": [],
   "source": [
    "# create a S1Scene class instance based on the scene identifier coming from our \"latest scene dataframe\"\n",
    "latest_scene = Sentinel1Scene(latest_df['identifier'].values[0])\n",
    "\n",
    "# print summarising infos\n",
    "latest_scene.info()\n",
    "\n",
    "# print file location\n",
    "file_location = latest_scene.get_path(s1_project.download_dir)\n",
    "\n",
    "print(' File is located at: ')\n",
    "print(' ' + str(file_location))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MkKvHRSl538D"
   },
   "source": [
    "### 9* - Produce a subsetted ARD product\n",
    "\n",
    "The creation of the ARD product follows the same logic as presented in OST Tutorial 1. However, for this case we introduce the subset argument to eh *create_ard* function.\n",
    "Subsetting is adviced if only a small portion of the whole image is of interest. It will speed up processing and uses less storage.\n",
    "\n",
    "In our case we use some helper functions within the OST package to create a squared buffer area fo 10000 meter around our point of interest defined as AOI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "W3Wyb_ny538E"
   },
   "outputs": [],
   "source": [
    "# 10 km buffer around AOI Point\n",
    "from shapely.wkt import loads\n",
    "from ost.helpers import vector as vec\n",
    "\n",
    "# turn WKT into shapely geometry object\n",
    "shp_aoi = loads(s1_project.aoi)\n",
    "\n",
    "# use OST helper's function to create a quadrant buffer for subset\n",
    "subset_area = vec.geodesic_point_buffer(shp_aoi.y, shp_aoi.x, 10000, envelope=True)\n",
    "\n",
    "print('-----------------------------------------------------------------------------')\n",
    "latest_scene.create_ard(\n",
    "    # we see our download path can be automatically generated by providing the Project's download directory\n",
    "    infile=latest_scene.get_path(download_dir=s1_project.download_dir), \n",
    "    # let's simply take our processing folder\n",
    "    out_dir=s1_project.processing_dir,   \n",
    "    # define the subset\n",
    "    subset=subset_area,\n",
    "    # in case already processed, we will re-process\n",
    "    overwrite=True\n",
    ")\n",
    "\n",
    "print('-----------------------------------------------------------------------------')\n",
    "print(' The path to our newly created ARD product can be obtained the following way:')\n",
    "latest_scene.ard_dimap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "m4N-5dBu538G"
   },
   "source": [
    "### 10* - Create a RGB color composite\n",
    "\n",
    "As already demonstrated in OST Tutorial 1, we create an RGB GeoTiff, and visualize it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "icq5Nf34538H"
   },
   "outputs": [],
   "source": [
    "latest_scene.create_rgb(outfile = s1_project.processing_dir.joinpath(f'{latest_scene.start_date}.tif'))\n",
    "latest_scene.visualise_rgb(shrink_factor=1)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "3 - Latest Sentinel-1 scene.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}