data/uk_gb_awi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ancient Woodland Inventory Overview\n",
    "\n",
    "This workflow deals with analysing Ancient Woodland Inventory data, made available and published by countries of the United Kingdom, where data is available.\n",
    "\n",
    "The main purpose of data is to try and understand which of the many areas declared as Woodland within the UK can actually be considered native, protected or otherwise \"real\" trees that aren't part of forestry management or future timber.\n",
    "\n",
    "### Methodology\n",
    "\n",
    "One if the driving factors behind this work was realisation that there's no comprehensive, UK- or GB-wide research into the area of Native Trees. There is one for Scotland, the incredible [Native Woodland Survey of Scotland (NWSS)](https://www.data.gov.uk/dataset/da3f8548-a130-4a0d-8ddd-45019adcf1f3/native-woodland-survey-of-scotland-nwss), but even within it there's a note saying that current fixation on \"Ancient Woodland Sites\" isn't always useful if you want to answer the question of \"How many \"good\" trees are there in the UK?\".\n",
    "\n",
    "With this being a project aimed at illustrating the extent of the issue, the methodology is simple:\n",
    "- We assume that all relevant AWI (Ancient Woodland Inventory) areas entries are Native Trees\n",
    "- We exclude supplementary areas such as Open habitat in the NWSS dataset from the Trees area\n",
    "- For the final estimation we will use AWI England, AWI Cymru (Wales) and NWSS, since AWI Scotland closely mirrors the NWSS results but is significantly less detailed\n",
    "\n",
    "#### On Data Recency\n",
    "\n",
    "Most of the data here deals with relatively static entities and assumes that no existing areas of AWI or NWSS have been removed or significantly altered, therefore it's feasible to apply it in aggregate regardless of it's date of publication in most cases.\n",
    "\n",
    "#### Mapping\n",
    "\n",
    "When it comes to visualisation, the following aggregated types of areas will be used:\n",
    "- Native Trees: Areas that are classified as \"Ancient Semi-Natural Woodland\" (ASNW) or \"Native Woods\" (NW) in the datasets\n",
    "- Semi-Native Trees: Areas that are classified as \"Plantation on Ancient Woodland Sites\" (PAWS), \"Nearly-native woods\" (NNW), \"Long-established woodlands of plantation origin\" (LEPO) or \"Restored Ancient Woodland Sites\" (RAWS) in the datasets and other, assumed to be lower quality/desirability areas.\n",
    "\n",
    "#### Terminology\n",
    "\n",
    "The terminology between three AWI sources is different but aligned enough to draw comparisons if need be. The NWSS terminology and approach is significantly different, but it's also the one that's closest to the \"real\" trees.\n",
    "\n",
    "Below are the terms as defined by their respective methodology documents and my notes on \"desirability\" of each when viewed from \"we want more real trees\" standpoint:\n",
    "- **AWI England:**\n",
    "  - **ASNW (Ancient Semi-Natural Woodland):** Retains a native tree and shrub cover, ultimate \"native tree\" area, most desirable.\n",
    "  - **PAWS (Plantation on Ancient Woodland Sites):** Original tree cover has been felled and replaced by planting, often with conifers. It was, however, maintained in that state since, often developing semi-natural characteristics, which makes them desirable.\n",
    "  - **AWP (Ancient Wood Pasture):** Trees are managed in tandem with a long established tradition of grazing, characteristically with at least some veteran trees or shrubs, which still makes them desirable.\n",
    "- **AWI Scotland:** \n",
    "  - **AW (Ancient Woodland antiquity 1a and 2a):** Interpreted as semi-natural woodland from maps of 1750 (1a) or 1860 (2a) and continuously wooded to the present day. If planted with non-native species during the 20th century they are referred to as PAWS, consistently with AWI England, most desirable.\n",
    "  - **LEPO (Long-established woodlands of plantation origin):** Interpreted as plantation from maps of 1750 (1b1) or 1860 (2b) and continuously wooded since. Many of these sites have developed semi-natural characteristics, especially the oldest ones, which may be as rich as AW, relatively similar to PAWS in desirability and characteristics.\n",
    "  - **OR (Other 'Roy' woodland):** Shown as unwooded on the 1st edition maps but as woodland on the Roy maps. Such sites have, at most, had only a short break in continuity of woodland cover and may still retain features of Ancient Woodland.\n",
    "- **NWSS:**\n",
    "  - **NW (Native woods):** woodland where over 50% of the canopy (defined as the upper layers present at\n",
    "any part of the site) is composed of native species. Can be on ancient woodland sites or other areas where woodland is of more recent origin, the absolute most desirable as factors in the species and not infers it from the age.\n",
    "  - **NNW (Nearly-native woods):** woodland where between 40% and 50% of the canopy is composed of native\n",
    "species. These woods have the greatest potential to become native woods with appropriate action \n",
    "  - **AWS (Ancient Woodland sites):** the extent of ancient woodland as estimated in the Scottish Ancient\n",
    "Woodland Inventory (AWI Scotland)\n",
    "  - **PAWS (Plantation on Ancient Woodland Sites):** As recorded in AWI Scotland, ancient woodland where the original semi-natural tree cover on an ancient woodland site had been replaced by planted stands, usually of non native species. \n",
    "- **AWI Cymru (Wales):**\n",
    "  - **ASNW (Ancient Semi-Natural Woodland):** Broadleaf woodlands with mainly native tree and shrub species which are believed to have been in existence for over 400 years, most desirable.\n",
    "  - **PAWS (Plantation on Ancient Woodland Sites):** Sites which are believed to have been continuously wooded for over 400 years and currently have a canopy cover of more than 50 percent non-native conifer tree species, as desirable as other PAWS.\n",
    "  - **RAWS (Restored Ancient Woodland Sites):** Woodlands which are believed to have been continually wooded for over 400 years. These woodlands will have gone through a phase when canopy cover was more than 50% non-native conifer tree species and now have a canopy cover of more than 50 percent broadleaf. The phrase 'restored ancient woodland' describes woodland which appears, with the use of remote sensing techniques, to have returned to a more natural condition. The inventory designation does not mean that the woodland is fully restored or that it is in good ecological condition, rather desirable but it would be great to know their condition before inclusion.\n",
    "  - **AWSU (Ancient Woodland Site of Unknown Category):** Woodlands which may be any of the three categories above. These areas are mainly in transition and existing tree cover is described as 'shrubs', 'young trees', 'felled' or 'ground prepared for planting', a small category but worthy of inclusion nonetheless.\n",
    "\n",
    "### Sources\n",
    "- England: [Ancient Woodland (England)](https://www.data.gov.uk/dataset/9461f463-c363-4309-ae77-fdcd7e9df7d3/ancient-woodland-england), 2023, [Open Government Licence](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). [Methodology](https://publications.naturalengland.org.uk/publication/4876500800634880).\n",
    "- Northern Ireland: *no data available*\n",
    "- Scotland: \n",
    "  - [Ancient Woodland Inventory (Scotland)](https://www.data.gov.uk/dataset/c2f57ed9-5601-4864-af5f-a6e73e977f54/ancient-woodland-inventory-scotland), supplementary, 2023, [Open Government Licence](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). [Methodology](https://www.nature.scot/doc/guide-understanding-scottish-ancient-woodland-inventory-awi).\n",
    "  - [Native Woodland Survey of Scotland (NWSS)](https://www.data.gov.uk/dataset/da3f8548-a130-4a0d-8ddd-45019adcf1f3/native-woodland-survey-of-scotland-nwss) - 2006-2013, [Open Government Licence](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). [Methodology](https://forestry.gov.scot/publications/74-scotland-s-native-woodlands-results-from-the-native-woodland-survey-of-scotland/viewdocument/74).\n",
    "- Wales: [Ancient Woodland Inventory](https://www.data.gov.uk/dataset/1db7cbc0-2eef-4c3b-85be-73c7b87d4c87/ancient-woodland-inventory), 2023, [Open Government Licence](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). [Methodology](https://naturalresources.wales/guidance-and-advice/environmental-topics/woodlands-and-forests/identifying-ancient-woodlands/?lang=en).\n",
    "\n",
    "### Source Mapping\n",
    "To help with result reproduction, the mappings are:\n",
    "- AWI England: `../data/source/Ancient_Woodland_England_8159391193868782027/Ancient_Woodland___Natural_England.shp` - from [Ancient Woodland (England)](https://www.data.gov.uk/dataset/9461f463-c363-4309-ae77-fdcd7e9df7d3/ancient-woodland-england), ShapeFile (it might take 2-3 minutes to generate the link so give it a moment), CRS EPSG:27700\n",
    "- AWI Scotland: `../data/source/AWI_SCOTLAND_SHP_27700/AWI_SCOTLAND.shp` - from [Ancient Woodland Inventory (Scotland)](https://www.data.gov.uk/dataset/c2f57ed9-5601-4864-af5f-a6e73e977f54/ancient-woodland-inventory-scotland), ShapeFile, CRS EPSG:27700\n",
    "- NWSS: `../data/source/Native_Woodland_Survey_of_Scotland_4228117159943345825/Native_Woodland_Survey_of_Scotland.shp` - from [Native Woodland Survey of Scotland (NWSS)](https://www.data.gov.uk/dataset/da3f8548-a130-4a0d-8ddd-45019adcf1f3/native-woodland-survey-of-scotland-nwss), ShapeFile, CRS EPSG:4326 (for some reason, we convert to 27700 in the workflow)\n",
    "- AWI Cymru (Wales): `../data/source/NRW_ANCIENT_WOODLAND_INVENTORY_2021/NRW_ANCIENT_WOODLAND_INVENTORY_2021Polygon.shp` - [Ancient Woodland Inventory](https://www.data.gov.uk/dataset/1db7cbc0-2eef-4c3b-85be-73c7b87d4c87/ancient-woodland-inventory), ShapeFile, CRS EPSG:27700\n",
    "\n",
    "In case a ShapeFile is unavailable, use the `GeoJSON` and mind the CRS, convert to EPSG:27700.\n",
    "\n",
    "### Notes\n",
    "\n",
    "All runtimes are indicated for Apple M1 Max 64GB. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import geopandas as gpd\n",
    "gpd.options.io_engine = 'pyogrio'\n",
    "import util.geo_ops as gops"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "awi_eng = gpd.read_file('../data/source/Ancient_Woodland_England_8159391193868782027/Ancient_Woodland___Natural_England.shp')\n",
    "awi_sco = gpd.read_file('../data/source/AWI_SCOTLAND_SHP_27700/AWI_SCOTLAND.shp')\n",
    "nwss = gpd.read_file('../data/source/Native_Woodland_Survey_of_Scotland_4228117159943345825/Native_Woodland_Survey_of_Scotland.shp')\n",
    "awi_cym = gpd.read_file('../data/source/NRW_ANCIENT_WOODLAND_INVENTORY_2021/NRW_ANCIENT_WOODLAND_INVENTORY_2021Polygon.shp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Expected: 364,914.32 ha\n",
    "print(f'Total AWI Area:', '{:,.2f}ha'.format(awi_eng['AREA'].sum()))\n",
    "gops.geodf_summary(awi_eng, 'THEMNAME', 'AREA')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Expected: 352,766.04 ha\n",
    "print(f'Total AWI Area:', '{:,.2f}ha'.format(awi_sco.HECTARE.sum()))\n",
    "gops.geodf_summary(awi_sco, 'ANTIQUITY', 'HECTARE')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Expected: 385,979.70, 364,208.23 ha w/o open land habitat\n",
    "print(f'Total AWI Area:', '{:,.2f}ha'.format(nwss.HECTARES.sum()), 'or', '{:,.2f}ha'.format(nwss[nwss['TYPE'] != 'Open land habitat'].HECTARES.sum()), 'w/o Open Land Habitat')\n",
    "gops.geodf_summary(nwss, 'TYPE', 'HECTARES')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Expected: 94,849.94 ha\n",
    "print(f'Total AWI Area:', '{:,.2f}ha'.format(awi_cym.area_hecta.sum()))\n",
    "gops.geodf_summary(awi_cym, 'catagory_n', 'area_hecta')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combined Dataset\n",
    "In order to represetnt the Native Trees across the GB we'll reduce the various AWI and NWSS data types to two: Native Trees and Semi-Native Trees. As mentioned above, we'll dismiss the AWI Scotland dataset in favour of NWSS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Merged mappings\n",
    "mappings = {\n",
    "    'England': {\n",
    "        'type_source': 'THEMNAME',\n",
    "        'type_aggregate': {\n",
    "            'Ancient & Semi-Natural Woodland': 'Ancient & Mostly Natural Woodland',\n",
    "            'Ancient Replanted Woodland': 'Plantation on Ancient Woodland Site',\n",
    "            'Ancient Wood Pasture': 'Other (unknown quality, pasture, etc.)'\n",
    "        },\n",
    "        'type_combined': {\n",
    "            'Ancient & Mostly Natural Woodland': 'Native Trees',\n",
    "            'Plantation on Ancient Woodland Site': 'Semi-Native Trees',\n",
    "            'Other (unknown quality, pasture, etc.)': 'Other (land, pasture, unknown, etc.)'\n",
    "        },\n",
    "        'area_source': 'AREA',\n",
    "        'area_target': 'area_ha'\n",
    "    },\n",
    "    'Scotland': {\n",
    "        'type_source': 'TYPE',\n",
    "        'type_aggregate': {\n",
    "            'Native woodland': 'Ancient & Mostly Natural Woodland',\n",
    "            'PAWS': 'Plantation on Ancient Woodland Site',\n",
    "            'Nearly-native woodland': 'Semi-Natural Woodland',\n",
    "            'Open land habitat': 'Other (unknown quality, pasture, etc.)'\n",
    "        },\n",
    "        'type_combined': {\n",
    "            'Ancient & Mostly Natural Woodland': 'Native Trees',\n",
    "            'Semi-Natural Woodland': 'Semi-Native Trees',\n",
    "            'Plantation on Ancient Woodland Site': 'Semi-Native Trees',\n",
    "            'Other (unknown quality, pasture, etc.)': 'Other (land, pasture, unknown, etc.)'\n",
    "        },\n",
    "        'area_source': 'HECTARES',\n",
    "        'area_target': 'area_ha'\n",
    "    },\n",
    "    'Wales': {\n",
    "        'type_source': 'catagory_n',\n",
    "        'type_aggregate': {\n",
    "            'Ancient Semi Natural Woodland': 'Ancient & Mostly Natural Woodland',\n",
    "            'Plantation on Ancient Woodland Site': 'Plantation on Ancient Woodland Site',\n",
    "            'Restored Ancient Woodland Site': 'Semi-Natural Woodland',\n",
    "            'Restored Ancient Semi Natural Woodland': 'Semi-Natural Woodland',\n",
    "            'Ancient Woodland Site of Unknown Category': 'Other (unknown quality, pasture, etc.)'\n",
    "        },\n",
    "        'type_combined': {\n",
    "            'Ancient & Mostly Natural Woodland': 'Native Trees',\n",
    "            'Semi-Natural Woodland': 'Semi-Native Trees',\n",
    "            'Plantation on Ancient Woodland Site': 'Semi-Native Trees',\n",
    "            'Other (unknown quality, pasture, etc.)': 'Other (land, pasture, unknown, etc.)'\n",
    "        },\n",
    "        'area_source': 'area_hecta',\n",
    "        'area_target': 'area_ha'\n",
    "    }\n",
    "}\n",
    "\n",
    "# Aligning the CRS before merging\n",
    "gdfs = {\n",
    "    'England': awi_eng,\n",
    "    'Scotland': nwss.to_crs(epsg=27700),\n",
    "    'Wales': awi_cym\n",
    "}\n",
    "\n",
    "# Applying the mappings\n",
    "for country, mapping in mappings.items():\n",
    "    gdf = gdfs[country]\n",
    "    gdf_harmonised = gpd.GeoDataFrame(gdf, geometry=gdf.geometry, crs=gdf.crs)\n",
    "    gdf_harmonised['type_source'] = gdf[mapping['type_source']]\n",
    "    gdf_harmonised['type_aggregate'] = gdf[mapping['type_source']].replace(mapping['type_aggregate'])\n",
    "    gdf_harmonised['type_combined'] = gdf_harmonised['type_aggregate'].replace(mapping['type_combined'])\n",
    "    gdf_harmonised[mapping['area_target']] = gdf[mapping['area_source']]\n",
    "    gdf_harmonised['country'] = country\n",
    "    gdfs[country] = gdf_harmonised\n",
    "\n",
    "# Merging the GeoDataFrames and rearranging the columns\n",
    "awi_dataset = pd.concat(gdfs.values(), ignore_index=True)\n",
    "awi_dataset['source_index'] = awi_dataset.index\n",
    "awi_dataset = awi_dataset[['source_index', 'country', 'type_combined', 'type_aggregate', 'type_source', 'area_ha', 'geometry']]\n",
    "\n",
    "# Writing to CSV\n",
    "types = ['combined', 'aggregate', 'source']\n",
    "for type in types:\n",
    "    awi_dataset.groupby(['country', f'type_{type}']).area_ha.sum().to_csv(f'../data/sheets/gb_awi_{type}_summary.csv')\n",
    "\n",
    "# Expected total area: 845,743.96 ha\n",
    "print(f'Total AWI Area:', '{:,.2f}ha'.format(awi_dataset.area_ha.sum()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Saving to Parquet for future use.\n",
    "# Runtime: 2s, Storage: 425MB\n",
    "awi_dataset.to_parquet(f'../data/processed/gb_awi_dataset.parquet')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}