Building_a_Network_Graph_of_Global_Trade.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ba3d9fee",
   "metadata": {},
   "source": [
    "<br><br><br>\n",
    "<span style=\"font-size:36px;color:#ffffff;font-weight:bold;background-color:#134DD1\">Building a Network Graph of Global Trade</span>\n",
    "<br><br><br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb0c3cb7",
   "metadata": {},
   "source": [
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">What is this graph for?</span>\n",
    "\n",
    "<b>It allows you to quickly gain insight into how international trade is organized for a particular product, identify trade links that countries have not reported, and identify countries and territories that have closed trade reporting or do not report trade at all.</b>\n",
    "\n",
    "The network connects each country/territory to its top N largest (in USD) commodity suppliers. It displays:\n",
    "\n",
    "- the direction of trade flow between the country/territory and its supplier (the difference between country-supplier exports and supplier-country exports);\n",
    "- total commodity trade (exports + imports in each country/territory;\n",
    "- a parameter by choice:\n",
    "\n",
    "    - the region each country/territory belongs to (Africa, Americas, Antarctica, Asia, Europe, Oceania, Special categories, i.e. bunkers, free zones);\n",
    "    - the balance between exports and imports of the commodity for each country/territory (10/90%, 50/50%, 70/30%, etc.);\n",
    "    - each country/territory's rank inside this network, depending on the number of its trade links (aka Google PageRank).\n",
    "    \n",
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">What do we get by executing the code?</span>\n",
    "\n",
    "An interactive network, colored according to one of our chosen parameters; its nodes, when clicked, will display the name of the country and will be highlighted with all its trade connections; this network can be saved in .png format.\n",
    " \n",
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">Limitations</span>\n",
    "\n",
    "To use the code, you need to register a Comtrade B2C account and get a free subscription key. [REGISTRATION GUIDE](https://unstats.un.org/wiki/display/comtrade/New+Comtrade+User+Guide#NewComtradeUserGuide-UNComtradeAPIManagement).\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">Contents</span><a class=\"anchor\" id=\"contents\"></a>\n",
    "\n",
    "* [Imports, Data Requests](#0)\n",
    "* [Data Processing](#1)\n",
    "* [Creating the layout in PyGraphviz](#2)\n",
    "* [From PyGraphviz to Cytoscape layout](#3)\n",
    "* [Building the graph](#4)\n",
    "\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">Imports, Data Requests</span><a class=\"anchor\" id=\"0\"></a>\n",
    "\n",
    "#### PACKAGES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1552c5fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import math\n",
    "\n",
    "import comtradeapicall                                      # To extract the data \n",
    "import time                                                 # from UN Comtrade\n",
    "\n",
    "import pygraphviz as pgv                                    # To calculate the initial network positions\n",
    "\n",
    "import networkx as nx                                       # To calculate nodes' PageRanks\n",
    "\n",
    "import dash_cytoscape as cyto                               # To build the network\n",
    "cyto.load_extra_layouts()\n",
    "from plotly.colors import hex_to_rgb\n",
    "\n",
    "from dash import Dash, html, Input, Output, callback, ctx   # To build the final layout\n",
    "import dash_bootstrap_components as dbc\n",
    "from datetime import datetime;\n",
    "\n",
    "import warnings\n",
    "\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd946ff",
   "metadata": {},
   "source": [
    "#### COMTRADE SUBSCRIPTION KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "72ed8de2",
   "metadata": {},
   "outputs": [],
   "source": [
    "subscription_key = 'YOUR_SUBSCRIPTION_CODE'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fbfd28d",
   "metadata": {},
   "source": [
    "#### COMMODITY AND COUNTRY CODES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0d781186",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>section</th>\n",
       "      <th>hscode</th>\n",
       "      <th>description</th>\n",
       "      <th>parent</th>\n",
       "      <th>level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I</td>\n",
       "      <td>01</td>\n",
       "      <td>Animals; live</td>\n",
       "      <td>TOTAL</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>I</td>\n",
       "      <td>0101</td>\n",
       "      <td>Horses, asses, mules and hinnies; live</td>\n",
       "      <td>01</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>I</td>\n",
       "      <td>010121</td>\n",
       "      <td>Horses; live, pure-bred breeding animals</td>\n",
       "      <td>0101</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>I</td>\n",
       "      <td>010129</td>\n",
       "      <td>Horses; live, other than pure-bred breeding an...</td>\n",
       "      <td>0101</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>I</td>\n",
       "      <td>010130</td>\n",
       "      <td>Asses; live</td>\n",
       "      <td>0101</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  section  hscode                                        description parent  \\\n",
       "0       I      01                                      Animals; live  TOTAL   \n",
       "1       I    0101             Horses, asses, mules and hinnies; live     01   \n",
       "2       I  010121           Horses; live, pure-bred breeding animals   0101   \n",
       "3       I  010129  Horses; live, other than pure-bred breeding an...   0101   \n",
       "4       I  010130                                        Asses; live   0101   \n",
       "\n",
       "   level  \n",
       "0      2  \n",
       "1      4  \n",
       "2      6  \n",
       "3      6  \n",
       "4      6  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "commodities = pd.read_csv('comtrade_codes/harmonized-system.csv')  # Commodity HS-codes used by Comtrade. Source: \n",
    "                                                                   # https://github.com/datasets/harmonized-system\n",
    "commodities.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "76b964af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>all</td>\n",
       "      <td>All</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12</td>\n",
       "      <td>Algeria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>20</td>\n",
       "      <td>Andorra</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id         text\n",
       "0  all          All\n",
       "1    4  Afghanistan\n",
       "2    8      Albania\n",
       "3   12      Algeria\n",
       "4   20      Andorra"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reporters = pd.read_csv('comtrade_codes/reporterAreas.csv')  # Comtrade reporters, partners, and their codes. Source: \n",
    "partners = pd.read_csv('comtrade_codes/partnerAreas.csv')    # https://unstats.un.org/wiki/display/comtrade/Comtrade+Country+Code+and+Name\n",
    "\n",
    "reporters.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fcab152",
   "metadata": {},
   "source": [
    "#### REQUESTING THE DATA"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cc6def9",
   "metadata": {},
   "source": [
    "In this notebook, I'll build a wheat and meslin (Comtrade code 1001) global trade network for the year 2022.\n",
    "\n",
    "Export and import data for each country are requested separately.\n",
    "\n",
    "The Comtrade API imposes rate limits, which is why I put a 10-second pause between my two requests here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "217fa1ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "comtrade_exp = comtradeapicall.getFinalData(\n",
    "    subscription_key,\n",
    "    typeCode='C',                                   # C = commodities\n",
    "    freqCode='A',                                   # A = annual\n",
    "    clCode='HS',\n",
    "    period=\n",
    "    '2022',                                         # year; several years should be listed in order and separated by commas\n",
    "    reporterCode=None,                              # all exporters\n",
    "    cmdCode=\n",
    "    '1001',                                         # wheat and meslin hs-code; several codes should be separated by commas\n",
    "    flowCode='X',                                   # X = export\n",
    "    partnerCode=None,                               # all importers\n",
    "    partner2Code='0',\n",
    "    customsCode='C00',\n",
    "    motCode='0',\n",
    "    maxRecords=250000)\n",
    "\n",
    "time.sleep(10)\n",
    "\n",
    "comtrade_imp = comtradeapicall.getFinalData(\n",
    "    subscription_key,\n",
    "    typeCode='C',\n",
    "    freqCode='A',\n",
    "    clCode='HS',\n",
    "    period='2022',\n",
    "    reporterCode=None,                              # all importers\n",
    "    cmdCode='1001',\n",
    "    flowCode='M',                                   # M = import\n",
    "    partnerCode=None,                               # all exporters\n",
    "    partner2Code='0',\n",
    "    customsCode='C00',\n",
    "    motCode='0',\n",
    "    maxRecords=250000)\n",
    "\n",
    "comtrade_imp = comtrade_imp[\n",
    "    comtrade_imp['reporterCode'] != comtrade_imp['partnerCode']]  # Getting rid of a country's trade with itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "dc7cac5f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>typeCode</th>\n",
       "      <th>freqCode</th>\n",
       "      <th>refPeriodId</th>\n",
       "      <th>refYear</th>\n",
       "      <th>refMonth</th>\n",
       "      <th>period</th>\n",
       "      <th>reporterCode</th>\n",
       "      <th>reporterISO</th>\n",
       "      <th>reporterDesc</th>\n",
       "      <th>flowCode</th>\n",
       "      <th>...</th>\n",
       "      <th>netWgt</th>\n",
       "      <th>isNetWgtEstimated</th>\n",
       "      <th>grossWgt</th>\n",
       "      <th>isGrossWgtEstimated</th>\n",
       "      <th>cifvalue</th>\n",
       "      <th>fobvalue</th>\n",
       "      <th>primaryValue</th>\n",
       "      <th>legacyEstimationFlag</th>\n",
       "      <th>isReported</th>\n",
       "      <th>isAggregate</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>X</td>\n",
       "      <td>...</td>\n",
       "      <td>333.053</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1506.098</td>\n",
       "      <td>1506.098</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>X</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>62.966</td>\n",
       "      <td>62.966</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>X</td>\n",
       "      <td>...</td>\n",
       "      <td>177.000</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1030.987</td>\n",
       "      <td>1030.987</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>X</td>\n",
       "      <td>...</td>\n",
       "      <td>153.053</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>404.934</td>\n",
       "      <td>404.934</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>X</td>\n",
       "      <td>...</td>\n",
       "      <td>2.000</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.211</td>\n",
       "      <td>7.211</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 47 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  typeCode freqCode  refPeriodId  refYear  refMonth period  reporterCode  \\\n",
       "0        C        A     20220101     2022        52   2022            24   \n",
       "1        C        A     20220101     2022        52   2022            24   \n",
       "2        C        A     20220101     2022        52   2022            24   \n",
       "3        C        A     20220101     2022        52   2022            24   \n",
       "4        C        A     20220101     2022        52   2022            24   \n",
       "\n",
       "  reporterISO reporterDesc flowCode  ...   netWgt  isNetWgtEstimated grossWgt  \\\n",
       "0        None         None        X  ...  333.053               True      0.0   \n",
       "1        None         None        X  ...    1.000              False      0.0   \n",
       "2        None         None        X  ...  177.000              False      0.0   \n",
       "3        None         None        X  ...  153.053               True      0.0   \n",
       "4        None         None        X  ...    2.000              False      0.0   \n",
       "\n",
       "  isGrossWgtEstimated  cifvalue  fobvalue primaryValue legacyEstimationFlag  \\\n",
       "0               False       NaN  1506.098     1506.098                    6   \n",
       "1               False       NaN    62.966       62.966                    0   \n",
       "2               False       NaN  1030.987     1030.987                    0   \n",
       "3               False       NaN   404.934      404.934                    6   \n",
       "4               False       NaN     7.211        7.211                    0   \n",
       "\n",
       "  isReported  isAggregate  \n",
       "0      False         True  \n",
       "1      False         True  \n",
       "2      False         True  \n",
       "3      False         True  \n",
       "4      False         True  \n",
       "\n",
       "[5 rows x 47 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comtrade_exp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6aad0b7e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>typeCode</th>\n",
       "      <th>freqCode</th>\n",
       "      <th>refPeriodId</th>\n",
       "      <th>refYear</th>\n",
       "      <th>refMonth</th>\n",
       "      <th>period</th>\n",
       "      <th>reporterCode</th>\n",
       "      <th>reporterISO</th>\n",
       "      <th>reporterDesc</th>\n",
       "      <th>flowCode</th>\n",
       "      <th>...</th>\n",
       "      <th>netWgt</th>\n",
       "      <th>isNetWgtEstimated</th>\n",
       "      <th>grossWgt</th>\n",
       "      <th>isGrossWgtEstimated</th>\n",
       "      <th>cifvalue</th>\n",
       "      <th>fobvalue</th>\n",
       "      <th>primaryValue</th>\n",
       "      <th>legacyEstimationFlag</th>\n",
       "      <th>isReported</th>\n",
       "      <th>isAggregate</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>20</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>M</td>\n",
       "      <td>...</td>\n",
       "      <td>6.618347e+04</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>3.089292e+04</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.089292e+04</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>20</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>M</td>\n",
       "      <td>...</td>\n",
       "      <td>6.618347e+04</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>3.089292e+04</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.089292e+04</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>M</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>3.857492e+08</td>\n",
       "      <td>3.282984e+08</td>\n",
       "      <td>3.857492e+08</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>M</td>\n",
       "      <td>...</td>\n",
       "      <td>2.025550e+08</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>1.073306e+08</td>\n",
       "      <td>8.765932e+07</td>\n",
       "      <td>1.073306e+08</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>C</td>\n",
       "      <td>A</td>\n",
       "      <td>20220101</td>\n",
       "      <td>2022</td>\n",
       "      <td>52</td>\n",
       "      <td>2022</td>\n",
       "      <td>24</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>M</td>\n",
       "      <td>...</td>\n",
       "      <td>2.908050e+07</td>\n",
       "      <td>False</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2.992217e+07</td>\n",
       "      <td>2.545128e+07</td>\n",
       "      <td>2.992217e+07</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 47 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  typeCode freqCode  refPeriodId  refYear  refMonth period  reporterCode  \\\n",
       "0        C        A     20220101     2022        52   2022            20   \n",
       "1        C        A     20220101     2022        52   2022            20   \n",
       "2        C        A     20220101     2022        52   2022            24   \n",
       "3        C        A     20220101     2022        52   2022            24   \n",
       "4        C        A     20220101     2022        52   2022            24   \n",
       "\n",
       "  reporterISO reporterDesc flowCode  ...        netWgt  isNetWgtEstimated  \\\n",
       "0        None         None        M  ...  6.618347e+04               True   \n",
       "1        None         None        M  ...  6.618347e+04               True   \n",
       "2        None         None        M  ...           NaN              False   \n",
       "3        None         None        M  ...  2.025550e+08               True   \n",
       "4        None         None        M  ...  2.908050e+07              False   \n",
       "\n",
       "  grossWgt isGrossWgtEstimated      cifvalue      fobvalue  primaryValue  \\\n",
       "0      0.0               False  3.089292e+04           NaN  3.089292e+04   \n",
       "1      0.0               False  3.089292e+04           NaN  3.089292e+04   \n",
       "2      0.0               False  3.857492e+08  3.282984e+08  3.857492e+08   \n",
       "3      0.0               False  1.073306e+08  8.765932e+07  1.073306e+08   \n",
       "4      0.0               False  2.992217e+07  2.545128e+07  2.992217e+07   \n",
       "\n",
       "  legacyEstimationFlag isReported  isAggregate  \n",
       "0                    6      False         True  \n",
       "1                    6      False         True  \n",
       "2                    0      False         True  \n",
       "3                    6      False         True  \n",
       "4                    0      False         True  \n",
       "\n",
       "[5 rows x 47 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comtrade_imp.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07e87f56",
   "metadata": {},
   "source": [
    "\n",
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">Data Processing</span><a class=\"anchor\" id=\"1\"></a>\n",
    "\n",
    "[UP](#contents)\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "We have two raw Comtrade dataframes now: \n",
    "\n",
    "* Each reporter's exports (in USD): total and to each of its partners. Exports are reported in Free On Board (FOB) terms.\n",
    "* Each reporter's imports (in USD): total and from each of its partners. Imports are reported in Cost, Insurance and Freight (CIF) terms.\n",
    "\n",
    "<b>Countries might not declare imports or exports to certain partners, partly or totally. For the network to be as complete as possible, we need to identify undeclared trade links:</b>\n",
    "\n",
    "* If country A didn't report exports to country B but country B declared imports from A, I rely on B's data. \n",
    "* If country A didn't report any exports at all but other countries declared imports from A, I sum these countries' imports as A's total exports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9e28f166",
   "metadata": {},
   "outputs": [],
   "source": [
    "export_totals = comtrade_exp[comtrade_exp['partnerCode'] == 0][[             # Total exports by country\n",
    "    'reporterCode', 'partnerCode', 'fobvalue'\n",
    "]].groupby(['reporterCode', 'partnerCode']).agg('sum').reset_index()\n",
    "\n",
    "import_totals = comtrade_imp[comtrade_imp['partnerCode'] == 0][[             # Total imports by country\n",
    "    'reporterCode', 'partnerCode', 'cifvalue'\n",
    "]].groupby(['reporterCode', 'partnerCode']).agg('sum').reset_index()\n",
    "\n",
    "export_by_country = comtrade_exp[comtrade_exp['partnerCode'] != 0][[         # Exports by exporter & importer\n",
    "    'reporterCode', 'partnerCode', 'fobvalue'\n",
    "]].groupby(['reporterCode', 'partnerCode']).agg('sum').reset_index()\n",
    "\n",
    "import_by_country = comtrade_imp[comtrade_imp['partnerCode'] != 0][[         # Imports by importer & exporter\n",
    "    'reporterCode', 'partnerCode', 'cifvalue'\n",
    "]].groupby(['reporterCode', 'partnerCode']).agg('sum').reset_index()\n",
    "\n",
    "\n",
    "for dataset in [                                                             # I'll countr the initial node positions \n",
    "        export_totals, import_totals, export_by_country, import_by_country   # with PyGraphviz.                                                                  \n",
    "]:                                                                           # This tool, when sizing the nodes, takes  \n",
    "    for col in ['reporterCode', 'partnerCode']:                              # into account their indexes' lengths; \n",
    "        dataset[col] = [                                                     # to neutralize this factor, I create \n",
    "            '00' + str(x) if len(str(x)) == 1 else '0' +                     # equal-length 3-character string indexes.          \n",
    "            str(x) if len(str(x)) == 2 else str(x)\n",
    "            for x in dataset[col].tolist()\n",
    "        ]\n",
    "\n",
    "export_by_country.columns = ['exporter', 'importer', 'value']\n",
    "import_by_country.columns = ['importer', 'exporter', 'value']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d1f0501",
   "metadata": {},
   "source": [
    "#### NODES \n",
    "\n",
    "For PyGraphviz, we need to set each node's width and height, which, in our case, mean the diameter (2\\*sqrt(area/pi)). I'll size the nodes' areas by their countries' total trade. However, the nodes won't be smaller than a certain size to stay visible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ebbaf977",
   "metadata": {},
   "outputs": [],
   "source": [
    "total_trade = export_totals[[                                                    # Exports and imports reported officially \n",
    "    'reporterCode',                                                              # by countries\n",
    "    'fobvalue'                                                               \n",
    "]].set_index('reporterCode').join(\n",
    "    import_totals[['reporterCode', 'cifvalue']].set_index('reporterCode'),\n",
    "    how='outer').reset_index().rename(columns={\n",
    "        'reporterCode': 'country_code',\n",
    "        'fobvalue': 'export',\n",
    "        'cifvalue': 'import'\n",
    "    }).set_index('country_code')\n",
    "\n",
    "for col in ['export', 'import']:                                                 # Marking the rows with missing official\n",
    "    total_trade[col + '_note'] = [                                               # exports or imports data\n",
    "        1 if math.isnan(x) else 0 for x in total_trade[col].tolist()\n",
    "    ]\n",
    "\n",
    "imports_to_add = export_by_country[[                                             # Partners' data: country's total exports \n",
    "    'exporter',                                                                  # according to its importers, and country's\n",
    "    'importer',                                                                  # total imports according to its exporters         \n",
    "    'value'\n",
    "]].groupby('importer').agg('sum')                                            \n",
    "exports_to_add = import_by_country[[\n",
    "    'importer',\n",
    "    'exporter',                   \n",
    "    'value'\n",
    "]].groupby('exporter').agg('sum')\n",
    "\n",
    "trade_to_add = exports_to_add.rename(columns={\n",
    "    'value': 'export_by_partners'\n",
    "}).join(imports_to_add.rename(columns={'value': 'import_by_partners'}),\n",
    "        how='outer')\n",
    "\n",
    "df_nodes = total_trade.join(trade_to_add, how='outer')[[\n",
    "    'export', 'import', 'export_by_partners', 'import_by_partners'\n",
    "]]\n",
    "\n",
    "df_nodes[['export', 'import', 'export_by_partners',\n",
    "          'import_by_partners']] = df_nodes[[\n",
    "              'export', 'import', 'export_by_partners', 'import_by_partners'\n",
    "          ]].fillna(0)\n",
    "\n",
    "df_nodes['trade'] = [                                                             # Total trade (sizing parameter)\n",
    "    ex + im if ex != 0 and im != 0                                                # Counting by partners' data if official \n",
    "    else exp + im if ex == 0 and im != 0                                          # data is lacking\n",
    "    else ex + imp if ex != 0 and im == 0 else exp + imp for ex, im, exp, imp in\n",
    "    zip(df_nodes['export'], df_nodes['import'], df_nodes['export_by_partners'],\n",
    "        df_nodes['import_by_partners'])\n",
    "]\n",
    "\n",
    "df_nodes = df_nodes[df_nodes['trade'] >                                           # Keep only the countries with non-zero trade\n",
    "                    0]  \n",
    "\n",
    "df_nodes['trade_rescaled'] = df_nodes['trade'] / df_nodes['trade'].max(\n",
    ") * 60                                                                            # Rescaling the sizes: max = 60 and min = 0.15\n",
    "df_nodes['trade_rescaled'] = [\n",
    "    0.15 if x <= 0.15 else x for x in df_nodes['trade_rescaled'].tolist()\n",
    "]\n",
    "\n",
    "df_nodes['diameter'] = [                                                          # Nodes' diameters\n",
    "    np.sqrt(x / np.pi) * 2 for x in df_nodes['trade_rescaled'].tolist()\n",
    "]\n",
    "\n",
    "diameter_dict = df_nodes['diameter'].to_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69a7e7e1",
   "metadata": {},
   "source": [
    "#### LINKS \n",
    "\n",
    "At the moment, our country-to-country trade data represents country A's exports to country B and country B's exports to country A as two separate rows. We need to unify these rows into a single \"A-B trade link\" row and define which country is the \"source\" and which is the \"target\" of this trade link. The source is the side that exports more than it gets back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b5a15ae9",
   "metadata": {},
   "outputs": [],
   "source": [
    "export_by_country = export_by_country[                                            # Keep only countries with non-zero trade\n",
    "    (export_by_country['exporter'].isin(df_nodes.index.tolist()))\n",
    "    & (export_by_country['importer'].isin(df_nodes.index.tolist()))]\n",
    "import_by_country = import_by_country[\n",
    "    (import_by_country['exporter'].isin(df_nodes.index.tolist()))\n",
    "    & (import_by_country['importer'].isin(df_nodes.index.tolist()))]\n",
    "\n",
    "\n",
    "df_list = []                                                                      # Creating a dataset of mutual trade flows\n",
    "                                                                                  # out of export and import dataframes\n",
    "for dataset in [export_by_country, import_by_country]:\n",
    "\n",
    "    dataset['source_target'] = [                                                  # a unique country-to-country link index\n",
    "        '_'.join(sorted([exporter, importer]))\n",
    "        for exporter, importer in zip(dataset['exporter'], dataset['importer'])\n",
    "    ] \n",
    "\n",
    "    dataset['order'] = dataset.groupby('source_target').cumcount()\n",
    "\n",
    "    df = dataset[dataset['order'] == 0].set_index('source_target')[[\n",
    "        'exporter', 'importer', 'value'\n",
    "    ]].rename(columns={\n",
    "        'value': 'to'\n",
    "    }).join(dataset[dataset['order'] == 1].set_index('source_target')[[\n",
    "        'value'\n",
    "    ]].rename(columns={'value': 'back'}))\n",
    "\n",
    "    df_list.append(df)\n",
    "\n",
    "\n",
    "source_target_df = df_list[0].combine_first(df_list[1])                           # Filling the missing values/interactions \n",
    "                                                                                  # from exporters' data with corresponding \n",
    "source_target_df = source_target_df.fillna(0)                                     # values/interactions declared by the importers\n",
    "\n",
    "source_target_df[[                                                                # Defining the source and target side, \n",
    "    'source', 'target'                                                            # depending on exports worth\n",
    "]] = [[exporter, importer] if to - back >= 0 else [importer, exporter]\n",
    "      for exporter, importer, to, back in\n",
    "      zip(source_target_df['exporter'], source_target_df['importer'],\n",
    "          source_target_df['to'], source_target_df['back'])]\n",
    "\n",
    "df_links = source_target_df.reset_index()[['source', 'target']]\n",
    "\n",
    "\n",
    "rank_df = export_by_country[['importer', 'exporter', 'value']].set_index([       # Ranking trade partners for each country \n",
    "    'importer', 'exporter'                                                       # depending on the total cost of the commodity \n",
    "]).combine_first(import_by_country[['importer', 'exporter', 'value'              # delivered\n",
    "                                    ]].set_index(['importer',\n",
    "                                                  'exporter'])).reset_index()\n",
    "\n",
    "rank_df['supplier_rank'] = rank_df.sort_values(\n",
    "    by='value', ascending=False).groupby('importer').cumcount() + 1\n",
    "\n",
    "supplier_rank_dict = rank_df.set_index(['importer', 'exporter'\n",
    "                                        ])['supplier_rank'].to_dict()\n",
    "\n",
    "df_links['supplier_rank_source'] = df_links.set_index(\n",
    "    ['target', 'source']).index.map(supplier_rank_dict)\n",
    "\n",
    "df_links['supplier_rank_target'] = df_links.set_index(\n",
    "    ['source', 'target']).index.map(supplier_rank_dict)\n",
    "\n",
    "\n",
    "link_dict = dict()                                                               # In the network, each country will be \n",
    "                                                                                 # connected to only its two largest suppliers. \n",
    "for e in export_by_country.exporter:                                             # At the same time, one of our ways of coloring\n",
    "    link_dict[e] = []                                                            # network nodes is the number of trade links. \n",
    "for i in export_by_country.importer:                                             # Here we create a dictionary in which we save \n",
    "    if i not in link_dict.keys():                                                # the real, and not truncated, number of \n",
    "        link_dict[i] = []                                                        # connections for each country.\n",
    "for i in import_by_country.importer:\n",
    "    if i not in link_dict.keys():\n",
    "        link_dict[i] = []\n",
    "for e in import_by_country.exporter:\n",
    "    if e not in link_dict.keys():\n",
    "        link_dict[e] = []\n",
    "        \n",
    "for e in export_by_country.exporter:\n",
    "    data = export_by_country[export_by_country.exporter == e]\n",
    "    importers_list = data['importer'].unique().tolist()\n",
    "    for i in importers_list:\n",
    "        link_dict[e].append(i)\n",
    "        \n",
    "for i in export_by_country.importer:\n",
    "    data = export_by_country[export_by_country.importer == i]\n",
    "    exporters_list = data['exporter'].unique().tolist()\n",
    "    for e in exporters_list:\n",
    "        link_dict[i].append(e)\n",
    "        \n",
    "for e in import_by_country.exporter:\n",
    "    data = import_by_country[import_by_country.exporter == e]\n",
    "    importers_list = data['importer'].unique().tolist()\n",
    "    for i in importers_list:\n",
    "        link_dict[e].append(i)\n",
    "        \n",
    "for i in import_by_country.importer:\n",
    "    data = import_by_country[import_by_country.importer == i]\n",
    "    exporters_list = data['exporter'].unique().tolist()\n",
    "    for e in exporters_list:\n",
    "        link_dict[i].append(e)\n",
    "        \n",
    "for key in link_dict.keys():\n",
    "    link_dict[key] = len(list(dict.fromkeys(link_dict[key])))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebdb1b00",
   "metadata": {},
   "source": [
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">Creating the layout in PyGraphviz</span><a class=\"anchor\" id=\"2\"></a>\n",
    "\n",
    "[UP](#contents)\n",
    "<br>\n",
    "<br>\n",
    "I'll connect each country to its top two wheat suppliers. You can control this number, depending on the particular network's density: sometimes there are too many unrelated countries, and sometimes they are so interconnected that three links already look messy.\n",
    "\n",
    "I'll build a network graph in several steps: first, I'll create a network from my links dataframe; then I'll set node parameters; then I'll create the layout similar to the spring layout in Networkx."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2fe394bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_links = 2 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8582125d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df_links[\n",
    "    (df_links['supplier_rank_source'].isin(list(np.arange(n_links + 1))[1:])) |\n",
    "    (df_links['supplier_rank_target'].isin(list(np.arange(n_links +\n",
    "                                                          1))[1:]))].copy()\n",
    "\n",
    "layout_dict = dict()                                                              # A dict we'll pass to PyGraphviz\n",
    "\n",
    "for s in df['source'].unique().tolist():                                          # For each country, set a list of trade  \n",
    "    s_list = df[df['source'] == s]['target'].unique().tolist()                    # partners to connect with\n",
    "    layout_dict[s] = s_list\n",
    "\n",
    "G = pgv.AGraph(layout_dict)                                                       # The network is created\n",
    "\n",
    "for i, node in enumerate(G.iternodes()):                                          # Some node attributes\n",
    "    node.attr['shape'] = 'circle'\n",
    "    node.attr['width'] = diameter_dict[node]\n",
    "    node.attr['height'] = diameter_dict[node]\n",
    "    node.attr['fixedsize'] = True\n",
    "    node.attr[                                                                    # Minimumizing the label sizes for them \n",
    "        'fontsize'] = 1                                                           # not to affect the nodes' sizes\n",
    "\n",
    "G.layout(prog='fdp')                                                              # Layout: a force-directed graph\n",
    "\n",
    "graph_width = int(G.graph_attr['bb'].split(',')[2])                               # The width and height of a graph\n",
    "graph_height = int(G.graph_attr['bb'].split(',')[3])                              # to fit its elements on a page"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "597c0f3d",
   "metadata": {},
   "source": [
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">From PyGraphviz to Cytoscape layout</span><a class=\"anchor\" id=\"3\"></a>\n",
    "\n",
    "[UP](#contents)\n",
    "<br>\n",
    "<br>\n",
    "Now I'll extract the PyGraphviz coordinates and add to them all the attributes needed to build the network chart in Dash Cytoscape. \n",
    "\n",
    "#### NODE COORDINATES\n",
    "\n",
    "To draw the nodes, we just extract their coordinates and diameters from PyGraphviz."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1df627c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "node_dict = dict()\n",
    "\n",
    "for node in G.nodes():\n",
    "    node_dict[node] = dict()\n",
    "    node_dict[node]['x'] = float(node.attr['pos'].split(',')[0])\n",
    "    node_dict[node]['y'] = float(node.attr['pos'].split(',')[1])\n",
    "    node_dict[node]['diameter'] = float(node.attr['width']) * 63.5\n",
    "\n",
    "node_data = pd.DataFrame.from_dict(node_dict, orient='index')\n",
    "\n",
    "for dataset in [reporters, partners]:                                       # Comtrade country codes: int -> str\n",
    "    dataset['id'] = [\n",
    "        '00' + x if len(x) == 1 else '0' + x if len(x) == 2 else x\n",
    "        for x in dataset['id'].tolist()\n",
    "    ]\n",
    "\n",
    "country_codes = reporters.set_index('id')['text'].to_dict()\n",
    "country_codes.update(partners.set_index('id')['text'].to_dict())\n",
    "country_codes['380'] = 'Italy'\n",
    "\n",
    "node_data['country'] = node_data.index.map(country_codes)                   # defining the country/territory for each node"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06544566",
   "metadata": {},
   "source": [
    "#### COLORING PARAMETERS\n",
    "\n",
    "The graph might be colored by one of the next parameters:\n",
    "\n",
    "- the region each country or territory belongs to;\n",
    "- the balance between exports and imports;\n",
    "- each country's or territory's rank inside the network (PageRank).\n",
    "\n",
    "Here we'll add all the necessary parameters to the nodes dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "f29cb24d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# UN Region\n",
    "\n",
    "ccode_to_region_dict = dict()                                                  # Defining world regions: \n",
    "                                                                               # country code -> UN Region dictionary\n",
    "regions = [\n",
    "    'Africa', 'Oceania', 'Antarctica', 'Americas', 'Asia', 'Europe',\n",
    "    'Special categories and unspecified areas'\n",
    "]\n",
    "\n",
    "ccode_lists = [\n",
    "    [\n",
    "        '012', '024', '072', '086', '108', '120', '132', '140', '148', '174', \n",
    "        '175', '178', '180', '204', '226', '231', '232', '260', '262', '266', \n",
    "        '270', '288', '324', '384', '404', '426', '430', '434', '450', '454',\n",
    "        '466', '478', '480', '504', '508', '516', '562', '566', '577', '624',\n",
    "        '638', '646', '654', '678', '686', '690', '694', '706', '710', '716', \n",
    "        '728', '729', '732', '736', '748', '768', '788', '800', '818', '834', \n",
    "        '854', '894'\n",
    "    ],\n",
    "    [\n",
    "        '016', '036', '090', '162', '166', '184', '242', '258', '296', '316',\n",
    "        '334', '520', '527', '540', '548', '554', '570', '574', '580', '581', \n",
    "        '583', '584', '585', '598', '612', '772', '776', '798', '876', '882'\n",
    "    ], ['010'], \n",
    "    [\n",
    "        '028', '032', '044', '052', '060', '068', '074', '076', '084', '092',\n",
    "        '124', '136', '152', '170', '188', '192', '212', '214', '218', '222',\n",
    "        '238', '239', '254', '304', '308', '312', '320', '328', '332', '340',\n",
    "        '388', '473', '474', '484', '500', '531', '533', '534', '535', '558', \n",
    "        '591', '600', '604', '630', '636', '637', '652', '659', '660', '662',\n",
    "        '663', '666', '670', '740', '780', '796', '840', '842', '850', '858', \n",
    "        '862'\n",
    "    ], \n",
    "    [\n",
    "        '004', '031', '048', '050', '051', '064', '096', '104', '116', '144',\n",
    "        '156', '196', '268', '275', '344', '356', '360', '364', '368', '376', \n",
    "        '392', '398', '400', '408', '410', '414', '417', '418', '422', '446',\n",
    "        '458', '462', '490', '496', '512', '524', '586', '608', '626', '634', \n",
    "        '682', '699', '702', '704', '760', '762', '764', '784', '792', '795',\n",
    "        '860', '887'\n",
    "    ], \n",
    "    [\n",
    "        '008', '020', '040', '056', '070', '100', '112', '191', '203', '208',\n",
    "        '233', '234', '246', '248', '250', '251', '276', '292', '300', '336', \n",
    "        '348', '352', '372', '380', '428', '438', '440', '442', '470', '492', \n",
    "        '498', '499', '528', '568', '578', '579', '616', '620', '642', '643',\n",
    "        '674', '680', '688', '703', '705', '724', '744', '752', '756', '757',\n",
    "        '804', '807', '826', '831', '832', '833'\n",
    "    ], ['837', '838', '839', '899']\n",
    "]\n",
    "\n",
    "for r, c in zip(regions, ccode_lists):\n",
    "    ccode_to_region_dict.update(dict.fromkeys(c, r))\n",
    "\n",
    "node_data = node_data[node_data.index.isin(                                    # Filtering out undefined codes (EU-28, etc.)\n",
    "    [item for sublist in ccode_lists for item in sublist])]\n",
    "\n",
    "node_data['region'] = node_data.index.map(ccode_to_region_dict)\n",
    "\n",
    "\n",
    "# Export/Import Balance\n",
    "\n",
    "node_data = node_data.join(                                                    # appending some parameters from nodes dataframe\n",
    "    df_nodes[['export', 'import',                                              # to the new dataframe\n",
    "              'export_by_partners', 'import_by_partners']])\n",
    "\n",
    "node_data['export_for_val'] = [\n",
    "    ex if ex != 0 else exp for ex, exp in zip(node_data['export'], \n",
    "                                              node_data['export_by_partners'])\n",
    "]\n",
    "node_data['import_for_val'] = [\n",
    "    im if im != 0 else imp for im, imp in zip(node_data['import'], \n",
    "                                              node_data['import_by_partners']) # Export/import balance in the country's\n",
    "]                                                                              # trade: if export >= 50%, then we qualify \n",
    "node_data['export_share_val'] = node_data['export_for_val'] / node_data[[      # the country as an exporter; otherwise, we   \n",
    "    'import_for_val', 'export_for_val']].sum(axis=1)                           # consider it an importer    \n",
    "                                                                                      \n",
    "node_data['export_share'] = pd.cut(                                            # Coding the values on a scale from 1 to 10\n",
    "    node_data['export_share_val'], list(np.linspace(0, 1, 11)),                \n",
    "    labels=['0' + str(n) if len(str(n)) == 1 else str(n)                       \n",
    "            for n in np.arange(1, 11, 1)], include_lowest=True)               \n",
    "                   \n",
    "    \n",
    "# Pagerank\n",
    "\n",
    "node_data['pagerank_val'] = node_data.index.map(                               # Country's network rank, aka Google PageRank,\n",
    "    nx.pagerank(nx.from_pandas_edgelist(df_links, 'source', 'target')))        # based on the number of links\n",
    "\n",
    "node_data['pagerank'] = pd.cut(                                                # Coding the values on a scale from 1 to 10.\n",
    "    node_data['pagerank_val'], list(np.linspace(\n",
    "        node_data['pagerank_val'].min(),\n",
    "        node_data['pagerank_val'].max(),\n",
    "        11)),\n",
    "    labels=['0' + str(n) if len(str(n)) == 1 else str(n)\n",
    "            for n in np.arange(1, 11, 1)], include_lowest=True)\n",
    "\n",
    "node_data['trade_links'] = node_data.index.map(link_dict)                      # Trade links number we'll use to decide\n",
    "                                                                               # which nodes to label\n",
    "node_data = node_data.drop(['export_for_val',\n",
    "                            'import_for_val', \n",
    "                            'export_share_val',\n",
    "                            'pagerank_val'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "786e735a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>diameter</th>\n",
       "      <th>country</th>\n",
       "      <th>region</th>\n",
       "      <th>export</th>\n",
       "      <th>import</th>\n",
       "      <th>export_by_partners</th>\n",
       "      <th>import_by_partners</th>\n",
       "      <th>export_share</th>\n",
       "      <th>pagerank</th>\n",
       "      <th>trade_links</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>666.44</td>\n",
       "      <td>3189.5</td>\n",
       "      <td>263.702800</td>\n",
       "      <td>Kazakhstan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>1.920362e+09</td>\n",
       "      <td>3.315042e+08</td>\n",
       "      <td>1.416165e+09</td>\n",
       "      <td>5.870542e+05</td>\n",
       "      <td>09</td>\n",
       "      <td>03</td>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>004</th>\n",
       "      <td>1616.20</td>\n",
       "      <td>3033.5</td>\n",
       "      <td>80.257650</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>2.081999e+08</td>\n",
       "      <td>01</td>\n",
       "      <td>01</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>031</th>\n",
       "      <td>873.57</td>\n",
       "      <td>3575.0</td>\n",
       "      <td>115.531900</td>\n",
       "      <td>Azerbaijan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>1.575000e+04</td>\n",
       "      <td>4.365512e+08</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>9.644589e+07</td>\n",
       "      <td>01</td>\n",
       "      <td>01</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>268</th>\n",
       "      <td>929.31</td>\n",
       "      <td>3458.6</td>\n",
       "      <td>44.978955</td>\n",
       "      <td>Georgia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>1.934000e+01</td>\n",
       "      <td>6.490595e+07</td>\n",
       "      <td>1.062091e+06</td>\n",
       "      <td>2.490408e+06</td>\n",
       "      <td>01</td>\n",
       "      <td>01</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>364</th>\n",
       "      <td>346.28</td>\n",
       "      <td>2944.4</td>\n",
       "      <td>141.109700</td>\n",
       "      <td>Iran</td>\n",
       "      <td>Asia</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>6.286929e+03</td>\n",
       "      <td>6.494414e+08</td>\n",
       "      <td>01</td>\n",
       "      <td>01</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           x       y    diameter      country region        export  \\\n",
       "398   666.44  3189.5  263.702800   Kazakhstan   Asia  1.920362e+09   \n",
       "004  1616.20  3033.5   80.257650  Afghanistan   Asia  0.000000e+00   \n",
       "031   873.57  3575.0  115.531900   Azerbaijan   Asia  1.575000e+04   \n",
       "268   929.31  3458.6   44.978955      Georgia   Asia  1.934000e+01   \n",
       "364   346.28  2944.4  141.109700         Iran   Asia  0.000000e+00   \n",
       "\n",
       "           import  export_by_partners  import_by_partners export_share  \\\n",
       "398  3.315042e+08        1.416165e+09        5.870542e+05           09   \n",
       "004  0.000000e+00        0.000000e+00        2.081999e+08           01   \n",
       "031  4.365512e+08        0.000000e+00        9.644589e+07           01   \n",
       "268  6.490595e+07        1.062091e+06        2.490408e+06           01   \n",
       "364  0.000000e+00        6.286929e+03        6.494414e+08           01   \n",
       "\n",
       "    pagerank  trade_links  \n",
       "398       03           36  \n",
       "004       01            5  \n",
       "031       01            6  \n",
       "268       01           10  \n",
       "364       01           11  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "node_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62259825",
   "metadata": {},
   "source": [
    "#### LINKS\n",
    "\n",
    "Link coordinates are not needed in CytoScape; you just need to extract the source and target ID of each link."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "cc465a26",
   "metadata": {},
   "outputs": [],
   "source": [
    "sources = []\n",
    "targets = []\n",
    "\n",
    "for edge in G.edges():\n",
    "    sources.append(edge[0])\n",
    "    targets.append(edge[1])\n",
    "\n",
    "edge_data = pd.DataFrame([sources, targets]).T\n",
    "edge_data.columns = ['source', 'target']\n",
    "\n",
    "for side in ['source', 'target']:                                    # Source and target parameters we take from the node data\n",
    "    edge_data['country_' + side] = edge_data[side].map(\n",
    "        node_data['country'].drop_duplicates().to_dict())\n",
    "\n",
    "for parameter in ['region', 'export_share', 'pagerank']:\n",
    "    for side in ['source', 'target']:\n",
    "        edge_data[parameter + '_' + side] = edge_data[side].map(\n",
    "            node_data[parameter].to_dict())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "95725296",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>target</th>\n",
       "      <th>country_source</th>\n",
       "      <th>country_target</th>\n",
       "      <th>region_source</th>\n",
       "      <th>region_target</th>\n",
       "      <th>export_share_source</th>\n",
       "      <th>export_share_target</th>\n",
       "      <th>pagerank_source</th>\n",
       "      <th>pagerank_target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>398</td>\n",
       "      <td>004</td>\n",
       "      <td>Kazakhstan</td>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>09</td>\n",
       "      <td>01</td>\n",
       "      <td>03</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>398</td>\n",
       "      <td>031</td>\n",
       "      <td>Kazakhstan</td>\n",
       "      <td>Azerbaijan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>09</td>\n",
       "      <td>01</td>\n",
       "      <td>03</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>398</td>\n",
       "      <td>268</td>\n",
       "      <td>Kazakhstan</td>\n",
       "      <td>Georgia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>09</td>\n",
       "      <td>01</td>\n",
       "      <td>03</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>398</td>\n",
       "      <td>364</td>\n",
       "      <td>Kazakhstan</td>\n",
       "      <td>Iran</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>09</td>\n",
       "      <td>01</td>\n",
       "      <td>03</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>398</td>\n",
       "      <td>417</td>\n",
       "      <td>Kazakhstan</td>\n",
       "      <td>Kyrgyzstan</td>\n",
       "      <td>Asia</td>\n",
       "      <td>Asia</td>\n",
       "      <td>09</td>\n",
       "      <td>01</td>\n",
       "      <td>03</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  source target country_source country_target region_source region_target  \\\n",
       "0    398    004     Kazakhstan    Afghanistan          Asia          Asia   \n",
       "1    398    031     Kazakhstan     Azerbaijan          Asia          Asia   \n",
       "2    398    268     Kazakhstan        Georgia          Asia          Asia   \n",
       "3    398    364     Kazakhstan           Iran          Asia          Asia   \n",
       "4    398    417     Kazakhstan     Kyrgyzstan          Asia          Asia   \n",
       "\n",
       "  export_share_source export_share_target pagerank_source pagerank_target  \n",
       "0                  09                  01              03              01  \n",
       "1                  09                  01              03              01  \n",
       "2                  09                  01              03              01  \n",
       "3                  09                  01              03              01  \n",
       "4                  09                  01              03              01  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "edge_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a530ce2",
   "metadata": {},
   "source": [
    "#### LABELS\n",
    "\n",
    "Labeling every node would create a mess, so I'll label only the largest ones (25%+ of maximum size) and also those with 50 trade links and more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "21e3ab0b",
   "metadata": {},
   "outputs": [],
   "source": [
    "label_dict = dict()\n",
    "\n",
    "label_dict_short = {                                                              # Some country names should be shortened\n",
    "    'Bolivia (Plurinational State of)': 'Bolivia',                                # for labeling\n",
    "    'Brunei Darussalam': 'Brunei',\n",
    "    'Central African Rep.': 'CAR',\n",
    "    'China, Hong Kong SAR': 'Hong Kong',\n",
    "    'China, Macao SAR': 'Macao',\n",
    "    \"Dem. People's Rep. of Korea\": 'North Korea',\n",
    "    'Dem. Rep. of the Congo': 'Congo-Kinshasa',\n",
    "    'Falkland Isds (Malvinas)': 'Falkland Isds',\n",
    "    'Holy See (Vatican City State)': 'Holy See',\n",
    "    \"Lao People's Dem. Rep.\": 'Laos',\n",
    "    'Neth. Antilles and Aruba': 'Neth. Antilles',\n",
    "    'Rep. of Korea': 'Korea',\n",
    "    'Rep. of Moldova': 'Moldova',\n",
    "    'Russian Federation': 'Russia',\n",
    "    'Saint Barthelemy': 'St Barthelemy',\n",
    "    'Saint Helena': 'St Helena',\n",
    "    'Saint Lucia': 'St Lucia',\n",
    "    'Saint Maarten': 'St Maarten',\n",
    "    'Saint Kitts and Nevis': 'St Kitts and Nevis',\n",
    "    'Saint Kitts, Nevis and Anguilla': 'St Kitts and Nevis',\n",
    "    'Saint Pierre and Miquelon': 'St Pierre and Miquelon',\n",
    "    'Saint Vincent and the Grenadines': 'St Vincent',\n",
    "    'So. African Customs Union': 'SACU',\n",
    "    'United Arab Emirates': 'UAE',\n",
    "    'United Rep. of Tanzania': 'Tanzania',\n",
    "    'USA (before 1981)': 'USA',\n",
    "    'Africa CAMEU region, nes': 'Africa CAMEU, nes',\n",
    "    'Br. Antarctic Terr.': 'Br. Antarctic',\n",
    "    'Br. Indian Ocean Terr.': 'Br. Indian Ocean',\n",
    "    'Fr. South Antarctic Terr.': 'Fr. South Antarctic',\n",
    "    'Heard Island and McDonald Islands': 'Heard and McDonald Isds',\n",
    "    'North America and Central America, nes': 'North and Centr America, nes',\n",
    "    'South Georgia and the South Sandwich Islands': 'SGSSI',\n",
    "    'United States Minor Outlying Islands': 'US Minor Outlying Isds'\n",
    "}\n",
    "\n",
    "for n in node_data['country'].sort_values().unique().tolist():\n",
    "    if len(n) > 10 and n in label_dict_short.keys():\n",
    "        label_dict[n] = label_dict_short[n]\n",
    "    else:\n",
    "        label_dict[n] = n\n",
    "        \n",
    "nlinks_filter = []                                                                # Countries with 50+ trade links\n",
    "\n",
    "for index in node_data.index.unique().tolist():\n",
    "    if node_data.loc[index]['trade_links'] >= 50:\n",
    "        nlinks_filter.append(index)\n",
    "        \n",
    "max_diameter = node_data['diameter'].max()                                        # Maximum node size\n",
    "\n",
    "node_data['label'] = [                                                            # Filtering the labels\n",
    "    label_dict[country] if diameter >= max_diameter * 0.25\n",
    "    or index in nlinks_filter else np.nan for country, diameter, index in zip(\n",
    "        node_data['country'], node_data['diameter'], node_data.index)\n",
    "]\n",
    "\n",
    "node_data['label_selected'] = [                                                   # A column with all the labels\n",
    "        label_dict[country] for country in node_data['country']\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e040899",
   "metadata": {},
   "source": [
    "#### COLORING\n",
    "\n",
    "In the graph, we need to color four groups of elements:\n",
    "* nodes\n",
    "* nodes' edges\n",
    "* nodes' labels\n",
    "* links\n",
    "\n",
    "Besides that, I will provide an opportunity to select a specific country, so that the country will be brightened while all the other elements will be shadowed.\n",
    "\n",
    "We'll color the links with a gradient to show the category change: to do this, we pass multiple HEX codes as a single string to Cytoscape.\n",
    "\n",
    "I'll set the palette for the nodes' edges and generate other elements' colors out of it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "0481efa9",
   "metadata": {},
   "outputs": [],
   "source": [
    "region_color_dict = {\n",
    "    'Antarctica': '#7E6EBD',\n",
    "    'Africa': '#FB9038',\n",
    "    'Asia': '#D1085C',\n",
    "    'Europe': '#08B0D1',\n",
    "    'Americas': '#d4e2e8',\n",
    "    'Oceania': '#134DD1',\n",
    "    'Special categories and unspecified areas': '#F9F871'\n",
    "}\n",
    "\n",
    "export_share_color_dict = {\n",
    "    '01': '#08b4d1',\n",
    "    '02': '#39c3da',\n",
    "    '03': '#6bd2e3',\n",
    "    '04': '#9ce1ed',\n",
    "    '05': '#cef0f6',\n",
    "    '06': '#ffe5cd',\n",
    "    '07': '#ffcc9b',\n",
    "    '08': '#ffb268',\n",
    "    '09': '#ff9936',\n",
    "    '10': '#ff7f04'\n",
    "}\n",
    "\n",
    "pagerank_color_dict = {\n",
    "    '01': '#3e26a8',\n",
    "    '02': '#4746eb',\n",
    "    '03': '#3e70ff',\n",
    "    '04': '#2797eb',\n",
    "    '05': '#08b4d1',\n",
    "    '06': '#32c69f',\n",
    "    '07': '#81cc59',\n",
    "    '08': '#dbbd28',\n",
    "    '09': '#fcd030',\n",
    "    '10': '#f9fb15'\n",
    "}\n",
    "\n",
    "background_color = '#010103'\n",
    "\n",
    "def rgb_to_hex(rgb):\n",
    "    r = rgb[0]\n",
    "    g = rgb[1]\n",
    "    b = rgb[2]\n",
    "    return '#{:02x}{:02x}{:02x}'.format(r, g, b)\n",
    "\n",
    "def hex_gradient_list(hex_color1, hex_color2, n_colors):\n",
    "\n",
    "    assert n_colors > 1\n",
    "    color1_rgb = np.array(hex_to_rgb(hex_color1)) / 255\n",
    "    color2_rgb = np.array(hex_to_rgb(hex_color2)) / 255\n",
    "    ordered = np.linspace(0, 1, n_colors)\n",
    "    gradient = [list(((1 - order) * color1_rgb + (order * color2_rgb)))\n",
    "                for order in ordered]\n",
    "    gradient_transformed = [[int(round(val * 255)) for val in color] for color in gradient]\n",
    "    return [rgb_to_hex(color) for color in gradient_transformed]\n",
    "\n",
    "def hex_gradient_str(hex_color1, hex_color2, n_colors):\n",
    "\n",
    "    assert n_colors > 1\n",
    "    color1_rgb = np.array(hex_to_rgb(hex_color1)) / 255\n",
    "    color2_rgb = np.array(hex_to_rgb(hex_color2)) / 255\n",
    "    ordered = np.linspace(0, 1, n_colors)\n",
    "    gradient = [list(((1 - order) * color1_rgb + (order * color2_rgb)))\n",
    "                for order in ordered]\n",
    "    gradient_transformed = [[int(round(val * 255)) for val in color] for color in gradient]\n",
    "    return \" \".join([rgb_to_hex(color) for color in gradient_transformed])\n",
    "\n",
    "\n",
    "# Node Edges\n",
    "\n",
    "node_data['region_edge_color'] = node_data['region'].map(region_color_dict)\n",
    "node_data['export_share_edge_color'] = node_data['export_share'].map(export_share_color_dict)\n",
    "node_data['pagerank_edge_color'] = node_data['pagerank'].map(pagerank_color_dict)\n",
    "\n",
    "# Nodes\n",
    "\n",
    "node_data['region_color'] = [\n",
    "    hex_gradient_list(color, background_color, 3)[1] for color in node_data['region_edge_color']\n",
    "]\n",
    "node_data['export_share_color'] = [\n",
    "    hex_gradient_list(color, background_color, 3)[1] for color in node_data['export_share_edge_color']\n",
    "]\n",
    "node_data['pagerank_color'] = [\n",
    "    hex_gradient_list(color, background_color, 3)[1] for color in node_data['pagerank_edge_color']\n",
    "]\n",
    "\n",
    "# Chosen Nodes\n",
    "\n",
    "node_data['chosen_region_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 7)[3] for color in node_data['region_color']\n",
    "    ]\n",
    "\n",
    "node_data['chosen_export_share_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 7)[3] for color in node_data['export_share_color']\n",
    "    ]\n",
    "\n",
    "node_data['chosen_pagerank_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 7)[3] for color in node_data['pagerank_color']\n",
    "    ]\n",
    "\n",
    "# Node Labels\n",
    "\n",
    "node_data['region_label_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 5)[3] for color in node_data['region_edge_color']\n",
    "]\n",
    "node_data['export_share_label_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 5)[3] for color in node_data['export_share_edge_color']\n",
    "]\n",
    "node_data['pagerank_label_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 5)[3] for color in node_data['pagerank_edge_color']\n",
    "]\n",
    "\n",
    "# Chosen Node Labels\n",
    "\n",
    "node_data['chosen_region_label_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 6)[5] for color in node_data['region_edge_color']\n",
    "]\n",
    "node_data['chosen_export_share_label_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 6)[5] for color in node_data['export_share_edge_color']\n",
    "]\n",
    "node_data['chosen_pagerank_label_color'] = [\n",
    "    hex_gradient_list(color, '#ffffff', 6)[5] for color in node_data['pagerank_edge_color']\n",
    "]\n",
    "\n",
    "# Edges\n",
    "\n",
    "for parameter, d in zip(['region', 'export_share', 'pagerank'],\n",
    "                        [region_color_dict, export_share_color_dict, pagerank_color_dict]):\n",
    "    for side in ['source', 'target']:\n",
    "        edge_data[parameter + '_color_' + side] = edge_data[parameter + '_' + side].map(d)\n",
    "        \n",
    "for parameter in ['region', 'export_share', 'pagerank']:\n",
    "    edge_data[parameter + '_colors_source'] = [\n",
    "        hex_gradient_list(background_color, color, 11)[2] for color in edge_data[parameter + '_color_source']\n",
    "    ]\n",
    "    edge_data[parameter + '_colors_target'] = [\n",
    "        hex_gradient_list(background_color, color, 11)[7] for color in edge_data[parameter + '_color_target']\n",
    "    ]\n",
    "\n",
    "for parameter in ['region', 'export_share', 'pagerank']:\n",
    "    edge_data[parameter + '_colors'] = [\n",
    "        hex_gradient_str(color_source, color_target, 10) for color_source, color_target in zip(\n",
    "            edge_data[parameter + '_colors_source'], edge_data[parameter + '_colors_target'])\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1590dc0",
   "metadata": {},
   "source": [
    "<br>\n",
    "<span style=\"font-size:20px;color:#ffffff;font-weight:bold;background-color:#000000\">Building the graph</span><a class=\"anchor\" id=\"4\"></a>\n",
    "\n",
    "[UP](#contents)\n",
    "<br>\n",
    "\n",
    "#### ELEMENTS\n",
    "\n",
    "Here we create dictionaries with all the parameters and node positions, which we will then pass to Cytoscape. \n",
    "\n",
    "I will load the codes into the dictionary to color the graph by the number of trade links."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "674c2814",
   "metadata": {},
   "outputs": [],
   "source": [
    "elements = []\n",
    "\n",
    "# Node parameters and positions\n",
    "\n",
    "for i in node_data.index:\n",
    "    el_dict = dict()\n",
    "    el_dict['data'] = dict()\n",
    "    el_dict['position'] = dict()\n",
    "    \n",
    "    el_dict['data']['id'] = i\n",
    "    el_dict['data']['label'] = node_data.loc[i]['label']\n",
    "    el_dict['data']['label_selected'] = node_data.loc[i]['label_selected']\n",
    "    el_dict['data']['color'] = node_data.loc[i]['pagerank_label_color']\n",
    "    el_dict['data']['color_chosen'] = node_data.loc[i]['chosen_pagerank_label_color']\n",
    "    el_dict['data']['background_color'] = hex_gradient_list(node_data.loc[i]['pagerank_color'], background_color, 4)[2] + ' ' + node_data.loc[i]['pagerank_color'] + ' ' + node_data.loc[i]['pagerank_edge_color']\n",
    "    el_dict['data']['background_color_chosen'] = hex_gradient_list(node_data.loc[i]['chosen_pagerank_color'], background_color, 4)[2] + ' ' + node_data.loc[i]['pagerank_edge_color'] + ' ' + node_data.loc[i]['pagerank_edge_color']\n",
    "    el_dict['data']['border_color'] = node_data.loc[i]['pagerank_edge_color']\n",
    "    el_dict['data']['size'] = node_data.loc[i]['diameter']\n",
    "    el_dict['data']['opacity'] = 0.75   \n",
    "    el_dict['data']['font_size'] = 40\n",
    "    \n",
    "    el_dict['position']['x'] = node_data.loc[i]['x']\n",
    "    el_dict['position']['y'] = node_data.loc[i]['y']\n",
    "    \n",
    "    elements.append(el_dict)\n",
    "\n",
    "# Edge parameters\n",
    "\n",
    "for i in edge_data[['source', 'target']].drop_duplicates().index:\n",
    "    el_dict_e = dict()\n",
    "    el_dict_e['data'] = dict()\n",
    "    el_dict_e['data']['id'] = edge_data.loc[i]['source'] + '_' + edge_data.loc[i]['target']\n",
    "    el_dict_e['data']['source'] = edge_data.loc[i]['source']\n",
    "    el_dict_e['data']['target'] = edge_data.loc[i]['target']\n",
    "    el_dict_e['data']['colors'] = edge_data.loc[i]['pagerank_colors']\n",
    "    elements.append(el_dict_e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a329a3e",
   "metadata": {},
   "source": [
    "#### FINAL LAYOUT\n",
    "\n",
    "Here we create the final network. At the top we will have a small panel that will indicate the country the mouse is hovering over, the country the user clicked on, and a button to download the graph in .png format.\n",
    "\n",
    "We need to set the default network stylesheet, as well as styles for the HTML elements of our page (buttons, text, etc.).\n",
    "\n",
    "There are three callbacks for a page:\n",
    "- display the name of the country over which you hovered the cursor;\n",
    "- show the selected country with all trading partners and display its name;\n",
    "- save the network as an image by clicking the button."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c6b0e01a",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"100%\"\n",
       "            height=\"650\"\n",
       "            src=\"http://127.0.0.1:8050/\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x17f7d38b430>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "app = Dash(__name__)\n",
    "\n",
    "default_stylesheet = [{                                                         # Default network attributes\n",
    "    'selector': 'node',\n",
    "    'style': {\n",
    "        'label': 'data(label)',\n",
    "        \"background-fill\": \"radial-gradient\",\n",
    "        \"background-gradient-stop-colors\": 'data(background_color)',\n",
    "        \"background-gradient-stop-positions\": '0, 80, 90, 100',\n",
    "        'color': 'data(color)',\n",
    "        'text-valign': 'center',\n",
    "        'text-halign': 'center',\n",
    "        'font-size': 'data(font_size)',\n",
    "        'border-color': 'data(border_color)',\n",
    "        'border-width': 1.5,\n",
    "        \"border-opacity\": 1,\n",
    "        'width': 'data(size)',\n",
    "        'height': 'data(size)',\n",
    "        'opacity': 0.98\n",
    "    }\n",
    "}, {\n",
    "    'selector': 'edge',\n",
    "    'style': {\n",
    "        \"line-fill\": \"linear-gradient\",\n",
    "        \"line-gradient-stop-colors\": 'data(colors)',\n",
    "        \"line-gradient-stop-positions\": \"10, 20, 30, 40, 50, 60, 70, 80, 90\",\n",
    "        'width': 2.5,\n",
    "        'curve-style': 'bezier',\n",
    "        'source-endpoint': 'outside-to-node',\n",
    "        'target-endpoint': 'outside-to-node'\n",
    "    }\n",
    "}]\n",
    "\n",
    "styles = {                                                                      # HTML Styling\n",
    "    'Lab': {\n",
    "        'height': '100%',\n",
    "        'color': '#ffffff',\n",
    "        'font-family': 'Courier New',\n",
    "        'font-size': '80%',\n",
    "        'padding-top': '10%',\n",
    "        'padding-bottom': '10%',\n",
    "        'padding-left': '10%'\n",
    "    },\n",
    "    'Output': {\n",
    "        'height': '100%',\n",
    "        'color': '#08B0D1',\n",
    "        'font-family': 'Courier New',\n",
    "        'font-size': '80%',\n",
    "        'padding-top': '16%',\n",
    "        'padding-bottom': '10%',\n",
    "        'padding-left': '10%'\n",
    "    },\n",
    "    'Button': {\n",
    "        'height': '100%',\n",
    "        'width': '100%',\n",
    "        'color': '#ffffff',\n",
    "        'background-color': '#010103',\n",
    "        'border-color': '#010103',\n",
    "        'border-width': 0,\n",
    "        'padding-left': '20%',\n",
    "        'font-family': 'Courier New',\n",
    "        'font-size': '80%',\n",
    "        'text-align': 'left',\n",
    "        'cursor': 'pointer'\n",
    "    }\n",
    "}\n",
    "\n",
    "app.layout = dbc.Container([                                                    # Page structure                                           \n",
    "    html.Div([\n",
    "        html.Div([\n",
    "            html.Div(html.P('Country Hovered:', style=styles['Lab']), \n",
    "                     style={'width': '15%', \n",
    "                            'height': '100%',\n",
    "                            'margin-left': '2%'}),\n",
    "            html.Div(html.Div(id='mouseoverNodeData',\n",
    "                              style=styles['Output']), \n",
    "                     style={'width': '25%'}),\n",
    "            html.Div(html.P('Country Highlighted:', \n",
    "                            style=styles['Lab']),\n",
    "                     style={'width': '15%', \n",
    "                            'height': '100%', \n",
    "                            'margin-left': '2%'}),\n",
    "            html.Div(html.Div(id='output-country', \n",
    "                              style=styles['Output']), \n",
    "                     style={'width': '25%'}),\n",
    "            html.Div(html.Button(\"Save PNG\", id=\"btn-get-png\",\n",
    "                                 style=styles['Button']), \n",
    "                     style={'width': '20%', \n",
    "                            'background-color': '#010103', \n",
    "                            'margin-left': '2%'})\n",
    "        ], style={'width': '100%', 'display': 'flex', 'background-color': '#010103'}),\n",
    "        html.Div([\n",
    "            cyto.Cytoscape(\n",
    "                id='cytoscape',\n",
    "                layout={'name': 'cola'},\n",
    "                style={\n",
    "                    'width': '800px',\n",
    "                    'height': str(graph_height*800/graph_width) + 'px', \n",
    "                    'background-color': '#010103'},\n",
    "                elements=elements,\n",
    "                stylesheet=default_stylesheet)\n",
    "        ], style={'width': '100%', 'background-color': '#010103'})\n",
    "    ], style={'display': 'inline-block', 'width': '800px'})\n",
    "], style={'display': 'flex', \n",
    "          'justify-content': 'center',\n",
    "          'margin-bottom': '50px'})\n",
    "   \n",
    "    \n",
    "@app.callback(                                                                  # User Callbacks\n",
    "    Output('mouseoverNodeData', 'children'),\n",
    "    Input('cytoscape', 'mouseoverNodeData'))\n",
    "def display_hover_data(data):\n",
    "    if data:\n",
    "        return data['label_selected']\n",
    "\n",
    "    \n",
    "@app.callback(\n",
    "    [\n",
    "        Output('cytoscape', 'stylesheet'),\n",
    "        Output('output-country', 'children')\n",
    "    ],\n",
    "    [\n",
    "        Input('cytoscape', 'tapNode'),\n",
    "        Input('cytoscape', 'selectedNodeData')\n",
    "    ]\n",
    ")\n",
    "def generate_stylesheet(node, data_list):\n",
    "\n",
    "    if not data_list:\n",
    "        return default_stylesheet, 'No country selected'\n",
    "\n",
    "    elif node:\n",
    "        node_id = node['data']['id']\n",
    "    \n",
    "        stylesheet = [\n",
    "            {\n",
    "                \"selector\": 'node',\n",
    "                'style': {\n",
    "                    'label': 'data(label)',\n",
    "                    \"background-fill\": \"radial-gradient\",\n",
    "                    \"background-gradient-stop-colors\": 'data(background_color)',\n",
    "                    \"background-gradient-stop-positions\": \"0, 80, 90, 100\",\n",
    "                    'color': 'data(color)',\n",
    "                    'text-valign': 'center',\n",
    "                    'text-halign': 'center',\n",
    "                    'font-size': 'data(font_size)',\n",
    "                    'border-color': 'data(border_color)',\n",
    "                    'border-width': 1.5,\n",
    "                    \"border-opacity\": 1,\n",
    "                    'width': 'data(size)',\n",
    "                    'height': 'data(size)',\n",
    "                    'opacity': 0.3\n",
    "                }\n",
    "            }, {\n",
    "                'selector': 'edge',\n",
    "                'style': {\n",
    "                    \"line-fill\": \"linear-gradient\",\n",
    "                    \"line-gradient-stop-colors\": 'data(colors)',\n",
    "                    \"line-gradient-stop-positions\": \"10, 20, 30, 40, 50, 60, 70, 80, 90\",\n",
    "                    'width': 2,\n",
    "                    'curve-style': 'bezier',\n",
    "                    'source-endpoint': 'outside-to-node',\n",
    "                    'target-endpoint': 'outside-to-node',\n",
    "                    'opacity': 0.2\n",
    "                }\n",
    "            }, {\n",
    "                \"selector\": 'node[id = \"{}\"]'.format(node_id),\n",
    "                \"style\": {\n",
    "                    'label': 'data(label_selected)',\n",
    "                    \"background-fill\": \"radial-gradient\",\n",
    "                    \"background-gradient-stop-colors\": 'data(background_color_chosen)',\n",
    "                    \"background-gradient-stop-positions\": \"0, 98, 99, 100\",\n",
    "                    'color': 'data(color_chosen)',\n",
    "                    'text-valign': 'center',\n",
    "                    'text-halign': 'center',\n",
    "                    'font-size': 'data(font_size)',\n",
    "                    'border-color': 'data(border_color)',\n",
    "                    'border-width': 1.5,\n",
    "                    \"border-opacity\": 1,\n",
    "                    'width': 'data(size)',\n",
    "                    'height': 'data(size)',\n",
    "                    'opacity': 0.98,\n",
    "                    'z-index': 9999\n",
    "                }\n",
    "            }\n",
    "        ]\n",
    "\n",
    "        for edge in node[\"edgesData\"]:\n",
    "            if edge['source'] == node_id:\n",
    "                stylesheet.append(\n",
    "                    {\n",
    "                        \"selector\": 'node[id = \"{}\"]'.format(edge['target']),\n",
    "                        \"style\": {\n",
    "                            'label': 'data(label)',\n",
    "                            \"background-fill\": \"radial-gradient\",\n",
    "                            \"background-gradient-stop-colors\": 'data(background_color)',\n",
    "                            \"background-gradient-stop-positions\": \"0, 80, 90, 100\",\n",
    "                            'color': 'data(color)',\n",
    "                            'text-valign': 'center',\n",
    "                            'text-halign': 'center',\n",
    "                            'font-size': 'data(font_size)',\n",
    "                            'border-color': 'data(border_color)',\n",
    "                            'border-width': 1.5,\n",
    "                            \"border-opacity\": 1,\n",
    "                            'width': 'data(size)',\n",
    "                            'height': 'data(size)',\n",
    "                            'opacity': 0.98\n",
    "                        }\n",
    "                    })\n",
    "                stylesheet.append(\n",
    "                    {\n",
    "                        \"selector\": 'edge[id= \"{}\"]'.format(edge['id']),\n",
    "                        \"style\": {\n",
    "                            \"line-fill\": \"linear-gradient\",\n",
    "                            \"line-gradient-stop-colors\": 'data(colors)',\n",
    "                            \"line-gradient-stop-positions\": \"10, 20, 30, 40, 50, 60, 70, 80, 90\",\n",
    "                            'width': 7,\n",
    "                            'curve-style': 'bezier',\n",
    "                            'source-endpoint': 'outside-to-node',\n",
    "                            'target-endpoint': 'outside-to-node',\n",
    "                            'opacity': 0.98,\n",
    "                            'z-index': 5000\n",
    "                        }\n",
    "                    })\n",
    "\n",
    "            if edge['target'] == node_id:\n",
    "                stylesheet.append(\n",
    "                    {\n",
    "                        \"selector\": 'node[id = \"{}\"]'.format(edge['source']),\n",
    "                        \"style\": {\n",
    "                            'label': 'data(label)',\n",
    "                            \"background-fill\": \"radial-gradient\",\n",
    "                            \"background-gradient-stop-colors\": 'data(background_color)',\n",
    "                            \"background-gradient-stop-positions\": \"0, 80, 90, 100\",\n",
    "                            'color': 'data(color)',\n",
    "                            'text-valign': 'center',\n",
    "                            'text-halign': 'center',\n",
    "                            'font-size': 'data(font_size)',\n",
    "                            'border-color': 'data(border_color)',\n",
    "                            'border-width': 1.5,\n",
    "                            \"border-opacity\": 1,\n",
    "                            'width': 'data(size)',\n",
    "                            'height': 'data(size)',\n",
    "                            'opacity': 0.98,\n",
    "                            'z-index': 9999\n",
    "                        }\n",
    "                    })\n",
    "                stylesheet.append(\n",
    "                    {\n",
    "                        \"selector\": 'edge[id= \"{}\"]'.format(edge['id']),\n",
    "                        \"style\": {\n",
    "                            \"line-fill\": \"linear-gradient\",\n",
    "                            \"line-gradient-stop-colors\": 'data(colors)',\n",
    "                            \"line-gradient-stop-positions\": \"10, 20, 30, 40, 50, 60, 70, 80, 90\",\n",
    "                            'width': 6,\n",
    "                            'curve-style': 'bezier',\n",
    "                            'source-endpoint': 'outside-to-node',\n",
    "                            'target-endpoint': 'outside-to-node',\n",
    "                            'opacity': 0.98,\n",
    "                            'z-index': 5000\n",
    "                        }\n",
    "                    })\n",
    "\n",
    "        return stylesheet, node['data']['label_selected']\n",
    "\n",
    "    \n",
    "@callback(\n",
    "    Output(\"cytoscape\", \"generateImage\"),\n",
    "    Input(\"btn-get-png\", \"n_clicks\"),\n",
    "    prevent_initial_call=True\n",
    ")\n",
    "def get_image(get_png_clicks):\n",
    "    \n",
    "    if ctx.triggered_id == 'btn-get-png':\n",
    "        now = datetime.now().strftime(\"%Y-%m-%d_%H-%M-%S\")\n",
    "        return {'type': \"png\",\n",
    "                'action': \"download\",\n",
    "                'options': {'bg': '#010103'}, \n",
    "                'filename': f'wheat_and_meslin_{now}'}\n",
    "\n",
    "    \n",
    "if __name__ == \"__main__\":\n",
    "    app.run_server(debug=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f242b3eb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}