notebooks/models/nested_logit.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Nested Logit Model\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/artefactory/choice-learn/blob/main/notebooks/models/nested_logit.ipynb)\n",
    "\n",
    "The Nested Logit model considers sub-groups of alternatives totally substitutables, called 'nests'. The general idea is that a customer might choose its transportation mode between publics transport and its private car. And then, if he decides to use public transportations the customer chooses between taking the train or the bus.\\\n",
    "The classical Conditional Logit does not account for such decision process. Hence the introduction of the Nested Logit. More detailed information are available [here](https://cran.r-project.org/web/packages/mlogit/vignettes/c4.relaxiid.html#:~:text=The%20nested%20logit%20model&text=It%20is%20a%20generalization%20of,different%20nests%20are%20still%20uncorrelated.).\n",
    "\n",
    "\n",
    "In this notebook we reproduce results from other packages showing how to speficy a Nested Logit model with Choice-Learn and that we reach the right results.\n",
    "\n",
    "## Summary\n",
    "- [1] [Nested Logit on the SwissMetro dataset](#1--nested-logit-on-the-swissmetro-dataset)\n",
    "    - [Short Introduction to the Nested Logit model](#short-introduction-to-nested-logit)\n",
    "    - [Specification and estimation with Choice-Learn](#specification-and-estimation-with-choice-learn)\n",
    "- [2] [Nested Logit on the HC dataset](#2--nested-logit-with-the-hc-dataset)\n",
    "    - [First formulation](#first-formulation)\n",
    "    - [Second formulation](#second-formulation)\n",
    "    - [Third formulation](#third-formulation)\n",
    "    - [Fourth formulation](#fourth-formulation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install necessary requirements\n",
    "\n",
    "# If you run this notebook on Google Colab, or in standalone mode, you need to install the required packages.\n",
    "# Uncomment the following lines:\n",
    "\n",
    "# !pip install choice-learn\n",
    "\n",
    "# If you run the notebook within the GitHub repository, you need to run the following lines, that can skipped otherwise:\n",
    "import os\n",
    "import sys\n",
    "\n",
    "sys.path.append(\"../../\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"\"\n",
    "\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the Nested Logit from Choice-Learn !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from choice_learn.models import NestedLogit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1- Nested Logit on the SwissMetro dataset\n",
    "\n",
    "We reproduce the results from [Biogeme](https://biogeme.epfl.ch/sphinx/auto_examples/swissmetro/plot_b09nested.html) that is also reproduced in [PyLogit](https://github.com/timothyb0912/pylogit/blob/master/examples/notebooks/Nested%20Logit%20Example--Python%20Biogeme%20benchmark--09NestedLogit.ipynb).\\\n",
    "This example uses the SwissMetro dataset further described in the [data introduction](../introduction/2_data_handling.ipynb).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "%=====================================================================%\n",
      "%%% Summary of the dataset:\n",
      "%=====================================================================%\n",
      "Number of items: 3\n",
      "Number of choices: 6768\n",
      "%=====================================================================%\n",
      " No Shared Features by Choice registered\n",
      "\n",
      "\n",
      " Items Features by Choice:\n",
      " 2 items features \n",
      " with names: (['cost', 'travel_time'],)\n",
      "%=====================================================================%\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from choice_learn.datasets import load_swissmetro\n",
    "swiss_dataset = load_swissmetro(preprocessing=\"biogeme_nested\")\n",
    "print(swiss_dataset.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Short Introduction to Nested Logit\n",
    "\n",
    "The model specified in Biogeme defines two nests:\n",
    "- The existing modes nest with the train and car *(items indexes of 0 and 2)*\n",
    "- The future modes nest with the swissmetro *(item index of 1)*\n",
    "\n",
    "And the utility form is the following:\\\n",
    "&nbsp; &nbsp; &nbsp; $U(i) = \\beta^{inter}_i + \\beta^{tt} \\cdot TT(i) + \\beta^{co} \\cdot CO(i)$\\\n",
    "with:\n",
    "- $TT(i)$ the travel time of alternative $i$\n",
    "- $CO(i)$ the cost of alternative $i$\n",
    "- $\\beta^{inter}_{sm} = 0$\n",
    "\n",
    "The Nested Logit formulates the following probabilities from such utility:\n",
    "\n",
    "$$\n",
    "\\mathbb{P}(i) = \\frac{e^{\\frac{U(i)}{\\gamma_{nest(i)}}} \\cdot \\left( \\sum_{k \\in nest(i)} e^{\\frac{U(k)}{\\gamma_{nest(i)}}} \\right)^{\\gamma_{nest(i)}-1}}{\\sum_{m \\in \\mathcal{Nests}} \\left( \\sum_{k \\in m} e^{\\frac{U(k)}{\\gamma_{m}}} \\right)^{\\gamma_m}}\n",
    "$$\n",
    "\n",
    "To better understand this expression, one way is to split it into two parts:\n",
    "\n",
    "\n",
    "$$\n",
    "\\mathbb{P}(i) = \\mathbb{P}(nest(i)) \\cdot \\mathbb{P}(i | nest(i))\n",
    "$$\n",
    "\n",
    "with:\\\n",
    "$\n",
    "\\mathbb{P}(i | nest(i)) = \\frac{e^{\\frac{U(i)}{\\gamma_{nest(i)}}} }{\\sum_{k \\in nest(i)} e^{\\frac{U(k)}{\\gamma_{nest(i)}}}}\n",
    "$ &nbsp;\n",
    "and &nbsp;\n",
    "$\n",
    "\\mathbb{P}(nest(i))= \\frac{\\left( \\sum_{k \\in nest(i)} e^{\\frac{U(k)}{\\gamma_{nest(i)}}} \\right)^{\\gamma_{nest(i)}}}{\\sum_{m \\in \\mathcal{Nests}} \\left( \\sum_{k \\in m} e^{\\frac{U(k)}{\\gamma_{m}}} \\right)^{\\gamma_m}}\n",
    "$\n",
    "\n",
    "Therefore we have 4 weights in the utility function and the $\\gamma_{nest}$ values to estimate. The 'new' nest containing only one alternative, its correlation value $\\gamma^{new}$ has no impact, we only need to estimate $\\gamma^{old}$.\n",
    "\n",
    "\n",
    "### Specification and estimation with Choice-Learn\n",
    "\n",
    "With Choice-Learn, the Nested Logit model specification is similar to the [Conditional Logit specification](./../introduction/3_model_clogit.ipynb). The few differences are:\n",
    "- When the model is instantiated, the nested need to be specified as a list of nests with the concerned items indexes. In the example, we specify `items_nests=[[0, 2], [1]]` saying that first nest contains the items of indexes 0 (train) and 2 (car) and the second nest the item of index 1 (swiss metro).\n",
    "- The \"fast\" dict-base specifications has another alternative with `coefficients={feature_name: \"nest\"}` creating for the feature feature_name one coefficient to estimate by nest, this coefficient being shared by all alternatives of the nest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialization of the model\n",
    "swiss_model = NestedLogit(optimizer=\"lbfgs\", items_nests=[[0, 2], [1]])\n",
    "\n",
    "# Intercept for train & sm\n",
    "swiss_model.add_coefficients(feature_name=\"intercept\", items_indexes=[0, 2])\n",
    "\n",
    "# betas TT and CO shared by train and sm\n",
    "swiss_model.add_shared_coefficient(feature_name=\"travel_time\",\n",
    "                                   items_indexes=[0, 1, 2])\n",
    "swiss_model.add_shared_coefficient(feature_name=\"cost\",\n",
    "                                   items_indexes=[0, 1, 2])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.Socket(zmq.PUSH) at 0x7f46435341a0>> and will run it as-is.\n",
      "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
      "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
      "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:At least one gamma value for nests is below 0.05 and is\n",
      "        clipped to 0.05 for numeric optimization purposes.\n",
      "WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.Socket(zmq.PUSH) at 0x7f46435341a0>> and will run it as-is.\n",
      "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
      "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
      "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING: AutoGraph could not transform <bound method Socket.send of <zmq.Socket(zmq.PUSH) at 0x7f46435341a0>> and will run it as-is.\n",
      "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
      "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
      "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:L-BFGS Opimization finished:\n",
      "WARNING:root:---------------------------------------------------------------\n",
      "WARNING:root:Number of iterations: 21\n",
      "WARNING:root:Algorithm converged before reaching max iterations: True\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using L-BFGS optimizer, setting up .fit() function\n"
     ]
    }
   ],
   "source": [
    "# Estimation of the model\n",
    "history = swiss_model.fit(swiss_dataset, get_report=True, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Variable 'beta_intercept:0' shape=(1, 2) dtype=float32, numpy=array([[-0.51194817, -0.1671558 ]], dtype=float32)>,\n",
       " <tf.Variable 'beta_travel_time:0' shape=(1, 1) dtype=float32, numpy=array([[-0.89866394]], dtype=float32)>,\n",
       " <tf.Variable 'beta_cost:0' shape=(1, 1) dtype=float32, numpy=array([[-0.85666543]], dtype=float32)>,\n",
       " <tf.Variable 'gammas_nests:0' shape=(1, 1) dtype=float32, numpy=array([[0.4868395]], dtype=float32)>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swiss_model.trainable_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficient Name</th>\n",
       "      <th>Coefficient Estimation</th>\n",
       "      <th>Std. Err</th>\n",
       "      <th>z_value</th>\n",
       "      <th>P(.&gt;z)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>beta_intercept_0</td>\n",
       "      <td>-0.511948</td>\n",
       "      <td>0.046159</td>\n",
       "      <td>-11.090872</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>beta_intercept_1</td>\n",
       "      <td>-0.167156</td>\n",
       "      <td>0.036682</td>\n",
       "      <td>-4.556830</td>\n",
       "      <td>0.000005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>beta_travel_time</td>\n",
       "      <td>-0.898664</td>\n",
       "      <td>0.054548</td>\n",
       "      <td>-16.474657</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>beta_cost</td>\n",
       "      <td>-0.856665</td>\n",
       "      <td>0.046482</td>\n",
       "      <td>-18.430079</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>gammas_nests</td>\n",
       "      <td>0.486840</td>\n",
       "      <td>0.029635</td>\n",
       "      <td>16.427719</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Coefficient Name  Coefficient Estimation  Std. Err    z_value    P(.>z)\n",
       "0  beta_intercept_0               -0.511948  0.046159 -11.090872  0.000000\n",
       "1  beta_intercept_1               -0.167156  0.036682  -4.556830  0.000005\n",
       "2  beta_travel_time               -0.898664  0.054548 -16.474657  0.000000\n",
       "3         beta_cost               -0.856665  0.046482 -18.430079  0.000000\n",
       "4      gammas_nests                0.486840  0.029635  16.427719  0.000000"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "swiss_model.report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Variable 'beta_intercept:0' shape=(1, 2) dtype=float32, numpy=array([[-0.51194817, -0.1671558 ]], dtype=float32)>,\n",
       " <tf.Variable 'beta_travel_time:0' shape=(1, 1) dtype=float32, numpy=array([[-0.89866394]], dtype=float32)>,\n",
       " <tf.Variable 'beta_cost:0' shape=(1, 1) dtype=float32, numpy=array([[-0.85666543]], dtype=float32)>,\n",
       " <tf.Variable 'gammas_nests:0' shape=(1, 1) dtype=float32, numpy=array([[0.4868395]], dtype=float32)>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Looking at the weights\n",
    "swiss_model.trainable_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=5236.8994>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Estimating the total summed Negative Log-Likelihood\n",
    "swiss_model.evaluate(swiss_dataset) * len(swiss_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[0.15937711 0.6218435  0.21877931]\n",
      " [0.1940201  0.6445151  0.16146483]\n",
      " [0.11813082 0.597691   0.28417817]\n",
      " [0.12110618 0.52606976 0.3528241 ]], shape=(4, 3), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# Probabilities can be easily computed:\n",
    "probas = swiss_model.predict_probas(swiss_dataset)\n",
    "print(probas[:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interpretation and comparison with Biogeme results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2- Nested Logit with the HC Dataset\n",
    "\n",
    "We reproduce results from [mlogit](https://cran.r-project.org/web/packages/mlogit/vignettes/e2nlogit.html) that are also presented in [Torch-Choice](https://gsbdbi.github.io/torch-choice/nested_logit_model_house_cooling/).\n",
    "\n",
    "\n",
    "### First Formulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from choice_learn.datasets import load_hc\n",
    "from choice_learn.data import ChoiceDataset\n",
    "\n",
    "# Loading\n",
    "hc_df = load_hc(as_frame=True)\n",
    "\n",
    "# HC dataset is loaded as a pandas.DataFrame for the example.\n",
    "# It can be downloaded as a ChoiceDataset with the argument `as_frame=False`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rownames</th>\n",
       "      <th>depvar</th>\n",
       "      <th>ich.gcc</th>\n",
       "      <th>ich.ecc</th>\n",
       "      <th>ich.erc</th>\n",
       "      <th>ich.hpc</th>\n",
       "      <th>ich.gc</th>\n",
       "      <th>ich.ec</th>\n",
       "      <th>ich.er</th>\n",
       "      <th>icca</th>\n",
       "      <th>och.gcc</th>\n",
       "      <th>och.ecc</th>\n",
       "      <th>och.erc</th>\n",
       "      <th>och.hpc</th>\n",
       "      <th>och.gc</th>\n",
       "      <th>och.ec</th>\n",
       "      <th>och.er</th>\n",
       "      <th>occa</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>erc</td>\n",
       "      <td>9.70</td>\n",
       "      <td>7.86</td>\n",
       "      <td>8.79</td>\n",
       "      <td>11.36</td>\n",
       "      <td>24.08</td>\n",
       "      <td>24.50</td>\n",
       "      <td>7.37</td>\n",
       "      <td>27.28</td>\n",
       "      <td>2.26</td>\n",
       "      <td>4.09</td>\n",
       "      <td>3.85</td>\n",
       "      <td>1.73</td>\n",
       "      <td>2.26</td>\n",
       "      <td>4.09</td>\n",
       "      <td>3.85</td>\n",
       "      <td>2.95</td>\n",
       "      <td>20.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>hpc</td>\n",
       "      <td>8.77</td>\n",
       "      <td>8.69</td>\n",
       "      <td>7.09</td>\n",
       "      <td>9.37</td>\n",
       "      <td>28.00</td>\n",
       "      <td>32.71</td>\n",
       "      <td>9.33</td>\n",
       "      <td>26.49</td>\n",
       "      <td>2.30</td>\n",
       "      <td>2.69</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1.65</td>\n",
       "      <td>2.30</td>\n",
       "      <td>2.69</td>\n",
       "      <td>3.45</td>\n",
       "      <td>1.63</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>gcc</td>\n",
       "      <td>7.43</td>\n",
       "      <td>8.86</td>\n",
       "      <td>6.94</td>\n",
       "      <td>11.70</td>\n",
       "      <td>25.71</td>\n",
       "      <td>31.68</td>\n",
       "      <td>8.14</td>\n",
       "      <td>22.63</td>\n",
       "      <td>2.28</td>\n",
       "      <td>5.25</td>\n",
       "      <td>4.35</td>\n",
       "      <td>1.44</td>\n",
       "      <td>2.28</td>\n",
       "      <td>5.25</td>\n",
       "      <td>4.35</td>\n",
       "      <td>2.18</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>gcc</td>\n",
       "      <td>9.18</td>\n",
       "      <td>8.93</td>\n",
       "      <td>7.22</td>\n",
       "      <td>12.13</td>\n",
       "      <td>29.72</td>\n",
       "      <td>26.73</td>\n",
       "      <td>8.04</td>\n",
       "      <td>25.33</td>\n",
       "      <td>2.62</td>\n",
       "      <td>4.89</td>\n",
       "      <td>4.85</td>\n",
       "      <td>1.93</td>\n",
       "      <td>2.62</td>\n",
       "      <td>4.89</td>\n",
       "      <td>4.85</td>\n",
       "      <td>2.70</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>gcc</td>\n",
       "      <td>8.05</td>\n",
       "      <td>7.02</td>\n",
       "      <td>8.44</td>\n",
       "      <td>10.51</td>\n",
       "      <td>23.90</td>\n",
       "      <td>28.35</td>\n",
       "      <td>7.15</td>\n",
       "      <td>25.45</td>\n",
       "      <td>2.52</td>\n",
       "      <td>3.71</td>\n",
       "      <td>3.64</td>\n",
       "      <td>1.63</td>\n",
       "      <td>2.52</td>\n",
       "      <td>3.71</td>\n",
       "      <td>3.64</td>\n",
       "      <td>2.77</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   rownames depvar  ich.gcc  ich.ecc  ich.erc  ich.hpc  ich.gc  ich.ec  \\\n",
       "0         1    erc     9.70     7.86     8.79    11.36   24.08   24.50   \n",
       "1         2    hpc     8.77     8.69     7.09     9.37   28.00   32.71   \n",
       "2         3    gcc     7.43     8.86     6.94    11.70   25.71   31.68   \n",
       "3         4    gcc     9.18     8.93     7.22    12.13   29.72   26.73   \n",
       "4         5    gcc     8.05     7.02     8.44    10.51   23.90   28.35   \n",
       "\n",
       "   ich.er   icca  och.gcc  och.ecc  och.erc  och.hpc  och.gc  och.ec  och.er  \\\n",
       "0    7.37  27.28     2.26     4.09     3.85     1.73    2.26    4.09    3.85   \n",
       "1    9.33  26.49     2.30     2.69     3.45     1.65    2.30    2.69    3.45   \n",
       "2    8.14  22.63     2.28     5.25     4.35     1.44    2.28    5.25    4.35   \n",
       "3    8.04  25.33     2.62     4.89     4.85     1.93    2.62    4.89    4.85   \n",
       "4    7.15  25.45     2.52     3.71     3.64     1.63    2.52    3.71    3.64   \n",
       "\n",
       "   occa  income  \n",
       "0  2.95    20.0  \n",
       "1  1.63    50.0  \n",
       "2  2.18    50.0  \n",
       "3  2.70    50.0  \n",
       "4  2.77    60.0  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hc_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to pre-process the dataset like in the examples to 'easily' specify the Nested Logit model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "items_id = [\"gcc\", \"ecc\", \"erc\", \"hpc\", \"gc\", \"ec\", \"er\"]\n",
    "cooling_modes = [\"gcc\", \"ecc\", \"erc\", \"hpc\"]\n",
    "room_modes = [\"erc\", \"er\"]\n",
    "non_cooling_modes = [\"gc\", \"ec\", \"er\"]\n",
    "\n",
    "for mode in items_id:\n",
    "    if mode in cooling_modes:\n",
    "        hc_df[f\"icca.{mode}\"] = hc_df[\"icca\"]\n",
    "        hc_df[f\"occa.{mode}\"] = hc_df[\"occa\"]\n",
    "    else:\n",
    "        hc_df[f\"icca.{mode}\"] = 0.\n",
    "        hc_df[f\"occa.{mode}\"] = 0.\n",
    "\n",
    "for item in items_id:\n",
    "    if item in cooling_modes:\n",
    "        hc_df[f\"int_cooling.{item}\"] = 1.\n",
    "        hc_df[f\"inc_cooling.{item}\"] = hc_df.income\n",
    "    else:\n",
    "        hc_df[f\"int_cooling.{item}\"] = 0.\n",
    "        hc_df[f\"inc_cooling.{item}\"] = 0.\n",
    "    if item in room_modes:\n",
    "        hc_df[f\"inc_room.{item}\"] = hc_df.income\n",
    "    else:\n",
    "        hc_df[f\"inc_room.{item}\"] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating the dataset from this preprocessed dataframe\n",
    "dataset = ChoiceDataset.from_single_wide_df(df=hc_df,\n",
    "                                            items_features_prefixes=[\"ich\", \"och\", \"occa\", \"icca\",\n",
    "                                                                     \"int_cooling\", \"inc_cooling\",\n",
    "                                                                     \"inc_room\"],\n",
    "                                            delimiter=\".\",\n",
    "                                            items_id=items_id,\n",
    "                                            choices_column=\"depvar\",\n",
    "                                            choice_format=\"items_id\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the fast specification using a dictionnary with the 'constant' keyword."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using L-BFGS optimizer, setting up .fit() function\n"
     ]
    }
   ],
   "source": [
    "spec = {\n",
    "    \"ich\": \"constant\",\n",
    "    \"och\": \"constant\",\n",
    "    \"occa\": \"constant\",\n",
    "    \"icca\": \"constant\",\n",
    "    \"int_cooling\":\"constant\",\n",
    "    \"inc_cooling\": \"constant\",\n",
    "    \"inc_room\": \"constant\"\n",
    "}\n",
    "model = NestedLogit(\n",
    "    coefficients=spec,\n",
    "    items_nests=[[0, 1, 2, 3], [4, 5, 6]],\n",
    "    optimizer=\"lbfgs\",\n",
    "    shared_gammas_over_nests=True # Note the argument specifying that all nests have the same gamma value\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:At least one gamma value for nests is below 0.05 and is\n",
      "        clipped to 0.05 for numeric optimization purposes.\n",
      "WARNING:root:L-BFGS Opimization finished:\n",
      "WARNING:root:---------------------------------------------------------------\n",
      "WARNING:root:Number of iterations: 55\n",
      "WARNING:root:Algorithm converged before reaching max iterations: True\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using L-BFGS optimizer, setting up .fit() function\n"
     ]
    }
   ],
   "source": [
    "hist = model.fit(dataset, get_report=True, verbose=1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficient Name</th>\n",
       "      <th>Coefficient Estimation</th>\n",
       "      <th>Std. Err</th>\n",
       "      <th>z_value</th>\n",
       "      <th>P(.&gt;z)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ich_w_0</td>\n",
       "      <td>-0.554876</td>\n",
       "      <td>0.174471</td>\n",
       "      <td>-3.180328</td>\n",
       "      <td>0.001471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>och_w_1</td>\n",
       "      <td>-0.857881</td>\n",
       "      <td>0.300332</td>\n",
       "      <td>-2.856437</td>\n",
       "      <td>0.004284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>occa_w_2</td>\n",
       "      <td>-1.089362</td>\n",
       "      <td>1.056226</td>\n",
       "      <td>-1.031371</td>\n",
       "      <td>0.302367</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>icca_w_3</td>\n",
       "      <td>-0.225069</td>\n",
       "      <td>0.112268</td>\n",
       "      <td>-2.004749</td>\n",
       "      <td>0.044990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>int_cooling_w_4</td>\n",
       "      <td>-6.000777</td>\n",
       "      <td>4.986898</td>\n",
       "      <td>-1.203308</td>\n",
       "      <td>0.228857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>inc_cooling_w_5</td>\n",
       "      <td>0.249571</td>\n",
       "      <td>0.053589</td>\n",
       "      <td>4.657146</td>\n",
       "      <td>0.000003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>inc_room_w_6</td>\n",
       "      <td>-0.378969</td>\n",
       "      <td>0.116035</td>\n",
       "      <td>-3.265980</td>\n",
       "      <td>0.001091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gamma_nests</td>\n",
       "      <td>0.585920</td>\n",
       "      <td>0.242312</td>\n",
       "      <td>2.418043</td>\n",
       "      <td>0.015604</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Coefficient Name  Coefficient Estimation  Std. Err   z_value    P(.>z)\n",
       "0          ich_w_0               -0.554876  0.174471 -3.180328  0.001471\n",
       "1          och_w_1               -0.857881  0.300332 -2.856437  0.004284\n",
       "2         occa_w_2               -1.089362  1.056226 -1.031371  0.302367\n",
       "3         icca_w_3               -0.225069  0.112268 -2.004749  0.044990\n",
       "4  int_cooling_w_4               -6.000777  4.986898 -1.203308  0.228857\n",
       "5  inc_cooling_w_5                0.249571  0.053589  4.657146  0.000003\n",
       "6     inc_room_w_6               -0.378969  0.116035 -3.265980  0.001091\n",
       "7      gamma_nests                0.585920  0.242312  2.418043  0.015604"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another possibility is to keep the dataset as is and specify manually the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating the dataset\n",
    "dataset = ChoiceDataset.from_single_wide_df(df=hc_df,\n",
    "                                            shared_features_columns=[\"income\"],\n",
    "                                            items_features_prefixes=[\"ich\", \"och\", \"occa\", \"icca\"],\n",
    "                                            delimiter=\".\",\n",
    "                                            items_id=items_id,\n",
    "                                            choices_column=\"depvar\",\n",
    "                                            choice_format=\"items_id\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the manual specification we define each weight and the indexes of the concerned items."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using L-BFGS optimizer, setting up .fit() function\n"
     ]
    }
   ],
   "source": [
    "model = NestedLogit(items_nests=[[0, 1, 2, 3], [4, 5, 6]],\n",
    "                    optimizer=\"lbfgs\",\n",
    "                    shared_gammas_over_nests=True)\n",
    "# Coefficients that are for all the alternatives\n",
    "model.add_shared_coefficient(feature_name=\"ich\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"och\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"icca\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"occa\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "\n",
    "# The coefficients concerning the income are split into two groups of alternatives:\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[0, 1, 2, 3], coefficient_name=\"income_cooling\")\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[2, 6], coefficient_name=\"income_room\")\n",
    "\n",
    "# Finally only one nest has an intercept\n",
    "model.add_shared_coefficient(feature_name=\"intercept\", items_indexes=[0, 1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:At least one gamma value for nests is below 0.05 and is\n",
      "        clipped to 0.05 for numeric optimization purposes.\n",
      "WARNING:root:L-BFGS Opimization finished:\n",
      "WARNING:root:---------------------------------------------------------------\n",
      "WARNING:root:Number of iterations: 39\n",
      "WARNING:root:Algorithm converged before reaching max iterations: True\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using L-BFGS optimizer, setting up .fit() function\n",
      "WARNING:tensorflow:5 out of the last 5 calls to <function pfor.<locals>.f at 0x7f459f242b60> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:5 out of the last 5 calls to <function pfor.<locals>.f at 0x7f459f242b60> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:6 out of the last 6 calls to <function pfor.<locals>.f at 0x7f459f16e520> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:6 out of the last 6 calls to <function pfor.<locals>.f at 0x7f459f16e520> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    }
   ],
   "source": [
    "hist = model.fit(dataset, get_report=True, verbose=1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficient Name</th>\n",
       "      <th>Coefficient Estimation</th>\n",
       "      <th>Std. Err</th>\n",
       "      <th>z_value</th>\n",
       "      <th>P(.&gt;z)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>beta_ich</td>\n",
       "      <td>-0.554878</td>\n",
       "      <td>0.174468</td>\n",
       "      <td>-3.180405</td>\n",
       "      <td>0.001471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>beta_och</td>\n",
       "      <td>-0.857885</td>\n",
       "      <td>0.300328</td>\n",
       "      <td>-2.856494</td>\n",
       "      <td>0.004283</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>beta_icca</td>\n",
       "      <td>-0.225067</td>\n",
       "      <td>0.112267</td>\n",
       "      <td>-2.004751</td>\n",
       "      <td>0.044990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>beta_occa</td>\n",
       "      <td>-1.089337</td>\n",
       "      <td>1.056223</td>\n",
       "      <td>-1.031352</td>\n",
       "      <td>0.302376</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>income_cooling</td>\n",
       "      <td>0.249571</td>\n",
       "      <td>0.053588</td>\n",
       "      <td>4.657206</td>\n",
       "      <td>0.000003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>income_room</td>\n",
       "      <td>-0.378970</td>\n",
       "      <td>0.116032</td>\n",
       "      <td>-3.266064</td>\n",
       "      <td>0.001091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>beta_intercept</td>\n",
       "      <td>-6.000929</td>\n",
       "      <td>4.986704</td>\n",
       "      <td>-1.203386</td>\n",
       "      <td>0.228827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gamma_nests</td>\n",
       "      <td>0.585921</td>\n",
       "      <td>0.242307</td>\n",
       "      <td>2.418097</td>\n",
       "      <td>0.015602</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Coefficient Name  Coefficient Estimation  Std. Err   z_value    P(.>z)\n",
       "0         beta_ich               -0.554878  0.174468 -3.180405  0.001471\n",
       "1         beta_och               -0.857885  0.300328 -2.856494  0.004283\n",
       "2        beta_icca               -0.225067  0.112267 -2.004751  0.044990\n",
       "3        beta_occa               -1.089337  1.056223 -1.031352  0.302376\n",
       "4   income_cooling                0.249571  0.053588  4.657206  0.000003\n",
       "5      income_room               -0.378970  0.116032 -3.266064  0.001091\n",
       "6   beta_intercept               -6.000929  4.986704 -1.203386  0.228827\n",
       "7      gamma_nests                0.585921  0.242307  2.418097  0.015602"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.trainable_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The gamma value can be retrieved to compute the correlation as follow:\n",
    "correlation = 1 -model.trainable_weights[-1][0][0].numpy()\n",
    "print(\"Correlation over alternatives within each nest:\", correlation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Second Formulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = NestedLogit(items_nests=[[0, 1, 3, 4, 5], [2, 6]],\n",
    "                    optimizer=\"lbfgs\",\n",
    "                    shared_gammas_over_nests=True)\n",
    "# Coefficients that are for all the alternatives\n",
    "model.add_shared_coefficient(feature_name=\"ich\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"och\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"icca\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"occa\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "\n",
    "# The coefficients concerning the income are split into two groups of alternatives:\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[0, 1, 2, 3],\n",
    "                             coefficient_name=\"income_cooling\")\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[2, 6],\n",
    "                             coefficient_name=\"income_room\")\n",
    "\n",
    "# Finally only one nest has an intercept\n",
    "model.add_shared_coefficient(feature_name=\"intercept\", items_indexes=[0, 1, 2, 3],\n",
    "coefficient_name=\"int.cooling\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist = model.fit(dataset, get_report=True, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficient Name</th>\n",
       "      <th>Coefficient Estimation</th>\n",
       "      <th>Std. Err</th>\n",
       "      <th>z_value</th>\n",
       "      <th>P(.&gt;z)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>beta_ich</td>\n",
       "      <td>-1.106775</td>\n",
       "      <td>1.271377</td>\n",
       "      <td>-0.870533</td>\n",
       "      <td>0.384009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>beta_och</td>\n",
       "      <td>-1.774187</td>\n",
       "      <td>2.188208</td>\n",
       "      <td>-0.810794</td>\n",
       "      <td>0.417484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>beta_icca</td>\n",
       "      <td>-0.329113</td>\n",
       "      <td>0.308635</td>\n",
       "      <td>-1.066349</td>\n",
       "      <td>0.286266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>beta_occa</td>\n",
       "      <td>-1.990666</td>\n",
       "      <td>2.507875</td>\n",
       "      <td>-0.793766</td>\n",
       "      <td>0.427332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>income_cooling</td>\n",
       "      <td>0.405710</td>\n",
       "      <td>0.425043</td>\n",
       "      <td>0.954515</td>\n",
       "      <td>0.339823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>income_room</td>\n",
       "      <td>-0.737662</td>\n",
       "      <td>0.742359</td>\n",
       "      <td>-0.993673</td>\n",
       "      <td>0.320382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>int.cooling</td>\n",
       "      <td>-13.468433</td>\n",
       "      <td>18.365850</td>\n",
       "      <td>-0.733341</td>\n",
       "      <td>0.463350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gamma_nests</td>\n",
       "      <td>1.323886</td>\n",
       "      <td>1.889999</td>\n",
       "      <td>0.700469</td>\n",
       "      <td>0.483634</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Coefficient Name  Coefficient Estimation   Std. Err   z_value    P(.>z)\n",
       "0         beta_ich               -1.106775   1.271377 -0.870533  0.384009\n",
       "1         beta_och               -1.774187   2.188208 -0.810794  0.417484\n",
       "2        beta_icca               -0.329113   0.308635 -1.066349  0.286266\n",
       "3        beta_occa               -1.990666   2.507875 -0.793766  0.427332\n",
       "4   income_cooling                0.405710   0.425043  0.954515  0.339823\n",
       "5      income_room               -0.737662   0.742359 -0.993673  0.320382\n",
       "6      int.cooling              -13.468433  18.365850 -0.733341  0.463350\n",
       "7      gamma_nests                1.323886   1.889999  0.700469  0.483634"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=180.03148>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# NLL:\n",
    "model.evaluate(dataset) * len(dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Third formulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = NestedLogit(items_nests=[[0, 1, 2, 3], [4, 5, 6]],\n",
    "                    optimizer=\"lbfgs\",\n",
    "                    shared_gammas_over_nests=False)\n",
    "# Coefficients that are for all the alternatives\n",
    "model.add_shared_coefficient(feature_name=\"ich\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"och\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"icca\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"occa\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "\n",
    "# The coefficients concerning the income are split into two groups of alternatives:\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[0, 1, 2, 3], coefficient_name=\"income_cooling\")\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[2, 6], coefficient_name=\"income_room\")\n",
    "\n",
    "# Finally only one nest has an intercept\n",
    "model.add_shared_coefficient(feature_name=\"intercept\", items_indexes=[0, 1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist = model.fit(dataset, get_report=False, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tf.Variable 'beta_ich:0' shape=(1, 1) dtype=float32, numpy=array([[-0.5336982]], dtype=float32)>,\n",
       " <tf.Variable 'beta_och:0' shape=(1, 1) dtype=float32, numpy=array([[-0.83580786]], dtype=float32)>,\n",
       " <tf.Variable 'beta_icca:0' shape=(1, 1) dtype=float32, numpy=array([[-0.22605541]], dtype=float32)>,\n",
       " <tf.Variable 'beta_occa:0' shape=(1, 1) dtype=float32, numpy=array([[-1.1159254]], dtype=float32)>,\n",
       " <tf.Variable 'income_cooling:0' shape=(1, 1) dtype=float32, numpy=array([[0.24864283]], dtype=float32)>,\n",
       " <tf.Variable 'income_room:0' shape=(1, 1) dtype=float32, numpy=array([[-0.36347896]], dtype=float32)>,\n",
       " <tf.Variable 'beta_intercept:0' shape=(1, 1) dtype=float32, numpy=array([[-5.644782]], dtype=float32)>,\n",
       " <tf.Variable 'gammas_nests:0' shape=(1, 2) dtype=float32, numpy=array([[0.57864326, 0.42460972]], dtype=float32)>]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.trainable_weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fourth Formulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = NestedLogit(items_nests=[[0, 1, 2], [3], [4, 5, 6]],\n",
    "                    optimizer=\"lbfgs\",\n",
    "                    shared_gammas_over_nests=True)\n",
    "# Coefficients that are for all the alternatives\n",
    "model.add_shared_coefficient(feature_name=\"ich\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"och\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"icca\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "model.add_shared_coefficient(feature_name=\"occa\", items_indexes=[0, 1, 2, 3, 4, 5, 6])\n",
    "\n",
    "# The coefficients concerning the income are split into two groups of alternatives:\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[0, 1, 2, 3], coefficient_name=\"income_cooling\")\n",
    "model.add_shared_coefficient(feature_name=\"income\", items_indexes=[2, 6], coefficient_name=\"income_room\")\n",
    "\n",
    "# Finally only one nest has an intercept\n",
    "model.add_shared_coefficient(feature_name=\"intercept\", items_indexes=[0, 1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist = model.fit(dataset, get_report=True, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coefficient Name</th>\n",
       "      <th>Coefficient Estimation</th>\n",
       "      <th>Std. Err</th>\n",
       "      <th>z_value</th>\n",
       "      <th>P(.&gt;z)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>beta_ich</td>\n",
       "      <td>-0.838395</td>\n",
       "      <td>0.119850</td>\n",
       "      <td>-6.995352</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>beta_och</td>\n",
       "      <td>-1.331599</td>\n",
       "      <td>0.251385</td>\n",
       "      <td>-5.297053</td>\n",
       "      <td>1.192093e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>beta_icca</td>\n",
       "      <td>-0.256129</td>\n",
       "      <td>0.127123</td>\n",
       "      <td>-2.014806</td>\n",
       "      <td>4.392505e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>beta_occa</td>\n",
       "      <td>-1.405654</td>\n",
       "      <td>1.152076</td>\n",
       "      <td>-1.220105</td>\n",
       "      <td>2.224251e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>income_cooling</td>\n",
       "      <td>0.311357</td>\n",
       "      <td>0.055357</td>\n",
       "      <td>5.624510</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>income_room</td>\n",
       "      <td>-0.571352</td>\n",
       "      <td>0.082983</td>\n",
       "      <td>-6.885149</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>beta_intercept</td>\n",
       "      <td>-10.413467</td>\n",
       "      <td>5.268852</td>\n",
       "      <td>-1.976421</td>\n",
       "      <td>4.810715e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gamma_nests</td>\n",
       "      <td>0.956544</td>\n",
       "      <td>0.430562</td>\n",
       "      <td>2.221618</td>\n",
       "      <td>2.630913e-02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Coefficient Name  Coefficient Estimation  Std. Err   z_value        P(.>z)\n",
       "0         beta_ich               -0.838395  0.119850 -6.995352  0.000000e+00\n",
       "1         beta_och               -1.331599  0.251385 -5.297053  1.192093e-07\n",
       "2        beta_icca               -0.256129  0.127123 -2.014806  4.392505e-02\n",
       "3        beta_occa               -1.405654  1.152076 -1.220105  2.224251e-01\n",
       "4   income_cooling                0.311357  0.055357  5.624510  0.000000e+00\n",
       "5      income_room               -0.571352  0.082983 -6.885149  0.000000e+00\n",
       "6   beta_intercept              -10.413467  5.268852 -1.976421  4.810715e-02\n",
       "7      gamma_nests                0.956544  0.430562  2.221618  2.630913e-02"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tf_env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}