DS_technical_test_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the test scorer\n",
    "def competition_scorer(y_true, y_pred):\n",
    "    return log_loss(y_true, y_pred, sample_weight=10**y_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Information about the test\n",
    "\n",
    "In this test, the `requests` datasets contains information about the requests made by group of individuals (or family) to the french emergency housing public service. A sample of the `requests` dataset corresponds to a unique request. The `individuals` datasets contains information about each individual for all requests.\n",
    "You can use the column `request_id` to link the two datasets.\n",
    "\n",
    "The goal is to predict the categorical variable `granted_number_of_nights` which represents the number of nights of emergency housing granted to a group. You can train your model on the `requests_train`, the predictions should be made for requests listed in the `requests_test` dataset. The competition score should also be computed on the `requests_test`.\n",
    "\n",
    "The evaluation metric is given by the `competition_scorer` defined above. It corresponds to a weighted log-loss with weights 1, 10, 100, or 1000 if the `granted_number_of_nights` takes the value 0, 1, 2, or 3 respectively. Thus beware that you will be penalized harder for classification mistakes made on the higher labels.\n",
    "\n",
    "The score for a random prediction is shown at the end of this notebook (~1.6), your trained models should at least reach a **score below 1** to be significantly better.\n",
    "\n",
    "Good luck!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data description"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### File requests_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Field name                       | Description                                                                                                                                                        | Type                     |\n",
    "|----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|\n",
    "| request\\_id                      | \"the id of request, which is unique in this dataset\\.\"                                                                                                             | string                   |\n",
    "| animal\\_presence                 | flage indicating whether an animal is accompanying the group                                                                                                       | boolean                  |\n",
    "| answer\\_creation\\_date           | the timestamp when the anwer was registered                                                                                                                        | timestamp \\(iso format\\) |\n",
    "| child\\_situation                 | the number representing the situation of the children in the group \\(if any\\)                                                                                      | integer                  |\n",
    "| child\\_to\\_come                  | flag indicating whether the group is expecting a baby                                                                                                              | boolean                  |\n",
    "| district                         | the id of the administrative district where the request is made\\. Each district has its own emergency center responsible for receiving and managing the requests\\. | integer                  |\n",
    "| granted\\_number\\_of\\_nights      | \"the target \\(train dataset only\\), the categorized number of nights granted to the request\"                                                                       | integer                  |\n",
    "| group\\_composition\\_id           | the number representing the group composition                                                                                                                      | integer                  |\n",
    "| group\\_composition\\_label        | the description of the group composition                                                                                                                           | string                   |\n",
    "| group\\_creation\\_date            | the timestamp when the group was created in the database                                                                                                           | timestamp \\(iso format\\) |\n",
    "| group\\_id                        | \"the id of the group\\. A group can be linked to several requests, so not unique\"                                                                                   | string                   |\n",
    "| group\\_main\\_requester\\_id       | the id of the individual making the request\\. it cannot be joined with individual\\_id                                                                              | string                   |\n",
    "| group\\_type                      | whether the group is a made of several people \\(GROUP\\) or just one person \\(PERSON\\)                                                                              | string                   |\n",
    "| housing\\_situation\\_id           | the id of the housing situation of the group when making the request                                                                                               | integer                  |\n",
    "| housing\\_situation\\_label        | the description of the housing situation of the group when making the request                                                                                      | string                   |\n",
    "| long\\_term\\_housing\\_request     | flag indicating whether the groupe has filed a request for a long term housing solution\\.                                                                          | boolean                  |\n",
    "| number\\_of\\_underage             | the number of individual under 18 in the group                                                                                                                     | integer                  |\n",
    "| request\\_backoffice\\_creator\\_id | the id of the emergency call center employee who received the request                                                                                              | string                   |\n",
    "| request\\_creation\\_date          | the timestamp when the request was made\\.                                                                                                                          | timestamp \\(iso format\\) |\n",
    "| requester\\_type                  | the type of person who called the emergency service \\(a user / a social patrol / a third party\\)                                                                   | string                   |\n",
    "| social\\_situation\\_id            | the id representing the social situation of the group in the database                                                                                              | string                   |\n",
    "| town                             | the town where the requester is living                                                                                                                             | string                   |\n",
    "| victim\\_of\\_violence             | flag indicating whether the group includes an individual victim of violence                                                                                        | boolean                  |\n",
    "| victim\\_of\\_violence\\_type       | type of the individual victim of violence \\(and if it is a case of domestic violence\\)                                                                             | string                   |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>request_id</th>\n",
       "      <th>animal_presence</th>\n",
       "      <th>answer_creation_date</th>\n",
       "      <th>child_situation</th>\n",
       "      <th>child_to_come</th>\n",
       "      <th>district</th>\n",
       "      <th>granted_number_of_nights</th>\n",
       "      <th>group_composition_id</th>\n",
       "      <th>group_composition_label</th>\n",
       "      <th>group_creation_date</th>\n",
       "      <th>...</th>\n",
       "      <th>housing_situation_label</th>\n",
       "      <th>long_term_housing_request</th>\n",
       "      <th>number_of_underage</th>\n",
       "      <th>request_backoffice_creator_id</th>\n",
       "      <th>request_creation_date</th>\n",
       "      <th>requester_type</th>\n",
       "      <th>social_situation_id</th>\n",
       "      <th>town</th>\n",
       "      <th>victim_of_violence</th>\n",
       "      <th>victim_of_violence_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>130667d2964de43c511d0ead7ac66b33</td>\n",
       "      <td>f</td>\n",
       "      <td>2019-02-11 22:30:00</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>85</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>man alone</td>\n",
       "      <td>2018-05-03 12:10:40.416</td>\n",
       "      <td>...</td>\n",
       "      <td>other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1</td>\n",
       "      <td>eecd6110b7e9157e6423428b22a28159</td>\n",
       "      <td>2019-02-11 22:30:00</td>\n",
       "      <td>third party</td>\n",
       "      <td>9a84febc47fedaf2c81d7c755e9edc85</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b1cd62fcf24eebb0f685d5eaf55317a8</td>\n",
       "      <td>f</td>\n",
       "      <td>2019-02-12 09:00:00</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>85</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>man alone</td>\n",
       "      <td>2019-03-07 12:11:08.545</td>\n",
       "      <td>...</td>\n",
       "      <td>other</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1</td>\n",
       "      <td>eecd6110b7e9157e6423428b22a28159</td>\n",
       "      <td>2019-02-12 09:00:00</td>\n",
       "      <td>user</td>\n",
       "      <td>11f64f0547e855c5c59329b9edc63063</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>12cc0a18890bd4959fe42df3ae58838e</td>\n",
       "      <td>f</td>\n",
       "      <td>2019-02-01 21:00:00</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>59</td>\n",
       "      <td>1</td>\n",
       "      <td>80</td>\n",
       "      <td>isolated child/underage</td>\n",
       "      <td>2019-03-13 12:54:15.887</td>\n",
       "      <td>...</td>\n",
       "      <td>street</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1</td>\n",
       "      <td>686d962128c7622aa79ed91da13f5c77</td>\n",
       "      <td>2019-02-01 21:00:00</td>\n",
       "      <td>user</td>\n",
       "      <td>898ab2883f410f4c6fb042ef16329fbb</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ae2d5b4dc181d29e430132f145da1556</td>\n",
       "      <td>f</td>\n",
       "      <td>2019-02-25 15:12:05.037</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>50</td>\n",
       "      <td>2</td>\n",
       "      <td>20</td>\n",
       "      <td>woman alone</td>\n",
       "      <td>2018-10-09 14:37:29.773</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1</td>\n",
       "      <td>ec29519464c381a6630f4e480a0c000e</td>\n",
       "      <td>2019-02-25 15:12:05.037</td>\n",
       "      <td>user</td>\n",
       "      <td>3998c8e890ddc980527dd1af45dfa672</td>\n",
       "      <td>NaN</td>\n",
       "      <td>t</td>\n",
       "      <td>woman</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>d13a17ce36c832514fda2464e11c4a9f</td>\n",
       "      <td>f</td>\n",
       "      <td>2019-02-20 09:59:20.603</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>93</td>\n",
       "      <td>2</td>\n",
       "      <td>60</td>\n",
       "      <td>group with child(ren)</td>\n",
       "      <td>2018-10-02 10:47:36.931</td>\n",
       "      <td>...</td>\n",
       "      <td>hotel paid by the emergency centre</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-1</td>\n",
       "      <td>c8b5cc926b1d7212224a08baa9619518</td>\n",
       "      <td>2019-02-20 09:59:20.603</td>\n",
       "      <td>user</td>\n",
       "      <td>74a10d96eae43586bc3c46e89719b821</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                         request_id animal_presence     answer_creation_date  \\\n",
       "0  130667d2964de43c511d0ead7ac66b33               f      2019-02-11 22:30:00   \n",
       "1  b1cd62fcf24eebb0f685d5eaf55317a8               f      2019-02-12 09:00:00   \n",
       "2  12cc0a18890bd4959fe42df3ae58838e               f      2019-02-01 21:00:00   \n",
       "3  ae2d5b4dc181d29e430132f145da1556               f  2019-02-25 15:12:05.037   \n",
       "4  d13a17ce36c832514fda2464e11c4a9f               f  2019-02-20 09:59:20.603   \n",
       "\n",
       "   child_situation child_to_come  district  granted_number_of_nights  \\\n",
       "0               -1           NaN        85                         1   \n",
       "1               -1           NaN        85                         1   \n",
       "2               -1           NaN        59                         1   \n",
       "3               -1           NaN        50                         2   \n",
       "4               -1           NaN        93                         2   \n",
       "\n",
       "   group_composition_id  group_composition_label      group_creation_date  \\\n",
       "0                    10                man alone  2018-05-03 12:10:40.416   \n",
       "1                    10                man alone  2019-03-07 12:11:08.545   \n",
       "2                    80  isolated child/underage  2019-03-13 12:54:15.887   \n",
       "3                    20              woman alone  2018-10-09 14:37:29.773   \n",
       "4                    60    group with child(ren)  2018-10-02 10:47:36.931   \n",
       "\n",
       "   ...             housing_situation_label long_term_housing_request  \\\n",
       "0  ...                               other                       NaN   \n",
       "1  ...                               other                       NaN   \n",
       "2  ...                              street                       NaN   \n",
       "3  ...                                 NaN                       NaN   \n",
       "4  ...  hotel paid by the emergency centre                       NaN   \n",
       "\n",
       "  number_of_underage     request_backoffice_creator_id  \\\n",
       "0                 -1  eecd6110b7e9157e6423428b22a28159   \n",
       "1                 -1  eecd6110b7e9157e6423428b22a28159   \n",
       "2                 -1  686d962128c7622aa79ed91da13f5c77   \n",
       "3                 -1  ec29519464c381a6630f4e480a0c000e   \n",
       "4                 -1  c8b5cc926b1d7212224a08baa9619518   \n",
       "\n",
       "     request_creation_date requester_type               social_situation_id  \\\n",
       "0      2019-02-11 22:30:00    third party  9a84febc47fedaf2c81d7c755e9edc85   \n",
       "1      2019-02-12 09:00:00           user  11f64f0547e855c5c59329b9edc63063   \n",
       "2      2019-02-01 21:00:00           user  898ab2883f410f4c6fb042ef16329fbb   \n",
       "3  2019-02-25 15:12:05.037           user  3998c8e890ddc980527dd1af45dfa672   \n",
       "4  2019-02-20 09:59:20.603           user  74a10d96eae43586bc3c46e89719b821   \n",
       "\n",
       "  town victim_of_violence victim_of_violence_type  \n",
       "0  NaN                  f                     NaN  \n",
       "1  NaN                  f                     NaN  \n",
       "2  NaN                  f                     NaN  \n",
       "3  NaN                  t                   woman  \n",
       "4  NaN                  f                     NaN  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "requests_train = pd.read_csv(filepath_or_buffer='data/requests_train.csv',\n",
    "                             sep=',',\n",
    "                             low_memory=False,\n",
    "                             error_bad_lines=False)\n",
    "\n",
    "requests_test = pd.read_csv(filepath_or_buffer='data/requests_test.csv',\n",
    "                            sep=',',\n",
    "                            low_memory=False,\n",
    "                            error_bad_lines=False)\n",
    "\n",
    "\n",
    "requests_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### File individuals_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Field name                      | Description                                                                                                               | Type                     |\n",
    "|---------------------------------|---------------------------------------------------------------------------------------------------------------------------|--------------------------|\n",
    "| request\\_id                     | the id of the request\\. It is not unique in this dataset \\(a request can be for several individual in case of a group\\)\\. | string                   |\n",
    "| individual\\_id                  | the id of the individual                                                                                                  | string                   |\n",
    "| birth\\_month                    | month of birth of the individual                                                                                          | integer                  |\n",
    "| birth\\_year                     | the year of birth of the individual                                                                                       | integer                  |\n",
    "| childcare\\_center\\_supervision  | flag representing the fact that a child receives follow\\-up care                                                          | boolean                  |\n",
    "| disabled\\_worker\\_certification | flag representing the fact that an individual is considered disabled regarding the employement law                        | boolean                  |\n",
    "| gender                          | the gender of the individual                                                                                              | boolean                  |\n",
    "| housing\\_situation\\_2\\_id       | housing situation category number of the group when making the request \\(more precise\\)                                   | integer                  |\n",
    "| housing\\_situation\\_2\\_label    | description of the housing situation of the group when making the request \\(more precise\\)                                | string                   |\n",
    "| individual\\_creation\\_date      | timestamp of the individual creation in the database                                                                      | timestamp \\(iso format\\) |\n",
    "| individual\\_role                | description of the role of the individual                                                                                 | string                   |\n",
    "| individual\\_role\\_2\\_id         | category number of the role of the individual                                                                             | integer                  |\n",
    "| individual\\_role\\_2\\_label      | \"description of the role of the individual, more precise than the former\"                                                 | string                   |\n",
    "| marital\\_status\\_id             | id of the marital status of the individual                                                                                | integer                  |\n",
    "| marital\\_status\\_label          | description of the marital status of the individual                                                                       | string                   |\n",
    "| pregnancy                       | flag representing whether the individual is pregnant \\(if applicable\\)                                                    | boolean                  |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>request_id</th>\n",
       "      <th>individual_id</th>\n",
       "      <th>housing_situation_2_id</th>\n",
       "      <th>housing_situation_2_label</th>\n",
       "      <th>birth_month</th>\n",
       "      <th>birth_year</th>\n",
       "      <th>childcare_center_supervision</th>\n",
       "      <th>disabled_worker_certification</th>\n",
       "      <th>gender</th>\n",
       "      <th>individual_creation_date</th>\n",
       "      <th>individual_role</th>\n",
       "      <th>individual_role_2_id</th>\n",
       "      <th>individual_role_2_label</th>\n",
       "      <th>marital_status_id</th>\n",
       "      <th>marital_status_label</th>\n",
       "      <th>pregnancy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>130667d2964de43c511d0ead7ac66b33</td>\n",
       "      <td>b24a8cfaaf3663bac45ca3bcd24f0b36</td>\n",
       "      <td>20</td>\n",
       "      <td>on the street</td>\n",
       "      <td>5</td>\n",
       "      <td>1977</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>male</td>\n",
       "      <td>2018-01-29</td>\n",
       "      <td>isolated person</td>\n",
       "      <td>30</td>\n",
       "      <td>isolated adult</td>\n",
       "      <td>10</td>\n",
       "      <td>single person</td>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b1cd62fcf24eebb0f685d5eaf55317a8</td>\n",
       "      <td>eef783d1bda82661a751c12c34aff8c2</td>\n",
       "      <td>20</td>\n",
       "      <td>on the street</td>\n",
       "      <td>5</td>\n",
       "      <td>1980</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>male</td>\n",
       "      <td>2019-03-07</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30</td>\n",
       "      <td>isolated adult</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>12cc0a18890bd4959fe42df3ae58838e</td>\n",
       "      <td>927b6fa58d6be263c411332835b11fec</td>\n",
       "      <td>20</td>\n",
       "      <td>on the street</td>\n",
       "      <td>1</td>\n",
       "      <td>2003</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>male</td>\n",
       "      <td>2019-03-13</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>isolated underage/child</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ae2d5b4dc181d29e430132f145da1556</td>\n",
       "      <td>94181d5cf4a56cc84d12677df84e2e03</td>\n",
       "      <td>20</td>\n",
       "      <td>on the street</td>\n",
       "      <td>7</td>\n",
       "      <td>1986</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>female</td>\n",
       "      <td>2018-10-09</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30</td>\n",
       "      <td>isolated adult</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>d13a17ce36c832514fda2464e11c4a9f</td>\n",
       "      <td>e79d2e3f9ace50049f1796c098008c13</td>\n",
       "      <td>35</td>\n",
       "      <td>hotel paid by the emergency structure</td>\n",
       "      <td>9</td>\n",
       "      <td>1980</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>female</td>\n",
       "      <td>2017-06-22</td>\n",
       "      <td>husband/spouse</td>\n",
       "      <td>1</td>\n",
       "      <td>partner</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         request_id                     individual_id  \\\n",
       "0  130667d2964de43c511d0ead7ac66b33  b24a8cfaaf3663bac45ca3bcd24f0b36   \n",
       "1  b1cd62fcf24eebb0f685d5eaf55317a8  eef783d1bda82661a751c12c34aff8c2   \n",
       "2  12cc0a18890bd4959fe42df3ae58838e  927b6fa58d6be263c411332835b11fec   \n",
       "3  ae2d5b4dc181d29e430132f145da1556  94181d5cf4a56cc84d12677df84e2e03   \n",
       "4  d13a17ce36c832514fda2464e11c4a9f  e79d2e3f9ace50049f1796c098008c13   \n",
       "\n",
       "   housing_situation_2_id              housing_situation_2_label  birth_month  \\\n",
       "0                      20                          on the street            5   \n",
       "1                      20                          on the street            5   \n",
       "2                      20                          on the street            1   \n",
       "3                      20                          on the street            7   \n",
       "4                      35  hotel paid by the emergency structure            9   \n",
       "\n",
       "   birth_year childcare_center_supervision disabled_worker_certification  \\\n",
       "0        1977                          NaN                           NaN   \n",
       "1        1980                          NaN                           NaN   \n",
       "2        2003                          NaN                           NaN   \n",
       "3        1986                          NaN                           NaN   \n",
       "4        1980                          NaN                           NaN   \n",
       "\n",
       "   gender individual_creation_date  individual_role  individual_role_2_id  \\\n",
       "0    male               2018-01-29  isolated person                    30   \n",
       "1    male               2019-03-07              NaN                    30   \n",
       "2    male               2019-03-13              NaN                    10   \n",
       "3  female               2018-10-09              NaN                    30   \n",
       "4  female               2017-06-22   husband/spouse                     1   \n",
       "\n",
       "   individual_role_2_label  marital_status_id marital_status_label pregnancy  \n",
       "0           isolated adult                 10        single person         f  \n",
       "1           isolated adult                 -1                  NaN         f  \n",
       "2  isolated underage/child                 -1                  NaN         f  \n",
       "3           isolated adult                 -1                  NaN         f  \n",
       "4                  partner                 -1                  NaN         f  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "individuals_train = pd.read_csv(filepath_or_buffer='data/individuals_train.csv',\n",
    "                                sep=',',\n",
    "                                low_memory=False,\n",
    "                                error_bad_lines=False)\n",
    "\n",
    "individuals_test = pd.read_csv(filepath_or_buffer='data/individuals_test.csv',\n",
    "                               sep=',',\n",
    "                               low_memory=False,\n",
    "                               error_bad_lines=False)\n",
    "\n",
    "individuals_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test score: 1.6465241024382162\n"
     ]
    }
   ],
   "source": [
    "random_preds = np.random.uniform(size=(requests_test.shape[0], 4))\n",
    "y_true = requests_test.granted_number_of_nights.values\n",
    "random_score = competition_scorer(y_true, random_preds)\n",
    "\n",
    "print(f'test score: {random_score}')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}