examples/headline_summarization/Quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bdc2f8db",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Arthur Sandbox Example: Headline Summarization\n",
    "In this guide, we'll use the XSum headline summarization model and dataset from Huggingface to onboard a new streaming model to the Arthur platform. Then we will use Arthur to analyze our model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bbdeef0",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "tags": []
   },
   "source": [
    "# Outline\n",
    "\n",
    "Read on for an overview of everything this notebook will cover. **[Click here to dive straight into the code.](#Imports)**\n",
    "\n",
    "## Onboarding\n",
    "\n",
    "Onboarding is the process of setting up your model to be monitored by Arthur. You specify the type of data your model ingests, send a reference dataset to provide a baseline of the distribution of your data, and you configure additional settings among the services Arthur offers.\n",
    "\n",
    "**Arthur does not need your model object itself to monitor performance - only predictions are required**\n",
    "\n",
    "All you need to monitor your model with Arthur is to upload the predictions your model makes: Arthur computes analytics about your model based on that prediction data. This data can be computed directly by your model in a script or notebook like this one to be uploaded to the platform, or can be fetched from an external database to be sent to Arthur.\n",
    "\n",
    "### Getting Model Predictions\n",
    "We'll prepare a sample from a headline summarization dataset and generate answers from a pretrained model from Huggingface.\n",
    "\n",
    "### Registering Model with Arthur\n",
    "We'll configure our model's attributes and save the model to the Arthur platform.\n",
    "\n",
    "### Sending Inferences\n",
    "We'll send model inferences (inputs and predictions) to the Arthur platform.\n",
    "\n",
    "\n",
    "## Model Monitoring and Analysis\n",
    "\n",
    "Once onboarding is complete and you have inferences uploaded to the platform, you can use Arthur to get model monitoring insights.\n",
    "\n",
    "We will analyze the token-likelihood sequences produced by the model, measuring their length and their spikiness in changes in token likelihood from one token to the next."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d1b958a",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ce1bc2e",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    },
    "tags": []
   },
   "source": [
    "# Setup & Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9c907c68",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# ensure required packages are installed\n",
    "#  don't worry, our requirements are flexible!\n",
    "\n",
    "! pip install -r requirements.txt > /dev/null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9c1dfc67",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-09-02T19:08:14.390559Z",
     "start_time": "2021-09-02T19:08:12.824796Z"
    },
    "pycharm": {
     "name": "#%%\n"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from datetime import datetime, timedelta\n",
    "from IPython.display import display, HTML\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beb37338",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0fbaa73",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Onboarding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e3639de",
   "metadata": {},
   "source": [
    "## Prepare data and predictions\n",
    "We will load a sample of our training split fromt the XSum dataset. This dataset is used as a reference distribution defining the expected space of tasks for this generative text model. This way, Arthur can monitor for drift and stability of the texts inputted to your model.\n",
    "\n",
    "In this example we will use the Pegasus encoder-decoder model finetuned on the XSum dataset to produce model predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "165764b7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Found cached dataset xsum (/Users/maxcembalest/.cache/huggingface/datasets/xsum/default/1.2.0/082863bf4754ee058a5b6f6525d0cb2b18eadb62c7b370b095d1364050a52b71)\n"
     ]
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n",
    "\n",
    "xsum_dataset = load_dataset(\"xsum\", split='test')\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"google/pegasus-xsum\")\n",
    "\n",
    "lm_model = AutoModelForSeq2SeqLM.from_pretrained(\"google/pegasus-xsum\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee628cbb",
   "metadata": {},
   "source": [
    "The below generation function takes as input a string and returns a tuple containing the token ids of the generated sequence as well as the log probabilities as an array of shape (sequence length, vocab size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0b2c5bff",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Tuple, List\n",
    "def generate_summary(article: str) -> Tuple[List[int], np.array]:\n",
    "    tokens = tokenizer(article, max_length=512, return_tensors=\"pt\")\n",
    "    generation = lm_model.generate(**tokens, return_dict_in_generate=True, output_scores=True, num_beams=1, renormalize_logits=True)\n",
    "    return generation.sequences, np.stack(generation.scores).squeeze()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "360e6a05",
   "metadata": {},
   "source": [
    "Get a mapping from ids to tokens in the vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "097ed9c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "vocab = tokenizer.get_vocab()\n",
    "id_to_vocab = {id_: token for token, id_ in vocab.items()}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dbf4159",
   "metadata": {},
   "source": [
    "Next, we use the Arthur SDK function `tensors_to_arthur_inference` to convert our model outputs to an Arthur-ready format. We can choose how many (up to 5) of the top probability tokens to log for each inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c3ba2e98",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    }
   ],
   "source": [
    "from arthurai.core.model_utils import tensors_to_arthur_inference\n",
    "\n",
    "inferences = []\n",
    "for i in range(5):\n",
    "    # get model output\n",
    "    generation = generate_summary(xsum_dataset[i]['document'])\n",
    "    # decode tokens to final generated text\n",
    "    seq = tokenizer.batch_decode(generation[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)\n",
    "    likelihoods = tensors_to_arthur_inference(generation[1], id_to_vocab, 3)\n",
    "    inference = {\n",
    "        'document': xsum_dataset[i]['document'],\n",
    "        'generated_summary': seq[0],\n",
    "        'summary_token_probs': likelihoods\n",
    "    }\n",
    "    inferences.append(inference)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "eaa61a70",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'document': 'Prison Link Cymru had 1,099 referrals in 2015-16 and said some ex-offenders were living rough for up to a year before finding suitable accommodation.\\nWorkers at the charity claim investment in housing would be cheaper than jailing homeless repeat offenders.\\nThe Welsh Government said more people than ever were getting help to address housing problems.\\nChanges to the Housing Act in Wales, introduced in 2015, removed the right for prison leavers to be given priority for accommodation.\\nPrison Link Cymru, which helps people find accommodation after their release, said things were generally good for women because issues such as children or domestic violence were now considered.\\nHowever, the same could not be said for men, the charity said, because issues which often affect them, such as post traumatic stress disorder or drug dependency, were often viewed as less of a priority.\\nAndrew Stevens, who works in Welsh prisons trying to secure housing for prison leavers, said the need for accommodation was \"chronic\".\\n\"There\\'s a desperate need for it, finding suitable accommodation for those leaving prison there is just a lack of it everywhere,\" he said.\\n\"It could take six months to a year, without a lot of help they could be on the streets for six months.\\n\"When you think of the consequences of either being on the street, especially with the cold weather at the moment or you may have a roof over your head, sometimes there is only one choice.\"\\nMr Stevens believes building more one-bedroom flats could help ease the problem.\\n\"The average price is a hundred pounds a week to keep someone in a rented flat, prison is a lot more than that so I would imagine it would save the public purse quite a few pounds,\" he said.\\nOfficial figures show 830 one-bedroom properties were built in the year to March 2016, of an overall total of 6,900 new properties in Wales.\\nMarc, 50, who has been in and out of prison for the past 20 years for burglary offences, said he struggled to find accommodation each time he was released.\\nHe said he would ask himself: \"Where am I going to stay? Where am I going to live? Have I got somewhere where I can see my daughter.\"\\n\"You\\'re put out among the same sort of people doing the same sort of thing, and it\\'s difficult, it\\'s difficult to get away from it. It\\'s like every man for himself, there\\'s nothing.\"\\nMarc has now found stable accommodation with homeless charity Emmaus and said it had been life changing.\\n\"You feel safe, you got hot food, you\\'ve got company of people in similar situations to yourself but all dealing with different issues. It\\'s a constructive, helpful atmosphere,\" he said.\\nTom Clarke, chief executive of Emmaus South Wales, agreed there was not enough support available.\\n\"We do still see [people] homeless on the streets, so clearly they haven\\'t got accommodation and haven\\'t got provision,\" he said.\\n\"I think the key is connecting people with the services they need. I don\\'t delude myself that Emmaus can offer a one size fits all for everyone, we can\\'t.\\n\"But there must be other opportunities and given suitable encouragement I believe that can and should happen.\"\\nA Welsh Government spokesman said the national pathway for homeless services to children, young people and adults in the secure estate had prevented many people from losing their home whilst serving their prison sentence.\\nIt added there were already significant demands for one-bedroom flats across the public and private sector and it was providing 20,000 new affordable homes in the next five years.',\n",
       "  'generated_summary': 'A charity has said it has seen a \"desperate\" need for housing for homeless people in Wales.',\n",
       "  'summary_token_probs': [{'▁More': 0.14739905297756195,\n",
       "    '▁The': 0.15617099404335022,\n",
       "    '▁A': 0.1668444722890854},\n",
       "   {'▁homeless': 0.05522293969988823,\n",
       "    '▁\"': 0.2275698035955429,\n",
       "    '▁charity': 0.4529936909675598},\n",
       "   {'▁which': 0.10341085493564606,\n",
       "    '▁says': 0.11643338948488235,\n",
       "    '▁has': 0.3602505624294281},\n",
       "   {'▁called': 0.13903598487377167,\n",
       "    '▁warned': 0.18098242580890656,\n",
       "    '▁said': 0.3734981119632721},\n",
       "   {'▁there': 0.13141843676567078,\n",
       "    '▁homeless': 0.1574133038520813,\n",
       "    '▁it': 0.20769721269607544},\n",
       "   {'▁received': 0.05590398982167244,\n",
       "    '▁is': 0.15843385457992554,\n",
       "    '▁has': 0.34066715836524963},\n",
       "   {'▁been': 0.04716835170984268,\n",
       "    '▁had': 0.12852980196475983,\n",
       "    '▁seen': 0.5675621032714844},\n",
       "   {'▁\"': 0.04353978484869003,\n",
       "    '▁a': 0.7346400618553162,\n",
       "    '▁an': 0.07529696822166443},\n",
       "   {'▁40%': 0.027278512716293335,\n",
       "    '▁rise': 0.09127599000930786,\n",
       "    '▁\"': 0.3018726110458374},\n",
       "   {'dist': 0.02170277014374733,\n",
       "    'di': 0.06995752453804016,\n",
       "    'd': 0.7411702275276184},\n",
       "   {'anger': 0.00111184932757169,\n",
       "    'read': 0.063139908015728,\n",
       "    'esper': 0.9169283509254456},\n",
       "   {'ation': 0.003937354311347008,\n",
       "    'ately': 0.07379543036222458,\n",
       "    'ate': 0.8817391395568848},\n",
       "   {'▁Need': 0.004620884545147419,\n",
       "    '▁need': 0.3047276437282562,\n",
       "    '\"': 0.6056327223777771},\n",
       "   {'▁increase': 0.15280920267105103,\n",
       "    '▁rise': 0.1943100243806839,\n",
       "    '▁need': 0.19728246331214905},\n",
       "   {'▁in': 0.0029658828862011433,\n",
       "    '▁to': 0.055987462401390076,\n",
       "    '▁for': 0.8625580072402954},\n",
       "   {'▁affordable': 0.11721357703208923,\n",
       "    '▁more': 0.24277189373970032,\n",
       "    '▁housing': 0.2509273290634155},\n",
       "   {'▁accommodation': 0.02548299916088581,\n",
       "    '▁for': 0.7337383031845093,\n",
       "    '▁help': 0.02847117930650711},\n",
       "   {'▁former': 0.10322487354278564,\n",
       "    '▁men': 0.12142349034547806,\n",
       "    '▁homeless': 0.24715575575828552},\n",
       "   {'▁men': 0.14614203572273254,\n",
       "    '▁former': 0.18016108870506287,\n",
       "    '▁people': 0.1847544014453888},\n",
       "   {'▁leaving': 0.07088559865951538,\n",
       "    '▁in': 0.20193831622600555,\n",
       "    '▁after': 0.15997382998466492},\n",
       "   {'▁Welsh': 0.013427690602838993,\n",
       "    '▁Wales': 0.8253042697906494,\n",
       "    '▁the': 0.023584609851241112},\n",
       "   {'▁over': 0.021732943132519722,\n",
       "    ',': 0.05551162362098694,\n",
       "    '.': 0.7140709757804871},\n",
       "   {'.': 9.655363828642294e-05,\n",
       "    '▁\"': 0.0003459836298134178,\n",
       "    '</s>': 0.9349042773246765}]},\n",
       " {'document': 'Officers searched properties in the Waterfront Park and Colonsay View areas of the city on Wednesday.\\nDetectives said three firearms, ammunition and a five-figure sum of money were recovered.\\nA 26-year-old man who was arrested and charged appeared at Edinburgh Sheriff Court on Thursday.',\n",
       "  'generated_summary': 'A man has been charged after firearms and money were seized by police in Edinburgh.',\n",
       "  'summary_token_probs': [{'▁An': 0.009951921179890633,\n",
       "    '▁Police': 0.04644511640071869,\n",
       "    '▁A': 0.832900881767273},\n",
       "   {'▁woman': 0.0006877636187709868,\n",
       "    '▁26-': 0.0016444444190710783,\n",
       "    '▁man': 0.9207310080528259},\n",
       "   {'▁who': 0.0009694563923403621,\n",
       "    '▁has': 0.9235783219337463,\n",
       "    '▁is': 0.008910998702049255},\n",
       "   {'▁died': 0.0005632047541439533,\n",
       "    '▁been': 0.9080358743667603,\n",
       "    '▁appeared': 0.01719382032752037},\n",
       "   {'▁accused': 0.004897481761872768,\n",
       "    '▁arrested': 0.1702582687139511,\n",
       "    '▁charged': 0.7189240455627441},\n",
       "   {'▁following': 0.14367060363292694,\n",
       "    '▁after': 0.4145500361919403,\n",
       "    '▁in': 0.1933813840150833},\n",
       "   {'▁a': 0.10204783827066422,\n",
       "    '▁police': 0.2405659407377243,\n",
       "    '▁firearms': 0.36837926506996155},\n",
       "   {',': 0.03414512425661087,\n",
       "    '▁were': 0.0824623852968216,\n",
       "    '▁and': 0.7969264388084412},\n",
       "   {'▁a': 0.1313030868768692,\n",
       "    '▁cash': 0.1679534614086151,\n",
       "    '▁money': 0.2802054286003113},\n",
       "   {'▁was': 0.007942081429064274,\n",
       "    '▁with': 0.011951232329010963,\n",
       "    '▁were': 0.8840651512145996},\n",
       "   {'▁found': 0.08153212070465088,\n",
       "    '▁recovered': 0.26476386189460754,\n",
       "    '▁seized': 0.5870693922042847},\n",
       "   {'▁during': 0.06771715730428696,\n",
       "    '▁in': 0.31669050455093384,\n",
       "    '▁by': 0.4764735698699951},\n",
       "   {'▁Police': 0.015553689561784267,\n",
       "    '▁police': 0.8184274435043335,\n",
       "    '▁armed': 0.01920345425605774},\n",
       "   {'▁during': 0.015437500551342964,\n",
       "    '▁investigating': 0.20598174631595612,\n",
       "    '▁in': 0.5863365530967712},\n",
       "   {'▁the': 0.011391673237085342,\n",
       "    '▁an': 0.016664989292621613,\n",
       "    '▁Edinburgh': 0.8543614149093628},\n",
       "   {'▁in': 0.0013960031792521477,\n",
       "    ',': 0.00601107906550169,\n",
       "    '.': 0.9181209206581116},\n",
       "   {')': 3.914666740456596e-05,\n",
       "    '.': 8.346551476279274e-05,\n",
       "    '</s>': 0.9207320213317871}]},\n",
       " {'document': 'Jordan Hill, Brittany Covington and Tesfaye Cooper, all 18, and Tanishia Covington, 24, appeared in a Chicago court on Friday.\\nThe four have been charged with hate crimes and aggravated kidnapping and battery, among other things.\\nAn online fundraiser for their victim has collected $51,000 (Â£42,500) so far.\\nDenying the four suspects bail, Judge Maria Kuriakos Ciesil asked: \"Where was your sense of decency?\"\\nProsecutors told the court the beating started in a van and continued at a house, where the suspects allegedly forced the 18-year-old white victim, who suffers from schizophrenia and attention deficit disorder, to drink toilet water and kiss the floor.\\nPolice allege the van was earlier stolen by Mr Hill, who is also accused of demanding $300 from the victim\\'s mother while they held him captive, according to the Chicago Tribune.\\nThe court was also told the suspects stuffed a sock into his mouth, taped his mouth shut and bound his hands with a belt.\\nIn a video made for Facebook Live which was watched millions of times, the assailants can be heard making derogatory statements against white people and Donald Trump.\\nThe victim had been dropped off at a McDonalds to meet Mr Hill - who was one of his friends - on 31 December.\\nHe was found by a police officer on Tuesday, 3 January, a day after he was reported missing by his parents.\\nProsecutors say the suspects each face two hate crimes counts, one because of the victim\\'s race and the other because of his disabilities.',\n",
       "  'generated_summary': 'Four black people accused of beating a mentally disabled man and broadcasting the attack on Facebook have been denied bail.',\n",
       "  'summary_token_probs': [{'▁The': 0.031043965369462967,\n",
       "    '▁A': 0.07126393914222717,\n",
       "    '▁Four': 0.7035706639289856},\n",
       "   {'▁white': 0.024705447256565094,\n",
       "    '▁black': 0.7900169491767883,\n",
       "    '▁people': 0.06329590082168579},\n",
       "   {'▁teenagers': 0.029675383120775223,\n",
       "    '▁Americans': 0.05551106482744217,\n",
       "    '▁people': 0.650540292263031},\n",
       "   {'▁in': 0.06361347436904907,\n",
       "    '▁have': 0.2720164358615875,\n",
       "    '▁accused': 0.44093674421310425},\n",
       "   {'▁over': 0.0009448024211451411,\n",
       "    '▁in': 0.008143513463437557,\n",
       "    '▁of': 0.9236844182014465},\n",
       "   {'▁torture': 0.030659008771181107,\n",
       "    '▁torturing': 0.20328350365161896,\n",
       "    '▁beating': 0.4224582612514496},\n",
       "   {'▁an': 0.01298152282834053,\n",
       "    '▁a': 0.692047655582428,\n",
       "    '▁and': 0.225032240152359},\n",
       "   {'▁white': 0.05361773446202278,\n",
       "    '▁disabled': 0.20468999445438385,\n",
       "    '▁mentally': 0.47232839465141296},\n",
       "   {'▁ill': 0.03281280770897865,\n",
       "    '-': 0.0637507289648056,\n",
       "    '▁disabled': 0.7702006697654724},\n",
       "   {'▁black': 0.05080603435635567,\n",
       "    '▁white': 0.26410579681396484,\n",
       "    '▁man': 0.4465253949165344},\n",
       "   {'▁for': 0.024473298341035843,\n",
       "    '▁in': 0.30203190445899963,\n",
       "    '▁and': 0.5432246923446655},\n",
       "   {'▁broadcast': 0.08738340437412262,\n",
       "    '▁posting': 0.14907076954841614,\n",
       "    '▁broadcasting': 0.6322811245918274},\n",
       "   {'▁their': 0.011422928422689438,\n",
       "    '▁the': 0.7138702869415283,\n",
       "    '▁it': 0.18000441789627075},\n",
       "   {'▁ordeal': 0.04824370890855789,\n",
       "    '▁assault': 0.18628427386283875,\n",
       "    '▁attack': 0.4448767304420471},\n",
       "   {'▁online': 0.05846288800239563,\n",
       "    '▁live': 0.17527522146701813,\n",
       "    '▁on': 0.678325355052948},\n",
       "   {'Facebook': 0.0025630691088736057,\n",
       "    '▁social': 0.0744195356965065,\n",
       "    '▁Facebook': 0.867451548576355},\n",
       "   {'▁live': 0.08001943677663803,\n",
       "    '▁have': 0.5348222851753235,\n",
       "    '▁Live': 0.22765043377876282},\n",
       "   {'▁pleaded': 0.09178975224494934,\n",
       "    '▁been': 0.4477890729904175,\n",
       "    '▁appeared': 0.1569116711616516},\n",
       "   {'▁refused': 0.04165413975715637,\n",
       "    '▁ordered': 0.19564737379550934,\n",
       "    '▁denied': 0.5309064984321594},\n",
       "   {'▁release': 0.0013296786928549409,\n",
       "    '▁bond': 0.005088170059025288,\n",
       "    '▁bail': 0.9483442902565002},\n",
       "   {',': 0.03317275643348694, '▁by': 0.1073116585612297, '.': 0.7604940533638},\n",
       "   {'▁The': 6.332824705168605e-05,\n",
       "    '</s>': 0.9272734522819519,\n",
       "    '.': 7.498930062865838e-05}]},\n",
       " {'document': 'The 48-year-old former Arsenal goalkeeper played for the Royals for four years.\\nHe was appointed youth academy director in 2000 and has been director of football since 2003.\\nA West Brom statement said: \"He played a key role in the Championship club twice winning promotion to the Premier League in 2006 and 2012.\"',\n",
       "  'generated_summary': 'West Brom have appointed Reading director of football Nick Hammond as their new head coach.',\n",
       "  'summary_token_probs': [{'▁Former': 0.1021934375166893,\n",
       "    '▁Reading': 0.13672815263271332,\n",
       "    '▁West': 0.3758265972137451},\n",
       "   {'▁Ham': 0.0022582849487662315,\n",
       "    '▁Brom': 0.6563436985015869,\n",
       "    '▁Bromwich': 0.3223938047885895},\n",
       "   {'▁director': 0.1486688256263733,\n",
       "    '▁have': 0.32000941038131714,\n",
       "    \"'\": 0.15512751042842865},\n",
       "   {'▁confirmed': 0.10667479783296585,\n",
       "    '▁appointed': 0.5629857182502747,\n",
       "    '▁named': 0.1374897062778473},\n",
       "   {'▁ex': 0.012552681379020214,\n",
       "    '▁Reading': 0.4821060001850128,\n",
       "    '▁former': 0.3911462128162384},\n",
       "   {'▁boss': 0.050467804074287415,\n",
       "    \"'\": 0.28495895862579346,\n",
       "    '▁director': 0.45971211791038513},\n",
       "   {'▁and': 0.0019829533994197845,\n",
       "    '▁Chris': 0.0022781032603234053,\n",
       "    '▁of': 0.8813804388046265},\n",
       "   {'▁rugby': 0.0021453152876347303,\n",
       "    '▁Football': 0.003481058171018958,\n",
       "    '▁football': 0.9409065246582031},\n",
       "   {'▁Paul': 0.04699036478996277,\n",
       "    '▁Steve': 0.09641291201114655,\n",
       "    '▁Nick': 0.21064096689224243},\n",
       "   {'▁Powell': 0.04762494936585426,\n",
       "    '▁Black': 0.05642753466963768,\n",
       "    '▁Hammond': 0.7262939214706421},\n",
       "   {'▁in': 0.0050231534987688065,\n",
       "    '▁to': 0.04070688411593437,\n",
       "    '▁as': 0.850091814994812},\n",
       "   {'▁assistant': 0.03743184357881546,\n",
       "    '▁their': 0.7262060642242432,\n",
       "    '▁the': 0.03817055746912956},\n",
       "   {'▁head': 0.009419913403689861,\n",
       "    '▁new': 0.911216676235199,\n",
       "    '▁assistant': 0.011399498209357262},\n",
       "   {'▁manager': 0.118656225502491,\n",
       "    '▁first': 0.13887925446033478,\n",
       "    '▁head': 0.40777599811553955},\n",
       "   {'▁football': 0.001614608452655375,\n",
       "    '▁of': 0.05606364086270332,\n",
       "    '▁coach': 0.890636146068573},\n",
       "   {',': 0.06682197749614716,\n",
       "    '.': 0.5826202034950256,\n",
       "    '▁on': 0.09497235715389252},\n",
       "   {'▁\"': 4.2901188862742856e-05,\n",
       "    '.': 9.863796731224284e-05,\n",
       "    '</s>': 0.9206230640411377}]},\n",
       " {'document': 'Restoring the function of the organ - which helps control blood sugar levels - reversed symptoms of diabetes in animal experiments.\\nThe study, published in the journal Cell, says the diet reboots the body.\\nExperts said the findings were \"potentially very exciting\" as they could become a new treatment for the disease.\\nThe experiments were on mice put on a modified form of the \"fasting-mimicking diet\".\\nWhen people go on it they spend five days on a low calorie, low protein, low carbohydrate but high unsaturated-fat diet.\\nIt resembles a vegan diet with nuts and soups, but with around 800 to 1,100 calories a day.\\nThen they have 25 days eating what they want - so overall it mimics periods of feast and famine.\\nPrevious research has suggested it can slow the pace of ageing.\\nBut animal experiments showed the diet regenerated a special type of cell in the pancreas called a beta cell.\\nThese are the cells that detect sugar in the blood and release the hormone insulin if it gets too high.\\nDr Valter Longo, from the University of Southern California, said: \"Our conclusion is that by pushing the mice into an extreme state and then bringing them back - by starving them and then feeding them again - the cells in the pancreas are triggered to use some kind of developmental reprogramming that rebuilds the part of the organ that\\'s no longer functioning.\"\\nThere were benefits in both type 1 and type 2 diabetes in the mouse experiments.\\nType 1 is caused by the immune system destroying beta cells and type 2 is largely caused by lifestyle and the body no longer responding to insulin.\\nFurther tests on tissue samples from people with type 1 diabetes produced similar effects.\\nDr Longo said: \"Medically, these findings have the potential to be very important because we\\'ve shown - at least in mouse models - that you can use diet to reverse the symptoms of diabetes.\\n\"Scientifically, the findings are perhaps even more important because we\\'ve shown that you can use diet to reprogram cells without having to make any genetic alterations.\"\\nBBC reporter Peter Bowes took part in a separate trial with Dr Valter Longo.\\nHe said: \"During each five-day fasting cycle, when I ate about a quarter of the average person\\'s diet, I lost between 2kg and 4kg (4.4-8.8lbs).\\n\"But before the next cycle came round, 25 days of eating normally had returned me almost to my original weight.\\n\"But not all consequences of the diet faded so quickly.\"\\nHis blood pressure was lower as was a hormone called IGF-1, which is linked to some cancers.\\nHe said: \"The very small meals I was given during the five-day fast were far from gourmet cooking, but I was glad to have something to eat\"\\nPeter Bowes: Fasting for science\\nPeter Bowes: Intermittent fasting and the good things it did to my body\\nSeparate trials of the diet in people have been shown to improve blood sugar levels. The latest findings help to explain why.\\nHowever, Dr Longo said people should not rush off and crash diet.\\nHe told the BBC: \"It boils down to do not try this at home, this is so much more sophisticated than people realise.\"\\nHe said people could \"get into trouble\" with their health if it was done without medical guidance.\\nDr Emily Burns, research communications manager at Diabetes UK, said: \"This is potentially very exciting news, but we need to see if the results hold true in humans before we\\'ll know more about what it means for people with diabetes.\\n\"People with type-1 and type-2 diabetes would benefit immensely from treatments that can repair or regenerate insulin-producing cells in the pancreas.\"\\nFollow James on Twitter.',\n",
       "  'generated_summary': 'A high-calorie diet that mimics famine can regenerate the pancreas, US scientists say.',\n",
       "  'summary_token_probs': [{'▁Fast': 0.017989788204431534,\n",
       "    '▁A': 0.5080286264419556,\n",
       "    '▁The': 0.2851622700691223},\n",
       "   {'▁low': 0.11085092276334763,\n",
       "    '▁diet': 0.13187547028064728,\n",
       "    '▁high': 0.13223262131214142},\n",
       "   {'▁fat': 0.0064838784746825695,\n",
       "    '▁calorie': 0.04954179748892784,\n",
       "    '-': 0.8612539172172546},\n",
       "   {'carbohydrate': 0.027305105701088905,\n",
       "    'fat': 0.2612987756729126,\n",
       "    'calorie': 0.6095458269119263},\n",
       "   {'▁\"': 0.03147401660680771,\n",
       "    '▁diet': 0.5352458953857422,\n",
       "    ',': 0.277593195438385},\n",
       "   {'▁which': 0.075015589594841,\n",
       "    '▁can': 0.1862523853778839,\n",
       "    '▁that': 0.3312217593193054},\n",
       "   {'▁includes': 0.0038098329678177834,\n",
       "    '▁resembles': 0.015741098672151566,\n",
       "    '▁mimics': 0.8717220425605774},\n",
       "   {'▁the': 0.0678253024816513,\n",
       "    '▁a': 0.1654101461172104,\n",
       "    '▁famine': 0.40547698736190796},\n",
       "   {'▁has': 0.06205296143889427,\n",
       "    '▁can': 0.6433141231536865,\n",
       "    '▁may': 0.08012349903583527},\n",
       "   {'▁turn': 0.04472120851278305,\n",
       "    '▁\"': 0.0569496788084507,\n",
       "    '▁regenerate': 0.30242758989334106},\n",
       "   {'▁damaged': 0.09043093025684357,\n",
       "    '▁parts': 0.107944056391716,\n",
       "    '▁the': 0.38754773139953613},\n",
       "   {'▁part': 0.02684798091650009,\n",
       "    '▁pancreas': 0.7312498688697815,\n",
       "    '▁damaged': 0.07382874935865402},\n",
       "   {'▁and': 0.019185593351721764,\n",
       "    '▁in': 0.12262051552534103,\n",
       "    ',': 0.7541837692260742},\n",
       "   {'▁scientists': 0.12426663935184479,\n",
       "    '▁US': 0.28088653087615967,\n",
       "    '▁say': 0.20705962181091309},\n",
       "   {'▁research': 0.23396334052085876,\n",
       "    '▁scientists': 0.34277093410491943,\n",
       "    '▁researchers': 0.3229208290576935},\n",
       "   {'▁suggest': 0.09139368683099747,\n",
       "    '▁have': 0.17819902300834656,\n",
       "    '▁say': 0.42053690552711487},\n",
       "   {',': 0.0007838623714633286,\n",
       "    '</s>': 0.0011747569078579545,\n",
       "    '.': 0.9092552661895752},\n",
       "   {'▁Dr': 0.0006981710321269929,\n",
       "    '▁\"': 0.0009599138284102082,\n",
       "    '</s>': 0.9313077330589294}]}]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inferences"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8a34755",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Registering Model With Arthur"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff737732",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Setting Up Connection\n",
    "Supply your login to authenticate with the platform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c7368682",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Please enter password for admin: ········\n"
     ]
    }
   ],
   "source": [
    "from arthurai import ArthurAI\n",
    "# connect to Arthur\n",
    "# UNCOMMENT the two lines below and enter your details\n",
    "arthur = ArthurAI(\n",
    "    url=\"https://dev.arthur.ai\",  # you can also pass this through the ARTHUR_ENDPOINT_URL environment variable\n",
    "    login=\"admin\",  # you can also pass this through the ARTHUR_LOGIN environment variable\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db1d29f4",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Registering Model Type"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2d68c19",
   "metadata": {},
   "source": [
    "Next, we need to register the schema of the model with Arthur. Since this is a text to text model, we need to supply:\n",
    "\n",
    "* a name for the model's input text\n",
    "* a name for the model's output text\n",
    "* optionally, additional column names for tokenized text and likelihoods\n",
    "* optionally, ground truth or human generated texts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "757fe858",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We'll instantiate an [`ArthurModel`](https://docs.arthur.ai/sdk/sdk_v3/apiref/arthurai.core.models.ArthurModel.html) with the `ArthurAI.model()` method, which constructs a new local `ArthurModel` object. Later we'll use `ArthurModel.save()` to register this model with the Arthur platform.\n",
    "\n",
    "We give the model a user-friendly `display_name` and allow the unique `partner_model_id` field to be automatically generated, but you can supply a unique identifier if it helps you map your models in Arthur to your other MLOps systems.\n",
    "\n",
    "The `InputType` of a model specifies the general type of data your model ingests. The `OutputType` of a model specifies the modeling task at hand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b06ec787",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-09-02T19:08:20.533507Z",
     "start_time": "2021-09-02T19:08:20.531272Z"
    },
    "pycharm": {
     "name": "#%%\n"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from arthurai.common.constants import InputType, OutputType\n",
    "\n",
    "# instantiate an ArthurModel object to be registered \n",
    "# with the Arthur platform with a name and a type\n",
    "arthur_model = arthur.model(\n",
    "    partner_model_id=f\"NewsHeadlineSummarization_QS-{datetime.now().strftime('%Y%m%d%H%M%S')}\",\n",
    "    display_name=\"News Headline Generation\",\n",
    "    input_type=InputType.NLP,\n",
    "    output_type=OutputType.TokenSequence\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37c8628e",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We have just registered a streaming model, which is the default for an ArthurModel. A streaming model receives instances of data as they come into the deployed model. A batch model, in contrast, receives data in groups, and is often preferred if your model runs as a job rather than operating in realtime or over a data stream. To indicated a batch model, simply add an `is_batch=True` parameter to the call above.\n",
    "\n",
    "You can read more about [streaming and batch models in our documentation](https://docs.arthur.ai/user-guide/basic_concepts.html#basic-concepts-streaming-vs-batch)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cb2b457",
   "metadata": {},
   "source": [
    "### Building the model by specifying attributes\n",
    "\n",
    "We use a helper function to register the model attributes for the input text (`input_column`) and output text (`output_text_column`) the model will process.\n",
    "\n",
    "In addition, we register an attribute for the `output_likelihood_column`, which will be the attribute from our data that represents the likelihood of generated tokens, as well as an attribute called `ground_truth_text_column`, which will be the ground truth we evaluate our output text against."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8cfb8a7b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>stage</th>\n",
       "      <th>value_type</th>\n",
       "      <th>categorical</th>\n",
       "      <th>is_unique</th>\n",
       "      <th>categories</th>\n",
       "      <th>bins</th>\n",
       "      <th>range</th>\n",
       "      <th>monitor_for_bias</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>document</td>\n",
       "      <td>PIPELINE_INPUT</td>\n",
       "      <td>UNSTRUCTURED_TEXT</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>summary_token_probs</td>\n",
       "      <td>PREDICTED_VALUE</td>\n",
       "      <td>TOKEN_LIKELIHOODS</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>generated_summary</td>\n",
       "      <td>PREDICTED_VALUE</td>\n",
       "      <td>UNSTRUCTURED_TEXT</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>summary</td>\n",
       "      <td>GROUND_TRUTH</td>\n",
       "      <td>UNSTRUCTURED_TEXT</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  name            stage         value_type categorical  \\\n",
       "0             document   PIPELINE_INPUT  UNSTRUCTURED_TEXT        True   \n",
       "1  summary_token_probs  PREDICTED_VALUE  TOKEN_LIKELIHOODS       False   \n",
       "2    generated_summary  PREDICTED_VALUE  UNSTRUCTURED_TEXT        True   \n",
       "3              summary     GROUND_TRUTH  UNSTRUCTURED_TEXT        True   \n",
       "\n",
       "  is_unique categories  bins         range monitor_for_bias  \n",
       "0      True         []  None  [None, None]            False  \n",
       "1     False         []  None  [None, None]            False  \n",
       "2     False         []  None  [None, None]            False  \n",
       "3      True         []  None  [None, None]            False  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arthur_model.build_token_sequence_model(input_column ='document',\n",
    "                                 output_text_column='generated_summary',\n",
    "                                 output_likelihood_column='summary_token_probs',\n",
    "                                 ground_truth_text_column='summary')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74b92e62",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Saving the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb1c1bba",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Before saving, be sure to review your model to make sure everything is correct. We already saw the model schema returned by `ArthurModel.build()`, but we have since changed our attribute congiruations. Therefore we call `ArthurModel.review()` to see that our changed attributes look correct before saving to the platform. See the [onboarding walkthrough on the Arthur docs](https://docs.arthur.ai/user-guide/walkthroughs/model-onboarding/index.html#review-model) for tips on reviewing your model.\n",
    "\n",
    "Note that while we capture the ranges of the attributes in this schema, they don’t need to be exact and won’t affect any performance calculations. They’re used as metadata to configure plots in the online Arthur dashboard, but never affect data drift or any other computations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "baaeedba",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>stage</th>\n",
       "      <th>value_type</th>\n",
       "      <th>categorical</th>\n",
       "      <th>is_unique</th>\n",
       "      <th>categories</th>\n",
       "      <th>bins</th>\n",
       "      <th>range</th>\n",
       "      <th>monitor_for_bias</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>document</td>\n",
       "      <td>PIPELINE_INPUT</td>\n",
       "      <td>UNSTRUCTURED_TEXT</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>summary_token_probs</td>\n",
       "      <td>PREDICTED_VALUE</td>\n",
       "      <td>TOKEN_LIKELIHOODS</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>generated_summary</td>\n",
       "      <td>PREDICTED_VALUE</td>\n",
       "      <td>UNSTRUCTURED_TEXT</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>summary</td>\n",
       "      <td>GROUND_TRUTH</td>\n",
       "      <td>UNSTRUCTURED_TEXT</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>[]</td>\n",
       "      <td>None</td>\n",
       "      <td>[None, None]</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  name            stage         value_type categorical  \\\n",
       "0             document   PIPELINE_INPUT  UNSTRUCTURED_TEXT        True   \n",
       "1  summary_token_probs  PREDICTED_VALUE  TOKEN_LIKELIHOODS       False   \n",
       "2    generated_summary  PREDICTED_VALUE  UNSTRUCTURED_TEXT        True   \n",
       "3              summary     GROUND_TRUTH  UNSTRUCTURED_TEXT        True   \n",
       "\n",
       "  is_unique categories  bins         range monitor_for_bias  \n",
       "0      True         []  None  [None, None]            False  \n",
       "1     False         []  None  [None, None]            False  \n",
       "2     False         []  None  [None, None]            False  \n",
       "3      True         []  None  [None, None]            False  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# review the model attribute properties in the model schema\n",
    "arthur_model.review()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d0635e3c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'document': 'Prison Link Cymru had 1,099 referrals in 2015-16 and said some ex-offenders were living rough for up to a year before finding suitable accommodation.\\nWorkers at the charity claim investment in housing would be cheaper than jailing homeless repeat offenders.\\nThe Welsh Government said more people than ever were getting help to address housing problems.\\nChanges to the Housing Act in Wales, introduced in 2015, removed the right for prison leavers to be given priority for accommodation.\\nPrison Link Cymru, which helps people find accommodation after their release, said things were generally good for women because issues such as children or domestic violence were now considered.\\nHowever, the same could not be said for men, the charity said, because issues which often affect them, such as post traumatic stress disorder or drug dependency, were often viewed as less of a priority.\\nAndrew Stevens, who works in Welsh prisons trying to secure housing for prison leavers, said the need for accommodation was \"chronic\".\\n\"There\\'s a desperate need for it, finding suitable accommodation for those leaving prison there is just a lack of it everywhere,\" he said.\\n\"It could take six months to a year, without a lot of help they could be on the streets for six months.\\n\"When you think of the consequences of either being on the street, especially with the cold weather at the moment or you may have a roof over your head, sometimes there is only one choice.\"\\nMr Stevens believes building more one-bedroom flats could help ease the problem.\\n\"The average price is a hundred pounds a week to keep someone in a rented flat, prison is a lot more than that so I would imagine it would save the public purse quite a few pounds,\" he said.\\nOfficial figures show 830 one-bedroom properties were built in the year to March 2016, of an overall total of 6,900 new properties in Wales.\\nMarc, 50, who has been in and out of prison for the past 20 years for burglary offences, said he struggled to find accommodation each time he was released.\\nHe said he would ask himself: \"Where am I going to stay? Where am I going to live? Have I got somewhere where I can see my daughter.\"\\n\"You\\'re put out among the same sort of people doing the same sort of thing, and it\\'s difficult, it\\'s difficult to get away from it. It\\'s like every man for himself, there\\'s nothing.\"\\nMarc has now found stable accommodation with homeless charity Emmaus and said it had been life changing.\\n\"You feel safe, you got hot food, you\\'ve got company of people in similar situations to yourself but all dealing with different issues. It\\'s a constructive, helpful atmosphere,\" he said.\\nTom Clarke, chief executive of Emmaus South Wales, agreed there was not enough support available.\\n\"We do still see [people] homeless on the streets, so clearly they haven\\'t got accommodation and haven\\'t got provision,\" he said.\\n\"I think the key is connecting people with the services they need. I don\\'t delude myself that Emmaus can offer a one size fits all for everyone, we can\\'t.\\n\"But there must be other opportunities and given suitable encouragement I believe that can and should happen.\"\\nA Welsh Government spokesman said the national pathway for homeless services to children, young people and adults in the secure estate had prevented many people from losing their home whilst serving their prison sentence.\\nIt added there were already significant demands for one-bedroom flats across the public and private sector and it was providing 20,000 new affordable homes in the next five years.',\n",
       "  'generated_summary': 'A charity has said it has seen a \"desperate\" need for housing for homeless people in Wales.',\n",
       "  'summary_token_probs': [{'▁More': 0.14739905297756195,\n",
       "    '▁The': 0.15617099404335022,\n",
       "    '▁A': 0.1668444722890854},\n",
       "   {'▁homeless': 0.05522293969988823,\n",
       "    '▁\"': 0.2275698035955429,\n",
       "    '▁charity': 0.4529936909675598},\n",
       "   {'▁which': 0.10341085493564606,\n",
       "    '▁says': 0.11643338948488235,\n",
       "    '▁has': 0.3602505624294281},\n",
       "   {'▁called': 0.13903598487377167,\n",
       "    '▁warned': 0.18098242580890656,\n",
       "    '▁said': 0.3734981119632721},\n",
       "   {'▁there': 0.13141843676567078,\n",
       "    '▁homeless': 0.1574133038520813,\n",
       "    '▁it': 0.20769721269607544},\n",
       "   {'▁received': 0.05590398982167244,\n",
       "    '▁is': 0.15843385457992554,\n",
       "    '▁has': 0.34066715836524963},\n",
       "   {'▁been': 0.04716835170984268,\n",
       "    '▁had': 0.12852980196475983,\n",
       "    '▁seen': 0.5675621032714844},\n",
       "   {'▁\"': 0.04353978484869003,\n",
       "    '▁a': 0.7346400618553162,\n",
       "    '▁an': 0.07529696822166443},\n",
       "   {'▁40%': 0.027278512716293335,\n",
       "    '▁rise': 0.09127599000930786,\n",
       "    '▁\"': 0.3018726110458374},\n",
       "   {'dist': 0.02170277014374733,\n",
       "    'di': 0.06995752453804016,\n",
       "    'd': 0.7411702275276184},\n",
       "   {'anger': 0.00111184932757169,\n",
       "    'read': 0.063139908015728,\n",
       "    'esper': 0.9169283509254456},\n",
       "   {'ation': 0.003937354311347008,\n",
       "    'ately': 0.07379543036222458,\n",
       "    'ate': 0.8817391395568848},\n",
       "   {'▁Need': 0.004620884545147419,\n",
       "    '▁need': 0.3047276437282562,\n",
       "    '\"': 0.6056327223777771},\n",
       "   {'▁increase': 0.15280920267105103,\n",
       "    '▁rise': 0.1943100243806839,\n",
       "    '▁need': 0.19728246331214905},\n",
       "   {'▁in': 0.0029658828862011433,\n",
       "    '▁to': 0.055987462401390076,\n",
       "    '▁for': 0.8625580072402954},\n",
       "   {'▁affordable': 0.11721357703208923,\n",
       "    '▁more': 0.24277189373970032,\n",
       "    '▁housing': 0.2509273290634155},\n",
       "   {'▁accommodation': 0.02548299916088581,\n",
       "    '▁for': 0.7337383031845093,\n",
       "    '▁help': 0.02847117930650711},\n",
       "   {'▁former': 0.10322487354278564,\n",
       "    '▁men': 0.12142349034547806,\n",
       "    '▁homeless': 0.24715575575828552},\n",
       "   {'▁men': 0.14614203572273254,\n",
       "    '▁former': 0.18016108870506287,\n",
       "    '▁people': 0.1847544014453888},\n",
       "   {'▁leaving': 0.07088559865951538,\n",
       "    '▁in': 0.20193831622600555,\n",
       "    '▁after': 0.15997382998466492},\n",
       "   {'▁Welsh': 0.013427690602838993,\n",
       "    '▁Wales': 0.8253042697906494,\n",
       "    '▁the': 0.023584609851241112},\n",
       "   {'▁over': 0.021732943132519722,\n",
       "    ',': 0.05551162362098694,\n",
       "    '.': 0.7140709757804871},\n",
       "   {'.': 9.655363828642294e-05,\n",
       "    '▁\"': 0.0003459836298134178,\n",
       "    '</s>': 0.9349042773246765}]},\n",
       " {'document': 'Officers searched properties in the Waterfront Park and Colonsay View areas of the city on Wednesday.\\nDetectives said three firearms, ammunition and a five-figure sum of money were recovered.\\nA 26-year-old man who was arrested and charged appeared at Edinburgh Sheriff Court on Thursday.',\n",
       "  'generated_summary': 'A man has been charged after firearms and money were seized by police in Edinburgh.',\n",
       "  'summary_token_probs': [{'▁An': 0.009951921179890633,\n",
       "    '▁Police': 0.04644511640071869,\n",
       "    '▁A': 0.832900881767273},\n",
       "   {'▁woman': 0.0006877636187709868,\n",
       "    '▁26-': 0.0016444444190710783,\n",
       "    '▁man': 0.9207310080528259},\n",
       "   {'▁who': 0.0009694563923403621,\n",
       "    '▁has': 0.9235783219337463,\n",
       "    '▁is': 0.008910998702049255},\n",
       "   {'▁died': 0.0005632047541439533,\n",
       "    '▁been': 0.9080358743667603,\n",
       "    '▁appeared': 0.01719382032752037},\n",
       "   {'▁accused': 0.004897481761872768,\n",
       "    '▁arrested': 0.1702582687139511,\n",
       "    '▁charged': 0.7189240455627441},\n",
       "   {'▁following': 0.14367060363292694,\n",
       "    '▁after': 0.4145500361919403,\n",
       "    '▁in': 0.1933813840150833},\n",
       "   {'▁a': 0.10204783827066422,\n",
       "    '▁police': 0.2405659407377243,\n",
       "    '▁firearms': 0.36837926506996155},\n",
       "   {',': 0.03414512425661087,\n",
       "    '▁were': 0.0824623852968216,\n",
       "    '▁and': 0.7969264388084412},\n",
       "   {'▁a': 0.1313030868768692,\n",
       "    '▁cash': 0.1679534614086151,\n",
       "    '▁money': 0.2802054286003113},\n",
       "   {'▁was': 0.007942081429064274,\n",
       "    '▁with': 0.011951232329010963,\n",
       "    '▁were': 0.8840651512145996},\n",
       "   {'▁found': 0.08153212070465088,\n",
       "    '▁recovered': 0.26476386189460754,\n",
       "    '▁seized': 0.5870693922042847},\n",
       "   {'▁during': 0.06771715730428696,\n",
       "    '▁in': 0.31669050455093384,\n",
       "    '▁by': 0.4764735698699951},\n",
       "   {'▁Police': 0.015553689561784267,\n",
       "    '▁police': 0.8184274435043335,\n",
       "    '▁armed': 0.01920345425605774},\n",
       "   {'▁during': 0.015437500551342964,\n",
       "    '▁investigating': 0.20598174631595612,\n",
       "    '▁in': 0.5863365530967712},\n",
       "   {'▁the': 0.011391673237085342,\n",
       "    '▁an': 0.016664989292621613,\n",
       "    '▁Edinburgh': 0.8543614149093628},\n",
       "   {'▁in': 0.0013960031792521477,\n",
       "    ',': 0.00601107906550169,\n",
       "    '.': 0.9181209206581116},\n",
       "   {')': 3.914666740456596e-05,\n",
       "    '.': 8.346551476279274e-05,\n",
       "    '</s>': 0.9207320213317871}]},\n",
       " {'document': 'Jordan Hill, Brittany Covington and Tesfaye Cooper, all 18, and Tanishia Covington, 24, appeared in a Chicago court on Friday.\\nThe four have been charged with hate crimes and aggravated kidnapping and battery, among other things.\\nAn online fundraiser for their victim has collected $51,000 (Â£42,500) so far.\\nDenying the four suspects bail, Judge Maria Kuriakos Ciesil asked: \"Where was your sense of decency?\"\\nProsecutors told the court the beating started in a van and continued at a house, where the suspects allegedly forced the 18-year-old white victim, who suffers from schizophrenia and attention deficit disorder, to drink toilet water and kiss the floor.\\nPolice allege the van was earlier stolen by Mr Hill, who is also accused of demanding $300 from the victim\\'s mother while they held him captive, according to the Chicago Tribune.\\nThe court was also told the suspects stuffed a sock into his mouth, taped his mouth shut and bound his hands with a belt.\\nIn a video made for Facebook Live which was watched millions of times, the assailants can be heard making derogatory statements against white people and Donald Trump.\\nThe victim had been dropped off at a McDonalds to meet Mr Hill - who was one of his friends - on 31 December.\\nHe was found by a police officer on Tuesday, 3 January, a day after he was reported missing by his parents.\\nProsecutors say the suspects each face two hate crimes counts, one because of the victim\\'s race and the other because of his disabilities.',\n",
       "  'generated_summary': 'Four black people accused of beating a mentally disabled man and broadcasting the attack on Facebook have been denied bail.',\n",
       "  'summary_token_probs': [{'▁The': 0.031043965369462967,\n",
       "    '▁A': 0.07126393914222717,\n",
       "    '▁Four': 0.7035706639289856},\n",
       "   {'▁white': 0.024705447256565094,\n",
       "    '▁black': 0.7900169491767883,\n",
       "    '▁people': 0.06329590082168579},\n",
       "   {'▁teenagers': 0.029675383120775223,\n",
       "    '▁Americans': 0.05551106482744217,\n",
       "    '▁people': 0.650540292263031},\n",
       "   {'▁in': 0.06361347436904907,\n",
       "    '▁have': 0.2720164358615875,\n",
       "    '▁accused': 0.44093674421310425},\n",
       "   {'▁over': 0.0009448024211451411,\n",
       "    '▁in': 0.008143513463437557,\n",
       "    '▁of': 0.9236844182014465},\n",
       "   {'▁torture': 0.030659008771181107,\n",
       "    '▁torturing': 0.20328350365161896,\n",
       "    '▁beating': 0.4224582612514496},\n",
       "   {'▁an': 0.01298152282834053,\n",
       "    '▁a': 0.692047655582428,\n",
       "    '▁and': 0.225032240152359},\n",
       "   {'▁white': 0.05361773446202278,\n",
       "    '▁disabled': 0.20468999445438385,\n",
       "    '▁mentally': 0.47232839465141296},\n",
       "   {'▁ill': 0.03281280770897865,\n",
       "    '-': 0.0637507289648056,\n",
       "    '▁disabled': 0.7702006697654724},\n",
       "   {'▁black': 0.05080603435635567,\n",
       "    '▁white': 0.26410579681396484,\n",
       "    '▁man': 0.4465253949165344},\n",
       "   {'▁for': 0.024473298341035843,\n",
       "    '▁in': 0.30203190445899963,\n",
       "    '▁and': 0.5432246923446655},\n",
       "   {'▁broadcast': 0.08738340437412262,\n",
       "    '▁posting': 0.14907076954841614,\n",
       "    '▁broadcasting': 0.6322811245918274},\n",
       "   {'▁their': 0.011422928422689438,\n",
       "    '▁the': 0.7138702869415283,\n",
       "    '▁it': 0.18000441789627075},\n",
       "   {'▁ordeal': 0.04824370890855789,\n",
       "    '▁assault': 0.18628427386283875,\n",
       "    '▁attack': 0.4448767304420471},\n",
       "   {'▁online': 0.05846288800239563,\n",
       "    '▁live': 0.17527522146701813,\n",
       "    '▁on': 0.678325355052948},\n",
       "   {'Facebook': 0.0025630691088736057,\n",
       "    '▁social': 0.0744195356965065,\n",
       "    '▁Facebook': 0.867451548576355},\n",
       "   {'▁live': 0.08001943677663803,\n",
       "    '▁have': 0.5348222851753235,\n",
       "    '▁Live': 0.22765043377876282},\n",
       "   {'▁pleaded': 0.09178975224494934,\n",
       "    '▁been': 0.4477890729904175,\n",
       "    '▁appeared': 0.1569116711616516},\n",
       "   {'▁refused': 0.04165413975715637,\n",
       "    '▁ordered': 0.19564737379550934,\n",
       "    '▁denied': 0.5309064984321594},\n",
       "   {'▁release': 0.0013296786928549409,\n",
       "    '▁bond': 0.005088170059025288,\n",
       "    '▁bail': 0.9483442902565002},\n",
       "   {',': 0.03317275643348694, '▁by': 0.1073116585612297, '.': 0.7604940533638},\n",
       "   {'▁The': 6.332824705168605e-05,\n",
       "    '</s>': 0.9272734522819519,\n",
       "    '.': 7.498930062865838e-05}]},\n",
       " {'document': 'The 48-year-old former Arsenal goalkeeper played for the Royals for four years.\\nHe was appointed youth academy director in 2000 and has been director of football since 2003.\\nA West Brom statement said: \"He played a key role in the Championship club twice winning promotion to the Premier League in 2006 and 2012.\"',\n",
       "  'generated_summary': 'West Brom have appointed Reading director of football Nick Hammond as their new head coach.',\n",
       "  'summary_token_probs': [{'▁Former': 0.1021934375166893,\n",
       "    '▁Reading': 0.13672815263271332,\n",
       "    '▁West': 0.3758265972137451},\n",
       "   {'▁Ham': 0.0022582849487662315,\n",
       "    '▁Brom': 0.6563436985015869,\n",
       "    '▁Bromwich': 0.3223938047885895},\n",
       "   {'▁director': 0.1486688256263733,\n",
       "    '▁have': 0.32000941038131714,\n",
       "    \"'\": 0.15512751042842865},\n",
       "   {'▁confirmed': 0.10667479783296585,\n",
       "    '▁appointed': 0.5629857182502747,\n",
       "    '▁named': 0.1374897062778473},\n",
       "   {'▁ex': 0.012552681379020214,\n",
       "    '▁Reading': 0.4821060001850128,\n",
       "    '▁former': 0.3911462128162384},\n",
       "   {'▁boss': 0.050467804074287415,\n",
       "    \"'\": 0.28495895862579346,\n",
       "    '▁director': 0.45971211791038513},\n",
       "   {'▁and': 0.0019829533994197845,\n",
       "    '▁Chris': 0.0022781032603234053,\n",
       "    '▁of': 0.8813804388046265},\n",
       "   {'▁rugby': 0.0021453152876347303,\n",
       "    '▁Football': 0.003481058171018958,\n",
       "    '▁football': 0.9409065246582031},\n",
       "   {'▁Paul': 0.04699036478996277,\n",
       "    '▁Steve': 0.09641291201114655,\n",
       "    '▁Nick': 0.21064096689224243},\n",
       "   {'▁Powell': 0.04762494936585426,\n",
       "    '▁Black': 0.05642753466963768,\n",
       "    '▁Hammond': 0.7262939214706421},\n",
       "   {'▁in': 0.0050231534987688065,\n",
       "    '▁to': 0.04070688411593437,\n",
       "    '▁as': 0.850091814994812},\n",
       "   {'▁assistant': 0.03743184357881546,\n",
       "    '▁their': 0.7262060642242432,\n",
       "    '▁the': 0.03817055746912956},\n",
       "   {'▁head': 0.009419913403689861,\n",
       "    '▁new': 0.911216676235199,\n",
       "    '▁assistant': 0.011399498209357262},\n",
       "   {'▁manager': 0.118656225502491,\n",
       "    '▁first': 0.13887925446033478,\n",
       "    '▁head': 0.40777599811553955},\n",
       "   {'▁football': 0.001614608452655375,\n",
       "    '▁of': 0.05606364086270332,\n",
       "    '▁coach': 0.890636146068573},\n",
       "   {',': 0.06682197749614716,\n",
       "    '.': 0.5826202034950256,\n",
       "    '▁on': 0.09497235715389252},\n",
       "   {'▁\"': 4.2901188862742856e-05,\n",
       "    '.': 9.863796731224284e-05,\n",
       "    '</s>': 0.9206230640411377}]},\n",
       " {'document': 'Restoring the function of the organ - which helps control blood sugar levels - reversed symptoms of diabetes in animal experiments.\\nThe study, published in the journal Cell, says the diet reboots the body.\\nExperts said the findings were \"potentially very exciting\" as they could become a new treatment for the disease.\\nThe experiments were on mice put on a modified form of the \"fasting-mimicking diet\".\\nWhen people go on it they spend five days on a low calorie, low protein, low carbohydrate but high unsaturated-fat diet.\\nIt resembles a vegan diet with nuts and soups, but with around 800 to 1,100 calories a day.\\nThen they have 25 days eating what they want - so overall it mimics periods of feast and famine.\\nPrevious research has suggested it can slow the pace of ageing.\\nBut animal experiments showed the diet regenerated a special type of cell in the pancreas called a beta cell.\\nThese are the cells that detect sugar in the blood and release the hormone insulin if it gets too high.\\nDr Valter Longo, from the University of Southern California, said: \"Our conclusion is that by pushing the mice into an extreme state and then bringing them back - by starving them and then feeding them again - the cells in the pancreas are triggered to use some kind of developmental reprogramming that rebuilds the part of the organ that\\'s no longer functioning.\"\\nThere were benefits in both type 1 and type 2 diabetes in the mouse experiments.\\nType 1 is caused by the immune system destroying beta cells and type 2 is largely caused by lifestyle and the body no longer responding to insulin.\\nFurther tests on tissue samples from people with type 1 diabetes produced similar effects.\\nDr Longo said: \"Medically, these findings have the potential to be very important because we\\'ve shown - at least in mouse models - that you can use diet to reverse the symptoms of diabetes.\\n\"Scientifically, the findings are perhaps even more important because we\\'ve shown that you can use diet to reprogram cells without having to make any genetic alterations.\"\\nBBC reporter Peter Bowes took part in a separate trial with Dr Valter Longo.\\nHe said: \"During each five-day fasting cycle, when I ate about a quarter of the average person\\'s diet, I lost between 2kg and 4kg (4.4-8.8lbs).\\n\"But before the next cycle came round, 25 days of eating normally had returned me almost to my original weight.\\n\"But not all consequences of the diet faded so quickly.\"\\nHis blood pressure was lower as was a hormone called IGF-1, which is linked to some cancers.\\nHe said: \"The very small meals I was given during the five-day fast were far from gourmet cooking, but I was glad to have something to eat\"\\nPeter Bowes: Fasting for science\\nPeter Bowes: Intermittent fasting and the good things it did to my body\\nSeparate trials of the diet in people have been shown to improve blood sugar levels. The latest findings help to explain why.\\nHowever, Dr Longo said people should not rush off and crash diet.\\nHe told the BBC: \"It boils down to do not try this at home, this is so much more sophisticated than people realise.\"\\nHe said people could \"get into trouble\" with their health if it was done without medical guidance.\\nDr Emily Burns, research communications manager at Diabetes UK, said: \"This is potentially very exciting news, but we need to see if the results hold true in humans before we\\'ll know more about what it means for people with diabetes.\\n\"People with type-1 and type-2 diabetes would benefit immensely from treatments that can repair or regenerate insulin-producing cells in the pancreas.\"\\nFollow James on Twitter.',\n",
       "  'generated_summary': 'A high-calorie diet that mimics famine can regenerate the pancreas, US scientists say.',\n",
       "  'summary_token_probs': [{'▁Fast': 0.017989788204431534,\n",
       "    '▁A': 0.5080286264419556,\n",
       "    '▁The': 0.2851622700691223},\n",
       "   {'▁low': 0.11085092276334763,\n",
       "    '▁diet': 0.13187547028064728,\n",
       "    '▁high': 0.13223262131214142},\n",
       "   {'▁fat': 0.0064838784746825695,\n",
       "    '▁calorie': 0.04954179748892784,\n",
       "    '-': 0.8612539172172546},\n",
       "   {'carbohydrate': 0.027305105701088905,\n",
       "    'fat': 0.2612987756729126,\n",
       "    'calorie': 0.6095458269119263},\n",
       "   {'▁\"': 0.03147401660680771,\n",
       "    '▁diet': 0.5352458953857422,\n",
       "    ',': 0.277593195438385},\n",
       "   {'▁which': 0.075015589594841,\n",
       "    '▁can': 0.1862523853778839,\n",
       "    '▁that': 0.3312217593193054},\n",
       "   {'▁includes': 0.0038098329678177834,\n",
       "    '▁resembles': 0.015741098672151566,\n",
       "    '▁mimics': 0.8717220425605774},\n",
       "   {'▁the': 0.0678253024816513,\n",
       "    '▁a': 0.1654101461172104,\n",
       "    '▁famine': 0.40547698736190796},\n",
       "   {'▁has': 0.06205296143889427,\n",
       "    '▁can': 0.6433141231536865,\n",
       "    '▁may': 0.08012349903583527},\n",
       "   {'▁turn': 0.04472120851278305,\n",
       "    '▁\"': 0.0569496788084507,\n",
       "    '▁regenerate': 0.30242758989334106},\n",
       "   {'▁damaged': 0.09043093025684357,\n",
       "    '▁parts': 0.107944056391716,\n",
       "    '▁the': 0.38754773139953613},\n",
       "   {'▁part': 0.02684798091650009,\n",
       "    '▁pancreas': 0.7312498688697815,\n",
       "    '▁damaged': 0.07382874935865402},\n",
       "   {'▁and': 0.019185593351721764,\n",
       "    '▁in': 0.12262051552534103,\n",
       "    ',': 0.7541837692260742},\n",
       "   {'▁scientists': 0.12426663935184479,\n",
       "    '▁US': 0.28088653087615967,\n",
       "    '▁say': 0.20705962181091309},\n",
       "   {'▁research': 0.23396334052085876,\n",
       "    '▁scientists': 0.34277093410491943,\n",
       "    '▁researchers': 0.3229208290576935},\n",
       "   {'▁suggest': 0.09139368683099747,\n",
       "    '▁have': 0.17819902300834656,\n",
       "    '▁say': 0.42053690552711487},\n",
       "   {',': 0.0007838623714633286,\n",
       "    '</s>': 0.0011747569078579545,\n",
       "    '.': 0.9092552661895752},\n",
       "   {'▁Dr': 0.0006981710321269929,\n",
       "    '▁\"': 0.0009599138284102082,\n",
       "    '</s>': 0.9313077330589294}]}]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inferences"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ae9b659",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Now, we save the model. \n",
    "\n",
    "Note that this will be the first call to send data to the Arthur platform so far in this example - no information has been sent yet to the platform.\n",
    "\n",
    "The method `ArthurModel.save()` sends an API request to Arthur to validate your model - if there are any problems with your model schema, this method will result in an error informing you how to correct your model's configuration. If no errors are found, the model will be saved to the platform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4bfcd1d9",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:30:19 - arthurai - We have registered the  model with Arthur and are getting it ready to accept inferences...\n",
      "16:31:05 - arthurai - Model Creation Completed successfully, you can now send Data to Arthur.\n"
     ]
    }
   ],
   "source": [
    "# validate the model and save it onto the Arthur platform\n",
    "model_id = arthur_model.save()\n",
    "with open(\"quickstart_model_id.txt\", \"w\") as f:\n",
    "    f.write(model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f024bd9",
   "metadata": {},
   "source": [
    "### Set reference data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "17d5a02e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:31:05 - arthurai - Starting upload (0.149 MB in 1 files), depending on data size this may take a few minutes\n",
      "16:31:05 - arthurai - Upload completed: reference.parquet\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "({'counts': {'success': 50, 'failure': 0, 'total': 50}, 'failures': [[]]},\n",
       " {'dataset_close_result': {'message': 'success'}})"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arthur_model.set_reference_data(directory_path='./')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82c4bb5d",
   "metadata": {},
   "source": [
    "### Send Inferences\n",
    "\n",
    "One way to send inferences to Arthur is to use the `send_inferences` function, where we can pass in the predictions generated above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c71f193d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:31:05 - arthurai - 5 rows were missing inference_timestamp fields, so the current time was populated\n",
      "16:31:05 - arthurai - 5 rows were missing partner_inference_id fields, so UUIDs were generated, see return values\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'counts': {'failure': 0, 'success': 5, 'total': 5},\n",
       " 'results': [{'message': 'success',\n",
       "   'status': 200,\n",
       "   'partner_inference_id': 'TtQvNoxk6E4Fi8bzTu7tqJ'},\n",
       "  {'message': 'success',\n",
       "   'status': 200,\n",
       "   'partner_inference_id': 'UXpYpRoiT5nfRe24rCRckj'},\n",
       "  {'message': 'success',\n",
       "   'status': 200,\n",
       "   'partner_inference_id': 'hgsTdNLVxyKHCokFrRVbnj'},\n",
       "  {'message': 'success',\n",
       "   'status': 200,\n",
       "   'partner_inference_id': '587bdV9jrGx9Vt7hDoPPzn'},\n",
       "  {'message': 'success',\n",
       "   'status': 200,\n",
       "   'partner_inference_id': 'oQmtoESKKyJfy74icQGbQQ'}]}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arthur_model.send_inferences(inferences)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39a33e0e",
   "metadata": {},
   "source": [
    "We can also send a bulk json inference upload of inferences to the model by providing a path to the location of the inferences file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ffabddf0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:31:05 - arthurai - Starting upload (0.321 MB in 1 files), depending on data size this may take a few minutes\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-19 16:31:05,939 - arthurai.client.http.requests - WARNING - retries was specified as 3 but validation_response_code was not set, response contents will not be evaluated.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:31:06 - arthurai - Upload completed: inferences/inferences.json\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "({'counts': {'success': 50, 'failure': 0, 'total': 50}, 'failures': [[]]},\n",
       " None)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arthur_model.send_bulk_inferences(directory_path='./inferences/')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "954c0892",
   "metadata": {},
   "source": [
    "When your ground truth values are available for your model's inferences, you can update each inference by Partner Inference ID with its corresponding label. You will need the partner_inference_ids for your data so that we can include them with our upload of ground truth values and match each prediction with its corresponding ground truth. Here, we will inspect one ground truth data point and then update in bulk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "0403543d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ground_truth_timestamp': '2023-04-12T01:46:45.432216+00:00',\n",
       " 'ground_truth_data': {'summary': 'Prison Link Cymru had 1,099 referrals in 2015-16 and said some ex-offenders were living rough for up to a year before finding suitable accommodation.\\nWorkers at the charity claim investment in housing would be cheaper than jailing homeless repeat offenders.\\nThe Welsh Government said more people than ever were getting help to address housing problems.\\nChanges to the Housing Act in Wales, introduced in 2015, removed the right for prison leavers to be given priority for accommodation.\\nPrison Link Cymru, which helps people find accommodation after their release, said things were generally good for women because issues such as children or domestic violence were now considered.\\nHowever, the same could not be said for men, the charity said, because issues which often affect them, such as post traumatic stress disorder or drug dependency, were often viewed as less of a priority.\\nAndrew Stevens, who works in Welsh prisons trying to secure housing for prison leavers, said the need for accommodation was \"chronic\".\\n\"There\\'s a desperate need for it, finding suitable accommodation for those leaving prison there is just a lack of it everywhere,\" he said.\\n\"It could take six months to a year, without a lot of help they could be on the streets for six months.\\n\"When you think of the consequences of either being on the street, especially with the cold weather at the moment or you may have a roof over your head, sometimes there is only one choice.\"\\nMr Stevens believes building more one-bedroom flats could help ease the problem.\\n\"The average price is a hundred pounds a week to keep someone in a rented flat, prison is a lot more than that so I would imagine it would save the public purse quite a few pounds,\" he said.\\nOfficial figures show 830 one-bedroom properties were built in the year to March 2016, of an overall total of 6,900 new properties in Wales.\\nMarc, 50, who has been in and out of prison for the past 20 years for burglary offences, said he struggled to find accommodation each time he was released.\\nHe said he would ask himself: \"Where am I going to stay? Where am I going to live? Have I got somewhere where I can see my daughter.\"\\n\"You\\'re put out among the same sort of people doing the same sort of thing, and it\\'s difficult, it\\'s difficult to get away from it. It\\'s like every man for himself, there\\'s nothing.\"\\nMarc has now found stable accommodation with homeless charity Emmaus and said it had been life changing.\\n\"You feel safe, you got hot food, you\\'ve got company of people in similar situations to yourself but all dealing with different issues. It\\'s a constructive, helpful atmosphere,\" he said.\\nTom Clarke, chief executive of Emmaus South Wales, agreed there was not enough support available.\\n\"We do still see [people] homeless on the streets, so clearly they haven\\'t got accommodation and haven\\'t got provision,\" he said.\\n\"I think the key is connecting people with the services they need. I don\\'t delude myself that Emmaus can offer a one size fits all for everyone, we can\\'t.\\n\"But there must be other opportunities and given suitable encouragement I believe that can and should happen.\"\\nA Welsh Government spokesman said the national pathway for homeless services to children, young people and adults in the secure estate had prevented many people from losing their home whilst serving their prison sentence.\\nIt added there were already significant demands for one-bedroom flats across the public and private sector and it was providing 20,000 new affordable homes in the next five years.'},\n",
       " 'partner_inference_id': '0-2023-03-13'}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import json\n",
    "with open('ground_truth/gt.json', 'r') as f:\n",
    "    gt = json.load(f)\n",
    "gt[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "558898de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:31:06 - arthurai - Starting upload (0.126 MB in 1 files), depending on data size this may take a few minutes\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-19 16:31:06,166 - arthurai.client.http.requests - WARNING - retries was specified as 3 but validation_response_code was not set, response contents will not be evaluated.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16:31:06 - arthurai - Upload completed: ground_truth/gt.json\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'counts': {'success': 50, 'failure': 0, 'total': 50}, 'failures': [[]]}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "arthur_model.send_bulk_ground_truths(directory_path='./ground_truth/')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b4d26ea",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## See Model in Dashboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2b1a8396",
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<br> <a style=\"font-size:200%\" href=https://dev.arthur.ai/model/a0c06ef2-0293-498b-aa15-01cb22e6c755/overview>See your model (News Headline Generation) in the Arthur Dashboard</a> <br>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# the code below will render a link for you to view your model in the Arthur Dashboard\n",
    "\n",
    "def render_arthur_model_dashboard_link(arthur, arthur_model):\n",
    "    url = 'https://' + ''.join(arthur.client.api_base_url.split('/')[1:-2])\n",
    "    link_text = f\"See your model ({arthur_model.display_name}) in the Arthur Dashboard\"\n",
    "    href_string = f\"{url}/model/{arthur_model.id}/overview\"\n",
    "    html_string = f'<br> <a style=\"font-size:200%\" href={href_string}>{link_text}</a> <br>'\n",
    "    display(HTML(html_string))\n",
    "\n",
    "render_arthur_model_dashboard_link(arthur, arthur_model) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f51dca1",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "Once your inference data has been uploaded to the platform, you can see your model by following the above link to the model dashboard page to see an overview of the model and browse its inference data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a47d8b1a",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}