7-4,DeepFM模型.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ca129acc",
   "metadata": {},
   "source": [
    "\n",
    "# 7-4，DeepFM网络\n",
    "\n",
    "推荐系统和广告CTR预估主流模型的演化有两条主要路线。\n",
    "\n",
    "第一条是显式建模特征交互，提升模型对交叉特征的捕获能力。(如Wide&Deep,PNN,FNN,DCN,DeepFM,AutoInt等)\n",
    "\n",
    "第二条是加入注意力机制，提升模型的自适应能力和解释性。(如DIN,DIEN,DSIN,FiBiNET,AutoInt等)\n",
    "\n",
    "在所有这些模型中，DeepFM属于性价比非常高的模型（结构简洁，计算高效，指标有竞争力）。\n",
    "\n",
    "张俊林大佬 在2019年的时候甚至建议 沿着 LR->FM->DeepFM->干点别的 这样的路线去迭代推荐系统。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4da1dc29",
   "metadata": {},
   "source": [
    "参考文档：\n",
    "\n",
    "* 《推荐系统CTR预估学习路线》：https://zhuanlan.zhihu.com/p/351078721 \n",
    "\n",
    "* criteo数据集榜单：https://paperswithcode.com/dataset/criteo\n",
    "\n",
    "* DeepFM论文： https://arxiv.org/abs/1703.04247\n",
    "\n",
    "* 《清晰易懂，基于pytorch的DeepFM的完整实验代码》： https://zhuanlan.zhihu.com/p/332786045\n",
    "\n",
    "* torch实现参考：https://github.com/rixwew/pytorch-fm/blob/master/torchfm/model/dfm.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d1ac96f",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "<font color=\"red\">\n",
    " \n",
    "公众号 **算法美食屋** 回复关键词：**pytorch**， 获取本项目源码和所用数据集百度云盘下载链接。\n",
    "    \n",
    "</font> \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "56892bf7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.__version__ =  2.4.0\n",
      "torchkeras.__version__ =  4.0.0\n"
     ]
    }
   ],
   "source": [
    "import torch \n",
    "import torchkeras \n",
    "print(\"torch.__version__ = \", torch.__version__)\n",
    "print(\"torchkeras.__version__ = \", torchkeras.__version__) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e254ad5",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "f31d986e",
   "metadata": {},
   "source": [
    "## 一，DeepFM原理解析"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c2c4ee8",
   "metadata": {},
   "source": [
    "DeepFM继承了DeepWide的主体结构，将高低特征进行融合。\n",
    "\n",
    "其主要创新点有2个。\n",
    "\n",
    "一是将Wide部分替换成了 FM结构，以更有效的捕获特征交互interaction.\n",
    "\n",
    "二是FM中的隐向量 和 Deep部分的 embedding 向量共享权重，减少模型复杂性。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2917d911",
   "metadata": {},
   "source": [
    "![](https://tva1.sinaimg.cn/large/e6c9d24egy1h1brqobat6j21260nq0xl.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "350eb2f8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cdcad12",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3eb3bba2",
   "metadata": {},
   "source": [
    "## 二，DeepFM的pytorch实现"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e601ff88",
   "metadata": {},
   "source": [
    "下面是DeepFM的一个pytorch实现。\n",
    "\n",
    "除了添加了一个并行的MLP模块用于捕获隐式高阶交叉和组合特征外，其余结构基本和FM的实现完全一致。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1b01f881",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch \n",
    "from torch import nn,Tensor \n",
    "import torch.nn.functional as F \n",
    "\n",
    "class NumEmbedding(nn.Module):\n",
    "    \"\"\"\n",
    "    连续特征用linear层编码\n",
    "    输入shape: [batch_size,features_num(n), d_in], # d_in 通常是1\n",
    "    输出shape: [batch_size,features_num(n), d_out]\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, n: int, d_in: int, d_out: int, bias: bool = False) -> None:\n",
    "        super().__init__()\n",
    "        self.weight = nn.Parameter(Tensor(n, d_in, d_out))\n",
    "        self.bias = nn.Parameter(Tensor(n, d_out)) if bias else None\n",
    "        with torch.no_grad():\n",
    "            for i in range(n):\n",
    "                layer = nn.Linear(d_in, d_out)\n",
    "                self.weight[i] = layer.weight.T\n",
    "                if self.bias is not None:\n",
    "                    self.bias[i] = layer.bias\n",
    "\n",
    "    def forward(self, x_num):\n",
    "        # x_num: batch_size, features_num, d_in\n",
    "        assert x_num.ndim == 3\n",
    "        #x = x_num[..., None] * self.weight[None]\n",
    "        #x = x.sum(-2)\n",
    "        x = torch.einsum(\"bfi,fij->bfj\",x_num,self.weight)\n",
    "        if self.bias is not None:\n",
    "            x = x + self.bias[None]\n",
    "        return x\n",
    "    \n",
    "class CatEmbedding(nn.Module):\n",
    "    \"\"\"\n",
    "    离散特征用Embedding层编码\n",
    "    输入shape: [batch_size,features_num], \n",
    "    输出shape: [batch_size,features_num, d_embed]\n",
    "    \"\"\"\n",
    "    def __init__(self, categories, d_embed):\n",
    "        super().__init__()\n",
    "        self.embedding = torch.nn.Embedding(sum(categories), d_embed)\n",
    "        self.offsets = nn.Parameter(\n",
    "                torch.tensor([0] + categories[:-1]).cumsum(0),requires_grad=False)\n",
    "        \n",
    "        torch.nn.init.xavier_uniform_(self.embedding.weight.data)\n",
    "\n",
    "    def forward(self, x_cat):\n",
    "        \"\"\"\n",
    "        :param x_cat: Long tensor of size ``(batch_size, features_num)``\n",
    "        \"\"\"\n",
    "        x = x_cat + self.offsets[None]\n",
    "        return self.embedding(x) \n",
    "    \n",
    "class CatLinear(nn.Module):\n",
    "    \"\"\"\n",
    "    离散特征用Embedding实现线性层（等价于先F.onehot再nn.Linear()）\n",
    "    输入shape: [batch_size,features_num], \n",
    "    输出shape: [batch_size,features_num, d_out]\n",
    "    \"\"\"\n",
    "    def __init__(self, categories, d_out=1):\n",
    "        super().__init__()\n",
    "        self.fc = nn.Embedding(sum(categories), d_out)\n",
    "        self.bias = nn.Parameter(torch.zeros((d_out,)))\n",
    "        self.offsets = nn.Parameter(\n",
    "                torch.tensor([0] + categories[:-1]).cumsum(0),requires_grad=False)\n",
    "\n",
    "    def forward(self, x_cat):\n",
    "        \"\"\"\n",
    "        :param x: Long tensor of size ``(batch_size, features_num)``\n",
    "        \"\"\"\n",
    "        x = x_cat + self.offsets[None]\n",
    "        return torch.sum(self.fc(x), dim=1) + self.bias \n",
    "    \n",
    "    \n",
    "class FMLayer(nn.Module):\n",
    "    \"\"\"\n",
    "    FM交互项\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, reduce_sum=True):\n",
    "        super().__init__()\n",
    "        self.reduce_sum = reduce_sum\n",
    "\n",
    "    def forward(self, x): #注意：这里的x是公式中的 <v_i> * xi\n",
    "        \"\"\"\n",
    "        :param x: Float tensor of size ``(batch_size, num_features, k)``\n",
    "        \"\"\"\n",
    "        square_of_sum = torch.sum(x, dim=1) ** 2\n",
    "        sum_of_square = torch.sum(x ** 2, dim=1)\n",
    "        ix = square_of_sum - sum_of_square\n",
    "        if self.reduce_sum:\n",
    "            ix = torch.sum(ix, dim=1, keepdim=True)\n",
    "        return 0.5 * ix\n",
    "    \n",
    "    \n",
    "    \n",
    "#deep部分\n",
    "class MultiLayerPerceptron(nn.Module):\n",
    "\n",
    "    def __init__(self, d_in, d_layers, dropout, \n",
    "                 d_out = 1):\n",
    "        super().__init__()\n",
    "        layers = []\n",
    "        for d in d_layers:\n",
    "            layers.append(nn.Linear(d_in, d))\n",
    "            layers.append(nn.BatchNorm1d(d))\n",
    "            layers.append(nn.ReLU())\n",
    "            layers.append(nn.Dropout(p=dropout))\n",
    "            d_in = d\n",
    "        layers.append(nn.Linear(d_layers[-1], d_out))\n",
    "        self.mlp = nn.Sequential(*layers)\n",
    "\n",
    "    def forward(self, x):\n",
    "        \"\"\"\n",
    "        :param x: Float tensor of size ``(batch_size, d_in)``\n",
    "        \"\"\"\n",
    "        return self.mlp(x)\n",
    "    \n",
    "    \n",
    "class DeepFM(nn.Module):\n",
    "    \"\"\"\n",
    "    DeepFM模型。\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, d_numerical, categories, d_embed,\n",
    "                 deep_layers, deep_dropout,\n",
    "                 n_classes = 1):\n",
    "        \n",
    "        super().__init__()\n",
    "        if d_numerical is None:\n",
    "            d_numerical = 0\n",
    "        if categories is None:\n",
    "            categories = []\n",
    "            \n",
    "        self.categories = categories\n",
    "        self.n_classes = n_classes\n",
    "        \n",
    "        self.num_linear = nn.Linear(d_numerical,n_classes) if d_numerical else None\n",
    "        self.cat_linear = CatLinear(categories,n_classes) if categories else None\n",
    "        \n",
    "        self.num_embedding = NumEmbedding(d_numerical,1,d_embed) if d_numerical else None\n",
    "        self.cat_embedding = CatEmbedding(categories, d_embed) if categories else None\n",
    "        \n",
    "        if n_classes==1:\n",
    "            self.fm = FMLayer(reduce_sum=True)\n",
    "            self.fm_linear = None\n",
    "        else:\n",
    "            assert n_classes>=2\n",
    "            self.fm = FMLayer(reduce_sum=False)\n",
    "            self.fm_linear = nn.Linear(d_embed,n_classes)\n",
    "            \n",
    "        self.deep_in = d_numerical*d_embed+len(categories)*d_embed\n",
    "        \n",
    "        self.deep = MultiLayerPerceptron(\n",
    "            d_in= self.deep_in,\n",
    "            d_layers = deep_layers,\n",
    "            dropout = deep_dropout,\n",
    "            d_out = n_classes\n",
    "        )\n",
    "        \n",
    "\n",
    "    def forward(self, x):\n",
    "        \n",
    "        \"\"\"\n",
    "        x_num: numerical features\n",
    "        x_cat: category features\n",
    "        \"\"\"\n",
    "        x_num,x_cat = x\n",
    "        #linear部分\n",
    "        x = 0.0\n",
    "        if self.num_linear:\n",
    "            x = x + self.num_linear(x_num) \n",
    "        if self.cat_linear:\n",
    "            x = x + self.cat_linear(x_cat)\n",
    "        \n",
    "        #fm部分\n",
    "        x_embedding = []\n",
    "        if self.num_embedding:\n",
    "            x_embedding.append(self.num_embedding(x_num[...,None]))\n",
    "        if self.cat_embedding:\n",
    "            x_embedding.append(self.cat_embedding(x_cat))\n",
    "        x_embedding = torch.cat(x_embedding,dim=1)\n",
    "        \n",
    "        if self.n_classes==1:\n",
    "            x = x + self.fm(x_embedding)\n",
    "        else: \n",
    "            x = x + self.fm_linear(self.fm(x_embedding)) \n",
    "            \n",
    "        #deep部分\n",
    "        x = x + self.deep(x_embedding.view(-1,self.deep_in))\n",
    "        \n",
    "        if self.n_classes==1:\n",
    "            x = x.squeeze(-1)\n",
    "        \n",
    "        return x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5d20a06d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-4.4453,  0.1339], grad_fn=<SqueezeBackward1>)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##测试 DeepFM\n",
    "\n",
    "model = DeepFM(d_numerical = 3, categories = [4,3,2],\n",
    "        d_embed = 4, deep_layers = [20,20], deep_dropout=0.1,\n",
    "        n_classes = 1)\n",
    "x_num = torch.randn(2,3)\n",
    "x_cat = torch.randint(0,2,(2,3))\n",
    "model((x_num,x_cat)) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8316d366",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "23b6a528",
   "metadata": {},
   "source": [
    "## 三，criteo数据集完整范例"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4c4fce3",
   "metadata": {},
   "source": [
    "Criteo数据集是一个经典的广告点击率CTR预测数据集。\n",
    "\n",
    "这个数据集的目标是通过用户特征和广告特征来预测某条广告是否会为用户点击。\n",
    "\n",
    "数据集有13维数值特征(I1 -> I13)和26维类别特征(C14 -> C39), 共39维特征, 特征中包含着许多缺失值。\n",
    "\n",
    "训练集4000万个样本，测试集600万个样本。数据集大小超过100G.\n",
    "\n",
    "此处使用的是采样100万个样本后的cretio_small数据集。 \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa1bb01c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "13245a35",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import pandas as pd \n",
    "import datetime \n",
    "\n",
    "from sklearn.model_selection import train_test_split \n",
    "\n",
    "import torch \n",
    "from torch import nn \n",
    "from torch.utils.data import Dataset,DataLoader  \n",
    "import torch.nn.functional as F \n",
    "import torchkeras \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e71b6b04",
   "metadata": {},
   "source": [
    "### 1，准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1a89c17a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder,QuantileTransformer\n",
    "from sklearn.pipeline import Pipeline \n",
    "from sklearn.impute import SimpleImputer \n",
    "\n",
    "dfdata = pd.read_csv(\"./eat_pytorch_datasets/criteo_small.zip\",sep=\"\\t\",header=None)\n",
    "dfdata.columns = [\"label\"] + [\"I\"+str(x) for x in range(1,14)] + [\n",
    "    \"C\"+str(x) for x in range(14,40)]\n",
    "\n",
    "cat_cols = [x for x in dfdata.columns if x.startswith('C')]\n",
    "num_cols = [x for x in dfdata.columns if x.startswith('I')]\n",
    "num_pipe = Pipeline(steps = [('impute',SimpleImputer()),('quantile',QuantileTransformer())])\n",
    "\n",
    "for col in cat_cols:\n",
    "    dfdata[col]  = LabelEncoder().fit_transform(dfdata[col])\n",
    "\n",
    "dfdata[num_cols] = num_pipe.fit_transform(dfdata[num_cols])\n",
    "\n",
    "categories = [dfdata[col].max()+1 for col in cat_cols]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b6f30f3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch \n",
    "from torch.utils.data import Dataset,DataLoader \n",
    "\n",
    "#DataFrame转换成torch数据集Dataset, 特征分割成X_num,X_cat方式\n",
    "class DfDataset(Dataset):\n",
    "    def __init__(self,df,\n",
    "                 label_col,\n",
    "                 num_features,\n",
    "                 cat_features,\n",
    "                 categories,\n",
    "                 is_training=True):\n",
    "        \n",
    "        self.X_num = torch.tensor(df[num_features].values).float() if num_features else None\n",
    "        self.X_cat = torch.tensor(df[cat_features].values).long() if cat_features else None\n",
    "        self.Y = torch.tensor(df[label_col].values).float() \n",
    "        self.categories = categories\n",
    "        self.is_training = is_training\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.Y)\n",
    "    \n",
    "    def __getitem__(self,index):\n",
    "        if self.is_training:\n",
    "            return ((self.X_num[index],self.X_cat[index]),self.Y[index])\n",
    "        else:\n",
    "            return (self.X_num[index],self.X_cat[index])\n",
    "    \n",
    "    def get_categories(self):\n",
    "        return self.categories\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8b0eba3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "dftrain_val,dftest = train_test_split(dfdata,test_size=0.2)\n",
    "dftrain,dfval = train_test_split(dftrain_val,test_size=0.2)\n",
    "\n",
    "ds_train = DfDataset(dftrain,label_col = \"label\",num_features = num_cols,cat_features = cat_cols,\n",
    "                    categories = categories, is_training=True)\n",
    "\n",
    "ds_val = DfDataset(dfval,label_col = \"label\",num_features = num_cols,cat_features = cat_cols,\n",
    "                    categories = categories, is_training=True)\n",
    "\n",
    "ds_test = DfDataset(dftest,label_col = \"label\",num_features = num_cols,cat_features = cat_cols,\n",
    "                    categories = categories, is_training=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "04f56e49",
   "metadata": {},
   "outputs": [],
   "source": [
    "dl_train = DataLoader(ds_train,batch_size = 2048,shuffle=True)\n",
    "dl_val = DataLoader(ds_val,batch_size = 2048,shuffle=False)\n",
    "dl_test = DataLoader(ds_test,batch_size = 2048,shuffle=False)\n",
    "\n",
    "for features,labels in dl_train:\n",
    "    break \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d67386df",
   "metadata": {},
   "source": [
    "### 2，定义模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "11d303b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------------------------------------------------------\n",
      "Layer (type)                            Output Shape              Param #\n",
      "==========================================================================\n",
      "Linear-1                                     [-1, 1]                   14\n",
      "Embedding-2                              [-1, 26, 1]            1,296,709\n",
      "NumEmbedding-3                           [-1, 13, 8]                  104\n",
      "Embedding-4                              [-1, 26, 8]           10,373,672\n",
      "FMLayer-5                                    [-1, 1]                    0\n",
      "Linear-6                                   [-1, 128]               40,064\n",
      "BatchNorm1d-7                              [-1, 128]                  256\n",
      "ReLU-8                                     [-1, 128]                    0\n",
      "Dropout-9                                  [-1, 128]                    0\n",
      "Linear-10                                   [-1, 64]                8,256\n",
      "BatchNorm1d-11                              [-1, 64]                  128\n",
      "ReLU-12                                     [-1, 64]                    0\n",
      "Dropout-13                                  [-1, 64]                    0\n",
      "Linear-14                                   [-1, 32]                2,080\n",
      "BatchNorm1d-15                              [-1, 32]                   64\n",
      "ReLU-16                                     [-1, 32]                    0\n",
      "Dropout-17                                  [-1, 32]                    0\n",
      "Linear-18                                    [-1, 1]                   33\n",
      "==========================================================================\n",
      "Total params: 11,721,380\n",
      "Trainable params: 11,721,380\n",
      "Non-trainable params: 0\n",
      "--------------------------------------------------------------------------\n",
      "Input size (MB): 0.000084\n",
      "Forward/backward pass size (MB): 0.009438\n",
      "Params size (MB): 44.713516\n",
      "Estimated Total Size (MB): 44.723038\n",
      "--------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "def create_net():\n",
    "    net = DeepFM(\n",
    "        d_numerical= ds_train.X_num.shape[1],\n",
    "        categories= ds_train.get_categories(),\n",
    "        d_embed = 8, deep_layers = [128,64,32], deep_dropout=0.25,\n",
    "        n_classes = 1\n",
    "        \n",
    "    )\n",
    "    return net \n",
    "\n",
    "from torchkeras import summary\n",
    "\n",
    "net = create_net()\n",
    "summary(net,input_data=features);\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8ec71ed",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "cae075b1",
   "metadata": {},
   "source": [
    "### 3，训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f19a40a0-bbb6-4164-8f47-451842664326",
   "metadata": {},
   "source": [
    "我们使用梦中情炉torchkeras来实现最优雅的训练循环。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e9f5987a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torchkeras.metrics import AUC\n",
    "from torchkeras import KerasModel \n",
    "\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "\n",
    "metrics_dict = {\"auc\":AUC()}\n",
    "\n",
    "optimizer = torch.optim.Adam(net.parameters(), lr=0.002, weight_decay=0.001) \n",
    "\n",
    "model = KerasModel(net,\n",
    "                   loss_fn = loss_fn,\n",
    "                   metrics_dict= metrics_dict,\n",
    "                   optimizer = optimizer\n",
    "                  )      \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2d3ec3f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< 🐌 cpu is used >>>>>>\u001b[0m\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAikAAAGJCAYAAABPZ6NtAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB74UlEQVR4nO3dd1xTV/8H8E8SSNh7DwH3Yigq1dZWK4qjtIrWXXHUiValttVWnK30p9Xi1lZq26duxbaPWlulah04HqwDFRRFUWQrICAEkvP745pIIEDAwA3wfb9e90Vyc+6554bA/eZMAWOMgRBCCCFExwj5LgAhhBBCiDoUpBBCCCFEJ1GQQgghhBCdREEKIYQQQnQSBSmEEEII0UkUpBBCCCFEJ1GQQgghhBCdREEKIYQQQnQSBSmEEEII0UkUpBCdt2TJEggEAmRlZfFdlHpz//59CAQC/Pjjj3wXhdShGTNmoG/fvnwXo16NHz8eJiYmfBdDxciRIzF8+HC+i0HUoCCFkEqsWLECv/76K9/FaNRycnIwZcoU2NrawtjYGL1798bly5c1OlYgEFS6lb/xp6amYsqUKfDw8IChoSFatGiB0NBQZGdnV8j31q1b6N+/P0xMTGBlZYUPPvgAmZmZFdLJ5XKsXLkSHh4eMDAwgJeXF3bt2qXxtSclJWHbtm34/PPPNUpfUFAAmUymcf6N1cWLFzFjxgz4+vpCX18fAoGg0rS5ubn49NNP0apVKxgaGsLNzQ2TJk1CcnKySrrPPvsMBw4cwNWrV+u6+KSG9PguACG6asWKFRg2bBgGDx7Md1EaJblcjkGDBuHq1av45JNPYGNjg02bNqFXr16IjY1Fq1atqjz+P//5T4V9//vf/7B27Vr069dPuS8/Px/du3dHQUEBZsyYAVdXV1y9ehUbNmzAiRMnEBsbC6GQ+7726NEjvPnmmzA3N8eKFSuQn5+Pb775BtevX8fFixchFouV+X7xxRf4+uuvMXnyZHTt2hW//fYbRo8eDYFAgJEjR1Z7/WvXroWHhwd69+5daZpjx45hy5Yt+Pvvv5GTkwORSAQPDw8MGzYMs2fPhoODQ7XnaWyOHDmCbdu2wcvLC82bN8ft27fVppPL5ejbty9u3ryJGTNmoHXr1khMTMSmTZvw559/4tatWzA1NQUAdOrUCV26dMHq1avx888/1+flkOowQnTc4sWLGQCWmZlZr+c1NjZmwcHB9XpOhaSkJAaAbd++nZfz14c9e/YwAGzfvn3KfRkZGczCwoKNGjWqVnlOmjSJCQQC9vDhQ+W+HTt2MADs0KFDKmkXLVrEALDLly8r902fPp0ZGhqyBw8eKPcdO3aMAWBbt25V7nv06BHT19dnISEhyn1yuZz17NmTubi4sNLS0irLKZVKmY2NDVu4cKHa1/Pz89nQoUOZQCBgAwYMYOvXr2eHDh1ie/fuZYsWLWKtWrViFhYWbP/+/Zq9MTokODiYGRsb1/r4tLQ0VlhYyBhjLCQkhFV2Gzt79iwDwDZs2KCy/4cffmAAWFRUlMr+b775hhkbG7Nnz57VumxE+6i5hzQYWVlZGD58OMzMzGBtbY3Zs2ejqKioQrpffvkFvr6+MDQ0hJWVFUaOHImHDx+qpLlz5w6GDh0KBwcHGBgYwMXFBSNHjkRubi4ArimhoKAAP/30k7IJYfz48WrLlZ6eDj09PSxdurTCawkJCRAIBNiwYQMA4MmTJ5g3bx48PT1hYmICMzMzDBgwQCvVzFKpFIsWLYKvry/Mzc1hbGyMnj174sSJEyrpTp48CYFAgJMnT6rsr6wfTHx8PIYPHw5bW1sYGhqiTZs2+OKLL165vPv374e9vT2CgoKU+2xtbTF8+HD89ttvKC4urlF+xcXFOHDgAN566y24uLgo9+fl5QEA7O3tVdI7OjoCAAwNDZX7Dhw4gHfeeQfNmjVT7vP390fr1q2xd+9e5b7ffvsNJSUlmDFjhnKfQCDA9OnT8ejRI8TExFRZ1jNnziArKwv+/v4VXistLcU777yDS5cu4cKFCzhy5AhmzpyJQYMG4f3338fSpUtx8+ZNLFiwAKNHj8bhw4cr5BEfH49hw4bBysoKBgYG6NKlC37//XeVND/++CMEAgH++ecfTJ06FdbW1jAzM8O4cePw9OnTCnlu2rQJHTp0gEQigZOTE0JCQpCTk1Mh3YULFzBw4EBYWlrC2NgYXl5eWLt2bYV0KSkpGDx4MExMTGBra4t58+Zp1Jxlb2+v8jurTE1+7wDQt29fFBQU4NixY9XmTeoR31ESIdVR1KR4enqywMBAtmHDBjZ27FgGgH3wwQcqab/88ksmEAjYiBEj2KZNm9jSpUuZjY0Nc3d3Z0+fPmWMMVZcXMw8PDyYk5MT+/LLL9m2bdvY0qVLWdeuXdn9+/cZY4z95z//YRKJhPXs2ZP95z//Yf/5z3/YuXPnKi3j22+/zdq3b19h/9KlS5lIJGJpaWmMMcYuXbrEWrRowebPn8+2bt3Kli1bxpydnZm5uTlLSUlRHlebmpTMzEzm6OjIQkND2ebNm9nKlStZmzZtmL6+Pvv333+V6U6cOMEAsBMnTqgcr+6cV69eZWZmZsza2potWLCAbd26lX366afM09NTmUYqlbLMzEyNNplMpjyuZcuWbMCAARWuY9u2bQwAu3btmsbXzhhjUVFRDAD7/vvvVfbfuHGDCYVC1qNHDxYTE8MePnzIDh8+zFxcXNjgwYOV6R49esQAsP/7v/+rkPfYsWOZlZWV8vmHH37IjI2NmVwuV0mXmJjIALB169ZVWVbF5zQ3N7fCa8uWLWOOjo7s8ePHyn0ymYzl5+crHytqFTdt2sTs7OxYXl6eMm1cXBwzNzdn7du3Z//3f//HNmzYwN58800mEAhUag+2b9+u/Lvq2bMnW7duHQsJCWFCoZC9+eabKtem+Bv09/dn69evZzNnzmQikYh17dqVSaVSZbq//vqLicVi5ubmxhYvXsw2b97MPvroI+bv769MExwczAwMDFiHDh3YxIkT2ebNm9nQoUMZALZp06Yq37fyqqpJyczMZMbGxqxt27YsOjqaPXr0iJ08eZJ5enqyrl27spKSEpX0JSUlzNDQkH388cc1KgOpWxSkEJ2n+Af57rvvquyfMWMGA8CuXr3KGGPs/v37TCQSsa+++kol3fXr15menp5y/7///luhmUGdmjT3bN26lQFg169fV9nfvn179vbbbyufFxUVqdyoGeOCA4lEwpYtW6ayr6ZBSmlpKSsuLlbZ9/TpU2Zvb88mTpyo3FeTIOXNN99kpqamKs0fjDGVG5giP022pKQk5XHGxsYq5VI4fPgwA8COHj2q8bUzxtjQoUOZRCJRBqNlbdu2jVlYWKiUJTg4WOVGdenSJQaA/fzzzxWO/+STTxgAVlRUxBhjbNCgQax58+YV0hUUFDAAbP78+VWWdezYscza2rrC/tzcXGZmZsZ+/fVX5b7vvvuOWVpaMgCsQ4cO7MCBAyo35s6dO7PvvvtO+bxPnz7M09NTWVbGuN9Xjx49WKtWrZT7FEGKr6+vSqCxcuVKBoD99ttvjDGuCU4sFrN+/fqpfHY3bNjAALAffviBMcZ9/jw8PJibm1uF30HZz0twcDADoPJ5Z4yxTp06MV9f3yrft/KqClIYY+zQoUPM0dFR5fceEBBQaZNO69at1QbOhD/U3EMajJCQEJXns2bNAsB1pAOAqKgoyOVyDB8+HFlZWcrNwcEBrVq1UjZ7mJubAwD+/PNPFBYWaqVsQUFB0NPTw549e5T74uLicPPmTYwYMUK5TyKRKDtpymQyZGdnw8TEBG3atNF4VEtlRCKRsmOnXC7HkydPUFpaii5dutQq78zMTPzzzz+YOHGiSvMHAJURFd7e3jh27JhGW9mOns+fP4dEIqlwXgMDA+XrmsrLy8Phw4cxcOBAWFhYVHjd2dkZ3bp1Q0REBA4ePIjQ0FDs2LED8+fPVykPAI3K9Kplz87OhqWlZYX9f/31F6ysrPDuu+8CAC5fvoypU6di6NChOHjwIEaMGIHJkyerHPPee+8pm+6ePHmCv//+G8OHD8ezZ8+UfwPZ2dkICAjAnTt3kJKSonL8lClToK+vr3w+ffp06OnpKf+ujh8/DqlUijlz5ig/uwAwefJkmJmZKZub/v33XyQlJWHOnDkVfgfqRuBMmzZN5XnPnj1x7969qt62GrO1tUWnTp3w1Vdf4ddff8WSJUtw+vRpTJgwQW16S0vLJjXVQUNAo3tIg1F+tEeLFi0gFApx//59AFw/E8ZYpaNCFP+IPTw8EBoaijVr1mDHjh3o2bMn3n33XYwdO1YZwNSUjY0N+vTpg71792L58uUAgD179kBPT0+lz4VcLsfatWuxadMmJCUlqbTBW1tb1+rcZf30009YvXo14uPjUVJSotzv4eFR47wUN4yOHTtWmc7S0lJt34rqGBoaqu13ouhnpEm/A4UDBw6gqKgIY8aMqfDa2bNn8c477+D8+fPo0qULAGDw4MEwMzPD0qVLMXHiRLRv3155Pk3KpI2yM8Yq7IuNjcVbb72lvKlv27YNvXr1wvfff68st0wmU+n/ZG9vjzNnzgAAEhMTwRhDWFgYwsLC1J43IyMDzs7Oyufl/15MTEzg6Oio/Lt68OABAKBNmzYq6cRiMZo3b658/e7duwCq/7wAXDBna2urss/S0lJtX5jaunfvHnr37o2ff/4ZQ4cOBcAFdO7u7hg/fjz++OMPDBgwQOUYxliVQ5pJ/aMghTRY5f+ZyOVyCAQC/PHHHxCJRBXSl51AavXq1Rg/fjx+++03/PXXX/joo48QHh6O8+fPq3S6rImRI0diwoQJuHLlCnx8fLB371706dMHNjY2yjQrVqxAWFgYJk6ciOXLl8PKygpCoRBz5syBXC6v1XkVfvnlF4wfPx6DBw/GJ598Ajs7O4hEIoSHhytvIID6b7UAaj0Hh1QqxZMnTzRKa2trq/zdODo6IjU1tUIaxT4nJyeNy7Bjxw6Ym5vjnXfeqfDa1q1bYW9vrwxQFN59910sWbIE586dQ/v27ZUdKisrk5WVlbL2xNHRESdOnKhwU9O07NbW1mpvyNnZ2SrH3r9/H127dlVJ061bN5XnDx8+VAa4is/QvHnzEBAQoPbcLVu2rLJs9UHd36e2/fjjjygqKqrwmVDUUp09e7ZCkPL06dNqh76T+kVBCmkw7ty5o1IjkJiYCLlcDnd3dwBczQpjDB4eHmjdunW1+Xl6esLT0xMLFy7EuXPn8Prrr2PLli348ssvAVR+M6/M4MGDMXXqVGWTz+3bt7FgwQKVNPv370fv3r0RGRmpsj8nJ0clmKmN/fv3o3nz5oiKilIp++LFi1XSKZoZyo/MUHwjVmjevDkArtmqKufOnatyro+ykpKSlL8vHx8fnD59GnK5XKUZ4cKFCzAyMtLodwhwgcGJEycwfvx4tU0w6enpagMwRU1TaWkpAK5JyNbWFv/73/8qpL148SJ8fHyUz318fLBt2zbcunUL7du3Vym74vWqtG3bFjt27EBubq5K7Z2ZmZlyhBkAODg4qASYAFSaRIqKivCf//wHixYtAvDyd6avr69x7dadO3dUfn/5+flITU3FwIEDAQBubm4AuJFqivwBLjhNSkpSnqdFixYAuM9LbWrWtC09PR2MsQq/+/K/d4XS0lI8fPhQGcQQ3UB9UkiDsXHjRpXn69evBwDlt6GgoCCIRCIsXbq0QlU6Y0w5u2heXl6Ff1Cenp4QCoUqVfjGxsZqh1hWxsLCAgEBAdi7dy92794NsVhcYSI4kUhUoWz79u2r0E+gNhTfTsvmf+HChQrDYd3c3CASifDPP/+o7N+0aZPKc1tbW7z55pv44YcfKszQWfYcte2TMmzYMKSnpyMqKkq5LysrC/v27UNgYKBKwHH37t0KN2uF3bt3Qy6Xq23qAYDWrVsjPT29wpBrxeywnTp1Uu4bOnQoDh06pDJkPTo6Grdv38b777+v3Pfee+9BX19f5T1jjGHLli1wdnZGjx491JZFoXv37mCMITY2VmV/u3btlIEOAAwZMgQHDx7Exo0b8eDBAxw5cgQrVqwAAJw+fRr9+vWDpaUlxo4dCwCws7NDr169sHXrVrU1Qupmzv3uu+9UmgY3b96M0tJS5d+Vv78/xGIx1q1bp/J7j4yMRG5uLgYNGgQA6Ny5Mzw8PBAREVHh70Zd01Zda926NRhjKkPHAfW/dwC4efMmioqKqv3dkXpW7111Camh8kOQN27cqByCPHr0aJW04eHhDADr0aMHW7lyJdu8eTP79NNPWatWrdiqVasYY4wdPHiQOTs7szlz5rBNmzaxdevWsa5duzJ9fX0WExOjzGvgwIHM2NiYrV69mu3atYudP3++2rL+8ssvDAAzNTVlgYGBFV5XTCA2fvx49t1337FZs2YxKysr1rx5c/bWW28p09VmdI9ikqp3332Xbd26lc2fP59ZWFiwDh06MDc3N5W0I0eOZHp6eiw0NJRt3LiRDRgwgPn6+lY455UrV5iJiYlyCPJ3333HPv/8c+bt7a1xuSpTWlrKXnvtNWZiYsKWLl3KNm7cyDp06MBMTU1ZfHy8Slo3N7cK16Dg6+vLnJycKoyaUoiPj2fGxsbMxMSELViwgG3ZsoWNGjWKAWB9+/ZVSZucnMysra1ZixYt2Lp169iKFSuYpaVlhdEyjL0c8TNlyhT2/fffs0GDBjEAbMeOHdVee3FxsfI9LevRo0dMT0+vwgRzeDEyxcjIiK1atYoBYEKhkA0fPrzCJIc3btxglpaWzNrams2fP5999913bPny5WzgwIHMy8tLma78EGTF0GKhUMjeeOMNtUOQ+/XrxzZs2MBmzZqldgjy0aNHmb6+PnNzc2NLlixhW7duZXPnzmX9+vVTpqlsMjfFOapz//59tnz5crZ8+XLm5+fHACiflx2ZlZWVxRwcHJhYLGYfffQR27p1K5s6dSoTiUSsQ4cOFUbCffPNN8zIyEhlODfhHwUpROcp/nndvHmTDRs2jJmamjJLS0s2c+ZM9vz58wrpDxw4wN544w1mbGysnCchJCSEJSQkMMYYu3fvHps4cSJr0aIFMzAwYFZWVqx3797s+PHjKvnEx8ezN998kxkaGiqHrFYnLy9Pmf6XX36p8HpRURH7+OOPmaOjIzM0NGSvv/46i4mJYW+99dYrBylyuZytWLGCubm5MYlEwjp16sQOHTrEgoODK9zgMzMz2dChQ5mRkRGztLRkU6dOZXFxcWrPGRcXx4YMGcIsLCyYgYEBa9OmDQsLC9O4XFV58uQJmzRpErO2tmZGRkbsrbfeYpcuXaqQrrIgJT4+ngFgoaGhVZ4nPj6eDRs2jLm6uipvovPmzWMFBQUV0sbFxbF+/foxIyMjZmFhwcaMGaOc56YsmUymfL/FYjHr0KGD2t95ZT766CPWsmXLCvuDg4OZn5+fyk307t277PTp0+zp06fs+fPnLCYmhuXk5FSa9927d9m4ceOYg4MD09fXZ87Ozuydd95RmaFWEaScOnWKTZkyhVlaWjITExM2ZswYlp2dXSHPDRs2sLZt2zJ9fX1mb2/Ppk+frna495kzZ1jfvn2ZqakpMzY2Zl5eXmz9+vUq1/cqQUpVQ97L/g0xxgV9EydOZB4eHkwsFjNHR0c2efJktbNX+/n5sbFjx1Z7flK/BIzxUA9HCCFN3L1799C2bVv88ccf6NOnj3J/VlYWfH190bFjR+zatQtmZmYVjpXJZDh48CCGDRtW6/P/+OOPmDBhAi5dulShU3FTc+XKFXTu3BmXL1+utj8RqV/UJ4UQQnjQvHlzTJo0CV9//bXKfhsbGxw7dgy3b99Gq1atsHz5cpw/fx7JycmIi4vDli1b4O3tjWnTplXoK0Rq5+uvv8awYcMoQNFBVJNCiI7TZIivubl5jeYVIbrv2bNnWLVqFbZt26bSCdbU1BRjxozBokWLlMOma4NqUkhDQEOQCdFxmgzx3b59e6ULIJKGydTUFMuWLcPSpUuRmJiItLQ0mJmZoV27dsqZhQlp7HivSdm4cSNWrVqFtLQ0eHt7Y/369RUmKyorIiICmzdvRnJyMmxsbDBs2DCEh4crp6N2d3evMN8DAMyYMUM5hLVXr144deqUyutTp07Fli1btHhlhGjH06dPKwxVLa9Dhw6v9K2aEEJ0Ea81KXv27EFoaCi2bNkCPz8/REREICAgAAkJCbCzs6uQfufOnZg/fz5++OEH9OjRA7dv38b48eMhEAiwZs0aAMClS5dUJu+Ji4tD3759VeY4ALh1J5YtW6Z8bmRkVEdXScirqe2084QQ0tDxGqSsWbMGkydPVi72tGXLFhw+fBg//PCDysJfCopZQUePHg2AqzUZNWqUyuRH5deD+Prrr9GiRQu89dZbKvuNjIxUJpYihBBCiG7hLUiRSqWIjY1VmTZcKBTC39+/wgyZCj169MAvv/yCixcvolu3brh37x6OHDmCDz74oNJz/PLLLwgNDa0wxfmOHTvwyy+/wMHBAYGBgQgLC6uyNqW4uFhlNlLFKrPW1ta0IBUhhBBSA4wxPHv2DE5OTirLYqhLyIuUlBQGgJ07d05l/yeffMK6detW6XFr165l+vr6TE9PjwFg06ZNqzTtnj17mEgkYikpKSr7t27dyo4ePcquXbvGfvnlF+bs7MyGDBlSZXkVEw3RRhtttNFGG23a2R4+fFjlvbdBje45efIkVqxYgU2bNsHPzw+JiYmYPXs2li9frnZZ8sjISAwYMKDCiqRTpkxRPvb09ISjoyP69OmDu3fvKhfJKm/BggUIDQ1VPs/NzUWzZs3w8OFDtZMtEUIIIUS9vLw8uLq6wtTUtMp0vAUpNjY2EIlESE9PV9mfnp5eaV+RsLAwfPDBB/jwww8BcAFGQUEBpkyZgi+++EKlyujBgwc4fvy4yuJllfHz8wPArapbWZAikUjUrrBqZmZGQQohhBBSC9V1l+BtxlmxWAxfX19ER0cr98nlckRHR6N79+5qjyksLKzQdqVu5VeAmzfCzs5OuUJnVa5cuQIANISTEEII0SG8NveEhoYiODgYXbp0Qbdu3RAREYGCggLlaJ9x48bB2dkZ4eHhAIDAwECsWbMGnTp1Ujb3hIWFITAwUBmsAFyws337dgQHB0NPT/US7969i507d2LgwIGwtrbGtWvXMHfuXLz55pvw8vKqv4snhBBCSJV4DVJGjBiBzMxMLFq0CGlpafDx8cHRo0dhb28PAEhOTlapOVm4cCEEAgEWLlyIlJQU2NraIjAwEF999ZVKvsePH0dycjImTpxY4ZxisRjHjx9XBkSurq4YOnQoFi5cWLcXSwghhJAa4X3G2YYqLy8P5ubmyM3NrbRPCmMMpaWlKpPLkYZBJBJBT0+PhpcTQkgd0OQeCtDaPXVGKpUiNTUVhYWFfBeF1JKRkREcHR1pnRRCCOEJBSl1QC6XIykpCSKRCE5OThCLxfSNvAFhjEEqlSIzMxNJSUlo1apV1ZMNEUIIqRMUpNQBqVQKuVwOV1dXWhOogTI0NIS+vj4ePHgAqVSqXMCSEEKaEpkMOH0aSE0FHB2Bnj2BMuNU6hwFKXWIvn03bPT7I4Q0ZVFRwOzZwKNHL/e5uABr1wJBQfVTBvovTAghhBAVUVHAsGGqAQoApKRw+zWYJ1UrKEghhBBCiJJMxtWgqBv7q9g3Zw6Xrq5RkKLDZDLg5Elg1y7uZ0Mbyezu7o6IiAi+i0EIIaQGjh2rWINSFmPAw4dcX5W6Rn1SdBRfbYG9evWCj4+PVoKLS5cuwdjY+NULRQghRCO16egqlQJnzgCHDgGHDwMmJpqdKzX11ctbHQpSdJCiLbB8VZuiLXD//vrrtFQeYwwymazCcgPq2Nra1kOJCCGEADX7cpuZCfzxBxeY/PknkJf38jVNv1vWx3J31NxTjwoKKt+Kirg0mrQFzp6t2vRTWZ41NX78eJw6dQpr166FQCCAQCDAjz/+CIFAgD/++AO+vr6QSCQ4c+YM7t69i/feew/29vYwMTFB165dcfz4cZX8yjf3CAQCbNu2DUOGDIGRkRFatWqF33//XaOyyWQyTJo0CR4eHjA0NESbNm2wdu1alTS9evXCnDlzVPYNHjwY48ePVz4vLi7GZ599BldXV0gkErRs2RKRkZE1ep8IIUTXVNfR9cAB1f3DhwPBwcC+fVyAYmsLjB/PfQl++JALbiqb3ksgAFxduVqaukY1KfWoqiq0gQO5arbTp6tvC3z0iEvXqxe3z90dyMpSn7Ym1q5di9u3b6Njx45YtmwZAODGjRsAgPnz5+Obb75B8+bNYWlpiYcPH2LgwIH46quvIJFI8PPPPyMwMBAJCQlo1qxZpedYunQpVq5ciVWrVmH9+vUYM2YMHjx4ACsrqyrLJpfL4eLign379sHa2hrnzp3DlClT4OjoiOHDh2t8jePGjUNMTAzWrVsHb29vJCUlIUvdm0cIIQ2EJl9uR4zg7h0ODtzzd94BcnO5n++8A3TpApSddWHtWi64EQhU81UELhER9TNfCgUpOkbTNr66aAs0NzeHWCyGkZERHF58kuPj4wEAy5YtQ9++fZVprays4O3trXy+fPlyHDx4EL///jtmzpxZ6TnGjx+PUaNGAQBWrFiBdevW4eLFi+jfv3+VZdPX18fSpUuVzz08PBATE4O9e/dqHKTcvn0be/fuxbFjx+Dv7w8AaN68uUbHEkKIrqruyy3ABTJr1wLh4dzz0FDg448rTx8UxNWqqGs+ioiovy4HFKTUo/z8yl9TRKSatvGVTXf/fq2LpLEuXbqoPM/Pz8eSJUtw+PBhpKamorS0FM+fP0dycnKV+Xh5eSkfGxsbw8zMDBkZGRqVYePGjfjhhx+QnJyM58+fQyqVwsfHR+NruHLlCkQiEd566y2NjyGEEF2n6ZfWtm1fPtZkpZagIOC992jG2SZDk85IPXtykWpKivqqO4GAe71sW2B9DKApP0pn3rx5OHbsGL755hu0bNkShoaGGDZsGKRSaZX56OvrqzwXCASQy+XVnn/37t2YN28eVq9eje7du8PU1BSrVq3ChQsXlGmEQiHKL+pdUlKifGxoaFjteQghpKHR9Mutm1vN8xaJXnYt4AN1nNUxIhFXJQdUjHTroy1QLBZDpsGELGfPnsX48eMxZMgQeHp6wsHBAffrsErn7Nmz6NGjB2bMmIFOnTqhZcuWuHv3rkoaW1tbpJb5SiGTyRAXF6d87unpCblcjlOnTtVZOQkhpL4pvtzqQkdXbaMgRQcp2gKdnVX3u7jU/fBjd3d3XLhwAffv30dWVlaltRytWrVCVFQUrly5gqtXr2L06NEa1YjUVqtWrfC///0Pf/75J27fvo2wsDBcunRJJc3bb7+Nw4cP4/Dhw4iPj8f06dORk5Ojcm3BwcGYOHEifv31VyQlJeHkyZPYu3dvnZWbEELqGt9fbusSBSk6KiiI62ty4gSwcyf3Mymp7jsrzZs3DyKRCO3bt4etrW2lfUzWrFkDS0tL9OjRA4GBgQgICEDnzp3rrFxTp05FUFAQRowYAT8/P2RnZ2PGjBkqaSZOnIjg4GCMGzcOb731Fpo3b47evXurpNm8eTOGDRuGGTNmoG3btpg8eTIKajNemxBCdAifX27rkoCVb8QnGsnLy4O5uTlyc3NhZmam8lpRURGSkpLg4eEBAwMDnkpIXhX9HgkhDcWPPwKWlsCgQdzssXx1dNVUVffQsqjjLCGEENKAPXnCLfiXmwv897/cvCeNBTX3EJ0wbdo0mJiYqN2mTZvGd/EIIURnff01F6B4eXETgzYmVJNCdMKyZcswb948ta9VVRVICCFN2aNHwPr13OMVK1RnjW0MKEghOsHOzg52dnZ8F4MQQhqUpUu5td969mx8tSgANfcQQgghDVJ8PPDDD9zj8HDNZpFtaChIIYQQQhqghQsBuRwIDARef53v0tQNau4hhBBCGqDx44HERK4vSmNFQQohhBDSAL3zDjcvSmNs5lHgvbln48aNcHd3h4GBAfz8/HDx4sUq00dERKBNmzYwNDSEq6sr5s6di6KiIuXrS5YsgUAgUNnall36EdwkXSEhIbC2toaJiQmGDh2K9PT0Ork+QgghRJvKTsHamAMUgOcgZc+ePQgNDcXixYtx+fJleHt7IyAgABkZGWrT79y5E/Pnz8fixYtx69YtREZGYs+ePfj8889V0nXo0AGpqanK7cyZMyqvz507F//973+xb98+nDp1Co8fP0aQDs4ZLGMMJ58+xa70dJx8+hSyBjA5sLu7OyIiIvguBiGENEpyORAQAKxcCTx/zndp6h6vzT1r1qzB5MmTMWHCBADAli1bcPjwYfzwww+YP39+hfTnzp3D66+/jtGjRwPgboijRo3ChQsXVNLp6enBwcFB7Tlzc3MRGRmJnTt34u233wYAbN++He3atcP58+fx2muvafMSay0qMxOzExPxqLhYuc9FIsHali0RZGvLY8kIIYTw5cAB4NgxICaG65NiaMh3ieoWbzUpUqkUsbGx8Pf3f1kYoRD+/v6IiYlRe0yPHj0QGxurbBK6d+8ejhw5goHlBoffuXMHTk5OaN68OcaMGaOySF5sbCxKSkpUztu2bVs0a9as0vMCQHFxMfLy8lS2uhKVmYlhN26oBCgAkFJcjGE3biAqM7POzk0IIUQ3lZQAX3zBPZ43D2gKU0vxFqRkZWVBJpPB3t5eZb+9vT3S0tLUHjN69GgsW7YMb7zxBvT19dGiRQv06tVLpbnHz88PP/74I44ePYrNmzcjKSkJPXv2xLNnzwAAaWlpEIvFsLCw0Pi8ABAeHg5zc3Pl5urqqvG1MsZQIJNptOWVluKjO3egrmFHsW92YiLySks1yq8m60d+9913cHJyglwuV9n/3nvvYeLEibh79y7ee+892Nvbw8TEBF27dsXx48c1zr+8NWvWwNPTE8bGxnB1dcWMGTOQn5+vfH3JkiXw8fFROSYiIgLu7u4q+3744Qd06NABEokEjo6OmDlzZq3LRAghumr7duDOHcDGBggN5bs09aNBje45efIkVqxYgU2bNsHPzw+JiYmYPXs2li9fjrCwMADAgAEDlOm9vLzg5+cHNzc37N27F5MmTar1uRcsWIDQMp+KvLw8jQOVQrkcJqdP1/rcZTEAj4qLYV6un01l8nv2hLGGS2C+//77mDVrFk6cOIE+ffoAAJ48eYKjR4/iyJEjyM/Px8CBA/HVV19BIpHg559/RmBgIBISEtCsWbMaX4tQKMS6devg4eGBe/fuYcaMGfj000+xadMmjfPYvHkzQkND8fXXX2PAgAHIzc3F2bNna1wWQgjRZYWF3OyyADc/iqkpv+WpL7wFKTY2NhCJRBVG1aSnp1fanyQsLAwffPABPvzwQwCAp6cnCgoKMGXKFHzxxRcQqlm0wMLCAq1bt0ZiYiIAwMHBAVKpFDk5OSq1KVWdFwAkEgkkEklNL7NBsbS0xIABA7Bz505lkLJ//37Y2Nigd+/eEAqF8Pb2VqZfvnw5Dh48iN9//71WtRdz5sxRPnZ3d8eXX36JadOm1ShI+fLLL/Hxxx9j9uzZyn1du3atcVkIIUSXrV8PPH4MuLkBTWnNVd6CFLFYDF9fX0RHR2Pw4MEAALlcjujo6EpveIWFhRUCEdGLWoLKmjXy8/Nx9+5dfPDBBwAAX19f6OvrIzo6GkOHDgUAJCQkIDk5Gd27d9fGpVVgJBQiv2dPjdL+k5ODgdevV5vuiKcn3izXZFXZuWtizJgxmDx5MjZt2gSJRIIdO3Zg5MiREAqFyM/Px5IlS3D48GGkpqaitLQUz58/V+nzUxPHjx9HeHg44uPjkZeXh9LSUhQVFaGwsBBGRkbVHp+RkYHHjx8rAypCCGmMioqAb77hHi9bBjTy78sqeG3uCQ0NRXBwMLp06YJu3bohIiICBQUFytE+48aNg7OzM8LDwwEAgYGBWLNmDTp16qRs7gkLC0NgYKAyWJk3bx4CAwPh5uaGx48fY/HixRCJRBg1ahQAwNzcHJMmTUJoaCisrKxgZmaGWbNmoXv37nU2skcgEGjc5NLPygouEglSiovV9ksRgBvl08/KCqI6GCAfGBgIxhgOHz6Mrl274vTp0/j2228BcO/tsWPH8M0336Bly5YwNDTEsGHDIJVKa3ye+/fv45133sH06dPx1VdfwcrKCmfOnMGkSZMglUphZGQEoVBYIfgsKSlRPjZs7N3aCSEEgIEB8M8/wNatwJgxfJemfvEapIwYMQKZmZlYtGgR0tLS4OPjg6NHjyo70yYnJ6vUnCxcuBACgQALFy5ESkoKbG1tERgYiK+++kqZ5tGjRxg1ahSys7Nha2uLN954A+fPn4dtmWG73377LYRCIYYOHYri4mIEBATUqImhLokEAqxt2RLDbtyAAFAJVBQhSUTLlnUSoACAgYEBgoKCsGPHDiQmJqJNmzbo3LkzAODs2bMYP348hgwZAoCrpbp//36tzhMbGwu5XI7Vq1crf8d79+5VSWNra4u0tDQwxiB4cb1XrlxRvm5qagp3d3dER0ejd+/etSoHIYQ0BO3aAU1yCipGaiU3N5cBYLm5uRVee/78Obt58yZ7/vx5rfM/kJHBXM6dYzhxQrm5njvHDmRkvEqxNXLs2DEmkUhYmzZt2PLly5X7hwwZwnx8fNi///7Lrly5wgIDA5mpqSmbPXu2Mo2bmxv79ttvqz3HlStXGAAWERHB7t69y37++Wfm7OzMALCnT58yxhi7efMmEwgE7Ouvv2aJiYlsw4YNzNLSkrm5uSnz+fHHH5mBgQFbu3Ytu337NouNjWXr1q3Tyvugjd8jIYS8ivR0vktQN6q6h5bF+7T4RL0gW1vcf+01nPD2xs527XDC2xtJr71WLxO5vf3227CyskJCQoJy4jyAGzJsaWmJHj16IDAwEAEBAcpalpry9vbGmjVr8H//93/o2LEjduzYoWzWU2jXrh02bdqEjRs3wtvbGxcvXsS8efNU0gQHByMiIgKbNm1Chw4d8M477+DOnTu1KhMhhOiS27eBZs2ASZO4OVKaIgFjDWCudR2Ul5cHc3Nz5ObmwszMTOW1oqIiJCUlwcPDAwYGBjyVkLwq+j0SQvg0YgSwdy+3iOChQ3yXRruquoeWRTUphBBCiI6JjeUCFIEAWLGC79Lwh4IUUid27NgBExMTtVuHDh34Lh4hhOi0BQu4n2PGAF5e/JaFTw1qxlnScLz77rvw8/NT+5q+vn49l4YQQhqO6GhuEUF9/ZezzDZVFKSQOmFqagrTpjJvMyGEaAljL2tRpk4Fmjfntzx8oyClDlGf5IaNfn+EkPogkwGnTwOpqYBczo3qMTbm1uhp6ihIqQOK5ozCwkKaFbUBKywsBEDNU4SQuhMVBcyeDTx69HKfkxPw4YfAi3lNmzQKUuqASCSChYUFMjIyAABGRkbKGVOJ7mOMobCwEBkZGbCwsFAuuUAIIdoUFQUMG8Y18ZSVmgosXw54ewNBQfyUTVfQPCm1VN0Yb8YY0tLSkJOTU/+FI1phYWEBBwcHCjAJIVonkwHu7qo1KGUJBICLC5CUBDTG70mazpNCNSl1RCAQwNHREXZ2diqL4pGGQV9fn2pQCCF15vTpygMUgKtdefiQS9erV70VS+dQkFLHRCIR3ewIIYSoSE3VbrrGiiZzI4QQQuqZo6N20zVWFKQQQggh9axnT67PSWVd3gQCwNWVS9eUUZBCCCGE1DORCFi7Vv1risAlIqJxdpqtCQpSCCGEEB4EBQFhYRX3u7gA+/fT8GOAOs4SQgghvMnL434GBgKjRnF9UHr2pBoUBQpSCCGEEJ6sXg0MGQLY2QFt2/JdGt1DQQohhBDCE6EQePNNvkuhu6hPCiGEEMIDmu+9ehSkEEIIIfXs+XOgdWsgJATIz+e7NLqLghRCCCGknv3xB5CYCBw6BBgb810a3UVBCiGEEFLPdu/mfo4YUfmEboSCFEIIIaRe5edzNSgAF6SQylGQQgghhNSj//6X65PSsiXQuTPfpdFtFKQQQggh9YiaejTHe5CyceNGuLu7w8DAAH5+frh48WKV6SMiItCmTRsYGhrC1dUVc+fORVFRkfL18PBwdO3aFaamprCzs8PgwYORkJCgkkevXr0gEAhUtmnTptXJ9RFCCCEKOTnA0aPc45EjeS1Kg8BrkLJnzx6EhoZi8eLFuHz5Mry9vREQEICMjAy16Xfu3In58+dj8eLFuHXrFiIjI7Fnzx58/vnnyjSnTp1CSEgIzp8/j2PHjqGkpAT9+vVDQUGBSl6TJ09Gamqqclu5cmWdXishhBBSUgLMng307w907Mh3aXSfgDH+ppPx8/ND165dsWHDBgCAXC6Hq6srZs2ahfnz51dIP3PmTNy6dQvR0dHKfR9//DEuXLiAM2fOqD1HZmYm7OzscOrUKbz5Ylq/Xr16wcfHBxERERqXtbi4GMXFxcrneXl5cHV1RW5uLszMzDTOhxBCCGnq8vLyYG5uXu09lLeaFKlUitjYWPj7+78sjFAIf39/xMTEqD2mR48eiI2NVTYJ3bt3D0eOHMHAgQMrPU9ubi4AwMrKSmX/jh07YGNjg44dO2LBggUoLCyssrzh4eEwNzdXbq6urhpdJyGEEEJqh7e1e7KysiCTyWBvb6+y397eHvHx8WqPGT16NLKysvDGG2+AMYbS0lJMmzZNpbmnLLlcjjlz5uD1119HxzL1aqNHj4abmxucnJxw7do1fPbZZ0hISEBUVFSl5V2wYAFCQ0OVzxU1KYQQQogmTp4EioqAPn0AfX2+S9MwNKgFBk+ePIkVK1Zg06ZN8PPzQ2JiImbPno3ly5cjLCysQvqQkBDExcVVaAqaMmWK8rGnpyccHR3Rp08f3L17Fy1atFB7bolEAolEot0LIoQQ0mQsWwacOMGtfFzmOy+pAm9Bio2NDUQiEdLT01X2p6enw8HBQe0xYWFh+OCDD/Dhhx8C4AKMgoICTJkyBV988QWEwpetVzNnzsShQ4fwzz//wMXFpcqy+Pn5AQASExMrDVIIIYSQ2kpN5WpSACAoiNeiNCi89UkRi8Xw9fVV6QQrl8sRHR2N7t27qz2msLBQJRABAJFIBABQ9P9ljGHmzJk4ePAg/v77b3h4eFRblitXrgAAHB0da3MphBBCSJX27+dWPX7tNcDdne/SNBy8NveEhoYiODgYXbp0Qbdu3RAREYGCggJMmDABADBu3Dg4OzsjPDwcABAYGIg1a9agU6dOyuaesLAwBAYGKoOVkJAQ7Ny5E7/99htMTU2RlpYGADA3N4ehoSHu3r2LnTt3YuDAgbC2tsa1a9cwd+5cvPnmm/Dy8uLnjSCEENKoKSZwo7lRaobXIGXEiBHIzMzEokWLkJaWBh8fHxw9elTZmTY5OVml5mThwoUQCARYuHAhUlJSYGtri8DAQHz11VfKNJs3bwbADTMua/v27Rg/fjzEYjGOHz+uDIhcXV0xdOhQLFy4sO4vmBBCSJOTnAycO8fNLvv++3yXpmHhdZ6UhkzTMd6EEEKatm++AT75BHjrrZf9Upo6nZ8nhRBCCGkKLlzgftKKxzXXoIYgE0IIIQ3N3r3AtWtAs2Z8l6ThoSCFEEIIqUMCAeDtzXcpGiZq7iGEEELqSJkl30gtUJBCCCGE1IGbNwEbG2DSJG6OFFJzFKQQQgghdWDPHiA/H8jI4Jp8SM1RkEIIIYRoGWMvJ3CjUT21R0EKIYQQomVXrgC3bwMGBsC77/JdmoaLghRCCCFEy/bs4X4OGgTQfJ+1R0EKIYQQokXU1KM9FKQQQgghWnThAvDgAWBszNWkkNqjydwIIYQQLWrWDFi+HHj+HDAy4rs0DRsFKYQQQogWOTkBCxfyXYrGgZp7CCGEEKKTKEghhBBCtGTbNm5BwcJCvkvSOFCQQgghhGhBaSnw+efciJ7Tp/kuTeNAQQohhBCiBSdOAJmZgLU18PbbfJemcaAghRBCCNECxQRuw4YB+vr8lqWxoCCFEEIIeUVSKXDgAPd45Eh+y9KYUJBCCCGEvKK//gJycgAHB6BnT75L03hQkEIIIYS8IkVTz/DhgEjEb1kaEwpSCCGEkFfAGJCayj2mph7tohlnCSGEkFcgEADHjwP37gEeHnyXpnGhIIUQQgjRgubN+S5B40PNPYQQQkgtFRVxHWZJ3eA9SNm4cSPc3d1hYGAAPz8/XLx4scr0ERERaNOmDQwNDeHq6oq5c+eiqKioRnkWFRUhJCQE1tbWMDExwdChQ5Genq71ayOEENK4HTwI2NsDH33Ed0kaJ16DlD179iA0NBSLFy/G5cuX4e3tjYCAAGRkZKhNv3PnTsyfPx+LFy/GrVu3EBkZiT179uDzzz+vUZ5z587Ff//7X+zbtw+nTp3C48ePERQUVOfXSwghpHGQyYCTJ4HVq7k5UkxM+C5RI8V41K1bNxYSEqJ8LpPJmJOTEwsPD1ebPiQkhL399tsq+0JDQ9nrr7+ucZ45OTlMX1+f7du3T5nm1q1bDACLiYnRuOy5ubkMAMvNzdX4GEIIIQ3fgQOMubgwxo3r4TZ7e24/0Yym91DealKkUiliY2Ph7++v3CcUCuHv74+YmBi1x/To0QOxsbHK5pt79+7hyJEjGDhwoMZ5xsbGoqSkRCVN27Zt0axZs0rPCwDFxcXIy8tT2QghhDQtUVHctPePHqnuz8jg9kdF8VOuxoq3ICUrKwsymQz29vYq++3t7ZGWlqb2mNGjR2PZsmV44403oK+vjxYtWqBXr17K5h5N8kxLS4NYLIaFhYXG5wWA8PBwmJubKzdXV9eaXjIhhJAGTCYDZs/m6k7KU+ybM4dLR7SD946zNXHy5EmsWLECmzZtwuXLlxEVFYXDhw9j+fLldX7uBQsWIDc3V7k9fPiwzs9JCCFEd5w+XbEGpSzGgIcPuXREO3ibJ8XGxgYikajCqJr09HQ4ODioPSYsLAwffPABPvzwQwCAp6cnCgoKMGXKFHzxxRca5eng4ACpVIqcnByV2pSqzgsAEokEEomkNpdKCCGkEVDMKqutdKR6vNWkiMVi+Pr6Ijo6WrlPLpcjOjoa3bt3V3tMYWEhhELVIoteLJLAGNMoT19fX+jr66ukSUhIQHJycqXnJYQQQqr4HqvC0bFuy9GU8DrjbGhoKIKDg9GlSxd069YNERERKCgowIQJEwAA48aNg7OzM8LDwwEAgYGBWLNmDTp16gQ/Pz8kJiYiLCwMgYGBymClujzNzc0xadIkhIaGwsrKCmZmZpg1axa6d++O1157jZ83ghBCiM5iDIiIAGJjARcXICVFfb8UgYB7nVZB1h5eg5QRI0YgMzMTixYtQlpaGnx8fHD06FFlx9fk5GSVmpOFCxdCIBBg4cKFSElJga2tLQIDA/HVV19pnCcAfPvttxAKhRg6dCiKi4sREBCATZs21d+FE0IIaRDy8oBJk4D9+7nnCxcCX33FBSRlAxWBgPsZEUGrIGuTgDF18SCpTl5eHszNzZGbmwszMzO+i0MIIUTL4uKAoUOB27cBfX3g22+BGTO4WWZnz1btROvqygUoNC+oZjS9h9ICg4QQQkg5v/wCTJ0KFBZyAci+fYCfH/daUBDw3nvcKJ7UVK4PSs+eVINSFyhIIYQQQspQNOkAQL9+wI4dgI2NahqRCOjVq96L1uQ0qHlSCCGEkLrWty/XvLN4MXDkSMUAhdQfqkkhhBDS5GVmAra23OO33gISE4FmzfgtE6GaFEIIIU2EYuXiXbu4nzIZt4WFAS1aAPHxL9NSgKIbqCaFEEJIoxcVVXFEjpMT15Rz7Rr3/NdfgfnzeSkeqQQFKYQQQho1xcrF5SfcePyY28RiYPt2YPRofspHKkdBCiGEEJ0lk73aUN+qVi5WsLICRox49bIS7aM+KYQQQnRSVBTg7g707s3VcvTuzT2PitI8j3/+qXrlYgBIS6OVi3UV1aQQQgjRmlet+VCorIkmJYXbv3+/6uyuhYXAnTtc59eEhJc/b9zQ7Hy0crFuoiCFEEKaOG0GFuU7p7q4AGvX1my6+KqaaBT7pk7lZn1VlLNFC65GpLZo5WLdREEKIYTUE20FA9rMS1uBRU1rPsoqLQWys4Fnz4CWLbnrqq6JJiuLS6eY9bV1a6C4GGjbltvatOG2Vq2AgACugyytXNzwUJBCCKkVXbzh6nJe2goGtJnXqwQWZVVX8yEQAHPmcDUfS5YAV69yQUZWFjeJWk4Ol9bRkQsmyje9eOMKVmABPkc4rsJHpZwKf/wBGBq+XI24rHXruOuhlYsbIEZqJTc3lwFgubm5fBeF6JjSUsZOnGBs507uZ2mpbuSlzfwOHGDMxYUx7l8+t7m4cPspL/V5CQSqeQHcPoGgZnlqK6/S0orXV36zsmJszRrGli9n7NNPGZs+nbGxY7mfZXXtWnU+iu3ECcZ69qz8dQcHxuRyLl3Z/cuwkDGALUVYhfxq8r6Vv15X19r9Psmr0/QeSkFKLVGQQtTR1ZukNvPTxRuuLudVXTAgEHA3y7IBY04OY5mZjKWmMvboEWP37zOWmMjYjRvcjbyqQMDCgrHFixmbN4+xFStUyzJkCGPt23PnMzHRLLBQt9nbq+bbpo1mx+3cydj+/Yx99x1jUVGM/fMPYzdvMpaRwVhJScX3TPE7+BfejAHsMnwqfc80/V1oM+gntafpPVTAWFWjx0ll8vLyYG5ujtzcXJiZmfFdHPKKtFGtX1nVuaJKWdOqc23npc38ZDJuCGhl/QUU7ftJSdW/f/Wdl7MzN7OoXM71gSgp4TZDQ8DBgUsnlwNnznDvVWZm1eW6d49ruigqUr95e3Mr6Z48yQ2drc6JEy/7Vzg4AOnp1R9TnTZtVKd69/ICrl+vWR6vvQZ06ACYmr7crK2BiRNfpvnlF+CDD6rPq+w1VkfxmbVj6UiDg3K/PdKRKbCr8d8A0S2a3kOpTwpp8rTRvl+TNnlNbrjV5TV7NtCnz8v0is3MDDAy4vbn53PXJJUCM2ZUnd+sWUBBAXfzlkq5DohS6cutXz+gR4/qOzQyBjx8+LJD44MHwMcfq0+bmalZXr6+gIkJd8OaM4d7LSODy7+0lLvu/HxuX1V5PXrETdpV3qhRwM6d3GOZjFtcriqKcp05A6xcyQU66hQVcT81HdpaNp3iMyIQAHp6LzeZjBtqW5233wY6deKmfS9r0yauvKamwM2bQHBw9XmFh1cfWIwaBSxYwPURUfc5q03n1KAgLnj+58M/gadlzmX1J978/gMKUJoIClJIk/aqHQefP+duLocPa3bD9fEBjI2BNWu4mz7AnWP+fO6GW1rK3YSePq06r0ePAAuLiq/99BMwbhz3+MQJ4N13K8+nbH6PH788Th1TU668Nb3h5uUBBw5odkxlrl7lfnbq9HIfY8CtW6+Wr77+y01BTw+ws6s64FFITeWCRYEAMDCouLm6cuk0HdpaNt39+1ygIiw33aamtTJhYeoDizfeePm4Uyfgiy+0E1iIRFxQr+3OqUFBwHu7D0N+QAShXAaZSITV/ochCtKg2oY0ChSkkAarLqfLLltjoaj9+OsvLghITX255ebWrMxxcdzPJ09e7isoAO7erVk+6ggEXHOFgkQCWFpygc+zZ9Uf37Ej4ObGrWNSfvP25tLU9Ibr5MR9e1fn9m3uxlWdxYu5ZooWLV7us7TkbtgiERdYXL0KTJtWfV5Hj3I1UCKR+lEgAgGwZ49mgYCjI7BqVfXpevbkbvY1CQbKBk6vmldltB1YKGo+PprDkGKVA1hLgWwxnJ9aYO23gsqD/ZSUStu2/n7yBF0PH4KpXMaVWSZD3uFD+N/x43hbXbUYANjbc217asgYw+mcHKRKpXAUi9HTwgIidR8EojvqpYdMI0QdZ/mljU6g5UcQVDUigTHGNm9W/7qBQfWdGRXb0qWM/fYb1yFSIT2dsXPnGLt4kbHLlxmLjNQsrz/+YKyoiOtwKJdr7zqrUr5DoyadQJtyXgrKjrgiOYP3E4a307ifInmtO/WWL19tOvUq8tPWqJcDGRnM5ew5hhMnlJvL2XPsQEZG5Qe9/XaVH0xZuQst/7zC1qdP5WU7V65s56opG6kz1HG2jlHHWf7UtBNoXh7X/n7jBrdNmcJN9rRrl2arnu7cybW5x8Vx38QdHVU3c3OuBsPdvfpvuDXpBKqNvOoiP8X7D6j/9l2bDsK6mhcTMqBjjrJWAHEWEMgFteq0+el/M7FGmgiZdbFynyhbglBxS6wMtK1RXur6Ubm6cjUftemrIS1l2HQmB3efStHCUowZb1hArFezGoaozEwMu3ED5T9iilz2d+iAIFs117lvH/dHqZgs5VVYWADffQe8/752ykbqjKb3UApSaomCFH5oMorD3p4LPm7d4oKS5GTVNNu3A+PH127URVV09YZbNj9t3XSjoipW67tUV61fT3lp6+at1aCiDm6SWptxNjMTsxMT8aj45XW6SCRY27KlxmWSMQb38+dV8ijPQiTCp82agb1IX8oYZIxBBsAgKwtDwsLg9ddfkAsEENbgtsQEAggYQ/qgQbj/7bfQd3CAsUgEY6EQRiIRDIRCtLlwAY+kUrXHC15cb9Jrr1HTTz2iIKWOUZDCD00Di/IcHblhlB07AiNHAn5+2q9hALR7k9T2t2Vt33Rn30nEI2mZG5tYgrWtNL+x1UVeAM+1AooyyOXIKy3FM5kMT0tL0f/aNWRWMgyotjdJbfSvqOl15peW4kFxMe4XFeF+URGSnj/H/aIiXM/Px23FcKZX8P6JE9iyZg3MCguhV7aDVSVKhULkGRlhWmgo9tXmH0MZJ7y90cvS8pXyIJqjIKWOUZDCj//8p+pRKAr9+gFDhnBBSfv26oeeAtqvsQB0c1p2bX6T19W8FPlppVYgJqbSb94AYCwUYpC1NfJlMuTJZHhWWsr9lMmQV1oKaS3+rU51dESgjQ06GBmhmYEBhFUEHPVV+2EqEqGfpaUyMMmqbLy1ht40N0crQ0OIBAKIBALovfgpApTPzbKzMeDjj+F5+jSqCrkYgPM9emDNl18ixdwchTIZCuRy7qdMhkK5HCU1+D3sbNcOo+ztX+n6iOYoSKljFKTUr8JCYMUKbqRIVcNzFWo6aZQ2ayy0SRvflqu7GWnyTV7OGKRyOQrlcnS8dAmpVVSdO4nFiO/WDUYiUZU3Wm2UqyxNAp5Aa2ukS6V4LJUiVSpFanGxyuNUqRRJRUV4Wlpa7fk0YSQUQk8gQJ5MVqPjjIVCtDM2RnsjI3Qo89PNwAC/ZmXVKLArkcuRWVKCNKkUaVIp0l/8jH32DAeysmp8TRZ6enA3MIC7gQE8Xvx8VlqKhffvV3usprUV8oULIf/6a+hV8b6VikQQLlgA4fLllaYpkcvx55MnCFQMq6tCVIcOGEL9UuoNBSl1jIKU+iWXc0NQ79/n5o6orCa4Nk00gJZrP7Q0zFEb35aLZDLsy8zEuLLTjlaiuYEB9AUCFDOGYrn85fai/0BtiABIhEKIhUKIBQLu8YufxXI57mrQROBvaQlnsRjCMt+6heW+gQPA96mpeFbFTU0IoPoGBM19YG+PXhYWMBOJYKanB9OyP0UimIhE0BMKcfLpU/RWTPZShd7m5sgsLUVCYWGlNQAGAgFkQJU1BEZCIV4zNUV6SQnSS0peufbjAzs7DLWzg7uBAdwkElioGR+tCDhTiosrBE9ALZq0fHzArl6ttiZF4OMD/PtvlVlVVzYFI4EAc1xd8bGrK6wqGwNOtKZBBSkbN27EqlWrkJaWBm9vb6xfvx7dunVTm7ZXr144depUhf0DBw7E4cOHAQCCSv4IVq5ciU8++QQA4O7ujgcPHqi8Hh4ejvnz52tUZgpSakeTYCA5Gdi2DTh0CDh/npunAwD27n0ZoIwcye3TVhONtmgjsFDko8m35SKZDMll+giU3yqr8WjK9AQCOIjFcBSL4SQWw1EiUXmcUlyMqbdvV5uPprUCNb2Bl8rlSHz+HDcLC3GzoAA3XvyMLyysVTMSwAWLdmIx7MViOIjFsNfXh1Qux67K5v4vQ9PrVHxmAahcZ42b7tLSKkzIo+hMq7ZTbVoa11u+lmVj4AL0ey8CZjORCB+7umKOiwvM9GgqsbpSp0FKbm4uZDIZrMo19D958gR6eno1umnv2bMH48aNw5YtW+Dn54eIiAjs27cPCQkJsLOzq5D+yZMnkJb5x5udnQ1vb29s27YN48ePBwCkpaWpHPPHH39g0qRJSExMRPPmzQFwQcqkSZMwefJkZTpTU1MYGxtrVG4KUmququnn33sPOHIE2LqVW3JdUVOyd2+F0YSV5vUqTTR8dEKsqixuMTFIqSLAEAsEsNLTQ5oG35INBAIUafBnvqp5c3QzM1PWdCg3RQ2IUIgLubkI0GDxl0MdO8LPzAzSF7Uy0hfNRWUfX3z2DJ/du1dtXtOcnNDcwEA5EkReZlSIjDHIGUNcQQEOl50hrxJbWrXCZCcnjZqhtFYrAO3cwEvlcqxLScHHGsz8N8PJCUNsbJRBibW+foVrrqvrLB+ku0okiKhJkP7TT9zwuxeYSIRSExPET5qEtpGR0MvPh6BsjVnZaZZrWbYhNjb4PTsbYUlJuF5QAACw0tPDZ82aIcTZGca1rVYllarTIGXAgAEIDAzEjBkzVPZv2bIFv//+O44cOaJxXn5+fujatSs2bNgAAJDL5XB1dcWsWbM0qtWIiIjAokWLkJqaWmmAMXjwYDx79gzR0dHKfe7u7pgzZw7mKBYDqSEKUmqmqrlNGOMWLMvOfrm/d29g6lRg8GBu5lR1tDGKA6i/ToiWenpY0KwZnslkyC0t5bYXHS0Vj3NLS/G0pAQ16RFhLBQq+wgo+woYGiofm4tE8LhwQSs3I23e2LSZl6ZNKvVeK1Auz1e9gTeE63zlgH/ECK46VDE925AhwJYtL9crmDYNOHiQ++chEHDfYnbv1krZ5IxhX2YmFiclIeH5cwCAvb4+PndzwxRHRxiUCVZo9tpXU6dBipWVFc6ePYt27dqp7I+Pj8frr7+O7LJ3mypIpVIYGRlh//79GDx4sHJ/cHAwcnJy8Ntvv1Wbh6enJ7p3747vvvtO7evp6elwcXHBTz/9hNFlZu5yd3dHUVERSkpK0KxZM4wePRpz586FXiXVe8XFxSgu888lLy8Prq6uFKRooLq5TRSsrIAJE7h5nVq3rjptfTeryBlDVkkJ0l90PEx/0RFR8fxWQQEu5edrfF5t+NLdHVOdnGCtr19pE6eCNm9GupiXztYKqCnnq9zYGsp11lppKfeNJS+Pm5ht61Zg+PCK6fbu5b7F5ORwq2o+eVL7TmTqiiGXY2dGBpbcv4+kF81ALhIJFrq5YYKDAw5lZ2vl/09TVqerIBcXF6NUTe/3kpISPH8RfWoiKysLMpkM9uXaE+3t7RGvQUe/ixcvIi4uDpGRkZWm+emnn2Bqaoqgcm0AH330ETp37gwrKyucO3cOCxYsQGpqKtasWaM2n/DwcCxdulSDqyLlVbdyrsKuXdzQ4epUFlikFBdj2I0bGt3Y5Iwhu6QEIXfuqP1nr9g38uZNWOnpIaukBDUbn6He62Zm8DYxgbmeHreJRDDX04NZmcc3CwowUoPV8143N4eNosNONYJsbbG/Qwe1/1hrejPSxbxEAgHWtmyJYTduKPsZKChu1REtW9YoIAiytcV7NjZa/bYsEgheaS6OhnKdtfb8OdC8OeDh8bL2RJ3hw7nhe9Omcb3pCwu5VTC1RE8oxDgHB4yys8P2tDQsf/AAj4qLMe32bSxKSkKGmmbWmvz/IZqrVU1K79690bFjR6xfv15lf0hICK5du4bTp09rlM/jx4/h7OyMc+fOoXv37sr9n376KU6dOoULFy5UefzUqVMRExODa9euVZqmbdu26Nu3b4WylvfDDz9g6tSpyM/Ph0RN+wLVpNReTaefr4omzSrWenpY6u6Op6WlyC4txZOSEmSXlCC7tJT7WVKCp6WlVfb0rypvZefDMh0Rn5aU4P8ePqz2eE2q4evi23LZvLV1M9LFvHSqVqAONerrlMlqPjSvjvuMFMlk+D41FV/ev4+MKoan0+y1mqvTmpQvv/wS/v7+uHr1Kvr06QMAiI6OxqVLl/DXX39pnI+NjQ1EIhHSy62AmZ6eDgcHhyqPLSgowO7du7Fs2bJK05w+fRoJCQnYs2dPtWXx8/NDaWkp7t+/jzZt2lR4XSKRqA1eSPVUPn9CBnjmvJyW/boFIOf+mKtbYZcxhn0ZGVUGKACQXVqKmYmJr1Tmsr7y8MB4BwfY6utDXyhUm0bGGHZkZFQbWPS0sKj2fHXxbbls3tqaVVMX89KpWoE61Kivs6YBRz10ajUQiTDLxQWtDA0xoIqO4wzAw+JinM7JodlrtaRWQcrrr7+OmJgYrFq1Cnv37oWhoSG8vLwQGRmJVq1aaZyPWCyGr68voqOjlX1S5HI5oqOjMXPmzCqP3bdvH4qLizF27NhK00RGRsLX1xfeinXmq3DlyhUIhUK1I4pI7cXHA6GhL570zARmJgJ2ZYKMDAmwsSVck2wrLC+fKZXi0rNnuPTsGS7m5eHSs2eVTi1eXhdTU3gbG8NaX1+5WenpvXyup4drBQUIqKIWTqGHmRmcqglQtR1YaLNJpanRZvCky5rKdeoSTSf5o6H/2lPrQeA+Pj7YsWPHKxcgNDQUwcHB6NKlC7p164aIiAgUFBRgwoQJAIBx48bB2dkZ4eHhKsdFRkZi8ODBsLa2VptvXl4e9u3bh9WrV1d4LSYmBhcuXEDv3r1hamqKmJgYzJ07F2PHjoUl/dFrza+/ciMDnz0DTAZmIn/ejYqJbIqBJTcQlN8Wp/MkXEDy7Bku5eXhgZoaE00n5FrVvHm1/8BtxWK4vJgX41VrPwDtBxaN+tsyIQ2Qo4Z9wDRNR6pXqyAlufyysuU0a9ZM47xGjBiBzMxMLFq0CGlpafDx8cHRo0eVnWmTk5MhLFfFnpCQgDNnzlTZtLR7924wxjBKTScHiUSC3bt3Y8mSJSguLoaHhwfmzp2LUOVXfqIN9+5xAcqbvRjuLEhEfgleVisoCAEwYK1pPNaqGVnZ1sgIXU1N0dXUFN3MzNDRyAhtL13S2WYVbQcW9G2ZEN3R08JCq19sSPVq1XFWKBRWOeRRVsN1KhoimieleowBv/wCOA54ir5x1c/tYKevj57m5uhqZoaupqbwNTWFuZoh4dqe26FRd0IkhGhVZf9/FA7Q6B6N1GnH2X/LrZVQUlKCf//9F2vWrMFXX31VmyxJI3D1KvD559xoHjMzbp6lDz4AdqVr1j4b0bKlRquQUrMKIYQvlf3/AbgJG/2p5lOrtLp2z+HDh7Fq1SqcPHlSW1nqLKpJUbVrFzBpEjfNQUgI8GICYQDA71lZeE+DVUg1nSVTgWZ8JITwpez/H2s9Pcy4cwd3i4ow29kZETUYQNJU1WlNSmXatGmDS5cuaTNLouNKS4FPPwW+/ZZ7HhAAlB0VnlBYiNBqhgPXth2X+msQQvhS/v/PZoEA/a5dw/qUFIxzcEBnLU4u15Spn/ShGnl5eSpbbm4u4uPjsXDhwhoNQSYNW0YG0LfvywDl88+Bw4e56e0B4NiTJ/CLjcXdoiLY6OlBgIr9Zl913g9CCNEFfa2sMMrODnIAU2/fhkx7jRRNWq1qUiwsLCp0nGWMwdXVFbs1XOiJNAwyGTetfWoqN9laz57c3EnXrgGDBnHT3ZuYcAuRll15YGNKCmbfuQMZuHlGDnbsiDO5uTTvByGk0VrTogWOZGfjf8+eYcvjxwhxdua7SA1erfqknDp1SuW5UCiEra0tWrZsWekCfY1NU+iTEhUFzJ6tuu6Oiwuwdi3Qowfg68sFKAcPAu3bc6+XyOWYnZiIzY8fAwDG2dvjuzZtIHkxjJz6kRBCGrNNKSkIuXMHZiIR4rt1gyPNVK5Wna6CrHDz5k0kJydDWm52vXfffbe2WTYYjT1IiYoChg3jhhGXpYgn9u/nVip2dQXMzbl9T0pKMPzGDUTn5EAA4OvmzfGJq2u1K/QSQkhjIWMM3S9fxqVnzzDSzg67FN/giIo6DVLu3buHoKAgXLt2DQKBAIosFDcjmielYZPJAHf3ylcuFgi4GpWkpJfLZiQUFiLw+nXcef4cxkIhdrZvj3dtbOqtzIQQoisuP3uGrrGxkAP408sL/RQd9YiSpvfQWnWcnT17Ntzd3ZGRkQEjIyPExcXhn3/+QZcuXZrE8OPG7vTpygMUgKtdefiQSwdwHWRfu3wZd54/RzOJBOc6d6YAhRDSZHU2NcVHLi4AgBm3b+N5E/jiXldqFaTExMRg2bJlsLGxgVAohEgkwhtvvIHw8HB89NFH2i4jqWepqZqn25iSggHXriGntBQ9zMxw0dcXXiYmdVtAQgjRccvc3eEsFuNuURHCq1lKhlSuVkGKTCaD6Ysx4DY2Nnj8opOkm5sbEhIStFc6wgtHRw0SieTY43gbM1+M4PnA3h7R3t6wp4W1CCEEpnp6WPtiSo6vk5MRX1DAc4kaploFKR07dsTVq9xaLH5+fli5ciXOnj2LZcuWoXnz5lotIKl/PXtyfU4EAgBCBng/Bd5O534KGWBaAknEdfyGxxAACPfwwE9t28JA0UGFEEIIgmxsMNDKCiWMYcadO9DiBO9NRq3GCy9cuBAFL6LCZcuW4Z133kHPnj1hbW2NPXv2aLWApP6JRNww46ERmcDMRMCuzPoU2WKAAcU2UhgLhfilXTsMpjlOCCGkAoFAgA2tWqH9pUs4kZODHenpGOvgwHexGhStrd3z5MkTWFpaNpnhpo15dA8A7M/IxPsvVvpUmSaWcc+t9fQQ7eMDb+p/QgghVQp/8ACfJyXBVl8f8d26wUpfn+8i8a5OR/eoY2Vl1WQClMZOxhjm3k1EVfPYS4RCdDQ2ru+iEUJIg/OxqyvaGxkhs6QEC+7d47s4DYrWghTSeJzOyamwBHl5j6VSnM7JqZ8CEUJIAyYWCrG5dWsAwHepqTiXm8tziRoOClKIiocPgalfSKtPCCBVqlk6Qghp6t60sMCEF/1Rpt2+jRK5nOcSNQwUpBAVH38M3D6n2TBiRxpuTAghGlvZvDms9PRwvaAA61JS+C5Og0BBClE6fhzYtw8Q3LCAhaDygV8CAK4SCXpaWNRb2QghpKGzEYuxqkULAMCipCQkFxXxXCLdR0EKAQBIpcCsWdzjIQtzUQD10zgr+tFGtGxJqxcTQkgNjXdwQE9zcxTK5fjozh2+i6PzKEghAIB164D4eMDSNx/He19HCWPwNTGBc7kmHReJBPs7dEAQzY1CCCE1JhQIsLl1a+gJBPgtOxu/ZWXxXSSdVqvJ3Ejj8vgxsHQpAIfnkH99DXlyGd4wN8efXl6QCIU4nZODVKkUjmIxelpYUA0KIYS8gg7Gxpjn6oqvk5Mx684d9LGwgIke3Y7VoXeFYNs2IF9fCsn6a8jVk8LT2Bi/d+wIoxfT3PeytOS5hIQQ0riEublhd0YG7hcVYcn9+3jH2pq+DKqhtRlnm5rGNONsbkkpvE9cwQNxPtwNDHC2Uyc4SSR8F4sQQhq1w9nZeOf69Qr7XSQSrG3ZslE3q9f7jLOkYSqSyTDkRhweiPNhq6+Pv7y8KEAhhJB6UFzJXCkpxcUYduMGojIz67lEuoeClCbs1GmGEddv4URODkxFIhz18kIrIyO+i0UIIY2ejDHMTkxU+5qieWNOYiJkTbyxQyeClI0bN8Ld3R0GBgbw8/PDxYsXK03bq1cvCASCCtugQYOUacaPH1/h9f79+6vk8+TJE4wZMwZmZmawsLDApEmTkJ+fX2fXqGvS0hj6HrmN33OyoA8BfuvYEZ1NTfkuFiGENAnVLT/CADwsLm7yy4/wHqTs2bMHoaGhWLx4MS5fvgxvb28EBAQgIyNDbfqoqCikpqYqt7i4OIhEIrz//vsq6fr376+SbteuXSqvjxkzBjdu3MCxY8dw6NAh/PPPP5gyZUqdXaeuCdibhJKAVEAO7GzXHr2pcywhhNQbTZcVaerLj/A+umfNmjWYPHkyJkyYAADYsmULDh8+jB9++AHz58+vkN7Kykrl+e7du2FkZFQhSJFIJHB4sU5Cebdu3cLRo0dx6dIldOnSBQCwfv16DBw4EN988w2cnJwqHFNcXIziMlFvXl5ezS5Uh8w5/QjXvJIBAJ+JW2OYfePtnEUIIbpI02VFmvryI7zWpEilUsTGxsLf31+5TygUwt/fHzExMRrlERkZiZEjR8LY2Fhl/8mTJ2FnZ4c2bdpg+vTpyM7OVr4WExMDCwsLZYACAP7+/hAKhbhw4YLa84SHh8Pc3Fy5ubq61uRSdcZ/UtOxVsa1g3b+1wNfv1ExICOEEFK3elpYwEUiQVUDjV1o+RF+g5SsrCzIZDLY29ur7Le3t0daWlq1x1+8eBFxcXH48MMPVfb3798fP//8M6Kjo/F///d/OHXqFAYMGACZjJvqPS0tDXZ2dirH6OnpwcrKqtLzLliwALm5ucrt4cOHNblUnXA0Oxvj4+MBAOJDzjgyuhnPJSKEkKZJJBBgbcuWAFBpoNLG0JD/Phk8472551VERkbC09MT3bp1U9k/cuRI5WNPT094eXmhRYsWOHnyJPr06VOrc0kkEkga8NDcmNxcDI27AbmAAcftsMq9JeztabIgQgjhS5CtLfZ36IDZiYkqnWht9PWRXVKC6JwchCUl4cvmzXksJb94DVJsbGwgEomQnp6usj89Pb3S/iQKBQUF2L17N5YtW1bteZo3bw4bGxskJiaiT58+cHBwqNAxt7S0FE+ePKn2vA2BjDGVqeyt9PUx6Pp1FDI5HB9aweaPtphxgQIUQgjhW5CtLd6zsamw/MgPqamYcvs2vkpOhoNYjJkuLnwXlRe8BilisRi+vr6Ijo7G4MGDAQByuRzR0dGYOXNmlcfu27cPxcXFGDt2bLXnefToEbKzs+Ho6AgA6N69O3JychAbGwtfX18AwN9//w25XA4/P79XuyieRWVmVojKhQDkAPxMTRE9ugMQJAQtE0EIIbpBJBBUWH5kspMTUqVSLL5/Hx8lJsJeLMb75bopNAW8N3eFhobi+++/x08//YRbt25h+vTpKCgoUI72GTduHBYsWFDhuMjISAwePBjW1tYq+/Pz8/HJJ5/g/PnzuH//PqKjo/Hee++hZcuWCAgIAAC0a9cO/fv3x+TJk3Hx4kWcPXsWM2fOxMiRI9WO7GkoojIzMezGjQpj7xVzGk53doaxSIRyfYwJIYTooDA3N0x3cgIDMPbWLZx4+pTvItU73r9PjxgxApmZmVi0aBHS0tLg4+ODo0ePKjvTJicnQyhUjaUSEhJw5swZ/PXXXxXyE4lEuHbtGn766Sfk5OTAyckJ/fr1w/Lly1X6lOzYsQMzZ85Enz59IBQKMXToUKxbt65uL7YOKWYvrHRuQgYsuJOEsfb2tHAVIYQ0AAKBAOtbtUKGVIoDWVl4Ly4O//j4wKceJ94s332gvhc/pAUGa0nXFhg8+fQpel+9Wm26E97etKoxIYQ0IEUyGfpfu4ZTublwEItxtlMnNDc0rPPzqus+oK3FD2mBwSaGZi8khJDGyUAkwq8dO8LL2BhpUikCrl1DRh3/L6+s+0B9L35IQUojQbMXEkJI42Whr48/vLzgJpEg8flzDLp+HfmlpXVyrqq6D9T34ocUpDQSPS0sYCrXQ6WdUuSAtYxmLySEkIbKSSLBn97esNbTw/+ePcPQGzcglcurP7CGdGnxQwpSGokrefl4Virjpi4sH6jIwe3f2BKQU6dZQghpqNoYGeGIlxeMhEL89fQpJsTHQ67FGo2r+flYmJSkUdr66D5AQUoj8Li4GP3/vQ6IGZBoDGSWa9LJlACLOyD7oC1On+anjIQQQrSjm5kZDnToAD2BADszMvDJ3buvlB9jDH8/fYr+V6/C53//w1kNF9Ctj+4DvA9BJq+mQCZD4PXryIIUuG8EzPEBnusBnjmAtRTIFgPXLZQ1KKmpfJaWEEKINvS3tsYPbdpgXHw81jx6BEexGPOa1Ww9tlK5HAeysrAyORmX8/MBcDUXw2xtcTInB5klJWp7EAhQf4sfUpDSgMkZw9hbt3A5Px/m0Efu555AgT734lX1w4xfTLpLCCGkgfvAwQHpUik+uXcPn9y7B3uxGKPt7aud16RQJsP2tDSsfvgQSUVFAABDoRCTHB0R6uICD0ND5eie8j0IFDlFtGxZL/Ol0DwptaQL86R8evcuVj18CIlAgGNePhjtZY6UFEDdb1QgAFxcgKQkQCSq/7ISQgipGx8nJmLNo0cQArDS10dWSYnytbLzmmRJpdj4+DHWP3qE7Bcjg6z19DDLxQUhTk6wKdd8o26eFFeJBBH1OE8KBSm1xHeQsu3xY0y+fRsAsKNdO4y2t0dUFDBsWMUgRRHs7t8PBAXVc0EJIYTUKTlj6PXvvzitpi+Joq6jv5UVTubk4PmL0UAeBgaY5+qK8Q4OMKrim2tdzTir6T2UmnsaoOinTzH9zh0AwBJ3d4x+sYRAUBAXiAwfDshkL9O7uAARERSgEEJIY8QA3HvRbKPuNQD448kTAICviQk+bdYMQTY20BNWP3ZG3eKH9YmClAYmvqAAQ+PiUMoYRtvZYZGbm8rrvXq9DFC+/x5o2RLo2ZOaeAghpLE6nZODFA2GA69u0QJzXVwgaEDrt1GQ0oBkSaUYdP06cmUy9DAzQ2SbNhU+bJcvcz9btAA+/JCHQhJCCKlXms5X4igWN6gABaB5UhqMYrkcQ27cwL2iIngYGODXjh1hoKZ6RBGk+PrWcwEJIYTwojEvi0JBSgPAGMOHCQk4k5sLc5EIhzw9YVvJh00RpHTuXI8FJIQQwpueFhZwkUhQWR2JANyonIa4LAoFKQ3Alw8e4Jf0dIgA7O/QAe2NjStNS0EKIYQ0LSKBAGtbtgSACoFKfc9rom0UpOi43enpWHT/PgBgU+vW8LeyqjTt8+dAWhr3mIIUQghpOoJsbbG/Qwc4SyQq+10kEuzv0OGV5zXhC3Wc1WExubkYHx8PAAh1ccEUJ6cq0xsaAk+fAvfuAdbW9VFCQgghuiLI1hbv2djUybwmfKEgRUclPX+O9+LiUMwY3rW2xsoWLTQ6TiQCWrWq48IRQgjRSXzPa6JtFKToiLKz+pmKRPj07l1klpTAx8QEO9q1a9CRMCGEEFIbFKToAHXrIwCApZ4e/tuxI0z0NPs1vfsuYGoKfPkl4OFRFyUlhBBC6g91nOWZYqXJ8gEKADwtLcXFZ880yqegADh8GNi5k+ubQgghhDR0FKTwSMYYZicmorIVHgUA5iQmQqbBGpBXrwJyOeDoCDg4aLWYhBBCCC8oSOHR6ZwctTUoCgzAw+JinM7JqTYvmmmWEEJIY0NBCo80XW9Bk3Q0iRshhJDGhoIUHmlzvQUKUgghhDQ2OhGkbNy4Ee7u7jAwMICfnx8uXrxYadpevXpBIBBU2AYNGgQAKCkpwWeffQZPT08YGxvDyckJ48aNw+PHj1XycXd3r5DH119/XafXWZ621lsoKgJu3OAeU5BCCCGkseA9SNmzZw9CQ0OxePFiXL58Gd7e3ggICEBGRoba9FFRUUhNTVVucXFxEIlEeP/99wEAhYWFuHz5MsLCwnD58mVERUUhISEB7777boW8li1bppLXrFmz6vRay9PWegtpaUD79lynWRcX7ZeTEEII4YOAMQ2GjtQhPz8/dO3aFRs2bAAAyOVyuLq6YtasWZg/f361x0dERGDRokVITU2FcSUL7126dAndunXDgwcP0KxZMwBcTcqcOXMwZ86cWpU7Ly8P5ubmyM3NhZmZWa3yUFA3T4qrRIKIli1rtN5CaSmg4ZQqhBBCCG80vYfyWpMilUoRGxsLf39/5T6hUAh/f3/ExMRolEdkZCRGjhxZaYACALm5uRAIBLAo12zy9ddfw9raGp06dcKqVatQWlpaaR7FxcXIy8tT2bQlyNYW9197DSe8vbGzXTuc8PZG0muv1XhBKApQCCGENCa83taysrIgk8lgb2+vst/e3h7xLxbWq8rFixcRFxeHyMjIStMUFRXhs88+w6hRo1SitY8++gidO3eGlZUVzp07hwULFiA1NRVr1qxRm094eDiWLl2q4ZXV3KustyCTcWv2EEIIIY1Jg/7uHRkZCU9PT3Tr1k3t6yUlJRg+fDgYY9i8ebPKa6GhocrHXl5eEIvFmDp1KsLDwyEpt9Q1ACxYsEDlmLy8PLi6umrpSmqvpASwtQXatAH++AOwsuK7RIQQQoh28NrcY2NjA5FIhPT0dJX96enpcKhm2tSCggLs3r0bkyZNUvu6IkB58OABjh07Vm2/ET8/P5SWluL+/ftqX5dIJDAzM1PZdMGNG0BuLpCQADSihS8JIYQQfoMUsVgMX19fREdHK/fJ5XJER0eje/fuVR67b98+FBcXY+zYsRVeUwQod+7cwfHjx2FtbV1tWa5cuQKhUAg7O7uaXwiPys6PQgslE0IIaUx4b+4JDQ1FcHAwunTpgm7duiEiIgIFBQWYMGECAGDcuHFwdnZGeHi4ynGRkZEYPHhwhQCkpKQEw4YNw+XLl3Ho0CHIZDKkpaUBAKysrCAWixETE4MLFy6gd+/eMDU1RUxMDObOnYuxY8fCsoFVR9AkboQQQhor3oOUESNGIDMzE4sWLUJaWhp8fHxw9OhRZWfa5ORkCIWqFT4JCQk4c+YM/vrrrwr5paSk4PfffwcA+Pj4qLx24sQJ9OrVCxKJBLt378aSJUtQXFwMDw8PzJ07V6XPSUNBa/YQQghprHifJ6Wh0uY8KbVVWgqYmQHPnwPx8VznWUIIIUTXNYh5UsirSUjgAhQTE6BVK75LQwghhGgX7809pPbkciAoCNDXB4QUbhJCCGlkKEhpwDw9gQMH+C4FIYQQUjfo+zchhBBCdBIFKQ2UXA48eABQt2dCCCGNFQUpDVRiIuDuDjRrxgUshBBCSGNDQUoDpZgfxcWFOs0SQghpnOj21kDRTLOEEEIaOwpSGigKUgghhDR2FKQ0QIxRkEIIIaTxoyClAbp/H3j6lJvErUMHvktDCCGE1A0KUhogRS2KpycgFvNbFkIIIaSu0IyzDVCrVsCnnwIODnyXhBBCCKk7FKQ0QF5e3EYIIYQ0ZtTcQwghhBCdREFKA/P0KRAdDTx5wndJCCGEkLpFQUoDc/o04O8PvPUW3yUhhBBC6hYFKQ2MYmSPry+/5SCEEELqGgUpDQxN4kYIIaSpoCClgaEghRBCSFNBQUoDkp4OpKQAAgHg48N3aQghhJC6RUFKA/Lvv9zPNm0AExN+y0IIIYTUNQpSGhBq6iGEENKU0IyzDcj77wM2NoCHB98lIYQQQuoeBSkNSKtW3EYIIYQ0BdTcQwghhBCdpBNBysaNG+Hu7g4DAwP4+fnh4sWLlabt1asXBAJBhW3QoEHKNIwxLFq0CI6OjjA0NIS/vz/u3Lmjks+TJ08wZswYmJmZwcLCApMmTUJ+fn6dXeOrunkT2LoVuHaN75IQQggh9YP3IGXPnj0IDQ3F4sWLcfnyZXh7eyMgIAAZGRlq00dFRSE1NVW5xcXFQSQS4f3331emWblyJdatW4ctW7bgwoULMDY2RkBAAIqKipRpxowZgxs3buDYsWM4dOgQ/vnnH0yZMqXOr7e2Dh0Cpk0DvvyS75IQQggh9YTxrFu3biwkJET5XCaTMScnJxYeHq7R8d9++y0zNTVl+fn5jDHG5HI5c3BwYKtWrVKmycnJYRKJhO3atYsxxtjNmzcZAHbp0iVlmj/++IMJBAKWkpKi0Xlzc3MZAJabm6tR+lc1YgRjAGMavi2EEEKIztL0HsprTYpUKkVsbCz8/f2V+4RCIfz9/RETE6NRHpGRkRg5ciSMjY0BAElJSUhLS1PJ09zcHH5+fso8Y2JiYGFhgS5duijT+Pv7QygU4sKFC2rPU1xcjLy8PJWtPtHwY0IIIU0Nr0FKVlYWZDIZ7O3tVfbb29sjLS2t2uMvXryIuLg4fPjhh8p9iuOqyjMtLQ12dnYqr+vp6cHKyqrS84aHh8Pc3Fy5ubq6Vn+BWpKXByi61FCQQgghpKngvU/Kq4iMjISnpye6detW5+dasGABcnNzldvDhw/r/JwKV65wP5s14+ZJIYQQQpoCXoMUGxsbiEQipKenq+xPT0+Hg4NDlccWFBRg9+7dmDRpksp+xXFV5eng4FChY25paSmePHlS6XklEgnMzMxUtvoSG8v9pFoUQgghTQmvQYpYLIavry+io6OV++RyOaKjo9G9e/cqj923bx+Ki4sxduxYlf0eHh5wcHBQyTMvLw8XLlxQ5tm9e3fk5OQgVnH3B/D3339DLpfDz89PG5emVdQfhRBCSFPE+4yzoaGhCA4ORpcuXdCtWzdERESgoKAAEyZMAACMGzcOzs7OCA8PVzkuMjISgwcPhrW1tcp+gUCAOXPm4Msvv0SrVq3g4eGBsLAwODk5YfDgwQCAdu3aoX///pg8eTK2bNmCkpISzJw5EyNHjoSTk1O9XHdNrFsHTJgAuLnxXRJCCCGk/vAepIwYMQKZmZlYtGgR0tLS4OPjg6NHjyo7viYnJ0MoVK3wSUhIwJkzZ/DXX3+pzfPTTz9FQUEBpkyZgpycHLzxxhs4evQoDAwMlGl27NiBmTNnok+fPhAKhRg6dCjWrVtXdxf6Ciwtgbff5rsUhBBCSP0SMMYY34VoiPLy8mBubo7c3Nx67Z9CCCGENHSa3kMb9OiepuC334B584BTp/guCSGEEFK/eG/uIVX77Tdg+3bAyAh46y2+S0MIIYTUH6pJ0XE0socQQkhTRUGKDisqAm7c4B77+vJbFkIIIaS+UZCiw+LigNJSbpZZFxe+S0MIIYTULwpSdFjZmWYFAn7LQgghhNQ3ClJ0GPVHIYQQ0pRRkKLD7t7lflJ/FEIIIU0RDUHWYceOAQ8fcjPOEkIIIU0NBSk6TCAAmjXjuxSEEEIIP6i5hxBCCCE6iYIUHbV0KTBkCNfkQwghhDRFFKToqD//BH79FcjM5LskhBBCCD8oSNFBMhlw5Qr3mIYfE0IIaaooSNFBCQnA8+eAiQnQujXfpSGEEEL4QUGKDlLMNOvjAwjpN0QIIaSJolugDqKZZgkhhBAKUnQSBSmEEEIIBSk6SSIBDAwoSCGEENK00YyzOuivv4DSUuqPQgghpGmjIEVH6dFvhhBCSBNH39V1jFzOdwkIIYQQ3UBBio4ZOBDw9AROneK7JIQQQgi/qFFBhzAGXLwIPH0KmJnxXRpCCCGEX1STokMePOACFH19oEMHvktDCCGE8IuCFB0hkwE//cQ9dncHRCJei0MIIYTwjvcgZePGjXB3d4eBgQH8/Pxw8eLFKtPn5OQgJCQEjo6OkEgkaN26NY4cOaJ83d3dHQKBoMIWEhKiTNOrV68Kr0+bNq3OrrE6UVFcYLJkCff8zh3ueVQUb0UihBBCeMdrn5Q9e/YgNDQUW7ZsgZ+fHyIiIhAQEICEhATY2dlVSC+VStG3b1/Y2dlh//79cHZ2xoMHD2BhYaFMc+nSJchkMuXzuLg49O3bF++//75KXpMnT8ayZcuUz42MjLR/gRqIigKGDeP6o5SVksLt378fCAripWiEEEIIrwSMlb891h8/Pz907doVGzZsAADI5XK4urpi1qxZmD9/foX0W7ZswapVqxAfHw99fX2NzjFnzhwcOnQId+7cgUAgAMDVpPj4+CAiIkLjshYXF6O4uFj5PC8vD66ursjNzYVZLXu5ymRcjcmjR+pfFwgAFxcgKYmafwghhDQeeXl5MDc3r/Yeyltzj1QqRWxsLPz9/V8WRiiEv78/YmJi1B7z+++/o3v37ggJCYG9vT06duyIFStWqNSclD/HL7/8gokTJyoDFIUdO3bAxsYGHTt2xIIFC1BYWFhlecPDw2Fubq7cXF1da3jFFZ0+XXmAAnC1Kw8fcukIIYSQpoa35p6srCzIZDLY29ur7Le3t0d8fLzaY+7du4e///4bY8aMwZEjR5CYmIgZM2agpKQEixcvrpD+119/RU5ODsaPH6+yf/To0XBzc4OTkxOuXbuGzz77DAkJCYiqohPIggULEBoaqnyuqEl5Famp2k1HCCGENCYNap4UuVwOOzs7fPfddxCJRPD19UVKSgpWrVqlNkiJjIzEgAED4OTkpLJ/ypQpyseenp5wdHREnz59cPfuXbRo0ULtuSUSCSQSiVavx9FRu+kIIYSQxoS35h4bGxuIRCKkp6er7E9PT4eDg4PaYxwdHdG6dWuIynTQaNeuHdLS0iCVSlXSPnjwAMePH8eHH35YbVn8/PwAAImJiTW9jFfSsyfX56RcS5SSQAC4unLpCCGEkKaGtyBFLBbD19cX0dHRyn1yuRzR0dHo3r272mNef/11JCYmQl5mgZvbt2/D0dERYrFYJe327dthZ2eHQYMGVVuWK1euAOCCoPokEgFr13KPywcqiucREdRplhBCSNPE6zwpoaGh+P777/HTTz/h1q1bmD59OgoKCjBhwgQAwLhx47BgwQJl+unTp+PJkyeYPXs2bt++jcOHD2PFihUqc6AAXLCzfft2BAcHQ6/ccsJ3797F8uXLERsbi/v37+P333/HuHHj8Oabb8LLy6vuL7qcoCBumLGzs+p+FxcafkwIIaRp47VPyogRI5CZmYlFixYhLS0NPj4+OHr0qLIzbXJyMoTCl3GUq6sr/vzzT8ydOxdeXl5wdnbG7Nmz8dlnn6nke/z4cSQnJ2PixIkVzikWi3H8+HFERESgoKAArq6uGDp0KBYuXFi3F1uFoCDgvfe4UTypqVwflJ49qQaFEEJI08brPCkNmaZjvAkhhBCiSufnSSGEEEIIqQoFKYQQQgjRSRSkEEIIIUQnUZBCCCGEEJ1EQQohhBBCdBIFKYQQQgjRSQ1q7R5dohi5nZeXx3NJCCGEkIZFce+sbhYUClJq6dmzZwDwyishE0IIIU3Vs2fPYG5uXunrNJlbLcnlcjx+/BiMMTRr1gwPHz6kSd14kJeXB1dXV3r/eULvP7/o/ecXvf+1xxjDs2fP4OTkpDKzfHlUk1JLQqEQLi4uyiorMzMz+pDyiN5/ftH7zy96//lF73/tVFWDokAdZwkhhBCikyhIIYQQQohOoiDlFUkkEixevBgSiYTvojRJ9P7zi95/ftH7zy96/+sedZwlhBBCiE6imhRCCCGE6CQKUgghhBCikyhIIYQQQohOoiCFEEIIITqJgpRXsHHjRri7u8PAwAB+fn64ePEi30VqEpYsWQKBQKCytW3blu9iNWr//PMPAgMD4eTkBIFAgF9//VXldcYYFi1aBEdHRxgaGsLf3x937tzhp7CNUHXv//jx4yv8TfTv35+fwjYy4eHh6Nq1K0xNTWFnZ4fBgwcjISFBJU1RURFCQkJgbW0NExMTDB06FOnp6TyVuHGhIKWW9uzZg9DQUCxevBiXL1+Gt7c3AgICkJGRwXfRmoQOHTogNTVVuZ05c4bvIjVqBQUF8Pb2xsaNG9W+vnLlSqxbtw5btmzBhQsXYGxsjICAABQVFdVzSRun6t5/AOjfv7/K38SuXbvqsYSN16lTpxASEoLz58/j2LFjKCkpQb9+/VBQUKBMM3fuXPz3v//Fvn37cOrUKTx+/BhBQUE8lroRYaRWunXrxkJCQpTPZTIZc3JyYuHh4TyWqmlYvHgx8/b25rsYTRYAdvDgQeVzuVzOHBwc2KpVq5T7cnJymEQiYbt27eKhhI1b+fefMcaCg4PZe++9x0t5mpqMjAwGgJ06dYoxxn3W9fX12b59+5Rpbt26xQCwmJgYvorZaFBNSi1IpVLExsbC399fuU8oFMLf3x8xMTE8lqzpuHPnDpycnNC8eXOMGTMGycnJfBepyUpKSkJaWprK34O5uTn8/Pzo76EenTx5EnZ2dmjTpg2mT5+O7OxsvovUKOXm5gIArKysAACxsbEoKSlR+fy3bdsWzZo1o8+/FlCQUgtZWVmQyWSwt7dX2W9vb4+0tDSeStV0+Pn54ccff8TRo0exefNmJCUloWfPnnj27BnfRWuSFJ95+nvgT//+/fHzzz8jOjoa//d//4dTp05hwIABkMlkfBetUZHL5ZgzZw5ef/11dOzYEQD3+ReLxbCwsFBJS59/7aBVkEmDM2DAAOVjLy8v+Pn5wc3NDXv37sWkSZN4LBkh/Bg5cqTysaenJ7y8vNCiRQucPHkSffr04bFkjUtISAji4uKoD1w9opqUWrCxsYFIJKrQezs9PR0ODg48larpsrCwQOvWrZGYmMh3UZokxWee/h50R/PmzWFjY0N/E1o0c+ZMHDp0CCdOnICLi4tyv4ODA6RSKXJyclTS0+dfOyhIqQWxWAxfX19ER0cr98nlckRHR6N79+48lqxpys/Px927d+Ho6Mh3UZokDw8PODg4qPw95OXl4cKFC/T3wJNHjx4hOzub/ia0gDGGmTNn4uDBg/j777/h4eGh8rqvry/09fVVPv8JCQlITk6mz78WUHNPLYWGhiI4OBhdunRBt27dEBERgYKCAkyYMIHvojV68+bNQ2BgINzc3PD48WMsXrwYIpEIo0aN4rtojVZ+fr7Kt/KkpCRcuXIFVlZWaNasGebMmYMvv/wSrVq1goeHB8LCwuDk5ITBgwfzV+hGpKr338rKCkuXLsXQoUPh4OCAu3fv4tNPP0XLli0REBDAY6kbh5CQEOzcuRO//fYbTE1Nlf1MzM3NYWhoCHNzc0yaNAmhoaGwsrKCmZkZZs2ahe7du+O1117jufSNAN/Dixqy9evXs2bNmjGxWMy6devGzp8/z3eRmoQRI0YwR0dHJhaLmbOzMxsxYgRLTEzku1iN2okTJxiACltwcDBjjBuGHBYWxuzt7ZlEImF9+vRhCQkJ/Ba6Eanq/S8sLGT9+vVjtra2TF9fn7m5ubHJkyeztLQ0vovdKKh73wGw7du3K9M8f/6czZgxg1laWjIjIyM2ZMgQlpqayl+hGxEBY4zVf2hECCGEEFI16pNCCCGEEJ1EQQohhBBCdBIFKYQQQgjRSRSkEEIIIUQnUZBCCCGEEJ1EQQohhBBCdBIFKYQQQgjRSRSkEEIIIUQnUZBCCCEvnDx5EgKBoMJicYQQflCQQgghhBCdREEKIYQQQnQSBSmEEJ0hl8sRHh4ODw8PGBoawtvbG/v37wfwsinm8OHD8PLygoGBAV577TXExcWp5HHgwAF06NABEokE7u7uWL16tcrrxcXF+Oyzz+Dq6gqJRIKWLVsiMjJSJU1sbCy6dOkCIyMj9OjRAwkJCXV74YQQtShIIYTojPDwcPz888/YsmULbty4gblz52Ls2LE4deqUMs0nn3yC1atX49KlS7C1tUVgYCBKSkoAcMHF8OHDMXLkSFy/fh1LlixBWFgYfvzxR+Xx48aNw65du7Bu3TrcunULW7duhYmJiUo5vvjiC6xevRr/+9//oKenh4kTJ9bL9RNCyuF7GWZCCGGMsaKiImZkZMTOnTunsn/SpEls1KhR7MSJEwwA2717t/K17OxsZmhoyPbs2cMYY2z06NGsb9++Ksd/8sknrH379owxxhISEhgAduzYMbVlUJzj+PHjyn2HDx9mANjz58+1cp2EEM1RTQohRCckJiaisLAQffv2hYmJiXL7+eefcffuXWW67t27Kx9bWVmhTZs2uHXrFgDg1q1beP3111Xyff3113Hnzh3IZDJcuXIFIpEIb731VpVl8fLyUj52dHQEAGRkZLzyNRJCakaP7wIQQggA5OfnAwAOHz4MZ2dnldckEolKoFJbhoaGGqXT19dXPhYIBAC4/jKEkPpFNSmEEJ3Qvn17SCQSJCcno2XLliqbq6urMt358+eVj58+fYrbt2+jXbt2AIB27drh7NmzKvmePXsWrVu3hkgkgqenJ+RyuUofF0KI7qKaFEKITjA1NcW8efMwd+5cyOVyvPHGG8jNzcXZs2dhZmYGNzc3AMCyZctgbW0Ne3t7fPHFF7CxscHgwYMBAB9//DG6du2K5cuXY8SIEYiJicGGDRuwadMmAIC7uzuCg4MxceJErFu3Dt7e3njw4AEyMjIwfPhwvi6dEFIJClIIITpj+fLlsLW1RXh4OO7duwcLCwt07twZn3/+ubK55euvv8bs2bNx584d+Pj44L///S/EYjEAoHPnzti7dy8WLVqE5cuXw9HREcuWLcP48eOV59i8eTM+//xzzJgxA9nZ2WjWrBk+//xzPi6XEFINAWOM8V0IQgipzsmTJ9G7d288ffoUFhYWfBeHEFIPqE8KIYQQQnQSBSmEEEII0UnU3EMIIYQQnUQ1KYQQQgjRSRSkEEIIIUQnUZBCCCGEEJ1EQQohhBBCdBIFKYQQQgjRSRSkEEIIIUQnUZBCCCGEEJ1EQQohhBBCdNL/A1CKURyzGcddAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* background: */\n",
       "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
       "    progress {background-color: #CDCDCD;}\n",
       "\n",
       "    /* value: */\n",
       "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
       "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
       "    progress {color: #00BFFF ;}\n",
       "\n",
       "    /* optional */\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #000000;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      <progress value='23' class='progress-bar-interrupted' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      23.00% [23/100] [12:17<41:09]\n",
       "      <br>\n",
       "      ████████████████████100.00% [79/79] [val_loss=0.5473, val_auc=0.7431]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0;31m<<<<<< val_auc without improvement in 5 epoch,early stopping >>>>>> \n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "dfhistory = model.fit(train_data = dl_train,\n",
    "    val_data = dl_val,\n",
    "    epochs=100,\n",
    "    ckpt_path='checkpoint',\n",
    "    patience=5,\n",
    "    monitor='val_auc',\n",
    "    mode='max',\n",
    "    plot=True,\n",
    "    cpu=True\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26087443",
   "metadata": {},
   "source": [
    "### 4，评估模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4cb64f90-8df9-4045-b371-458646316756",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████| 98/98 [00:04<00:00, 23.15it/s, val_auc=0.782, val_loss=0.464]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'val_loss': 0.4642997295880804, 'val_auc': 0.7817735075950623}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(dl_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cead8403-a736-4e0f-8a40-5363f9a20720",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "27826a5a",
   "metadata": {},
   "source": [
    "### 5，使用模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b1fd052c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7817740760992791\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import roc_auc_score\n",
    "model.eval()\n",
    "dl_test = model.accelerator.prepare(dl_test)\n",
    "with torch.no_grad():\n",
    "    result = torch.cat([model.forward(t[0]) for t in dl_test])\n",
    "\n",
    "preds = F.sigmoid(result)\n",
    "labels = torch.cat([x[-1] for x in dl_test])\n",
    "\n",
    "val_auc = roc_auc_score(labels.numpy(),preds.numpy())\n",
    "print(val_auc)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df8aa18e",
   "metadata": {},
   "source": [
    "### 6，保存模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "961764d9-2459-43b8-87a0-ed97bfdcd247",
   "metadata": {},
   "source": [
    "模型最佳权重已经保存在 model.fit(ckpt_path) 传入的参数中了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c578b927",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net_clone = create_net()\n",
    "net_clone.load_state_dict(torch.load(model.ckpt_path))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0ce2aa6",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "0d010313",
   "metadata": {},
   "source": [
    "**如果本书对你有所帮助，想鼓励一下作者，记得给本项目加一颗星星star⭐️，并分享给你的朋友们喔😊!** \n",
    "\n",
    "如果对本书内容理解上有需要进一步和作者交流的地方，欢迎在公众号\"算法美食屋\"下留言。作者时间和精力有限，会酌情予以回复。\n",
    "\n",
    "也可以在公众号后台回复关键字：**加群**，加入读者交流群和大家讨论。\n",
    "\n",
    "![算法美食屋logo.png](https://tva1.sinaimg.cn/large/e6c9d24egy1h41m2zugguj20k00b9q46.jpg)"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "formats": "ipynb,md"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}