playground.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 文本分类的训练代码\n",
    "\n",
    "以`TextCNN`为例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先引入训练器`Trainer`和配置器`Config`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from trainer import Trainer\n",
    "from config import Config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 实例化一个配置器, 在构造函数里指定预设配置\n",
    "- 其他还需要额外指定的参数可以单独设置\n",
    "- 注意这里设置`embedding`预训练文件得用`set_embedding_pretrained`方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config = Config(from_json='./configs/TextCNN.json') # 从json文件中读取配置\n",
    "config.num_classes = 10 # 数据集里一共有多少个类别\n",
    "config.vocab_path = './THUCNews/data/vocab.pkl' # 词表路径\n",
    "config.set_embedding_pretrained('./THUCNews/data/embedding_Tencent.npz') # 预训练好的词表Embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设置好后就可以开始训练啦, 实例化一个训练器, 然后指定训练集路径, 验证集路径, 配置器.\n",
    "上述是必选参数\n",
    "此外你可以额外指定\n",
    "- 句子最大长度`padding_length`\n",
    "- 批训练大小`batch_size`\n",
    "- 验证集批训练大小`batch_size_eval`\n",
    "- 本次任务的名字`task_name`(随便取, 不过记得每次都要修改下名字, 这样结果保存下来才不会重名)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer = Trainer('./THUCNews/data/train.txt', './THUCNews/data/dev.txt', config, padding_length=64, batch_size=256, batch_size_eval=64, task_name='TextCNN_Try')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后定义一个for循环, 里面随便做点什么就可以开始训练了\n",
    "\n",
    "`trainer`里还可以指定\n",
    "- `num_epochs`迭代次数(默认30)\n",
    "- `lr`学习率(默认1e-3)\n",
    "- `gpu`个数, 必须传入数组, 如[0], [0, 1, 2, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in trainer():\n",
    "    a = i"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你从某个已经训练好的模型继续训练, 请在`trainer`中加入`resume_path`和`resume_step`参数\n",
    "- resume_path: 已经训练好的模型路径\n",
    "- resume_step: 当前这个模型的步数, 可以自己设定一个数字\n",
    "\n",
    "例如\n",
    "\n",
    "```python\n",
    "for i in trainer(resume_path='./model.pth', resume_step=572)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 文本分类的预测代码\n",
    "\n",
    "如果想使用自己训练好的模型, 可以运行以下代码\n",
    "\n",
    "以`TextCNN`为例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from predictor import Predictor\n",
    "from config import Config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config = Config(from_json='./configs/TextCNN.json')\n",
    "config.num_classes = 10\n",
    "config.vocab_path = './THUCNews/data/vocab.pkl'\n",
    "config.set_embedding_pretrained('./THUCNews/data/embedding_Tencent.npz')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里需要配置`resume_path`的路径, 即你训练好的模型存放路径."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred = Predictor(config, resume_path='你的模型路径')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后就可以开心的预测数据的分类了!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred(['你真的很有趣!'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6.13 ('pcpower')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "7872dec8fee77bb117962a585331ef85b9564312d4ed2770c3f26192a3127be2"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}