paddleOCR_CharacterDetection_demo.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bG__3cDM5WNX"
      },
      "source": [
        "References:  \n",
        "GitHub  \n",
        "https://github.com/PaddlePaddle/PaddleOCR  \n",
        "  \n",
        "Google Colaboratoryで開く  \n",
        "<a href=\"https://colab.research.google.com/github/kaz12tech/ai_demos/blob/master/paddleOCR_CharacterDetection_demo.ipynb\" target=\"_blank\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bGC_UFNq_Kfe"
      },
      "source": [
        "# ランタイムの設定\n",
        "「ランタイム」→「ランタイムのタイプを変更」→「ハードウェアアクセラレータ」をGPUに変更"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_zXqKYmR7YAE"
      },
      "source": [
        "# 実行方法\n",
        "「ランタイム」→「すべてのセルを実行」を選択"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "smO6lfjf7ao9"
      },
      "source": [
        "# Google Driveのマウント\n",
        "Google ColaboratoryとGoogle Driveを接続します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sGdg-xhrj95T",
        "outputId": "2011c70d-9588-402c-a23c-5d6dbf7842d3"
      },
      "outputs": [],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aorzje_t7lb4"
      },
      "source": [
        "# 環境セットアップ"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uQ9igWO9A_v7"
      },
      "source": [
        "## PaddlePaddleのインストール"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "01ER-UfHBEnv",
        "outputId": "ce6cbb7c-32cf-48e8-e948-22ebc8ca340d"
      },
      "outputs": [],
      "source": [
        "!python3 -m pip install paddlepaddle-gpu -i https://mirror.baidu.com/pypi/simple"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O8h4uAlaB1B5"
      },
      "source": [
        "## PaddlePaddleのインストール確認\n",
        "PaddlePaddleのバージョンと認識しているGPUの数を出力"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wctsVdv9Brbk",
        "outputId": "61b409f7-1a56-4e87-8457-d718a9ae0bb6"
      },
      "outputs": [],
      "source": [
        "!python3 -c \"import paddle; print(paddle.__version__)\"\n",
        "!python3 -c \"import paddle; paddle.utils.run_check()\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iDZo4Rnv_9-0"
      },
      "source": [
        "## PaddleOCRのインストール"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "azkQw2wa45hp",
        "outputId": "7715c944-4564-464e-ee26-95ecdc53ab2e"
      },
      "outputs": [],
      "source": [
        "%cd /content/\n",
        "!git clone https://github.com/PaddlePaddle/PaddleOCR.git"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cmLgo8zuAk6T"
      },
      "source": [
        "## 関連するライブラリのインストール"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YRcWaNslAQMs",
        "outputId": "e205164c-084b-4387-d67f-e4bad28fd2e5"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "!pip install -r requirements.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J8-t4NOR7x6P"
      },
      "source": [
        "# 文字検出\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RestewMTDFMc"
      },
      "source": [
        "## テスト用画像のダウンロード"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VZxznR99DJVf",
        "outputId": "b07545e9-5e80-4b48-db63-cfb1a75a1c72"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "\n",
        "!wget https://paddleocr.bj.bcebos.com/dygraph_v2.1/ppocr_img.zip\n",
        "!unzip ppocr_img.zip > /dev/null"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wwKO2JspK-q2"
      },
      "source": [
        "## 学習済みモデルのダウンロード"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PF0UUHxYLCGU",
        "outputId": "e3806f3e-0094-486e-e1af-f0fa4167cc0e"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "!mkdir pretrain_models\n",
        "%cd /content/PaddleOCR/pretrain_models\n",
        "\n",
        "!wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar\n",
        "!tar -xvf ch_ppocr_server_v2.0_det_infer.tar"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DUzyiNw_IPI6"
      },
      "source": [
        "## 文字検出"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NEgPkMu0P1bx"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import matplotlib.pyplot as plt\n",
        "from PIL import Image, ImageDraw"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "85umh7j_IS17",
        "outputId": "8241197e-0f4f-4c4a-b20f-2e961aaa346c"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "!python3 tools/infer/predict_det.py \\\n",
        "  --det_algorithm=\"DB\" \\\n",
        "  --det_model_dir=\"./pretrain_models/ch_ppocr_server_v2.0_det_infer\" \\\n",
        "  --image_dir=\"./ppocr_img/imgs/12.jpg\" \\\n",
        "  --use_gpu=True"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "clIO6WRQPjUy"
      },
      "source": [
        "## 検出結果の表示"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 414
        },
        "id": "Fa3g9MySPmi7",
        "outputId": "5bd8e687-eeb9-40e5-b125-6e1cf3e85513"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "result_path = \"./inference_results/det_res_12.jpg\"\n",
        "\n",
        "# 画像の表示\n",
        "image = Image.open(result_path).convert(\"RGB\")\n",
        "plt.figure(figsize=(6, 6))\n",
        "plt.imshow(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WLz3oJKXQUD3"
      },
      "source": [
        "# 文字検出モデルのトレーニング"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U1yVfNArRWCD"
      },
      "source": [
        "## データ準備"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nJ8QffgjR1SW",
        "outputId": "cf37c8e3-72d5-4843-ec67-b7cf5c805f8c"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "\n",
        "# 登録必須\n",
        "# 学習データ(ICDAR2015)ダウンロード\n",
        "!wget --no-check-certificate https://rrc.cvc.uab.es/downloads/ch4_training_images.zip\n",
        "!wget --no-check-certificate https://rrc.cvc.uab.es/downloads/ch4_test_images.zip\n",
        "!unzip ch4_training_images.zip -d ./icdar_c4_train_imgs > /dev/null\n",
        "!unzip ch4_test_images.zip -d ./ch4_test_images > /dev/null\n",
        "\n",
        "!wget https://paddleocr.bj.bcebos.com/dataset/train_icdar2015_label.txt\n",
        "!wget https://paddleocr.bj.bcebos.com/dataset/test_icdar2015_label.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6or4PivPWbJY"
      },
      "source": [
        "## 学習済みモデルのダウンロード"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZhfQiKIMWjJv",
        "outputId": "0a28064a-9cbc-45b1-a344-8f2d436b1c12"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR/pretrain_models\n",
        "\n",
        "!wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_train.tar\n",
        "!tar -xvf ch_ppocr_server_v2.0_det_train.tar"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Pf0ew2AkNfG"
      },
      "source": [
        "## model保存用ディレクトリ作成"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "papoURWAkWIG"
      },
      "outputs": [],
      "source": [
        "!mkdir -p /content/drive/MyDrive/Colab\\ Notebooks/output/ch_db_res18/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N7LZ7Z8RXUQD"
      },
      "source": [
        "## モデルのトレーニング"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "g2ReusG5XXCH",
        "outputId": "8fa5d602-3574-48aa-9dd2-b6a73cf47c14"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "\n",
        "# 学習途中のモデルを確認\n",
        "ckpt_path = \"\"\n",
        "# 最新エポックのモデルを確認\n",
        "if os.path.exists(\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/latest.pdopt\"):\n",
        "  ckpt_path = \"/content/drive/MyDrive/Colab\\ Notebooks/output/ch_db_res18/latest\"\n",
        "# eval後のモデルを確認。evalがあれば最新エポックのモデルではなくeval後を利用\n",
        "if os.path.exists(\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/best_accuracy.pdopt\"):\n",
        "  ckpt_path = \"/content/drive/MyDrive/Colab\\ Notebooks/output/ch_db_res18/best_accuracy\"\n",
        "if ckpt_path == \"\":\n",
        "  print(\"There is no ckpt. Train a new model.\")\n",
        "else:\n",
        "  print(\"Resume training from\", ckpt_path)\n",
        "\n",
        "#-o Global.pretrained_model=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/latest\" \\\n",
        "!python3 tools/train.py \\\n",
        "  -c configs/det/det_res18_db_v2.0.yml \\\n",
        "  -o \\\n",
        "  Global.epoch_num=30 \\\n",
        "  Global.eval_batch_step=[725,125] \\\n",
        "  Global.pretrained_model=\"./pretrain_models/ch_ppocr_server_v2.0_det_train/best_accuracy\" \\\n",
        "  Global.checkpoints={ckpt_path} \\\n",
        "  Global.save_model_dir=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/\" \\\n",
        "  Global.save_res_path=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/\" \\\n",
        "  Train.dataset.data_dir=./ \\\n",
        "  Train.dataset.label_file_list=[./train_icdar2015_label.txt] \\\n",
        "  Eval.dataset.data_dir=./ \\\n",
        "  Eval.dataset.label_file_list=[./test_icdar2015_label.txt] \\"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VtBAclBY4W67"
      },
      "source": [
        "# モデルの評価"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-segQkUR8ajH"
      },
      "source": [
        "## precision, recall, hmeanの算出"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TgCZmAb_4aXc",
        "outputId": "05fcf296-13bb-454c-bf3b-a474544d99bf"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "\n",
        "# latest以外にbest_accracyが出力されている場合はbest_accuracyを使用\n",
        "ckpt_path = \"/content/drive/MyDrive/Colab\\ Notebooks/output/ch_db_res18/latest\"\n",
        "if os.path.exists(\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/best_accuracy.pdopt\"):\n",
        "  ckpt_path = \"/content/drive/MyDrive/Colab\\ Notebooks/output/ch_db_res18/best_accuracy\"\n",
        "print(\"ckpt path\", ckpt_path)\n",
        "\n",
        "!python3 tools/eval.py \\\n",
        "  -c configs/det/det_res18_db_v2.0.yml \\\n",
        "  -o \\\n",
        "  Global.pretrained_model=\"./pretrain_models/ch_ppocr_server_v2.0_det_train/best_accuracy\" \\\n",
        "  Global.checkpoints={ckpt_path} \\\n",
        "  Global.save_model_dir=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/\" \\\n",
        "  Global.save_res_path=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/\" \\\n",
        "  Train.dataset.data_dir=./ \\\n",
        "  Train.dataset.label_file_list=[./train_icdar2015_label.txt] \\\n",
        "  Eval.dataset.data_dir=./ \\\n",
        "  Eval.dataset.label_file_list=[./test_icdar2015_label.txt] \\\n",
        "  PostProcess.box_thresh=0.6 \\\n",
        "  PostProcess.unclip_ratio=1.5"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "glC3aEwp_Bb0"
      },
      "source": [
        "## 学習用モデルを推論用モデルに変換"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7bw3GfYA_FUZ",
        "outputId": "ba6b3fa1-6eb9-4f98-aeef-e19c7630075a"
      },
      "outputs": [],
      "source": [
        "!python3 tools/export_model.py \\\n",
        "  -c configs/det/det_res18_db_v2.0.yml \\\n",
        "  -o \\\n",
        "  Global.pretrained_model={ckpt_path} \\\n",
        "  Global.save_inference_dir=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/det_db_inference/\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UA9RMXMx8U4i"
      },
      "source": [
        "## トレーニングしたモデルを使って文字検出"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VfdJBfcm8oyT",
        "outputId": "57c343f5-cc85-46b9-9f50-b3e4cf07be6e"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "!python3 tools/infer/predict_det.py \\\n",
        "  --det_algorithm=\"DB\" \\\n",
        "  --det_model_dir=\"/content/drive/MyDrive/Colab Notebooks/output/ch_db_res18/det_db_inference\" \\\n",
        "  --image_dir=\"./ch4_test_images/img_394.jpg\" \\\n",
        "  --use_gpu=True"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "id": "3GcqSHf6gzjs",
        "outputId": "9c9f9a53-fbcb-46f4-8bcf-52c0f17d4fc7"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "result_path = \"./inference_results/det_res_img_394.jpg\"\n",
        "!mv ./inference_results/det_res_img_394.jpg ./inference_results/train_det_res_img_394.jpg\n",
        "result_path = \"./inference_results/train_det_res_img_394.jpg\"\n",
        "\n",
        "# 画像の表示\n",
        "image = Image.open(result_path).convert(\"RGB\")\n",
        "plt.figure(figsize=(6, 6))\n",
        "plt.imshow(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZNNW7epokzGf"
      },
      "source": [
        "## 配布されているモデルと比較"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xvGySVLfkyyw",
        "outputId": "a65fe895-160e-4888-d790-d6058c4786a6"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "!python3 tools/infer/predict_det.py \\\n",
        "  --det_algorithm=\"DB\" \\\n",
        "  --det_model_dir=\"./pretrain_models/ch_ppocr_server_v2.0_det_infer\" \\\n",
        "  --image_dir=\"./ch4_test_images/img_394.jpg\" \\\n",
        "  --use_gpu=True"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 276
        },
        "id": "SG-S7Iixk90Y",
        "outputId": "3764752c-07f5-4482-f054-d0999ad8510e"
      },
      "outputs": [],
      "source": [
        "%cd /content/PaddleOCR\n",
        "pretrain_result_path = \"./inference_results/det_res_img_394.jpg\"\n",
        "\n",
        "image = Image.open(result_path).convert(\"RGB\")\n",
        "pre_image = Image.open(pretrain_result_path).convert(\"RGB\")\n",
        "\n",
        "dst = Image.new('RGB', (image.width + pre_image.width, image.height))\n",
        "dst.paste(image, (0, 0))\n",
        "dst.paste(pre_image, (image.width, 0))\n",
        "\n",
        "# 画像の表示\n",
        "plt.figure(figsize=(12, 12))\n",
        "plt.imshow(dst)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "paddleOCR_CharacterDetection_demo.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}