plot_gexternal_xgboost.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\n# Convert a pipeline with a XGBoost model\n\n.. index:: XGBoost\n\n:epkg:`sklearn-onnx` only converts :epkg:`scikit-learn` models\ninto :epkg:`ONNX` but many libraries implement :epkg:`scikit-learn`\nAPI so that their models can be included in a :epkg:`scikit-learn`\npipeline. This example considers a pipeline including a :epkg:`XGBoost`\nmodel. :epkg:`sklearn-onnx` can convert the whole pipeline as long as\nit knows the converter associated to a *XGBClassifier*. Let's see\nhow to do it.\n\n## Train a XGBoost classifier\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from pyquickhelper.helpgen.graphviz_helper import plot_graphviz\nfrom mlprodict.onnxrt import OnnxInference\nimport numpy\nimport onnxruntime as rt\nfrom sklearn.datasets import load_iris, load_diabetes, make_classification\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import StandardScaler\nfrom xgboost import XGBClassifier, XGBRegressor, DMatrix, train as train_xgb\nfrom skl2onnx.common.data_types import FloatTensorType\nfrom skl2onnx import convert_sklearn, to_onnx, update_registered_converter\nfrom skl2onnx.common.shape_calculator import (\n    calculate_linear_classifier_output_shapes,\n    calculate_linear_regressor_output_shapes)\nfrom onnxmltools.convert.xgboost.operator_converters.XGBoost import (\n    convert_xgboost)\nfrom onnxmltools.convert import convert_xgboost as convert_xgboost_booster\n\n\ndata = load_iris()\nX = data.data[:, :2]\ny = data.target\n\nind = numpy.arange(X.shape[0])\nnumpy.random.shuffle(ind)\nX = X[ind, :].copy()\ny = y[ind].copy()\n\npipe = Pipeline([('scaler', StandardScaler()),\n                 ('xgb', XGBClassifier(n_estimators=3))])\npipe.fit(X, y)\n\n# The conversion fails but it is expected.\n\ntry:\n    convert_sklearn(pipe, 'pipeline_xgboost',\n                    [('input', FloatTensorType([None, 2]))],\n                    target_opset=12)\nexcept Exception as e:\n    print(e)\n\n# The error message tells no converter was found\n# for :epkg:`XGBoost` models. By default, :epkg:`sklearn-onnx`\n# only handles models from :epkg:`scikit-learn` but it can\n# be extended to every model following :epkg:`scikit-learn`\n# API as long as the module knows there exists a converter\n# for every model used in a pipeline. That's why\n# we need to register a converter."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register the converter for XGBClassifier\n\nThe converter is implemented in :epkg:`onnxmltools`:\n`onnxmltools...XGBoost.py\n<https://github.com/onnx/onnxmltools/blob/master/onnxmltools/convert/\nxgboost/operator_converters/XGBoost.py>`_.\nand the shape calculator:\n`onnxmltools...Classifier.py\n<https://github.com/onnx/onnxmltools/blob/master/onnxmltools/convert/\nxgboost/shape_calculators/Classifier.py>`_.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "update_registered_converter(\n    XGBClassifier, 'XGBoostXGBClassifier',\n    calculate_linear_classifier_output_shapes, convert_xgboost,\n    options={'nocl': [True, False], 'zipmap': [True, False]})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Convert again\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "model_onnx = convert_sklearn(\n    pipe, 'pipeline_xgboost',\n    [('input', FloatTensorType([None, 2]))],\n    target_opset=12)\n\n# And save.\nwith open(\"pipeline_xgboost.onnx\", \"wb\") as f:\n    f.write(model_onnx.SerializeToString())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Compare the predictions\n\nPredictions with XGBoost.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(\"predict\", pipe.predict(X[:5]))\nprint(\"predict_proba\", pipe.predict_proba(X[:1]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Predictions with onnxruntime.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "sess = rt.InferenceSession(\"pipeline_xgboost.onnx\")\npred_onx = sess.run(None, {\"input\": X[:5].astype(numpy.float32)})\nprint(\"predict\", pred_onx[0])\nprint(\"predict_proba\", pred_onx[1][:1])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Final graph\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "oinf = OnnxInference(model_onnx)\nax = plot_graphviz(oinf.to_dot())\nax.get_xaxis().set_visible(False)\nax.get_yaxis().set_visible(False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Same example with XGBRegressor\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "update_registered_converter(\n    XGBRegressor, 'XGBoostXGBRegressor',\n    calculate_linear_regressor_output_shapes, convert_xgboost)\n\n\ndata = load_diabetes()\nx = data.data\ny = data.target\nX_train, X_test, y_train, _ = train_test_split(x, y, test_size=0.5)\n\npipe = Pipeline([('scaler', StandardScaler()),\n                 ('xgb', XGBRegressor(n_estimators=3))])\npipe.fit(X_train, y_train)\n\nprint(\"predict\", pipe.predict(X_test[:5]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "ONNX\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "onx = to_onnx(pipe, X_train.astype(numpy.float32))\n\nsess = rt.InferenceSession(onx.SerializeToString())\npred_onx = sess.run(None, {\"X\": X_test[:5].astype(numpy.float32)})\nprint(\"predict\", pred_onx[0].ravel())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Some discrepencies may appear. In that case,\nyou should read `l-example-discrepencies-float-double`.\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Same with a Booster\n\nA booster cannot be inserted in a pipeline. It requires\na different conversion function because it does not\nfollow :epkg:`scikit-learn` API.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "x, y = make_classification(n_classes=2, n_features=5,\n                           n_samples=100,\n                           random_state=42, n_informative=3)\nX_train, X_test, y_train, _ = train_test_split(x, y, test_size=0.5,\n                                               random_state=42)\n\ndtrain = DMatrix(X_train, label=y_train)\n\nparam = {'objective': 'multi:softmax', 'num_class': 3}\nbst = train_xgb(param, dtrain, 10)\n\ninitial_type = [('float_input', FloatTensorType([None, X_train.shape[1]]))]\nonx = convert_xgboost_booster(bst, \"name\", initial_types=initial_type)\n\nsess = rt.InferenceSession(onx.SerializeToString())\ninput_name = sess.get_inputs()[0].name\nlabel_name = sess.get_outputs()[0].name\npred_onx = sess.run(\n    [label_name], {input_name: X_test.astype(numpy.float32)})[0]\nprint(pred_onx)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}