node_classification.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\nNode Classification with DGL\n============================\n\nGNNs are powerful tools for many machine learning tasks on graphs. In\nthis introductory tutorial, you will learn the basic workflow of using\nGNNs for node classification, i.e.\u00a0predicting the category of a node in\na graph.\n\nBy completing this tutorial, you will be able to\n\n-  Load a DGL-provided dataset.\n-  Build a GNN model with DGL-provided neural network modules.\n-  Train and evaluate a GNN model for node classification on either CPU\n   or GPU.\n\nThis tutorial assumes that you have experience in building neural\nnetworks with PyTorch.\n\n(Time estimate: 13 minutes)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import dgl\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Overview of Node Classification with GNN\n----------------------------------------\n\nOne of the most popular and widely adopted tasks on graph data is node\nclassification, where a model needs to predict the ground truth category\nof each node. Before graph neural networks, many proposed methods are\nusing either connectivity alone (such as DeepWalk or node2vec), or simple\ncombinations of connectivity and the node's own features.  GNNs, by\ncontrast, offers an opportunity to obtain node representations by\ncombining the connectivity and features of a *local neighborhood*.\n\n`Kipf et\nal., <https://arxiv.org/abs/1609.02907>`__ is an example that formulates\nthe node classification problem as a semi-supervised node classification\ntask. With the help of only a small portion of labeled nodes, a graph\nneural network (GNN) can accurately predict the node category of the\nothers.\n\nThis tutorial will show how to build such a GNN for semi-supervised node\nclassification with only a small number of labels on the Cora\ndataset,\na citation network with papers as nodes and citations as edges. The task\nis to predict the category of a given paper. Each paper node contains a\nword count vector as its features, normalized so that they sum up to one,\nas described in Section 5.2 of\n`the paper <https://arxiv.org/abs/1609.02907>`__.\n\nLoading Cora Dataset\n--------------------\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import dgl.data\n\ndataset = dgl.data.CoraGraphDataset()\nprint('Number of categories:', dataset.num_classes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "A DGL Dataset object may contain one or multiple graphs. The Cora\ndataset used in this tutorial only consists of one single graph.\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "g = dataset[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "A DGL graph can store node features and edge features in two\ndictionary-like attributes called ``ndata`` and ``edata``.\nIn the DGL Cora dataset, the graph contains the following node features:\n\n- ``train_mask``: A boolean tensor indicating whether the node is in the\n  training set.\n\n- ``val_mask``: A boolean tensor indicating whether the node is in the\n  validation set.\n\n- ``test_mask``: A boolean tensor indicating whether the node is in the\n  test set.\n\n- ``label``: The ground truth node category.\n\n-  ``feat``: The node features.\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print('Node features')\nprint(g.ndata)\nprint('Edge features')\nprint(g.edata)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Defining a Graph Convolutional Network (GCN)\n--------------------------------------------\n\nThis tutorial will build a two-layer `Graph Convolutional Network\n(GCN) <http://tkipf.github.io/graph-convolutional-networks/>`__. Each\nlayer computes new node representations by aggregating neighbor\ninformation.\n\nTo build a multi-layer GCN you can simply stack ``dgl.nn.GraphConv``\nmodules, which inherit ``torch.nn.Module``.\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from dgl.nn import GraphConv\n\nclass GCN(nn.Module):\n    def __init__(self, in_feats, h_feats, num_classes):\n        super(GCN, self).__init__()\n        self.conv1 = GraphConv(in_feats, h_feats)\n        self.conv2 = GraphConv(h_feats, num_classes)\n    \n    def forward(self, g, in_feat):\n        h = self.conv1(g, in_feat)\n        h = F.relu(h)\n        h = self.conv2(g, h)\n        return h\n    \n# Create the model with given dimensions\nmodel = GCN(g.ndata['feat'].shape[1], 16, dataset.num_classes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "DGL provides implementation of many popular neighbor aggregation\nmodules. You can easily invoke them with one line of code.\n\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Training the GCN\n----------------\n\nTraining this GCN is similar to training other PyTorch neural networks.\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def train(g, model):\n    optimizer = torch.optim.Adam(model.parameters(), lr=0.01)\n    best_val_acc = 0\n    best_test_acc = 0\n\n    features = g.ndata['feat']\n    labels = g.ndata['label']\n    train_mask = g.ndata['train_mask']\n    val_mask = g.ndata['val_mask']\n    test_mask = g.ndata['test_mask']\n    for e in range(100):\n        # Forward\n        logits = model(g, features)\n\n        # Compute prediction\n        pred = logits.argmax(1)\n\n        # Compute loss\n        # Note that you should only compute the losses of the nodes in the training set.\n        loss = F.cross_entropy(logits[train_mask], labels[train_mask])\n\n        # Compute accuracy on training/validation/test\n        train_acc = (pred[train_mask] == labels[train_mask]).float().mean()\n        val_acc = (pred[val_mask] == labels[val_mask]).float().mean()\n        test_acc = (pred[test_mask] == labels[test_mask]).float().mean()\n\n        # Save the best validation accuracy and the corresponding test accuracy.\n        if best_val_acc < val_acc:\n            best_val_acc = val_acc\n            best_test_acc = test_acc\n\n        # Backward\n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n\n        if e % 5 == 0:\n            print('In epoch {}, loss: {:.3f}, val acc: {:.3f} (best {:.3f}), test acc: {:.3f} (best {:.3f})'.format(\n                e, loss, val_acc, best_val_acc, test_acc, best_test_acc))\nmodel = GCN(g.ndata['feat'].shape[1], 16, dataset.num_classes)\ntrain(g, model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Training on GPU\n---------------\n\nTraining on GPU requires to put both the model and the graph onto GPU\nwith the ``to`` method, similar to what you will do in PyTorch.\n\n.. code:: python\n\n   g = g.to('cuda')\n   model = GCN(g.ndata['feat'].shape[1], 16, dataset.num_classes).to('cuda')\n   train(g, model)\n\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "What\u2019s next?\n------------\n\n-  :doc:`How does DGL represent a graph <2_dglgraph>`?\n-  :doc:`Write your own GNN module <3_message_passing>`.\n-  :doc:`Link prediction (predicting existence of edges) on full\n   graph <4_link_predict>`.\n-  :doc:`Graph classification <5_graph_classification>`.\n-  :doc:`Make your own dataset <6_load_data>`.\n-  `The list of supported graph convolution\n   modules <apinn-pytorch>`.\n-  `The list of datasets provided by DGL <apidata>`.\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Thumbnail credits: Stanford CS224W Notes\n# sphinx_gallery_thumbnail_path = '_static/blitz_1_introduction.png'"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}