09-theano/09.15-tensor-basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Theano tensor 模块：基础"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "张量是向量在数学上的一种推广，具体内容可以参考维基百科：\n",
    "https://en.wikipedia.org/wiki/Tensor\n",
    "\n",
    "在 Theano 中有一个专门处理张量变量的模块：`theano.tensor` （以下简称 `T`）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using gpu device 1: Tesla C2075 (CNMeM is disabled)\n"
     ]
    }
   ],
   "source": [
    "import theano\n",
    "import theano.tensor as T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构造符号变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以用 `tensor` 模块创造符号变量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'theano.tensor.var.TensorVariable'>\n",
      "<class 'theano.tensor.type.TensorType'>\n"
     ]
    }
   ],
   "source": [
    "x = T.fmatrix()\n",
    "\n",
    "print type(x)\n",
    "print type(T.fmatrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面可以看到，`T.fmatrix()` 创造出的是一个 `TensorVariable` 类，而 `T.fmatrix` 本身是一个 `TensorType` 类。\n",
    "\n",
    "除了使用 `fmatrix`，我们还可以通过指定 `matrix` 的 `dtype` 参数来定义，例如下面的三种方式都是产生一个 `int32` 型的标量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = T.scalar('myvar', dtype='int32')\n",
    "x = T.iscalar('myvar')\n",
    "x = T.TensorType(dtype='int32', broadcastable=())('myvar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "常用的构造函数有：\n",
    "\n",
    "- `T.scalar(name=None, dtype=config.floatX)`\n",
    "- `T.vector(name=None, dtype=config.floatX)`\n",
    "- `T.row(name=None, dtype=config.floatX)`\n",
    "- `T.col(name=None, dtype=config.floatX)`\n",
    "- `T.matrix(name=None, dtype=config.floatX)`\n",
    "- `T.tensor3(name=None, dtype=config.floatX)`\n",
    "- `T.tensor4(name=None, dtype=config.floatX)`\n",
    "\n",
    "还可以使用一个构造多个变量：\n",
    "- `T.scalars`\n",
    "- `T.vectors`\n",
    "- `T.rows`\n",
    "- `T.cols`\n",
    "- `T.matrices`\n",
    "\n",
    "除此之外，我们还可以用 `TensorType` 类自定义的符号变量：\n",
    "\n",
    "`T.TensorType(dtype, broadcastable, name=None)`\n",
    "\n",
    "- `dtype: str`：对应于 `numpy` 中的类型 \n",
    "- `broadcastable: tuple, list, or array of boolean values`：如果是 `True` 表示该维的维度只能为 1；长度表示符号变量的维度。\n",
    "\n",
    "|pattern|interpretation|\n",
    "|---|---|\n",
    "| [] | scalar |\n",
    "| [True] | 1D scalar (vector of length 1) |\n",
    "| [True, True] | 2D scalar (1x1 matrix) |\n",
    "| [False] | vector |\n",
    "| [False, False] | matrix |\n",
    "| [False] * n | nD tensor |\n",
    "| [True, False]\t| row (1xN matrix) |\n",
    "| [False, True]\t| column (Mx1 matrix) |\n",
    "| [False, True, False] | A Mx1xP tensor (a) |\n",
    "| [True, False, False] | A 1xNxP tensor (b) |\n",
    "| [False, False, False] | A MxNxP tensor (pattern of a + b) |\n",
    "\n",
    "产生一个五维的变量类型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dtensor5 = T.TensorType('float64', (False,)*5)\n",
    "\n",
    "x = dtensor5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 变量方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .dim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "维度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    }
   ],
   "source": [
    "print x.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "类型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorType(float64, 5D)\n"
     ]
    }
   ],
   "source": [
    "print x.type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "包含的变量类型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "float64\n"
     ]
    }
   ],
   "source": [
    "print x.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .reshape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "传入一个变量对 x 进行 `reshape`，通常需要指定 `shape` 的 `ndim`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "shape = T.ivector(\"shape\")\n",
    "\n",
    "y = x.reshape(shape, ndim=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`y` 是 `x` 的一个 `view`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5 3\n"
     ]
    }
   ],
   "source": [
    "print x.ndim, y.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .dimshuffle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`dimshuffle` 改变维度的顺序，返回原始变量的一个 `view`：\n",
    "\n",
    "输入是一个包含 `0,1,...,ndim-1` 和任意数目的 `'x'` 的组合：\n",
    "\n",
    "例如：\n",
    "\n",
    "- `('x')`：将标量变成 1 维数组\n",
    "- `(0, 1)`：与原始的 2 维数组相同\n",
    "- `(1, 0)`：交换 2 维数组的两个维度，形状从 `N × M` 变 `M × N`\n",
    "- `('x', 0)`：形状从 `N` 变成 `1 × N`\n",
    "- `(0, 'x')`：形状从 `N` 变成 `N × 1`\n",
    "- `(2, 0, 1)`： 形状从 `A × B × C` 变成 `C × A × B`\n",
    "- `(0, 'x', 1)`： 形状从 `A × B` 变成 `A × 1 × B`\n",
    "- `(1, 'x', 0)`： 形状从 `A × B` 变成 `B × 1 × A`\n",
    "- `(1,)`： 将第 0 维除去，除去的维度的大小必须为 1。形状从 `1 × A` 变成 `A`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DimShuffle{x,1,2,0}.0\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "z = y.dimshuffle((\"x\", 1, 2, 0))\n",
    "\n",
    "print z\n",
    "print z.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .flatten"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`flatten(ndim=1)` 返回原始变量的一个 `view`，将变量降为 `ndim` 维："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "z = x.flatten(ndim=2)\n",
    "\n",
    "print z.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .ravel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与 `flatten` 一样。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### .T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "转置，注意，一维数组或者变量的转置是其本身，要想将行列向量互相转换，需要使用 `reshape` 或者 `dimshuffle`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 其他方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['T', 'all', 'any', 'arccos', 'arccosh', 'arcsin', 'arcsinh', 'arctan', 'arctanh', 'argmax', 'argmin', 'argsort', 'astype', 'broadcastable', 'ceil', 'choose', 'clip', 'clone', 'compress', 'conj', 'conjugate', 'copy', 'cos', 'cosh', 'cumprod', 'cumsum', 'diagonal', 'dimshuffle', 'dot', 'dtype', 'eval', 'exp', 'fill', 'flatten', 'floor', 'imag', 'index', 'log', 'max', 'mean', 'min', 'name', 'ndim', 'nonzero', 'norm', 'owner', 'prod', 'ptp', 'ravel', 'real', 'repeat', 'reshape', 'round', 'shape', 'sin', 'sinh', 'size', 'sort', 'sqrt', 'squeeze', 'std', 'sum', 'swapaxes', 'tag', 'take', 'tan', 'tanh', 'trace', 'transpose', 'trunc', 'type', 'var']\n"
     ]
    }
   ],
   "source": [
    "print filter(lambda t: t.isalpha(), dir(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模块函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了与 `numpy` 兼容，`tensor`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`shape(x)` 返回一个存储变量 `x` 形状的变量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape.0\n"
     ]
    }
   ],
   "source": [
    "print T.shape(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.shape_padleft, T.shape_padright"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在最左边/右边加上 n 个大小为 1 的 1 个维度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DimShuffle{x,0,1,2}.0\n",
      "DimShuffle{0,1,2,x}.0\n"
     ]
    }
   ],
   "source": [
    "x = T.tensor3()\n",
    "\n",
    "print T.shape_padleft(x)\n",
    "print T.shape_padright(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.shape_padaxis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在指定位置插入大小为 1 的 1 个维度："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DimShuffle{0,x,1,2}.0\n",
      "DimShuffle{x,0,1,2}.0\n",
      "DimShuffle{0,1,2,x}.0\n"
     ]
    }
   ],
   "source": [
    "print T.shape_padaxis(x, 1)\n",
    "print T.shape_padaxis(x, 0)\n",
    "print T.shape_padaxis(x, -1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "插入这些大小为 `1` 的维度，主要目的是 `broadcast` 化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.unbroadcast"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以使用 `unbroadcast(x, *axes)` 使得 `x` 的某些维度不可 `broadcast`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.tile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`tile(x, reps)` 按照规则重复 `x`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 产生张量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.zeros_like(x), T.ones_like(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "产生一个与 x 形状相同的全 0 或全 1 变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.fill(a, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用 `b` 的值去填充 `a`，`b` 是一个数值或者 `theano scalar`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.alloc(value, *shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "返回指定形状的变量，并初始化为 `value`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.eye(n, m=None, k=0, dtype=theano.config.floatX)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "单位矩阵"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.basic.choose(a, choices)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`a` 是一个 `index` 数组变量，对应于 `choices` 中的位置。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 降维"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.max(x), T.argmax(x), T.max_and_argmax(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最大值，最大值位置，最大值和最大值位置。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.min(x), T.argmin(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最小值，最小值位置。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.sum(x), T.prod(x), T.mean(x), T.var(x), T.std(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和，积，均值，方差，标准差"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### T.all(x), T.any(x)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}