week04_[recap]_deep_learning

Note: This week's materials cover the basics of neural nets and deep learning and teach you how to use auto-diff frameworks. If you're already fluent in tensorflow OR pytorch OR theano - feel free to skip this week entirely..

Materials

• Lecture slides

• In russian:

  • Basic lecture on deep learning - video
  • Deep learning frameworks - video
  • Pytorch tutorial recommended
  • Tensorflow tutorial (english only for now. Links are welcome)
  • Theano tutorial

• In english:

  • Intro to neural nets and backprop (english) - video
  • Intro to convnets - video
  • Deep learning frameworks - video
  • Tensorflow tutorial
  • Theano tutorial
  • Pytorch tutorial

Bonus materials

• Karpathy's course on deep learning (english) - http://cs231n.github.io/
• A neat little play-ground where you can train small NNs and see what they actually learn - playground
• Nuts and Bolts of deep learning by Andrew Ng (english) - video
• Deep learning philosophy: our humble take (english)
• Deep learning demystified - video
• Karpathy's lecture on deep learning for computer vision - https://www.youtube.com/watch?v=u6aEYuemt0M
• Our humble DL course: HSE'fall17, Skoltech/YSDA'spring16 courses on deep learning (english).
• Srsly, just google "deep learning %s"%s for s in what_you_want_to_know.

Practice

Colab url (pytorch) From now on, we'll have two tracks: theano and tensorflow. We'll also add pytorch seminars as soon as they're ready.

Please pick seminar_theano.ipynb, seminar_tensorflow.ipynb or seminar_pytorch.ipynb.

Note: in this and all following weeks you're only required to get through practice in one of the frameworks. Looking into other alternatives is great for self-education but never mandatory.

What to choose?

• The simplest choice is PyTorch: it's basically ye olde numpy with automatic gradients and a lot of pre-implemented DL stuff... except all the functions have different names.

• If you want to be familiar with production-related stuff from day 1, choose TensorFlow. It's much more convenient to deploy (to non-python or to mobiles). The catch is that all those conveniences become inconveniences once you want to write something simple in jupyter.

• Theano works like tensorflow but it offers a numpy-compatible interface and comes with built-in graph optimization. The payoff is that theano is not as popular as the first two. It is also not meant as a producton framework so deploying to mobiles may be a problem.

• It's not like choosing house at Hogwarts, you'll be able to switch between frameworks easily once you master the underlying principles.