This is a repository of course materials for the Deep Learning for Biology course.
The course is taught Fall 2019 at Higher School of Economics (Moscow), Faculty of Computer Science, Master’s Programme 'Data Analysis in Biology and Medicine'.
- Course slides
- Course Jupyter notebooks (using Tensorflow 2.0). Later in the course we switched to Colab notebooks. You can download them if you want.
Topics:
- Short history
- Current results in Deep Learning
- Images and Video
- Speech and Sound
- Text and Language
- Robotic control
- ML for systems
- Problems with DL
- Other approaches to AI
- Knowledge and Representation
- Symbolic approaches
- Evolutionary computations and Swarm intelligence
- Hardware
Slides:
Video:
Topics:
- Intro into NN: neuron, neural network, backpropagation,
- Feed-forward NNs (FNN)
- Autoencoders (AE)
Slides:
Video:
Code:
- Tensorflow 2 Intro (FFN: Binary classification, Multi-class classification, Regression)
- Autoencoders (shallow, deep, regularized/sparse, denoising)
Video:
Topics:
- What is CNN
Code:
- CNN for classification, CNN autoencoders, Saving and Loading models, How to use pretrained models in Tensorflow
Slides:
Video:
- part 1
- other parts are missing :(
Topics:
- Activations, Regularization, Augmentation, Optimization etc
- Models: LeNet, AlexNet, VGG, GoogLeNet, Inception, ResNet, DenseNet, XCeption, NASNet
Slides:
Video:
- video 1, ~75%
- waiting for other recordings
Video:
Video:
Slides:
Video:
Topics:
- Theory of Transfer Learning
Code:
- How to use pretrained models in Tensorflow
Slides:
Video:
Topics:
- 1D, 3D, dilated convolutions
- Detection: R-CNN, Fast R-CNN, Faster R-CNN, YOLO
- Fully-convolutional CNNs (FCNs)
- Deconvolutional networks (Transposed convolution)
- Generative Adversarial Networks (GANs)
- Style Transfer
Code:
- Variational autoencoder
Slides:
Video:
Topics:
- RNN basics, Backpropagation through time
- Long short-term memory (LSTM)
Code:
- Generating text using RNNs
- Time series forecasting
Slides:
Video:
Topics:
- Advanced RNNs: Bidirectional RNNs, Multidimensional RNNs
- Working with texts: vectorizing, one-hot encoding, word embeddings, word2vec, BPE etc
Code:
- Building text classifiers (LSTM, Deep LSTM, Bidirectional LSTM, 1D-CNN, CNN+LSTM)
Slides:
Video:
Video:
Topics:
- Multimodal Learning
- Seq2seq
- Encoder-Decoder
- Beam search
- Attention mechanisms, Visualizing attention, Hard and Soft attention, Self-Attention
- Augmented RNNs
- Connectionist Temporal Classification (CTC)
- Non-RNN Sequence Learning, problems with RNNs
- Convolutional Sequence Learning
Slides:
Video:
Topics:
- Self-Attention Neural Networks (SAN): Transformer Architecture
- Transformer: The next steps (Image Transformer, Universal Transformer, Transformer-XL)
- BERT & Co (RoBERTa, XLNet, ALBERT, etc), GPT-2, etc
Slides:
Code:
Video: