Transformer For Images:

A transformer implementation for images: Steps include:

1. Patch embedding
2. Attention Module
3. Multi layer perceptron
4. Transformer block
5. Final Vision Transformer

Pretraining using ViT on Imagenet dataset and tested on new image.

Additional topics:

1. How linear layer behave, when we have 3-dim or more dim tensors.
2. Basic Property of Layer Normalization.

Inspired from https://www.youtube.com/watch?v=ovB0ddFtzzA