Here we will be building very basic and simple neural network . We will be training on MNIST dataset and test the results.

We had 3 files here

• load_dataset : which had a function "load_dataset" which loads our MNIST dataset .
• implementation.py : Here we had the functions "SGD" (which is essentially mini batch)
  We also had "update_mini_batch" which updates parameters batch wise
  "feed_forward" it just return the prediction value based on weights and biases
  "backpropogate" which essentially does backpropogation .
  and others are small helper functions.

Backpropagation algorithm

• The backpropagation algorithm provide us with a way of computing the gradient of the cost function by performing the following operations
• Feed forward :for each l=2,3...L we compute z^l = (w^l)(a^(l-1)) + b^l and a^l = σ(z^l)
• Output error: 𝛿^l = ∇_a(cost_function) * σ¹(z^l) (Note: "*" here is dot product)
• Backpropagation error : 𝛿^l = (w^l+1) ^T 𝛿^l+1 *σ ¹(z^l)

Neural Network's Output

• Our neural network has only one hidden layer having 30 neurons.(its a hyperparameter we got to know if this is 30 its showing minimum error rate .)
• The final accuracy we are getting is around 95%.