Back-Prop-Learning.md

BACK-PROP-LEARNING

AIMA3e

function BACK-PROP-LEARNING(examples, network) returns a neural network
 inputs examples, a set of examples, each with input vector x and output vector y
    network, a multilayer network with L layers, weights w_i,j, activation function g
 local variables: Δ, a vector of errors, indexed by network node

 repeat
   for each weight w_i,j in network do
     w_i,j ← a small random number
   for each example (x, y) in examples do
     /* Propagate the inputs forward to compute the outputs */
     for each node i in the input layer do
       a_i ← x_i
     for l = 2 to L do
       for each node j in layer l do
         in_j ← Σ_i w_i,j a_i
         a_j ← g(in_j)
     /* Propagate deltas backward from output layer to input layer */
     for each node j in the output layer do
       Δ[j] ← g′(in_j) × (y_i − a_j)
     for l = L − 1 to 1 do
       for each node i in layer l do
         Δ[i] ← g′(in_i) Σ_j w_i,j Δ[j]
     /* Update every weight in network using deltas */
     for each weight w_i,j in network do
       w_i,j ← w_i,j + α × a_i × Δ[j]
 until some stopping criterion is satisfied
 return network

---

Figure ?? The back-propagation algorithm for learning in multilayer networks.