1. To convert the input into something that the network can
process, we normalise the size of the input and convert it to an
array of pixels (here a 10 x 13 array).
HIDDEN
                           LAYER

           INPUT
                                           OUTPUT
           LAYER
                                            LAYER

  1



  -1
             …




                                              …
  -1
                              …




2. Inputs are sent into the network, possibly ‘1’ for a dark
pixel and ‘-1’ for a light pixel.
HIDDEN
                          LAYER

           INPUT
                                         OUTPUT
           LAYER
                                          LAYER

  1          1


 -1          -1
            …




                                           …
 -1          -1
                            …




3. Each node in the hidden layer evaluates a weighted sum
of its inputs, and determines its state. We will demonstrate
this for 1 node.
HIDDEN
                                LAYER

                INPUT
                                                OUTPUT
                LAYER
                                                 LAYER

       1          1


      -1          -1
                  …




                                                  …
       -1         -1
                                     …




4. Let this hidden node be j, and nodes connecting to it be i. We find

                                     w ij x j
                             all i
HIDDEN
                                      LAYER

                         INPUT
                                                OUTPUT
                         LAYER   .4              LAYER

                   1             .2
                           1

                                 .7
               -1          -1
                          …




                                                  …
               -1          -1
                                       …



 There are 26 input nodes; suppose we determine that

        w ij x j       (0.4 1) (0.2        1) ... (0.7   1)   0.3
all i
HIDDEN
                                    LAYER

                    INPUT
                                                    OUTPUT
                    LAYER     .4                     LAYER

            1                 .2
                       1

                              .7
           -1         -1
                      …




                                                      …
           -1         -1
                                       …



5. We then decide based on this sum how to determine the state of this node.
A possible rule would be that if the sum is nonnegative, the state should be
1 (the neuron fires), and if it is negative, the state should be -1 (it doesn’t).
HIDDEN
                                   LAYER

                   INPUT              -1
                                                     OUTPUT
                   LAYER     .4                       LAYER

          1                  .2
                      1

                             .7
          -1         -1
                     …




                                                       …
          -1         -1
                                      …



Thus, as –0.3 < 0, this node’s state is set to –1.
HIDDEN
                                  LAYER

                   INPUT            -1
                                                 OUTPUT
                   LAYER
                                                  LAYER

          1          1               1



         -1         -1               1
                    …




                                                   …
                                    -1
         -1         -1
                                    …
                                    -1

6. We repeat this process for every other node in the hidden layer.
HIDDEN
                                        LAYER

                           INPUT           -1
                                                         OUTPUT
                           LAYER                      .3 LAYER

                  1                        1     .4
                             1                             1      1
                                                 .4

                 -1         -1             1     .1
                            …




                                                          …
                                                 .2
                                           -1
                 -1         -1
                                          …
                                           -1

7. With all of the states in the hidden layer computed, we begin to compute
the states of the output layer in the same way. Here, suppose
           w ij x j   0.2 0, so we set the node's state to 1.
   all i
HIDDEN
                                 LAYER

                  INPUT             -1
                                                OUTPUT
                  LAYER
                                                 LAYER

          1         1               1              1         1


         -1         -1              1              -1        -1
                    …




                                                   …
                                    -1
         -1         -1                             -1        -1
                                    …
                                    -1

8. Repeat for every node in the output layer. We now have our set of outputs.
Each node in the output layer would correspond to 1 digit.
HIDDEN




                                                                     1
                                   LAYER

                   INPUT             -1
                                                  OUTPUT
                   LAYER
                                                   LAYER

          1          1                1               1         1


         -1          -1               1              -1         -1
                     …




                                                     …
                                     -1
         -1          -1                              -1         -1
                                     …
                                     -1

9. Hopefully, we have only one output node firing; this means that the
network has identified the input to be the digit corresponding to that output.
If none / more than one fire, the network has not successfully determined
what digit was input.

Neural Network Classification-Explanation

  • 1.
    1. To convertthe input into something that the network can process, we normalise the size of the input and convert it to an array of pixels (here a 10 x 13 array).
  • 2.
    HIDDEN LAYER INPUT OUTPUT LAYER LAYER 1 -1 … … -1 … 2. Inputs are sent into the network, possibly ‘1’ for a dark pixel and ‘-1’ for a light pixel.
  • 3.
    HIDDEN LAYER INPUT OUTPUT LAYER LAYER 1 1 -1 -1 … … -1 -1 … 3. Each node in the hidden layer evaluates a weighted sum of its inputs, and determines its state. We will demonstrate this for 1 node.
  • 4.
    HIDDEN LAYER INPUT OUTPUT LAYER LAYER 1 1 -1 -1 … … -1 -1 … 4. Let this hidden node be j, and nodes connecting to it be i. We find w ij x j all i
  • 5.
    HIDDEN LAYER INPUT OUTPUT LAYER .4 LAYER 1 .2 1 .7 -1 -1 … … -1 -1 … There are 26 input nodes; suppose we determine that w ij x j (0.4 1) (0.2 1) ... (0.7 1) 0.3 all i
  • 6.
    HIDDEN LAYER INPUT OUTPUT LAYER .4 LAYER 1 .2 1 .7 -1 -1 … … -1 -1 … 5. We then decide based on this sum how to determine the state of this node. A possible rule would be that if the sum is nonnegative, the state should be 1 (the neuron fires), and if it is negative, the state should be -1 (it doesn’t).
  • 7.
    HIDDEN LAYER INPUT -1 OUTPUT LAYER .4 LAYER 1 .2 1 .7 -1 -1 … … -1 -1 … Thus, as –0.3 < 0, this node’s state is set to –1.
  • 8.
    HIDDEN LAYER INPUT -1 OUTPUT LAYER LAYER 1 1 1 -1 -1 1 … … -1 -1 -1 … -1 6. We repeat this process for every other node in the hidden layer.
  • 9.
    HIDDEN LAYER INPUT -1 OUTPUT LAYER .3 LAYER 1 1 .4 1 1 1 .4 -1 -1 1 .1 … … .2 -1 -1 -1 … -1 7. With all of the states in the hidden layer computed, we begin to compute the states of the output layer in the same way. Here, suppose w ij x j 0.2 0, so we set the node's state to 1. all i
  • 10.
    HIDDEN LAYER INPUT -1 OUTPUT LAYER LAYER 1 1 1 1 1 -1 -1 1 -1 -1 … … -1 -1 -1 -1 -1 … -1 8. Repeat for every node in the output layer. We now have our set of outputs. Each node in the output layer would correspond to 1 digit.
  • 11.
    HIDDEN 1 LAYER INPUT -1 OUTPUT LAYER LAYER 1 1 1 1 1 -1 -1 1 -1 -1 … … -1 -1 -1 -1 -1 … -1 9. Hopefully, we have only one output node firing; this means that the network has identified the input to be the digit corresponding to that output. If none / more than one fire, the network has not successfully determined what digit was input.