This document discusses a 2006 paper that introduced a fast learning algorithm for deep belief nets. It allowed these types of networks to be trained efficiently for the first time. The paper presented contrastive divergence, an approximation algorithm that makes it possible to train deep belief networks layer-by-layer in a greedy, unsupervised manner. Additional reading on contrastive divergence and notes are also referenced.

1. © SuperDataScienceDeep Learning A-Z

2. © SuperDataScienceDeep Learning A-Z

3. © SuperDataScienceDeep Learning A-Z
Hidden
Nodes
Visible
Nodes

4. © SuperDataScienceDeep Learning A-Z
…

5. © SuperDataScienceDeep Learning A-Z
Energy
Data Space

6. © SuperDataScienceDeep Learning A-Z
…

7. © SuperDataScienceDeep Learning A-Z
Energy
Data Space

8. © SuperDataScienceDeep Learning A-Z
Energy
Data Space

9. © SuperDataScienceDeep Learning A-Z
Energy
Data Space

10. © SuperDataScienceDeep Learning A-Z
A fast learning algorithm for
deep belief nets
By Geoffrey Hinton et al. (2006)
Link:
https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf
Additional Reading:

11. © SuperDataScienceDeep Learning A-Z
Notes on Contrastive Divergence
By Oliver Woodford (2012?)
Link:
http://www.robots.ox.ac.uk/~ojw/files/NotesOnCD.pdf
Additional Reading: