Understanding Rbm by WangYuanTao

1,413 views
1,344 views

Published on

Published in: Technology
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total views
1,413
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
47
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Understanding Rbm by WangYuanTao

  1. 1. Understanding RBM Wang Yuantao 08/22/2009
  2. 2. Outline <ul><li>RBM Model </li></ul><ul><ul><li>For Netflix Prize Problem </li></ul></ul><ul><li>RBM Algorithm </li></ul><ul><ul><li>Implementation </li></ul></ul><ul><ul><li>Technical detail </li></ul></ul><ul><li>Contribution </li></ul><ul><ul><li>Model </li></ul></ul><ul><ul><li>Result </li></ul></ul>
  3. 3. Taxonomy <ul><li>By Determinacy </li></ul><ul><ul><li>Deterministic: SVD/kNN </li></ul></ul><ul><ul><li>Stochastic: RBM </li></ul></ul><ul><li>By Optimisation </li></ul><ul><ul><li>Empirical: kNN </li></ul></ul><ul><ul><li>Optimal </li></ul></ul><ul><ul><ul><li>Gradient descent: SVD </li></ul></ul></ul><ul><ul><ul><li>Maximum likelihood: RBM </li></ul></ul></ul>
  4. 4. Model Structure By Hinton
  5. 5. Model Assumption <ul><li>1 </li></ul><ul><li>2 </li></ul>
  6. 6. Optimal Object Maximize:
  7. 7. Training Sampling
  8. 8. Training Phases 1. Init v0 by real rating 2. Sample h0 by v0 3. Reconstruct v1 by h0 4. Re-sample h1 by v1 5. Update W 6. Compute Eh
  9. 9. Prediction
  10. 10. Technical detail <ul><li>Sparseness </li></ul><ul><li>Sample </li></ul><ul><ul><li>Gibbs 1-step sampling </li></ul></ul><ul><li>Update </li></ul><ul><ul><li>Batch v.s. online(per-case) </li></ul></ul><ul><ul><li>Training bias </li></ul></ul><ul><li>Learning rate </li></ul><ul><ul><li>Weight Decay </li></ul></ul><ul><ul><li>Momentum </li></ul></ul><ul><ul><li>Anneal </li></ul></ul>
  11. 11. Temporal RBM <ul><ul><li>F=16, lrate=0.002 </li></ul></ul>0.9373 0.9381 0.9385 0.9392 Original RBM
  12. 12. Contribution 0.8694 0.8688 RBM
  13. 13. Thanks <ul><li>Contact </li></ul><ul><ul><li>王元涛 </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>13521106828 </li></ul></ul><ul><li>Q&A </li></ul>

×