Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- NIPS読み会2013: One-shot learning by i... by nozyh 17211 views
- 論文紹介 Semi-supervised Learning with ... by Seiya Tokui 138644 views
- NIPS2015読み会: Ladder Networks by Eiichi Matsumoto 55977 views
- IIBMP2016 深層生成モデルによる表現学習 by Preferred Networks 35702 views
- An Online Spark Pipeline: Semi-Supe... by Databricks 958 views
- 機械学習とコンピュータビジョン入門 by Kinki University 6380 views

No Downloads

Total views

24,614

On SlideShare

0

From Embeds

0

Number of Embeds

12,172

Shares

0

Downloads

251

Comments

0

Likes

35

No embeds

No notes for slide

- 1. Introduction to Semi- Supervised Learning Thursday, January 21, 2010
- 2. Outline • • • Self Traininga • Multiview Algorithms • Generative Models • S3VMs(TSVMs) • Graph-Based Algorithms • • Thursday, January 21, 2010
- 3. ? Thursday, January 21, 2010
- 4. • • • costly... • • • : Thursday, January 21, 2010
- 5. • • : ? => SSL Thursday, January 21, 2010
- 6. ? Thursday, January 21, 2010
- 7. • • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 8. SSL • Self Training • Multiview Algorithms • Generative Models • S3VMs(TSVMs) • Graph-Based Algorithms Thursday, January 21, 2010
- 9. Self Training Thursday, January 21, 2010
- 10. Self Training • : • : p(y=1|x) 0.99 1 ! • Thursday, January 21, 2010
- 11. Self Training • f • f •( ) • Thursday, January 21, 2010
- 12. Self Training • • ! • • • • • Thursday, January 21, 2010
- 13. Multiview Algorithms Thursday, January 21, 2010
- 14. Co-training • Self Training => • ? • Thursday, January 21, 2010
- 15. : Web http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 16. • • ) (BOF • • Co-training idea • • Thursday, January 21, 2010
- 17. Co-training • • • Thursday, January 21, 2010
- 18. Co-training http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 19. Co-training • • • Self Training • • • Thursday, January 21, 2010
- 20. Co-training • Co-EM • EM • Fake feature split • • Multiview • • => Thursday, January 21, 2010
- 21. Generative Models Thursday, January 21, 2010
- 22. Generative Models • • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 23. Generative Models • • • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 24. Generative Models • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 25. Generative Models • • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 26. Generative Models • : • ( ) • Thursday, January 21, 2010
- 27. Generative Models • • • EM • • • EM • • • Baum-Welch algorithm Thursday, January 21, 2010
- 28. • • • : PRML => EM MLE Thursday, January 21, 2010
- 29. • ( ) • y 2 • EM MLE Thursday, January 21, 2010
- 30. E-step and M-step http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 31. Generative Models • • • • • • EM ( ) • => page Thursday, January 21, 2010
- 32. Generative Models http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 33. • • λ(λ < 1) Thursday, January 21, 2010
- 34. S3VMs(TSVMs) Thursday, January 21, 2010
- 35. TSVMs • ( ) http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 36. (RKHS) • SVM • => (RKHS:reproducing kernel Hilbert space) • Thursday, January 21, 2010
- 37. k k f Thursday, January 21, 2010
- 38. • • • ( ) • Thursday, January 21, 2010
- 39. • (F ) • x F (F !) Thursday, January 21, 2010
- 40. •f F • • f f( x ) => f => F => !! Thursday, January 21, 2010
- 41. Thursday, January 21, 2010
- 42. SVM RKHS ! hw RKHS Thursday, January 21, 2010
- 43. hinge loss hinge loss !! http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 44. • • ? • f(x)>1 or f(x)<-1 • ! => http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 45. • Joachims(1999) • branch- and-bound search( ) • 10000 Thursday, January 21, 2010
- 46. S3VM • • • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 47. CCCP (Concave-Convex procedure) • Yuille (2003) • • ) ( • Difference of Convex (DC ) • Update Thursday, January 21, 2010
- 48. CCCP (Concave-Convex procedure) • update • • (update ) • !! Thursday, January 21, 2010
- 49. TSVMs + CCCP • CCCP TSVMs • • Iteration 2 http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf http://www.stat.umn.edu/~xshen/paper/tsvm.pdf Thursday, January 21, 2010
- 50. L. Wang (2007) • SVM TSMVs CCCP TSMVs http://www.stat.umn.edu/~xshen/paper/tsvm.pdf Thursday, January 21, 2010
- 51. Graph-Based Algorithms Thursday, January 21, 2010
- 52. : • • http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 53. • !! • • etc http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 54. • sparseness http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf Thursday, January 21, 2010
- 55. http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf https://www.aaai.org/Papers/ICML/2003/ICML03-118.pdf Thursday, January 21, 2010
- 56. Graph-based SSL • : edge node • : • (instance ) • kNN : k 1 0 => sparse • : • dense Thursday, January 21, 2010
- 57. Graph-based SSL • mincut • harmonic • manifold regularization Thursday, January 21, 2010
- 58. • • 0 1 ﬁx • Thursday, January 21, 2010
- 59. • => • sink source • : http://john.blitzer.com/tutorial/ssl_tutorial.pdf Thursday, January 21, 2010
- 60. source sink : source ∞ : sink -∞ Thursday, January 21, 2010
- 61. harmonic => Laplacian • 0-1 y • Zhu (2003a) Goldberg (2006) • Thursday, January 21, 2010
- 62. harmonic => Laplacian (SVM ) ( L a U b f ... ) → !! Thursday, January 21, 2010
- 63. http://pages.cs.wisc.edu/~goldberg/publications/goldbergTextgraphs.pdf Thursday, January 21, 2010
- 64. y_i hat{y}_i http://pages.cs.wisc.edu/~goldberg/publications/goldbergTextgraphs.pdf Thursday, January 21, 2010
- 65. Laplacian •f 0 ! => Thursday, January 21, 2010
- 66. Goldberg (2006) • SVMR Metric labeling • SSL • http://pages.cs.wisc.edu/~goldberg/publications/goldbergTextgraphs.pdf Thursday, January 21, 2010
- 67. Manifold regularization • Harmonic • ... • • RKHS • :) Thursday, January 21, 2010
- 68. Manifold regularization • • • SVM • ! Thursday, January 21, 2010
- 69. : • • 2 • ( )RKHS • Thursday, January 21, 2010
- 70. • f • f Thursday, January 21, 2010
- 71. • • • Thursday, January 21, 2010
- 72. • • Iterated Laplacians • Heat semigroup • • Squared norm of the Hessian • Belkin (2004) http://www.geocities.co.jp/Technopolis/5893/publication/kernel.pdf Thursday, January 21, 2010
- 73. Laplacian Regularized Least Squres(LapRLS) • + 0 Regularized Least Squres • closed Thursday, January 21, 2010
- 74. Laplacian Support Vector Machines • SVM • 0 SVM Thursday, January 21, 2010
- 75. ? • (Harmonic ) • • Thursday, January 21, 2010
- 76. Thursday, January 21, 2010
- 77. mixture model, EM TSMVs Co-training i.i.d ( Graph-based ) Thursday, January 21, 2010
- 78. • no pain, no gain • no model assumption, no gain • wrong model assumption, no gain, a lot of pain Thursday, January 21, 2010
- 79. ( | ) • • Co-boosting • bootstrap • Directed graphs • Information Regularization • Structural Learning • Large Data • • PAC Statistical Learning Theory Thursday, January 21, 2010
- 80. Thursday, January 21, 2010
- 81. (Tutorial etc) • • NAACL 2006 Tutorial: Inductive Semi-supervised Learning with Applicability to NLP, A. Sarkar and G. Haffari. • ICML 2007 Tutorial: Semi-supervised Learning, Xiaojin Zhu. • Blitzer, J. and Zhu, J. (2008). ACL 2008 tutorial on Semi- Supervised learning. http://ssl-acl08.wikidot.com/. • • X. Zhu. Semi-supervised learning literature survey. Technical report, Computer Sciences, University of Wisconsin- Madison, 2007. • Zhu, X. (2005). Semi-supervised learning with graphs. Doctoral dissertation, Carnegie Mellon University. CMU- LTI-05-192. Thursday, January 21, 2010
- 82. (Generative model) • Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). Text classiﬁcation from labeled and unlabeled documents using EM. Machine Learning, 39, 103–134. • Liu, B., Lee, W. S.,Yu, P. S., & Li, X. (2002). Partially supervised classiﬁcation of text documents. Proceedings of the Nineteenth International Conference on Machine Learning (ICML). • Lee, W. S., & Liu, B. (2003). Learning with positive and unlabeled examples using weighted logistic regression. Proceedings of the Twentieth International Conference on Machine Learning (ICML). • Denis, F., Gilleron, R., & Tommasi, M. (2002). Text classiﬁcation from positive and unlabeled examples. The 9th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems(IPMU). Thursday, January 21, 2010
- 83. (TSVMs) • Joachims, T. (1999). Transductive inference for text classiﬁcation using support vector machines. Proc. 16th International Conf. on Machine Learning (pp. 200– 209). Morgan Kaufmann, San Francisco, CA. • Yuille, A.L., Rangara jan, A. The concave-convex procedure. Neural Computation 15(4) (2003) 915–936. • L. Wang, X. Shen, and W. Pan. On transductive support vector machines. In J. Verducci, X. Shen, and J. Lafferty, editors, Prediction and Discovery. American Mathematical Society, 2007. • R. Collobert, et al. (2006). Large Scale Transductive SVMs. Journal of Machine Learning Research 7:1687-1712. Thursday, January 21, 2010
- 84. (Graph-based) • Blum, A., & Chawla, S. (2001). Learning from labeled and unlabeled data using graph mincuts. Proc. 18th International Conf. on Machine Learning. • Zhu, X., Ghahramani, Z., & Lafferty, J. (2003a). Semi-supervised lear ning using Gaussian ﬁelds and harmonic functions. The 20th International Conference on Machine Learning (ICML). • Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 888–905. • Pang, B., & Lee, L. (2004). A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. Proceedings of the Association for Computational Linguistics (pp. 271–278). • Goldberg, A., & Zhu, X. (2006). Seeing stars when there aren’t many s tars: Graph- based semi-supervised learning for sentiment categorization. HLT-NAACL 2006 Workshop on Textgraphs: Graph-based Algorithms for Natural Language Processing. New York, NY. • Belkin, M., Niyogi, P., & Sindhwani, V. (2004b). Manifold regularization: A geometric framework for learning from examples (Technical Report TR-2004-06). University of Chicago. • M. Belkin & P. Niyogi (2002). `Using Manifold Structure for Partially Labelled Classiﬁcation'. In NIPS, pp. 929+. Thursday, January 21, 2010
- 85. ( ) • Seeger, M. (2001). Learning with labeled and unlabeled data (Technical Report). University of Edinburgh. • François Denis, Bat M, Universit'e De Lille I. PAC Learning from Positive Statistical Queries. Proc. 9th International Conference on Algorithmic Learning Theory - ALT '98 • • Thursday, January 21, 2010

No public clipboards found for this slide

Be the first to comment