Diploma of Advanced Studies

519 views
394 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
519
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Diploma of Advanced Studies

  1. 1. Outline MD Classification MDSSL Application to SA Conclusions Future Work DEA - Diploma of Advanced Studies Semi-supervised Learning of Multi-dimensional Class Bayesian Network Classifiers: Application to Sentiment Analysis Jonathan Ortigosa-Hern´ndez a advised by Jos´ A. Lozano and I˜aki Inza e n Intelligent Systems Group Computer Science and Artificial Intelligence Department University of the Basque Country November 4th, 2010 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  2. 2. Outline MD Classification MDSSL Application to SA Conclusions Future WorkCurriculum Vitae Undergraduate Education 2003-2008: Masters of Science Degree in Computer Engineering, University of the Basque Country. 2007-2008: Bachelor of Science Degree in Informatics, Coventry University. Postgraduate Education 2008-Present: PhD Student, ISG Group, University of the Basque Country (Four-Year MEC-FPU Grant). Doctorate Program: Probabilistic Graphical Models for Artificial Intelligence and Data Mining. Doctorate Lectures: Fundamentals of Probabilistic Graphical Models Inference in PGMs Learning PGMs Bioinformatic Applications of PGMs Scientific Research Methodology Statistical and Computational Basis for PGMs Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  3. 3. Outline MD Classification MDSSL Application to SA Conclusions Future WorkResearch Interest Methodology Multi-dimensional Classification Multi-dimensional Class Bayesian Network Classifiers Semi-supervised Learning Applications Opinion Mining and Sentiment Analysis Affect Analysis Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  4. 4. Outline MD Classification MDSSL Application to SA Conclusions Future Work 1 Multi-dimensional Supervised Classification 2 Multi-dimensional Semi-supervised Learning 3 Application to Sentiment Analysis 4 Conclusions 5 Current and Future Work Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  5. 5. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSupervised Classification It consists of building a classifier Ψ from a given labelled training dataset D, by using an induction algorithm A (A(D) = Ψ), X1 X2 ... Xn C (1) (1) (1) x1 x2 ... xn c (1) (2) (2) (2) x1 x2 ... xn c (2) ... ... ... ... ... (N) (N) (N) x1 x2 ... xn c (N) in order to predict the value of a class variable C for any new unlabelled instance x (Ψ(x) = c). Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  6. 6. Outline MD Classification MDSSL Application to SA Conclusions Future WorkUni-dimensional and Multi-dimensional Classification Uni-dimensional classification tries to predict a single class variable based on a dataset composed of a set of labelled examples. (Uni-dimensional Class) Bayesian Network Classifiers (Larra˜aga et al, 2005). n Multi-dimensional classification is the generalisation of the single-class classification task to the simultaneous prediction of a set of class variables. Multi-dimensional Class Bayesian Network Classifiers (v.d. Gaag and d. Waal, 2006). Do not confuse with multi-class and multi-label classification. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  7. 7. Outline MD Classification MDSSL Application to SA Conclusions Future WorkMulti-dimensional Supervised Learning A typical supervised training dataset X1 X2 ... Xn C1 C2 ... Cm (1) (1) (1) (1) (1) (1) x1 x2 ... xn c1 c2 ... cm (2) (2) (2) (2) (2) (2) x1 x2 ... xn c1 c2 ... cm ... ... ... ... ... ... ... ... (N) (N) (N) (N) (N) (N) x1 x2 ... xn c1 c2 ... cm Each instance of the dataset contains both the values of the attributes and m labels which characterise the attributes. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  8. 8. Outline MD Classification MDSSL Application to SA Conclusions Future WorkBayesian Network Classifiers C X1 X2 X3 X4 X5 X6 Figure: A (uni-dimensional) naive Bayes structure. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  9. 9. Outline MD Classification MDSSL Application to SA Conclusions Future WorkMulti-dimensional Class Bayesian Network Classifiers(MDBNC) C1 C2 C3 X1 X2 X3 X4 X5 X6 Figure: A multi-dimensional naive Bayes structure. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  10. 10. Outline MD Classification MDSSL Application to SA Conclusions Future WorkMDBNC Structure C1 C2 C3 C1 C2 C3 X1 X2 X3 X4 X5 X1 X2 X3 X4 X5 (a) Complete graph (b) Feature selection subgraph C1 C2 C3 X1 X2 X3 X4 X5 (c) Class subgraph (d) Feature subgraph Figure: A MDNBC structure and its division. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  11. 11. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC (a) Multi-dimensional naive Bayes (b) Multi-dimensional tree-augmented network (c) Multi-dimensional J/K dependence Bayesian (2/3) Figure: Different subfamilies of MDBNC. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  12. 12. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC - MDnB Multi-dimensional naive Bayes (MDnB) The class and feature subgraphs are empty. Each class variable is parent of all the features. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  13. 13. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC - MDnB Multi-dimensional naive Bayes (MDnB) It has a fixed structure. Thus, it has no structural learning (v.d. Gaag and d. Waal, 2006). Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  14. 14. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC - MDnB Multi-dimensional tree-augmented network classifier (MDTAN) The class and feature subgraphs are trees. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  15. 15. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC - MDnB Multi-dimensional tree-augmented network classifier (MDTAN) A wrapper structural learning algorithm is proposed in (v.d. Gaag and d. Waal, 2006). Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  16. 16. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC - MDnB Multi-dimensional J/K dependence Bayesian classifier (MD J/K ) The class subgraph is a J-dependence graph. The feature subgraph is a K -dependence graph. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  17. 17. Outline MD Classification MDSSL Application to SA Conclusions Future WorkSub-families of MDBNC - MDnB Multi-dimensional J/K dependence Bayesian classifier (MD J/K ) There was not a specific structural learning algorithm. So, we proposed a learning algorithm in (Ortigosa-Hern´ndez et a al, 2010). Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  18. 18. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 0 - Initialisation Establish the maximum C1 C2 C3 C4 number of parents in both class and feature subgraphs, i.e. J = 2 X1 X2 X3 X4 X5 X6 X7 X8 and K = 2. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  19. 19. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 1 - Learn the structure between the class variables (Ac ) C1 C2 C3 C4 Calculate the mutual information MI (Ci , Cj ) for each pair of class X1 X2 X3 X4 X5 X6 X7 X8 variables. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  20. 20. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 1 - Learn the structure between the class variables (Ac ) Calculate the p-values (significance of each mutual information) C1 C2 C3 C4 using independence test. C1 C2 C3 X1 X2 X3 X4 X5 X6 X7 X8 C4 0.36 0.57 0.01 C3 0.27 0.63 C2 0.06 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  21. 21. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 1 - Learn the structure between the class variables (Ac ) Remove the p-values greater than the C1 C2 C3 C4 threshold α = 0.1. C1 C2 C3 C4 0.36 0.57 0.01 X1 X2 X3 X4 X5 X6 X7 X8 C3 0.27 0.63 C2 0.06 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  22. 22. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 1 - Learn the structure between the class variables (Ac ) From the lowest value, start adding arcs to the graph fulfilling the conditions of no cycles C1 C2 C3 C4 and no more than J-parents per class variable. X1 X2 X3 X4 X5 X6 X7 X8 C1 C2 C3 C4 x x 0.01 C3 x x C2 0.06 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  23. 23. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 1 - Learn the structure between the class variables (Ac ) From the lowest value, start adding arcs to the graph fulfilling the conditions of no cycles C1 C2 C3 C4 and no more than J-parents per class variable. X1 X2 X3 X4 X5 X6 X7 X8 C1 C2 C3 C4 x x x C3 x x C2 0.06 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  24. 24. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 2 - Learn the structure between the class variables and the features (ACF ) C1 C2 C3 C4 Calculate the mutual information MI (Ci , Xj ) for each pair Ci and Xj . X1 X2 X3 X4 X5 X6 X7 X8 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  25. 25. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 2 - Learn the structure between the class variables and the features (ACF ) Calculate the p-value of each mutual information. C1 C2 C3 C4 C1 C2 C3 C4 X1 0.64 0.00 0.77 0.98 X2 0.82 0.03 0.11 0.37 X3 0.00 0.06 0.00 0.01 X4 0.68 0.09 0.00 0.55 X1 X2 X3 X4 X5 X6 X7 X8 X5 0.81 0.12 0.81 0.65 X6 0.57 0.24 0.00 0.00 X7 0.25 0.26 0.00 0.00 X8 0.32 0.15 0.00 0.44 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  26. 26. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 2 - Learn the structure between the class variables and the features (ACF ) Remove the p-values greater than the threshold α = 0.1. C1 C2 C3 C4 C1 C2 C3 C4 X1 0.64 0.00 0.77 0.98 X2 0.82 0.03 0.11 0.37 X3 0.00 0.06 0.00 0.01 X4 0.68 0.09 0.00 0.55 X1 X2 X3 X4 X5 X6 X7 X8 X5 0.81 0.12 0.81 0.65 X6 0.57 0.24 0.00 0.00 X7 0.25 0.26 0.00 0.00 X8 0.32 0.15 0.00 0.44 Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  27. 27. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 2 - Learn the structure between the class variables and the features (ACF ) Add all the arcs to the structure. C1 C2 C3 C4 C1 C2 C3 C4 X1 x 0.00 x x X2 x 0.03 x x X3 0.00 0.06 0.00 0.01 X4 x 0.09 0.00 x X1 X2 X3 X4 X5 X6 X7 X8 X5 x x x x X6 x x 0.00 0.00 X7 x x 0.00 0.00 X8 x x 0.00 x Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  28. 28. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 3 - Learn the structure between the features(AF ) Calculate the conditional mutual information C1 C2 C3 C4 MI (Xi , Xj ||Pac (Xj )). Calculate the p-values. X1 X2 X3 X4 X5 X6 X7 X8 Remove the p-values greater than the threshold α = 0.1. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  29. 29. Outline MD Classification MDSSL Application to SA Conclusions Future WorkA supervised method to learn a MD J/K structure(Ortigosa-Hern´ndez et al, 2010) a Step 3 - Learn the structure between the features(AF ) Add arcs between the C1 C2 C3 C4 features fulfilling the conditions of no cycles between the features X1 X2 X3 X4 X5 X6 X7 X8 and no more than K -parents per feature. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  30. 30. Outline MD Classification MDSSL Application to SA Conclusions Future WorkMajor Problem of Supervised Learning However, in many real world problems, obtaining data is relatively easy, while labelling is difficult, expensive or labor intensive (usually done by an external mechanism, e.g. human beings). This problem is accentuated when using multiple target variables. DESIRE: Learning algorithms able to incorporate a large number of unlabelled data with a small number of labeled data when learning competitive classifiers. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  31. 31. Outline MD Classification MDSSL Application to SA Conclusions Future WorkMulti-dimensional Semi-supervised Learning A typical semi-supervised training dataset X1 X2 ... Xn C1 C2 ... Cm (1) (1) (1) (1) (1) (1) x1 x2 ... xn c1 c2 ... cm (2) (2) (2) (2) (2) (2) x1 x2 ... xn c1 c2 ... cm ... ... ... ... ... ... ... ... (L) (L) (L) (L) (L) (L) x1 x2 ... xn c1 c2 ... cm (L+1) (L+1) (L+1) x1 x2 ... xn ? ? ... ? (L+2) (L+2) (L+2) x1 x2 ... xn ? ? ... ? ... ... ... ... ... ... ... ... (N) (N) (N) x1 x2 ... xn ? ? ... ? Semi-supervised Learning fulfils this desire. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  32. 32. Outline MD Classification MDSSL Application to SA Conclusions Future WorkThe Expectation-Maximisation Algorithm The EM algorithm (Dempster et al, 1977) Learn an initial model. Repeat until convergence: (a) Expectation step: Using the current model, estimate the missing values of the data. (b) Maximisation step: Using the whole data and the previous estimations, learn a new current model. Any MDBNC learning algorithm can be used as model in this algorithm. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  33. 33. Outline MD Classification MDSSL Application to SA Conclusions Future WorkArtificial Experimentation 20 different mult-dimensional datasets are sampled. Training subset: 10, 000 instances (L:100/U:10, 000) Test subset: 5, 000 Feat. Class V. Num. 5 to 20 2 to 4 Card. 2 to 4 2 to 3 Learning algorithms: 4 uni-dimensional (nB, TAN, 2-DB and 3-DB) 4 multi-dimensional (MDnB, MDTAN, MD 2/2 and MD 2/3) Scenario: Supervised / Semi-supervised Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  34. 34. Outline MD Classification MDSSL Application to SA Conclusions Future WorkArtificial Experimentation Numerical results can be found in: http://www.sc.ehu.es/ccwbayes/members/jonathan/home/News_and_Notables/Entries/2010/11/30_ Artificial_Experiments_2010.html (Semi-supervised) Multi-dimensional algorithms V (Supervised) Multi-dimensional algorithms V Uni-dimensional algorithms Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  35. 35. Outline MD Classification MDSSL Application to SA Conclusions Future WorkApplication to Sentiment Analysis Sentiment Analysis (AKA Opinion Mining) is the computational study of opinions, sentiments and emotions expressed in text (Liu, 2010). When treating Sentiment Analysis as a classification problem, several different (but related) problems appear. For example: 1 Subjectivity Classification. Its aim is to classify a text as subjective or objective. 2 Sentiment Classification. It classifies an opinionated text as expressing a positive, neutral, or negative opinion. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  36. 36. Outline MD Classification MDSSL Application to SA Conclusions Future WorkMotivation for Using Semi-supervised Learning ofMulti-dimensional Classifiers 1 Up to now, these subproblems have been studied in isolation despite of being closely related. So, probably it would be helpful to use multi-dimensional classifiers. 2 Obtaining enough labeled examples for a classifier may be costly and time consuming. This motivates us to deal with unlabelled examples in a semi-supervised framework. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  37. 37. Outline MD Classification MDSSL Application to SA Conclusions Future WorkHypothesis Formulated First Hypothesis The explicit use of the relationships between different class variables can be beneficial to improve their recognition rates. Second Hypothesis Multi-dimensional techniques can work with unlabelled data in order to improve the classification rates in Sentiment Analysis. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  38. 38. Outline MD Classification MDSSL Application to SA Conclusions Future WorkProperties of the Dataset Collected by Socialware Company S.A., from the ASOMO service of mobilised opinion analysis. It consists of 2, 542 Spanish reviews extracted from a blog: 150 documents have been labeled in isolation by an expert. 2, 392 posts are left unlabelled. Figure: The ASOMO corpus. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  39. 39. Outline MD Classification MDSSL Application to SA Conclusions Future WorkProperties of Each Document Each document is represented as: 14 features Obtained by using an open source morphological analyser (Carreras et al, 2006). Each feature provide different information related to part-of-speech (POS). Eg. First Persons, Agreement Expressions, Imperatives, Prediction Verbs (future), Questions, Positive Adjectives, etc. Represented as a real number between 0 and 1. 3 class variables Will to Influence: {declarative sentence, soft WI, medium WI, strong WI} Sentiment: {very negative, negative, neutral, positive, very positive} Subjectivity: {Yes (subjective), No (objective)} Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  40. 40. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 1 - Set Up I First Hypothesis The explicit use of the relationships between different class variables can be beneficial to improve their recognition rates. The ASOMO corpus has been used to learn: 3 (uni-dimensional) naive Bayes classifiers, one per each class variable. A (uni-dimensional) naive Bayes classifier with a compound class variable. A multi-dimensional naive Bayes classifier. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  41. 41. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 1 - Set Up II First Hypothesis The explicit use of the relationships between different class variables can be beneficial to improve their recognition rates. Features from ASOMO dataset are discretised into 3 values using equal frequency. In addition to the ASOMO feature set, two state-of-the-art feature sets are used: Unigrams Unigrams + Bigrams Results averaged over 5 × 5 fold cross validation. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  42. 42. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 1- JOINT Accuracies Figure: JOINT accuracies on ASOMO corpus using three different feature sets in both uni and multi-dimensional scenarios (5 × 5cv) Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  43. 43. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 1 - Computation Time Figure: Computational times of the learning algorithms using different feature sets. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  44. 44. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 2 - Set Up Second Hypothesis Multi-dimensional techniques can work with unlabelled data in order to improve the classification rates in Sentiment Analysis. The ASOMO dataset has been used to learn: 3 (uni-dimensional) Bayesian network classifiers: nB, TAN and 2DB. 5 MDBNC: MDnB, MDTAN, MD 2/2, MD 2/3 and MD 2/4. In both Supervised and Semi-supervised (EM algorithm) learning frameworks. Features from ASOMO dataset are discretised into 3 values using equal frequency. Results averaged over 5 × 5 fold cross validation. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  45. 45. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 2 - JOINT Accuracy 25 20 15 10 5 nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4 Figure: JOINT accuracies on ASOMO dataset in the supervised and semi-supervised learning frameworks. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  46. 46. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 2 - Will to Influence 70 60 50 40 30 nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4 Figure: Accuracies for the Will to Influence class variable on ASOMO dataset in the supervised and semi-supervised learning frameworks. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  47. 47. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 2 - Sentiment Polarity 40 35 30 25 20 nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4 Figure: Accuracies for the Sentiment Polarity class variable on ASOMO dataset in the supervised and semi-supervised learning frameworks. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  48. 48. Outline MD Classification MDSSL Application to SA Conclusions Future WorkExperiment 2 - Subjectivity 90 80 70 60 50 nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4 Figure: Accuracies for the Subjectivity class variable on ASOMO dataset in the supervised and semi-supervised learning frameworks. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  49. 49. Outline MD Classification MDSSL Application to SA Conclusions Future WorkConclusions I - Methodology Multi-dimensional classification and semi-supervised learning are two different branches of machine learning. With this research, we have established a bridge between them showing that: Uni-dimensional approaches cannot capture the real nature of multi-dimensional problems. More accurate classifiers can be found using the multi-dimensional learning approaches. The use of large amounts of unlabelled data can be beneficial to improve recognition rates. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  50. 50. Outline MD Classification MDSSL Application to SA Conclusions Future WorkConclusions II - Application With respect to the Sentiment Analysis application, we have proposed a novel perspective to solve the problem. Experimental results demonstrate that the use of multi-dimensional classification, as well as the use of unlabelled data, can lead us to more accurate classifiers. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  51. 51. Outline MD Classification MDSSL Application to SA Conclusions Future WorkConclusions III - Publications Publication J. Ortigosa-Hernandez, J.D. Rodriguez, L. Alzate, I. Inza, J.A. Lozano. (2010). A Semi-supervised Approach to Multi-dimensional Classification with Application to Sentiment Analysis. CEDI 2010, V Simposio de Teoria y Aplicaciones de Mineria de Datos (TAMIDA2010), Valencia, Spain. We are writing the final draft of a paper for the Special Issue on Data Mining Applications and Case Studies at Neurocomputing. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  52. 52. Outline MD Classification MDSSL Application to SA Conclusions Future WorkShort Term Future Work Title: Application to Affect Analysis Description: Use the methodology proposed in this presentation to deal with the problem of Affect Analysis. Collaboration: Socialware S.A. Motivation (Abbasi et al., 2008) Affect Analysis is concerned with the analysis of text containing emotions and it tries to extract a large number of potential emotions, e.g. happiness, sadness, anger, hate, violence, excitement, etc. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  53. 53. Outline MD Classification MDSSL Application to SA Conclusions Future WorkShort Term Future Work We want to take a step forward in this problem taking advantage of the potential possibilities of the MDBNC to model complex relationships between the class variables. (a) Plutchik’s affect model (b) Chromatic affect model Figure: Psychological Human Affection models considered for this project. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  54. 54. Outline MD Classification MDSSL Application to SA Conclusions Future WorkLong Term Future Work Wrapper Structural Search Algorithm. Adapt other semi-supervised learning approaches to the multi-dimensional classification domain, e.g. Co-training, Active Learning, ... The scalability of MDBNC is a problem that has to be studied (high dimensionality of multi-label problems). “As discussed in the literature, currently there is no coherent strategy for handling unlabelled data, so some creativity must be exercised.” (Cohen’s thesis, 2003) Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  55. 55. Outline MD Classification MDSSL Application to SA Conclusions Future WorkQuestions THANK YOU jonathan.ortigosa@ehu.es Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies
  56. 56. Outline MD Classification MDSSL Application to SA Conclusions Future WorkReferences Abbasi, A. and Chen, H. and Thoms, S. and Fu, T. (2008). Affect Analysis of Web Forums and Blogs using Correlation Ensembles. IEEE Transactions on Knowledge and Data Engineering, Vol. 20(9), pp. 1168–1180. Carreras X., Chao I., Padro L. and Padro M. (2006). An Open-Source Suite of Language Analyzers. In Proceedings of the 4th Int. Conference on Language Resources and Evaluation, Vol. 10, pp. 239–342. Cozman, F. and Cohen, I. (2006). Risk of Semi-Supervised Learning. In: Chapelle, O. Scholkopf, B. and Zien, A. Semi-Supervised Learning. The MIT Press. pp 57-72. Dempster A., Laird N. and Rubin D. (1977). Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society. Series B, 39(1): 1–38. Friedman, N. (1998.) The Bayesian Structural EM algorithm. In Proc. 14th Conf. on Uncertainty in Artificial Intelligence. Morgan Kaufmann, San Francisco, CA, pp. 129–138. van der Gaag L. and d. Waal P. (2006). Multi-dimensional Bayesian Classifiers. In Proceedings of the Third European Workshop in Probabilistic Graphical Models, pages 107–114. Larra˜aga, P., Lozano, J.A., Pe˜a, J.M. and Inza, I. (2005). Special Issue on Probabilistic Graphical Models for n n Classification. Machine Learning, 59(3). Liu B. (2010). Sentiment Analysis and Subjectivity. In: Indurkhya N. and Damerau F.J. Handbook of Natural Language Processing, Chapman & Hall, 2nd Ed. Ortigosa-Hernandez J., Rodriguez J.D., Alzate L., Inza I. and Lozano J.A. (2010). A Semi-supervised Approach to Multi-dimensional Classification with Application to Sentiment Analysis. In Proc. of the V Simposio de Teoria y Aplicaciones de Mineria de Datos (TAMIDA2010), CEDI 2010, Valencia, Spain. Rodriguez, J.D. and Lozano, J.A. (2008). Multi-objective learning of multi-dimensional Bayesian classifiers. In Proceedings of the Eighth International Conference on Hybrid Intelligent Systems, HIS 2008, Barcelona, Spain. pp. 501-506. Jonathan Ortigosa-Hern´ndez a DEA - Diploma of Advanced Studies

×