Interactive Machine Learning Tutorial.

1. Department of Computer Science
Interactive
Machine Learning (Appendix)
Zitao Liu
ztliu@cs.pitt.edu
School of Arts of and Sciences
Department of Computer Science

2. Department of Computer Science
Examples: CueFlik(CHI2008)
•Goal: allows users to quickly create their own rules for re- ranking images based on their visual characteristics.
•Workflow: user provides CueFlik with examples of images that each rule should match, is shown the rule that CueFlik has learned, and then continues to provide CueFlik with examples of images that should be matched or rejected until they are satisfied with the learned rules.
•ML Model: Nearest Neighbor; similarity measure; active learning.
•Specialties: None.

3. Department of Computer Science
Examples: CueFlik(CHI2008)
Positive examples
Negative examples

4. Department of Computer Science
Examples: CueT(CHI2011)
•Goal: learns from the triaging decisions of operators and help them quickly and accurately triage alarms in a highly dynamic environment.
•Workflow: constantly updating and learning: alarms arrive, CueT provides recommendations and operators make decisions and CueT learns from it.
•ML Model: Nearest Neighbor; Adaptive similarity measure (online metric learning).
•Specialties: When to generate a new ticket(class)

5. Department of Computer Science
Examples: CueT(CHI2011)
Alarm view
Ticket view
Severity
Similarity visualization

6. Department of Computer Science
Examples: Apolo(CHI2011)
•Goal: guides the user to incrementally and interactively explore large network data(demonstrated by scientific literature).
•Workflow: iterative exploration: browse examples -> create groups(labels) -> label examples -> browse new examples…
•ML Model: belief propagation.
•Specialties: None.

7. Department of Computer Science
Examples: Apolo(CHI2011)

8. Department of Computer Science
Examples: Info. Extraction(CHI2009)
•Goal: proposes a novel synergistic method for jointly amplifying community content creation and learning based information extraction.
•Workflow: automatically extracts potential infobox and invites visitors to manually examined and explicitly confirmed it.
•ML Model: Doc/Sentence level classifiers; Information Extraction(CRF).
•Specialties: None.

9. Department of Computer Science
Examples: Info. Extraction(CHI2009)
info.
box
dialog confirmation

10. Department of Computer Science
Examples: EnsembleMatrix(CHI2009)
•Goal: presents a graphical view of confusion matrices to help users understand relative merits of various classifiers and helps users build combination multiclass classifiers.
•Workflow: users can directly manipulate the visualizations in order to build combination multiclass classifiers(Partition and Reweight).
•ML Model: ensemble classifier.
•Specialties: none.

11. Department of Computer Science
Examples: EnsembleMatrix(CHI2009)
Main ensemble classifier
each classifier
linear combination widget
Heat map
representation

12. Department of Computer Science
Examples: ManiMatrix(CHI2010)
•Goal: provides controls and visualizations that enable system builders to refine the behavior of classification systems in an intuitive manner.
•Workflow: users refine parameters of a confusion matrix (modify decision boundaries of classifiers) via an interactive cycle of reclassification and visualization.
•ML Model: assigning sets of costs of misclassification in a confusion matrix. Costs of misclassification are non-uniform across classes.
•Specialties: decision-theoretic analysis(푅푖푠푘푗= 푝푖⋅퐶표푠푡푖푗 푐푖 =1)

13. Department of Computer Science
Examples: ManiMatrix(CHI2010)
6 cloudy days were misclassified as rainy
changes after user’s click
lock the cell with a desired direction
desired direction of the value changes (increasing)

14. Department of Computer Science
Examples: ReGroup(CHI2012)
•Goal: helps people create custom, on-demand groups in online social networks.
•Workflow: ReGroup observes a person’s normal interaction of adding members to a group, it learns a probabilistic model of group membership in order to suggest both additional members and group characteristics for filtering a friend list. It continually update its membership model based on interactive user feedback.
•ML Model: Naïve Bayes(probability of each friend being a member of the group). Re-trained every time a person adds friends to a group.
•Specialties: obtain implicit negative examples; unlearnable groups; missing data

15. Department of Computer Science
Examples: ReGroup(CHI2012)
Top 5 relevant group characteristics
Relevant friend suggestions
A static, hierarchical list of all feature value filters

16. Department of Computer Science
Examples: Visual-FSSEM(KDD2000)
•Goal: guides the feature selection procedures and enable a deeper understanding of the data in unsupervised setting.
•Workflow: performs a greedy sequential forward or backward search for features as measured by the chosen performance criterion(scatter separability, maximum likelihood, cluster entropy and probability of error) Users can select initial feature subset and limit the pool features to search from. The user selects the direction forward or backward in every step and the number of steps.
•ML Model: LDA(linear discriminant analysis) for visualization; mixture multivariate Gaussians for clustering.
•Specialties: EM local minima;