Many times what your machine learning algorithm is optimizing for is not what you really want. Worse, many times you do not really know what you want. In interactive machine learning the human is an integral part of the learning process leading to cooperative human-machine solutions. In this talk, we will see a few such problems from the field of Natural Language Processing (NLP), their challenges, common pitfalls and some solutions.
Where f typically such that
𝑓 = argmin 𝑓∈𝐹
𝐿 𝑓 𝑥𝑖 , 𝑦𝑖 + 𝜆𝑅 𝑓
I know what I want
(and can formalize it)
I have time & money to label lots of data
Example: Machine Translation
Given a text s and its proposed translation p, how to measure its distance with
respect to a reference translation t ?
BLEU: n-gram overlap between t and p
typically: 1 ≤ 𝑛 ≤ 4, precision only, brevity penalty
bonus points for matching stems and synonyms
Statistical Machine Translation
Consequences of not formalizing correctly
Users do not use your model
Computer-Assisted Translation used rule-based systems for years
Automatic Post Edition
Where Z(X) capture some prior:
I am not sure what I want
I have a (big) corpus with assumed patterns
Example: Exploratory Search
Whenever your task is:
– Broad / under-specified
– Searcher’s understanding inadequate at the beginning
– Searcher’s understanding evolves as results are gradually retrieved.
The answer to what you search is “I know it when I see it”
Exploratory Search: examples
• Vo, Ngoc Phuoc An, et al. "DISCO: A System Leveraging Semantic Search in Document Review." COLING (Demos). 2016.
• Privault, Caroline, et al. "A new tangible user interface for machine learning document review." Artificial Intelligence and Law 18.4 (2010): 459-479.
• Ferrero, Germán, Audi Primadhanty, and Ariadna Quattoni. "InToEventS: An Interactive Toolkit for Discovering and Building Event Schemas." EACL 2017 (2017): 104.
Example: Active Learning
Give initiative to the algorithm
allow action of type: “please, label instance x”
Cognitive effort of labeling a document 3-5x higher than labelling a word 
• type(feedback) ≠ type(label)
• information load of a word label is small
• word sense disambiguation
 Raghavan, Hema, Omid Madani, and Rosie Jones. "Active learning with feedback on features and instances." Journal of
Machine Learning Research7.Aug (2006): 1655-1686.
If you really want to solve a problem, don’t be prisoner of your
1. Does it really capture success?
does it align with human judgment?
2. What does the [machine | human] best?
3. Can you remove the burden from humans by smarter algorithms?