IntroToML_Lecture1.pdf

Introduction to
Machine Learning
Winter Term 2019/20
Markus Schedl, Günter Klambauer
markus.schedl@jku.at, klambauer@ml.jku.at
http://www.cp.jku.at, http://www.bioinf.jku.at

Introduction to Machine Learning
© 2019 Markus Schedl 2
Schedule
May be subject to change; for latest information, have a look at KUSSS

What is Machine Learning?
• Term “Machine Learning” coined by Arthur Samuel (1959):
“How can computers learn to solve problems without
being explicitly programmed?”
• Tom M. Mitchell provided a more formal definition (1997):
“A computer program is said to learn from experience E
with respect to some class of tasks T and performance
measure P if its performance at tasks in T, as measured
by P, improves with experience E.”
→ continuous improvement as important characteristics
• Nowadays, ML techniques/algorithms are omnipresent, albeit
not always visible.

Important and Current Areas of Application
• Automatic machine translation (from text and image), e.g.:
https://translate.google.com
https://ai.googleblog.com/2015/07/how-google-translate-squeezes-deep.html

• Automatic machine translation (from text and image), e.g.:
https://translate.google.com (Google)
https://ai.googleblog.com/2015/07/how-google-translate-squeezes-deep.html

• Semantic image tagging:
https://www.cv-
foundation.org/openaccess/content_iccv_2015/papers/Fu_Relaxing_From_V
ocabulary_ICCV_2015_paper.pdf (Microsoft)

• Semantic music video tagging:
http://www.ifs.tuwien.ac.at/~schindler/pubs/ACMTIST2016.pdf

• Self-driving cars (autonomous vehicles):
https://www.zdnet.com/article/dossier-the-leaders-in-self-driving-cars
https://www.navigantresearch.com/reports/navigant-research-
leaderboard-automated-driving-vehicles

• Playing (computer) games: winning Chess, Go, Arcade
games, etc. against best human champions
(IBM Watson, Google Deepmind, etc.)
https://deepmind.com/research/dqn
https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf

• Playing (computer) games: winning Chess, Go, etc. against
best human champions
https://deepmind.com/research/dqn
https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf

• Text-to-speech (e.g., Google Wavnet):
https://deepmind.com/blog/wavenet-generative-model-raw-audio
Previous Wavnet
• Facial expression/emotion detection:
https://imotions.com/biosensor/fea-facial-expression-analysis
• Face detection, recognition, and classification:
https://azure.microsoft.com/en-us/services/cognitive-
services/face
• Age detection:
https://www.how-old.net
• Object classification from hand-drawings:
https://quickdraw.withgoogle.com

Arts (e.g., automatic generation of music and images)
• Singing synthesis (e.g., using as input MIDI + lyrics):
http://www.dtic.upf.edu/~mblaauw/NPSS
• Music “composition” (e.g., Flow Machine by Sony CSL):
• Reorchestration of the EU Anthem in various music styles
(by Sony CSL, Paris): https://www.youtube.com/watch?v=0qnTaAz-
xtQ&list=PLuOoXrWK6Kz5ySULxGMtAUdZEg9SkXDoq
Female Japanese singer
“Daddy’s Car” in the style of The Beatles

Arts (e.g., automatic generation of text and images)
• Automatic handwriting: https://arxiv.org/abs/1308.0850
• Fake research (math) papers:
https://cs.stanford.edu/people/jcjohns/fake-math/4.pdf
• Fake Wikipedia articles, source code, Shakespeare sonnets, etc.:
http://karpathy.github.io/2015/05/21/rnn-effectiveness

Arts (e.g., automatic generation of text and images)
• Learning style of a painter and transferring it to other images
https://www.cv-
foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_
Transfer_CVPR_2016_paper.pdf
https://deepart.io (try with your own photos)
https://deepdreamgenerator.com (Deep Dream generator)
https://github.com/luanfujun/deep-photo-
styletransfer/blob/master/README.md (Deep photo style transfer)

Learning style of a painter and transferring it to other images:
Mona Lisa restyled by…
Picasso van Gogh Monet
Cubist Expressionist Impressionist

Learning style from training images and transferring it to other
images: Mona Lisa restyled by…
Hieroglyphs Crab Nebula Google Maps

And many more…
• Spam/junk mail detection
• Healthcare (e.g., diagnosis of cancer)
• Recommender systems (preference learning, rating prediction)
• Conversational user interfaces (voice assistants or chat bots)
e.g., Alexa, Siri, Google Assistant, various car manufacturers)
• Predictive analytics, e.g.:
• predicting if/when a natural disaster will happen
• predicting outcome of an election
• Analyzing causal relationships (e.g. from time series data,
predict whether elections, product launches, disasters, etc.
influence people’s purchasing/consumption behavior; stock
market prediction)
• Automatic music playlist generation/continuation (sequential)
• Lethal autonomous weapons
Even more examples (in German): https://heise.de/-4537812

Categorization of ML Techniques
Supervised learning:
Labeled training data to learn from is given, i.e. data
items (features) come with semantic information
(typically classes or continuous output values)
Goal: learn a rule/function that maps input (data items) to output
(classes or values)
Unsupervised learning:
No semantic class labels (or continuous values) for data items to
learn from are given, i.e. desired output is unknown
Goal: discover structure in the data, detect similar
groups (clusters) of data items

Categorization of ML Techniques
Reinforcement learning:
An agent acts in an unknown environment, trying to gather
knowledge and learn from interactions with the environment,
maximizing some reward (e.g., playing games, autonomous driving)

Supervised Learning: More Formally
A model is learned from labeled (training) data, which can then be used to
predict the outcome for unseen (test) data.
= { , … , } data set ( s are typically vectors)
Classification: C set of classes
A classifier learns a function: → or =
Ex.: Predicting month from weather data
{location=Austria, temp=35°, sunny=yes, rain=no} → month=August
{location=Spain, temp=15°, sunny=no, rain=yes} → month=December
{location=Sweden, temp=-7°, sunny=no, rain=no} → month=???
Binary classifier: Decision is whether an unseen instance belongs to the
(only) class or not, e.g., = {!"#, $%}

Supervised Learning: More Formally
A model is learned from labeled (training) data, which can then be used to
predict the outcome for unseen (test) data.
= { , … , } data set ( s are typically vectors)
Regression: Y continuous range ℝ
A regression algorithm learns a function: → ! or = + ∈ ℝ
Ex.: Predicting personality trait “neuroticism” in [1, 5] for users from their
social media footprints [Skowron et al., WWW 2016]
{image_color=red, #followers=317, text_sentiment=-0.9} → neurot=4.7
{image_color=green, #followers=3, text_sentiment=0.65} → neurot=2.5
{image_color=green, #followers=42, text_sentiment=0.8} → neurot=?

Other Popular Supervised Variants
Semi-supervised learning:
Labeled and (usually many more) unlabeled examples are given as
input to build a learner/classifier; from unlabeled instances learner
can learn patterns in the data
Active learning:
Special case of semi-supervised learning: user is asked to provide
labels interactively (e.g. for data items on which the current model is
most uncertain as to which class they belong to)

Unsupervised Learning
Clustering:
Unlabeled input data is automatically categorized into a (typically
unknown) number of clusters (groups of similar items), without
knowing clusters beforehand (for automatic organization of data
sets)
Dimensionality reduction:
Typically high-dimensional input data (feature vectors of >> 100
dimensions) is mapped into a lower-dimensional space such that
information loss is minimized (e.g., recommender systems, topic
modeling, data visualization, data compression)

Examples From Our Own Research
Examples from the Institute of Computational Perception that
make use of machine learning:
• Intelligent iPod (automatic playlist generation)
• Automatic Page Turner (musical score following)
• ROBOD (real-time beat tracking)
• nepTune (music browsing interface)
• Predicting user characteristics from social media traces

Intelligent organization of music
for “one-touch access”
music collections become larger
and larger (on PCs as well as on
mobile players)
most UIs of music players still
only allow organization and
searching by textual properties
accoding to scheme
(genre-)artist-album-track
→ novel and innovative strategies
to access music are sought in MIR
Intelligent iPod

Task:
automatic music playlist generation
Method:
1. audio feature extraction
(rhythmic descriptors)
2. similarity/distance estimation
between these features (i.e. the
respective songs)
3. determining a path that passes
all songs in the collection, while
minimizing path length (overall
distance)
4. visualize this path in an intuitive
way and provide user with
means of navigation
Intelligent iPod

(Arzt, Widmer; 2010)
Automatic Page Turner
Task:
musical score following
Method:
1. computer “follows” what a
pianist is playing
2. constantly matches the
played notes with the
notes given in the score
sheet
3. triggers an actor to turn
the page when reaching
the end of a score sheet

(Arzt, Widmer; 2010)
ROBOD
Task:
real-time beat tracking
Finalist of the IEEE Signal Processing Cup 2017
https://www.youtube.com/watch?v=21X9SWqOgmw
https://www.youtube.com/watch?v=f7FoDmUWEDU
https://www.youtube.com/watch?v=sjkZg8bvMWw

Task:
cluster music collections
Method:
1.extract audio features
2.train an unsupervised learner
(clustering algorithm) from these
features
3.visualize clusters by density
estimation and computing a
height map (number of pieces
per cluster → height)
4.create an artificial landscape
the user can navigate through
nepTune: Intelligent Music Browsing Interface
http://www.cp.jku.at/projects/nepTune

nepTune: Different Modes
meta-data (artist and song)
descriptive terms mined from the web
images gathered from the web

• Exploit digital traces from multiple social media
sites to predict personality traits
• Modeling user traces: visual features (Instagram), linguistic
features (Instagram and Twitter), and metadata features (Twitter)
• Modeling personality traits: “Big Five Inventory” (BFI): Openness,
Conscientiousness, Extraversion, Agreeableness,
and Neuroticism (OCEAN)
• Ground truth (on which classifier/regressor is trained): 44-item BFI
personality questionnaire with quality assurance, OCEAN value
range: [1…5]
[Skowron et al., 2016]: Skowron, M., Ferwerda, B., Tkalčič, M., and Schedl, M. Fusing Social Media
Cues: Personality Prediction from Twitter and Instagram, Proceedings of the 25th International
World Wide Web Conference (WWW), Montreal, Canada, April 2016.
Predicting Personality from Social Media Data

Predicting Personality from Social Media Data
• Visual features (Instagram):
- Brightness, saturation
- Pleasure-Arousal-Dominance (PAD) → emotion
- Content-based features, e.g., presence of face or full body
• Linguistic features (Twitter and Instagram):
- NLP for sentiment analysis: LIWC, ANEW, etc.
- Pre-trained classifiers: dialog acts (e.g., question,
statement, greeting), sentiment
• Metadata features (Twitter):
- User reputation and influence scores: e.g., number of followers
and followees, Klout and adaptation of TIME influence scores
[Skowron et al., 2016]: Skowron, M., Ferwerda, B., Tkalčič, M., and Schedl, M. Fusing Social Media
Cues: Personality Prediction from Twitter and Instagram, Proceedings of the 25th International
World Wide Web Conference (WWW), Montreal, Canada, April 2016.

Personality Prediction: Results from Meta Study
[Azucar et al., 2018]: Azucar, D., Marengo, D., and Settanni, M. Predicting the Big 5 personality
traits from digital footprints on social media: A meta-analysis, Personality and Individual
Differences, 124:150-159, 2018.

[Azucar et al., 2018]: Azucar, D., Marengo, D., and Settanni, M. Predicting the Big 5 personality
traits from digital footprints on social media: A meta-analysis, Personality and Individual
Differences, 124:150-159, 2018.
Infer personality traits: Results compared in meta study
Skowron et al., 2016
Forest plot
of effect
sizes
(correlations)

IntroToML_Lecture1.pdf

Recommended

Recommended

More Related Content

Similar to IntroToML_Lecture1.pdf

Similar to IntroToML_Lecture1.pdf (20)

Recently uploaded

Recently uploaded (20)

IntroToML_Lecture1.pdf