SlideShare a Scribd company logo

Dhn2018-A Study on Word2Vec on a Historical Swedish Newspaper Corpus

N
N
Nina TahmasebiAssociate Professor in NLP

Presentation from The Di­gital Hu­man­it­ies in the Nor­dic Countries 3rd Conference (DHN2018)

Dhn2018-A Study on Word2Vec on a Historical Swedish Newspaper Corpus

1 of 26
A Study on Word2Vec on a
Historical Swedish
Newspaper Corpus
Nina Tahmasebi, PhD
University of Gothenburg
DHN2018
2018-03-08
What is new?
time
Background
Increasing amount of historical texts
in digital format
Easy digital access for anyone!
Not only scholars.
Possibility to digitally analyze
historical documents
at large scale.
Information from primary resources
Not only modern interpretations.
Culturomics:
Tracking the development of
cultural and language
phenomena over time.
E.g., topics, opinions …
time
LanguageChange
• Spelling variations
• Diachronic word replacement (w2w)
Teutschland Deutschlandfelitious happy
Background
time
LanguageChange
• Spelling variations
• Diachronic word replacement (w2w)
• Named entity change
Petrograd St. Petersburg
Background
LanguageChange
• Spelling variations
• Diachronic word replacement (w2w)
• Named entity change
• Word sense change
He was an awesome leader
time
Background
time
LanguageChange
• Spelling variations
• Diachronic word replacement (w2w)
• Named entity change
• Word sense change
• General LanguageChange
Background
Kona  Qwinna  Qvinna  Kvinna
Ad

Recommended

Tartu-DHtalk-final.pdf
Tartu-DHtalk-final.pdfTartu-DHtalk-final.pdf
Tartu-DHtalk-final.pdfNina Tahmasebi
 
2022-10-18-KBR-for publication.pdf
2022-10-18-KBR-for publication.pdf2022-10-18-KBR-for publication.pdf
2022-10-18-KBR-for publication.pdfNina Tahmasebi
 
2020 10-26-language change-stuttgart-workshop
2020 10-26-language change-stuttgart-workshop2020 10-26-language change-stuttgart-workshop
2020 10-26-language change-stuttgart-workshopNina Tahmasebi
 
2020 09-28-odense-final-forpublication
2020 09-28-odense-final-forpublication2020 09-28-odense-final-forpublication
2020 09-28-odense-final-forpublicationNina Tahmasebi
 
Workshop on Digital Literacy - Digital text and data-intensive research
Workshop on Digital Literacy - Digital text and data-intensive researchWorkshop on Digital Literacy - Digital text and data-intensive research
Workshop on Digital Literacy - Digital text and data-intensive researchNina Tahmasebi
 
Detecting language change for the digital humanities
Detecting language change for the digital humanitiesDetecting language change for the digital humanities
Detecting language change for the digital humanitiesNina Tahmasebi
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024Albert Qian
 

More Related Content

Recently uploaded

ROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsx
ROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsxROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsx
ROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsxAnkitChoudhary955647
 
Analytical Coursework - Molly Winterbottom.pdf
Analytical Coursework - Molly Winterbottom.pdfAnalytical Coursework - Molly Winterbottom.pdf
Analytical Coursework - Molly Winterbottom.pdfMollyWinterbottom
 
Open Access Publishing and the Open Journal of Astrophysics
Open Access Publishing and the Open Journal of AstrophysicsOpen Access Publishing and the Open Journal of Astrophysics
Open Access Publishing and the Open Journal of AstrophysicsPeter Coles
 
Volatile Oils-Introduction for pharmacy students and graduates
Volatile Oils-Introduction for pharmacy students and graduatesVolatile Oils-Introduction for pharmacy students and graduates
Volatile Oils-Introduction for pharmacy students and graduatesAhmed Metwaly
 
A galactic microquasar mimicking winged radio galaxies
A galactic microquasar mimicking winged radio galaxiesA galactic microquasar mimicking winged radio galaxies
A galactic microquasar mimicking winged radio galaxiesSérgio Sacani
 
Quality safety and legislations of cosmetics.pptx
Quality safety and legislations of cosmetics.pptxQuality safety and legislations of cosmetics.pptx
Quality safety and legislations of cosmetics.pptxDeviSky1
 
Earth and Planetary Science | Volume 01 | Issue 01 | April 2022
Earth and Planetary Science | Volume 01 | Issue 01 | April 2022Earth and Planetary Science | Volume 01 | Issue 01 | April 2022
Earth and Planetary Science | Volume 01 | Issue 01 | April 2022Nan Yang Academy of Sciences
 
From Leaf to Lab: Uncovering the Molecular Mysteries of Cannabis
From Leaf to Lab: Uncovering the Molecular Mysteries of CannabisFrom Leaf to Lab: Uncovering the Molecular Mysteries of Cannabis
From Leaf to Lab: Uncovering the Molecular Mysteries of CannabisMarkus Roggen
 
Planeta 9 - A Pan-STARRS1 Search for Planet Nine
Planeta 9 - A Pan-STARRS1 Search for Planet NinePlaneta 9 - A Pan-STARRS1 Search for Planet Nine
Planeta 9 - A Pan-STARRS1 Search for Planet NineSérgio Sacani
 
Thornyissue testing of slideshow for website
Thornyissue testing of slideshow for websiteThornyissue testing of slideshow for website
Thornyissue testing of slideshow for websitesuelcarter1
 
Elbow joint - Anatomy of the Elbow joint
Elbow joint - Anatomy of the Elbow jointElbow joint - Anatomy of the Elbow joint
Elbow joint - Anatomy of the Elbow jointTELISHA2
 
Duchenne Muscular Dystrophy or DMD .pptx
Duchenne Muscular Dystrophy or DMD .pptxDuchenne Muscular Dystrophy or DMD .pptx
Duchenne Muscular Dystrophy or DMD .pptxNavanidhan.M
 
Physics Chapter Three - Electric Fields and Charges
Physics Chapter Three - Electric Fields and ChargesPhysics Chapter Three - Electric Fields and Charges
Physics Chapter Three - Electric Fields and Chargesalinford
 
IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1
IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1
IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1Sérgio Sacani
 
The ExoGRAVITY project - observations of exoplanets from the ground with opti...
The ExoGRAVITY project - observations of exoplanets from the ground with opti...The ExoGRAVITY project - observations of exoplanets from the ground with opti...
The ExoGRAVITY project - observations of exoplanets from the ground with opti...Advanced-Concepts-Team
 
LIGHT Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...
LIGHT  Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...LIGHT  Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...
LIGHT Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...Abhinav S
 
Study of X - Ray Spectra and its types
Study  of X  - Ray Spectra and its typesStudy  of X  - Ray Spectra and its types
Study of X - Ray Spectra and its typestanishashukla147
 

Recently uploaded (20)

ROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsx
ROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsxROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsx
ROLES OF MICROBES IN BIOCONTROL BY ANKIT CHOUDHARY.ppsx
 
Analytical Coursework - Molly Winterbottom.pdf
Analytical Coursework - Molly Winterbottom.pdfAnalytical Coursework - Molly Winterbottom.pdf
Analytical Coursework - Molly Winterbottom.pdf
 
Open Access Publishing and the Open Journal of Astrophysics
Open Access Publishing and the Open Journal of AstrophysicsOpen Access Publishing and the Open Journal of Astrophysics
Open Access Publishing and the Open Journal of Astrophysics
 
Research methods in ethnobotany- Exploring Traditional Wisdom
Research methods in ethnobotany- Exploring Traditional WisdomResearch methods in ethnobotany- Exploring Traditional Wisdom
Research methods in ethnobotany- Exploring Traditional Wisdom
 
Volatile Oils-Introduction for pharmacy students and graduates
Volatile Oils-Introduction for pharmacy students and graduatesVolatile Oils-Introduction for pharmacy students and graduates
Volatile Oils-Introduction for pharmacy students and graduates
 
A galactic microquasar mimicking winged radio galaxies
A galactic microquasar mimicking winged radio galaxiesA galactic microquasar mimicking winged radio galaxies
A galactic microquasar mimicking winged radio galaxies
 
Quality safety and legislations of cosmetics.pptx
Quality safety and legislations of cosmetics.pptxQuality safety and legislations of cosmetics.pptx
Quality safety and legislations of cosmetics.pptx
 
Earth and Planetary Science | Volume 01 | Issue 01 | April 2022
Earth and Planetary Science | Volume 01 | Issue 01 | April 2022Earth and Planetary Science | Volume 01 | Issue 01 | April 2022
Earth and Planetary Science | Volume 01 | Issue 01 | April 2022
 
From Leaf to Lab: Uncovering the Molecular Mysteries of Cannabis
From Leaf to Lab: Uncovering the Molecular Mysteries of CannabisFrom Leaf to Lab: Uncovering the Molecular Mysteries of Cannabis
From Leaf to Lab: Uncovering the Molecular Mysteries of Cannabis
 
Planeta 9 - A Pan-STARRS1 Search for Planet Nine
Planeta 9 - A Pan-STARRS1 Search for Planet NinePlaneta 9 - A Pan-STARRS1 Search for Planet Nine
Planeta 9 - A Pan-STARRS1 Search for Planet Nine
 
Thornyissue testing of slideshow for website
Thornyissue testing of slideshow for websiteThornyissue testing of slideshow for website
Thornyissue testing of slideshow for website
 
INTRODUCTION TO PLANT TAXONOMY WITH DIVERSE TAXONOMIC APPROACHES
INTRODUCTION TO PLANT TAXONOMY WITH DIVERSE TAXONOMIC APPROACHESINTRODUCTION TO PLANT TAXONOMY WITH DIVERSE TAXONOMIC APPROACHES
INTRODUCTION TO PLANT TAXONOMY WITH DIVERSE TAXONOMIC APPROACHES
 
Elbow joint - Anatomy of the Elbow joint
Elbow joint - Anatomy of the Elbow jointElbow joint - Anatomy of the Elbow joint
Elbow joint - Anatomy of the Elbow joint
 
Duchenne Muscular Dystrophy or DMD .pptx
Duchenne Muscular Dystrophy or DMD .pptxDuchenne Muscular Dystrophy or DMD .pptx
Duchenne Muscular Dystrophy or DMD .pptx
 
Physics Chapter Three - Electric Fields and Charges
Physics Chapter Three - Electric Fields and ChargesPhysics Chapter Three - Electric Fields and Charges
Physics Chapter Three - Electric Fields and Charges
 
IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1
IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1
IM-1 Press Kit - Kit de Imprensa do Lançmento da Missão IM-1
 
The ExoGRAVITY project - observations of exoplanets from the ground with opti...
The ExoGRAVITY project - observations of exoplanets from the ground with opti...The ExoGRAVITY project - observations of exoplanets from the ground with opti...
The ExoGRAVITY project - observations of exoplanets from the ground with opti...
 
LIGHT Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...
LIGHT  Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...LIGHT  Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...
LIGHT Community Medicine LIGHT IS A SOURCE OF ENERGY THERE ARE TWO TYPE OF S...
 
VEM 023- LESSON 1.pdf
VEM 023- LESSON 1.pdfVEM 023- LESSON 1.pdf
VEM 023- LESSON 1.pdf
 
Study of X - Ray Spectra and its types
Study  of X  - Ray Spectra and its typesStudy  of X  - Ray Spectra and its types
Study of X - Ray Spectra and its types
 

Featured

Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsKurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at WorkGetSmarter
 
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...DevGAMM Conference
 
Barbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationErica Santiago
 
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellSaba Software
 
Introduction to C Programming Language
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming LanguageSimplilearn
 
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...Palo Alto Software
 

Featured (20)

Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
 
ChatGPT webinar slides
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slides
 
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike RoutesMore than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
 
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
 
Barbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy Presentation
 
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
 
Introduction to C Programming Language
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming Language
 
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
 

Dhn2018-A Study on Word2Vec on a Historical Swedish Newspaper Corpus

  • 1. A Study on Word2Vec on a Historical Swedish Newspaper Corpus Nina Tahmasebi, PhD University of Gothenburg DHN2018 2018-03-08
  • 2. What is new? time Background Increasing amount of historical texts in digital format Easy digital access for anyone! Not only scholars. Possibility to digitally analyze historical documents at large scale. Information from primary resources Not only modern interpretations. Culturomics: Tracking the development of cultural and language phenomena over time. E.g., topics, opinions …
  • 3. time LanguageChange • Spelling variations • Diachronic word replacement (w2w) Teutschland Deutschlandfelitious happy Background
  • 4. time LanguageChange • Spelling variations • Diachronic word replacement (w2w) • Named entity change Petrograd St. Petersburg Background
  • 5. LanguageChange • Spelling variations • Diachronic word replacement (w2w) • Named entity change • Word sense change He was an awesome leader time Background
  • 6. time LanguageChange • Spelling variations • Diachronic word replacement (w2w) • Named entity change • Word sense change • General LanguageChange Background Kona  Qwinna  Qvinna  Kvinna
  • 7. St. Petersburg What is the problem? 1500 1525 1550 1575 1600 1625 1650 1675 1700 1725 1750 1775 1800 1825 1850 1875 1900 1925 1950 1975 2000 time Petrograd Finding relevant content • Spelling variations, named entity change Interpreting content • Word sense evolution, term to term evolution Aligning e.g., topics or opinions across time E.g., ”Iphone 5 is awesome”  positive or negative review?
  • 8. What is the problem? Finding relevant content • Spelling variations, named entity change Interpreting content • Word sense change, diachronic word replacements Aligning e.g., topics or opinions across time E.g., ”Iphone 5 is awesome”  positive or negative review? 1500 1525 1550 1575 1600 1625 1650 1675 1700 1725 1750 1775 1800 1825 1850 1875 1900 1925 1950 1975 2000 time St. Petersburg
  • 9. ”Sestini’s benefit last night at the Opera House was overflowing with the fashionable and gay” The Problem – Interpreting
  • 10. The Problem – Interpreting
  • 11. The Problem – Interpreting ”Sebastini’s benefit last night at the Opera House was overflowing with the fashionable and gay” TheTimes, April 27th, 1787
  • 12. Aims To find word sense changes automatically by 1. Modeling word senses 2. Comparing these over time To find what changes, how it changed and when it changed
  • 13. We are not the first, nor the last • Context vectors • Topic models • Graph-based methods • Word embedding methods context vectors ti tj w The meanings of words are not fixed but in fact undergo change The meanings of words are not fixed but in fact undergo change Finally, we conduct a preliminary evaluation in which we apply our methods to the task of Finally, we conduct a preliminary evaluation in which we apply our methods to the task of BNC ukWaC
  • 14. Word embeddings W(‘‘woman")−W(‘‘man") ≃ W(‘‘aunt")−W(‘‘uncle") W(‘‘woman")−W(‘‘man") ≃ W(‘‘queen")−W(‘‘king“) Word embeddings shown in 2D instead of 50-100000 Image: Nieto Pina and Johansson, RANLP’15 Iraq - Violence = Jordan Human - Animal = Ethics President - Power = Prime Minister Library - Books = Hall
  • 15. Word embedding-based models Kulkarni et al. WWW’15 • Project a word onto a vector/point (POS, frequency and embeddings) • Track vectors over time Kim et al. LACSS 2014 Basile et al. CLiC-it 2016 Zhang et al. WWW’16 Hamilton et al. ACL 2016 Bamler and Mandt ICML 2017 Image: Kulkarni et al. WWW’15
  • 16. Downsides with word embeddings 1. Random in • Initialization • Order in which the training examples are seen 2. 100 Million tokens per time span 3. Typically learn one vector per word Stable/less dominant senses get lost! Stone Music Lifestyle Rock
  • 17. Our study • Word2Vec (W2V) • a two-layer neural net (skip-gram) out-of-the-box using the Deeplearning4j (DL4J) for Java • Kubhist • Swedish Newspapers • 1749-1925 • Trained yearly vectors - 2000 000 4000 000 6000 000 8000 000 10000 000 12000 000 14000 000 16000 000 1749 1757 1779 1787 1795 1803 1811 1819 1827 1835 1843 1851 1859 1867 1875 1883 1891 1899 1907 1915 1923 Numberoftokens Year Size of Kubhist in tokens tokens
  • 18. What did we do? • 11 (10) words over time nyhet 'news', tidning 'newspaper', politik 'politics', telefon 'telephone', telegraf 'telegraph', kvinna 'woman', man 'man', glad, 'happy', retorik 'rethoric', resa 'travel' and musik 'music'. A = {happy, smiling, glad} B = {happy, joyful, cheerful, excited} Overlap = 1 Unique = 3+4-1 = 6 Jaccard similarity = 1/6
  • 19. Result summary Avg. Jaccard similarity, normalized frequency and Spearman correlation
  • 20. Study on W2V for Kubhist • The more frequent the term, the more stable the vectors • 0.11-0.19 overlap between years  2-3 words in common each year 1 word  jacc = 0.05 2 words  jacc = 0.11 3 words  jacc = 0.18 4 words  jacc = 0.25
  • 21. Some Swedish results Women: 1912: 'kvinna': [valbarhet, valrätt, rösträtt, själfförsörjande, sexuell, okunnig, högerparti, politisk, radikal, vänsterparti] 1908: 'kvinna': [österåsen, ung, rösträtt, ljusglimt, flicka, iförda, knäböjande, begåfvad, värnlös, jubla] 1895: 'kvinna': [qvinna, varelse, människa, öfvermåttan, flicka, reptil, gosse, förälskade, öfvergifven, högväxt] 1879: 'kvinna': [qvarlefva, vålnad, öfvade, rättskaffens, begåfvade, skenbart, skummande, vilde, herskar, mygga] 1867: 'kvinna': [äes, kvrk, kunäe, mle, näo, nuvaranäe, äer, v«r«, uä, äig] 1868: 'kvinna': [piller, kvilken, mis, kade, klo, nde, äock, reäan, äsom, bvilken]
  • 22. Some Swedish results Politics: 1925: 'politik': [näring, trygghet, kamp, arbetarrörelse, konservativ, nationell, strävan, europa, neutralitet, önskad] 1922: 'politik': [åskådning, socialistisk, ägnad, demokrati, utrikespolitisk, sakligt, situation, representativ, auktoritet, ärlig] 1900: 'politik': [enig, bvad, finlands, politisk, konstitutionel, revolution, armenien, citera, civiliserade, dementi] 1872: 'politik': [republikansk, opposition, kränka, reaktionär, neutral, republikan, tillbakavisa, changarniers, påfvedöme, hora 1858: 'politik': [asylrätt, allians, frankrikes, konstitutionell, konflikt, försonlig, rysslands, press, makt, fördrag] 1844: 'politik': [tadla, allians, vägran, irländsk, frankrikes, bemedling, tribun, segra, ministeriell, fördrag]
  • 23. Next step • OCR errors • Spelling normalization utbetalt34 1 utbetalta 15 utbetaltu 1 utbe¬ 2 utbfjudes 1 utbftte 1 utbi 1 utbiJdning 1 utbiedd 1 utbiedning 2 breflådör 1 breflåifas 1 breflörsändning 1 breflösen 5 brefmassan 1 brefmottagningsställen 1 brefmärken 4 brefmärkena 2
  • 24. Current and Future work • Test neural word embeddings on Kubhist • Correct OCR errors • Spelling variations • Find diachronic word replacements • Handikappad  funktionsnedsatt  funktionshindrad • Sense-based embeddings for word sense change
  • 25. Thank you! Nina Tahmasebi, PhD Språkbanken (The Swedish Language Bank), Center for Digital humanities University of Gothenburg Nina.tahmasebi@GU.se
  • 26. Algorithm Overview Step 1: Word sense discr. (curvature clustering) individual time slices O(|S|T) Step 2: Detecting stable senses  units Step 3: Relating units Paths Stone Music Lifestyle Rock Tahmasebi & Risse, RANLP2017