SlideShare a Scribd company logo
Mining knowledge in news
for personalization
Dung Chu and David Graus,
data scientists
FD Mediagroep
Outline
• Introduction FDMG
• AI use cases:
– FD personalized newsletter
– CompanyInfo entity linker
– Smart Radio
FD mediagroep
Het Financieele Dagblad
Het Financieele Dagblad is dé nieuws- en inspiratiebron die op elk
moment van de dag financieel-economische betekenis geeft aan
ontwikkelingen in de wereld.
Company info geeft altijd real time toegang tot actuele bedrijfs- en
prospectinformatie van 2,5 miljoen bedrijven in Nederland
FD/BNR Networks
FD/BNR Networks brengt ideeën, meningen en talentrijke
ondernemers en professionals samen door de organisatie van o.a.
forums, debatten, netwerksessies, en evenementenreeksen.
BNR Nieuwsradio
BNR Nieuwsradio is de enige radiozender in Nederland waar
ambitieuze en ondernemende mensen 24 uur per dag op de hoogte
worden gehouden van relevant nieuws.
FD Mediagroep helpt om betere beslissingen te nemen door overzicht, inzicht én
(vooruit)zicht te bieden met journalistieke merken en hoogstaande kwalitatieve producten.
FD personalized newsletter
FD personalized newsletter: goals
• Diversify content sent to readers
• Improve engagement
FD personalized newsletters
• Rich data:
– Logged-in data with unique client ID
• Recommendation model: Item-to-item collaborative
filtering
Published
last week
Older
articles
PySpark:
+ Convert reader-article matrix to IndexedRowMatrix
+ columnSimilarities(): fast compute of cosine
similarity between columns
Results: free clients
T-test
Results: paid clients (fixed 2000)
T-test
Content diversification
Popularity emails
Personalized emails
7 articles 178 articles (60 clicked)
Long tail
articles
25000
15000
CompanyInfo entity linker
Ref: https://www.slideshare.net/davidgraus/financial-news-mining-pydata-amsterdam
11
12
Situation before
• Manual process
• Not able to process many articles
Rich data
1. Knowledge Base (Company.info):
– >2.8M organization profiles
– Rich metadata: sector information, financial information,
people, buildings, etc…
2. News articles:
– >1200 sources (Dutch, online)
• From Het Financieele Dagblad to the Groninger Gezinsbode
– Around ~4k articles per day
– Multiple years of hand-labeled archive
Model
• Two stage approach:
– Named-entity recognition
– Entity Linking using binary classification
• Predict whether mention m should be linked to entity candidate c
Pipeline
16
1. 0. 1. 1. 0.
Doc
NER mention1 org1
org2
org3
…
org n
Extract
Features
1. 0. 0. 1. 0.
1. 1. 0. 0. 0.
1. 0. 1. 1. 0.
RF
1
0
mention2
mention3
0.647
…
0.873
0.780
0.342
…
Pipeline
17
1. 0. 1. 1. 0.
NER mention1 org1
org2
org3
org n
Extract
Features
1. 0. 0. 1. 0.
1. 1. 0. 0. 0.
1. 0. 1. 1. 0.
RF
1
0
mention2
mention3
0.647
0.873
0.780
0.342
…… …
Pipeline
18
1. 0. 1. 1. 0.
NER mention1 org2
org3
org1
org n
Extract
Features
1. 0. 0. 1. 0.
1. 1. 0. 0. 0.
1. 0. 1. 1. 0.
RF
1
0
mention2
mention3
0.873
0.780
0.647
0.342
…… …
org3 1. 0. 1. 1. 0. 0.780
Pipeline
19
1. 0. 1. 1. 0.
NER mention1
org1
…
org n
Extract
Features
1. 0. 0. 1. 0.
RF
1
0
mention2
mention3 0.647
0.342
…
org2 1. 1. 0. 0. 0. 0.873
…
Evaluation
• Take data, make train/test-split
– NER: ~85%
– EL: ~85%
• But: Data is noisy/biased
– + Manual inspection
20
Results
– Replace previous manual process
– Number of articles per day: approx. +160%
– Number of linked orgs: approx. +310%
– More “long tail” articles
21
Smart Radio
Smart Radio – audio personalization
• Current audio listening experience:
– Passive listening
– Long audio data with mix of topics
• New way:
– On demand and personalized listening
TV
Radio
Smart Radio - BNR app
Smart Radio – pipeline “serverless”
Take-home messages
• AI projects add values to existing businesses
• Diversify content delivered to customers: enable
“long tail” articles/audio fragments
• In combination with rich data, simple AI models work
• Our solutions are re-usable
Upcoming projects
• Fully automatic audio segmentation
• Personalized summarization of news articles
Twitter: @dungchu
Email: dung.manh.chu@fdmediagroep.nl
LinkedIn: https://www.linkedin.com/in/dungmanhchu/

More Related Content

Similar to Mining knowledge in news for personalization

Connections Drive Digital Transformation
Connections Drive Digital TransformationConnections Drive Digital Transformation
Connections Drive Digital Transformation
Neo4j
 
Social media radar | Artur Karda
Social media radar | Artur KardaSocial media radar | Artur Karda
Social media radar | Artur Karda
Artur Karda
 
Kiev congress social media radar - artur karda 2011-10-21
Kiev congress   social media radar - artur karda 2011-10-21Kiev congress   social media radar - artur karda 2011-10-21
Kiev congress social media radar - artur karda 2011-10-21Oksana Kushnir
 
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Connected Data World
 
B.Pearson_Ten Trends Reshaping our Industry_r01
B.Pearson_Ten Trends Reshaping our Industry_r01B.Pearson_Ten Trends Reshaping our Industry_r01
B.Pearson_Ten Trends Reshaping our Industry_r01W2O Group
 
Open Data and News Analytics Demo
Open Data and News Analytics DemoOpen Data and News Analytics Demo
Open Data and News Analytics Demo
Ontotext
 
Predictive Analytics World Chicago 2015
Predictive Analytics World Chicago 2015Predictive Analytics World Chicago 2015
Predictive Analytics World Chicago 2015
Dan Potter
 
Advanced Analytics for Any Data at Real-Time Speed
Advanced Analytics for Any Data at Real-Time SpeedAdvanced Analytics for Any Data at Real-Time Speed
Advanced Analytics for Any Data at Real-Time Speed
danpotterdwch
 
IABE Big Data information paper - An actuarial perspective
IABE Big Data information paper - An actuarial perspectiveIABE Big Data information paper - An actuarial perspective
IABE Big Data information paper - An actuarial perspective
Mateusz Maj
 
Single Cell RNA Sequencing Market.pdf
 Single Cell RNA Sequencing Market.pdf Single Cell RNA Sequencing Market.pdf
Single Cell RNA Sequencing Market.pdf
BIS Research Inc.
 
Technology Planning Document_v1.1
Technology Planning Document_v1.1Technology Planning Document_v1.1
Technology Planning Document_v1.1Raju Nair
 
Technology Planning Document V1.1
Technology Planning Document V1.1Technology Planning Document V1.1
Technology Planning Document V1.1digital.signage
 
Climate change action through artificial intelligence
Climate change action through artificial intelligenceClimate change action through artificial intelligence
Climate change action through artificial intelligence
weADAPT
 
News Media Metadata - The Current Landscape
News Media Metadata - The Current LandscapeNews Media Metadata - The Current Landscape
News Media Metadata - The Current Landscape
Richard Wallis
 
Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...
Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...
Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...
TISP Project
 
Moving Targets: Harnessing Real-time Value from Data in Motion
Moving Targets: Harnessing Real-time Value from Data in Motion Moving Targets: Harnessing Real-time Value from Data in Motion
Moving Targets: Harnessing Real-time Value from Data in Motion
Inside Analysis
 
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
Patrick Van Renterghem
 
2011 CBSi Overview
2011 CBSi Overview2011 CBSi Overview
2011 CBSi Overview
markvandenhaak
 
GanttChart for ClearQuest User Guide
GanttChart for ClearQuest User GuideGanttChart for ClearQuest User Guide
GanttChart for ClearQuest User GuideAlexander Novichkov
 
Powering the Intelligent Enterprise
Powering the Intelligent EnterprisePowering the Intelligent Enterprise
Powering the Intelligent Enterprise
Christine Connors
 

Similar to Mining knowledge in news for personalization (20)

Connections Drive Digital Transformation
Connections Drive Digital TransformationConnections Drive Digital Transformation
Connections Drive Digital Transformation
 
Social media radar | Artur Karda
Social media radar | Artur KardaSocial media radar | Artur Karda
Social media radar | Artur Karda
 
Kiev congress social media radar - artur karda 2011-10-21
Kiev congress   social media radar - artur karda 2011-10-21Kiev congress   social media radar - artur karda 2011-10-21
Kiev congress social media radar - artur karda 2011-10-21
 
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
Powerful Information Discovery with Big Knowledge Graphs –The Offshore Leaks ...
 
B.Pearson_Ten Trends Reshaping our Industry_r01
B.Pearson_Ten Trends Reshaping our Industry_r01B.Pearson_Ten Trends Reshaping our Industry_r01
B.Pearson_Ten Trends Reshaping our Industry_r01
 
Open Data and News Analytics Demo
Open Data and News Analytics DemoOpen Data and News Analytics Demo
Open Data and News Analytics Demo
 
Predictive Analytics World Chicago 2015
Predictive Analytics World Chicago 2015Predictive Analytics World Chicago 2015
Predictive Analytics World Chicago 2015
 
Advanced Analytics for Any Data at Real-Time Speed
Advanced Analytics for Any Data at Real-Time SpeedAdvanced Analytics for Any Data at Real-Time Speed
Advanced Analytics for Any Data at Real-Time Speed
 
IABE Big Data information paper - An actuarial perspective
IABE Big Data information paper - An actuarial perspectiveIABE Big Data information paper - An actuarial perspective
IABE Big Data information paper - An actuarial perspective
 
Single Cell RNA Sequencing Market.pdf
 Single Cell RNA Sequencing Market.pdf Single Cell RNA Sequencing Market.pdf
Single Cell RNA Sequencing Market.pdf
 
Technology Planning Document_v1.1
Technology Planning Document_v1.1Technology Planning Document_v1.1
Technology Planning Document_v1.1
 
Technology Planning Document V1.1
Technology Planning Document V1.1Technology Planning Document V1.1
Technology Planning Document V1.1
 
Climate change action through artificial intelligence
Climate change action through artificial intelligenceClimate change action through artificial intelligence
Climate change action through artificial intelligence
 
News Media Metadata - The Current Landscape
News Media Metadata - The Current LandscapeNews Media Metadata - The Current Landscape
News Media Metadata - The Current Landscape
 
Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...
Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...
Paola Mazzucchi, AIE and mEDRA Project Manager, Converging metadata for conve...
 
Moving Targets: Harnessing Real-time Value from Data in Motion
Moving Targets: Harnessing Real-time Value from Data in Motion Moving Targets: Harnessing Real-time Value from Data in Motion
Moving Targets: Harnessing Real-time Value from Data in Motion
 
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
 
2011 CBSi Overview
2011 CBSi Overview2011 CBSi Overview
2011 CBSi Overview
 
GanttChart for ClearQuest User Guide
GanttChart for ClearQuest User GuideGanttChart for ClearQuest User Guide
GanttChart for ClearQuest User Guide
 
Powering the Intelligent Enterprise
Powering the Intelligent EnterprisePowering the Intelligent Enterprise
Powering the Intelligent Enterprise
 

Recently uploaded

platelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptxplatelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptx
muralinath2
 
extra-chromosomal-inheritance[1].pptx.pdfpdf
extra-chromosomal-inheritance[1].pptx.pdfpdfextra-chromosomal-inheritance[1].pptx.pdfpdf
extra-chromosomal-inheritance[1].pptx.pdfpdf
DiyaBiswas10
 
Hemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptxHemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptx
muralinath2
 
Nutraceutical market, scope and growth: Herbal drug technology
Nutraceutical market, scope and growth: Herbal drug technologyNutraceutical market, scope and growth: Herbal drug technology
Nutraceutical market, scope and growth: Herbal drug technology
Lokesh Patil
 
Lateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensiveLateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensive
silvermistyshot
 
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Sérgio Sacani
 
Toxic effects of heavy metals : Lead and Arsenic
Toxic effects of heavy metals : Lead and ArsenicToxic effects of heavy metals : Lead and Arsenic
Toxic effects of heavy metals : Lead and Arsenic
sanjana502982
 
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptxBody fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
muralinath2
 
Orion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWSOrion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWS
Columbia Weather Systems
 
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
Sérgio Sacani
 
Mammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also FunctionsMammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also Functions
YOGESH DOGRA
 
general properties of oerganologametal.ppt
general properties of oerganologametal.pptgeneral properties of oerganologametal.ppt
general properties of oerganologametal.ppt
IqrimaNabilatulhusni
 
GBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture MediaGBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture Media
Areesha Ahmad
 
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiologyBLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
NoelManyise1
 
bordetella pertussis.................................ppt
bordetella pertussis.................................pptbordetella pertussis.................................ppt
bordetella pertussis.................................ppt
kejapriya1
 
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Sérgio Sacani
 
Nucleic Acid-its structural and functional complexity.
Nucleic Acid-its structural and functional complexity.Nucleic Acid-its structural and functional complexity.
Nucleic Acid-its structural and functional complexity.
Nistarini College, Purulia (W.B) India
 
role of pramana in research.pptx in science
role of pramana in research.pptx in sciencerole of pramana in research.pptx in science
role of pramana in research.pptx in science
sonaliswain16
 
Chapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisisChapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisis
tonzsalvador2222
 
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Ana Luísa Pinho
 

Recently uploaded (20)

platelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptxplatelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptx
 
extra-chromosomal-inheritance[1].pptx.pdfpdf
extra-chromosomal-inheritance[1].pptx.pdfpdfextra-chromosomal-inheritance[1].pptx.pdfpdf
extra-chromosomal-inheritance[1].pptx.pdfpdf
 
Hemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptxHemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptx
 
Nutraceutical market, scope and growth: Herbal drug technology
Nutraceutical market, scope and growth: Herbal drug technologyNutraceutical market, scope and growth: Herbal drug technology
Nutraceutical market, scope and growth: Herbal drug technology
 
Lateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensiveLateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensive
 
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...
 
Toxic effects of heavy metals : Lead and Arsenic
Toxic effects of heavy metals : Lead and ArsenicToxic effects of heavy metals : Lead and Arsenic
Toxic effects of heavy metals : Lead and Arsenic
 
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptxBody fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
Body fluids_tonicity_dehydration_hypovolemia_hypervolemia.pptx
 
Orion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWSOrion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWS
 
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
 
Mammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also FunctionsMammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also Functions
 
general properties of oerganologametal.ppt
general properties of oerganologametal.pptgeneral properties of oerganologametal.ppt
general properties of oerganologametal.ppt
 
GBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture MediaGBSN - Microbiology (Lab 4) Culture Media
GBSN - Microbiology (Lab 4) Culture Media
 
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiologyBLOOD AND BLOOD COMPONENT- introduction to blood physiology
BLOOD AND BLOOD COMPONENT- introduction to blood physiology
 
bordetella pertussis.................................ppt
bordetella pertussis.................................pptbordetella pertussis.................................ppt
bordetella pertussis.................................ppt
 
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
Observation of Io’s Resurfacing via Plume Deposition Using Ground-based Adapt...
 
Nucleic Acid-its structural and functional complexity.
Nucleic Acid-its structural and functional complexity.Nucleic Acid-its structural and functional complexity.
Nucleic Acid-its structural and functional complexity.
 
role of pramana in research.pptx in science
role of pramana in research.pptx in sciencerole of pramana in research.pptx in science
role of pramana in research.pptx in science
 
Chapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisisChapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisis
 
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...
 

Mining knowledge in news for personalization

  • 1. Mining knowledge in news for personalization Dung Chu and David Graus, data scientists FD Mediagroep
  • 2. Outline • Introduction FDMG • AI use cases: – FD personalized newsletter – CompanyInfo entity linker – Smart Radio
  • 3. FD mediagroep Het Financieele Dagblad Het Financieele Dagblad is dé nieuws- en inspiratiebron die op elk moment van de dag financieel-economische betekenis geeft aan ontwikkelingen in de wereld. Company info geeft altijd real time toegang tot actuele bedrijfs- en prospectinformatie van 2,5 miljoen bedrijven in Nederland FD/BNR Networks FD/BNR Networks brengt ideeën, meningen en talentrijke ondernemers en professionals samen door de organisatie van o.a. forums, debatten, netwerksessies, en evenementenreeksen. BNR Nieuwsradio BNR Nieuwsradio is de enige radiozender in Nederland waar ambitieuze en ondernemende mensen 24 uur per dag op de hoogte worden gehouden van relevant nieuws. FD Mediagroep helpt om betere beslissingen te nemen door overzicht, inzicht én (vooruit)zicht te bieden met journalistieke merken en hoogstaande kwalitatieve producten.
  • 5. FD personalized newsletter: goals • Diversify content sent to readers • Improve engagement
  • 6. FD personalized newsletters • Rich data: – Logged-in data with unique client ID • Recommendation model: Item-to-item collaborative filtering Published last week Older articles PySpark: + Convert reader-article matrix to IndexedRowMatrix + columnSimilarities(): fast compute of cosine similarity between columns
  • 8. Results: paid clients (fixed 2000) T-test
  • 9. Content diversification Popularity emails Personalized emails 7 articles 178 articles (60 clicked) Long tail articles 25000 15000
  • 10. CompanyInfo entity linker Ref: https://www.slideshare.net/davidgraus/financial-news-mining-pydata-amsterdam
  • 11. 11
  • 12. 12
  • 13. Situation before • Manual process • Not able to process many articles
  • 14. Rich data 1. Knowledge Base (Company.info): – >2.8M organization profiles – Rich metadata: sector information, financial information, people, buildings, etc… 2. News articles: – >1200 sources (Dutch, online) • From Het Financieele Dagblad to the Groninger Gezinsbode – Around ~4k articles per day – Multiple years of hand-labeled archive
  • 15. Model • Two stage approach: – Named-entity recognition – Entity Linking using binary classification • Predict whether mention m should be linked to entity candidate c
  • 16. Pipeline 16 1. 0. 1. 1. 0. Doc NER mention1 org1 org2 org3 … org n Extract Features 1. 0. 0. 1. 0. 1. 1. 0. 0. 0. 1. 0. 1. 1. 0. RF 1 0 mention2 mention3 0.647 … 0.873 0.780 0.342 …
  • 17. Pipeline 17 1. 0. 1. 1. 0. NER mention1 org1 org2 org3 org n Extract Features 1. 0. 0. 1. 0. 1. 1. 0. 0. 0. 1. 0. 1. 1. 0. RF 1 0 mention2 mention3 0.647 0.873 0.780 0.342 …… …
  • 18. Pipeline 18 1. 0. 1. 1. 0. NER mention1 org2 org3 org1 org n Extract Features 1. 0. 0. 1. 0. 1. 1. 0. 0. 0. 1. 0. 1. 1. 0. RF 1 0 mention2 mention3 0.873 0.780 0.647 0.342 …… …
  • 19. org3 1. 0. 1. 1. 0. 0.780 Pipeline 19 1. 0. 1. 1. 0. NER mention1 org1 … org n Extract Features 1. 0. 0. 1. 0. RF 1 0 mention2 mention3 0.647 0.342 … org2 1. 1. 0. 0. 0. 0.873 …
  • 20. Evaluation • Take data, make train/test-split – NER: ~85% – EL: ~85% • But: Data is noisy/biased – + Manual inspection 20
  • 21. Results – Replace previous manual process – Number of articles per day: approx. +160% – Number of linked orgs: approx. +310% – More “long tail” articles 21
  • 23. Smart Radio – audio personalization • Current audio listening experience: – Passive listening – Long audio data with mix of topics • New way: – On demand and personalized listening TV Radio
  • 24. Smart Radio - BNR app
  • 25. Smart Radio – pipeline “serverless”
  • 26. Take-home messages • AI projects add values to existing businesses • Diversify content delivered to customers: enable “long tail” articles/audio fragments • In combination with rich data, simple AI models work • Our solutions are re-usable
  • 27. Upcoming projects • Fully automatic audio segmentation • Personalized summarization of news articles
  • 28. Twitter: @dungchu Email: dung.manh.chu@fdmediagroep.nl LinkedIn: https://www.linkedin.com/in/dungmanhchu/

Editor's Notes

  1. Business related news; over 4k articles a day from over >900 sources
  2. You can go from news to profile
  3. Tying it all together
  4. Tying it all together
  5. Tying it all together
  6. Tying it all together
  7. Bias: Typically 1 or 2 orgs per article (humans). Errors. “Misses.”