Los Angeles | London | New Delhi | Singapore | Washington DC
Enriching the
Social Sciences
through Text
Mining
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
1,400,000 journal articles 366,000 numerical data series
100,000 book chapters 4,000 reports
1,300 case studies 12 debate series
112,000 reference entries 1,300 videos
400 literature reviews 1,000 podcasts
The two sides of text mining
Internal: content enrichment
• Auto-tagging
• Links to related material
External: enabling users to
mine our content
Los Angeles | London | New Delhi | Singapore | Washington DC
Case study: SAGE Knowledge
• Problem:
• Almost 4,000 books and encyclopedias, hardly any of
which have keywords
• Even fewer have keywords at chapter level
• No links to other relevant content on other SAGE sites
• Possible solutions:
• Get someone to read it all
• Get someone to not read it all
• Get the authors to do it
• Get a computer to do it
Los Angeles | London | New Delhi | Singapore | Washington DC
Case study: SAGE Knowledge
• Problem:
• Almost 4,000 books and encyclopedias, hardly any of
which have keywords
• Even fewer have keywords at chapter level
• Possible solutions:
• Get someone to read it all
• Get someone to not read it all
• Get the authors to do it
• Get a computer to do it
Los Angeles | London | New Delhi | Singapore | Washington DC
Case study: SAGE Knowledge
● New problem:
• Lack of a standard vocabulary to describe social science
concepts
● Possible solutions:
• Pay someone to make one
• Find/license one
• Build one
Los Angeles | London | New Delhi | Singapore | Washington DC
Forming a common vocabulary
How do we identify
the most important
concepts in our
content?
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Improving recall and
disambiguation
Los Angeles | London | New Delhi | Singapore | Washington DC
Improving recall and
disambiguation
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014. In
February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd
Winter Olympic Games. It was the first Olympics held in Russia since the Soviet
Union dissolved in 1991, and the country—along with Russian president Vladimir
Putin—was heavily invested in the games’ success, hoping to impress the rest of
the world. In the years and months preceding the games, political developments,
security concerns, and allegations of corruption drew harsh scrutiny from
outside Russia and increased speculation that the games could be a disaster.
Internal Turmoil. Perhaps chief among the concerns surrounding Russia's
Olympics was ensuring the security of those attending the games. Sochi is
located in a part of Russia that is close to the country's North Caucasus, which
encompasses several regions with peoples who are seeking independence from
Russia.
Los Angeles | London | New Delhi | Singapore | Washington DC
Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014.
In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd
Winter Olympic Games.
It was the first Olympics held in Russia since the Soviet Union dissolved in 1991,
and the country—along with Russian president Vladimir Putin—was heavily
invested in the games’ success, hoping to impress the rest of the world.
In the years and months preceding the games, political developments, security
concerns, and allegations of corruption drew harsh scrutiny from outside Russia
and increased speculation that the games could be a disaster.
Internal Turmoil.
Perhaps chief among the concerns surrounding Russia's Olympics was ensuring
the security of those attending the games.
Sochi is located in a part of Russia that is close to the country's North Caucasus,
which encompasses several regions with peoples who are seeking independence
from Russia.
Los Angeles | London | New Delhi | Singapore | Washington DC
Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014.
In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd
Winter Olympic Games.
It was the first Olympics held in Russia since the Soviet Union dissolved in 1991,
and the country—along with Russian president Vladimir Putin—was heavily
invested in the games’ success, hoping to impress the rest of the world.
In the years and months preceding the games, political developments, security
concerns, and allegations of corruption drew harsh scrutiny from outside Russia
and increased speculation that the games could be a disaster.
Internal Turmoil.
Perhaps chief among the concerns surrounding Russia's Olympics was ensuring
the security of those attending the games.
Sochi is located in a part of Russia that is close to the country's North Caucasus,
which encompasses several regions with peoples who are seeking independence
from Russia.
Los Angeles | London | New Delhi | Singapore | Washington DC
Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014.
In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd
Winter Olympic Games.
It was the first Olympics held in Russia since the Soviet Union dissolved in 1991,
and the country—along with Russian president Vladimir Putin—was heavily
invested in the games’ success, hoping to impress the rest of the world.
In the years and months preceding the games, political developments, security
concerns, and allegations of corruption drew harsh scrutiny from outside Russia
and increased speculation that the games could be a disaster.
Internal Turmoil.
Perhaps chief among the concerns surrounding Russia's Olympics was ensuring
the security of those attending the games.
Sochi is located in a part of Russia that is close to the country's North Caucasus,
which encompasses several regions with peoples who are seeking independence
from Russia.
Los Angeles | London | New Delhi | Singapore | Washington DC
Sochi 0.130
IOC 0.079
athlete 0.068
Olympics 0.058
Olympic Games 0.057
Putin 0.044
International Olympic Committee 0.029
IOC president 0.027
Olympic project 0.027
Olympic athlete 0.026
Los Angeles | London | New Delhi | Singapore | Washington DC
Olympic Games 1.808
games 1.062
Russia 0.772
athletes 0.642
Vladimir Putin 0.511
sport 0.411
skiing 0.305
homosexuality 0.300
Black Sea 0.291
committees 0.277
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
False friends
• Acceleration is a meaningful term in engineering, but not in
social sciences.
• Solution: social science content does not get tagged with
‘acceleration’
Los Angeles | London | New Delhi | Singapore | Washington DC
Mini case study:
SAGE Research Methods
Los Angeles | London | New Delhi | Singapore | Washington DC
Mini case study:
SAGE Research Methods
• Improved consistency
• Ability to make global changes across a corpus
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Los Angeles | London | New Delhi | Singapore | Washington DC
Are we all doomed?
Los Angeles | London | New Delhi | Singapore | Washington DC
Are we all doomed?
• Computers are still sub-human at processing natural
language
• Computers can’t read between the lines
• Computers still need to be taught and trained
Los Angeles | London | New Delhi | Singapore | Washington DC
Thank you!
alan.maloney@sagepub.co.uk

November 18, 2015 NISO Webinar: Text Mining: Digging Deep for Knowledge

  • 1.
    Los Angeles |London | New Delhi | Singapore | Washington DC Enriching the Social Sciences through Text Mining
  • 2.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 3.
    Los Angeles |London | New Delhi | Singapore | Washington DC 1,400,000 journal articles 366,000 numerical data series 100,000 book chapters 4,000 reports 1,300 case studies 12 debate series 112,000 reference entries 1,300 videos 400 literature reviews 1,000 podcasts
  • 4.
    The two sidesof text mining Internal: content enrichment • Auto-tagging • Links to related material External: enabling users to mine our content
  • 5.
    Los Angeles |London | New Delhi | Singapore | Washington DC Case study: SAGE Knowledge • Problem: • Almost 4,000 books and encyclopedias, hardly any of which have keywords • Even fewer have keywords at chapter level • No links to other relevant content on other SAGE sites • Possible solutions: • Get someone to read it all • Get someone to not read it all • Get the authors to do it • Get a computer to do it
  • 6.
    Los Angeles |London | New Delhi | Singapore | Washington DC Case study: SAGE Knowledge • Problem: • Almost 4,000 books and encyclopedias, hardly any of which have keywords • Even fewer have keywords at chapter level • Possible solutions: • Get someone to read it all • Get someone to not read it all • Get the authors to do it • Get a computer to do it
  • 7.
    Los Angeles |London | New Delhi | Singapore | Washington DC Case study: SAGE Knowledge ● New problem: • Lack of a standard vocabulary to describe social science concepts ● Possible solutions: • Pay someone to make one • Find/license one • Build one
  • 8.
    Los Angeles |London | New Delhi | Singapore | Washington DC Forming a common vocabulary How do we identify the most important concepts in our content?
  • 9.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 10.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 11.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 12.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 13.
    Los Angeles |London | New Delhi | Singapore | Washington DC Improving recall and disambiguation
  • 14.
    Los Angeles |London | New Delhi | Singapore | Washington DC Improving recall and disambiguation
  • 15.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 16.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 17.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 18.
    Los Angeles |London | New Delhi | Singapore | Washington DC Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014. In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd Winter Olympic Games. It was the first Olympics held in Russia since the Soviet Union dissolved in 1991, and the country—along with Russian president Vladimir Putin—was heavily invested in the games’ success, hoping to impress the rest of the world. In the years and months preceding the games, political developments, security concerns, and allegations of corruption drew harsh scrutiny from outside Russia and increased speculation that the games could be a disaster. Internal Turmoil. Perhaps chief among the concerns surrounding Russia's Olympics was ensuring the security of those attending the games. Sochi is located in a part of Russia that is close to the country's North Caucasus, which encompasses several regions with peoples who are seeking independence from Russia.
  • 19.
    Los Angeles |London | New Delhi | Singapore | Washington DC Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014. In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd Winter Olympic Games. It was the first Olympics held in Russia since the Soviet Union dissolved in 1991, and the country—along with Russian president Vladimir Putin—was heavily invested in the games’ success, hoping to impress the rest of the world. In the years and months preceding the games, political developments, security concerns, and allegations of corruption drew harsh scrutiny from outside Russia and increased speculation that the games could be a disaster. Internal Turmoil. Perhaps chief among the concerns surrounding Russia's Olympics was ensuring the security of those attending the games. Sochi is located in a part of Russia that is close to the country's North Caucasus, which encompasses several regions with peoples who are seeking independence from Russia.
  • 20.
    Los Angeles |London | New Delhi | Singapore | Washington DC Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014. In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd Winter Olympic Games. It was the first Olympics held in Russia since the Soviet Union dissolved in 1991, and the country—along with Russian president Vladimir Putin—was heavily invested in the games’ success, hoping to impress the rest of the world. In the years and months preceding the games, political developments, security concerns, and allegations of corruption drew harsh scrutiny from outside Russia and increased speculation that the games could be a disaster. Internal Turmoil. Perhaps chief among the concerns surrounding Russia's Olympics was ensuring the security of those attending the games. Sochi is located in a part of Russia that is close to the country's North Caucasus, which encompasses several regions with peoples who are seeking independence from Russia.
  • 21.
    Los Angeles |London | New Delhi | Singapore | Washington DC Sochi Hosts the 2014 Winter Olympic Games: February 4 and 7, 2014. In February 2014, the coastal resort city of Sochi, Russia, played host to the 22nd Winter Olympic Games. It was the first Olympics held in Russia since the Soviet Union dissolved in 1991, and the country—along with Russian president Vladimir Putin—was heavily invested in the games’ success, hoping to impress the rest of the world. In the years and months preceding the games, political developments, security concerns, and allegations of corruption drew harsh scrutiny from outside Russia and increased speculation that the games could be a disaster. Internal Turmoil. Perhaps chief among the concerns surrounding Russia's Olympics was ensuring the security of those attending the games. Sochi is located in a part of Russia that is close to the country's North Caucasus, which encompasses several regions with peoples who are seeking independence from Russia.
  • 22.
    Los Angeles |London | New Delhi | Singapore | Washington DC Sochi 0.130 IOC 0.079 athlete 0.068 Olympics 0.058 Olympic Games 0.057 Putin 0.044 International Olympic Committee 0.029 IOC president 0.027 Olympic project 0.027 Olympic athlete 0.026
  • 23.
    Los Angeles |London | New Delhi | Singapore | Washington DC Olympic Games 1.808 games 1.062 Russia 0.772 athletes 0.642 Vladimir Putin 0.511 sport 0.411 skiing 0.305 homosexuality 0.300 Black Sea 0.291 committees 0.277
  • 24.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 25.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 26.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 27.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 28.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 29.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 30.
    Los Angeles |London | New Delhi | Singapore | Washington DC False friends • Acceleration is a meaningful term in engineering, but not in social sciences. • Solution: social science content does not get tagged with ‘acceleration’
  • 31.
    Los Angeles |London | New Delhi | Singapore | Washington DC Mini case study: SAGE Research Methods
  • 32.
    Los Angeles |London | New Delhi | Singapore | Washington DC Mini case study: SAGE Research Methods • Improved consistency • Ability to make global changes across a corpus
  • 33.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 34.
    Los Angeles |London | New Delhi | Singapore | Washington DC
  • 35.
    Los Angeles |London | New Delhi | Singapore | Washington DC Are we all doomed?
  • 36.
    Los Angeles |London | New Delhi | Singapore | Washington DC Are we all doomed? • Computers are still sub-human at processing natural language • Computers can’t read between the lines • Computers still need to be taught and trained
  • 37.
    Los Angeles |London | New Delhi | Singapore | Washington DC Thank you! alan.maloney@sagepub.co.uk