SlideShare a Scribd company logo
Who’s Doing What for Whom, and How?
The Social Media Analysis Solution Space


 Seth Grimes
 @sethgrimes
Deconstruction

The topic “Knowledge Extraction and Consolidation
   from Social Media” is comprised of:
    • Knowledge Extraction.
    • Knowledge Consolidation.
    • Social Media.




Sentiment, opinion mining, and analysis are involved.
I’ll talk about these matters.
Deconstruction, 2

My topic: Who’s Doing What for Whom?
    • Who = Solution providers:
      researchers, software, services.
    • What = Social media analysis (SMA), “social business,”
      analytics-infused advisory services.
    • For Whom = Business users.
    • How = Technologies.
I’ll talk about these elements as well, starting with the
     applications, then moving to tech, then to
     providers.
Theses

Social Media = Platforms + Networks + Content.
Knowledge = Contextualized, interrelated information.
Knowledge, in automated settings, must be structured
  to be usable .
Consolidation involves
  collection, filtering, analysis, reduction, integration, i
  nference, and presentation… iteratively.
“Business is a collection of activities carried on for
whatever purpose, be it
science, technology, commerce, industry, law, governm
ent, defense, et cetera.”
Business Questions

What are people saying? What’s hot/trending?
What are they saying about {topic|person|product} X?
 ... about X versus {topic|person|product} Y?
How has opinion about X and Y evolved?
How has opinion correlated with
 {our|competitors’|general}
 {news|marketing|sales|events}?
What’s behind opinion, the root causes?
    • (How) Can we link opinions & transactions?
    • (How) Can we link opinion & intent?
Who are opinion leaders?
Business Needs

How do these factors affect my business?
How can answers to these questions help me
 improve business processes?


We have a decision support need and an operational
  need. We=
    • Consumers.
    • Marketers.
    • Competitors.
    • Managers.
Analysis Approaches

In industry settings, we (should) work backward:
    Mission  Goals  Presentation  Methods &
    Data
    • What are your business goals?
    • What insights will help your reach them?
    • What data, transformation, and presentations will
      generate those insights?
    • For each option, what will it cost and what is it worth:
      What is the expected/projected ROI?
Sometimes we work this way, and sometimes we
  want to explore…
Data, Information & Knowledge



                             “Where America’s Racist
                             Tweets Come From”




   http://mashable.com/2012/11/11/racist-tweets/
Document
    input and
    processing




   Knowledge
   handling is     Desk Set (1957): Computer engineer
   key             Richard Sumner (Spencer Tracy)
                   and television network librarian
                   Bunny Watson (Katherine Hepburn)
H.P. Luhn, “A      and the "electronic brain" EMERAC.
Business
Intelligence
System,” IBM
Journal, October
1958
Intelligence


Business intelligence (BI) was first defined in 1958:
  “In this paper, business is a collection of activities carried on
  for whatever purpose, be it
  science, technology, commerce, industry, law, government, d
  efense, et cetera... The notion of intelligence is also defined
  here... as ‘the ability to apprehend the interrelationships of
  presented facts in such a way as to guide action towards a
  desired goal.’”
                                                -- Hans Peter Luhn
                                  “A Business Intelligence System”
                                        IBM Journal, October 1958
Applies to --
The Popular, Misguided View, 2
Incomplete!

All media are social.
Incomplete, 2

  Personal. Mobile. Knowledge Infused.




http://timoelliott.com/blog/2010/10/sap-businessobjects-augmented-
explorer-now-available-resources-to-test-it.html
What Is Our Vision? Our Goal?

The inclusion of social data and social-derived insights
   (a.k.a. information) in a global knowledge network?
The social Semantic Web?
The Semantic Social Web?


Why extract knowledge from social media?
    • The academic challenge is interesting but not enough.
    • We want to create better social-computing experiences.
    • We want to infuse social into other computing realms.
Our Social Knowledge Goal?
                                                   http://www.cambridgesemantics.com/sema
                                                   ntic-university/semantic-search-and-the-
                                                   semantic-web




                                       http://img.freebase.com/api/trans/raw/m/02dtnzv




“The Semantic Web has been and remains a
  parallel, incomplete, never-up-to-date subset of the World Wide
  Web and the databases accessible through it.” (Me, 2010)
Business Driven Approaches

 Pragmatic knowledge structuring.




https://developers.facebook.com/docs/opengraph/


      <div itemscope itemtype="http://schema.org/Organization">
       <span itemprop="name">Google.org (GOOG)</span>

      Contact Details:
       <div itemprop="address" itemscope itemtype="http://schema.org/PostalAddress">
        Main address:
          <span itemprop="streetAddress">38 avenue de l'Opera</span>
          <span itemprop="postalCode">F-75002</span>
          <span itemprop="addressLocality">Paris, France</span> ,
       </div>
        Tel:<span itemprop="telephone">( 33 1) 42 68 53 00 </span>,                    http://open.blogs.nytimes.com/2012/02/16/rnews-
        Fax:<span itemprop="faxNumber">( 33 1) 42 68 53 01 </span>,
        E-mail: <span itemprop="email">secretariat(at)google.org</span>                                 is-here-and-this-is-what-it-means/
      </div>

      http://schema.org/Organization
Data pipes   Business Driven Approaches, 2a
Business Driven Approaches, 3

Social media monitoring.




  http://www.goldbachinteractive.com/current-news/technical-papers/social-media-
  monitoring-a-small-market-overview-sysomos-radian6-and-more
Business Driven Approaches, 3’

Dashboards and engagement consoles.
Fusions: Analysis
Business Driven, 4

Infographics: Old wine, new bottles.
    − Static, non-collaborative.
    + I like narrative.
Business Driven Approaches, 5




A
Semanticized
Web
Business Driven, 6

Question Authorities.
                   https://secure.wikimedia.org/wiki
                   pedia/en/wiki/File:Watson_Jeopar
                   dy.jpg
The Race
Milestones

Language+ understanding.
    • Text, speech, and video.
    • Narrative, discourse, and argument.
Information extraction.


Knowledge structuring and integration.
Inference; synthesis.
Language generation.
Conversation; interaction; autonomy.
≈> Convergence, a.k.a. Singularity
What does the market say?




Free report download via http://altaplana.com/TA2011
Users (current & potential) say
Important sources

What textual information are you analyzing or do
  you plan to analyze?
blogs and other social media (twitter, social-   62% (2011)
network sites, etc.)                             47% (2009)

news articles                                    41% (2011)
                                                 44% (2009)
on-line forums                                   35% (2011)
                                                 35% (2009)
customer/market surveys                          35% (2011)
                                                 34% (2009)
reviews                                          30% (2011)
                                                 21% (2009)
e-mail and correspondence                        29% (2011)
                                                 36% (2009)
Information in text
Applications

Text analytics has applications in –
  • Intelligence & law enforcement.
  • Life sciences.
  • Media & publishing including social-media analysis and
    contextual advertizing.
  • Competitive intelligence.
  • Voice of the Customer: CRM, product management &
    marketing.
  • Legal, tax & regulatory (LTR) including compliance.
  • Recruiting.
Online Commerce

Text analytics is applied for marketing, search
   optimization, competitive intelligence.
    • Analyze social media and enterprise feedback to
      understand opportunities, threats, trends.
    • Categorize product and service offerings for on-site
      search and faceted navigation and to enrich content
      delivery.
    • Annotate pages to enhance Web-search
      findability, ranking.
    • Scrape competitor sites for offers and pricing.
    • Analyze social and news media for competitive
      information.
Voice of the Customer

Text analytics is applied to enhance customer service
   and satisfaction.
    • Analyze customer interactions and opinions –
          •    E-mail, contact-center notes, survey responses.
          •    Forum & blog posting and other social media.
    • – to –
          •    Address customer product & service issues.
          •    Improve quality.
          •    Manage brand & reputation.
    • If you can link qualitative information from text you can –
          •    Link feedback to transactions.
          •    Assess customer value.
          •    Understand root causes.
          •    Mine data for measures such as churn likelihood.
E-Discovery and Compliance

Text analytics is applied for compliance, fraud and
   risk, and e-discovery.
    • Regulatory mandates and corporate practices dictate –
          •   Monitoring corporate communications.
          •   Managing electronic stored information for production in event of
              litigation.
    • Sources include e-mail (!!), news, social media
    • Risk avoidance and fraud detection are key to effective
      decision making
          •   Text analytics mines critical data from unstructured sources.
          •   Integrated text-transactional analytics provides rich insights.
Knowledge, Enrichment & Integration

Semantics enables join across types and/or sources
  and/or structures, using meaningful identifiers, to
  create an ensemble that is greater than the sum of
  the parts.
Interrelate information to represent knowledge.
Enrichment and integration involve:
    • Mappings and transformations.
    • Aggregation and collection.
    • All the typical data concerns:
      cleansing, profiling, consistency, security,…
A Big Data analytics architecture
          (HPCC’s)
http://hpccsystems.com/




          http://www.geeklawblog.com/2011/12/lexis-advance-platform-launch-two.html
Text+ Technology Mashups

Text analytics generates semantics to bridge
   search, BI, and applications, enabling next-
   generation information systems.
 Semantic search                              Information access
 (search + text)                              (search + text + BI)


Search based            Search         BI
applications
                                              Integrated analytics
(search + text +
                                              (text + BI)
apps)
                            Applica-
    Text analytics           tions          NextGen
    (inner circle)                          CRM, EFM, MR, mar
                                            keting, …
Social Sources




Dealing with social
sources requires
flexibility, data/con
tent
sophistication, and
timeliness.
Sentiment Analysis

“Sentiment analysis is the task of identifying positive
and negative opinions, emotions, and evaluations.”
      -- Wilson, Wiebe & Hoffman, 2005, “Recognizing Contextual Polarity in
                                          Phrase-Level Sentiment Analysis”

“Sentiment analysis or opinion mining is the
computational study of opinions, sentiments and
emotions expressed in text… An opinion on a feature f is
a positive or negative view, attitude, emotion or
appraisal on f from an opinion holder.”
     -- Bing Liu, 2010, “Sentiment Analysis and Subjectivity,” in Handbook of
                                                Natural Language Processing
Beyond Polarity
Intent Analysis




http://sentibet.com/



                                 http://www.aiaioo.com/whitepapers/intention_analysis_use_cases.pdf
Complications

Sentiment may be of interest at multiple levels.
   Corpus / data space, i.e., across multiple sources.
   Document.
   Statement / sentence.
   Entity / topic / concept.
Human language is noisy and chaotic!
   Jargon, slang, irony, ambiguity, anaphora, polysemy, synonym
     y, etc.
   Context is key. Discourse analysis comes into play.
Must distinguish the sentiment holder from the object:
   “Geithner said the recession may worsen.”
Milestones Re-viewed

✔ Language+ understanding.
    Text, speech, and video.
    ✖ Narrative, discourse, and argument.
✔ Information extraction.
✔ Knowledge structuring and integration.
? Inference; synthesis.
Language generation.
Conversation; interaction; autonomy.
≈> Convergence, a.k.a. Singularity
Text Tech Initiatives

Now and near future.
    • Broader & deeper international language support.
    • Sentiment analysis, beyond polarity.
      Emotions, intent signals. etc.
    • Identity resolution & profile extraction.
      Online-social-enterprise data integration.
    • Semantic data integration, Complex Data.
    • Speech analytics.
    • Discourse analysis.
      Because isolated messages are not conversations.
    • Rich-media content analytics.
    • Augmented reality; new human-computer interfaces.
A Focus on Information & Applications

Now and near future.
    • Signal detection.
      Sentiment, emotion, identity, intent.
    • Semanticized applications.
      Linkable, mashable, enrichable.
    • Rich information.
      Context sensitive, situational.
Σ = Sense-making…
Primary Solution Considerations

Adaptation or specialization: To a business or cultural
 domain, information type (e.g., text, speech, images)
 & source (e.g., Twitter, e-mail, news articles).
By-user customization possibilities: For instance, via
 custom taxonomies, rules, lexicons.
Sentiment resolution: Aggregate, message, or feature
 level. (What features? Topics, coreferenced entities?)
Primary Considerations, cont.

Outputs: E.g., annotated
 text, models, indicators, dashboards, exploratory data
 interfaces.
Usage mode: As-a-service (via API) or
 installed/hosted/cloud.
Capacity: Volume, performance, throughput.
Cost.
Software & Platform Options

Text-analytics options may be grouped generally.
    • Installed text-analysis application, whether desktop or
      server or deployed in-database.
    • Data mining workbench.
    • Hosted.
    • Programming tool.
    • As-a-service, via an application programming interface
      (API).
    • Code library or component of a business/vertical
      application, for instance for CRM, e-discovery, search.
Text analytics is frequently embedded in search or
   other end-user applications.
Analytical Assets (Open Source)




                        >>> import nltk
                        >>> sentence = """At eight o'clock on Thursday
                        morning... Arthur didn't feel very good."""
                        >>> tokens = nltk.word_tokenize(sentence)
                        >>> tokens
                        ['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
                        'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']
                        >>> tagged = nltk.pos_tag(tokens)
                        >>> tagged[0:6]
                        [('At', 'IN'), ('eight', 'CD'), ("o'clock", 'JJ'), ('on', 'IN'),
                        ('Thursday', 'NNP'), ('morning', 'NN')]

                                                        http://nltk.org/
tm: Text Mining Package
A framework for text mining
applications within R.
Providers 1 (non-exhaustive) –

Human analysis.
  Converseon (to date).
  KD Paine Associates.
  Synthesio.
Human crowdsourced:
  Amazon Mechanical Turk.
  CrowdFlower.
Providers 2 (non-exhaustive) –

As-a-service:
   AlchemyAPI.
   Converseon ConveyAPI.
   OpenAmplify.
   Saplo.
Software libraries:
   GATE
   LingPipe.
   Python NLTK.
   R.
   RapidMiner.
Providers 3 (non-exhaustive) –

Financial markets applications.
   Digital Trowel.
   Dow Jones.
   RavenPack.
   Thomson Reuters NewsScope.
Providers 4 (non-exhaustive) –

Other-domain applications.
   Attensity.                Clarabridge.
   Crimson Hexagon.          Expert System.
   IBM.                      Kana/Overtone.
   Lexalytics.               Medallia.
   NetBase.                  OpenText/Nstein.
   SAP.                      SAS.
   Sysomos.                  WiseWindow.
Who’s Doing What for Whom, and How?
The Social Media Analysis Solution Space


 Seth Grimes
 @sethgrimes

More Related Content

Viewers also liked

Turing
TuringTuring
7. knowledge acquisition, representation and organization 8. semantic network...
7. knowledge acquisition, representation and organization 8. semantic network...7. knowledge acquisition, representation and organization 8. semantic network...
7. knowledge acquisition, representation and organization 8. semantic network...
AhL'Dn Daliva
 
The Turing Test - A sociotechnological analysis and prediction - Machine Inte...
The Turing Test - A sociotechnological analysis and prediction - Machine Inte...The Turing Test - A sociotechnological analysis and prediction - Machine Inte...
The Turing Test - A sociotechnological analysis and prediction - Machine Inte...
piero scaruffi
 
Integration Of Declarative and Procedural Knowledge for The Management of Chr...
Integration Of Declarative and Procedural Knowledge for The Management of Chr...Integration Of Declarative and Procedural Knowledge for The Management of Chr...
Integration Of Declarative and Procedural Knowledge for The Management of Chr...
Health Informatics New Zealand
 
Representation of knowledge
Representation of knowledgeRepresentation of knowledge
Representation of knowledge
Veera Balaji kumar veeraswamy
 
Turing test
Turing testTuring test
Turing test
Dipesh Senseii
 
Lecture 4 Meta Knowledge
Lecture 4 Meta KnowledgeLecture 4 Meta Knowledge
Lecture 4 Meta Knowledge
Simon Shurville
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial Intelligence
sanjay_asati
 
Knowledge Representation in Artificial intelligence
Knowledge Representation in Artificial intelligence Knowledge Representation in Artificial intelligence
Knowledge Representation in Artificial intelligence
Yasir Khan
 
Knowledge representation and Predicate logic
Knowledge representation and Predicate logicKnowledge representation and Predicate logic
Knowledge representation and Predicate logic
Amey Kerkar
 
Knowledge representation in AI
Knowledge representation in AIKnowledge representation in AI
Knowledge representation in AI
Vishal Singh
 

Viewers also liked (11)

Turing
TuringTuring
Turing
 
7. knowledge acquisition, representation and organization 8. semantic network...
7. knowledge acquisition, representation and organization 8. semantic network...7. knowledge acquisition, representation and organization 8. semantic network...
7. knowledge acquisition, representation and organization 8. semantic network...
 
The Turing Test - A sociotechnological analysis and prediction - Machine Inte...
The Turing Test - A sociotechnological analysis and prediction - Machine Inte...The Turing Test - A sociotechnological analysis and prediction - Machine Inte...
The Turing Test - A sociotechnological analysis and prediction - Machine Inte...
 
Integration Of Declarative and Procedural Knowledge for The Management of Chr...
Integration Of Declarative and Procedural Knowledge for The Management of Chr...Integration Of Declarative and Procedural Knowledge for The Management of Chr...
Integration Of Declarative and Procedural Knowledge for The Management of Chr...
 
Representation of knowledge
Representation of knowledgeRepresentation of knowledge
Representation of knowledge
 
Turing test
Turing testTuring test
Turing test
 
Lecture 4 Meta Knowledge
Lecture 4 Meta KnowledgeLecture 4 Meta Knowledge
Lecture 4 Meta Knowledge
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial Intelligence
 
Knowledge Representation in Artificial intelligence
Knowledge Representation in Artificial intelligence Knowledge Representation in Artificial intelligence
Knowledge Representation in Artificial intelligence
 
Knowledge representation and Predicate logic
Knowledge representation and Predicate logicKnowledge representation and Predicate logic
Knowledge representation and Predicate logic
 
Knowledge representation in AI
Knowledge representation in AIKnowledge representation in AI
Knowledge representation in AI
 

Similar to Knowledge Extraction from Social Media

Business Intelligence and Big Data in Cloud
Business Intelligence and Big Data in CloudBusiness Intelligence and Big Data in Cloud
Business Intelligence and Big Data in Cloud
Ding Li
 
Agile data science
Agile data scienceAgile data science
Agile data science
Joel Horwitz
 
Big Data - Analytics iC-360
Big Data - Analytics iC-360Big Data - Analytics iC-360
Big Data - Analytics iC-360
davemishra
 
PCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakaraj
PCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakarajPCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakaraj
PCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakaraj
sudhakarrun
 
Improve Customer Experience Management with Text Analytics - MeaningCloud web...
Improve Customer Experience Management with Text Analytics - MeaningCloud web...Improve Customer Experience Management with Text Analytics - MeaningCloud web...
Improve Customer Experience Management with Text Analytics - MeaningCloud web...
MeaningCloud
 
Leveraging Persuasive Architecture
Leveraging Persuasive ArchitectureLeveraging Persuasive Architecture
Leveraging Persuasive Architecture
Michael Rawlins
 
Smart Content = Smart Business
Smart Content = Smart BusinessSmart Content = Smart Business
Smart Content = Smart Business
Seth Grimes
 
Business Social Media - Central CT SIM Meeting
Business Social Media - Central CT SIM MeetingBusiness Social Media - Central CT SIM Meeting
Business Social Media - Central CT SIM Meeting
Michael Rawlins
 
Computer Applications and Systems - Workshop V
Computer Applications and Systems - Workshop VComputer Applications and Systems - Workshop V
Computer Applications and Systems - Workshop V
Raji Gogulapati
 
Information Architecture: Putting the "I" back in IT
Information Architecture:  Putting the "I" back in ITInformation Architecture:  Putting the "I" back in IT
Information Architecture: Putting the "I" back in IT
Louis Rosenfeld
 
Minne analytics presentation 2018 12 03 final compressed
Minne analytics presentation 2018 12 03 final   compressedMinne analytics presentation 2018 12 03 final   compressed
Minne analytics presentation 2018 12 03 final compressed
Bonnie Holub
 
Social Media Analytics: The Value Proposition
Social Media Analytics: The Value PropositionSocial Media Analytics: The Value Proposition
Social Media Analytics: The Value Proposition
Content Savvy
 
The power of social media anlaytics
The power of social media anlayticsThe power of social media anlaytics
The power of social media anlaytics
Ajay Ram
 
Advanced Analytics and Data Science Expertise
Advanced Analytics and Data Science ExpertiseAdvanced Analytics and Data Science Expertise
Advanced Analytics and Data Science Expertise
SoftServe
 
Minne analytics presentation 2018 12 03 final compressed
Minne analytics presentation 2018 12 03 final   compressedMinne analytics presentation 2018 12 03 final   compressed
Minne analytics presentation 2018 12 03 final compressed
Bonnie Holub
 
20130427 What's Your Social IQ?
20130427 What's Your Social IQ?20130427 What's Your Social IQ?
20130427 What's Your Social IQ?
BlueMetalInc
 
Big Data & Business Analytics: Understanding the Marketspace
Big Data & Business Analytics: Understanding the MarketspaceBig Data & Business Analytics: Understanding the Marketspace
Big Data & Business Analytics: Understanding the Marketspace
Bala Iyer
 
Social Media Data Analysis and Visualization Tools
Social Media Data Analysis and Visualization ToolsSocial Media Data Analysis and Visualization Tools
Social Media Data Analysis and Visualization Tools
Sayani Majumder
 
Big Data Customer Experience Analytics -- The Next Big Opportunity for You
Big Data Customer Experience Analytics -- The Next Big Opportunity for You Big Data Customer Experience Analytics -- The Next Big Opportunity for You
Big Data Customer Experience Analytics -- The Next Big Opportunity for You
Dr.Dinesh Chandrasekar PhD(hc)
 
Social Media Insights
Social Media InsightsSocial Media Insights
Social Media Insights
Paul Fennemore MSc (Dist)
 

Similar to Knowledge Extraction from Social Media (20)

Business Intelligence and Big Data in Cloud
Business Intelligence and Big Data in CloudBusiness Intelligence and Big Data in Cloud
Business Intelligence and Big Data in Cloud
 
Agile data science
Agile data scienceAgile data science
Agile data science
 
Big Data - Analytics iC-360
Big Data - Analytics iC-360Big Data - Analytics iC-360
Big Data - Analytics iC-360
 
PCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakaraj
PCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakarajPCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakaraj
PCM STFF 2010 AIMS ACCENTURE e mc2 sudhakar kanakaraj
 
Improve Customer Experience Management with Text Analytics - MeaningCloud web...
Improve Customer Experience Management with Text Analytics - MeaningCloud web...Improve Customer Experience Management with Text Analytics - MeaningCloud web...
Improve Customer Experience Management with Text Analytics - MeaningCloud web...
 
Leveraging Persuasive Architecture
Leveraging Persuasive ArchitectureLeveraging Persuasive Architecture
Leveraging Persuasive Architecture
 
Smart Content = Smart Business
Smart Content = Smart BusinessSmart Content = Smart Business
Smart Content = Smart Business
 
Business Social Media - Central CT SIM Meeting
Business Social Media - Central CT SIM MeetingBusiness Social Media - Central CT SIM Meeting
Business Social Media - Central CT SIM Meeting
 
Computer Applications and Systems - Workshop V
Computer Applications and Systems - Workshop VComputer Applications and Systems - Workshop V
Computer Applications and Systems - Workshop V
 
Information Architecture: Putting the "I" back in IT
Information Architecture:  Putting the "I" back in ITInformation Architecture:  Putting the "I" back in IT
Information Architecture: Putting the "I" back in IT
 
Minne analytics presentation 2018 12 03 final compressed
Minne analytics presentation 2018 12 03 final   compressedMinne analytics presentation 2018 12 03 final   compressed
Minne analytics presentation 2018 12 03 final compressed
 
Social Media Analytics: The Value Proposition
Social Media Analytics: The Value PropositionSocial Media Analytics: The Value Proposition
Social Media Analytics: The Value Proposition
 
The power of social media anlaytics
The power of social media anlayticsThe power of social media anlaytics
The power of social media anlaytics
 
Advanced Analytics and Data Science Expertise
Advanced Analytics and Data Science ExpertiseAdvanced Analytics and Data Science Expertise
Advanced Analytics and Data Science Expertise
 
Minne analytics presentation 2018 12 03 final compressed
Minne analytics presentation 2018 12 03 final   compressedMinne analytics presentation 2018 12 03 final   compressed
Minne analytics presentation 2018 12 03 final compressed
 
20130427 What's Your Social IQ?
20130427 What's Your Social IQ?20130427 What's Your Social IQ?
20130427 What's Your Social IQ?
 
Big Data & Business Analytics: Understanding the Marketspace
Big Data & Business Analytics: Understanding the MarketspaceBig Data & Business Analytics: Understanding the Marketspace
Big Data & Business Analytics: Understanding the Marketspace
 
Social Media Data Analysis and Visualization Tools
Social Media Data Analysis and Visualization ToolsSocial Media Data Analysis and Visualization Tools
Social Media Data Analysis and Visualization Tools
 
Big Data Customer Experience Analytics -- The Next Big Opportunity for You
Big Data Customer Experience Analytics -- The Next Big Opportunity for You Big Data Customer Experience Analytics -- The Next Big Opportunity for You
Big Data Customer Experience Analytics -- The Next Big Opportunity for You
 
Social Media Insights
Social Media InsightsSocial Media Insights
Social Media Insights
 

More from Seth Grimes

Recent Advances in Natural Language Processing
Recent Advances in Natural Language ProcessingRecent Advances in Natural Language Processing
Recent Advances in Natural Language Processing
Seth Grimes
 
Creating an AI Startup: What You Need to Know
Creating an AI Startup: What You Need to KnowCreating an AI Startup: What You Need to Know
Creating an AI Startup: What You Need to Know
Seth Grimes
 
NLP 2020: What Works and What's Next
NLP 2020: What Works and What's NextNLP 2020: What Works and What's Next
NLP 2020: What Works and What's Next
Seth Grimes
 
Efficient Deep Learning in Natural Language Processing Production, with Moshe...
Efficient Deep Learning in Natural Language Processing Production, with Moshe...Efficient Deep Learning in Natural Language Processing Production, with Moshe...
Efficient Deep Learning in Natural Language Processing Production, with Moshe...
Seth Grimes
 
From Customer Emotions to Actionable Insights, with Peter Dorrington
From Customer Emotions to Actionable Insights, with Peter DorringtonFrom Customer Emotions to Actionable Insights, with Peter Dorrington
From Customer Emotions to Actionable Insights, with Peter Dorrington
Seth Grimes
 
Intro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AI
Intro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AIIntro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AI
Intro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AI
Seth Grimes
 
Emotion AI
Emotion AIEmotion AI
Emotion AI
Seth Grimes
 
Text Analytics Market Trends
Text Analytics Market TrendsText Analytics Market Trends
Text Analytics Market Trends
Seth Grimes
 
Text Analytics for NLPers
Text Analytics for NLPersText Analytics for NLPers
Text Analytics for NLPers
Seth Grimes
 
Our FinTech Future – AI’s Opportunities and Challenges?
Our FinTech Future – AI’s Opportunities and Challenges? Our FinTech Future – AI’s Opportunities and Challenges?
Our FinTech Future – AI’s Opportunities and Challenges?
Seth Grimes
 
Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...
Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...
Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...
Seth Grimes
 
The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...
The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...
The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...
Seth Grimes
 
Fairness in Machine Learning and AI
Fairness in Machine Learning and AIFairness in Machine Learning and AI
Fairness in Machine Learning and AI
Seth Grimes
 
Classification with Memes–Uber case study
Classification with Memes–Uber case studyClassification with Memes–Uber case study
Classification with Memes–Uber case study
Seth Grimes
 
Aspect Detection for Sentiment / Emotion Analysis
Aspect Detection for Sentiment / Emotion AnalysisAspect Detection for Sentiment / Emotion Analysis
Aspect Detection for Sentiment / Emotion Analysis
Seth Grimes
 
Content AI: From Potential to Practice
Content AI: From Potential to PracticeContent AI: From Potential to Practice
Content AI: From Potential to Practice
Seth Grimes
 
Text Analytics Market Insights: What's Working and What's Next
Text Analytics Market Insights: What's Working and What's NextText Analytics Market Insights: What's Working and What's Next
Text Analytics Market Insights: What's Working and What's Next
Seth Grimes
 
An Industry Perspective on Subjectivity, Sentiment, and Social
An Industry Perspective on Subjectivity, Sentiment, and SocialAn Industry Perspective on Subjectivity, Sentiment, and Social
An Industry Perspective on Subjectivity, Sentiment, and Social
Seth Grimes
 
The Insight Value of Social Sentiment
The Insight Value of Social SentimentThe Insight Value of Social Sentiment
The Insight Value of Social Sentiment
Seth Grimes
 
Text Analytics 2014: User Perspectives on Solutions and Providers
Text Analytics 2014: User Perspectives on Solutions and ProvidersText Analytics 2014: User Perspectives on Solutions and Providers
Text Analytics 2014: User Perspectives on Solutions and Providers
Seth Grimes
 

More from Seth Grimes (20)

Recent Advances in Natural Language Processing
Recent Advances in Natural Language ProcessingRecent Advances in Natural Language Processing
Recent Advances in Natural Language Processing
 
Creating an AI Startup: What You Need to Know
Creating an AI Startup: What You Need to KnowCreating an AI Startup: What You Need to Know
Creating an AI Startup: What You Need to Know
 
NLP 2020: What Works and What's Next
NLP 2020: What Works and What's NextNLP 2020: What Works and What's Next
NLP 2020: What Works and What's Next
 
Efficient Deep Learning in Natural Language Processing Production, with Moshe...
Efficient Deep Learning in Natural Language Processing Production, with Moshe...Efficient Deep Learning in Natural Language Processing Production, with Moshe...
Efficient Deep Learning in Natural Language Processing Production, with Moshe...
 
From Customer Emotions to Actionable Insights, with Peter Dorrington
From Customer Emotions to Actionable Insights, with Peter DorringtonFrom Customer Emotions to Actionable Insights, with Peter Dorrington
From Customer Emotions to Actionable Insights, with Peter Dorrington
 
Intro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AI
Intro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AIIntro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AI
Intro to Deep Learning for Medical Image Analysis, with Dan Lee from Dentuit AI
 
Emotion AI
Emotion AIEmotion AI
Emotion AI
 
Text Analytics Market Trends
Text Analytics Market TrendsText Analytics Market Trends
Text Analytics Market Trends
 
Text Analytics for NLPers
Text Analytics for NLPersText Analytics for NLPers
Text Analytics for NLPers
 
Our FinTech Future – AI’s Opportunities and Challenges?
Our FinTech Future – AI’s Opportunities and Challenges? Our FinTech Future – AI’s Opportunities and Challenges?
Our FinTech Future – AI’s Opportunities and Challenges?
 
Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...
Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...
Preposition Semantics: Challenges in Comprehensive Corpus Annotation and Auto...
 
The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...
The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...
The Ins and Outs of Preposition Semantics:
 Challenges in Comprehensive Corpu...
 
Fairness in Machine Learning and AI
Fairness in Machine Learning and AIFairness in Machine Learning and AI
Fairness in Machine Learning and AI
 
Classification with Memes–Uber case study
Classification with Memes–Uber case studyClassification with Memes–Uber case study
Classification with Memes–Uber case study
 
Aspect Detection for Sentiment / Emotion Analysis
Aspect Detection for Sentiment / Emotion AnalysisAspect Detection for Sentiment / Emotion Analysis
Aspect Detection for Sentiment / Emotion Analysis
 
Content AI: From Potential to Practice
Content AI: From Potential to PracticeContent AI: From Potential to Practice
Content AI: From Potential to Practice
 
Text Analytics Market Insights: What's Working and What's Next
Text Analytics Market Insights: What's Working and What's NextText Analytics Market Insights: What's Working and What's Next
Text Analytics Market Insights: What's Working and What's Next
 
An Industry Perspective on Subjectivity, Sentiment, and Social
An Industry Perspective on Subjectivity, Sentiment, and SocialAn Industry Perspective on Subjectivity, Sentiment, and Social
An Industry Perspective on Subjectivity, Sentiment, and Social
 
The Insight Value of Social Sentiment
The Insight Value of Social SentimentThe Insight Value of Social Sentiment
The Insight Value of Social Sentiment
 
Text Analytics 2014: User Perspectives on Solutions and Providers
Text Analytics 2014: User Perspectives on Solutions and ProvidersText Analytics 2014: User Perspectives on Solutions and Providers
Text Analytics 2014: User Perspectives on Solutions and Providers
 

Recently uploaded

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 

Recently uploaded (20)

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 

Knowledge Extraction from Social Media

  • 1. Who’s Doing What for Whom, and How? The Social Media Analysis Solution Space Seth Grimes @sethgrimes
  • 2. Deconstruction The topic “Knowledge Extraction and Consolidation from Social Media” is comprised of: • Knowledge Extraction. • Knowledge Consolidation. • Social Media. Sentiment, opinion mining, and analysis are involved. I’ll talk about these matters.
  • 3. Deconstruction, 2 My topic: Who’s Doing What for Whom? • Who = Solution providers: researchers, software, services. • What = Social media analysis (SMA), “social business,” analytics-infused advisory services. • For Whom = Business users. • How = Technologies. I’ll talk about these elements as well, starting with the applications, then moving to tech, then to providers.
  • 4. Theses Social Media = Platforms + Networks + Content. Knowledge = Contextualized, interrelated information. Knowledge, in automated settings, must be structured to be usable . Consolidation involves collection, filtering, analysis, reduction, integration, i nference, and presentation… iteratively. “Business is a collection of activities carried on for whatever purpose, be it science, technology, commerce, industry, law, governm ent, defense, et cetera.”
  • 5. Business Questions What are people saying? What’s hot/trending? What are they saying about {topic|person|product} X? ... about X versus {topic|person|product} Y? How has opinion about X and Y evolved? How has opinion correlated with {our|competitors’|general} {news|marketing|sales|events}? What’s behind opinion, the root causes? • (How) Can we link opinions & transactions? • (How) Can we link opinion & intent? Who are opinion leaders?
  • 6. Business Needs How do these factors affect my business? How can answers to these questions help me improve business processes? We have a decision support need and an operational need. We= • Consumers. • Marketers. • Competitors. • Managers.
  • 7. Analysis Approaches In industry settings, we (should) work backward: Mission  Goals  Presentation  Methods & Data • What are your business goals? • What insights will help your reach them? • What data, transformation, and presentations will generate those insights? • For each option, what will it cost and what is it worth: What is the expected/projected ROI? Sometimes we work this way, and sometimes we want to explore…
  • 8. Data, Information & Knowledge “Where America’s Racist Tweets Come From” http://mashable.com/2012/11/11/racist-tweets/
  • 9. Document input and processing Knowledge handling is Desk Set (1957): Computer engineer key Richard Sumner (Spencer Tracy) and television network librarian Bunny Watson (Katherine Hepburn) H.P. Luhn, “A and the "electronic brain" EMERAC. Business Intelligence System,” IBM Journal, October 1958
  • 10. Intelligence Business intelligence (BI) was first defined in 1958: “In this paper, business is a collection of activities carried on for whatever purpose, be it science, technology, commerce, industry, law, government, d efense, et cetera... The notion of intelligence is also defined here... as ‘the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.’” -- Hans Peter Luhn “A Business Intelligence System” IBM Journal, October 1958 Applies to --
  • 13. Incomplete, 2 Personal. Mobile. Knowledge Infused. http://timoelliott.com/blog/2010/10/sap-businessobjects-augmented- explorer-now-available-resources-to-test-it.html
  • 14. What Is Our Vision? Our Goal? The inclusion of social data and social-derived insights (a.k.a. information) in a global knowledge network? The social Semantic Web? The Semantic Social Web? Why extract knowledge from social media? • The academic challenge is interesting but not enough. • We want to create better social-computing experiences. • We want to infuse social into other computing realms.
  • 15. Our Social Knowledge Goal? http://www.cambridgesemantics.com/sema ntic-university/semantic-search-and-the- semantic-web http://img.freebase.com/api/trans/raw/m/02dtnzv “The Semantic Web has been and remains a parallel, incomplete, never-up-to-date subset of the World Wide Web and the databases accessible through it.” (Me, 2010)
  • 16. Business Driven Approaches Pragmatic knowledge structuring. https://developers.facebook.com/docs/opengraph/ <div itemscope itemtype="http://schema.org/Organization"> <span itemprop="name">Google.org (GOOG)</span> Contact Details: <div itemprop="address" itemscope itemtype="http://schema.org/PostalAddress"> Main address: <span itemprop="streetAddress">38 avenue de l'Opera</span> <span itemprop="postalCode">F-75002</span> <span itemprop="addressLocality">Paris, France</span> , </div> Tel:<span itemprop="telephone">( 33 1) 42 68 53 00 </span>, http://open.blogs.nytimes.com/2012/02/16/rnews- Fax:<span itemprop="faxNumber">( 33 1) 42 68 53 01 </span>, E-mail: <span itemprop="email">secretariat(at)google.org</span> is-here-and-this-is-what-it-means/ </div> http://schema.org/Organization
  • 17. Data pipes Business Driven Approaches, 2a
  • 18. Business Driven Approaches, 3 Social media monitoring. http://www.goldbachinteractive.com/current-news/technical-papers/social-media- monitoring-a-small-market-overview-sysomos-radian6-and-more
  • 19. Business Driven Approaches, 3’ Dashboards and engagement consoles.
  • 21. Business Driven, 4 Infographics: Old wine, new bottles. − Static, non-collaborative. + I like narrative.
  • 22. Business Driven Approaches, 5 A Semanticized Web
  • 23. Business Driven, 6 Question Authorities. https://secure.wikimedia.org/wiki pedia/en/wiki/File:Watson_Jeopar dy.jpg
  • 25. Milestones Language+ understanding. • Text, speech, and video. • Narrative, discourse, and argument. Information extraction. Knowledge structuring and integration. Inference; synthesis. Language generation. Conversation; interaction; autonomy. ≈> Convergence, a.k.a. Singularity
  • 26. What does the market say? Free report download via http://altaplana.com/TA2011
  • 27. Users (current & potential) say
  • 28. Important sources What textual information are you analyzing or do you plan to analyze? blogs and other social media (twitter, social- 62% (2011) network sites, etc.) 47% (2009) news articles 41% (2011) 44% (2009) on-line forums 35% (2011) 35% (2009) customer/market surveys 35% (2011) 34% (2009) reviews 30% (2011) 21% (2009) e-mail and correspondence 29% (2011) 36% (2009)
  • 30.
  • 31. Applications Text analytics has applications in – • Intelligence & law enforcement. • Life sciences. • Media & publishing including social-media analysis and contextual advertizing. • Competitive intelligence. • Voice of the Customer: CRM, product management & marketing. • Legal, tax & regulatory (LTR) including compliance. • Recruiting.
  • 32. Online Commerce Text analytics is applied for marketing, search optimization, competitive intelligence. • Analyze social media and enterprise feedback to understand opportunities, threats, trends. • Categorize product and service offerings for on-site search and faceted navigation and to enrich content delivery. • Annotate pages to enhance Web-search findability, ranking. • Scrape competitor sites for offers and pricing. • Analyze social and news media for competitive information.
  • 33. Voice of the Customer Text analytics is applied to enhance customer service and satisfaction. • Analyze customer interactions and opinions – • E-mail, contact-center notes, survey responses. • Forum & blog posting and other social media. • – to – • Address customer product & service issues. • Improve quality. • Manage brand & reputation. • If you can link qualitative information from text you can – • Link feedback to transactions. • Assess customer value. • Understand root causes. • Mine data for measures such as churn likelihood.
  • 34. E-Discovery and Compliance Text analytics is applied for compliance, fraud and risk, and e-discovery. • Regulatory mandates and corporate practices dictate – • Monitoring corporate communications. • Managing electronic stored information for production in event of litigation. • Sources include e-mail (!!), news, social media • Risk avoidance and fraud detection are key to effective decision making • Text analytics mines critical data from unstructured sources. • Integrated text-transactional analytics provides rich insights.
  • 35. Knowledge, Enrichment & Integration Semantics enables join across types and/or sources and/or structures, using meaningful identifiers, to create an ensemble that is greater than the sum of the parts. Interrelate information to represent knowledge. Enrichment and integration involve: • Mappings and transformations. • Aggregation and collection. • All the typical data concerns: cleansing, profiling, consistency, security,…
  • 36. A Big Data analytics architecture (HPCC’s) http://hpccsystems.com/ http://www.geeklawblog.com/2011/12/lexis-advance-platform-launch-two.html
  • 37. Text+ Technology Mashups Text analytics generates semantics to bridge search, BI, and applications, enabling next- generation information systems. Semantic search Information access (search + text) (search + text + BI) Search based Search BI applications Integrated analytics (search + text + (text + BI) apps) Applica- Text analytics tions NextGen (inner circle) CRM, EFM, MR, mar keting, …
  • 38. Social Sources Dealing with social sources requires flexibility, data/con tent sophistication, and timeliness.
  • 39. Sentiment Analysis “Sentiment analysis is the task of identifying positive and negative opinions, emotions, and evaluations.” -- Wilson, Wiebe & Hoffman, 2005, “Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis” “Sentiment analysis or opinion mining is the computational study of opinions, sentiments and emotions expressed in text… An opinion on a feature f is a positive or negative view, attitude, emotion or appraisal on f from an opinion holder.” -- Bing Liu, 2010, “Sentiment Analysis and Subjectivity,” in Handbook of Natural Language Processing
  • 41. Intent Analysis http://sentibet.com/ http://www.aiaioo.com/whitepapers/intention_analysis_use_cases.pdf
  • 42. Complications Sentiment may be of interest at multiple levels. Corpus / data space, i.e., across multiple sources. Document. Statement / sentence. Entity / topic / concept. Human language is noisy and chaotic! Jargon, slang, irony, ambiguity, anaphora, polysemy, synonym y, etc. Context is key. Discourse analysis comes into play. Must distinguish the sentiment holder from the object: “Geithner said the recession may worsen.”
  • 43. Milestones Re-viewed ✔ Language+ understanding. Text, speech, and video. ✖ Narrative, discourse, and argument. ✔ Information extraction. ✔ Knowledge structuring and integration. ? Inference; synthesis. Language generation. Conversation; interaction; autonomy. ≈> Convergence, a.k.a. Singularity
  • 44. Text Tech Initiatives Now and near future. • Broader & deeper international language support. • Sentiment analysis, beyond polarity. Emotions, intent signals. etc. • Identity resolution & profile extraction. Online-social-enterprise data integration. • Semantic data integration, Complex Data. • Speech analytics. • Discourse analysis. Because isolated messages are not conversations. • Rich-media content analytics. • Augmented reality; new human-computer interfaces.
  • 45. A Focus on Information & Applications Now and near future. • Signal detection. Sentiment, emotion, identity, intent. • Semanticized applications. Linkable, mashable, enrichable. • Rich information. Context sensitive, situational. Σ = Sense-making…
  • 46. Primary Solution Considerations Adaptation or specialization: To a business or cultural domain, information type (e.g., text, speech, images) & source (e.g., Twitter, e-mail, news articles). By-user customization possibilities: For instance, via custom taxonomies, rules, lexicons. Sentiment resolution: Aggregate, message, or feature level. (What features? Topics, coreferenced entities?)
  • 47. Primary Considerations, cont. Outputs: E.g., annotated text, models, indicators, dashboards, exploratory data interfaces. Usage mode: As-a-service (via API) or installed/hosted/cloud. Capacity: Volume, performance, throughput. Cost.
  • 48. Software & Platform Options Text-analytics options may be grouped generally. • Installed text-analysis application, whether desktop or server or deployed in-database. • Data mining workbench. • Hosted. • Programming tool. • As-a-service, via an application programming interface (API). • Code library or component of a business/vertical application, for instance for CRM, e-discovery, search. Text analytics is frequently embedded in search or other end-user applications.
  • 49. Analytical Assets (Open Source) >>> import nltk >>> sentence = """At eight o'clock on Thursday morning... Arthur didn't feel very good.""" >>> tokens = nltk.word_tokenize(sentence) >>> tokens ['At', 'eight', "o'clock", 'on', 'Thursday', 'morning', 'Arthur', 'did', "n't", 'feel', 'very', 'good', '.'] >>> tagged = nltk.pos_tag(tokens) >>> tagged[0:6] [('At', 'IN'), ('eight', 'CD'), ("o'clock", 'JJ'), ('on', 'IN'), ('Thursday', 'NNP'), ('morning', 'NN')] http://nltk.org/ tm: Text Mining Package A framework for text mining applications within R.
  • 50. Providers 1 (non-exhaustive) – Human analysis. Converseon (to date). KD Paine Associates. Synthesio. Human crowdsourced: Amazon Mechanical Turk. CrowdFlower.
  • 51. Providers 2 (non-exhaustive) – As-a-service: AlchemyAPI. Converseon ConveyAPI. OpenAmplify. Saplo. Software libraries: GATE LingPipe. Python NLTK. R. RapidMiner.
  • 52. Providers 3 (non-exhaustive) – Financial markets applications. Digital Trowel. Dow Jones. RavenPack. Thomson Reuters NewsScope.
  • 53. Providers 4 (non-exhaustive) – Other-domain applications. Attensity. Clarabridge. Crimson Hexagon. Expert System. IBM. Kana/Overtone. Lexalytics. Medallia. NetBase. OpenText/Nstein. SAP. SAS. Sysomos. WiseWindow.
  • 54. Who’s Doing What for Whom, and How? The Social Media Analysis Solution Space Seth Grimes @sethgrimes