Real Time Competitive Marketing Intelligence


Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Real Time Competitive Marketing Intelligence

    1. 1. A System for Real-time Competitive Market Intelligence SIGKDD 02 Edmonton, Alberta, Canada Copyright 2002 ACM Sholom M. Weiss and Naval K. Verma IBM T.J. Watson Research Center, P.O. Box 218, Yorktown Heights, NY 10598, USA sholom @, nverma@
    2. 2. Agenda <ul><li>Introduction </li></ul><ul><li>Related technical research areas </li></ul><ul><li>Methods and procedures </li></ul><ul><li>Lightweight rule induction </li></ul><ul><li>Results and discussion </li></ul>
    3. 3. Introduction <ul><li>Goal </li></ul><ul><ul><li>Detect critical differences in the text written about a company vs. the text for its competitors. </li></ul></ul><ul><li>Real-time market intelligence and competitive analysis. </li></ul><ul><li>Many sources of news that feed in real-time. </li></ul><ul><li>So much information is available on-line and is immediately accessible. </li></ul>
    4. 4. Introduction (Continued) <ul><li>Overall design consists of the following components: </li></ul><ul><ul><li>Real-time crawler. </li></ul></ul><ul><ul><li>conditional document retriever. </li></ul></ul><ul><ul><li>Text analysis techniques that convert the documents to numerical format. </li></ul></ul><ul><ul><li>Rule induction methods for finding patterns in data. </li></ul></ul><ul><ul><li>Display result. </li></ul></ul>
    5. 5. Related technical research areas <ul><li>Researchers: </li></ul><ul><ul><li>Learning rules for extraction of useful data from unstructured information. </li></ul></ul><ul><ul><li>Using an information retrieval score (tf/idf) to rank words and phrases that characterize each web site. </li></ul></ul><ul><li>In this paper: </li></ul><ul><ul><li>The goal is also to learn, not for the purpose of assembling data, but to find patterns in unstructured text that distinguish among competitors. </li></ul></ul><ul><ul><li>Have a real-time crawler </li></ul></ul><ul><ul><li>We apply a complete pattern. The patterns are not just sets of words but more meaningful conjunctions and disjunctions. </li></ul></ul>
    6. 6. Methods and Procedures
    7. 7. Methods and Procedures (Continued.) <ul><li>Crawl the net in real time for articles about the competitors. </li></ul><ul><li>Specify conditions for separating the documents into groups for comparison. </li></ul><ul><li>Transform the text into a numerical form in preparation for applying machine learning methods. </li></ul><ul><li>Apply machine learning methods. ( Decision rule induction methods ) </li></ul><ul><li>Determine interesting word patterns for specific companies. </li></ul>
    8. 8. Methods and Procedures (Continued.) <ul><li>Assembling the documents for a designed group of competitors </li></ul>
    9. 9. Methods and Procedures (Continued.)
    10. 10. Methods and Procedures (Continued.)
    11. 11. Lightweight Rule Induction method (LRI) <ul><li>The method learns compact disjunctive normal form (DNF). </li></ul><ul><ul><li>DNF (Disjunctive Normal Form) 或語範式 </li></ul></ul><ul><ul><ul><li>A wff is a DNF, iff (1) clauses 之間是 disjunctive, 且 (2)clauses 之內是 conjunctive 。 </li></ul></ul></ul><ul><ul><ul><li>例: (P  Q)  (X  Y  Z) </li></ul></ul></ul><ul><li>Figure 5 shows and example of a typical DNF rule generated by LRI. </li></ul><ul><ul><li>In this example, the rule has a length of three with two disjuncts. </li></ul></ul>
    12. 12. Result and discussion <ul><li>Step1: </li></ul><ul><ul><li>Starting with stories dated after September1, 2001 crawl the newswires. </li></ul></ul><ul><ul><li>Collect stories for IBM, Microsoft, Dell, Compaq, and Sun. </li></ul></ul><ul><ul><li>Sample very 15 miniutes and add any new materials. </li></ul></ul><ul><ul><li>Clean and convert to XML. </li></ul></ul><ul><ul><li>Add stories to current data base. </li></ul></ul><ul><li>Step2: </li></ul><ul><ul><li>Indicate conditions for forming analytical groups and labels </li></ul></ul><ul><ul><ul><li>IBM stories: December 1, 2001 ~ December 10, 2001 V.S. December 11~ December 31. </li></ul></ul></ul>
    13. 13. Result and discussion (Continued) <ul><li>Step3: </li></ul><ul><ul><li>Compare using rules for form A or B , where A and B are no more than 2 words each. </li></ul></ul><ul><ul><li>Resulting patterns: </li></ul></ul><ul><ul><ul><li>Service or network </li></ul></ul></ul><ul><ul><ul><li>York or work </li></ul></ul></ul><ul><li>Step4: </li></ul><ul><ul><li>Delete york and invoke a new comparison. </li></ul></ul><ul><ul><li>Resulting patterns: </li></ul></ul><ul><ul><ul><li>Sign or systems is added as the second pattern. </li></ul></ul></ul><ul><ul><li>Display documents and highlight words </li></ul></ul>
    14. 14. Result and discussion (Continued) <ul><li>Step5: New conditions </li></ul><ul><ul><li>IBM vs. Sun for the month of December. </li></ul></ul><ul><li>Step6: Compare </li></ul><ul><ul><li>Resulting patterns: </li></ul></ul><ul><ul><ul><li>IBM: data or sign. </li></ul></ul></ul><ul><li>Step7: New conditions </li></ul><ul><ul><li>IBM to competitors market cap increases vs. IBM market cap decreases, same time period. </li></ul></ul>
    15. 15. Snapshot comparing IBM and Microsoft Newswires