Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Unsung Hero of Big Data in Manufacturing: Unstructured Content


Published on

Presented by: Rik Tamm-­‐Daniels, VP Technology, Attivio

TIBCO Spotfire and Teradata: First to Insight, First to Action; Warehousing, Analytics and Visualizations for the High Tech Industry Conference
July 22, 2013 The Four Seasons Hotel Palo Alto, CA

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

The Unsung Hero of Big Data in Manufacturing: Unstructured Content

  1. 1. The  Unsung  Hero  of  Big  Data   in  Manufacturing:     Unstructured  Content   Rik  Tamm-­‐Daniels,  VP  Technology   TwiBer:  @riktammdaniels  
  2. 2. Proprietary   Big  Data   Big  Data  encompasses  geEng  insight  and  making  smarter   decisions  from  any  informaFon  that  is  not  easily  tapped  using   tradiFonal  BI  and  AnalyFcs  technology  stacks.   Source:  'Big  Data'  Is  Only  the  Beginning  of  Extreme  Informa7on  Management,  April  7,  2011,  Gartner  Group  
  3. 3. Proprietary   Big  Data   Structured   Data   Unstructured   Data   Unstructured   Data   Unstructured   Content   Beyond  Structured  vs.  Unstructured   <xyz>   </xyz>   1-­‐2-­‐2013  abc/bed/fnd  87  998   1-­‐2-­‐2013  abc/bed/fnd  87  998   1-­‐2-­‐2013  abc/bed/fnd  87  998   1-­‐2-­‐2013  abc/bed/fnd  87  998     57°   64°   71°   120°   Understanding  the  nature  of  data  and  how  to  get  insight  from  it  is   fundamental  to  succeeding  with  Big  Data  
  4. 4. Proprietary   Unstructured  Data   •  Brings  the  promise  of  inferring  customer   intent,  idenFfying  paBerns  of  behavior  and   predicFng  future  acFons   •  But  gaining  this  insight  requires  large  amounts   of  raw  data,  much  of  which  is  noise,  to  get   what  amounts  to  indirect  insight   1000  rows  of  impression  logs  to  get  1   click-­‐thru;  6  months  of  click  data  to   be  able  to  develop  staFsFcally   meaningful  user  segments  
  5. 5. Proprietary   Unstructured  Content   •  Consider  this  CRM  case:   Customer:  Joe  Customer   Product:  Mobile  Phone  X1000   Summary:  BaBery  is  dying  aber  only  1  hour  of  use   Date:  3-­‐1-­‐2013  01:35:00   Comments:     I  just  bought  the  new  X1000  because  I  saw  it  had  fantasFc  reviews   online,  but  aber  only  one  week,  the  baBery  dies  aber  just  an  hour  of   use.    This  is  completely  unacceptable  and  I’ve  been  unable  to  get  help   from  ACME  mobile  where  I  bought  the  phone.    They  keep  telling  me   that  depending  on  the  apps  I’m  running,  this  might  be  expected.    I’m   only  running  the  basic  pre-­‐installed  apps!    Also,  I’ve  noFced  that  the   screen  is  always  super  bright  and  starts  to  hurt  my  eyes  aber  20   minutes  of  use.  I  would  really  appreciate  it  if  you  could  send  me  a  new   phone  or  tell  me  how  to  fix  mine.  
  6. 6. Proprietary   What  does  this  single  CRM  case  tell  us?   Customer:  Joe  Customer   Product:  Mobile  Phone  X1000   Summary:  BaBery  is  dying  aber  only  1  hour  of  use   Date:  3-­‐1-­‐2013  01:35:00   Comments:     I  just  bought  the  new  X1000  because  I  saw  it  had  fantasFc  reviews  online,  but  aber   only  one  week  of  use,  the  baBery  dies  aber  just  an  hour.    This  is  completely   unacceptable  and  I’ve  been  unable  to  get  help  from  ACME  mobile  where  I  bought  the   phone.    They  keep  telling  me  that  depending  on  the  apps  I’m  running,  this  might  be   expected.    I’m  only  running  the  basic  pre-­‐installed  apps!  This  is  completely   unacceptable  from  a  retailer  selling  your  products.  Also,  I’ve  noFced  that  the  screen  is   always  super  bright  and  starts  to  hurt  my  eyes  aber  20  minutes  of  use.  I  would  really   appreciate  it  if  you  could  send  me  a  new  phone  or  tell  me  how  to  fix  mine.   Customer  email  Customer  name   PosiFve   SenFment   NegaFve   SenFment   Retail  Outlet   PotenFal   Liability   Key  concept   Industry  term  
  7. 7. Proprietary   What’s  the  business  value  of  this  CRM  case?   •  When  a  help  desk  user  gets  assigned  this  case,   their  view  could  be:     Customer:  Joe  Customer   Email  Address:   Summary:  …   Case:   I  just  bought  the  new  X1000  because  I  saw  it  had  fantasFc   reviews  online,  but  aber  only  one  week  of  use,  the  baBery   dies  aber  just  an  hour  of  use.    This  is  completely   unacceptable  and  I’ve  been  unable  to  get  help  from  ACME   mobile  where  I  bought  the  phone.    They  keep  telling  me   that  depending  on  the  apps  I’m  running,  this  might  be   expected.    I’m  only  running  the  basic  pre-­‐installed  apps!     Also,  I’ve  noFced  that  the  screen  is  always  super  bright   and  starts  to  hurt  my  eyes  aber  20  minutes  of  use.  I  would   really  appreciate  it  if  you  could  send  me  a  new  phone  or   tell  me  how  to  fix  mine.     Related  CRM  cases:     Screen  and  baBery  issue   …     Screen  too  bright   ….   Issue  trend  line   for  screen  and   baBery  issues   for  the  X1000  
  8. 8. Proprietary   What  could  the  value  of  a  lot  of  CRM  cases  be?   LifeFme  Customer  Value   %  NegaFve  Customer   Service  InteracFons   ACME  Mobile   PhoneX   CoolPhones   Joe’s  Phones   BigBell   MallMobile       Percent  /  per  retailer   BaBery  dies   Apps  I’m  running   Screen     Super  Bright  
  9. 9. Proprietary   Unstructured  Content  in  Customer   Experience  Management   Every  item  from  every  one   of  these  channels  is  your   customer  telling  you   directly  what  they  like,   what  they  don’t  like,  where   they  would  like  to  see  you   take  your  products,  etc…  
  10. 10. Proprietary   Unstructured  Content  is  Everywhere!   CRM  case   notes   Email   Surveys   Warranty   claims   Product   manuals   Maintenance   reports   Online   reviews   Social  Media  
  11. 11. Unlocking  Unstructured   Content  
  12. 12. Proprietary   Unlocking  Unstructured  Content   •  Fundamental:  Structure  the  Unstructured   •  For  Unstructured  Content  this  is  done  through   Text  AnalyFcs:   – SenFment  Analysis   – ClassificaFon   – EnFty  ExtracFon   – Key  Concept  Analysis   – Taxonomies  and  Ontologies     New  Dimensions/New  Insights   Key   Concept   Analysis   EnFty   ExtracFon   SenFment   Analysis  
  13. 13. Proprietary   Enterprise  Text  AnalyFcs  Spectrum   Taxonomies/   Ontologies   Gazeteer/   Dic.onary-­‐ Driven  En.ty   Extrac.on   Pa@ern   Based  En.ty   Extrac.on   Machine-­‐ Learning:   Sen.ment  &   Classifica.on   En.ty   Extrac.on   Language   Model-­‐Based   Keyphrase   Analysis   Directed   Discovery  
  14. 14. Proprietary   Enterprise  Text  AnalyFcs  in  PracFce   •  IteraFon,  iteraFon,   iteraFon  –  the  faster   you  can  iterate,  the   greater  the  ROI   Text  Analy.cs  are  
  15. 15. Proprietary   Enterprise  Text  AnalyFcs  Best  PracFces   •  Domain-­‐specific  analyFcs   •  Extensible/flexible  frameworks   •  Directed  and  Discovery  text  analyFcs   •  Agile  data  environments   –  Text  analyFc  data  can  produce  high-­‐degrees  of  cardinality   –  Metadata  will  be  variable  by  “row”/“object”  
  16. 16. The  Big  Picture  
  17. 17. Proprietary   Big  Data  Maturity  Model   Level  0:   Current   State   Level  1:  Single   source  of  Big   Data  Analyzed   Level  2:   MulFple   types  of  Big   Data  sources   Analyzed   Level  3:   Unified  Big   Data   Architecture   Looking  at  a  single  source  of   Big  Data:  unstructured  data   or  unstructured  content   Looking  at  mulFple   sources  of  Big  Data:   unstructured  data  and   unstructured  content   Single  point  of  access   for  all  Big  Data  access   and  analysis  with   variety  of  access  modes   to  support  variety  of   business  cases   Structured  data  analysis:   EDW  +  BI   18  
  18. 18. Proprietary   •  Unified  informaFon  access  plauorms  provide  single  point  of   access  to  informaFon  from  mulFple  sources,  integraFng   and  finding  relaFonships  across  sources   –  Efficiently  combines  features  of  database,  business   intelligence  and  search  technologies  in  a  single   architecture  in  real-­‐Fme   Unified  InformaFon  Access   Unified  Informa.on  Access   Note:  IDC  June  2012.   Unified  Informa.on   Access  &  Analysis   Content   Analy.cs   Databases   Search  and   Discovery   Decision   Management   Business   Intelligence           Data   Warehouses   Unified  Informa.on  Access  &  Analysis   •  Serves  as  foundaFon  for  new  informaFon  management  and   access  stack  for  the  enterprise   •  May  replace  data  warehouses  if  applicaFons  require  quick  ad   hoc  access  to  collecFons  of  heterogeneous  informaFon   •  Will  eventually  replace  the  tradiFonal  enterprise  search   engine   Key  Elements  
  19. 19. Customer  Experience   Management  for  Complex   Device  Manufacturers  
  20. 20. Proprietary   Case  Study       Global  Manufacturer   •  Create new BI platform to proactively manage customer engine fleets at new level of breadth and depth of detail beyond just data trends •  Greatly improve efficiency, customer satisfaction and repeat business •  Analyze Everything platform for complete agile BI: integrates, correlates and presents data and content, with no advance data modeling required: •  Engine sensor data, generated in “Big Data” volumes •  Service status data, quality metrics, CRM and other databases •  Customer case management notes •  Engine maintenance system notes by service technicians •  Supports BI tools with native SQL support & ODBC/JDBC connectivity •  BI pilot completed in just 5 weeks – “a new standard for BI time to market” •  Managers analyze and discover new correlations between changes in engine KPIs, sensor data, recurring key phrases from service notes & more •  New insights into root causes behind service issues – not just the numbers •  “No data left behind…Time from ‘data to decision’ drastically reduced” ProblemWhyAIE?Results
  21. 21. Proprietary           SEARCH API SQL over ODBC/JDBC QUERY/RESPONSE WORKFLOWS INGESTION WORKFLOWS CONTENT API TIBCO Spotfire DATA & CONTENT CONNECTORS UNIVERSAL INDEX     EDW Customer Service Email CRM OperaFonal  Events   Device  Generated  Data   Complex Event Processing Engine Maintenance Reports Textual  data  is   enriched  with  text   analyFcs  (senFment,   keyphrases,  enFty   extracFon)   CUSTOMERS  DEVICES  OWNED  EMAILS  CRM  CASES  OPERATIONAL   EVENTS   KPIs   Dashboards  using  TIBCO   Spouire  contain  a  mix  of  in-­‐ memory  tables  loaded  from   AIE  with  on-­‐demand  detail   drill  down   Logical  Tables  from   the  BI  tool   perspecFve,  AIE   does  not  store  data   in  physical  tables  
  22. 22. TIBCO  Spouire  and   AEvio  AIE  
  23. 23. Proprietary   Benefits  of  combining  AEvio  and  TIBCO  Spouire     VisualizaFon  of  unstructured  content  and  data  that  does  not  currently  exist   in  the  BI  stack,  providing  criFcal  business  process  and  analysis  context   Rich  Text  AnalyFcs  to  gain  insight  from  unstructured  content  that  can   be  visualized  in  Spouire   Full-­‐text  search  within  Spouire  Dashboards  -­‐  providing  a  complete  Data   Discovery  experience   Agile,  complete,  UIA  technology  stack  
  24. 24. Proprietary   Spotfire Server Extracted  or     On-­‐Demand  via  JDBC   AIE  and  TIBCO  Spouire  Reference  Architecture   Direct   via  ODBC   Spotfire