Are You Ready for the Era of Big Data and Extreme Information Management?


Published on

This is the first part of a 5-part series on the Information Challenges facing organizations. A white paper describing these challenges can be found here -

Published in: Business, Technology, Education
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Are You Ready for the Era of Big Data and Extreme Information Management?

  1. The  New  Normal:  Get  Ready   for  the  Era  of  Extreme   Informa7on  Management  John  Mancini  President,  AIIM  @jmancini77  
  2. Giving  Credit  Where  Credit  is  Due…    I  didn’t  make  up  the  term  “Extreme  Informa7on  Management”    I  first  heard  it  used  by  Gartner.    But  I  like  it  a  lot.  
  3. Big  Data  is  not  just  “more  data.”    -­‐-­‐Thornton  May  
  4. The  Chessboard  Fable  
  5. We  are  reaching  the  2nd  half  of  “Moore’s  Chessboard,”  drama7cally  changing  what  informa7on  means  to  our  organiza7ons  and  how  it  must  be  managed.  
  7. Systems  of  Engagement   Social  and   Era   Mainframe   Mini   PC   Internet   Cloud   Systems  of  Record   Years   1960-­‐1975   1975-­‐1992   1992-­‐2001   2001-­‐2009   2010-­‐2015   Typical   A  batch   A  dept   A   An   thing   A  web  page   trans   process   document   interac7on   managed  Best  known   Digital   IBM   Microso`   Google   Facebook   company   Equipment   Social   Content   Image   Document   Content   Microfilm   Business  mgmt  focus   Mgmt   Mgmt   Mgmt   Systems  
  8. Considera*on   Systems  of  Record   Systems  of  Engagement  Focus   Transac7ons   Interac7ons  Governance   Command  &  Control   Collabora7on  Core  Elements   Facts  &  Commitments   Ideas  &  Nuances  Value   Single  Source  of  Truth   Discovery  &  Dialog  Standard   Accurate  &  Complete   Immediate  &  Accessible  Content   Authored   Communal  Primary  Record  Type   Documents   Conversa7ons  Searchability   Easy   Hard  Usability   User  is  trained   User  “knows”    Accessibility   Regulated  &  Contained   Ad  Hoc  &  Open  Reten7on   Permanent   Transient  Policy  Focus   Security  (Protect  Assets)   Privacy  (Protect  Users)  
  9. Systems  of  Engagement  •  For  the  past  decade,  companies  have  been  accumula5ng  data   in  what  we  call  a  system  of  record.    Those  who  survive  going   forward  will  also  have  systems  of  engagement  –   h=p://  -­‐-­‐  start  with  evalua5ng   how  you  can  have  a  relevant  conversa5on  with  each   individual  customer  across  all  channels.    And  insuring  you   have  the  analy5cal  capability  and  the  data  to  support  that   analysis.    That  is  where  the  linkage  is  between  the  system  of   record  data  to  system  of  engagement.  On  the  technology  side,   we  believe  the  future  of  handling  this  volume  lies  in  leveraging   the  capability  of  the  cloud.   •  Yuchon  Lee,  Vice  President,  IBM  
  10. Our  5-­‐Point  Manifesto  1.  Commit  to  the  cloud.  2.  Mobilize  everything.  3.  Make  the  business   social.  4.  Digi7ze  anything  that   moves.  5.  Prepare  for   informa7on   management  on  a   massive  scale.  
  11. According  to  IDC  –  Between  now  and  2020…  •  44X  growth  in   informa7on  •  75X  growth  in   informa7on  “containers”  BUT…  •  1.4X  growth  in  IT   professionals  
  12. The  drama7c  changes  in  the  consumer  space  provide  a  hint  as  to  what  is  coming…  
  13. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  14. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  15. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  16. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  17. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  18. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  19. Source  =  hnp://­‐day-­‐in-­‐the-­‐internet/  
  20. Prepare  for  extreme  informa7on  management.  
  21. We  are  moving  from  the  Systems  of  Record  era  in  which  our  focus  was  on  high-­‐value  informa7on  assets  to  the  Systems  of  Engagement  era  in  which  volume  and  complexity  and  velocity  are  increasingly  drama7cally.  
  22. Value  of  Informa7on  per  Unit  to  Organiza7on   HIGH  DENSITY   Systems  of  Record   1   Managed  via  Structured   tradi7onal  BI  Informa7on   and  Data  i.e.,  “data”   Warehousing  Original  concept  –  Freeform  Dynamics  
  23. Value  of  Informa7on  per  Unit  to  Organiza7on   HIGH  Value/Byte   Systems  of  Record   1   Managed  via  Structured   tradi7onal  BI  Informa7on   and  Data  i.e.,  “data”   Warehousing   2  Unstructured   Currently  Informa7on   unmanaged  i.e.,  “content”   Managed  in   ECM  &  ERM   systems  Original  concept  –  Freeform  Dynamics  
  24. Value  of  Informa7on  per  Unit  to  Organiza7on   HIGH  Value/Byte   LOW  Value/Byte   Systems  of  Record   Systems  of  Engagement   1   3   BIG  DATA   Managed  via  Structured   tradi7onal  BI   Internet  of  things  –  e.g.,  climate  Informa7on   and  Data   data,  transac7on  records,  phone  i.e.,  “data”   Warehousing   GPS  data  –  intelligent,   interconnected,  and  everywhere   Volume,  Velocity,  Variety,  Complexity   2   2.5  quin7llion  bytes/day  Unstructured   Currently   4  Informa7on   unmanaged   BIG  CONTENT  i.e.,  “content”   Social,  images,  audio,  video,  text,   Managed  in   office  apps,  web  traffic,  print   ECM  &  ERM   streams,  email,  documents   systems  Original  concept  –  Freeform  Dynamics  
  25. Considered  overall,  to  what  degree  does  your   organiza7on  exploit  its  informa7on  assets  for   analysis  and  decision  making  purposes?   0%   20%   40%   60%   80%   100%   Structured  data   Unstructured  data   5  Fully   4   3   2   1-­‐Poorly   Unsure  Source:    Online  survey  of  Register  readers,  122  respondents,  first  half  of  November  2011,  Freeform  Dynamics  
  26. The  New  Normal   •  Volume:  Enterprises  are  awash  with  ever-­‐growing  data  of  all  types,  easily   amassing  terabytes—even  petabytes—of  informa7on.   –  Turn  12  terabytes  of  Tweets  created  daily  into  improved  product  sen7ment  analysis   –  Convert  350  billion  meter  readings  per  annum  to  bener  predict  power  consump7on   •  Velocity:  Some7mes  2  minutes  is  too  late.  For  7me-­‐sensi7ve  processes   such  as  catching  fraud,  big  data  must  be  used  as  it  streams  into  your   enterprise  in  order  to  maximize  its  value.   –  Scru7nize  5  million  trade  events  per  day  to  iden7fy  poten7al  fraud   –  Analyze  500  million  call  detail  records  per  day  in  real-­‐7me  to  predict  customer  churn   faster   •  Variety:  Big  data  is  any  type  of  data  -­‐  structured  and  unstructured  data   such  as  text,  sensor  data,  audio,  video,  click  streams,  log  files  and  more.   New  insights  are  found  when  analyzing  these  data  types  together.   –  Use  100’s  of  live  video  feeds  from  surveillance  cameras  to  monitor  points  of  interest   –  Take  advantage  of  the  80%  data  growth  in  images,  video  and  documents  to  improve   customer  sa7sfac7on  Source  =  IBM  
  27. The  New  Normal   •  The  vast  majority  of  the  world’s  informa7on  is   unstructured.   •  Unstructured  informa7on  growing  15X  faster   than  structured.   •  Raw  compu7ng  power  growing  so  fast  that  an   off-­‐the-­‐shelf  box  approaching  the  compu7ng   power  of  a  super  computer  5  years  ago.   •  “Democra7za7on”  of  informa7on  access.  Source  =  Understanding  Big  Data:    Analy5cs  for  Enterprise  Class  Hadoop  and  Streamng  Data  
  28. Irra7onal  thinking  •  Get  rid  of  as  much  as   •  Save  everything  that  you   you  can:   can:   –  Li7ga7on  risk   –  Might  need  it  “someday”   –  Compliance  risk   –  Poten7al  aggregated  value   –  Storage  cost   –  Disposi7on  uncertainty      High  Value/Byte     Low  Value/Byte  
  29. Welcome  to  the  Era  of  Extreme  Informa7on  
  30. 5  Things  to  Remember  About  Extreme  Informa7on  
  31. #1:  Most  organiza7ons  do  not  have  the  basics  in  place.  
  32. •  61%  report  “content  chaos”  re  unstructured  informa7on.  •  80%  have  no  extended  search  across  mul7ple  repositories.  •  For  70%,  harder  to  find  your  own  stuff  than  stuff  on  the  web.  
  33. #2:  No  one  except  “the  industry”  cares  about  “structured”  vs.  “unstructured.”   61%  would  find  it  “very  useful”  to  link  structured  and   unstructured  datasets.  
  34. #3:  Manual  records  management  is  dead.  
  35. •  66%  have  an  informa7on   management  strategy,  but   only  22  percent  use  it.   •  79%  have  an  informa7on   reten7on  policy,  but  only  32   percent  enforce  it.     •  58%  say  that  a  single   enterprise  records   management  model   underlying  all  content   systems  is  their  goal,  yet   only  9%  have  achieved  this.  Source:  AIIM,  Process  Revolu7on:  Moving  Your  Business  from  Paper  to  PC  to  Tablet  
  36. #4:  The  future  is  in  metadata.  
  37. •  “The  solu7on  to  the   over  abundance  of   informa7on  is  more   informa7on.”   –  David  Weinberger  •  “Data  that  is  seman7c   means  exactly  the  same   thing  to  any  system  or   person  who  uses  it.”   –  David  Siegel  
  38. The  New  Normal  •  The  standardiza7on  of  data  sets  across  industries,  the   separa7on  of  data  from  its  descrip7on,  and  the  exposure  of   this  informa7on  in  the  cloud  create  enormous   opportuni7es.  •  We  can  now  analyze  problems  that  were  previously   undiges7ble  do  to  the  sheer  scale  of  compu7ng  power   required  to  address  them.   –  The  cloud,  HADOOP,  and  MapReduce  driving  division  of  vast   data  into  small  pieces  and  parsing  compu7ng  across  large   numbers  of  computers.  •  We  can  now  solve  the  metadata  problem  (i.e.,  there  is   none!)  for  vast  landfills  of  unstructured  informa7on   –  We  can  now  use  seman7c  technology  to  apply  metadata  where   it  didn’t  previously  exist.    
  39. For  example…  
  40. Individual  Paper  records   A  Short  History  •  Copied  and  aggregated  paper   of  Financial  •  Manual  compliance  and  “reading  rooms”   Repor7ng…   Individual  Computerized  records   •  Aggregated  computerized  records   •  Spot  audits  and  online  viewing  via  EDGAR   Separa7on  of  data  from  viewing   •  XBRL  standards  –  values,  tags,  dic7onaries   •  Internal  process  standardiza7on   Industry  and  regulatory  standardiza7on   •  Adop7on  as  GAAP   •  Mandated  by  110  countries   Standardized  data  moves  to  the  cloud   •  Automated  compliance  and  availability   •  Big  data  analy7c  opportuni7es  Source  =  Pull:  The  Power  of  the  Seman7c  Web  
  41. Mining  social   streams…   for  predic7ons   about  the  next   hit…  
  42. Mining  social  streams…for  the  best  food  
  43. Mining  social  streams…for  drug  info  
  44. Using  text  analy7cs  to  mo7vate  voters  
  45. Big  Data  Process  Applica7ons   •  Financial  Services   •  U7li7es   –  Fraud  detec7on   –  Weather  analysis   –  360°  View  of  the  Customer   –  Smart  grid  management   •  Transporta7on   •  Intelligence   –  Logis7cs  op7miza7on   –  System  Log  Analysis   –  Traffic  conges7on   –  Cybersecurity   •  Health  &  Life  Sciences   •  Retail   –  Epidemic  early  warning   –  360°  View  of  the  Customer   –  ICU  monitoring   –  Real-­‐7me  promo7ons   •  Telecommunica7ons   •  Law  Enforcement   –  Geomapping  /  marke7ng   –  Mul7modal  surveillance   –  Network  monitoring   –  Cyber  security  detec7on  Source  =  IBM  
  46. The  combina7on   This  revolu7on  of  seman7cs  and   will  once  and  for  accessibility  of   all  require  the  data  in  the  cloud   elimina7on  of  is  revolu7onary.   paper  and  Across  industries   dictate  the   management  of  Across   unstructured  geography   informa7on   assets.  
  47. #5:  We  need  T-­‐Shaped  people  to  address  this  “extreme  informa7on”  opportunity.  
  48. The  emerging  informa7on  professional  •  The  vast  majority  of  organiza5ons  see  the  need  to  manage   informa5on  as  an  enterprise  resource  rather  than  in  separate   "silos,"  departments  or  systems,  but  they  dont  know  how  to   begin  to  address  the  challenge,  as  it  is  so  large...  •  Professional  roles  focused  on  informa5on  management  will  be   different  to  that  of  established  IT  roles.    •  An  "informa5on  professional"  will  not  be  one  type  of  role  or   skill  set,  but  will  in  fact  have  a  number  of  specializa5ons.   –  Deb  Logan  and  Regina  Casonata,  Gartner  
  49. Who  are  these  people?   IT  Legal  professional   Risk/Liability  Focus   Records  Manager   Digital  Archivist   Business  Process  Owners   Professionals   Informa7on   Value  Focus   Business  Analyst   Knowledge  Manager   Informa7on/Data  Scien7st   Ent  Informa7on  Manager   Governance  Focus   Info/Data  Stewards   Ent  Informa7on  Architect   Social  Focus   Informa7on  Curators   Community  Managers  Most  roles  from  Deb  Logan  and  Regina  Casonata,  Gartner  
  50. We  need  T-­‐Shaped  Professionals   BROAD   DEEP  White  paper  here  –    hnp://    Free  prac*ce  exam/assessment  -­‐-­‐    hnp://­‐prac7ce-­‐exam