• Like

Operational Intelligence with WSO2 BAM

Uploaded on


More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Gaining  Opera+onal   Intelligence  with  WSO2   BAM   Director,  Research,  WSO2  Inc.      Visi6ng  Faculty,  University  of  Moratuwa      Member,  Apache  SoEware  Founda6on      Research  Scien6st,  Lanka  SoEware  Founda6on   Srinath  Perera  
  • 2. About  WSO2   ๏  Global  enterprise,  founded  in  2005   by  acknowledged  leaders  in  XML,   web  services    technologies,   standards    and  open  source   ๏  Provides  only  open  source   plaHorm-­‐as-­‐a-­‐service  for  private,   public  and  hybrid  cloud   deployments   ๏  All  WSO2  products  are  100%  open   source  and  released  under  the   Apache  License  Version  2.0.   ๏  Is  an  Ac+ve  Member  of  OASIS,   Cloud  Security  Alliance,  OSGi   Alliance,  AMQP  Working  Group,   OpenID  Founda+on  and  W3C.   2   ๏  Driven  by  Innova+on   ๏  Launched  first  open  source  API   Management  solu+on  in  2012   ๏  Launched  App  Factory  in  2Q  2013   ๏  Launched  Enterprise  Store  and   first  open  source  Mobile  solu+on   in  4Q  2013  
  • 3. 3   What  WSO2  delivers  
  • 4. Business  Model   4  
  • 5. More  Informa+on  !   ๏  Include  links  to  product   downloads,  white  paper   downloads  ,  etc.   5  
  • 6. Outline   § Connected  Business  and  Big  data  analy+cs   § Big  Data  Technologies  from  WSO2   § BAM  –  Batch  analy+cs   § CEP  –  Real  +me  analy+cs   § Lambda  Architecture  to  combine   § From  your  business  to  insights   § Iden+fy  KPIs   § Calculate  KPIs   § Build  Dashboards     § Add  Drilldown   § Alerts  and  Ac+ons    
  • 7. About  Connected  Business  webinar  series   §  March 05:Introduction to the Connected Business §  March 12: Enterprise Integration made easy with WSO2 ESB §  .. §  April 30: Gaining Operational Intelligence ‪with WSO2 BAM §  May 7: The WSO2 Advantage for a Connected Business
  • 8. Adap+ve  Connected  Business  
  • 9. Connected  Business  
  • 10. Be  Adap+ve   ๏  Capture  business  ac+vity   (iden+fied  by  messages,   transac+on  execu+on,  and  data   state  changes)  and  store  data   points  for  future  analy+cs   ๏  Deliver  automated  no+fica+ons  to   stakeholders  and  systems  based   on  business  ac+vity,  stakeholder   accountability,  and  authority.     ๏  Automa+cally  adapt  business   process  execu+on  based  on  events   and  current  condi+ons  
  • 11. Big  Picture  
  • 12. Collec+ng  Data   ๏  Data  collected  at  sensors  and  sent  to  big  data  system  via   events  or  flat  files   ๏  Event  Streams:  we  name  the  events  by  its  content/   originator     •  Get  data  through     – Point  to  Point   – Event  Bus   •  E.g.  Data  bridge   – a  thrib  based  transport  we   did  that  do  about  400k   events/  sec  
  • 13. Making  Sense  of  Data   ๏  To  know  (what  happened?)   ๏  Basic  analy+cs  +  visualiza+ons  (min,  max,   average,  histogram,  distribu+ons  …  )   ๏  Interac+ve  drill  down   ๏  To  explain  (why)   ๏  Data  mining,  classifica+ons,  building  models,   clustering         ๏  To  forecast     ๏  Neural  networks,  decision  models      
  • 14. Dashboards  and  last  Mile   §  Presenting information o  To end user o  To decision takers o  To scientist §  Interactive exploration §  Sending alerts http://www.flickr.com/photos/ stevefaeembra/3604686097/
  • 15. Big  Data  Architecture  
  • 16. Data  Collec+on   •  Can  receive  events  via   SOAP,  HTTP,  JMS,  ..   •  WSO2  Events  is  highly   op+mized  version   (400K  events  TPS)   •  Default  Agents  and   you  can  write  custom   agents.     Agent agent = new Agent(agentConfiguration); publisher = new AsyncDataPublisher( "tcp://localhost:7612", .. ); StreamDefinition definition = new StreamDefinition(STREAM_NAME, VERSION); definition.addPayloadData("sid", STRING); ... publisher.addStreamDefinition(definition ); ... Event event = new Event(); event.setPayloadData(eventData); publisher.publish(STREAM_NAME, VERSION, event);
  • 17. Business  Ac+vity  Monitor  
  • 18. BAM  Hive  Query   Find  how  much  +me  spent  in  each  cell.     CREATE EXTERNAL TABLE IF NOT EXISTS PlayStream … select sid, ceiling((y+33000)*7/10000 + x/10000) as cell, count(sid) from PlayStream GROUP BY sid, ceiling((y+33000)*7/10000 + x/10000);
  • 19. Complex  Event  Processor  
  • 20. CEP  Query   define partition sidPrt by PlayStream.sid, LocBySecStream.sid from PlayStream#window.timeBatch(1sec) select sid, avg(x) as xMean, avg(y) as yMean, avg(z) as zMean insert into LocBySecStream partition by sidPrt from every e1 = LocBySecStream -> e2 = LocBySecStream [e1.yMean + 10000 > yMean or yMean + 10000 > e1.yMean] within 2sec select e1.sid insert into LongAdvStream partition by sidPrt ; Calculate the mean location of each player every second Detect more than 10m run
  • 21. Lambda  Architecture  
  • 22. Extrac+ng  Opera+onal   Intelligence  from  your   Business  
  • 23. Key  Performance  Indicators   (KPIs)   ๏  To  monitor,  you  need  to  measure   ๏  KPIs  are  the  bojom  line     ๏  For  a  organiza+on  it  may  be  profit,  or  revenue     ๏  For  marke+ng,  it  may  be  LEADs  generated     ๏  They  are  oben  domain  specifics   ๏  For  each  KPI,  we  need  to  see  history,  find  rela+onships,   and  some+me  possible  future  trends.  
  • 24. Many  Stake  Holders,  Many   Views  
  • 25. Decision  Process  
  • 26. Dashboard     ๏  Idea  is  to  get  the  “Overall   idea”  in  a  glance   ๏  Like  your  car  dashboard     ๏  Support  for  personaliza+on,   you  can  build  your  own   dashboard.       ๏  This  is  the  entry  point  for  Drill   down  and  ac+vity  monitoring.  
  • 27. Drill  Down   ๏  Find  the  problem,  and  find  what  caused  it.   ๏  It  is  like  “Finding  CPU  bojleneck”   ๏  Idea  is  to  recursively  find  categories  which  are  responsible   for  the  problem   ๏  E.g.  See  year’s  sales,  then  drill  down  to  a  par+cular  year  to  see  per  month  data.     ๏  See  country  level  data,  and  drill  down  to  see  state  wide  data.     ๏  Ac+vity  monitoring  is  a  key  ac+vity  here      
  • 28. Case  Study:  Sobware  Support  Services   ๏  KPIs   ๏  Number  of  open  Issues     ๏  Mean  and  Max  +me  to  close  a  issue     ๏  SLA  viola+ons     ๏  List  of  long  running  issues     ๏  Issue  by  last  update  +me   ๏  Data     ๏  Collect  data  when  issue  is  created  or  updated  or  closed.     ๏  E.g.  IssueChange(id,  ac+on,  +me,  userID)  
  • 29. From  data  to  KPI   ๏  Group  issues  by  ID  grouping  each  issue  into  one  ac+vity  (Ac+vity  Monitoring)   ๏  Calculate  KPIs  for  each  ac+vity   define partition IssueParition IssueChange.id; from IssueChange[type=“open”] as s ->IssueChange[“closed”] as e select e.time-s.time .. Aggregate  data  in  different  level  to  support  drill  down   ๏  hour,  weekly,  and  monthly   ๏  Ver+cal,  product,    
  • 30. Aggregate  to  support  Drilldown   Dimension 1 Dimension 2
  • 31. Dashboard  (Build  with  UES)   Click to drill down Click to drill down Click to see details about Issues
  • 32. Drill  Down   ๏  Click  and  drill  down  issues  in  each  category     ๏  Can  select  a  issue  and  see  all  ac+vi+es  happened  within  the   issue   ๏  See  who  takes  most  +me  on  issues     ๏  Look  at  day,  week,  month  trends  
  • 33. Analy+cs  Process  
  • 34. Ques+ons?  
  • 35. THANK  YOU!!!   37