• Like

Operational Intelligence with WSO2 BAM

  • 601 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
601
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
40
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Gaining  Opera+onal   Intelligence  with  WSO2   BAM   Director,  Research,  WSO2  Inc.      Visi6ng  Faculty,  University  of  Moratuwa      Member,  Apache  SoEware  Founda6on      Research  Scien6st,  Lanka  SoEware  Founda6on   Srinath  Perera  
  • 2. About  WSO2   ๏  Global  enterprise,  founded  in  2005   by  acknowledged  leaders  in  XML,   web  services    technologies,   standards    and  open  source   ๏  Provides  only  open  source   plaHorm-­‐as-­‐a-­‐service  for  private,   public  and  hybrid  cloud   deployments   ๏  All  WSO2  products  are  100%  open   source  and  released  under  the   Apache  License  Version  2.0.   ๏  Is  an  Ac+ve  Member  of  OASIS,   Cloud  Security  Alliance,  OSGi   Alliance,  AMQP  Working  Group,   OpenID  Founda+on  and  W3C.   2   ๏  Driven  by  Innova+on   ๏  Launched  first  open  source  API   Management  solu+on  in  2012   ๏  Launched  App  Factory  in  2Q  2013   ๏  Launched  Enterprise  Store  and   first  open  source  Mobile  solu+on   in  4Q  2013  
  • 3. 3   What  WSO2  delivers  
  • 4. Business  Model   4  
  • 5. More  Informa+on  !   ๏  Include  links  to  product   downloads,  white  paper   downloads  ,  etc.   5  
  • 6. Outline   § Connected  Business  and  Big  data  analy+cs   § Big  Data  Technologies  from  WSO2   § BAM  –  Batch  analy+cs   § CEP  –  Real  +me  analy+cs   § Lambda  Architecture  to  combine   § From  your  business  to  insights   § Iden+fy  KPIs   § Calculate  KPIs   § Build  Dashboards     § Add  Drilldown   § Alerts  and  Ac+ons    
  • 7. About  Connected  Business  webinar  series   §  March 05:Introduction to the Connected Business §  March 12: Enterprise Integration made easy with WSO2 ESB §  .. §  April 30: Gaining Operational Intelligence ‪with WSO2 BAM §  May 7: The WSO2 Advantage for a Connected Business
  • 8. Adap+ve  Connected  Business  
  • 9. Connected  Business  
  • 10. Be  Adap+ve   ๏  Capture  business  ac+vity   (iden+fied  by  messages,   transac+on  execu+on,  and  data   state  changes)  and  store  data   points  for  future  analy+cs   ๏  Deliver  automated  no+fica+ons  to   stakeholders  and  systems  based   on  business  ac+vity,  stakeholder   accountability,  and  authority.     ๏  Automa+cally  adapt  business   process  execu+on  based  on  events   and  current  condi+ons  
  • 11. Big  Picture  
  • 12. Collec+ng  Data   ๏  Data  collected  at  sensors  and  sent  to  big  data  system  via   events  or  flat  files   ๏  Event  Streams:  we  name  the  events  by  its  content/   originator     •  Get  data  through     – Point  to  Point   – Event  Bus   •  E.g.  Data  bridge   – a  thrib  based  transport  we   did  that  do  about  400k   events/  sec  
  • 13. Making  Sense  of  Data   ๏  To  know  (what  happened?)   ๏  Basic  analy+cs  +  visualiza+ons  (min,  max,   average,  histogram,  distribu+ons  …  )   ๏  Interac+ve  drill  down   ๏  To  explain  (why)   ๏  Data  mining,  classifica+ons,  building  models,   clustering         ๏  To  forecast     ๏  Neural  networks,  decision  models      
  • 14. Dashboards  and  last  Mile   §  Presenting information o  To end user o  To decision takers o  To scientist §  Interactive exploration §  Sending alerts http://www.flickr.com/photos/ stevefaeembra/3604686097/
  • 15. Big  Data  Architecture  
  • 16. Data  Collec+on   •  Can  receive  events  via   SOAP,  HTTP,  JMS,  ..   •  WSO2  Events  is  highly   op+mized  version   (400K  events  TPS)   •  Default  Agents  and   you  can  write  custom   agents.     Agent agent = new Agent(agentConfiguration); publisher = new AsyncDataPublisher( "tcp://localhost:7612", .. ); StreamDefinition definition = new StreamDefinition(STREAM_NAME, VERSION); definition.addPayloadData("sid", STRING); ... publisher.addStreamDefinition(definition ); ... Event event = new Event(); event.setPayloadData(eventData); publisher.publish(STREAM_NAME, VERSION, event);
  • 17. Business  Ac+vity  Monitor  
  • 18. BAM  Hive  Query   Find  how  much  +me  spent  in  each  cell.     CREATE EXTERNAL TABLE IF NOT EXISTS PlayStream … select sid, ceiling((y+33000)*7/10000 + x/10000) as cell, count(sid) from PlayStream GROUP BY sid, ceiling((y+33000)*7/10000 + x/10000);
  • 19. Complex  Event  Processor  
  • 20. CEP  Query   define partition sidPrt by PlayStream.sid, LocBySecStream.sid from PlayStream#window.timeBatch(1sec) select sid, avg(x) as xMean, avg(y) as yMean, avg(z) as zMean insert into LocBySecStream partition by sidPrt from every e1 = LocBySecStream -> e2 = LocBySecStream [e1.yMean + 10000 > yMean or yMean + 10000 > e1.yMean] within 2sec select e1.sid insert into LongAdvStream partition by sidPrt ; Calculate the mean location of each player every second Detect more than 10m run
  • 21. Lambda  Architecture  
  • 22. Extrac+ng  Opera+onal   Intelligence  from  your   Business  
  • 23. Key  Performance  Indicators   (KPIs)   ๏  To  monitor,  you  need  to  measure   ๏  KPIs  are  the  bojom  line     ๏  For  a  organiza+on  it  may  be  profit,  or  revenue     ๏  For  marke+ng,  it  may  be  LEADs  generated     ๏  They  are  oben  domain  specifics   ๏  For  each  KPI,  we  need  to  see  history,  find  rela+onships,   and  some+me  possible  future  trends.  
  • 24. Many  Stake  Holders,  Many   Views  
  • 25. Decision  Process  
  • 26. Dashboard     ๏  Idea  is  to  get  the  “Overall   idea”  in  a  glance   ๏  Like  your  car  dashboard     ๏  Support  for  personaliza+on,   you  can  build  your  own   dashboard.       ๏  This  is  the  entry  point  for  Drill   down  and  ac+vity  monitoring.  
  • 27. Drill  Down   ๏  Find  the  problem,  and  find  what  caused  it.   ๏  It  is  like  “Finding  CPU  bojleneck”   ๏  Idea  is  to  recursively  find  categories  which  are  responsible   for  the  problem   ๏  E.g.  See  year’s  sales,  then  drill  down  to  a  par+cular  year  to  see  per  month  data.     ๏  See  country  level  data,  and  drill  down  to  see  state  wide  data.     ๏  Ac+vity  monitoring  is  a  key  ac+vity  here      
  • 28. Case  Study:  Sobware  Support  Services   ๏  KPIs   ๏  Number  of  open  Issues     ๏  Mean  and  Max  +me  to  close  a  issue     ๏  SLA  viola+ons     ๏  List  of  long  running  issues     ๏  Issue  by  last  update  +me   ๏  Data     ๏  Collect  data  when  issue  is  created  or  updated  or  closed.     ๏  E.g.  IssueChange(id,  ac+on,  +me,  userID)  
  • 29. From  data  to  KPI   ๏  Group  issues  by  ID  grouping  each  issue  into  one  ac+vity  (Ac+vity  Monitoring)   ๏  Calculate  KPIs  for  each  ac+vity   define partition IssueParition IssueChange.id; from IssueChange[type=“open”] as s ->IssueChange[“closed”] as e select e.time-s.time .. Aggregate  data  in  different  level  to  support  drill  down   ๏  hour,  weekly,  and  monthly   ๏  Ver+cal,  product,    
  • 30. Aggregate  to  support  Drilldown   Dimension 1 Dimension 2
  • 31. Dashboard  (Build  with  UES)   Click to drill down Click to drill down Click to see details about Issues
  • 32. Drill  Down   ๏  Click  and  drill  down  issues  in  each  category     ๏  Can  select  a  issue  and  see  all  ac+vi+es  happened  within  the   issue   ๏  See  who  takes  most  +me  on  issues     ๏  Look  at  day,  week,  month  trends  
  • 33. Analy+cs  Process  
  • 34. Ques+ons?  
  • 35. THANK  YOU!!!   37