Gaining	
  Opera+onal	
  
Intelligence	
  with	
  WSO2	
  
BAM	
  
Director,	
  Research,	
  WSO2	
  Inc.	
  
	
  	
  Visi...
About	
  WSO2	
  
๏  Global	
  enterprise,	
  founded	
  in	
  2005	
  
by	
  acknowledged	
  leaders	
  in	
  XML,	
  
we...
3	
  
What	
  WSO2	
  delivers	
  
Business	
  Model	
  
4	
  
More	
  Informa+on	
  !	
  
๏  Include	
  links	
  to	
  product	
  
downloads,	
  white	
  paper	
  
downloads	
  ,	
  et...
Outline	
  
§ Connected	
  Business	
  and	
  Big	
  data	
  analy+cs	
  
§ Big	
  Data	
  Technologies	
  from	
  WSO2	...
About	
  Connected	
  Business	
  webinar	
  series	
  
§  March 05:Introduction to
the Connected Business
§  March 12: ...
Adap+ve	
  Connected	
  Business	
  
Connected	
  Business	
  
Be	
  Adap+ve	
  
๏  Capture	
  business	
  ac+vity	
  
(iden+fied	
  by	
  messages,	
  
transac+on	
  execu+on,	
  and	
 ...
Big	
  Picture	
  
Collec+ng	
  Data	
  
๏  Data	
  collected	
  at	
  sensors	
  and	
  sent	
  to	
  big	
  data	
  system	
  via	
  
event...
Making	
  Sense	
  of	
  Data	
  
๏  To	
  know	
  (what	
  happened?)	
  
๏  Basic	
  analy+cs	
  +	
  visualiza+ons	
  (...
Dashboards	
  and	
  last	
  Mile	
  
§  Presenting information
o  To end user
o  To decision takers
o  To scientist
§  ...
Big	
  Data	
  Architecture	
  
Data	
  Collec+on	
  
•  Can	
  receive	
  events	
  via	
  
SOAP,	
  HTTP,	
  JMS,	
  ..	
  
•  WSO2	
  Events	
  is	
  h...
Business	
  Ac+vity	
  Monitor	
  
BAM	
  Hive	
  Query	
  
Find	
  how	
  much	
  +me	
  spent	
  in	
  each	
  cell.	
  	
  
CREATE EXTERNAL TABLE IF NOT E...
Complex	
  Event	
  Processor	
  
CEP	
  Query	
  
define partition sidPrt by PlayStream.sid, LocBySecStream.sid
from PlayStream#window.timeBatch(1sec)
sele...
Lambda	
  Architecture	
  
Extrac+ng	
  Opera+onal	
  
Intelligence	
  from	
  your	
  
Business	
  
Key	
  Performance	
  Indicators	
  
(KPIs)	
  
๏  To	
  monitor,	
  you	
  need	
  to	
  measure	
  
๏  KPIs	
  are	
  th...
Many	
  Stake	
  Holders,	
  Many	
  
Views	
  
Decision	
  Process	
  
Dashboard	
  	
  
๏  Idea	
  is	
  to	
  get	
  the	
  “Overall	
  
idea”	
  in	
  a	
  glance	
  
๏  Like	
  your	
  car	...
Drill	
  Down	
  
๏  Find	
  the	
  problem,	
  and	
  find	
  what	
  caused	
  it.	
  
๏  It	
  is	
  like	
  “Finding	
 ...
Case	
  Study:	
  Sobware	
  Support	
  Services	
  
๏  KPIs	
  
๏  Number	
  of	
  open	
  Issues	
  	
  
๏  Mean	
  and	...
From	
  data	
  to	
  KPI	
  
๏  Group	
  issues	
  by	
  ID	
  grouping	
  each	
  issue	
  into	
  one	
  ac+vity	
  (Ac...
Aggregate	
  to	
  support	
  Drilldown	
  
Dimension 1
Dimension 2
Dashboard	
  (Build	
  with	
  UES)	
  
Click to
drill down
Click to
drill down
Click to
see details
about
Issues
Drill	
  Down	
  
๏  Click	
  and	
  drill	
  down	
  issues	
  in	
  each	
  category	
  	
  
๏  Can	
  select	
  a	
  is...
Analy+cs	
  Process	
  
Ques+ons?	
  
THANK	
  YOU!!!	
  
37	
  
Operational Intelligence with WSO2 BAM
Operational Intelligence with WSO2 BAM
Upcoming SlideShare
Loading in …5
×

Operational Intelligence with WSO2 BAM

1,269 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,269
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
72
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Operational Intelligence with WSO2 BAM

  1. 1. Gaining  Opera+onal   Intelligence  with  WSO2   BAM   Director,  Research,  WSO2  Inc.      Visi6ng  Faculty,  University  of  Moratuwa      Member,  Apache  SoEware  Founda6on      Research  Scien6st,  Lanka  SoEware  Founda6on   Srinath  Perera  
  2. 2. About  WSO2   ๏  Global  enterprise,  founded  in  2005   by  acknowledged  leaders  in  XML,   web  services    technologies,   standards    and  open  source   ๏  Provides  only  open  source   plaHorm-­‐as-­‐a-­‐service  for  private,   public  and  hybrid  cloud   deployments   ๏  All  WSO2  products  are  100%  open   source  and  released  under  the   Apache  License  Version  2.0.   ๏  Is  an  Ac+ve  Member  of  OASIS,   Cloud  Security  Alliance,  OSGi   Alliance,  AMQP  Working  Group,   OpenID  Founda+on  and  W3C.   2   ๏  Driven  by  Innova+on   ๏  Launched  first  open  source  API   Management  solu+on  in  2012   ๏  Launched  App  Factory  in  2Q  2013   ๏  Launched  Enterprise  Store  and   first  open  source  Mobile  solu+on   in  4Q  2013  
  3. 3. 3   What  WSO2  delivers  
  4. 4. Business  Model   4  
  5. 5. More  Informa+on  !   ๏  Include  links  to  product   downloads,  white  paper   downloads  ,  etc.   5  
  6. 6. Outline   § Connected  Business  and  Big  data  analy+cs   § Big  Data  Technologies  from  WSO2   § BAM  –  Batch  analy+cs   § CEP  –  Real  +me  analy+cs   § Lambda  Architecture  to  combine   § From  your  business  to  insights   § Iden+fy  KPIs   § Calculate  KPIs   § Build  Dashboards     § Add  Drilldown   § Alerts  and  Ac+ons    
  7. 7. About  Connected  Business  webinar  series   §  March 05:Introduction to the Connected Business §  March 12: Enterprise Integration made easy with WSO2 ESB §  .. §  April 30: Gaining Operational Intelligence ‪with WSO2 BAM §  May 7: The WSO2 Advantage for a Connected Business
  8. 8. Adap+ve  Connected  Business  
  9. 9. Connected  Business  
  10. 10. Be  Adap+ve   ๏  Capture  business  ac+vity   (iden+fied  by  messages,   transac+on  execu+on,  and  data   state  changes)  and  store  data   points  for  future  analy+cs   ๏  Deliver  automated  no+fica+ons  to   stakeholders  and  systems  based   on  business  ac+vity,  stakeholder   accountability,  and  authority.     ๏  Automa+cally  adapt  business   process  execu+on  based  on  events   and  current  condi+ons  
  11. 11. Big  Picture  
  12. 12. Collec+ng  Data   ๏  Data  collected  at  sensors  and  sent  to  big  data  system  via   events  or  flat  files   ๏  Event  Streams:  we  name  the  events  by  its  content/   originator     •  Get  data  through     – Point  to  Point   – Event  Bus   •  E.g.  Data  bridge   – a  thrib  based  transport  we   did  that  do  about  400k   events/  sec  
  13. 13. Making  Sense  of  Data   ๏  To  know  (what  happened?)   ๏  Basic  analy+cs  +  visualiza+ons  (min,  max,   average,  histogram,  distribu+ons  …  )   ๏  Interac+ve  drill  down   ๏  To  explain  (why)   ๏  Data  mining,  classifica+ons,  building  models,   clustering         ๏  To  forecast     ๏  Neural  networks,  decision  models      
  14. 14. Dashboards  and  last  Mile   §  Presenting information o  To end user o  To decision takers o  To scientist §  Interactive exploration §  Sending alerts http://www.flickr.com/photos/ stevefaeembra/3604686097/
  15. 15. Big  Data  Architecture  
  16. 16. Data  Collec+on   •  Can  receive  events  via   SOAP,  HTTP,  JMS,  ..   •  WSO2  Events  is  highly   op+mized  version   (400K  events  TPS)   •  Default  Agents  and   you  can  write  custom   agents.     Agent agent = new Agent(agentConfiguration); publisher = new AsyncDataPublisher( "tcp://localhost:7612", .. ); StreamDefinition definition = new StreamDefinition(STREAM_NAME, VERSION); definition.addPayloadData("sid", STRING); ... publisher.addStreamDefinition(definition ); ... Event event = new Event(); event.setPayloadData(eventData); publisher.publish(STREAM_NAME, VERSION, event);
  17. 17. Business  Ac+vity  Monitor  
  18. 18. BAM  Hive  Query   Find  how  much  +me  spent  in  each  cell.     CREATE EXTERNAL TABLE IF NOT EXISTS PlayStream … select sid, ceiling((y+33000)*7/10000 + x/10000) as cell, count(sid) from PlayStream GROUP BY sid, ceiling((y+33000)*7/10000 + x/10000);
  19. 19. Complex  Event  Processor  
  20. 20. CEP  Query   define partition sidPrt by PlayStream.sid, LocBySecStream.sid from PlayStream#window.timeBatch(1sec) select sid, avg(x) as xMean, avg(y) as yMean, avg(z) as zMean insert into LocBySecStream partition by sidPrt from every e1 = LocBySecStream -> e2 = LocBySecStream [e1.yMean + 10000 > yMean or yMean + 10000 > e1.yMean] within 2sec select e1.sid insert into LongAdvStream partition by sidPrt ; Calculate the mean location of each player every second Detect more than 10m run
  21. 21. Lambda  Architecture  
  22. 22. Extrac+ng  Opera+onal   Intelligence  from  your   Business  
  23. 23. Key  Performance  Indicators   (KPIs)   ๏  To  monitor,  you  need  to  measure   ๏  KPIs  are  the  bojom  line     ๏  For  a  organiza+on  it  may  be  profit,  or  revenue     ๏  For  marke+ng,  it  may  be  LEADs  generated     ๏  They  are  oben  domain  specifics   ๏  For  each  KPI,  we  need  to  see  history,  find  rela+onships,   and  some+me  possible  future  trends.  
  24. 24. Many  Stake  Holders,  Many   Views  
  25. 25. Decision  Process  
  26. 26. Dashboard     ๏  Idea  is  to  get  the  “Overall   idea”  in  a  glance   ๏  Like  your  car  dashboard     ๏  Support  for  personaliza+on,   you  can  build  your  own   dashboard.       ๏  This  is  the  entry  point  for  Drill   down  and  ac+vity  monitoring.  
  27. 27. Drill  Down   ๏  Find  the  problem,  and  find  what  caused  it.   ๏  It  is  like  “Finding  CPU  bojleneck”   ๏  Idea  is  to  recursively  find  categories  which  are  responsible   for  the  problem   ๏  E.g.  See  year’s  sales,  then  drill  down  to  a  par+cular  year  to  see  per  month  data.     ๏  See  country  level  data,  and  drill  down  to  see  state  wide  data.     ๏  Ac+vity  monitoring  is  a  key  ac+vity  here      
  28. 28. Case  Study:  Sobware  Support  Services   ๏  KPIs   ๏  Number  of  open  Issues     ๏  Mean  and  Max  +me  to  close  a  issue     ๏  SLA  viola+ons     ๏  List  of  long  running  issues     ๏  Issue  by  last  update  +me   ๏  Data     ๏  Collect  data  when  issue  is  created  or  updated  or  closed.     ๏  E.g.  IssueChange(id,  ac+on,  +me,  userID)  
  29. 29. From  data  to  KPI   ๏  Group  issues  by  ID  grouping  each  issue  into  one  ac+vity  (Ac+vity  Monitoring)   ๏  Calculate  KPIs  for  each  ac+vity   define partition IssueParition IssueChange.id; from IssueChange[type=“open”] as s ->IssueChange[“closed”] as e select e.time-s.time .. Aggregate  data  in  different  level  to  support  drill  down   ๏  hour,  weekly,  and  monthly   ๏  Ver+cal,  product,    
  30. 30. Aggregate  to  support  Drilldown   Dimension 1 Dimension 2
  31. 31. Dashboard  (Build  with  UES)   Click to drill down Click to drill down Click to see details about Issues
  32. 32. Drill  Down   ๏  Click  and  drill  down  issues  in  each  category     ๏  Can  select  a  issue  and  see  all  ac+vi+es  happened  within  the   issue   ๏  See  who  takes  most  +me  on  issues     ๏  Look  at  day,  week,  month  trends  
  33. 33. Analy+cs  Process  
  34. 34. Ques+ons?  
  35. 35. THANK  YOU!!!   37  

×