Big	
  Brother	
  for	
  Enterprises:	
  	
  
Social	
  Media	
  Stream	
  Use	
  Cases	
  

Samisa	
  Abeysinghe	
  
VP	
...
About	
  the	
  Presenter	
  
๏ 

๏ 

2	
  

Samisa	
  Abeysinghe	
  
VP	
  Developer	
  Evangelism	
  
samisa@wso2.com	
 ...
About	
  WSO2	
  
๏ 
๏ 

๏ 

Global	
  enterprise,	
  founded	
  in	
  2005	
  
by	
  acknowledged	
  leaders	
  in	
  XML...
What	
  WSO2	
  Deliver	
  
4	
  
NSA like Monitoring for
Your Enterprise
๏ Analyze volumes of data 	

๏ Address correlation complexities in analytics	

๏ O...
Why should I bother?
๏ Deal with high volume (terra bites) of information 	

๏ In order to make decisions 	

๏ Real time &...
WSO2 :ONLY COMPLETE
& INTEGRATED Platform

๏ Complete and integrated for 	


๏ Data capture	

๏ Analysis: both real-time a...
Big Data: Strategy

8	
  
Responsive and Effective

9	
  
WSO2 Big Data Analytics
Platform for Your Enterprise

WSO2	
  Big	
  Data	
  Analy0cs	
  
Pla4orm	
  for	
  Your	
  Enterp...
Key	
  Elements	
  
Data	
  
CollecXon	
  

Taking	
  
AcXon	
  

Data	
  
Analysis	
  

Data	
  
VisualizaXon	
  

11	
  
WSO2 Big Data Analytics
Platform for Your Enterprise

Data Collection	

12	
  

Data Analysis	


Data Visualization &
Taki...
Use	
  Case	
  Scenario:	
  
Analyzing	
  Twiaer	
  Streams	
  	

	

	


13	
  
Social	
  media	
  …	
  !	
  
o 

Test	
  
o 

	
  	
  

Test	
  
Processing	
  Social	
  Media	
  !!!	
  

	
  	
  
Why	
  Monitor	
  Social	
  Media	
  ?	
  
●  To	
  get	
  useful	
  insight	
  about	
  the	
  market	
  	
  
○  CompeXto...
WSO2	
  Business	
  AcXvity	
  Monitor
	
  
(BAM)	
  
●  Can	
  capture	
  data	
  from	
  any	
  applicaXon/service	
  	
...
WSO2	
  Business	
  AcXvity	
  Monitor
	
  
(BAM)	
  

18	
  
WSO2	
  Complex	
  Event	
  Processor	
  
(CEP)	
  
●  CEP	
  process	
  mulXple	
  event	
  streams	
  generated	
  withi...
WSO2	
  Complex	
  Event	
  Processor	
  
(CEP)	
  

20	
  
E.g.	
  Usecases	
  
●  Finding	
  out	
  the	
  trend	
  of	
  #Cloud	
  Twiaer	
  tags	
  
over	
  Xme	
  	
  
●  IdenXf...
SoluXon	
  Architecture	
  

22	
  
Twiaer	
  Agent	
  
	
  	
  	
  	
  	
  	
  	
  //Configuring	
  Agent	
  
	
  	
  	
  	
  	
  	
  	
  	
  AgentConfiguraXon...
Visualizing	
  #Cloud	
  Twiaer	
  Social	
  
Stream
	
  

24	
  
25	
  
Finding	
  out	
  the	
  trend	
  of	
  #Cloud	
  
Twiaer	
  tags	
  over	
  Xme	
  
๏ 

๏ 
26	
  

insert	
  overwrite	
 ...
27	
  
IdenXfy	
  the	
  public	
  interest	
  on	
  
cloud	
  market	
  leaders	
  	
  
insert	
  overwrite	
  table	
  CountOnC...
29	
  
Comparing	
  interest	
  trends	
  on	
  
cloud	
  market
	
  

30	
  
31	
  
NoXfying	
  via	
  Email	
  when	
  an	
  
interesXng	
  tweet	
  was	
  posted!
	
  
&	
  Take	
  AcXon
	
  

32	
  
CEP:	
  Retweet	
  NoXficaXon	
  	
  
from	
  twiaerStream[tweet	
  contains	
  'RT	
  ']	
  	
  
insert	
  into	
  reTweet...
Demo	
  …..	
  
http://wso2.com/library/demonstrations/2014/02/screencast-bigbrother-for-the-enterprise-social-stream-anal...
Log	
  Analysis	
  Demos	
  …..	
  
http://wso2.com/library/demonstrations/2014/02/screencast-monitoring-system-logs-with-...
Business	
  Model	
  

36	
  
Call	
  to	
  acXon	
  page	
  
๏ 
๏ 

Big	
  Data	
  Webinar	
  Series:	
  
hap://wso2.com/landing/wso2-­‐bigbrother-­‐we...
Contact	
  us	
  !	
  
Upcoming SlideShare
Loading in …5
×

Social Stream Analysis Use Cases

953 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
953
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Social Stream Analysis Use Cases

  1. 1. Big  Brother  for  Enterprises:     Social  Media  Stream  Use  Cases   Samisa  Abeysinghe   VP  Developer  Evangelism   26  Feb  2014
  2. 2. About  the  Presenter   ๏  ๏  2   Samisa  Abeysinghe   VP  Developer  Evangelism   samisa@wso2.com     Samisa  Abeysinghe,  Vice   President  of  Developer   Evangelism  joined  the  company  in   September  2005.    Prior  to  the   current  role,  Samisa  used  to  be   VP  of  Engineering  and  managed   the  development  of  WSO2   Carbon  based  product  plaOorm.    
  3. 3. About  WSO2   ๏  ๏  ๏  Global  enterprise,  founded  in  2005   by  acknowledged  leaders  in  XML,   web  services    technologies,   standards    and  open  source   Provides  only  open  source   plaOorm-­‐as-­‐a-­‐service  for  private,   public  and  hybrid  cloud   deployments   ๏  ๏  3   All  WSO2  products  are  100%  open   source  and  released  under  the   Apache  License  Version  2.0.   Is  an  AcXve  Member  of  OASIS,   Cloud  Security  Alliance,  OSGi   Alliance,  AMQP  Working  Group,   OpenID  FoundaXon  and  W3C.   Driven  by  InnovaXon   ๏  Launched  first  open  source  API   Management  soluXon  in  2012   ๏  Launched  App  Factory  in  2Q  2013   ๏  Launched  Enterprise  Store  and   first  open  source  Mobile  soluXon   in  4Q  2013  
  4. 4. What  WSO2  Deliver   4  
  5. 5. NSA like Monitoring for Your Enterprise ๏ Analyze volumes of data ๏ Address correlation complexities in analytics ๏ Off line vs Real time operations implications ๏ Some operations got to be in real time, else the value is lost ๏ Summarized data over time (and other) dimensions for analytics A “Big Brother” that keeps an eye on the whole enterprise 5  
  6. 6. Why should I bother? ๏ Deal with high volume (terra bites) of information ๏ In order to make decisions ๏ Real time & Offline ๏ Take action 6  
  7. 7. WSO2 :ONLY COMPLETE & INTEGRATED Platform ๏ Complete and integrated for ๏ Data capture ๏ Analysis: both real-time and batch ๏ Visualization ๏ Action taking business process execution Making data driven intelligence for your enterprise easy 7  
  8. 8. Big Data: Strategy 8  
  9. 9. Responsive and Effective 9  
  10. 10. WSO2 Big Data Analytics Platform for Your Enterprise WSO2  Big  Data  Analy0cs   Pla4orm  for  Your  Enterprise   10  
  11. 11. Key  Elements   Data   CollecXon   Taking   AcXon   Data   Analysis   Data   VisualizaXon   11  
  12. 12. WSO2 Big Data Analytics Platform for Your Enterprise Data Collection 12   Data Analysis Data Visualization & Taking Action
  13. 13. Use  Case  Scenario:   Analyzing  Twiaer  Streams   13  
  14. 14. Social  media  …  !   o  Test   o      Test  
  15. 15. Processing  Social  Media  !!!      
  16. 16. Why  Monitor  Social  Media  ?   ●  To  get  useful  insight  about  the  market     ○  CompeXtor  Analysis   ○  Customer  Trend  and  Interest   ●  IdenXfy  and  Compare  Trends     ○  Monitoring  the  brand   ○  Comparing  the  trends  of  the  public  interest   16  
  17. 17. WSO2  Business  AcXvity  Monitor   (BAM)   ●  Can  capture  data  from  any  applicaXon/service     ●  APIs,  Services  or  source  including  file-­‐based  data   ○  WSO2  has  lightweight  agents  to  provide     ■  raw  data  from  numerous  sources     ■  as  well  as  support  for     ●  gateways  (lightweight  or  heavyweight)     ●  Service  and  API  proxies   ●  Data  imports  from  various  sources  (including  file  based  data)   ●  Any  other  source  (extended  agents)   17  
  18. 18. WSO2  Business  AcXvity  Monitor   (BAM)   18  
  19. 19. WSO2  Complex  Event  Processor   (CEP)   ●  CEP  process  mulXple  event  streams  generated  within  the   enterprise  to  construct  data  abstracXon  and  idenXfy   meaningful  paaerns  among  those  streams   ○  Powered  by  WSO2  Siddhi   ■  ■  Processes  more  than  2.5M  events/sec  on  single  server  (commodity  hardware)   Has  a  powerful  and  extensible  SQL  Like  Siddhi  query  language     ○  Integrates  with  any  enterprise  system  for  event  capture   ■  Support  mulXple  transports  &  event  formats:  HTTP,  JMS  &  SMTP  on  XML,  JSON,  Text  &  Map   ○  Efficient  large  volume  event  publishing  to  enterprise   systems  and  custom  noXficaXons     ■  ■  ■  19   XML,  JSON,  Text  over  HTTP,  JMS  &  SMTP       SMS  Support     MySQL  DB  and  Cassandra  Support  
  20. 20. WSO2  Complex  Event  Processor   (CEP)   20  
  21. 21. E.g.  Usecases   ●  Finding  out  the  trend  of  #Cloud  Twiaer  tags   over  Xme     ●  IdenXfy  the  public  interest  on  cloud  market   leaders  based  on  tweets  on  #Cloud  Twiaer  tags     ●  Comparing  interest  trends  on  cloud  market     ●  NoXfying  via  Email  when  an  interesXng  tweet   was  posted.   21  
  22. 22. SoluXon  Architecture   22  
  23. 23. Twiaer  Agent                //Configuring  Agent                  AgentConfiguraXon  agentConfiguraXon  =  new  AgentConfiguraXon();                  agentConfiguraXon.setTrustStorePassword("/Users/suho/wso2/dev/twiaer-­‐sample/wso2cep-­‐3.0.0/repository/"  +                                  "resources/security/client-­‐truststore.jks");                  agentConfiguraXon.setTrustStore("wso2carbon”);                  Agent  agent  =  new  Agent(agentConfiguraXon);                    //New  Data  publisher  to  BAM                  final  AsyncDataPublisher  asyncDataPublisher  =  new  AsyncDataPublisher("tcp://localhost:7611",                                  "admin",  "admin",  agent);                    //Defining  the  Stream                  String  STREAM_NAME  =  "twiaer_data_stream";                  String  VERSION  =  "1.0.0";                  String  streamDefiniXon  =  "{"  +                                  "  'name':'"  +  STREAM_NAME  +  "',"  +                                  "  'version':'"  +  VERSION  +  "',"  +                                  "  'nickName':  'Twiaer_Data_Stream',"  +                                  "  'descripXon':  'Stream  of  twiaer  data',"  +                                  "  'payloadData':["  +                                  "  {'name':'createdAt','type':'LONG'},"  +                                  "  {'name':'user','type':'STRING'},"  +                                  "  {'name':'tweet','type':'STRING'}"  +                                  "  ]"  +                                  "}";                  asyncDataPublisher.addStreamDefiniXon(streamDefiniXon,  STREAM_NAME,  VERSION);     23  
  24. 24. Visualizing  #Cloud  Twiaer  Social   Stream   24  
  25. 25. 25  
  26. 26. Finding  out  the  trend  of  #Cloud   Twiaer  tags  over  Xme   ๏  ๏  26   insert  overwrite  table  Twi@erDataStreamStatsPerMinute  select  count(1)  as   total_tweet_count,  year(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  ))  as  year,  month(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐ MM-­‐dd  HH:mm:ss.SSS'  ))  as  month,day(from_unixXme(cast(created_at/1000  as   BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  ))  as  day,hour(from_unixXme(cast(created_at/ 1000  as  BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  ))  as  hour,   minute(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  ))   as  minute,concat(substring(from_unixXme(cast(created_at/1000  as  BIGINT),  'yyyy-­‐MM-­‐ dd  HH:mm:ss'),0,16),':00')  as  Xme  from  TwiaerDataStream  group  by   year(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  )),   month(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),day(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),hour(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),minute(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),substring(from_unixXme(cast(created_at/1000  as  BIGINT),  'yyyy-­‐MM-­‐ dd  HH:mm:ss'),0,16);      
  27. 27. 27  
  28. 28. IdenXfy  the  public  interest  on   cloud  market  leaders     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'IBM'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where  instr(lower(tweet),   'ibm')  >0  ;     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'Google'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where   instr(lower(tweet),  'google')  >0  ;     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'VMware'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where   instr(lower(tweet),  'vmware')  >0  ;     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'Azure'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where  instr(lower(tweet),   'azure')  >0  ;       28  
  29. 29. 29  
  30. 30. Comparing  interest  trends  on   cloud  market   30  
  31. 31. 31  
  32. 32. NoXfying  via  Email  when  an   interesXng  tweet  was  posted!   &  Take  AcXon   32  
  33. 33. CEP:  Retweet  NoXficaXon     from  twiaerStream[tweet  contains  'RT  ']     insert  into  reTweetStream;         33  
  34. 34. Demo  …..   http://wso2.com/library/demonstrations/2014/02/screencast-bigbrother-for-the-enterprise-social-stream-analysis/ 34  
  35. 35. Log  Analysis  Demos  …..   http://wso2.com/library/demonstrations/2014/02/screencast-monitoring-system-logs-with-wso2-business-activity-monitor/ http://wso2.com/library/demonstrations/2014/02/screencast-http-log-monitoring-with-wso2-business-activity-monitor/ 35  
  36. 36. Business  Model   36  
  37. 37. Call  to  acXon  page   ๏  ๏  Big  Data  Webinar  Series:   hap://wso2.com/landing/wso2-­‐bigbrother-­‐webinar-­‐series/     ๏  WSO2  Business  AcXvity  Monitor:   hap://wso2.com/products/business-­‐acXvity-­‐monitor/     ๏  WSO2  BAM  Docs:   hap://docs.wso2.org/display/BAM240/WSO2+Business+AcXvity+Monitor +DocumentaXon     ๏  WSO2  Complex  Event  Processor:   hap://wso2.com/products/complex-­‐event-­‐processor/     ๏  37   Be  your  own  NSA:  hap://wso2.com/landing/nsa/     WSO2  CEP  Docs:   hap://docs.wso2.org/display/CEP300/WSO2+Complex+Event+Processor +DocumentaXon    
  38. 38. Contact  us  !  

×