Copyright	
  ©	
  2013	
  Splunk	
  Inc.	
  

Payment	
  Gateway	
  Real-­‐Jme	
  
Monitoring	
  with	
  Splunk!	
  
Marci...
About	
  Me	
  
! 

IT	
  Coordinator	
  on	
  R&D	
  department	
  for	
  the	
  last	
  4	
  years	
  
Java	
  Developer...
About	
  My	
  Company	
  	
  
!

Universo	
  Online	
  –	
  Brazilian	
  Internet	
  Group	
  formed	
  by	
  two	
  
ver...
Agenda	
  
!
!
! 
!
! 
!
! 

About	
  PagSeguro	
  @	
  Universo	
  Online	
  
Finding	
  Out	
  and	
  Choosing	
  Splunk...
PagSeguro	
  
Internet	
  Payment	
  Gateway	
  
About	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  .	
  

6	...
About	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  .	
  
Mob...
Finding	
  Out	
  and	
  Choosing	
  Splunk	
  
Finding	
  Out	
  and	
  Choosing	
  Splunk	
  
The	
  Challenge	
  
New	
  Backend	
  Service	
  

Credit	
  Card	
  Proc...
Finding	
  Out	
  and	
  Choosing	
  Splunk	
  
Monitoring	
  tool	
  needs	
  
!
!
!
! 

Centralized	
  Big	
  Data	
  
F...
Finding	
  Out	
  and	
  Choosing	
  Splunk	
  
Splunk	
  fits!	
  
! 
! 
!
!
!

Works	
  with	
  textual	
  events	
  
No	...
The	
  Java	
  Logging	
  
Architecture	
  
The	
  Java	
  Logging	
  Architecture	
  
TransacCon	
  Request	
  
Message	
  Unmarshalling	
  
CreditCard	
  
Sale	
  
...
The	
  Java	
  Logging	
  Architecture	
  
<14>Mar 12 08:18:44 a5-hertz2 app=psgw-core INFO
[br.com.uol.ps.gw.core.aspect....
Fast	
  Time	
  to	
  Value	
  
Dashboards	
  
Real-­‐Jme	
  Monitoring	
  Dashboard	
  

16	
  
Real-­‐Jme	
  Monitoring	
  Dashboard	
  
Average	
  Jme	
  of	
  communicaJon	
  

17	
  
Real-­‐Jme	
  Monitoring	
  Dashboard	
  
Comparing	
  transacJons	
  volume	
  with	
  last	
  week	
  

18	
  
Real-­‐Jme	
  Business	
  Value	
  Dashboard	
  

19	
  
Real-­‐Jme	
  Business	
  Value	
  Dashboard	
  
Average	
  Jme	
  spent	
  in	
  communicaJon	
  by	
  transacJon	
  

20...
Real-­‐Jme	
  Business	
  Value	
  Dashboard	
  
Disabled	
  legacy	
  partners	
  architecture	
  

21	
  
Big	
  Data	
  Java	
  Debugging	
  
Big	
  Data	
  Java	
  Debugging	
  
!
! 
!

PagSeguro	
  has	
  more	
  than	
  15	
  SCRUM	
  feature	
  teams	
  
Java	...
Big	
  Data	
  Java	
  Debugging	
  –	
  Anomalies	
  

24	
  
Big	
  Data	
  Java	
  Debugging	
  
Java	
  Garbage	
  CollecJon	
  versus	
  DAO	
  execuJons	
  

25	
  
OperaJonal	
  DC	
  
Intelligence	
  
OperaJonal	
  DC	
  Intelligence	
  
Actual	
  DC	
  monitoring	
  tool	
  

27	
  
OperaJonal	
  DC	
  Intelligence	
  
Pushing	
  actual	
  monitoring	
  tool	
  data	
  to	
  Splunk	
  

<134>Jun 3 17:26...
OperaJonal	
  DC	
  Intelligence	
  
Events	
  dashboards	
  by	
  product	
  

29	
  
OperaJonal	
  DC	
  Intelligence	
  
Weekly	
  incidents	
  number	
  
Splunk®	
  goes	
  live!	
  

30	
  
Non-­‐stop	
  Splunk	
  Upgrade	
  
in	
  ProducJon	
  
Non-­‐stop	
  Splunk	
  Upgrade	
  in	
  ProducJon	
  
Before:	
  2	
  Servers	
  with	
  1	
  Splunk	
  Instance	
  

Inc...
Non-­‐stop	
  Splunk	
  Upgrade	
  in	
  ProducJon	
  
Aher:	
  4	
  Splunk	
  Servers	
  With	
  4	
  Splunk	
  Instances...
Summary	
  
Summary	
  	
  
Universo	
  Online	
  is	
  using	
  Splunk	
  in	
  several	
  ways	
  
! 

Monitoring	
  PagSeguro	
  tr...
THANK	
  YOU	
  
Upcoming SlideShare
Loading in …5
×

Us67903 using universo_online_marcioghiraldelli_paymentgatewaymonitoringwithsplunk

208 views
179 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
208
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Us67903 using universo_online_marcioghiraldelli_paymentgatewaymonitoringwithsplunk

  1. 1. Copyright  ©  2013  Splunk  Inc.   Payment  Gateway  Real-­‐Jme   Monitoring  with  Splunk!   Marcio  Ghiraldelli   IT  Coordinator,  Universo  Online   #splunkconf  
  2. 2. About  Me   !  IT  Coordinator  on  R&D  department  for  the  last  4  years   Java  Developer  at  one  of  the  largest  ecommerce  operaJon  at  Brazil   !  CerJfied  Java  Enterprise  Architect  and  MBA  in  Project  Management   !  2  
  3. 3. About  My  Company     ! Universo  Online  –  Brazilian  Internet  Group  formed  by  two   verJcals:   –  UOL  CS  –  Internet  Content  and  Service  provider   –  UOL  DIVEO  –  IT  Infrastructure  and  Datacenter  Service  Provider   3  
  4. 4. Agenda   ! ! !  ! !  ! !  About  PagSeguro  @  Universo  Online   Finding  Out  and  Choosing  Splunk   The  Java  Logging  Architechture   Fast  Time  to  Value  Dashboards   Big  Data  Java  Debugging   OperaJonal  DC  Intelligence   Non  Stop  Splunk  Upgrade  in  ProducJon   4  
  5. 5. PagSeguro   Internet  Payment  Gateway  
  6. 6. About                                                      .   6  
  7. 7. About                                                      .   Mobile  soluJons   7  
  8. 8. Finding  Out  and  Choosing  Splunk  
  9. 9. Finding  Out  and  Choosing  Splunk   The  Challenge   New  Backend  Service   Credit  Card  Processors,   Banks,  etc   Monitoring  Tool   PCI-­‐DSS  Security   Compliance   9  
  10. 10. Finding  Out  and  Choosing  Splunk   Monitoring  tool  needs   ! ! ! !  Centralized  Big  Data   Flexible   Easy  to  use   Real-­‐Jme   10  
  11. 11. Finding  Out  and  Choosing  Splunk   Splunk  fits!   !  !  ! ! ! Works  with  textual  events   No  need  of  regex  if  logs  are  in  key  value  format   Choose  Splunk  BEFORE  build  the  Java  applicaJon   Refined  data  with  low  license  usage   Splunk  data  extracJon  language  is  easy  to  use   11  
  12. 12. The  Java  Logging   Architecture  
  13. 13. The  Java  Logging  Architecture   TransacCon  Request   Message  Unmarshalling   CreditCard   Sale   Chain   Command   Database  Persistence   CreditCard  DecrypCon   CreditCard  Sale   Database  Persistence   Response  Marshalling   TransacCon  Response   13   Commands  
  14. 14. The  Java  Logging  Architecture   <14>Mar 12 08:18:44 a5-hertz2 app=psgw-core INFO [br.com.uol.ps.gw.core.aspect.CommandExecuteLogger] (qtp564660841-420:) datetime=2013-03-12 08:18:44,528 command=CreditCardSaleCommand msecs=555   Command=CreditCardSaleCommand   msecs=555 {creditCardResponse=null, creditCardRequestModel={ merchantCode=113257430   creditCardTransaction={id=10943513, merchantCode=113257430, fiDetails={nsu=357008,authorizationCode=530841, time=Tue Mar 12 08:18:44 BRT 2013, rawMessage=Transacao capturada com sucesso, rawCode=0, fi=cielo, integratorCode=10169261863339501001}, numTransientToken=93853bde05bd43f1b2a93375b790759e, maskedNumber=547408******1525   isWallet=false, softDescriptor=CortaCortaco, maskedNumber=547408******1525, brand=mastercard   cvv=***, brand=mastercard, paymentStatus=PAGO 14  
  15. 15. Fast  Time  to  Value   Dashboards  
  16. 16. Real-­‐Jme  Monitoring  Dashboard   16  
  17. 17. Real-­‐Jme  Monitoring  Dashboard   Average  Jme  of  communicaJon   17  
  18. 18. Real-­‐Jme  Monitoring  Dashboard   Comparing  transacJons  volume  with  last  week   18  
  19. 19. Real-­‐Jme  Business  Value  Dashboard   19  
  20. 20. Real-­‐Jme  Business  Value  Dashboard   Average  Jme  spent  in  communicaJon  by  transacJon   20  
  21. 21. Real-­‐Jme  Business  Value  Dashboard   Disabled  legacy  partners  architecture   21  
  22. 22. Big  Data  Java  Debugging  
  23. 23. Big  Data  Java  Debugging   ! !  ! PagSeguro  has  more  than  15  SCRUM  feature  teams   Java  applicaJon  distributed  around  60  hosts   Difficult  to  trace  RCA  on  producJon  outages   23  
  24. 24. Big  Data  Java  Debugging  –  Anomalies   24  
  25. 25. Big  Data  Java  Debugging   Java  Garbage  CollecJon  versus  DAO  execuJons   25  
  26. 26. OperaJonal  DC   Intelligence  
  27. 27. OperaJonal  DC  Intelligence   Actual  DC  monitoring  tool   27  
  28. 28. OperaJonal  DC  Intelligence   Pushing  actual  monitoring  tool  data  to  Splunk   <134>Jun 3 17:26:40 localhost MACS_EVENT IDT_EVENT_OCURRENCE="104433559", CI="a4-osvaldo1", SERVICE="snmp_bd_win_mem_80_90", INTERFACE="if-0-a4osvaldo1", BACKEND="dc_gt", DATE="2013-06-03 17:26:34.0", TYPE="Ativo", IND_ALERT="1", IND_EVENT_ALARM="1", ATTEMPTS="3/3", INFORMATION="Physical Memory: 80%used(3281MB/4091MB) (>80%) : WARNING"! 28  
  29. 29. OperaJonal  DC  Intelligence   Events  dashboards  by  product   29  
  30. 30. OperaJonal  DC  Intelligence   Weekly  incidents  number   Splunk®  goes  live!   30  
  31. 31. Non-­‐stop  Splunk  Upgrade   in  ProducJon  
  32. 32. Non-­‐stop  Splunk  Upgrade  in  ProducJon   Before:  2  Servers  with  1  Splunk  Instance   Incoming  TCP  Data   Splunk  Search  User  Access   Search  Head  and  Indexer   NFS  Mount   Index  Cold  Buckets   32  
  33. 33. Non-­‐stop  Splunk  Upgrade  in  ProducJon   Aher:  4  Splunk  Servers  With  4  Splunk  Instances   Search  Head   Splunk  Search  User  Access   Indexer   TCP  Load  Balancer   TCP  Load  Balancer   Search  Head   Indexer   33   Incoming  TCP  Data  
  34. 34. Summary  
  35. 35. Summary     Universo  Online  is  using  Splunk  in  several  ways   !  Monitoring  PagSeguro  transacJons   !  SupporJng  PagSeguro  business  decisions   !  R&D  Sohware  Engineering  debugging   ! Dashboarding  DC  producJon  environment  via  logs  from     actual  monitoring  tool   35  
  36. 36. THANK  YOU  

×