Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DOXLON November 2016 - ELK Stack and Beats


Published on

Jon Hammant, Head of Cloud & DevOps for UK & EU for Epam Systems, presented an overview of using the ELK stack together with the Beats Plugin data shippers to provide detailed system metrics, network traffic, file analysis, and more. In addition, he provided an overview of how to monitor multiple Docker containers in a cloud native environment, with logs sent back to a central host.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

DOXLON November 2016 - ELK Stack and Beats

  1. 1. ELK STACK WITH BEATS November, 2016 Jon Hammant – Head of DevOps & Cloud UK/EU EPAM Systems
  2. 2. INTRO Head of Cloud & DevOps UK & EU email me. our careers portal.
  3. 3. ABOUT EPAM Q1 2016 Revenue $264.5M CONSTANT GROWTH 4 Continents 25 Countries REVENUE BY GEOGRAPHY North America Europe APAC CIS 58% 36% 2% 4% 20,000+ Engineers, designers and consultants FOUNDED IN 1993 US HEADQUARTERED PUBLIC COMPANY (NYSE:EPAM) SERVICE MIX Software Engineering & Product/Platform Development QA and Test Automation Managed Services Infrastructure & Licensing 20+% YOY organic growth 21 Reported Consecutive Quarters 2016 Revenue Guidance $1.15B Financial Services Travel & Consumer Software & Hi-tech Media & Entertainment Life sciences & Healthcare INDUSTRY FOCUS 27% 24% 14% 21% 8% Emerging 6%
  4. 4. PROBLEM Too many syste ms an d n ot e n ou gh visib ility Massive ly d istrib u te d In cre asin g n u mb e r of microser vices Fu ll d e -centralization Painfu l p roce ss We need log ging & metrics
  5. 5. WHY DO WE NEED METRICS? Bloodletting Starte d arou n d 100BCE C ontin u e d u ntil 19 th C e ntu r y H u n d re d s of Th ou san d s h ave d ie d It was d on e b e cau se p e op le cared Th ey ju st d id n ’t h ave right th e information
  6. 6. WHY ELK? Easy to setu p Massive ly Powe rfu l Scale s ve r y we ll Op e n sou rce Availab le as a se r vice 10 min u te setu p
  8. 8. • WHO WE SERVEWE NEED A WAY OF GETTING LOGS IN We don’t want to run Syslog everywhere Increasingly the applications are running on cloud native systems For a lightweight process we can’t add heavyweight logging No point writing loads of logging code
  9. 9. • WHO WE SERVEWHAT ARE BEATS? Beats are the Elasticsearch platform for single purpose, lightweight data shippers. Designed to be small & portable Logstash is still important for data enrichment, reformatting Replaces Logstash Forwarder & more
  10. 10. • WHO WE SERVECORE BEATS F i l e b e a t Pa c ke t b e a t M e t r i c b e a t W i n l o g b e a t
  11. 11. • WHO WE SERVEFILEBEAT Simplest of the Beat plugins Think of it as cat on steroids Can send a text file to central host Replaces Logstash Forwarder Has concept of backpressure to stop remote host being overloaded
  12. 12. • WHO WE SERVEMETRICBEAT System level monitoring – CPU, Memory, filesystem, IO statistics Includes modules for common services – Apache, Nginx, MongoDB, MySQL, Postgres & more Container ready – deploy one copy to monitor all other Docker containers
  13. 13. • WHO WE SERVEPACKETBEAT Network Packet Capture Understands application layer protocols – HTTP, DNS, ICMP, AMQP Great for security and latency analysis Can offer ”what went wrong” packet flow analysis
  14. 14. • WHO WE SERVEWINLOGBEAT Monitoring of Windows Log channels Pull Windows logs along with Linux Logs
  15. 15. WHEN LOGGING & METRICS WORK “Every th in g we kn ow in aviation , eve r y ru le in th e ru le b ook, eve r y p roce d u re we h ave , we kn ow b e cau se some on e somewh e re d ie d … We h ave p u rch ase d at gre at cost, lesson s literally b rou ght with b lood ” - " Su lly" Su lle n b e rge r
  16. 16. COMMUNITY BEATS Everything based on Go - libbeat Over 34 different community created Beats now available docs/communitybeats.asciidoc
  17. 17. • WHO WE SERVEOPENSOURCE HIGHLIGHTS h t t p b e a t Po l l a h t t p e n d p o i n t my s q l b e a t R u n a s c h e d u l e d q u e r y o n a my S q l s e r v e r M a n y m o r e u s e f u l B e a t s a v a i l a b l e o r w r i t e y o u r o w n C l o u d t ra i l b e a t , P i n g b e a t , C o n s u l b e a t e t c . . exe c b e a t Pe r i o d i c a l l y r u n c o m m a n d s a n d s e n d o u t p u t a n d e r r o r
  18. 18. • WHO WE SERVEdockbeat git clone clone wget chmod +x dockbeat-v1.0.0-x86_64 vi dockbeat/dockbeat.yml Replace Docker_Socket & Elasticsearch or Logstash host ./dockbeat-v1.0.0-x86_64 -c dockbeat/dockbeat.yml -v –e (can also be started in a container or swarm and permissioned)
  19. 19. • WHO WE SERVEEXAMPLE DASHBOARD - Metricbeat
  20. 20. • WHO WE SERVEUSE! D i s c o v e r L i s t h i s t o r i c C P U u s a g e F i n d o u t w h i c h c o n t a i n e r s w e r e r u n A n a l y ze fo r i n s e c u r e c o n t a i n e r s M e t r i c s S h o w r e a l t i m e m e t r i c s o f sy s t e m u s e D i s p l ay b u s i n e s s v a l u e V i e w t h e w h o l e sy s t e m a t o n e V i s u a l i z e L o o k b a c k a t p e r fo r m a n c e s t a t s C o r r e l a t e c o s t / p e r fo r m a n c e a n d r e v e n u e S h o w l o n g t e r m t r e n d s A l e r t U s e E l a s t A l e r t o n c o n t a i n e r s B e i n fo r m e d w h e n t h i n g s s t o p K n o w w h e n c a p a c i t y i s a n i s s u e
  21. 21. • WHO WE SERVESUCCESS! Storage is cheap Log everything and remove later Packetbeat is extremely useful go-audit (auditctl) and syslog are fantastic Black-box thinking, learn from mistakes