Splunk at CeryxDerek MockDirector, Software DevelopmentMay 3, 2012
About Me…      Director, Software Development      10 years at Ceryx      Previous log management experience includes     ...
Who we are …  A leader in providing enterprise unified communications via  the cloud model to large corporations with over...
Ceryx Services                                                                                    Cloud                   ...
The Initial Problem – Message Tracking•   Email logs across multiple platforms•   Growing infrastructure - increasing comp...
Message Tracking Before Splunk Had to sift through:      Multiple layers of servers      Each layer with redundant      ma...
Initial Splunk Deployment•   Used two data centers•   Deployed an indexer in each DC•   Deployed ~30 forwarders•   Setup d...
Message Tracking After Splunk “Aha”      Huge customer experience win:      End-to-end resolution now      takes 1 hour vs...
Why Splunk?      Securely see all logs from one place      Splunk is more cost-effective in licensing for throughput vs. p...
Our Environment      Deployed to multiple data centers      Deployed the forwarder to 400+ servers      6 indexers      50...
Machine Data SourcesMS Applications                                                                          Perf Monitori...
Security & Compliance   Since deploying Splunk we have completed:   – ISO 27001   – SAS Type II   Handling security logs f...
Ceryx Security App      Custom app developed for      Security team      – System access monitoring and        alerting   ...
Ceryx NOC     Custom apps developed for     Operations     – OS performance       Dashboards/Alerting     – App performanc...
Ceryx Service Delivery      Custom app developed for our      Support and Delivery Groups      – Assists in generation of ...
Ceryx Customer Support      Dashboard developed      by Support for Support      Enables live monitoring      of phone and...
Ceryx Software Development   Responsible for all in-house software development   – Cloud Control   – Monitoring Tools     ...
Cloud Control App      Custom app for Cloud      Control      –   Performance      –   Web Analytics      –   Usage      –...
Splunk Dashboards                                                   Wow!      Greatly accelerates creation of      weekly ...
Splunkbase      Some of the Apps we use:      –   Splunk App for Windows      –   Splunk App for Unix and Linux      –   G...
Other Customizations      Our NOC is very picky       Developed additional search commands      – Custom alert formats   ...
Splunk “Aha” Moments      Happen all the time now      Splunk graphical visualizations gave      immediate visibility     ...
Internal Transformation      Everyone got tired of “What was the system      doing last _____”      For many, Splunk has b...
Fielding Requests Beyond IT      Sales wanted to know where      users connected to Ceryx servers      throughout the worl...
What’s Next?      More Splunk:      –   Migrating many alerting functions to be centralized in Splunk      –   OS/Performa...
Hot Tips      Do a live demo with management to show non-technical people the      power of Splunk      Create a cheat-she...
Toronto Splunk Users Group      Started in March 2012      Meets monthly and open to current Splunk customers      Discuss...
Questions?
Thank youMay 3, 2012
Upcoming SlideShare
Loading in...5
×

SplunkLive! Toronto - Ceryx

1,338

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,338
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

SplunkLive! Toronto - Ceryx

  1. 1. Splunk at CeryxDerek MockDirector, Software DevelopmentMay 3, 2012
  2. 2. About Me… Director, Software Development 10 years at Ceryx Previous log management experience includes • Late nights, lots of files and grep Hobbies - possibly Fav Splunk T-Shirt: • Finding your faults, just like momCopyright © 2011, Splunk Inc. 2 Listen to your data.
  3. 3. Who we are … A leader in providing enterprise unified communications via the cloud model to large corporations with over 18+ years of messaging and collaboration experience Ceryx’s Cloud Control software dramatically reduces support costs while enhancing/enabling customer experience. Have developed a large integrator partner channel to address this target market.Copyright © 2011, Splunk Inc. 3 Listen to your data.
  4. 4. Ceryx Services Cloud Unified Archiving Encryption Security Management Mobility Messaging Capabilities Multi-Tenant (< 3000 seats) Dedicated (3000 + seats) Managed (3000 + seats) • Shared Server Hardware • Dedicated Exchange and Active • Solution managed at customers DC (or Directory locations of their choice) • Complete mapping to existing • Monitored/Supported to Ceryx SLA’s business processes with full integration to the customer’s business process and operational guidelinesCopyright © 2011, Splunk Inc. 4 Listen to your data.
  5. 5. The Initial Problem – Message Tracking• Email logs across multiple platforms• Growing infrastructure - increasing complexity• New retention and compliance requirements• Increasing number of helpdesk tickets being opened• Customer experience was declining - resolution times increasing• Training times for new staff were increasingCopyright © 2011, Splunk Inc. 5 Listen to your data.
  6. 6. Message Tracking Before Splunk Had to sift through: Multiple layers of servers Each layer with redundant machines Would run separate session searches and try to sift through all the dataCopyright © 2011, Splunk Inc. 6 Listen to your data.
  7. 7. Initial Splunk Deployment• Used two data centers• Deployed an indexer in each DC• Deployed ~30 forwarders• Setup distributed search• Provided access to support staff only• No customizations• No appsCopyright © 2011, Splunk Inc. 7 Listen to your data.
  8. 8. Message Tracking After Splunk “Aha” Huge customer experience win: End-to-end resolution now takes 1 hour vs. 1 day Organization has since grown, 2-3x number of servers If we didn’t have Splunk we would have needed to add additional staff to support our growth.Copyright © 2011, Splunk Inc. 8 Listen to your data.
  9. 9. Why Splunk? Securely see all logs from one place Splunk is more cost-effective in licensing for throughput vs. per agent or per user Search GUI intuitive with great visualizations Searching was quick and easy Flexibility – can do things you can’t even think of Apps – Can wrap our knowledge around the dataCopyright © 2011, Splunk Inc. 9 Listen to your data.
  10. 10. Our Environment Deployed to multiple data centers Deployed the forwarder to 400+ servers 6 indexers 50+ staff with Splunk access Indexing 45+GB per day And we aren’t done…Copyright © 2011, Splunk Inc. 10 Listen to your data.
  11. 11. Machine Data SourcesMS Applications Perf MonitoringExchange WindowsSharePoint LinuxLync/OCS Custom Applications Web Servers Email Authentication Event Monitoring App performance IIS logs Exchange MTA IAS Radius Windows event logs and availability Apache logs Sendmail MTA (VPN/Secure ID) Linux system logs Other MTA’s Copyright © 2011, Splunk Inc. 11 Listen to your data.
  12. 12. Security & Compliance Since deploying Splunk we have completed: – ISO 27001 – SAS Type II Handling security logs for both Ceryx corporate data as well as customer data Monitoring for violations etc.Copyright © 2011, Splunk Inc. 12 Listen to your data.
  13. 13. Ceryx Security App Custom app developed for Security team – System access monitoring and alerting – Abuse reporting – Ad-hoc searches as requiredCopyright © 2011, Splunk Inc. 13 Listen to your data.
  14. 14. Ceryx NOC Custom apps developed for Operations – OS performance Dashboards/Alerting – App performance Alerting – Event Log alerting – Usage reports – Capacity/Trending reports Allows our NOC fast and easy access to the information they need when responding to problemsCopyright © 2011, Splunk Inc. 14 Listen to your data.
  15. 15. Ceryx Service Delivery Custom app developed for our Support and Delivery Groups – Assists in generation of monthly reporting to our customers – Respond to trends in customer issues quicker – Generate system usage reportsCopyright © 2011, Splunk Inc. 15 Listen to your data.
  16. 16. Ceryx Customer Support Dashboard developed by Support for Support Enables live monitoring of phone and ticket queues View ticket and call distributions to ensure adequate resource utilizationCopyright © 2011, Splunk Inc. 16 Listen to your data.
  17. 17. Ceryx Software Development Responsible for all in-house software development – Cloud Control – Monitoring Tools  Health checks  Availability – Support Tools  ITSM systems Modifying our custom applications to log data easily ingested by Splunk Provide log data to developers without needing server access Increased our time to resolution on issues escalated by our Ops groupsCopyright © 2011, Splunk Inc. 17 Listen to your data.
  18. 18. Cloud Control App Custom app for Cloud Control – Performance – Web Analytics – Usage – Capacity – AuditingCopyright © 2011, Splunk Inc. 18 Listen to your data.
  19. 19. Splunk Dashboards Wow! Greatly accelerates creation of weekly status reports Share data on a large LED board along with phone stats—a “wow” moment for rest of org to see live Splunk stats CIOCopyright © 2011, Splunk Inc. 19 Listen to your data.
  20. 20. Splunkbase Some of the Apps we use: – Splunk App for Windows – Splunk App for Unix and Linux – Google Maps – Sideview Utils – Web Intelligence – Deployment Monitor – Splunk for use with aaMap – Maxmind (Geo Location) – PDF Report ServerCopyright © 2011, Splunk Inc. 20 Listen to your data.
  21. 21. Other Customizations Our NOC is very picky  Developed additional search commands – Custom alert formats – Custom filtering options – 3rd Party monitoring system integrationCopyright © 2011, Splunk Inc. 21 Listen to your data.
  22. 22. Splunk “Aha” Moments Happen all the time now Splunk graphical visualizations gave immediate visibility Allow us to look at data as a whole instead of only at a component level Easier access to historical data makes analysis much quickerCopyright © 2011, Splunk Inc. 22 Listen to your data.
  23. 23. Internal Transformation Everyone got tired of “What was the system doing last _____” For many, Splunk has become the go to application for troubleshooting issues Show them a few searches and they are hooked Getting almost daily requests for “can we get data X into Splunk?” Increased awareness in the value of our dataCopyright © 2011, Splunk Inc. 23 Listen to your data.
  24. 24. Fielding Requests Beyond IT Sales wanted to know where users connected to Ceryx servers throughout the world Took a few clicks and about a minute to generate the report Answering questions for the business paints IT as critical in providing the data that shapes business decisions.Copyright © 2011, Splunk Inc. 24 Listen to your data.
  25. 25. What’s Next? More Splunk: – Migrating many alerting functions to be centralized in Splunk – OS/Performance event data across the entire environment – Continue to customize our in-house applications to log to Splunk – The Splunk for VMware App Give internal groups better visibility with customized Splunk apps for: – Security-related info – Custom application data Give customers the ability to search Splunk data – Leverage the API to give customers visibility and self-serve ability – More visibility into messaging services – using data for their own internal reportingCopyright © 2011, Splunk Inc. 25 Listen to your data.
  26. 26. Hot Tips Do a live demo with management to show non-technical people the power of Splunk Create a cheat-sheet for people with practical examples Ask business owners what questions they’d like to ask/ answer, but don’t know who/ how to ask Storage sizing/ performance Get involved in the Splunk CommunityCopyright © 2011, Splunk Inc. 26 Listen to your data.
  27. 27. Toronto Splunk Users Group Started in March 2012 Meets monthly and open to current Splunk customers Discussions include: – Splunk use cases – Issues or problems people are having – Best practices – Beer Splunk T-shirts Contact me: derek.mock@ceryx.com or @derek_mockCopyright © 2011, Splunk Inc. 27 Listen to your data.
  28. 28. Questions?
  29. 29. Thank youMay 3, 2012

×