Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Clear the Mist from your Clouds with Splunk


Published on

With the majority of organisations using the cloud in one way or the other, cloud has become an essential ingredient of modern IT. But with increased cloud usage comes a greater need to have visibility across operations, security and cost control. Add in the complexity of multiple clouds, and things get quite misty very quickly. Whether you’re running workloads in AWS, Azure, or GCP (or all three)- Splunk has you covered to keep you on the right path in your cloud journey. This session will explore how Splunk integrates with the most popular cloud providers and the services they provide, allowing you to look in, on and through the Clouds.

Published in: Technology
  • Don't even THINK about buying any uterine fibroids product, drugs or going on a gimmick fibroids program until you read my revealing, no-holds barred holistic uterine fibroids cure book. ■■■
    Are you sure you want to  Yes  No
    Your message goes here
  • New E-book Reveals Unique Holistic Strategies to Cure Uterine Fibroids. Discover How To Quickly And Easily Cure Uterine Fibroids Permanently...Even If Everything Else You Tried had Failed...Without Drugs, Without Surgery, and Without Nasty Side Effects – Guaranteed! ▲▲▲
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Clear the Mist from your Clouds with Splunk

  1. 1. © 2019 SPLUNK INC.© 2019 SPLUNK INC. Clear the Mist from your Clouds with Splunk SplunkLive London - June 2019 Yuval Tenenbaum Director – SE Architects EMEA
  2. 2. © 2017 SPLUNK INC. Migration To Cloud & Hybrid Cloud Insights is Top Of Mind
  3. 3. © 2019 SPLUNK INC. ► Enables Least privileged model at the highest operational control ► Mitigates Risk – lower the ‘blast radius’ of impactful events ► Achieve Agility- deploy & run environments programmatically at scale ► Cost optimisation- clear ‘line of sight’ into the cost of running workloads Hybrid Cloud – Think Differently Legacy Model Least privileged
  4. 4. © 2019 SPLUNK INC. ► Split Investment may slow down your cloud adoption – Spreading your resources across multiple clouds means that you may not get critical mass or a fast ROI ► Portability - How many of us will actually move workloads around? ► Cloud Broker concept – Putting a “bloatware” between you and your cloud api’s instead of working natively with these cloud API’s Is it Really All Good Stuff? I used to be indecisive now I’m definitely going multi- cloud
  5. 5. © 2019 SPLUNK INC. Cloud - Same Challenges-Different Environments ► Security • Are we firewalled correctly? • Do we use all necessary security features? ► Compliance • Are we following all published standards? ► Networking • Placed servers on the correct network? ► Financial • Stayed within budget? ► Capacity Planning • Used resources optimally? And all of that in a decentralized Model…
  6. 6. © 2019 SPLUNK INC. Customer experience??? SAAS Hybrid Everything - What happens when we stack them? ON PREMISES Legacy systems (Mainframe…) Facilities Dev/PreProd Storage Backup Archive DR Security VMs Containers Micro services AWS (Application 1)Access / Security Database StorageDev Compute Containers App engine GCP (Big Data project 1) Dataflow AWS (Archive) Azure (Application 1) VMs Database VM sets Traffic mger
  7. 7. © 2017 SPLUNK INC. So How Can Splunk Clear up this Cloudy Mist? Know your Clouds…..
  8. 8. © 2019 SPLUNK INC. ► Splunk has working relationships with AWS, Azure, and GCP ► We have customers successfully running Splunk Enterprise BYOL within AWS, Azure, and GCP ► We have proven strategies to get data in from AWS, Azure, and GCP Cloud Vendor Relationships
  9. 9. © 2017 SPLUNK INC. Splunk’s Approach to Hybrid Cloud One Consolidated Solution Manage Hybrid Infrastructure Cost, Capacity and Resource Management Cloud Migration Splunk takes the place of the multitude of monitoring tools because sometimes one is better than many. Deploy Splunk in Hybrid setup (on-prem, saas, byol) and deal with Hybrid infrastructure complex monitoring Understand how your resources are performing – and how many are being used – then optimize utilization and billing. Get visibility at all stages of the migration process (landing zones)– whether before, during or long after.
  10. 10. © 2017 SPLUNK INC. In the Beginning…… Cloud Migration
  11. 11. © 2019 SPLUNK INC. What Customers Want To Achieve When Migrating to the Cloud ► Build - Differentiate yourself by building unique and valuable services ► Move Fast - From initial idea to a service which can be monetized ► Stay Secure - Make sure that what we build is secure and compliant ▶ Manage Cost – Control what you spend and gain visibility into future cost
  12. 12. © 2019 SPLUNK INC. Path To Successful Cloud Migration Measure the baseline user experience and performance, as well as define acceptable post-migration levels. Security assessment – build a well architected and compliant landing zones Performance metrics should be closely monitored & compared to the baseline. Throughout the migration, end-to-end monitoring can help SecOps teams stay ahead of any potential risks. Continuous monitoring should be used to measure acceptable metrics and success. Leverage a platform that shows insights into cost, shared services, monitoring, Security & compliance BEFORE DURING AFTER
  13. 13. © 2019 SPLUNK INC. Challenges With Building & Maintaining Landing Zones ▶ Define & maintain an Account structure ▶ Define your network architecture and monitor it continuously ▶ Define & maintain a security governance and compliance baseline Migrate Land Operate & Optimize
  14. 14. © 2019 SPLUNK INC. Additional Considerations ▶ Define & maintain centralized logging ▶ Define & maintain Cost Allocation
  15. 15. © 2019 SPLUNK INC. How Can Splunk Help (1)? ▶ Tell you who is accessing your accounts, from where and what are they doing?
  16. 16. © 2019 SPLUNK INC. How Can Splunk Help (2)? ▶ Tell you if anyone is breaking your security policies? • Is encryption used everywhere • Has the root account has MFA enabled • Suspicious AWS S3 Activities • IAM Password policies are kept as you defined in your security baseline?
  17. 17. © 2019 SPLUNK INC. How Can Splunk Help (3)? ▶ Help you understand your network topology and gain visibility into who is trying to access it ▶ Help you gain visibility into performance & right sizing of your key workloads ▶ Help you understand historic and future cost
  18. 18. © 2019 SPLUNK INC. AWS Analytic Stories - ES Content Updates
  19. 19. © 2019 SPLUNK INC. Migration Dashboards
  20. 20. © 2017 SPLUNK INC. So How Do We Collect Cloud Data to do this Hybrid Monitoring?
  21. 21. © 2017 SPLUNK INC. Getting Data In Cloud Patterns
  22. 22. © 2017 SPLUNK INC. General Getting Data In Routes Pull or Push, Add-Ons or Serverless Poll/Request API Data Data Cloud Serverless Code Add-On HEC “Push”
  23. 23. © 2017 SPLUNK INC. GDI : AWS
  24. 24. © 2019 SPLUNK INC. It May Look a Bit Complicated
  25. 25. © 2019 SPLUNK INC. ► AWS Config can be pulled with a Splunk Heavy Forwarder with the SQS Based S3. Anything via CloudWatch Logs or CW events, can be pushed with Kinesis Firehose to Splunk AWS Pull vs. Push Config Events SNS Topic Notification SQS Subscription Notification Pulls Event from S3 Bucket Splunk Pull SQS Notification HEC PushPull CloudWatch Logs
  26. 26. © 2019 SPLUNK INC. AWS Source Matrix There are many options to GDI in AWS but Splunk can help Data Type Recommended Input Type Billing Billing CloudWatch CloudWatch CloudFront Access Logs SQS based S3 Config SQS based S3 Config Rules Config Rules Description Description ELB Access Logs SQS based S3 Inspector Inspector CloudTrail SQS Based S3 S3 access logs SQS Based S3 VPC Flow Logs (CW Logs) Kinesis With SQS Based S3 you can scale out data collection by configuring multiple inputs to ingest logs from the same S3 bucket without creating duplicate data. Kinesis Firehose is recommended for CloudWatch Logs data collection
  27. 27. © 2017 SPLUNK INC. GDI : Azure & O365
  28. 28. © 2019 SPLUNK INC. 3 Log Types in Azure 1) Control/Management, 2) Data Plane, 3) Processed Events Control: System Configuration and Management Data Plane: Provisioned Service and Diagnostic Data Processed Events: Alerts & Recommendations
  29. 29. © 2019 SPLUNK INC. { REST } Storage Event Hub
  30. 30. © 2019 SPLUNK INC. ► Splunk can pull data from Azure using a Heavy Forwarder and collect data from either the MS Blob or a REST API using the modular input. Azure can push data using the Event Hub to Azure Functions which can be sent to Splunk’s HEC. Azure Pull vs. Push MSBlob HEC PushPull Splunk Indexers Activity Monitor Event Hub Azure Function Event Hub
  31. 31. © 2019 SPLUNK INC. Azure Add-on Landscape
  32. 32. © 2019 SPLUNK INC. Getting O365 Data In Azure Active Directory Application OAUTH2 REST Splunk Add-on for Microsoft O365 Office 365
  33. 33. © 2017 SPLUNK INC. GDI : Google Cloud
  34. 34. © 2019 SPLUNK INC. Getting GCP Data In REST Splunk Add-on for Google Cloud Platform Billing PubSub Monitoring StackDriver
  35. 35. © 2019 SPLUNK INC. ► Initial: • Most customers will generate around 1-10GB when they are setting up their Public Cloud deployments and enabling services. • As they mature - 10-50GB. ► More instances and deployed apps in Cloud, 50-200GB. ► Most customers are 100-200GB / day of Public Cloud data. ► All-in Cloud Companies : 500GB-1TB range. ► Less common >1TB ► O365 - ~400 to 500 KB per user per day (50K users = 25 GB/day) ► Best way to analyze the amount of data is to spin-off a test environment and look at the numbers. How Much Data?
  36. 36. © 2017 SPLUNK INC. Collection Deployment Architectures
  37. 37. © 2019 SPLUNK INC. ► Central Splunk Instance • One Instance to manage – lower “Instance/Storage” costs • Data egress cost considerations (data transfers from each cloud) • Local or Distributed Heavy Forwarders ► Splunk Instance per Cloud, 1 “Master” view • One Instance in each Cloud – potential higher “Instance/Storage” cost • Management of Splunk in each Cloud • “Master” Search Head needed for Hybrid Search – latency impact • Lower egress cost ► Hybrid • Mix of both options balancing out Costs/Hybrid Search Deployment Architecture 3 Patterns
  38. 38. © 2019 SPLUNK INC. Option 1 Public/Private Cloud / Splunk Cloud Single Splunk InstanceHeavy Forwarder (Add-On) Heavy Forwarder (Add-On) Heavy Forwarder (Add-On) Note Options for Serverless/HEC input direct to Central Instance Cloud Data
  39. 39. © 2019 SPLUNK INC. Option 2 Public/Private Cloud Distributed Hybrid SearchSplunk Indexer(s) Splunk Indexer(s) Splunk Indexer(s) Search Head Search Results
  40. 40. © 2019 SPLUNK INC. Option 3 Distributed Search Splunk Indexer(s) & Master Search Splunk Indexer(s) Heavy Forwarder (Add-On) Cloud Data Search Results
  41. 41. © 2019 SPLUNK INC.© 2017 SPLUNK INC. © 2017 SPLUNK INC. OUR MISSION ….Including Cloud data!
  42. 42. © 2019 SPLUNK INC. Hybrid Monitoring Collect & store machine data generated by on-premises IT sources and public cloud sources simultaneously, and can correlate across both to monitor, alert, analyse, troubleshoot and investigate.
  43. 43. © 2017 SPLUNK INC. Pulling it all together: Example Cloud Innovation, Integration and Use Case AWS Security Hub + Splunk Phantom Bi-Directional Integration
  44. 44. © 2019 SPLUNK INC. AWS Security Hub - Findings
  45. 45. © 2019 SPLUNK INC. Phantom - EC2 Instance- Investigate & Notify
  46. 46. © 2019 SPLUNK INC. Geo Location & IP Reputation
  47. 47. © 2019 SPLUNK INC. Prompting The Analyst- Quarantine Instance
  48. 48. © 2019 SPLUNK INC. Phantom- Isolate ES2 Instance Playbook
  49. 49. © 2019 SPLUNK INC.
  50. 50. © 2019 SPLUNK INC. Back To AWS Security Hub
  51. 51. © 2019 SPLUNK INC.© 2019 SPLUNK INC. Don't forget to rate this session in the .conf18 mobile app Thank You.