Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multi-Tenant Log Analytics SaaS Service using Solr: Presented by Chirag Gupta & Srivatsan Parthasarathy, Microsoft

744 views

Published on

Presented at Lucene/Solr Revolution 2016

Published in: Technology
  • Be the first to comment

Multi-Tenant Log Analytics SaaS Service using Solr: Presented by Chirag Gupta & Srivatsan Parthasarathy, Microsoft

  1. 1. O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
  2. 2. Multi-Tenant Log Analytics SaaS Service using SOLR Chirag Gupta (Principal Software Engineer) Srivatsan Parthasarathy (Partner Software Engineer) Microsoft Corp.
  3. 3. Solutions Search & Analytics Portal/API Microsoft Log Analytics
  4. 4. Real-time search Patching Configuration Change Tracking Linux Solutions Monitoring App Solutions Community Gallery Security Audit Data Protection Dashboards Integrations/Collectors Log Analytics Service Solution Platform Microso'  Log  Analy0cs  Solu0on  Pla4orm
  5. 5. 5 02 DEMO Microsoft Log Analytics Aka OMS (Operations Management Suite)
  6. 6. 6 02 •  50K+  customers  with  10M+  shards Usage Statistics •  20PB  searchable  documents   •  188  million  queries  per  week  
  7. 7. 7 02 Challenges •  SOLR  Cloud  boDlenecks   •  MulE-­‐tenant  support   •  Performance  issues  with  wide  queries  
  8. 8. 8 02 SOLR Cloud
  9. 9. IngesEon   Azure  File  Shares SOLR  Clusters SOLR   Query   Cluster  State   Microso'  Log  Analy0cs  Cloud Gateway   SOLR  Query   Merger  
  10. 10. SOLR  Cluster   Query Inges0on Predictable  Load Single  Customer/ Workload SOLR  Workloads Light  Load Moderate  Load Heavy  Load Long  tail  of  large  and  small  customers Growing/shrinking  without  warning
  11. 11. IngesEon   SOLR  Clusters Azure  File  Shares Cold  Cluster1  SOLR   Query   Microso'  Log  Analy0cs  Workload  Management Cluster  State   Gateway   SOLR  Query   Merger   Management  
  12. 12. SOLR  Cluster   Wide  Range  Query SOLR  Query
  13. 13. Query   Cold  Cluster1  SOLR   Hot  SOLR  Clusters Azure  File  Shares Cold  Cluster1  SOLR   Cold  Cluster Microso'  Log  Analy0cs  Cold  Query Cluster  State   Gateway   SOLR  Query   Merger  
  14. 14. 14 02 Recap •  Used  external  durable  storage  vs  replicaEon   •  Central  scoped  config  with  ApplicaEon  Load  Balancing  vs   Zookeeper   •  Shard  Lifecycle  Management  as  a  separate  component   •  Cold  Storage  based  Queries  
  15. 15. 15 02 Next Steps •  Further  opEmize  SOLR  for  Log  AnalyEcs  Scenario   •  IngesEon  pipeline   •  AnalyEcs  queries  
  16. 16. 16 02 Q/A
  17. 17. 17 02 Thank You Check out Microsoft Log Analytics @ http://microsoft.com/oms Blogs @ https://blogs.technet.microsoft.com/msoms/

×