The Loggly service utilizes Elasticsearch as the search engine underneath a lot of our core functionality. Log management imposes some tough requirements on search technology. To boil it down, it must be able to:
• Reliably perform near real-time indexing at huge scale – in our case, more than 100,000 log events per second
• Simultaneously handle high search volumes on the same index with solid performance and efficiency
When we were building our Gen2 log management service, we wanted to be sure that we were setting all of Elasticsearch’s configurations in the way that would deliver maximum performance for both indexing and search. Unfortunately, we found it very difficult to find this information in the Elasticsearch documentation because it’s not located in one place. This deck summarizes our learnings and can serve as a checklist of configuration properties you can reference to optimize ES for your application.
Get even more tips and insight on our full blog post → http://bit.ly/NineTipsOnES