Download as PDF, PPTX
Uploaded byElasticsearch
Nine Publishing: Building a modern infrastructure with the Elastic Stack
The document discusses the implementation of the Elastic Stack to enhance infrastructure agility, flexibility, and efficiency. Key takeaways include the complexities of managing a self-hosted Elasticsearch cluster, the importance of automation in maintenance, and the benefits of centralized logging and monitoring. Additionally, it highlights the successful open-sourcing of their own Helm chart and discusses various cases for improved user experience and cost optimization.
More Related Content
Similar to Nine Publishing: Building a modern infrastructure with the Elastic Stack
![Modernising One Legal Se@rch with Elastic Enterprise Search [Customer Story]](https://cdn.slidesharecdn.com/ss_thumbnails/modernisingonelegalserchwithelastic-jsedit14thjune2021-210625013449-thumbnail.jpg?width=640&height=640&fit=bounds)
Nine Publishing: Building a modern infrastructure with the Elastic Stack
- 1. 1 Michael Lorant Principal SystemsEngineer Nine Embracing the Elastic stack to solve the challenges of modern infrastructure
- 2.
- 3.
- 4.
- 5.
- 6.
- 7. 7 What We Needed ApplicationSearch Analytics Centralised Logging
- 8. 8 What We Built ApplicationSearch Analytics Centralised Logging
- 9.
- 10.
- 11.
- 12. 12 What Is Interesting Containerised Ourown Helm chart which we open sourced. https://github.com/fairfaxmedia/charts/tree/master/stable/elasticsearch
- 13. 13 What Is Interesting Optimised Highperformance / Cost effective Densely stacked
- 14. 14 What Is Interesting UpstreamCompatibility Upgrading will be time consuming if you derive too far from the base image.
- 15. 15 What Is Interesting Automation Upgradingand patching is important. If you have to make changes, it will require significant effort without automation.
- 16.
- 17.
- 18.
- 19. 19 What We Learned NetworkIntensive
- 20. 20 What We Learned NetworkIntensive
- 21. 21 What We Learned NodeMaintenance Understand how to replace data nodes without causing unnecessary replication. index.unassigned.node_left.delayed_timeout /_flush/synced https://www.elastic.co/guide/en/elasticsearch/reference/current/delayed-allocation.html
- 22. 22 What We Learned Challenging Runningyour own Elasticsearch cluster at scale is complex. Who will own it? Who will maintain it? Who will update it?
- 23.
- 24. 24 What Is Interesting SinglePurpose Many smaller pipelines.
- 25.
- 26. 26 What We Learned ResourceIntensive
- 27. 27 What We Learned SchemaComplexity dots.in.keys (╯°□°）╯︵ ┻━┻
- 28.
- 29. 29 What Is Interesting MetricbeatFilebeat APM
- 30. 30 What We Learned ArchitectSimilar To Logstash Adding modules such as AWS Kinesis output is easy. https://github.com/s12v/awsbeats
- 31. 31 What We Learned EasyTo Implement Helm charts well maintained and consistent. Most of the configuration you use for one beat can be used for all the others.
- 32.
- 33.
- 34.
- 35.
- 36. 36 What We Learned Developerssee Kibana as Elasticsearch =
- 37.
- 38. 38 Case 1: Troubleshooting Arewe able to identify requests to pages that don’t exist on the SMH?
- 39.
- 40.
- 41.
- 42. 42 Case 2: Cost Isour CDN optimally configured for the SMH and AFR to improve user experience and keep costs low?
- 43.
- 44.
- 45.
- 46.
- 47. 47 Case 3: Visibility Arewe able to spot problems with SMH or The Age before they impact users?
- 48.
- 49. 49 Case 4: Analytics AreCanberra Times users in Sydney or Melbourne using the site to get around the paywall?
- 50.
- 51.