Zabbix was experiencing performance issues due to large history tables in the database. To address this, the architecture was changed to store history data in Elasticsearch instead of database tables. This improved scalability and performance. The basic item and event data remained in the MariaDB database cluster. Zabbix proxies were also used to distribute load across multiple network segments. With this new architecture, history data is indexed in Elasticsearch without database tables, improving query speed and reducing database size.

1. Zabbix At scale with Elasticsearch
Zabbix Mysql ElasticSearch
Leandro Totino Pereira
System Engineer

2. Zabbix Problems
• Zabbix Server busy
• Database too many history records
• Database slow due to huge table
• Corrupted table
• Backup

3. Zabbix Tables
• History - table that stores all float values
• History_uint - table that stores all integer values
• History_text - table that stores all text values
• History_Str - table that store all string values
• History_Log - table that stores all log values
• Events - table that stores all events values
• Items - table that stores all items values

4. ES - Elasticsearch
• Scalable to one a many nodes.
• Really fast compared to sql history,logs tables.
• Index Based (auto-partitioning index based on number documents)
• Data Compression really good.

5. MariaDB - Galera/Binlog replication
Multi cluster architecture supported, the main ones:
Galera multi-master cluster
Mysql replication (master,multi-slave)
Binlog server (master, multi slaves)
If you have a huge amount items/events you should consider partitioning
events,items tables and run OPTIMIZE on them to reclaim disc space.

6. Hot Backup Mysql
• Backups that complete quickly and reliably
• Uninterrupted transaction processing during backups
• Savings on disk space and network bandwidth
• Automatic backup verification
• Higher uptime due to faster restore time
• Percona XtraBackup makes MySQL hot backups for all versions of Percona Server,
MySQL, and MariaDB. It performs streaming, compressed, and incremental MySQL
backups.
With Percona XtraBackup, you can achieve the following benefits:

7. Zabbix Architecture History* table are saved in
elasticsearch and the basics ones
items, events are saved on
mariadb cluster.
Zabbix proxies set up in many networks
segments
VIP address in front of Zabbix server
master/slave

8. Elasticsearch zabbix mapping

9. Elasticsearch indices, no table history in Zabbix
database
History_uint,History_str,History(dbl),history_text,History_log in
Elasticsearch and no table history in mysql cluster

10. Let check if It´s working

11. Thank you!
Questions ?
More Information:
Linkedin:
https://www.linkedin.com/in/leandro-totino-
pereira
Facebook:
https://www.facebook.com/leandro.totinopereira