- Improved usability for _time filters - Support for Promtail - Support for read-only mode - Better observability - Better data ingestion scalability - Roadmap update

1. © 2023 VictoriaMetrics
What’s new in VictoriaLogs
Q3 2023

2. Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes

3. Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last
5 minutes a hour ago

4. Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last
5 minutes a hour ago
● _time:[2023-09,2023-10) offset 5d - return monthly logs 5 days before
September, 2023

5. Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last
5 minutes a hour ago
● _time:[2023-09,2023-10) offset 5d - return monthly logs 5 days before
September, 2023
● Docs at https://docs.victoriametrics.com/VictoriaLogs/LogsQL.html#time-filter

7. Support for Promtail
● Accept logs from Promtail (aka logs’ shipper for Grafana Loki)
● Docs at
https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html

8. Support for Promtail
● Accept logs from Promtail (aka logs’ shipper for Grafana Loki)
● Docs at
https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html
● Previously supported log shippers: Filebeat, Fluentbit, Logstash, Vector
● See https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/

9. Support for read-only mode
● VictoriaLogs stops accepting new data if the remaining storage space becomes
lower than -storage.minFreeDiskSpace

10. Support for read-only mode
● VictoriaLogs stops accepting new data if the remaining storage space becomes
lower than -storage.minFreeDiskSpace
● It returns HTTP status code 429 (too many requests) to the client in this case

11. Support for read-only mode
● VictoriaLogs stops accepting new data if the remaining storage space becomes
lower than -storage.minFreeDiskSpace
● It returns HTTP status code 429 (too many requests) to the client in this case
● This protects from “out of disk space” errors

12. Better observability
● Expose vl_data_size_bytes{type=”storage”} and
vl_data_size_bytes{type=”indexdb”} metrics

13. Better observability
● Expose vl_data_size_bytes{type=”storage”} and
vl_data_size_bytes{type=”indexdb”} metrics
● Expose vl_http_request_duration_seconds histogram

14. Better data ingestion scalability
● VictoriaLogs had an artificial limit on data ingestion speed (~40K log entries per
second)

15. Better data ingestion scalability
● VictoriaLogs had an artificial limit on data ingestion speed (~40K log entries per
second)
● Now this limit has been lifted - data ingestion performance scales with CPU cores
available to VictoriaLogs

16. Bugfixes
● Fix possible panic when no data is written to VictoriaLogs for long time.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895
● Fix possible panic when the number of columns with unique names exceeds the
hard-coded limit.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
● Fix possible “too big columnsHeaderSize” panic.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4834

17. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald

18. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting

19. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)

20. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana

21. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups

22. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)

23. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
● Cluster version

24. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
● Cluster version
● Alerting

25. Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
● Cluster version
● Alerting
● See https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html