SlideShare a Scribd company logo
1 of 25
Download to read offline
© 2023 VictoriaMetrics
What’s new in VictoriaLogs
Q3 2023
Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last
5 minutes a hour ago
Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last
5 minutes a hour ago
● _time:[2023-09,2023-10) offset 5d - return monthly logs 5 days before
September, 2023
Improved usability for _time filters
● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last
5 minutes a hour ago
● _time:[2023-09,2023-10) offset 5d - return monthly logs 5 days before
September, 2023
● Docs at https://docs.victoriametrics.com/VictoriaLogs/LogsQL.html#time-filter
Support for Promtail
● Accept logs from Promtail (aka logs’ shipper for Grafana Loki)
● Docs at
https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html
Support for Promtail
● Accept logs from Promtail (aka logs’ shipper for Grafana Loki)
● Docs at
https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html
● Previously supported log shippers: Filebeat, Fluentbit, Logstash, Vector
● See https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/
Support for read-only mode
● VictoriaLogs stops accepting new data if the remaining storage space becomes
lower than -storage.minFreeDiskSpace
Support for read-only mode
● VictoriaLogs stops accepting new data if the remaining storage space becomes
lower than -storage.minFreeDiskSpace
● It returns HTTP status code 429 (too many requests) to the client in this case
Support for read-only mode
● VictoriaLogs stops accepting new data if the remaining storage space becomes
lower than -storage.minFreeDiskSpace
● It returns HTTP status code 429 (too many requests) to the client in this case
● This protects from “out of disk space” errors
Better observability
● Expose vl_data_size_bytes{type=”storage”} and
vl_data_size_bytes{type=”indexdb”} metrics
Better observability
● Expose vl_data_size_bytes{type=”storage”} and
vl_data_size_bytes{type=”indexdb”} metrics
● Expose vl_http_request_duration_seconds histogram
Better data ingestion scalability
● VictoriaLogs had an artificial limit on data ingestion speed (~40K log entries per
second)
Better data ingestion scalability
● VictoriaLogs had an artificial limit on data ingestion speed (~40K log entries per
second)
● Now this limit has been lifted - data ingestion performance scales with CPU cores
available to VictoriaLogs
Bugfixes
● Fix possible panic when no data is written to VictoriaLogs for long time.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895
● Fix possible panic when the number of columns with unique names exceeds the
hard-coded limit.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762
● Fix possible “too big columnsHeaderSize” panic.
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4834
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
● Cluster version
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
● Cluster version
● Alerting
Roadmap
● Support additional data ingestion protocols - fluentd, syslog, journald
● Add missing functionality to LogsQL:
○ Stream context (aka grep -A -B -C)
○ Data extraction and transformation
○ Post-filtering
○ Stats calculations
○ Sorting and limiting
● Live tailing (aka tail -f)
● Improved web UI + integration with Grafana
● Snapshots and backups
● Object storage support (S3, GCS, MinIO)
● Cluster version
● Alerting
● See https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html

More Related Content

Similar to What’s new in VictoriaLogs (Q3 2023)

Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)DataWorks Summit
 
Introduction to apache kafka
Introduction to apache kafkaIntroduction to apache kafka
Introduction to apache kafkaSamuel Kerrien
 
Time series denver an introduction to prometheus
Time series denver   an introduction to prometheusTime series denver   an introduction to prometheus
Time series denver an introduction to prometheusBob Cotton
 
Key considerations in productionizing streaming applications
Key considerations in productionizing streaming applicationsKey considerations in productionizing streaming applications
Key considerations in productionizing streaming applicationsKafkaZone
 
Sprint 45 review
Sprint 45 reviewSprint 45 review
Sprint 45 reviewManageIQ
 
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayDatadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayC4Media
 
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...Amazon Web Services
 
Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...
Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...
Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...HostedbyConfluent
 
Data Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFixData Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFixC4Media
 
What's new in MariaDB Platform X3
What's new in MariaDB Platform X3What's new in MariaDB Platform X3
What's new in MariaDB Platform X3MariaDB plc
 
Sprint 43 Review
Sprint 43 ReviewSprint 43 Review
Sprint 43 ReviewManageIQ
 
Taming Oracle EBS R12.x Accrual Reconciliation Load
Taming Oracle EBS R12.x Accrual Reconciliation LoadTaming Oracle EBS R12.x Accrual Reconciliation Load
Taming Oracle EBS R12.x Accrual Reconciliation LoadJoshua Johnson, MIS
 
Sprint 46 review
Sprint 46 reviewSprint 46 review
Sprint 46 reviewManageIQ
 
Cinder project update at OpenStack Boston Summit May 2017
Cinder project update at OpenStack Boston Summit May 2017Cinder project update at OpenStack Boston Summit May 2017
Cinder project update at OpenStack Boston Summit May 2017Miroslav Halas
 
Moving Towards Better Upgrades in Kafka Streams
Moving Towards Better Upgrades in Kafka StreamsMoving Towards Better Upgrades in Kafka Streams
Moving Towards Better Upgrades in Kafka StreamsHostedbyConfluent
 
OSMC 2023 | Large-scale logging made easy by Alexandr Valialkin
OSMC 2023 | Large-scale logging made easy by Alexandr ValialkinOSMC 2023 | Large-scale logging made easy by Alexandr Valialkin
OSMC 2023 | Large-scale logging made easy by Alexandr ValialkinNETWAYS
 
How to improve ELK log pipeline performance
How to improve ELK log pipeline performanceHow to improve ELK log pipeline performance
How to improve ELK log pipeline performanceSteven Shim
 
Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...
Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...
Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...Grant McAlister
 

Similar to What’s new in VictoriaLogs (Q3 2023) (20)

Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)Kafka to the Maxka - (Kafka Performance Tuning)
Kafka to the Maxka - (Kafka Performance Tuning)
 
Introduction to apache kafka
Introduction to apache kafkaIntroduction to apache kafka
Introduction to apache kafka
 
Time series denver an introduction to prometheus
Time series denver   an introduction to prometheusTime series denver   an introduction to prometheus
Time series denver an introduction to prometheus
 
Key considerations in productionizing streaming applications
Key considerations in productionizing streaming applicationsKey considerations in productionizing streaming applications
Key considerations in productionizing streaming applications
 
Sprint 45 review
Sprint 45 reviewSprint 45 review
Sprint 45 review
 
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayDatadog: a Real-Time Metrics Database for One Quadrillion Points/Day
Datadog: a Real-Time Metrics Database for One Quadrillion Points/Day
 
Perf Tuning Short
Perf Tuning ShortPerf Tuning Short
Perf Tuning Short
 
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
 
Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...
Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...
Analyzing Petabyte Scale Financial Data with Apache Pinot and Apache Kafka | ...
 
Data Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFixData Science in the Cloud @StitchFix
Data Science in the Cloud @StitchFix
 
What's new in MariaDB Platform X3
What's new in MariaDB Platform X3What's new in MariaDB Platform X3
What's new in MariaDB Platform X3
 
Sprint 43 Review
Sprint 43 ReviewSprint 43 Review
Sprint 43 Review
 
Taming Oracle EBS R12.x Accrual Reconciliation Load
Taming Oracle EBS R12.x Accrual Reconciliation LoadTaming Oracle EBS R12.x Accrual Reconciliation Load
Taming Oracle EBS R12.x Accrual Reconciliation Load
 
Sprint 46 review
Sprint 46 reviewSprint 46 review
Sprint 46 review
 
Sprint 47
Sprint 47Sprint 47
Sprint 47
 
Cinder project update at OpenStack Boston Summit May 2017
Cinder project update at OpenStack Boston Summit May 2017Cinder project update at OpenStack Boston Summit May 2017
Cinder project update at OpenStack Boston Summit May 2017
 
Moving Towards Better Upgrades in Kafka Streams
Moving Towards Better Upgrades in Kafka StreamsMoving Towards Better Upgrades in Kafka Streams
Moving Towards Better Upgrades in Kafka Streams
 
OSMC 2023 | Large-scale logging made easy by Alexandr Valialkin
OSMC 2023 | Large-scale logging made easy by Alexandr ValialkinOSMC 2023 | Large-scale logging made easy by Alexandr Valialkin
OSMC 2023 | Large-scale logging made easy by Alexandr Valialkin
 
How to improve ELK log pipeline performance
How to improve ELK log pipeline performanceHow to improve ELK log pipeline performance
How to improve ELK log pipeline performance
 
Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...
Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...
Amazon RDS for PostgreSQL - Postgres Open 2016 - New Features and Lessons Lea...
 

More from VictoriaMetrics

VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics
 
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community UpdateVictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community UpdateVictoriaMetrics
 
VictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas ClusterVictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas ClusterVictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly DetectionVictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly DetectionVictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics UpdateVictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics UpdateVictoriaMetrics
 
December 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics UpdatesDecember 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics UpdatesVictoriaMetrics
 
Q3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community UpdateQ3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community UpdateVictoriaMetrics
 
Managed VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & UpdateManaged VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & UpdateVictoriaMetrics
 
VM Anomaly Detection: Introduction
VM Anomaly Detection: IntroductionVM Anomaly Detection: Introduction
VM Anomaly Detection: IntroductionVictoriaMetrics
 
Q3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetricsQ3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetricsVictoriaMetrics
 
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023VictoriaMetrics
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseVictoriaMetrics
 
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetricsVictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetricsVictoriaMetrics
 
VictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 RoadmapVictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 RoadmapVictoriaMetrics
 
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features HighlightsVictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features HighlightsVictoriaMetrics
 

More from VictoriaMetrics (17)

VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024
 
VictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community UpdateVictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community Update
 
VictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas ClusterVictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas Cluster
 
WEDOS & VictoriaMetrics
WEDOS & VictoriaMetricsWEDOS & VictoriaMetrics
WEDOS & VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly DetectionVictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly Detection
 
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics UpdateVictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
 
December 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics UpdatesDecember 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics Updates
 
Q3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community UpdateQ3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community Update
 
Managed VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & UpdateManaged VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & Update
 
VM Anomaly Detection: Introduction
VM Anomaly Detection: IntroductionVM Anomaly Detection: Introduction
VM Anomaly Detection: Introduction
 
Q3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetricsQ3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetrics
 
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
 
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetricsVictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
VictoriaMetrics 15/12 Meet Up: Updates on Managed VictoriaMetrics
 
VictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 RoadmapVictoriaMetrics 2023 Roadmap
VictoriaMetrics 2023 Roadmap
 
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features HighlightsVictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
VictoriaMetrics 15/12 Meet Up: 2022 Features Highlights
 

Recently uploaded

Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalLionel Briand
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprisepreethippts
 
SoftTeco - Software Development Company Profile
SoftTeco - Software Development Company ProfileSoftTeco - Software Development Company Profile
SoftTeco - Software Development Company Profileakrivarotava
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
Salesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZSalesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZABSYZ Inc
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringHironori Washizaki
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf31events.com
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shardsChristopher Curtin
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...confluent
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Anthony Dahanne
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identityteam-WIBU
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingShane Coughlan
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Developmentvyaparkranti
 

Recently uploaded (20)

Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprise
 
SoftTeco - Software Development Company Profile
SoftTeco - Software Development Company ProfileSoftTeco - Software Development Company Profile
SoftTeco - Software Development Company Profile
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. Salesforce
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
Salesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZSalesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZ
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their Engineering
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
Sending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdfSending Calendar Invites on SES and Calendarsnack.pdf
Sending Calendar Invites on SES and Calendarsnack.pdf
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards
 
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
Catch the Wave: SAP Event-Driven and Data Streaming for the Intelligence Ente...
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identity
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Development
 

What’s new in VictoriaLogs (Q3 2023)

  • 1. © 2023 VictoriaMetrics What’s new in VictoriaLogs Q3 2023
  • 2. Improved usability for _time filters ● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes
  • 3. Improved usability for _time filters ● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes ● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last 5 minutes a hour ago
  • 4. Improved usability for _time filters ● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes ● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last 5 minutes a hour ago ● _time:[2023-09,2023-10) offset 5d - return monthly logs 5 days before September, 2023
  • 5. Improved usability for _time filters ● _time:5m instead of _time:[now-5m,now] - return logs for the last 5 minutes ● _time:5m offset 1h instead of time:[now-1h5m,now-1h] - return logs for the last 5 minutes a hour ago ● _time:[2023-09,2023-10) offset 5d - return monthly logs 5 days before September, 2023 ● Docs at https://docs.victoriametrics.com/VictoriaLogs/LogsQL.html#time-filter
  • 6.
  • 7. Support for Promtail ● Accept logs from Promtail (aka logs’ shipper for Grafana Loki) ● Docs at https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html
  • 8. Support for Promtail ● Accept logs from Promtail (aka logs’ shipper for Grafana Loki) ● Docs at https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/Promtail.html ● Previously supported log shippers: Filebeat, Fluentbit, Logstash, Vector ● See https://docs.victoriametrics.com/VictoriaLogs/data-ingestion/
  • 9. Support for read-only mode ● VictoriaLogs stops accepting new data if the remaining storage space becomes lower than -storage.minFreeDiskSpace
  • 10. Support for read-only mode ● VictoriaLogs stops accepting new data if the remaining storage space becomes lower than -storage.minFreeDiskSpace ● It returns HTTP status code 429 (too many requests) to the client in this case
  • 11. Support for read-only mode ● VictoriaLogs stops accepting new data if the remaining storage space becomes lower than -storage.minFreeDiskSpace ● It returns HTTP status code 429 (too many requests) to the client in this case ● This protects from “out of disk space” errors
  • 12. Better observability ● Expose vl_data_size_bytes{type=”storage”} and vl_data_size_bytes{type=”indexdb”} metrics
  • 13. Better observability ● Expose vl_data_size_bytes{type=”storage”} and vl_data_size_bytes{type=”indexdb”} metrics ● Expose vl_http_request_duration_seconds histogram
  • 14. Better data ingestion scalability ● VictoriaLogs had an artificial limit on data ingestion speed (~40K log entries per second)
  • 15. Better data ingestion scalability ● VictoriaLogs had an artificial limit on data ingestion speed (~40K log entries per second) ● Now this limit has been lifted - data ingestion performance scales with CPU cores available to VictoriaLogs
  • 16. Bugfixes ● Fix possible panic when no data is written to VictoriaLogs for long time. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4895 ● Fix possible panic when the number of columns with unique names exceeds the hard-coded limit. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4762 ● Fix possible “too big columnsHeaderSize” panic. https://github.com/VictoriaMetrics/VictoriaMetrics/issues/4834
  • 17. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald
  • 18. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting
  • 19. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f)
  • 20. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f) ● Improved web UI + integration with Grafana
  • 21. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f) ● Improved web UI + integration with Grafana ● Snapshots and backups
  • 22. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f) ● Improved web UI + integration with Grafana ● Snapshots and backups ● Object storage support (S3, GCS, MinIO)
  • 23. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f) ● Improved web UI + integration with Grafana ● Snapshots and backups ● Object storage support (S3, GCS, MinIO) ● Cluster version
  • 24. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f) ● Improved web UI + integration with Grafana ● Snapshots and backups ● Object storage support (S3, GCS, MinIO) ● Cluster version ● Alerting
  • 25. Roadmap ● Support additional data ingestion protocols - fluentd, syslog, journald ● Add missing functionality to LogsQL: ○ Stream context (aka grep -A -B -C) ○ Data extraction and transformation ○ Post-filtering ○ Stats calculations ○ Sorting and limiting ● Live tailing (aka tail -f) ● Improved web UI + integration with Grafana ● Snapshots and backups ● Object storage support (S3, GCS, MinIO) ● Cluster version ● Alerting ● See https://docs.victoriametrics.com/VictoriaLogs/Roadmap.html