SlideShare a Scribd company logo
What’s new in VictoriaLogs
Q2 2024
What is VictoriaLogs?
● Open source database for logs
What is VictoriaLogs?
● Open source database for logs
● Easy to setup and operate - just a single executable with sane default configs
What is VictoriaLogs?
● Open source database for logs
● Easy to setup and operate - just a single executable with sane default configs
● Works great with both structured and plaintext logs
What is VictoriaLogs?
● Open source database for logs
● Easy to setup and operate - just a single executable with sane default configs
● Works great with both structured and plaintext logs
● Uses up to 30x less RAM and up to 15x disk space than Elasticsearch
What is VictoriaLogs?
● Open source database for logs
● Easy to setup and operate - just a single executable with sane default configs
● Works great with both structured and plaintext logs
● Uses up to 30x less RAM and up to 15x disk space than Elasticsearch
● Provides simple yet powerful query language for logs - LogsQL
Improved querying HTTP API
Improved querying HTTP API
/select/logsql/query?query=...
● Returns all the log fields by default for simplified logs’ analysis (previously only
_time, _stream and _msg fields were returned)
Improved querying HTTP API
/select/logsql/query?query=...
● Returns all the log fields by default for simplified logs’ analysis (previously only
_time, _stream and _msg fields were returned)
● Accepts optional start and end args, which allow limiting query results to the
given time range
Improved querying HTTP API
/select/logsql/query?query=...
● Returns all the log fields by default for simplified logs’ analysis (previously only
_time, _stream and _msg fields were returned)
● Accepts optional start and end args, which allow limiting query results to the
given time range
● Accepts optional limit=N arg, which allows returning the last N log entries with
the biggest timestamps for the given query
Improved querying HTTP API
/select/logsql/hits?query=...&start=...&end=...&step=...
● Returns the number of matching logs for the given query per the given step on
the given [start … end] time range
Improved querying HTTP API
/select/logsql/hits?query=...&start=...&end=...&step=...
● Returns the number of matching logs for the given query per the given step on
the given [start … end] time range
● Allows building graphs for matching logs over time like VMUI for VictoriaMetrics
does
Improved querying HTTP API
/select/logsql/hits?query=...&start=...&end=...&step=...
● Returns the number of matching logs for the given query per the given step on
the given [start … end] time range
● Allows building graphs for matching logs over time like VMUI for VictoriaMetrics
does
● Already available in VMUI for VictoriaLogs!
Improved querying HTTP API
Improved querying HTTP API
/select/logsql/streams?query=...&start=...&end=...
● Returns log streams (aka application instances) for the given query on the given
[start … end] time range, with the number of logs in every stream
Improved querying HTTP API
/select/logsql/streams?query=...&start=...&end=...
● Returns log streams (aka application instances) for the given query on the given
[start … end] time range, with the number of logs in every stream
● Allows quickly locating streams with the biggest number of logs
Improved querying HTTP API
/select/logsql/streams?query=...&start=...&end=...
● Returns log streams (aka application instances) for the given query on the given
[start … end] time range, with the number of logs in every stream
● Allows quickly locating streams with the biggest number of logs
● Coming soon in VMUI for logs
Improved querying HTTP API
/select/logsql/field_names?query=...&start=...&end=...
● Returns field names seen in logs for the given query on the given [start … end]
time range, with the number of logs per each field name
Improved querying HTTP API
/select/logsql/field_names?query=...&start=...&end=...
● Returns field names seen in logs for the given query on the given [start … end]
time range, with the number of logs per each field name
● Allows building auto-suggestion for log field names in VMUI
Improved querying HTTP API
/select/logsql/field_names?query=...&start=...&end=...
● Returns field names seen in logs for the given query on the given [start … end]
time range, with the number of logs per each field name
● Allows building auto-suggestion for log field names in VMUI
● Coming soon in VMUI for logs
Improved querying HTTP API
/select/logsql/field_values?query=...&field=...&start=...&end=...
● Returns unique values for the given field seen in logs for the given query on the
given [start … end] time range, with the number of logs per each value
Improved querying HTTP API
/select/logsql/field_values?query=...&field=...&start=...&end=...
● Returns unique values for the given field seen in logs for the given query on the
given [start … end] time range, with the number of logs per each value
● Allows building auto-suggestion for values of the given field in VMUI
Improved querying HTTP API
/select/logsql/field_values?query=...&field=...&start=...&end=...
● Returns unique values for the given field seen in logs for the given query on the
given [start … end] time range, with the number of logs per each value
● Allows building auto-suggestion for values of the given field in VMUI
● Coming soon in VMUI for logs
Data ingestion via Syslog protocol
● Automatic parsing of Syslog fields for the following formats:
○ RFC3164
<PRI>MMM DD hh:mm:ss HOSTNAME APP-NAME[PROCID]: MESSAGE
Data ingestion via Syslog protocol
● Automatic parsing of Syslog fields for the following formats:
○ RFC3164
<PRI>MMM DD hh:mm:ss HOSTNAME APP-NAME[PROCID]: MESSAGE
○ RFC5424
<PRI>1 TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [STRUCTURED-DATA] MESSAGE
Data ingestion via Syslog protocol
● Supported transports:
○ UDP
○ TCP
○ TCP+TLS
Data ingestion via Syslog protocol
● Supported transports:
○ UDP
○ TCP
○ TCP+TLS
● Gzip and deflate compression support
Data ingestion via Syslog protocol
● Supported transports:
○ UDP
○ TCP
○ TCP+TLS
● Gzip and deflate compression support
● Ability to configure distinct TCP and UDP ports with distinct settings
Data ingestion via Syslog protocol
● Supported transports:
○ UDP
○ TCP
○ TCP+TLS
● Gzip and deflate compression support
● Ability to configure distinct TCP and UDP ports with distinct settings
● Automatic log streams with (hostname, app_name, app_id) fields
LogsQL improvements
● Filtering shorthands
● week_range and day_range filters
● Limiters
● Log analytics
● Data extraction and transformation
● Additional filtering
● Sorting
LogsQL improvements: filtering shorthands
● =”foo” is a shorthand for exact(“foo”)
LogsQL improvements: filtering shorthands
● =”foo” is a shorthand for exact(“foo”)
● ~”foo” is a shorthand for re(“foo”)
LogsQL improvements: filtering shorthands
● =”foo” is a shorthand for exact(“foo”)
● ~”foo” is a shorthand for re(“foo”)
● >N, >=N, <N, <=N are shorthands for range(N, inf), range[N, inf), range(-inf, N),
range(-inf, N]
LogsQL improvements: week_range and day_range
filters
● _time:week_range[Mon, Fri] selects logs with timestamps at workdays only
LogsQL improvements: week_range and day_range
filters
● _time:week_range[Mon, Fri] selects logs with timestamps at workdays only
● _time:day_range[08:00, 16:00) selects logs with timestamps between 08:00 and
16:00
LogsQL improvements: week_range and day_range
filters
● _time:week_range[Mon, Fri] selects logs with timestamps at workdays only
● _time:day_range[08:00, 16:00) selects logs with timestamps between 08:00 and
16:00
● Both filters support timezone offset:
○ _time:week_range[Mon, Fri] offset 5h30m
○ _time:day_range[08:00, 16:00) offset -2h
LogsQL improvements: limiters
● | fields f1, …, fN limits the number of returned log fields to the given f1, …, fN list
LogsQL improvements: limiters
● | fields f1, …, fN limits the number of returned log fields to the given f1, …, fN list
● | delete f1, …, fN drops f1, …, fN fields from results
LogsQL improvements: limiters
● | fields f1, …, fN limits the number of returned log fields to the given f1, …, fN list
● | delete f1, …, fN drops f1, …, fN fields from results
● | limit N limits the number of returned logs to N
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
○ q | min(f) returns the minimum value for f field seen in matching logs for q
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
○ q | min(f) returns the minimum value for f field seen in matching logs for q
○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
○ q | min(f) returns the minimum value for f field seen in matching logs for q
○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q
○ q | uniq_values(f) returns unique values for f field seen in matching logs for q
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
○ q | min(f) returns the minimum value for f field seen in matching logs for q
○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q
○ q | uniq_values(f) returns unique values for f field seen in matching logs for q
○ There are other functions - see
https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions
LogsQL improvements: log analytics
● | stats … calculates various stats:
○ q | count() calculates the number of matching logs for q
○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
○ q | min(f) returns the minimum value for f field seen in matching logs for q
○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q
○ q | uniq_values(f) returns unique values for f field seen in matching logs for q
○ There are other functions - see
https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions
● Multiple stats can be calculated in a single query:
_time:5m | stats count() rows, count_uniq(ip) ips
LogsQL improvements: log analytics
● | stats by (f1, …, fN) … calculates individual stats by the given (f1, …, fN) fields:
_time:5m | stats by (hostname)
count() hits,
count_uniq(ip) ips
LogsQL improvements: log analytics
● | stats by (f1, …, fN) … calculates individual stats by the given (f1, …, fN) fields:
_time:5m | stats by (hostname)
count() hits,
count_uniq(ip) ips
● Individual filters can be applied per each stats:
_time:5m | stats by (hostname)
count() as hits_total,
count() if (status_code:=200) hits_success,
count() if (status_code:>=400) hits_failure
LogsQL improvements: log analytics
● | stats by (f1, …, fN) … calculates individual stats by the given (f1, …, fN) fields:
_time:5m | stats by (hostname)
count() hits,
count_uniq(ip) ips
● Individual filters can be applied per each stats:
_time:5m | stats by (hostname)
count() as hits_total,
count() if (status_code:=200) hits_success,
count() if (status_code:>=400) hits_failure
● A sample of log entry can be selected per each group via row_any, row_min or
row_max functions:
_time:5m | stats by (hostname)
count() hits,
row_min(_time) first_log,
row_max(_time) last_log
LogsQL improvements: log analytics
● | stats by (_time:bucket) … groups stats by time buckets:
_time:10d | stats by (_time:day) count() | sort by (_time)
LogsQL improvements: log analytics
● | stats by (_time:bucket) … groups stats by time buckets:
_time:10d | stats by (_time:day) count() | sort by (_time)
● Timezone offset support:
_time:10d |
stats by (_time:day offset 5h30m) count() |
sort by (_time)
LogsQL improvements: log analytics
● | stats by (ip:/mask) … groups stats by IPv4 subnet masks:
_time:10d | stats by (ip:/24) count() hits | sort by (hits desc)
LogsQL improvements: data extraction and
transformation
● | extract … extracts data from any log field:
_time:5m |
extract “ip=<ip>,” |
stats by (ip) count() hits |
sort by (hits desc)
LogsQL improvements: data extraction and
transformation
● | extract … extracts data from any log field:
_time:5m |
extract “ip=<ip>,” |
stats by (ip) count() hits |
sort by (hits desc)
● There are also:
○ | extract_regexp … - extracts data with the help of regular expressions
○ | unpack_json - unpacks JSON fields from text
○ | unpack_logfmt - unpacks logfmt fields from text
○ | unpack_syslog - unpacks syslog fields from text
LogsQL improvements: data extraction and
transformation
● | format … creates a log field according to the given format, which can refer other
log fields
LogsQL improvements: data extraction and
transformation
● | format … creates a log field according to the given format, which can refer other
log fields
● | replace … replaces substrings in the given log field
LogsQL improvements: data extraction and
transformation
● | format … creates a log field according to the given format, which can refer other
log fields
● | replace … replaces substrings in the given log field
● | replace_regexp … replaces substrings matching the given regexps
LogsQL improvements: data extraction and
transformation
● | format … creates a log field according to the given format, which can refer other
log fields
● | replace … replaces substrings in the given log field
● | replace_regexp … replaces substrings matching the given regexps
● | math … performs mathematical calculations over log fields
LogsQL improvements: additional filtering
● The selected logs can be additionally filtered at any stage via | filter … pipe:
_time:5m |
stats by (host) count() hits |
filter hits:>1000
LogsQL improvements: sorting
● By default the returned results aren’t sorted because of performance reasons
LogsQL improvements: sorting
● By default the returned results aren’t sorted because of performance reasons
● | sort … pipe can be applied for sorting the results:
_time:5m |
stats by (host) count() hits |
sort by (hits desc) limit 10
LogsQL improvements: sorting
● By default the returned results aren’t sorted because of performance reasons
● | sort … pipe can be applied for sorting the results:
_time:5m |
stats by (host) count() hits |
sort by (hits desc) limit 10
● Sorting can be performed by multiple fields with individual order per each field:
_time:5m |
stats by (host) count() hits |
sort by (hits desc, host)
Roadmap
● Accept logs via OpenTelemetry protocol
Roadmap
● Accept logs via OpenTelemetry protocol
● VMUI improvements based on HTTP querying API
Roadmap
● Accept logs via OpenTelemetry protocol
● VMUI improvements based on HTTP querying API
● Improve Grafana plugin for VictoriaLogs -
https://github.com/VictoriaMetrics/victorialogs-datasource
Roadmap
● Accept logs via OpenTelemetry protocol
● VMUI improvements based on HTTP querying API
● Improve Grafana plugin for VictoriaLogs -
https://github.com/VictoriaMetrics/victorialogs-datasource
● Cluster version
○ Try single-node VictoriaLogs - it can replace 30-node Elasticsearch cluster in production
Roadmap
● Accept logs via OpenTelemetry protocol
● VMUI improvements based on HTTP querying API
● Improve Grafana plugin for VictoriaLogs -
https://github.com/VictoriaMetrics/victorialogs-datasource
● Cluster version
○ Try single-node VictoriaLogs - it can replace 30-node Elasticsearch cluster in production
● Transparent historical data migration to object storage
○ Try single-node VictoriaLogs with persistent volumes - it compresses 1TB of production logs from
Kubernetes to 20GB
Roadmap
● Accept logs via OpenTelemetry protocol
● VMUI improvements based on HTTP querying API
● Improve Grafana plugin for VictoriaLogs -
https://github.com/VictoriaMetrics/victorialogs-datasource
● Cluster version
○ Try single-node VictoriaLogs - it can replace 30-node Elasticsearch cluster in production
● Transparent historical data migration to object storage
○ Try single-node VictoriaLogs with persistent volumes - it compresses 1TB of production logs from
Kubernetes to 20GB
● See https://docs.victoriametrics.com/victorialogs/roadmap/
Try VictoriaLogs right now!

More Related Content

Similar to What’s New in VictoriaLogs - Q2 2024 Update

Sprint 44 review
Sprint 44 reviewSprint 44 review
Sprint 44 review
ManageIQ
 
Load testing in Zonky with Gatling
Load testing in Zonky with GatlingLoad testing in Zonky with Gatling
Load testing in Zonky with Gatling
Petr Vlček
 
(Open) MPI, Parallel Computing, Life, the Universe, and Everything
(Open) MPI, Parallel Computing, Life, the Universe, and Everything(Open) MPI, Parallel Computing, Life, the Universe, and Everything
(Open) MPI, Parallel Computing, Life, the Universe, and Everything
Jeff Squyres
 
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
NETWAYS
 
Guy Barrette: Afficher des données en temps réel dans PowerBI
Guy Barrette: Afficher des données en temps réel dans PowerBIGuy Barrette: Afficher des données en temps réel dans PowerBI
Guy Barrette: Afficher des données en temps réel dans PowerBI
MSDEVMTL
 
PostgreSQL FTS Solutions FOSDEM 2013 - PGDAY
PostgreSQL FTS Solutions FOSDEM 2013 - PGDAYPostgreSQL FTS Solutions FOSDEM 2013 - PGDAY
PostgreSQL FTS Solutions FOSDEM 2013 - PGDAY
Emanuel Calvo
 
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...
Ontico
 
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesThe Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
Databricks
 
Experiences building a distributed shared log on RADOS - Noah Watkins
Experiences building a distributed shared log on RADOS - Noah WatkinsExperiences building a distributed shared log on RADOS - Noah Watkins
Experiences building a distributed shared log on RADOS - Noah Watkins
Ceph Community
 
Query and audit logging in cassandra
Query and audit logging in cassandraQuery and audit logging in cassandra
Query and audit logging in cassandra
Vinay Kumar Chella
 
Nzitf Velociraptor Workshop
Nzitf Velociraptor WorkshopNzitf Velociraptor Workshop
Nzitf Velociraptor Workshop
Velocidex Enterprises
 
The burden of a successful feature: Scaling our real time logging platform
The burden of a successful feature: Scaling our real time logging platformThe burden of a successful feature: Scaling our real time logging platform
The burden of a successful feature: Scaling our real time logging platform
Fastly
 
How We Added Replication to QuestDB - JonTheBeach
How We Added Replication to QuestDB - JonTheBeachHow We Added Replication to QuestDB - JonTheBeach
How We Added Replication to QuestDB - JonTheBeach
javier ramirez
 
Revealing ALLSTOCKER
Revealing ALLSTOCKERRevealing ALLSTOCKER
Revealing ALLSTOCKER
Masashi Umezawa
 
OpenTelemetry For Operators
OpenTelemetry For OperatorsOpenTelemetry For Operators
OpenTelemetry For Operators
Kevin Brockhoff
 
Continuous Go Profiling & Observability
Continuous Go Profiling & ObservabilityContinuous Go Profiling & Observability
Continuous Go Profiling & Observability
ScyllaDB
 
Loggly - Benchmarking 5 Node.js Logging Libraries
Loggly - Benchmarking 5 Node.js Logging LibrariesLoggly - Benchmarking 5 Node.js Logging Libraries
Loggly - Benchmarking 5 Node.js Logging Libraries
SolarWinds Loggly
 
Logs aggregation and analysis
Logs aggregation and analysisLogs aggregation and analysis
Logs aggregation and analysis
Divante
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
MariaDB plc
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
MariaDB plc
 

Similar to What’s New in VictoriaLogs - Q2 2024 Update (20)

Sprint 44 review
Sprint 44 reviewSprint 44 review
Sprint 44 review
 
Load testing in Zonky with Gatling
Load testing in Zonky with GatlingLoad testing in Zonky with Gatling
Load testing in Zonky with Gatling
 
(Open) MPI, Parallel Computing, Life, the Universe, and Everything
(Open) MPI, Parallel Computing, Life, the Universe, and Everything(Open) MPI, Parallel Computing, Life, the Universe, and Everything
(Open) MPI, Parallel Computing, Life, the Universe, and Everything
 
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
OSMC 2023 | What’s new with Grafana Labs’s Open Source Observability stack by...
 
Guy Barrette: Afficher des données en temps réel dans PowerBI
Guy Barrette: Afficher des données en temps réel dans PowerBIGuy Barrette: Afficher des données en temps réel dans PowerBI
Guy Barrette: Afficher des données en temps réel dans PowerBI
 
PostgreSQL FTS Solutions FOSDEM 2013 - PGDAY
PostgreSQL FTS Solutions FOSDEM 2013 - PGDAYPostgreSQL FTS Solutions FOSDEM 2013 - PGDAY
PostgreSQL FTS Solutions FOSDEM 2013 - PGDAY
 
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...
 
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesThe Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
 
Experiences building a distributed shared log on RADOS - Noah Watkins
Experiences building a distributed shared log on RADOS - Noah WatkinsExperiences building a distributed shared log on RADOS - Noah Watkins
Experiences building a distributed shared log on RADOS - Noah Watkins
 
Query and audit logging in cassandra
Query and audit logging in cassandraQuery and audit logging in cassandra
Query and audit logging in cassandra
 
Nzitf Velociraptor Workshop
Nzitf Velociraptor WorkshopNzitf Velociraptor Workshop
Nzitf Velociraptor Workshop
 
The burden of a successful feature: Scaling our real time logging platform
The burden of a successful feature: Scaling our real time logging platformThe burden of a successful feature: Scaling our real time logging platform
The burden of a successful feature: Scaling our real time logging platform
 
How We Added Replication to QuestDB - JonTheBeach
How We Added Replication to QuestDB - JonTheBeachHow We Added Replication to QuestDB - JonTheBeach
How We Added Replication to QuestDB - JonTheBeach
 
Revealing ALLSTOCKER
Revealing ALLSTOCKERRevealing ALLSTOCKER
Revealing ALLSTOCKER
 
OpenTelemetry For Operators
OpenTelemetry For OperatorsOpenTelemetry For Operators
OpenTelemetry For Operators
 
Continuous Go Profiling & Observability
Continuous Go Profiling & ObservabilityContinuous Go Profiling & Observability
Continuous Go Profiling & Observability
 
Loggly - Benchmarking 5 Node.js Logging Libraries
Loggly - Benchmarking 5 Node.js Logging LibrariesLoggly - Benchmarking 5 Node.js Logging Libraries
Loggly - Benchmarking 5 Node.js Logging Libraries
 
Logs aggregation and analysis
Logs aggregation and analysisLogs aggregation and analysis
Logs aggregation and analysis
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
 

More from VictoriaMetrics

VictoriaMetrics in the Cloud: Meetup Q2 June 2024
VictoriaMetrics in the Cloud: Meetup Q2 June 2024VictoriaMetrics in the Cloud: Meetup Q2 June 2024
VictoriaMetrics in the Cloud: Meetup Q2 June 2024
VictoriaMetrics
 
VictoriaMetrics Community & News Update: Q2 Meet Up 2024
VictoriaMetrics Community & News Update: Q2 Meet Up 2024VictoriaMetrics Community & News Update: Q2 Meet Up 2024
VictoriaMetrics Community & News Update: Q2 Meet Up 2024
VictoriaMetrics
 
The Benefits of VictoriaMetrics: BlackSwift's Real-World Experience
The Benefits of VictoriaMetrics: BlackSwift's Real-World ExperienceThe Benefits of VictoriaMetrics: BlackSwift's Real-World Experience
The Benefits of VictoriaMetrics: BlackSwift's Real-World Experience
VictoriaMetrics
 
What’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 UpdateWhat’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 Update
VictoriaMetrics
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
VictoriaMetrics
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics
 
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community UpdateVictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics
 
VictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas ClusterVictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas Cluster
VictoriaMetrics
 
WEDOS & VictoriaMetrics
WEDOS & VictoriaMetricsWEDOS & VictoriaMetrics
WEDOS & VictoriaMetrics
VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly DetectionVictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics UpdateVictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics
 
December 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics UpdatesDecember 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics Updates
VictoriaMetrics
 
Q3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community UpdateQ3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community Update
VictoriaMetrics
 
Managed VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & UpdateManaged VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & Update
VictoriaMetrics
 
VM Anomaly Detection: Introduction
VM Anomaly Detection: IntroductionVM Anomaly Detection: Introduction
VM Anomaly Detection: Introduction
VictoriaMetrics
 
Q3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetricsQ3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetrics
VictoriaMetrics
 
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
VictoriaMetrics
 

More from VictoriaMetrics (20)

VictoriaMetrics in the Cloud: Meetup Q2 June 2024
VictoriaMetrics in the Cloud: Meetup Q2 June 2024VictoriaMetrics in the Cloud: Meetup Q2 June 2024
VictoriaMetrics in the Cloud: Meetup Q2 June 2024
 
VictoriaMetrics Community & News Update: Q2 Meet Up 2024
VictoriaMetrics Community & News Update: Q2 Meet Up 2024VictoriaMetrics Community & News Update: Q2 Meet Up 2024
VictoriaMetrics Community & News Update: Q2 Meet Up 2024
 
The Benefits of VictoriaMetrics: BlackSwift's Real-World Experience
The Benefits of VictoriaMetrics: BlackSwift's Real-World ExperienceThe Benefits of VictoriaMetrics: BlackSwift's Real-World Experience
The Benefits of VictoriaMetrics: BlackSwift's Real-World Experience
 
What’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 UpdateWhat’s new in VictoriaMetrics - Q2 2024 Update
What’s new in VictoriaMetrics - Q2 2024 Update
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
VictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community UpdateVictoriaMetrics December 2023 Meetup: Community Update
VictoriaMetrics December 2023 Meetup: Community Update
 
VictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas ClusterVictoriaMetrics for the Atlas Cluster
VictoriaMetrics for the Atlas Cluster
 
WEDOS & VictoriaMetrics
WEDOS & VictoriaMetricsWEDOS & VictoriaMetrics
WEDOS & VictoriaMetrics
 
VictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly DetectionVictoriaMetrics December 2023 Meetup: Anomaly Detection
VictoriaMetrics December 2023 Meetup: Anomaly Detection
 
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics UpdateVictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
VictoriaMetrics December 2023 Meetup: Managed VictoriaMetrics Update
 
December 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics UpdatesDecember 2024 Meetup: Welcome & VictoriaMetrics Updates
December 2024 Meetup: Welcome & VictoriaMetrics Updates
 
Q3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community UpdateQ3 Meet Up '23 - Community Update
Q3 Meet Up '23 - Community Update
 
Managed VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & UpdateManaged VictoriaMetrics: Intro & Update
Managed VictoriaMetrics: Intro & Update
 
VM Anomaly Detection: Introduction
VM Anomaly Detection: IntroductionVM Anomaly Detection: Introduction
VM Anomaly Detection: Introduction
 
Q3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetricsQ3 2023 Meet Up: What's New in VictoriaMetrics
Q3 2023 Meet Up: What's New in VictoriaMetrics
 
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
VictoriaMetrics: Welcome to the Virtual Meet Up March 2023
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
 

Recently uploaded

11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf
kalichargn70th171
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
rachitkumar09887
 
Attendance Tracking From Paper To Digital
Attendance Tracking From Paper To DigitalAttendance Tracking From Paper To Digital
Attendance Tracking From Paper To Digital
Task Tracker
 
ENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentationENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentation
sofiafernandezon
 
Google ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learningGoogle ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learning
VishrutGoyani1
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
neshakor5152
 
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
josephinedrea942
 
To Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance SheetsTo Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance Sheets
Task Tracker
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
akshesh doshi
 
welcome to presentation on Google Apps
welcome to   presentation on Google Appswelcome to   presentation on Google Apps
welcome to presentation on Google Apps
AsifKarimJim
 
NYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction InnovationNYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction Innovation
NYGGS Construction ERP Software
 
Mobile App Development Company in Noida - Drona Infotech
Mobile App Development Company in Noida - Drona InfotechMobile App Development Company in Noida - Drona Infotech
Mobile App Development Company in Noida - Drona Infotech
Mobile App Development Company in Noida - Drona Infotech
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
singhlata50dh
 
Odoo E-commerce website development guides
Odoo E-commerce website development guidesOdoo E-commerce website development guides
Odoo E-commerce website development guides
jhkdigitalmarketing
 
UMiami degree offer diploma Transcript
UMiami degree offer diploma TranscriptUMiami degree offer diploma Transcript
UMiami degree offer diploma Transcript
attueb
 
Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
confluent
 
Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …
908dutch
 
Folding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a seriesFolding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a series
Philip Schwarz
 
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
jealousviolet
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 

Recently uploaded (20)

11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf
 
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
Agra Girls Call Agra 0X0000000X Unlimited Short Providing Girls Service Avail...
 
Attendance Tracking From Paper To Digital
Attendance Tracking From Paper To DigitalAttendance Tracking From Paper To Digital
Attendance Tracking From Paper To Digital
 
ENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentationENISA Threat Landscape 2023 documentation
ENISA Threat Landscape 2023 documentation
 
Google ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learningGoogle ML-Kit - Understanding on-device machine learning
Google ML-Kit - Understanding on-device machine learning
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
HIRE A HACKER FOR CHEATING HUSBAND/WIFE)
 
To Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance SheetsTo Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance Sheets
 
ThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and DjangoThaiPy meetup - Indexes and Django
ThaiPy meetup - Indexes and Django
 
welcome to presentation on Google Apps
welcome to   presentation on Google Appswelcome to   presentation on Google Apps
welcome to presentation on Google Apps
 
NYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction InnovationNYGGS 360: A Complete ERP for Construction Innovation
NYGGS 360: A Complete ERP for Construction Innovation
 
Mobile App Development Company in Noida - Drona Infotech
Mobile App Development Company in Noida - Drona InfotechMobile App Development Company in Noida - Drona Infotech
Mobile App Development Company in Noida - Drona Infotech
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
 
Odoo E-commerce website development guides
Odoo E-commerce website development guidesOdoo E-commerce website development guides
Odoo E-commerce website development guides
 
UMiami degree offer diploma Transcript
UMiami degree offer diploma TranscriptUMiami degree offer diploma Transcript
UMiami degree offer diploma Transcript
 
Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
 
Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …Prada Group Reports Strong Growth in First Quarter …
Prada Group Reports Strong Growth in First Quarter …
 
Folding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a seriesFolding Cheat Sheet #7 - seventh in a series
Folding Cheat Sheet #7 - seventh in a series
 
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 

What’s New in VictoriaLogs - Q2 2024 Update

  • 1. What’s new in VictoriaLogs Q2 2024
  • 2. What is VictoriaLogs? ● Open source database for logs
  • 3. What is VictoriaLogs? ● Open source database for logs ● Easy to setup and operate - just a single executable with sane default configs
  • 4. What is VictoriaLogs? ● Open source database for logs ● Easy to setup and operate - just a single executable with sane default configs ● Works great with both structured and plaintext logs
  • 5. What is VictoriaLogs? ● Open source database for logs ● Easy to setup and operate - just a single executable with sane default configs ● Works great with both structured and plaintext logs ● Uses up to 30x less RAM and up to 15x disk space than Elasticsearch
  • 6. What is VictoriaLogs? ● Open source database for logs ● Easy to setup and operate - just a single executable with sane default configs ● Works great with both structured and plaintext logs ● Uses up to 30x less RAM and up to 15x disk space than Elasticsearch ● Provides simple yet powerful query language for logs - LogsQL
  • 8. Improved querying HTTP API /select/logsql/query?query=... ● Returns all the log fields by default for simplified logs’ analysis (previously only _time, _stream and _msg fields were returned)
  • 9. Improved querying HTTP API /select/logsql/query?query=... ● Returns all the log fields by default for simplified logs’ analysis (previously only _time, _stream and _msg fields were returned) ● Accepts optional start and end args, which allow limiting query results to the given time range
  • 10. Improved querying HTTP API /select/logsql/query?query=... ● Returns all the log fields by default for simplified logs’ analysis (previously only _time, _stream and _msg fields were returned) ● Accepts optional start and end args, which allow limiting query results to the given time range ● Accepts optional limit=N arg, which allows returning the last N log entries with the biggest timestamps for the given query
  • 11. Improved querying HTTP API /select/logsql/hits?query=...&start=...&end=...&step=... ● Returns the number of matching logs for the given query per the given step on the given [start … end] time range
  • 12. Improved querying HTTP API /select/logsql/hits?query=...&start=...&end=...&step=... ● Returns the number of matching logs for the given query per the given step on the given [start … end] time range ● Allows building graphs for matching logs over time like VMUI for VictoriaMetrics does
  • 13. Improved querying HTTP API /select/logsql/hits?query=...&start=...&end=...&step=... ● Returns the number of matching logs for the given query per the given step on the given [start … end] time range ● Allows building graphs for matching logs over time like VMUI for VictoriaMetrics does ● Already available in VMUI for VictoriaLogs!
  • 15. Improved querying HTTP API /select/logsql/streams?query=...&start=...&end=... ● Returns log streams (aka application instances) for the given query on the given [start … end] time range, with the number of logs in every stream
  • 16. Improved querying HTTP API /select/logsql/streams?query=...&start=...&end=... ● Returns log streams (aka application instances) for the given query on the given [start … end] time range, with the number of logs in every stream ● Allows quickly locating streams with the biggest number of logs
  • 17. Improved querying HTTP API /select/logsql/streams?query=...&start=...&end=... ● Returns log streams (aka application instances) for the given query on the given [start … end] time range, with the number of logs in every stream ● Allows quickly locating streams with the biggest number of logs ● Coming soon in VMUI for logs
  • 18. Improved querying HTTP API /select/logsql/field_names?query=...&start=...&end=... ● Returns field names seen in logs for the given query on the given [start … end] time range, with the number of logs per each field name
  • 19. Improved querying HTTP API /select/logsql/field_names?query=...&start=...&end=... ● Returns field names seen in logs for the given query on the given [start … end] time range, with the number of logs per each field name ● Allows building auto-suggestion for log field names in VMUI
  • 20. Improved querying HTTP API /select/logsql/field_names?query=...&start=...&end=... ● Returns field names seen in logs for the given query on the given [start … end] time range, with the number of logs per each field name ● Allows building auto-suggestion for log field names in VMUI ● Coming soon in VMUI for logs
  • 21. Improved querying HTTP API /select/logsql/field_values?query=...&field=...&start=...&end=... ● Returns unique values for the given field seen in logs for the given query on the given [start … end] time range, with the number of logs per each value
  • 22. Improved querying HTTP API /select/logsql/field_values?query=...&field=...&start=...&end=... ● Returns unique values for the given field seen in logs for the given query on the given [start … end] time range, with the number of logs per each value ● Allows building auto-suggestion for values of the given field in VMUI
  • 23. Improved querying HTTP API /select/logsql/field_values?query=...&field=...&start=...&end=... ● Returns unique values for the given field seen in logs for the given query on the given [start … end] time range, with the number of logs per each value ● Allows building auto-suggestion for values of the given field in VMUI ● Coming soon in VMUI for logs
  • 24. Data ingestion via Syslog protocol ● Automatic parsing of Syslog fields for the following formats: ○ RFC3164 <PRI>MMM DD hh:mm:ss HOSTNAME APP-NAME[PROCID]: MESSAGE
  • 25. Data ingestion via Syslog protocol ● Automatic parsing of Syslog fields for the following formats: ○ RFC3164 <PRI>MMM DD hh:mm:ss HOSTNAME APP-NAME[PROCID]: MESSAGE ○ RFC5424 <PRI>1 TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [STRUCTURED-DATA] MESSAGE
  • 26. Data ingestion via Syslog protocol ● Supported transports: ○ UDP ○ TCP ○ TCP+TLS
  • 27. Data ingestion via Syslog protocol ● Supported transports: ○ UDP ○ TCP ○ TCP+TLS ● Gzip and deflate compression support
  • 28. Data ingestion via Syslog protocol ● Supported transports: ○ UDP ○ TCP ○ TCP+TLS ● Gzip and deflate compression support ● Ability to configure distinct TCP and UDP ports with distinct settings
  • 29. Data ingestion via Syslog protocol ● Supported transports: ○ UDP ○ TCP ○ TCP+TLS ● Gzip and deflate compression support ● Ability to configure distinct TCP and UDP ports with distinct settings ● Automatic log streams with (hostname, app_name, app_id) fields
  • 30. LogsQL improvements ● Filtering shorthands ● week_range and day_range filters ● Limiters ● Log analytics ● Data extraction and transformation ● Additional filtering ● Sorting
  • 31. LogsQL improvements: filtering shorthands ● =”foo” is a shorthand for exact(“foo”)
  • 32. LogsQL improvements: filtering shorthands ● =”foo” is a shorthand for exact(“foo”) ● ~”foo” is a shorthand for re(“foo”)
  • 33. LogsQL improvements: filtering shorthands ● =”foo” is a shorthand for exact(“foo”) ● ~”foo” is a shorthand for re(“foo”) ● >N, >=N, <N, <=N are shorthands for range(N, inf), range[N, inf), range(-inf, N), range(-inf, N]
  • 34. LogsQL improvements: week_range and day_range filters ● _time:week_range[Mon, Fri] selects logs with timestamps at workdays only
  • 35. LogsQL improvements: week_range and day_range filters ● _time:week_range[Mon, Fri] selects logs with timestamps at workdays only ● _time:day_range[08:00, 16:00) selects logs with timestamps between 08:00 and 16:00
  • 36. LogsQL improvements: week_range and day_range filters ● _time:week_range[Mon, Fri] selects logs with timestamps at workdays only ● _time:day_range[08:00, 16:00) selects logs with timestamps between 08:00 and 16:00 ● Both filters support timezone offset: ○ _time:week_range[Mon, Fri] offset 5h30m ○ _time:day_range[08:00, 16:00) offset -2h
  • 37. LogsQL improvements: limiters ● | fields f1, …, fN limits the number of returned log fields to the given f1, …, fN list
  • 38. LogsQL improvements: limiters ● | fields f1, …, fN limits the number of returned log fields to the given f1, …, fN list ● | delete f1, …, fN drops f1, …, fN fields from results
  • 39. LogsQL improvements: limiters ● | fields f1, …, fN limits the number of returned log fields to the given f1, …, fN list ● | delete f1, …, fN drops f1, …, fN fields from results ● | limit N limits the number of returned logs to N
  • 40. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q
  • 41. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q ○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q
  • 42. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q ○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q ○ q | min(f) returns the minimum value for f field seen in matching logs for q
  • 43. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q ○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q ○ q | min(f) returns the minimum value for f field seen in matching logs for q ○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q
  • 44. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q ○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q ○ q | min(f) returns the minimum value for f field seen in matching logs for q ○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q ○ q | uniq_values(f) returns unique values for f field seen in matching logs for q
  • 45. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q ○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q ○ q | min(f) returns the minimum value for f field seen in matching logs for q ○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q ○ q | uniq_values(f) returns unique values for f field seen in matching logs for q ○ There are other functions - see https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions
  • 46. LogsQL improvements: log analytics ● | stats … calculates various stats: ○ q | count() calculates the number of matching logs for q ○ q | count_uniq(f) calculates the number of unique values for f field seen in matching logs for q ○ q | min(f) returns the minimum value for f field seen in matching logs for q ○ q | quantile(phi, f) returns phi-percentile for f field values seen in matching logs for q ○ q | uniq_values(f) returns unique values for f field seen in matching logs for q ○ There are other functions - see https://docs.victoriametrics.com/victorialogs/logsql/#stats-pipe-functions ● Multiple stats can be calculated in a single query: _time:5m | stats count() rows, count_uniq(ip) ips
  • 47. LogsQL improvements: log analytics ● | stats by (f1, …, fN) … calculates individual stats by the given (f1, …, fN) fields: _time:5m | stats by (hostname) count() hits, count_uniq(ip) ips
  • 48. LogsQL improvements: log analytics ● | stats by (f1, …, fN) … calculates individual stats by the given (f1, …, fN) fields: _time:5m | stats by (hostname) count() hits, count_uniq(ip) ips ● Individual filters can be applied per each stats: _time:5m | stats by (hostname) count() as hits_total, count() if (status_code:=200) hits_success, count() if (status_code:>=400) hits_failure
  • 49. LogsQL improvements: log analytics ● | stats by (f1, …, fN) … calculates individual stats by the given (f1, …, fN) fields: _time:5m | stats by (hostname) count() hits, count_uniq(ip) ips ● Individual filters can be applied per each stats: _time:5m | stats by (hostname) count() as hits_total, count() if (status_code:=200) hits_success, count() if (status_code:>=400) hits_failure ● A sample of log entry can be selected per each group via row_any, row_min or row_max functions: _time:5m | stats by (hostname) count() hits, row_min(_time) first_log, row_max(_time) last_log
  • 50. LogsQL improvements: log analytics ● | stats by (_time:bucket) … groups stats by time buckets: _time:10d | stats by (_time:day) count() | sort by (_time)
  • 51. LogsQL improvements: log analytics ● | stats by (_time:bucket) … groups stats by time buckets: _time:10d | stats by (_time:day) count() | sort by (_time) ● Timezone offset support: _time:10d | stats by (_time:day offset 5h30m) count() | sort by (_time)
  • 52. LogsQL improvements: log analytics ● | stats by (ip:/mask) … groups stats by IPv4 subnet masks: _time:10d | stats by (ip:/24) count() hits | sort by (hits desc)
  • 53. LogsQL improvements: data extraction and transformation ● | extract … extracts data from any log field: _time:5m | extract “ip=<ip>,” | stats by (ip) count() hits | sort by (hits desc)
  • 54. LogsQL improvements: data extraction and transformation ● | extract … extracts data from any log field: _time:5m | extract “ip=<ip>,” | stats by (ip) count() hits | sort by (hits desc) ● There are also: ○ | extract_regexp … - extracts data with the help of regular expressions ○ | unpack_json - unpacks JSON fields from text ○ | unpack_logfmt - unpacks logfmt fields from text ○ | unpack_syslog - unpacks syslog fields from text
  • 55. LogsQL improvements: data extraction and transformation ● | format … creates a log field according to the given format, which can refer other log fields
  • 56. LogsQL improvements: data extraction and transformation ● | format … creates a log field according to the given format, which can refer other log fields ● | replace … replaces substrings in the given log field
  • 57. LogsQL improvements: data extraction and transformation ● | format … creates a log field according to the given format, which can refer other log fields ● | replace … replaces substrings in the given log field ● | replace_regexp … replaces substrings matching the given regexps
  • 58. LogsQL improvements: data extraction and transformation ● | format … creates a log field according to the given format, which can refer other log fields ● | replace … replaces substrings in the given log field ● | replace_regexp … replaces substrings matching the given regexps ● | math … performs mathematical calculations over log fields
  • 59. LogsQL improvements: additional filtering ● The selected logs can be additionally filtered at any stage via | filter … pipe: _time:5m | stats by (host) count() hits | filter hits:>1000
  • 60. LogsQL improvements: sorting ● By default the returned results aren’t sorted because of performance reasons
  • 61. LogsQL improvements: sorting ● By default the returned results aren’t sorted because of performance reasons ● | sort … pipe can be applied for sorting the results: _time:5m | stats by (host) count() hits | sort by (hits desc) limit 10
  • 62. LogsQL improvements: sorting ● By default the returned results aren’t sorted because of performance reasons ● | sort … pipe can be applied for sorting the results: _time:5m | stats by (host) count() hits | sort by (hits desc) limit 10 ● Sorting can be performed by multiple fields with individual order per each field: _time:5m | stats by (host) count() hits | sort by (hits desc, host)
  • 63. Roadmap ● Accept logs via OpenTelemetry protocol
  • 64. Roadmap ● Accept logs via OpenTelemetry protocol ● VMUI improvements based on HTTP querying API
  • 65. Roadmap ● Accept logs via OpenTelemetry protocol ● VMUI improvements based on HTTP querying API ● Improve Grafana plugin for VictoriaLogs - https://github.com/VictoriaMetrics/victorialogs-datasource
  • 66. Roadmap ● Accept logs via OpenTelemetry protocol ● VMUI improvements based on HTTP querying API ● Improve Grafana plugin for VictoriaLogs - https://github.com/VictoriaMetrics/victorialogs-datasource ● Cluster version ○ Try single-node VictoriaLogs - it can replace 30-node Elasticsearch cluster in production
  • 67. Roadmap ● Accept logs via OpenTelemetry protocol ● VMUI improvements based on HTTP querying API ● Improve Grafana plugin for VictoriaLogs - https://github.com/VictoriaMetrics/victorialogs-datasource ● Cluster version ○ Try single-node VictoriaLogs - it can replace 30-node Elasticsearch cluster in production ● Transparent historical data migration to object storage ○ Try single-node VictoriaLogs with persistent volumes - it compresses 1TB of production logs from Kubernetes to 20GB
  • 68. Roadmap ● Accept logs via OpenTelemetry protocol ● VMUI improvements based on HTTP querying API ● Improve Grafana plugin for VictoriaLogs - https://github.com/VictoriaMetrics/victorialogs-datasource ● Cluster version ○ Try single-node VictoriaLogs - it can replace 30-node Elasticsearch cluster in production ● Transparent historical data migration to object storage ○ Try single-node VictoriaLogs with persistent volumes - it compresses 1TB of production logs from Kubernetes to 20GB ● See https://docs.victoriametrics.com/victorialogs/roadmap/