Open Tracing, to order and understand your mess. - ApiConf 2017

•

0 likes•1,344 views

This about how many api calls your applications were doing 3-4 years ago, and think about how many integration and difference services your requests is crossing before to come back to the final destination. How do you know this step of your pipeline is taking too much time? What is taking 2 seconds to answer? Is it the authentication service? Maybe it's the invoice generation service or the notification platform. Open Tracing is a distributed tracing cross vendor and open source that help you to understand bottleneck and to profile the requests from where they arrive at the final user. In an ecosystem where microservices and as a service concept are growing this can be a real challenge. During this presentation, we will see how it works from a general point of view to land in some real implementation, examples, and demo.

Technology

© 2017 InfluxData. All rights reserved.1
Open Tracing, to order and
understand your mess
@gianarb

I was just waiting for a new
standard!
cit. Troll

© 2017 InfluxData. All rights reserved.3
Typical problems with logs
¨ Which library do I need to use?
¨ Every library has a different format
¨ Every languages exposes a different format

© 2017 InfluxData. All rights reserved.4
Tracing is not something new
¨ There are vendors
¨ Every vendor has their own format

© 2017 InfluxData. All rights reserved.5
Every ten years, history repeats itself but now
¨ Microservices
¨ Polyglot architecture
¨ A different distribution

How do you “tell stories” about
concurrent systems?

© 2017 InfluxData. All rights reserved.7
log log log
log log
log
Parent Span Span Context / Baggage
Child
Child
Child Span
¨ Spans - Basic unit of timing and causality. Can be tagged with
key/value pairs.
¨ Logs - Structured data recorded on a span.
¨ Span Context - serializable format for linking spans across network
boundaries. Carries baggage, such as a request and client IDs.
¨ Tracers - Anything that plugs into the OpenTracing API to record
information.
¨ ZipKin, Jaeger, LightStep, others
¨ Also metrics (Prometheus) and logging

© 2017 InfluxData. All rights reserved.8
OpenTracing
API
application logic
µ-service frameworks
Lambda functions
RPC & control-flow frameworks
existing instrumentation
tracing infrastructure
main()
I N S T A N A
J a e g e r
microservice process

© 2017 InfluxData. All rights reserved.9
1.5 year old!
Tracer implementations: Zipkin, Jaeger, LightStep, SkyWalking, others
All sorts of companies use OpenTracing:

© 2017 InfluxData. All rights reserved.10
Rapidly growing OSS and vendor adoption
JDBIJava Webservlet
Jaxr

© 2017 InfluxData. All rights reserved.11
Some use cases
¨ Microservices
¨ Cache vs. Cache Miss
¨ Parallel vs Sequentially
¨ Queue system

© 2017 InfluxData. All rights reserved.12
Enterprise Edition
Telegraf Kapacitor
InfluxDB
Chronograf
InfluxDB and OpenTracing?!

© 2017 InfluxData. All rights reserved.13
InfluxDB and OpenTracing?!
Implement a comfortable
solution to store and quickly manipulate
all your traces.

Cardinality everywhere!
Monitoring, events, traces… All time series, but they are different!

© 2017 InfluxData. All rights reserved.15
There are different information that you can get from traces. In real time
and not.
Providing a good way to store data without the ability to visualize them
is not the point.
Visualisation

© 2017 InfluxData. All rights reserved.16
Netflix - https://youtu.be/Q4nniyAarbs?t=31m58s
PS I don’t know if it’s using OpenTracing (probably not) but it’s a nice visualisation.

© 2017 InfluxData. All rights reserved.17

Questions?
@gianarb
Site Reliability Engineer at InfluxData
Thanks OpenTracing for the support.

Tracing in the world of micro services has become a standard with people using distributed tracers like Zipkin, Jaeger, Appdash etc. But, with so many different tracers, its confusing to choose one tracer and then painful to replace a tracer. That's where OpenTracing comes in. OT provides a consistent, vendor-neutral API to allow users to choose whatever distributed tracer they need and can change the tracer with just an O(1) operation.

Overview and Opentracing in theory by Gianluca Arbezzano

Gianluca Arbezzano

Distributed Tracing with OpenTracing, ZipKin and Kubernetes

Container Solutions

The document discusses distributed tracing with Zipkin and Kubernetes. It defines microservices and their benefits and costs. Distributed tracing allows tracing requests across multiple services and systems. This helps identify bottlenecks and failures. The document discusses OpenTracing and Zipkin libraries for instrumenting services. It also covers Zipkin's architecture, performance considerations like sampling, and how Linkerd can help with legacy systems. Finally, it mentions answering questions about slow parts, optimizations, and failures using distributed tracing before cautioning about data volumes and interpretation.

Tracing 2000+ polyglot microservices at Uber with Jaeger and OpenTracing

Yuri Shkuro

Distributed tracing - get a grasp on your production

nklmish

Slides from my presentation on distributed tracing, explaining what is latency and why it matters. We took a look at openzipkin and its concepts like how the core annotations works, what are tags/logs, etc. Followed by a demo application created using golang and java (spring boot , spring cloud sleuth zipkin) . You can find source code here https://github.com/nklmish/go-distributed-tracing-demo https://github.com/nklmish/java-distributed-tracing-demo

Jaeger and OpenTracing Cloud Native Computing (CNCF) meetup Zurich

⛑ Pavol Loffay

Monitoring Pull vs Push, InfluxDB and Prometheus

Gianluca Arbezzano

This document discusses the differences between pull-based and push-based monitoring approaches. It notes that both approaches require instrumenting applications, and that sometimes one is easier than the other depending on the situation. Specifically, pull may be easier for monitoring millions of IoT devices globally, while push may be difficult for monitoring metrics from MySQL. The document also thanks Prometheus, the CNCF, and the OpenMetrics initiative for standardizing the pull method and metrics format. It highlights how Kapacitor now integrates Prometheus' service discovery and scraping code to enable pull-based monitoring using InfluxDB.

Distributed Tracing

Kevin Ingelman

This document discusses distributed tracing at Uber scale. It explains why companies implement distributed tracing, including for distributed transaction monitoring, performance optimization, root cause analysis, and service dependency analysis. It also discusses challenges with instrumentation and context propagation across services. Finally, it outlines strategies Uber used to encourage adoption of distributed tracing across many teams, such as appealing to developers' interests and measuring adoption through trace quality scores.

InfluxCloudi craft container orchestrator

Gianluca Arbezzano

Monitor your application and sleep

Gianluca Arbezzano

Our application speaks, time series are one of their languages. During this talk I will share how to use the open source Tick Stack to spin up a modern monitoring system for your application and your infrastructure. DevOps, cloud computing and containers changed how we are writing and running our applications. This talk shows what InfluxData and the community is building to have a modern and flexible monitoring toolkit.

Intro to open source observability with grafana, prometheus, loki, and tempo(...

LibbySchulze

This document provides an introduction to open source observability tools including Grafana, Prometheus, Loki, and Tempo. It summarizes each tool and how they work together. Prometheus is introduced as a time series database that collects metrics. Loki is described as a log aggregation system that handles logs at scale without high costs. Tempo is explained as a tracing system that allows tracing from logs, metrics, and between services. The document emphasizes that these tools can be run together to gain observability across an entire system from logs to metrics to traces.

Juraci Paixão Kröhling - All you need to know about OpenTelemetry

Juliano Costa

OpenTelemetry is one of the newest projects in the realm of Observability at the CNCF and is already the second most active project there. In this session, Juraci Paixão Kröhling will talk about the different subprojects and how to get started using them. Even if you heard about OpenTelemetry before, you'll leave this session with a better understanding of what this is all about, the several faces of OpenTelemetry, and what you can do to make your projects more observable.

Distributed Tracing

soasme

Distributed tracing allows monitoring and troubleshooting of transactions that span multiple services. It involves assigning unique IDs to requests as they pass through services and collecting timing data. Popular distributed tracing systems like Dapper, Zipkin, and OpenTracing use unique trace IDs and span IDs to correlate data from multiple services and timespans. They allow identifying slow or faulty services and transactions.

Opentracing 101

HungWei Chiu

In this slide, we go through the Google Dapper, OpenTracing, Jaeger to OpenTelemetry. By reading and studying the history of Dapper, we could lean the experience and design theory of a large-scale distributed tracing system and then know how it affects other solutions, like OpenTracing and Jaeger. We also discuss the difference between the OpenTracing and Jaeger and also demonstrate how Jaeger works and looks like. After, we talked about the future of OpenTracing, the new organization called OpenTelemetry, what's its goal and how to do that.

[WSO2Con EU 2018] Tooling for Observability

WSO2

Opentracing jaeger

Oracle Korea

Real time analytics with Netty, Storm, Kafka

Trieu Nguyen

This document discusses a real-time analytics architecture using Netty, Apache Kafka, and Storm. It includes an overview of the system architecture with Netty handling HTTP logging and producing to Apache Kafka for stream data storage. Storm analytics clusters then consume from Kafka topics to perform analytics functions like tokenization, parsing, aggregation, and saving results to Redis and a data warehouse.

Introduction to Distributed Tracing

petabridge

This document discusses distributed tracing and how it can help solve problems caused by microservices. It covers what distributed tracing is, how it works, popular implementations like OpenTracing and Zipkin, and best practices for using distributed tracing. OpenTracing is introduced as a vendor-neutral standard that helps library developers implement tracing and defines common formats for propagating traces between services. Code examples are provided for collecting trace data using OpenTracing and Zipkin.

Performance monitoring and call tracing in microservice environments

Martin Gutenbrunner

The document discusses challenges with monitoring microservice environments, including tracing calls between services. It describes how custom implementations can be complex due to different technologies. Commercial solutions like Dynatrace Ruxit provide unified monitoring with call tracing across technologies with minimal setup. They automatically detect issues without thresholds and include client-side monitoring.

Netflow Analysis using Elastic Stack - 조인중

NAVER D2

NetFlow Analysis with Elastic Stack summarizes using NetFlow data with the Elastic Stack, including Logstash, Elasticsearch, and Kibana. It discusses what NetFlow is, the typical NetFlow architecture with flow exporters, collectors, and analysis applications. It then covers how to use Logstash for NetFlow collection, Elasticsearch for storage and analysis, and Kibana for visualization. The presentation provides examples of Logstash configuration for NetFlow input, filtering, and output, as well as templates for indexing NetFlow data in Elasticsearch.

stackconf 2021 | Continuous Security – integrating security into your pipelines

NETWAYS

In the world of continuous delivery and cloud native, the boundaries between what is our application and what constitutes infrastructure is becoming increasing blurred. Our workloads, the containers they ship in, and our platform configuration is now often developed and deployed by the same teams, and development velocity is the key metric to success. This presents us with a challenge which the previous models of security as a final external gatekeeper step cannot keep up with. To ensure our apps and platforms are secure, we need to integrate security at all stages of our pipelines and ensure that our developers and engineering teams have tools and data with enable them to make decisions about security on an ongoing basis. In this session I will talk through the problem space, look at the kinds of security issues we need to consider, and look at where the integration points are to build in security as part of our CI/CD process.

Encode

Vanessa Lošić

The document describes the Application Research Group (ARG) and their work exploring applications built on Filecoin. ARG has created Estuary, an open source project that makes it easy to store data in the Filecoin network. Estuary provides a hosted node, website, and documentation. It handles storing user data through Filecoin storage deals with high replication for reliability. ARG shares details of Estuary publicly to help others learn and provide transparency into its performance and use of Filecoin miners. They aim to help others run their own Estuary nodes and improve the Filecoin storage experience.

Distributed tracing using open tracing & jaeger 2

Chandresh Pancholi

Encode Club workshop slides

Vanessa Lošić

This document provides an agenda and overview of SubQuery, an open source project that allows developers to index, transform, and query Substrate chain data. It discusses problems with current query speeds and parachain diversity that SubQuery aims to address. The basics of SubQuery are explained, including the components needed and key concepts like the manifest file, GraphQL schema, and mapping files that define how chain data is extracted, transformed, and persisted. Hands-on exercises are proposed to build a sample project. Production infrastructure for hosting SubQuery projects is also introduced.

Opa gatekeeper

Rita Zhang

This document discusses OPA Gatekeeper, which is an admission webhook that helps enforce policies and strengthen governance in Kubernetes clusters. It provides customizable admission controls via configuration instead of code. Gatekeeper uses the Open Policy Agent (OPA) to evaluate policies written in Rego against objects in the Kubernetes API. It started as kube-mgmt and has evolved through several versions. Gatekeeper allows defining policies as templates with parameters and matching rules, and instances of those policies that are enforced as custom resources. It provides capabilities like auditing, CI/CD integration, and replicating cluster state for offline policy checking. The document demonstrates example policies and invites the reader to get involved in the open source project.

Fluent Bit

Eduardo Silva Pereira

This document introduces Fluent Bit, an open source data collector for IoT and embedded devices. It provides a lightweight and customizable solution for collecting and transporting data from various sources to services like Fluentd. Fluent Bit uses a plugin architecture and supports inputs from devices, sensors, and operating systems as well as outputs to services like Treasure Data. It is designed for performance and low resource usage on embedded systems.

OSMC 2018 | Distributed Tracing FAQ by Gianluca Arbezzano

NETWAYS

Microservices, containers and more in general distributed systems have opened a different point of view on our system and applications. We need to understand how a single event or requests cross our app jumping over networks, containers, virtual machines and sometime clod provider. There is a specific practice called distributed tracing to increase observability of systems like that. After this talk, you will have a solid idea around what tracing means, how you can instrument your applications and you will be ready to trace your application across many languages using open source technologies like OpenTracing, OpenCensus, Zipkin, Jaeger, InfluxDB. You will ask yourself how you survived until today!

Monitor Kubernetes in Rancher using InfluxData

InfluxData

The document discusses monitoring Kubernetes in Rancher using InfluxData. It provides an introduction to Rancher and how it can deploy and manage Kubernetes clusters. It then discusses the time series data problem and how InfluxData solutions like Telegraf, InfluxDB, Chronograf and Kapacitor can collect, store and analyze the high volume time series data generated by containers. It highlights key features of Rancher and InfluxData and demonstrates how their tools can be used together to monitor Kubernetes workloads.

What's hot

Distributed tracing with OpenTracing and Jaeger @ getstream.io

Max Klyga

Distributed Tracing at UBER Scale: Creating a treasure map for your monitori...

Yuri Shkuro

InfluxCloudi craft container orchestrator

Gianluca Arbezzano

Monitor your application and sleep

Gianluca Arbezzano

Intro to open source observability with grafana, prometheus, loki, and tempo(...

LibbySchulze

Juraci Paixão Kröhling - All you need to know about OpenTelemetry

Juliano Costa

Distributed Tracing

soasme

Opentracing 101

HungWei Chiu

[WSO2Con EU 2018] Tooling for Observability

WSO2

Opentracing jaeger

Oracle Korea

Real time analytics with Netty, Storm, Kafka

Trieu Nguyen

Introduction to Distributed Tracing

petabridge

Performance monitoring and call tracing in microservice environments

Martin Gutenbrunner

Netflow Analysis using Elastic Stack - 조인중

NAVER D2

stackconf 2021 | Continuous Security – integrating security into your pipelines

NETWAYS

Encode

Vanessa Lošić

Distributed tracing using open tracing & jaeger 2

Chandresh Pancholi

Encode Club workshop slides

Vanessa Lošić

Opa gatekeeper

Rita Zhang

Fluent Bit

Eduardo Silva Pereira

What's hot (20)

Distributed tracing with OpenTracing and Jaeger @ getstream.io

Distributed Tracing at UBER Scale: Creating a treasure map for your monitori...

InfluxCloudi craft container orchestrator

Monitor your application and sleep

Intro to open source observability with grafana, prometheus, loki, and tempo(...

Juraci Paixão Kröhling - All you need to know about OpenTelemetry

Distributed Tracing

Opentracing 101

[WSO2Con EU 2018] Tooling for Observability

Opentracing jaeger

Real time analytics with Netty, Storm, Kafka

Introduction to Distributed Tracing

Performance monitoring and call tracing in microservice environments

Netflow Analysis using Elastic Stack - 조인중

stackconf 2021 | Continuous Security – integrating security into your pipelines

Encode

Distributed tracing using open tracing & jaeger 2

Encode Club workshop slides

Opa gatekeeper

Fluent Bit

Similar to Open Tracing, to order and understand your mess. - ApiConf 2017

OSMC 2018 | Distributed Tracing FAQ by Gianluca Arbezzano

NETWAYS

Monitor Kubernetes in Rancher using InfluxData

InfluxData

How to Use the TICK Stack, CoreOS, & Docker to Make Your SaaS Offering Better

Deborah Schalm

Other than InfluxDB and the open source Tick Stack, InfluxData provides a SaaS version of the TICK Stack, called InfluxCloud, that supports clustering, authentication and other enterprise features not found in the open source TICK Stack. InfluxCloud is designed to be easy to use and reliable for our customers, as they rely on this service to store time series data gathered from sensors, servers, and other devices that is the foundation for their own SLAs for their products and services. This means InfluxCloud requires an even higher standard of service in order to help them maintain their service level commitments.

How to Use the TICK Stack, CoreOS, & Docker to Make Your SaaS Offering Better

DevOps.com

NkSIP: The Erlang SIP application server

Carlos González Florido

NkSIP is an open-source, scalable Erlang SIP application server. It provides a distributed platform for building highly available communications applications. NkSIP aims to simplify SIP development by hiding complexity while providing full control. It uses Erlang for concurrency and fault tolerance and allows applications to be developed in multiple languages. NkSIP will soon support distribution based on the Dynamo model and integration with other protocols beyond SIP.

Introduction to Filecoin

Vanessa Lošić

This document provides an introduction to Filecoin, including: 1) Core concepts of Filecoin such as using IPFS for data retrieval and Filecoin for data persistence and verifiability on a decentralized storage network. 2) Examples of how storage helpers can simplify storing and retrieving data on Filecoin by handling dealmaking and verification. 3) An overview of the different layers that make up a Web3-enabled architecture using Filecoin and IPFS for decentralized storage.

2014 carlos gzlez florido nksip the erlang sip application server

VOIP2DAY

NkSIP is an open-source, scalable Erlang SIP application server that allows developers to easily create distributed SIP applications. It provides a full-featured SIP stack and plugin system to handle all SIP details, allowing applications to focus on business logic. The long term vision is for NkSIP to become a general distributed application server called NkCORE, serving as a platform for network functions and services across various protocols using Erlang on control nodes and Docker on workers.

Log everything!

ICANS GmbH

A True Story About Database Orchestration

InfluxData

Apache kafka

sureshraj43

IoT Architectural Overview - 3 use case studies from InfluxData

InfluxData

This SlideShare reviews how an IoT Data platform fits in with any IoT Architecture to manage the data requirements of every IoT implementation. It is based on the learnings from existing IoT practitioners that have adopted an IoT Data platform using InfluxData. These clients have a range of solutions–from home automation (thermostat monitoring & management), to infrastructure management (solar panel monitoring and control) to manufacturing (equipment monitoring & control) as well as environmental management (green wall monitoring & control). These learnings will help IoT adopters avoid the common pitfalls current clients faced on their journey to developing their IoT solution.

Introduction to IPFS & Filecoin - longer version

TinaBregovi

The document provides an overview of an introduction to IPFS and Filecoin. It discusses key concepts like IPFS being a peer-to-peer protocol for a decentralized web and Filecoin being an Airbnb-like network for data storage. It outlines the agenda which includes IPFS and Filecoin concepts, use cases, social good initiatives, opportunities, and resources. It then dives deeper into explaining technical aspects of IPFS, Filecoin, and related development tools.

Webinar: Multi-Cloud Monitoring

InfluxData

Software Stacks to enable SDN and NFV

Yoshihiro Nakajima

Yoshihiro Nakajima presented on software stacks that enable software-defined networking (SDN) and network functions virtualization (NFV). He discussed trends in SDN and NFV, introduced the Lagopus SDN software switch project, and described how Data Plane Development Kit (DPDK) helps optimize packet processing performance. The goal of his talk was to provide an NFV/SDN-aware software stack capable of 100Gbps switching through high-performance packet processing.

Red Hat Storage: Emerging Use Cases

Red_Hat_Storage

Introduction to IPFS & Filecoin

TinaBregovi

Intro elasticsearch taswarbhatti

Taswar Bhatti

This document provides an overview of Elastic Stack including ElasticSearch, Logstash, Kibana, and Beats. It discusses how Gemalto was using a monolithic solution to store logs from distributed systems and microservices, and wanted to implement a centralized scalable logging system. It describes various designs considered using Elastic Stack components like Logstash, Elasticsearch, and Filebeat to ingest, parse, store and visualize logs. Future plans discussed include using machine learning and Kafka.

Hail hydrate! from stream to lake using open source

Timothy Spann

(VIRTUAL) Hail Hydrate! From Stream to Lake Using Open Source - Timothy J Spann, StreamNative https://osselc21.sched.com/event/lAPi?iframe=no A cloud data lake that is empty is not useful to anyone. How can you quickly, scalably and reliably fill your cloud data lake with diverse sources of data you already have and new ones you never imagined you needed. Utilizing open source tools from Apache, the FLiP stack enables any data engineer, programmer or analyst to build reusable modules with low or no code. FLiP utilizes Apache NiFi, Apache Pulsar, Apache Flink and MiNiFi agents to load CDC, Logs, REST, XML, Images, PDFs, Documents, Text, semistructured data, unstructured data, structured data and a hundred data sources you could never dream of streaming before. I will teach you how to fish in the deep end of the lake and return a data engineering hero. Let's hope everyone is ready to go from 0 to Petabyte hero. https://osselc21.sched.com/event/lAPi/virtual-hail-hydrate-from-stream-to-lake-using-open-source-timothy-j-spann-streamnative

An Optics Life

Thomas Weible

Building a Secure and Resilient Foundation for Banking at Intesa Sanpaolo wit...

Docker, Inc.

Intesa Sanpaolo is one of the first banking groups in the Euro zone, with over 12 million customers and 4,600 branches in Italy. With a lot of traditional monolithic applications that are difficult to maintain and evolve, Intesa turned to Docker to help them both modernize the applications and improve their portability so that they could consider a multi-site architecture across multiple data centers. Using Docker Enterprise Edition (EE), Intesa took the first step to “break the monolith” by containerizing their infrastructure, self-described as an “Infrastructure-as-code” pattern, and now use Docker EE to orchestrate the applications across sites. In this talk Diego Braga, Infrastructure System Specialist at Intesa, and Lorenzo Fontana, DevOps Engineer at Kiratech will share how they implemented Docker EE along with software-defined networking and storage solutions to validate Intesa’s architectural model and to build a geographical distributed multi-data center cluster, all while saving infrastructure costs and remaining compliant with regulations. They will highlight their CI/CD process using Docker and Jenkins, how the developer and ops team are now working together to implement a DevOps methodology and Intesa’s ROI in using Docker EE. They will also share Intesa’s future plans, including creating mixed Linux/Windows clusters that use the same overlay network and on-prem/public cloud clusters opportunities.

Similar to Open Tracing, to order and understand your mess. - ApiConf 2017 (20)

OSMC 2018 | Distributed Tracing FAQ by Gianluca Arbezzano

Monitor Kubernetes in Rancher using InfluxData

How to Use the TICK Stack, CoreOS, & Docker to Make Your SaaS Offering Better

NkSIP: The Erlang SIP application server

Introduction to Filecoin

2014 carlos gzlez florido nksip the erlang sip application server

Log everything!

A True Story About Database Orchestration

Apache kafka

IoT Architectural Overview - 3 use case studies from InfluxData

Introduction to IPFS & Filecoin - longer version

Webinar: Multi-Cloud Monitoring

Software Stacks to enable SDN and NFV

Red Hat Storage: Emerging Use Cases

Introduction to IPFS & Filecoin

Intro elasticsearch taswarbhatti

Hail hydrate! from stream to lake using open source

An Optics Life

Building a Secure and Resilient Foundation for Banking at Intesa Sanpaolo wit...

More from Gianluca Arbezzano

Kubernetes debug like a pro

Gianluca Arbezzano

Go fit perfectly inside containers, you can ship apps as tiny images on k8s, distributing them across the globe. Gianluca will show how InfluxData debugs containers running on Kubernetes to allow sysadmins and developers to troubleshoot and replicate issues using core dump, debuggers, and logs. Go applications are perfect to be run inside a container. You can build a single binary, a tiny Docker image and you can ship them on your Kubernetes cluster. A successful production environment requires stability and simplicity, it needs to be easy to troubleshoot and operators need to be able to get all the information developers will need to fix a bug. During this talk, Gianluca will share what influxData is doing to allow developers and system administrator to work together, understanding problems running live at scale on Kubernetes and how to escalate them down to Software Engineer using logs, delve, gdb, core dumps, and traces to replicate and fix issues.

OSDC 2018 - Distributed monitoring

Gianluca Arbezzano

Modern software development is increasingly taking a “microservice” approach that has resulted in an explosion of complexity at the network level. We have more applications running distributed across different datacenters. Distributed tracing, events, and metrics are essential for observing and understanding modern microservice architectures. This talk is a deep dive on how to monitor your distributed system. You will get tools, methodologies, and experiences that will help you to realize what your applications expose and how to get value out from all these information. Gianluca Arbezzano, SRE at InfluxData will share how to monitor a distributed system, how to switch from a more traditional monitoring approach to observability. Stay focused on the server’s role and not on the hostname because it’s not really important anymore, our servers or containers are fast moving part and it’s easy to detach it from the right in case of trouble than call the server by name as a cute puppet. How to design a SLO for your core services and now to iterate on them. Instrument your services with tracing using tools like Zipkin or Jaeger to measure latency between in your network.

Value of your metrics: goodbye monitoring, welcome observability

Gianluca Arbezzano

Orbiter and how to extend Docker Swarm

Gianluca Arbezzano

Security Tips to run Docker in Production

Gianluca Arbezzano

When you are designing a production environment security is essential. All the Docker ecosystem but in particular Docker Swarm allows us to ship our containers out of our laptop, how can we make this process safe? During my talk, I will share tips around production environment, immutability and how troubleshooting common attack as code injection with Docker. Static analysis of our images, content trust with Notary to make our journey secure. How can we setup a cluster on the main cloud providers with VPN and node labeling to expose only a portion of our cluster? I will also show what Docker provides (Content Trust, Static Analysis) but also open source alternatives as Notary, centos/clair and Cilium. In the end of this talk, we had a better idea around how manage Docker in production.

Jenkins in the real world - DevOpsCon 2017

Gianluca Arbezzano

There are a lot of Continuous Integration services but Jenkins is still one of the most used in most programming languages. In this talk I will share the CurrencyFair experience, how our IT Team made of 40 engineers manage CurrencyFair delivery with GitHub, Jenkins, Hubot and Slack on different environments. Artifact to guarantee the stability of your codebase, pipeline and some Jenkins’s plugins in order to create the most comfortable delivery flow for your projects.

Tick Stack - Listen your infrastructure and please sleep

Gianluca Arbezzano

Docker Novosibirsk Meetup #3 - Docker in Production

Gianluca Arbezzano

Docker 1.13 introduces improvements that make Docker better for production environments. The Docker CLI is redesigned with top-level commands for different resources. Docker system commands are added for system management tasks. Compose file version 3 is now compatible with Swarm mode for defining multi-container applications. Secrets can now be used in Swarm mode to securely provide sensitive information to containers. Other features include squashing layers to reduce image sizes and viewing event streams. Examples and links are provided to demonstrate these Docker 1.13 features.

DockerDublin Meetup - News about Docker 1.13

Gianluca Arbezzano

Docker 1.13 includes several updates that improve the management of production environments, including a redesigned command line interface (CLI) and new system, plugin, and secret commands. The top-level CLI commands are now organized around common concepts like container, image, network and service. Docker 1.13 also introduces support for secrets in swarm mode and a --squash option to reduce image layers when building. The release makes experimental features fully supported.

Docker 1.12 and SwarmKit

Gianluca Arbezzano

Time Series Database and Tick Stack

Gianluca Arbezzano

This document discusses tools for working with time series data, including InfluxDB for storing time series data, Telegraf for collecting metrics, and Kapacitor for processing and alerting on metrics. It provides an overview of how to install and use InfluxDB, describes its HTTP and UDP APIs, query language, and advantages over alternatives. Continuous queries, input and output plugins for Telegraf, and alerting capabilities of Kapacitor are also summarized. The document encourages representing log lines and other time-indexed data as compact time series for scalability.

$Queue System and Zend\Queue implementation$ $Queue System and Zend\Queue implementation$

Queue System and Zend\Queue implementation

Gianluca Arbezzano

This document discusses a queue system for processing asynchronous tasks. It describes using a queue to send messages to be processed by worker consumers. Key aspects covered include queue adapters that can store messages in databases or message brokers, producing and consuming messages, and using the await method for workers to wait for new messages to process. Examples are provided of how to set up a queue, send and receive messages, and build asynchronous worker processes using the queue.

ZfDayIt 2014 - There is a module for everything

Gianluca Arbezzano

Vagrant - PugMI

Gianluca Arbezzano

This document discusses Vagrant, a tool that allows users to set up and configure development environments using configuration files. It covers topics like adding and removing Vagrant boxes, provisioning boxes using tools like Chef and Puppet, configuring networks, folders, and providers in the Vagrantfile, and creating custom Vagrant boxes using Bento. The overall message is that Vagrant streamlines setting up virtual development environments for projects.

Silex, iniziamo

Gianluca Arbezzano

More from Gianluca Arbezzano (15)

Kubernetes debug like a pro

OSDC 2018 - Distributed monitoring

Value of your metrics: goodbye monitoring, welcome observability

Orbiter and how to extend Docker Swarm

Security Tips to run Docker in Production

Jenkins in the real world - DevOpsCon 2017

Tick Stack - Listen your infrastructure and please sleep

Docker Novosibirsk Meetup #3 - Docker in Production

DockerDublin Meetup - News about Docker 1.13

Docker 1.12 and SwarmKit

Time Series Database and Tick Stack

$Queue System and Zend\Queue implementation$ $Queue System and Zend\Queue implementation$

Queue System and Zend\Queue implementation

ZfDayIt 2014 - There is a module for everything

Vagrant - PugMI

Silex, iniziamo

Recently uploaded

Uni Systems Copilot event_05062024_C.Vlachos.pdf

Uni Systems S.M.S.A.

Mind map of terminologies used in context of Generative AI

Kumud Singh

20240607 QFM018 Elixir Reading List May 2024

Matthew Sinclair

National Security Agency - NSA mobile device best practices

Quotidiano Piemontese

Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI

Vladimir Iglovikov, Ph.D.

Presented by Vladimir Iglovikov: - https://www.linkedin.com/in/iglovikov/ - https://x.com/viglovikov - https://www.instagram.com/ternaus/ This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation. Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners. This case study covers various aspects, including: People: The contributors and community that have supported Albumentations. Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions. Challenges: The hurdles in monetizing open-source projects and measuring user engagement. Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration. Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community. Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations. Mental Health: Maintaining balance and not feeling pressured by user demands. Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth. Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects. Explore more about Albumentations and join the community at: GitHub: https://github.com/albumentations-team/albumentations Website: https://albumentations.ai/ LinkedIn: https://www.linkedin.com/company/100504475 Twitter: https://x.com/albumentations

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Aggregage

“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...

Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/ Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit. In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing. van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024

Neo4j

Pushing the limits of ePRTC: 100ns holdover for 100 days

Adtran

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

RESUME BUILDER APPLICATION Project for students

KAMESHS29

Cosa hanno in comune un mattoncino Lego e la backdoor XZ?

Speck&Tech

ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune. Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile. BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).

Climate Impact of Software Testing at Nordic Testing Days

Kari Kakkonen

My slides at Nordic Testing Days 6.6.2024 Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Essentials of Automations: The Art of Triggers and Actions in FME

Safe Software

In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation. We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios. Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

Neo4j

TrustArc Webinar - 2024 Global Privacy Survey

TrustArc

How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024? In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores. See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe. This webinar will review: - The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey - The top challenges for privacy leaders, practitioners, and organizations in 2024 - Key themes to consider in developing and maintaining your privacy program

Full-RAG: A modern architecture for hyper-personalization

Zilliz

Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.

Data structures and Algorithms in Python.pdf

TIPNGVN2

Recently uploaded (20)

Uni Systems Copilot event_05062024_C.Vlachos.pdf

Mind map of terminologies used in context of Generative AI

20240607 QFM018 Elixir Reading List May 2024

National Security Agency - NSA mobile device best practices

Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI

Generative AI Deep Dive: Advancing from Proof of Concept to Production

“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...

Securing your Kubernetes cluster_ a step-by-step guide to success !

GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024

Pushing the limits of ePRTC: 100ns holdover for 100 days

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

RESUME BUILDER APPLICATION Project for students

Cosa hanno in comune un mattoncino Lego e la backdoor XZ?

Climate Impact of Software Testing at Nordic Testing Days

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Essentials of Automations: The Art of Triggers and Actions in FME

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

TrustArc Webinar - 2024 Global Privacy Survey

Full-RAG: A modern architecture for hyper-personalization

Data structures and Algorithms in Python.pdf

Open Tracing, to order and understand your mess. - ApiConf 2017

2. I was just waiting for a new standard! cit. Troll

6. How do you “tell stories” about concurrent systems?

7. © 2017 InfluxData. All rights reserved.7 log log log log log log Parent Span Span Context / Baggage Child Child Child Span ¨ Spans - Basic unit of timing and causality. Can be tagged with key/value pairs. ¨ Logs - Structured data recorded on a span. ¨ Span Context - serializable format for linking spans across network boundaries. Carries baggage, such as a request and client IDs. ¨ Tracers - Anything that plugs into the OpenTracing API to record information. ¨ ZipKin, Jaeger, LightStep, others ¨ Also metrics (Prometheus) and logging

8. © 2017 InfluxData. All rights reserved.8 OpenTracing API application logic µ-service frameworks Lambda functions RPC & control-flow frameworks existing instrumentation tracing infrastructure main() I N S T A N A J a e g e r microservice process

14. Cardinality everywhere! Monitoring, events, traces… All time series, but they are different!

15. © 2017 InfluxData. All rights reserved.15 There are different information that you can get from traces. In real time and not. Providing a good way to store data without the ability to visualize them is not the point. Visualisation

18. Questions? @gianarb Site Reliability Engineer at InfluxData Thanks OpenTracing for the support.

Open Tracing, to order and understand your mess. - ApiConf 2017

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Open Tracing, to order and understand your mess. - ApiConf 2017

Similar to Open Tracing, to order and understand your mess. - ApiConf 2017 (20)

More from Gianluca Arbezzano

More from Gianluca Arbezzano (15)

Recently uploaded

Recently uploaded (20)

Open Tracing, to order and understand your mess. - ApiConf 2017