ActiveSupport::Notifications inside out
•
0 likes•319 views
This document discusses Rails instrumentation using ActiveSupport::Notifications. It introduces notifications as a way to decouple runtime data collection from functional modules using a publish/subscribe pattern. Notifications can be used to instrument various Rails events and actions, and the data can power features like logging, profiling, performance tracking, and triggering background jobs.
Report
Share
Report
Share
Download to read offline
Recommended
How Sysbee Manages Infrastructures and Provides Advanced Monitoring by Using ...
Discover how Sysbee helps organizations bring DevOps culture to small and medium enterprises. Their team helps their customers by improving stability, security, scalability — by providing cost-effective IT infrastructure. Learn how monitoring everything can improve your processes and simplify debugging!
Sysbee’s introspection on monitoring tools over the years
How TSDB’s, and specifically InfluxDB, fits into improving observability
Their approach to using the TICK Stack to improve the web hosting industry
"Smooth Operator" [Bay Area NewSQL meetup]
This slide was delivered at the Bay Area NewSQL meetup in California on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
Yipit - AWS Start-Up Customer
Yipit is the largest daily deal aggregator in the US with 20 employees and 40+ servers hosted on AWS. It was built using Python, Django, Nginx, Gunicorn, Celery, MySQL, MongoDB, Redis and DynamoDB. AWS services currently used include S3, EC2, CloudWatch, CloudFront, RDS, ElastiCache, SQS, IAM, Route53 and DynamoDB. AWS provides benefits like speed of iteration, infrastructure as code, leveraging AWS expertise, and automation which helps Yipit as a startup. Yipit monitors its infrastructure using New Relic, Django Sentry, Papertrail, AppFirst and Stathat.
Micheal Pershyn "Coljure 4 Big Data"
This document discusses how Clojure can be used for big data applications. It introduces Clojure and several Clojure libraries for distributed stream processing (Apache Storm) and batch processing (Apache Trident, Incanter, Cascalog). Storm is described as a distributed real-time computation system. Trident provides high-level abstractions for exactly-once processing on Storm. Incanter and Cascalog allow interactive analysis and querying of big data using a Clojure API. The document argues Clojure is well-suited for big data due to its focus on immutability, functions, and map-reduce approaches.
AWS Finland meetup 2018 August
The document provides an agenda for an AWS Finland Meetup in August 2018 sponsored by Sanoma. The agenda includes doors opening at 17:30, the schedule starting at 18:00, and speakers presenting on a community update, new and noteworthy AWS services from Rolf Koski, and experiences using Fargate from Heidi Lahti. Additionally, Moritz Siuts will present on Kubernetes clusters for ride sharing at MOIA.
Grafana 7.0
Grafana 7.0 introduces new features including a tracing data viewer that allows users to view and correlate metrics, logs, and traces across data sources. It also includes new data transformations that allow users to transform data before it is queried. Additionally, Grafana 7.0 features a new plugin architecture that splits core functionality into packages and supports official backend plugins running as a separate process.
Introducing TiDB Operator [Cologne, Germany]
This slide was delivered at the Kubernetes/Docker meetup in Cologne, Germany, hosted by Giant Swarms on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
Dynomite - PerconaLive 2017
Netflix created and open sourced Dynomite project to provide reusable distributed database infrastructure that turns single server data stores into scalable, distributed databases. Dynomite supports pluggable protocols and pluggable storage engines, which allows us to add sharding and replication to a variety of non-distributed data stores. The entire database infrastructure can be reused across a variety of workloads from in-memory to on-disk, and across APIs from key/value to document databases. Dynomite allows application developers to choose the API that best fits their requirements, while DevOps can select the best operation database based on the workload. Dynomite is used by Netflix to handle millions of OPS in production leveraging Redis and RocksDB. In this talk, we are going to show how we achieved high availability by being able to terminate any Dynomite node without client side downtime, best practices and the challenges in deploying Dynomite in production.
Recommended
How Sysbee Manages Infrastructures and Provides Advanced Monitoring by Using ...
Discover how Sysbee helps organizations bring DevOps culture to small and medium enterprises. Their team helps their customers by improving stability, security, scalability — by providing cost-effective IT infrastructure. Learn how monitoring everything can improve your processes and simplify debugging!
Sysbee’s introspection on monitoring tools over the years
How TSDB’s, and specifically InfluxDB, fits into improving observability
Their approach to using the TICK Stack to improve the web hosting industry
"Smooth Operator" [Bay Area NewSQL meetup]
This slide was delivered at the Bay Area NewSQL meetup in California on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
Yipit - AWS Start-Up Customer
Yipit is the largest daily deal aggregator in the US with 20 employees and 40+ servers hosted on AWS. It was built using Python, Django, Nginx, Gunicorn, Celery, MySQL, MongoDB, Redis and DynamoDB. AWS services currently used include S3, EC2, CloudWatch, CloudFront, RDS, ElastiCache, SQS, IAM, Route53 and DynamoDB. AWS provides benefits like speed of iteration, infrastructure as code, leveraging AWS expertise, and automation which helps Yipit as a startup. Yipit monitors its infrastructure using New Relic, Django Sentry, Papertrail, AppFirst and Stathat.
Micheal Pershyn "Coljure 4 Big Data"
This document discusses how Clojure can be used for big data applications. It introduces Clojure and several Clojure libraries for distributed stream processing (Apache Storm) and batch processing (Apache Trident, Incanter, Cascalog). Storm is described as a distributed real-time computation system. Trident provides high-level abstractions for exactly-once processing on Storm. Incanter and Cascalog allow interactive analysis and querying of big data using a Clojure API. The document argues Clojure is well-suited for big data due to its focus on immutability, functions, and map-reduce approaches.
AWS Finland meetup 2018 August
The document provides an agenda for an AWS Finland Meetup in August 2018 sponsored by Sanoma. The agenda includes doors opening at 17:30, the schedule starting at 18:00, and speakers presenting on a community update, new and noteworthy AWS services from Rolf Koski, and experiences using Fargate from Heidi Lahti. Additionally, Moritz Siuts will present on Kubernetes clusters for ride sharing at MOIA.
Grafana 7.0
Grafana 7.0 introduces new features including a tracing data viewer that allows users to view and correlate metrics, logs, and traces across data sources. It also includes new data transformations that allow users to transform data before it is queried. Additionally, Grafana 7.0 features a new plugin architecture that splits core functionality into packages and supports official backend plugins running as a separate process.
Introducing TiDB Operator [Cologne, Germany]
This slide was delivered at the Kubernetes/Docker meetup in Cologne, Germany, hosted by Giant Swarms on how TiDB, an open source NewSQL distributed database, is deployed and managed on any Kubernetes-enabled cloud environment by applying the Operator pattern.
Dynomite - PerconaLive 2017
Netflix created and open sourced Dynomite project to provide reusable distributed database infrastructure that turns single server data stores into scalable, distributed databases. Dynomite supports pluggable protocols and pluggable storage engines, which allows us to add sharding and replication to a variety of non-distributed data stores. The entire database infrastructure can be reused across a variety of workloads from in-memory to on-disk, and across APIs from key/value to document databases. Dynomite allows application developers to choose the API that best fits their requirements, while DevOps can select the best operation database based on the workload. Dynomite is used by Netflix to handle millions of OPS in production leveraging Redis and RocksDB. In this talk, we are going to show how we achieved high availability by being able to terminate any Dynomite node without client side downtime, best practices and the challenges in deploying Dynomite in production.
Scylla Summit 2022: Overcoming the Performance Cost of Streaming Transactions
For a long time distributed transactions have been known to make systems slow and unavailable. However, recent academic and industry advancements such as RAMP, Occult, Calvin and TAPIR have changed the landscape and made transactions remarkably fast.
In this talk, Denis Rystsov, Staff Engineer at Vectorized will explain how the Redpanda streaming data platform utilizes modern transactional approaches and pushes the envelope further by adjusting these concepts to the streaming workload. He'll share benchmarks and explain what makes Redpanda transactions so fast.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Netflix Data Benchmark @ HPTS 2017
Cloud database vendors tend to report performance numbers for the sweet spot or running in highly optimized hardware with specific workload parameters.
Moreover, many of these systems are not tested under different failure scenarios that may appear in the public cloud.
At Netflix, as a cloud-native enterprise, our focus is on high availability. We achieve high availability by deploying at multiple regions.
Hence, our data store system performance is highly affected by our global deployment model, instance types and workload patterns.
Hence, we were interested in a Cloud database benchmark tool that could be deployed in a loosely-coupled fashion, as a microservice, and with the ability to dynamically change the configurations parameters at run time. In this paper, we present Netflix Data Benchmark (NDBench). NDBench offers pluggable patterns and loads and support for different client APIs. It offers the ability to deploy, manage and monitor multiple instances from a single point. NDBench was designed to run for infinite time. This nature offered us with the ability to test long-running database maintenance jobs, test database systems under conditions that affect the performance, such as compactions, repairs etc., and also gives us a view on client side issues like memory leaks and heap pressure. We have been running NDBench for almost 3 years having validated multiple database versions, tested numerous NoSQL systems running on the Cloud, and tested new functionalities. NDBench is major component of our testing and validation pipelines.
Cypher for Apache Spark
Presented at the Second openCypher Implementers Meeting in London, UK, May 2017 @ http://www.opencypher.org/blog/2017/06/13/ocim2-blog/
OSOM Operations in the Cloud
The document discusses operations in the cloud and best practices. It describes how companies like Zynga and others were able to scale games and applications using AWS services like EC2, S3, EBS, ELB, and RDS. It discusses high availability, making applications stateless, monitoring, and open source alternatives. Best practices include infrastructure as code, automated provisioning, eliminating single points of failure, caching, and following the sun for development.
OSOM - Operations in the Cloud
The document discusses operations in the cloud and best practices. It describes how companies like Zynga and others were able to scale games and applications using AWS services like EC2, S3, EBS, ELB, and RDS. It discusses high availability, making applications stateless, monitoring, and open source alternatives. Best practices include infrastructure as code, automated provisioning, eliminating single points of failure, caching, and following the sun for development.
Migrating batch ETLs to streaming Flink
Data engineering presentation about the evolution of data infrastructure and migrating batch analytics jobs to streaming pipelines with Apache Flink.
How a Time Series Database Contributes to a Decentralized Cloud Object Storag...
In this presentation, you'll learn how InfluxDB is a component to Storj’s Tardigrade service and workflows. John Gleeson and Ben Sirb of Storj Lab will Storj’s redefinition of a cloud object storage network, how InfluxData fits into Storj’s Open Source Partner Program, and how to collect and manage high-volume, real-time telemetry data from a distributed network.
Apache Cassandra Lunch #52: Airflow and Cassandra for Cluster Management
In Apache Cassandra Lunch #52: Airflow and Cassandra for Cluster Management, we discussed using Airflow to schedule tasks on a Cassandra cluster beyond what could be accomplished with the Cassandra provider package.
Introducing TiDB @ SF DevOps Meetup
This document introduces TiDB, an open source distributed SQL database developed by PingCAP. It provides a 3-part summary:
1) TiDB is a hybrid transactional/analytical database inspired by Google Spanner/F1 that provides horizontal scalability, MySQL compatibility, and ACID transactions. It consists of TiDB, TiKV, and Placement Driver.
2) Mobike, a bike sharing platform with 200 million users, uses TiDB to power operations like bike locking/unlocking tracking and real-time analytics to handle high concurrency and permanent storage needs.
3) Over 200 companies use TiDB for two major uses - MySQL scalability and hybrid OLTP/OLAP architecture
Evolving ALLSTOCKER: Agile increments with Pharo Smalltalk
ALLSTOCKER is an online marketplace for used construction equipment that uses Pharo Smalltalk. It evolved its development process and architecture over time to handle growth. Originally a single Pharo image, it adopted microservices, multiple Pharo images, and reactive technologies like Elasticsearch and Neo4j to improve search and auctions. This allowed scaling to over 400 machine listings per month while maintaining responsiveness.
AtlasCamp 2015: How to deliver radical architectural change without the custo...
1. Atlassian delivered radical architectural changes by migrating 60,000 virtual Linux containers and 120,000 application instances to a microservices architecture on their platform without major incidents.
2. They achieved this through careful planning including gradual rollouts, shadowing new services, and making data migrations idempotent.
3. End-to-end ownership of the changes by dedicated teams also helped ensure flexibility and minimized incidents during the migration.
Putting the Spark into Functional Fashion Tech Analystics
Metail uses Apache Spark and a functional programming approach to process and analyze data from its fashion recommendation application. It collects data through various pipelines to understand user journeys and optimize business processes like photography. Metail's data pipeline is influenced by functional paradigms like immutability and uses Spark on AWS to operate on datasets in a distributed, scalable manner. The presentation demonstrated Metail's use of Clojure, Spark, and AWS services to build a functional data pipeline for analytics purposes.
2020 07-30 elastic agent + ingest management
This document provides an agenda and overview for a technical deep dive presentation on Elastic Agent and Ingest Manager. The presentation will include demos of enrolling an agent, collecting data out of the box, and configuring data collection. It will also provide a technical overview of components like the new indexing strategy, Elastic Agent, config options, and the Elastic Package Registry for centralized package management. Questions from the audience will be taken at the end.
OpenStack MagnetoDB. Atlanta Summit 2014
MagnetoDB presentation at Atlanta OpenStack Summit.
Agenda
========
MagnetoDB overview
Live demo
MagnetoDB Juno Roadmap
Q&A
MagnetoDB - the key-value database service with OpenStack native RESTful and Amazon DynamoDB API, predictable performance and seamless scalability
MagnetoDB key features
==================
REST API
Schemaless, non-relational table-based model
Put/get/query/scan item operations
Eventual and strong consistency
Local Secondary indexes
Batch read/write operations
DynamoDB API support
Seamless throughput and storage scaling
Fault tolerance
MagnetoDB use cases
=================
High request traffic level applications
Time series data
Queries are distributed across the data
Any other applications
Migration of workloads between AWS and OpenStack
Architecture highlights
================
Layered architecture
Horizontally scalable
Pluggable database backend
Leverage of existing OpenStack services
OpenStack HEAT
OpenStack Neutron LoadBalancer
OpenStack Trove
Current status
============
MagnetoDB 2.0.x released
Stackforge project, openstack community processes
OpenStack native API
Keystone v3 support
Multi-tenancy enforcement
Tempest tests (149 passed)
Devstack integration
Deployable to Vanilla OpenStack
Compatible with existing DynamoDB clients
Juno roadmap
===========
Incubation
Community
Enlarge core team
Features
python-magnetodb client
Tunable consistency
Atomic counters
Ceilometer monitoring integration
Other features
Quota management
more https://etherpad.openstack.org/p/magnetodb-juno-roadmap
GraphQL API on a Serverless Environment
The document discusses using GraphQL to build a serverless API for a mobile app that detects construction errors. Key points include:
1. GraphQL provides an efficient way to fetch data from microservices in a single roundtrip and allows flexible field selection.
2. The previous monolithic REST API was split into microservices for data fetching, state management, and serving the API to improve performance and scalability.
3. Serverless is a good fit for the stateless GraphQL architecture as resolvers can trigger other serverless functions and the API can scale up and down easily.
Prometheus loves Grafana
GrafanaCon 2015 - http://grafanacon.org/
Tobias will be giving an overview of Prometheus, an open-source monitoring system with a multi-dimensional label system, expressive query language and dashboard editor called PromDash. Learn about the highlights and differences of PromDash compared to Grafana and discuss the options to make Grafana the primary dashboard editor of the Prometheus project.
Dynamic in C# 4.0
The document discusses dynamic programming in .NET 4. It covers the benefits of dynamic languages like easier refactoring. The Dynamic Language Runtime (DLR) allows dynamic features in static languages and sharing libraries. Dynamic objects implement their own binding and the DLR optimizes them. While dynamic coding avoids type declarations, it lacks compiler checking and IntelliSense support. The document also compares dynamic programming to using the object and var types.
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
This document discusses moving MongoDB to the cloud. It provides an overview of MongoDB hosting options including on-premises data centers, cloud providers, and hosted databases. It outlines some key reasons to move to the cloud, such as cost-effectiveness, reduced need for staffing, and improved availability. It also covers important considerations for strategy planning including instance types, high availability strategy, security, and migration/rollback strategies. Finally, it discusses two common strategies for migrating - adding a cloud server to an existing replica set with no downtime, or taking backups and restoring to the cloud which requires downtime.
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
Building data pipelines is pretty hard! Building a multi-datacenter active-active real time data pipeline for multiple classes of data with different durability, latency and availability guarantees is much harder.
Real time infrastructure powers critical pieces of Uber (think Surge) and in this talk we will discuss our architecture, technical challenges, learnings and how a blend of open source infrastructure (Apache Kafka and Samza) and in-house technologies have helped Uber scale.
Scaling Infrastructure at Picnic
Being able to respond fast and effective to changes in traffic is key for a reliable and cost effective platform. The Corona outbreak came with unique shopping behaviour and corresponding challenges.
Busra will show how Picnic adopted their infrastructure to these new challenges and how both scheduled and automated scaling are strategically combined.
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more. It is used at Netflix to power various services by sitting on top of Redis and providing high availability, scalability, and tunable consistency. Conductor is a workflow orchestration engine used by Netflix that stores workflow definitions and state in Dynomite to allow reusable and controllable workflow processes.
Dynomite @ RedisConf 2017
- Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more.
- Netflix uses Dynomite to provide high availability and scalability for several internal microservices and tools like Conductor, an orchestration engine that uses Redis.
- Conductor allows defining workflows as code and executing them in a distributed and scalable way using Dynomite and Dyno Queues.
More Related Content
What's hot
Scylla Summit 2022: Overcoming the Performance Cost of Streaming Transactions
For a long time distributed transactions have been known to make systems slow and unavailable. However, recent academic and industry advancements such as RAMP, Occult, Calvin and TAPIR have changed the landscape and made transactions remarkably fast.
In this talk, Denis Rystsov, Staff Engineer at Vectorized will explain how the Redpanda streaming data platform utilizes modern transactional approaches and pushes the envelope further by adjusting these concepts to the streaming workload. He'll share benchmarks and explain what makes Redpanda transactions so fast.
To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.
Netflix Data Benchmark @ HPTS 2017
Cloud database vendors tend to report performance numbers for the sweet spot or running in highly optimized hardware with specific workload parameters.
Moreover, many of these systems are not tested under different failure scenarios that may appear in the public cloud.
At Netflix, as a cloud-native enterprise, our focus is on high availability. We achieve high availability by deploying at multiple regions.
Hence, our data store system performance is highly affected by our global deployment model, instance types and workload patterns.
Hence, we were interested in a Cloud database benchmark tool that could be deployed in a loosely-coupled fashion, as a microservice, and with the ability to dynamically change the configurations parameters at run time. In this paper, we present Netflix Data Benchmark (NDBench). NDBench offers pluggable patterns and loads and support for different client APIs. It offers the ability to deploy, manage and monitor multiple instances from a single point. NDBench was designed to run for infinite time. This nature offered us with the ability to test long-running database maintenance jobs, test database systems under conditions that affect the performance, such as compactions, repairs etc., and also gives us a view on client side issues like memory leaks and heap pressure. We have been running NDBench for almost 3 years having validated multiple database versions, tested numerous NoSQL systems running on the Cloud, and tested new functionalities. NDBench is major component of our testing and validation pipelines.
Cypher for Apache Spark
Presented at the Second openCypher Implementers Meeting in London, UK, May 2017 @ http://www.opencypher.org/blog/2017/06/13/ocim2-blog/
OSOM Operations in the Cloud
The document discusses operations in the cloud and best practices. It describes how companies like Zynga and others were able to scale games and applications using AWS services like EC2, S3, EBS, ELB, and RDS. It discusses high availability, making applications stateless, monitoring, and open source alternatives. Best practices include infrastructure as code, automated provisioning, eliminating single points of failure, caching, and following the sun for development.
OSOM - Operations in the Cloud
The document discusses operations in the cloud and best practices. It describes how companies like Zynga and others were able to scale games and applications using AWS services like EC2, S3, EBS, ELB, and RDS. It discusses high availability, making applications stateless, monitoring, and open source alternatives. Best practices include infrastructure as code, automated provisioning, eliminating single points of failure, caching, and following the sun for development.
Migrating batch ETLs to streaming Flink
Data engineering presentation about the evolution of data infrastructure and migrating batch analytics jobs to streaming pipelines with Apache Flink.
How a Time Series Database Contributes to a Decentralized Cloud Object Storag...
In this presentation, you'll learn how InfluxDB is a component to Storj’s Tardigrade service and workflows. John Gleeson and Ben Sirb of Storj Lab will Storj’s redefinition of a cloud object storage network, how InfluxData fits into Storj’s Open Source Partner Program, and how to collect and manage high-volume, real-time telemetry data from a distributed network.
Apache Cassandra Lunch #52: Airflow and Cassandra for Cluster Management
In Apache Cassandra Lunch #52: Airflow and Cassandra for Cluster Management, we discussed using Airflow to schedule tasks on a Cassandra cluster beyond what could be accomplished with the Cassandra provider package.
Introducing TiDB @ SF DevOps Meetup
This document introduces TiDB, an open source distributed SQL database developed by PingCAP. It provides a 3-part summary:
1) TiDB is a hybrid transactional/analytical database inspired by Google Spanner/F1 that provides horizontal scalability, MySQL compatibility, and ACID transactions. It consists of TiDB, TiKV, and Placement Driver.
2) Mobike, a bike sharing platform with 200 million users, uses TiDB to power operations like bike locking/unlocking tracking and real-time analytics to handle high concurrency and permanent storage needs.
3) Over 200 companies use TiDB for two major uses - MySQL scalability and hybrid OLTP/OLAP architecture
Evolving ALLSTOCKER: Agile increments with Pharo Smalltalk
ALLSTOCKER is an online marketplace for used construction equipment that uses Pharo Smalltalk. It evolved its development process and architecture over time to handle growth. Originally a single Pharo image, it adopted microservices, multiple Pharo images, and reactive technologies like Elasticsearch and Neo4j to improve search and auctions. This allowed scaling to over 400 machine listings per month while maintaining responsiveness.
AtlasCamp 2015: How to deliver radical architectural change without the custo...
1. Atlassian delivered radical architectural changes by migrating 60,000 virtual Linux containers and 120,000 application instances to a microservices architecture on their platform without major incidents.
2. They achieved this through careful planning including gradual rollouts, shadowing new services, and making data migrations idempotent.
3. End-to-end ownership of the changes by dedicated teams also helped ensure flexibility and minimized incidents during the migration.
Putting the Spark into Functional Fashion Tech Analystics
Metail uses Apache Spark and a functional programming approach to process and analyze data from its fashion recommendation application. It collects data through various pipelines to understand user journeys and optimize business processes like photography. Metail's data pipeline is influenced by functional paradigms like immutability and uses Spark on AWS to operate on datasets in a distributed, scalable manner. The presentation demonstrated Metail's use of Clojure, Spark, and AWS services to build a functional data pipeline for analytics purposes.
2020 07-30 elastic agent + ingest management
This document provides an agenda and overview for a technical deep dive presentation on Elastic Agent and Ingest Manager. The presentation will include demos of enrolling an agent, collecting data out of the box, and configuring data collection. It will also provide a technical overview of components like the new indexing strategy, Elastic Agent, config options, and the Elastic Package Registry for centralized package management. Questions from the audience will be taken at the end.
OpenStack MagnetoDB. Atlanta Summit 2014
MagnetoDB presentation at Atlanta OpenStack Summit.
Agenda
========
MagnetoDB overview
Live demo
MagnetoDB Juno Roadmap
Q&A
MagnetoDB - the key-value database service with OpenStack native RESTful and Amazon DynamoDB API, predictable performance and seamless scalability
MagnetoDB key features
==================
REST API
Schemaless, non-relational table-based model
Put/get/query/scan item operations
Eventual and strong consistency
Local Secondary indexes
Batch read/write operations
DynamoDB API support
Seamless throughput and storage scaling
Fault tolerance
MagnetoDB use cases
=================
High request traffic level applications
Time series data
Queries are distributed across the data
Any other applications
Migration of workloads between AWS and OpenStack
Architecture highlights
================
Layered architecture
Horizontally scalable
Pluggable database backend
Leverage of existing OpenStack services
OpenStack HEAT
OpenStack Neutron LoadBalancer
OpenStack Trove
Current status
============
MagnetoDB 2.0.x released
Stackforge project, openstack community processes
OpenStack native API
Keystone v3 support
Multi-tenancy enforcement
Tempest tests (149 passed)
Devstack integration
Deployable to Vanilla OpenStack
Compatible with existing DynamoDB clients
Juno roadmap
===========
Incubation
Community
Enlarge core team
Features
python-magnetodb client
Tunable consistency
Atomic counters
Ceilometer monitoring integration
Other features
Quota management
more https://etherpad.openstack.org/p/magnetodb-juno-roadmap
GraphQL API on a Serverless Environment
The document discusses using GraphQL to build a serverless API for a mobile app that detects construction errors. Key points include:
1. GraphQL provides an efficient way to fetch data from microservices in a single roundtrip and allows flexible field selection.
2. The previous monolithic REST API was split into microservices for data fetching, state management, and serving the API to improve performance and scalability.
3. Serverless is a good fit for the stateless GraphQL architecture as resolvers can trigger other serverless functions and the API can scale up and down easily.
Prometheus loves Grafana
GrafanaCon 2015 - http://grafanacon.org/
Tobias will be giving an overview of Prometheus, an open-source monitoring system with a multi-dimensional label system, expressive query language and dashboard editor called PromDash. Learn about the highlights and differences of PromDash compared to Grafana and discuss the options to make Grafana the primary dashboard editor of the Prometheus project.
Dynamic in C# 4.0
The document discusses dynamic programming in .NET 4. It covers the benefits of dynamic languages like easier refactoring. The Dynamic Language Runtime (DLR) allows dynamic features in static languages and sharing libraries. Dynamic objects implement their own binding and the DLR optimizes them. While dynamic coding avoids type declarations, it lacks compiler checking and IntelliSense support. The document also compares dynamic programming to using the object and var types.
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
This document discusses moving MongoDB to the cloud. It provides an overview of MongoDB hosting options including on-premises data centers, cloud providers, and hosted databases. It outlines some key reasons to move to the cloud, such as cost-effectiveness, reduced need for staffing, and improved availability. It also covers important considerations for strategy planning including instance types, high availability strategy, security, and migration/rollback strategies. Finally, it discusses two common strategies for migrating - adding a cloud server to an existing replica set with no downtime, or taking backups and restoring to the cloud which requires downtime.
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
Building data pipelines is pretty hard! Building a multi-datacenter active-active real time data pipeline for multiple classes of data with different durability, latency and availability guarantees is much harder.
Real time infrastructure powers critical pieces of Uber (think Surge) and in this talk we will discuss our architecture, technical challenges, learnings and how a blend of open source infrastructure (Apache Kafka and Samza) and in-house technologies have helped Uber scale.
Scaling Infrastructure at Picnic
Being able to respond fast and effective to changes in traffic is key for a reliable and cost effective platform. The Corona outbreak came with unique shopping behaviour and corresponding challenges.
Busra will show how Picnic adopted their infrastructure to these new challenges and how both scheduled and automated scaling are strategically combined.
What's hot (20)
Scylla Summit 2022: Overcoming the Performance Cost of Streaming Transactions
Scylla Summit 2022: Overcoming the Performance Cost of Streaming Transactions
How a Time Series Database Contributes to a Decentralized Cloud Object Storag...
How a Time Series Database Contributes to a Decentralized Cloud Object Storag...
Apache Cassandra Lunch #52: Airflow and Cassandra for Cluster Management
Apache Cassandra Lunch #52: Airflow and Cassandra for Cluster Management
Evolving ALLSTOCKER: Agile increments with Pharo Smalltalk
Evolving ALLSTOCKER: Agile increments with Pharo Smalltalk
AtlasCamp 2015: How to deliver radical architectural change without the custo...
AtlasCamp 2015: How to deliver radical architectural change without the custo...
Putting the Spark into Functional Fashion Tech Analystics
Putting the Spark into Functional Fashion Tech Analystics
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
Similar to ActiveSupport::Notifications inside out
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more. It is used at Netflix to power various services by sitting on top of Redis and providing high availability, scalability, and tunable consistency. Conductor is a workflow orchestration engine used by Netflix that stores workflow definitions and state in Dynomite to allow reusable and controllable workflow processes.
Dynomite @ RedisConf 2017
- Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more.
- Netflix uses Dynomite to provide high availability and scalability for several internal microservices and tools like Conductor, an orchestration engine that uses Redis.
- Conductor allows defining workflows as code and executing them in a distributed and scalable way using Dynomite and Dyno Queues.
Big data Argentina meetup 2020-09: Intro to presto on docker
We will talk about how we are migrating our Presto clusters from AWS EMR to Docker using production-grade orchestrators
considering cluster management, configuration and monitoring. We will discuss between Hashicorp Nomad and Kubernetes as a base solution
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
How to let IaC support Unit Testing by Terratest, Integration Testing by Open Policy Agent, and GitOps by Atlantis
Deploying Perl apps on dotCloud
Perl Mongers presentation on deploying Perl web applications on dotCloud. Examples use a minimal Mojolicious::Lite app.
Scaling up uber's real time data analytics
Realtime infrastructure powers critical pieces of Uber. This talk will discuss the architecture, technical challenges, learnings and how a blend of open source infrastructure (Apache Kafka/Flink/Pinot) and in-house technologies have helped Uber scale and enabled SQL to power realtime decision making for city ops, data scientists, data analysts and engineers.
NetflixOSS Meetup season 3 episode 1
This summary provides an overview of the key points from the document in 3 sentences:
The document outlines the agenda for Season 3 Episode 1 of the Netflix OSS podcast, which includes lightning talks on 8 new projects including Atlas, Prana, Raigad, Genie 2, Inviso, Dynomite, Nicobar, and MSL. Representatives from Netflix, IBM Watson, Nike Digital, and Pivotal then each provide a 3-5 minute presentation on their featured project. The presentations describe the motivation, features and benefits of each project for observability, integration with the Netflix ecosystem, automation of Elasticsearch deployments, job scheduling, dynamic scripting for Java, message security, and developing microservices
Devops with Python by Yaniv Cohen DevopShift
This document discusses implementing DevOps with Python using Ansible. It provides an agenda for the presentation including discussing DevOps hotspots, infrastructure as code with Ansible, continuous integration/continuous delivery (CI/CD) using TravisCI and CircleCI, and an open discussion on monitoring and automated tests. It then covers problems commonly faced, how DevOps solves these problems, and the expected benefits of adopting a DevOps culture including standardized environments, infrastructure as code, automated delivery, monitoring, and improved collaboration. It provides an overview of Ansible concepts like inventories, ad-hoc commands, modules, playbooks, roles, and templates. It also demonstrates writing a custom Python module for Ansible and using it in a playbook. Finally, it
Monal Daxini - Beaming Flink to the Cloud @ Netflix
http://flink-forward.org/kb_sessions/beaming-flink-to-the-cloud-netflix/
Netflix is a data driven company and we process over 700 billion streaming events per day with at-least once processing semantics in the cloud. To enable extracting intelligence from this unbounded stream easily we are building Stream Processing as a Service (SPaaS) infrastructure so that the user can focus on extracting value and not have to worry about boilerplate infrastructure and scale. We will share our experience in building a scalable SPaaS using Flink, Apache Beam and Kafka as the foundation layer to process over 1.3 PB of event data without service disruption.
Beaming flink to the cloud @ netflix ff 2016-monal-daxini
Netflix is a data driven company and we process over 700 billion streaming events per day with at-least once processing semantics in the cloud. To enable extracting intelligence from this unbounded stream easily we are building Stream Processing as a Service (SPaaS) infrastructure so that the user can focus on extracting value and not have to worry about boilerplate infrastructure and scale.
We will share our experience in building a scalable SPaaS using Flink, Apache Beam and Kafka as the foundation layer to process over 1.3 PB of event data without service disruption.
A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)
This document describes BBVA's implementation of a Big Data Lake using Apache Spark for log collection, storage, and analytics. It discusses:
1) Using Syslog-ng for log collection from over 2,000 applications and devices, distributing logs to Kafka.
2) Storing normalized logs in HDFS and performing analytics using Spark, with outputs to analytics, compliance, and indexing systems.
3) Choosing Spark because it allows interactive, batch, and stream processing with one system using RDDs, SQL, streaming, and machine learning.
Writing and deploying serverless python applications
This document provides an overview of serverless computing using AWS Lambda. It discusses what serverless means, how it addresses issues with traditional server-based architectures like capacity planning and scaling. It then covers how to build and deploy serverless Python applications using AWS Lambda, including choosing templates and triggers, adding configuration and code, testing, and writing clients. Alternatives like Google Cloud Functions and the Zappa framework for deploying serverless apps are also mentioned.
PaaSTA: Autoscaling at Yelp
Capacity planning is a difficult challenge faced by most companies. If you have too few machines, you will not have enough compute resources available to deal with heavy loads. On the other hand, if you have too many machines, you are wasting money. This is why companies have started investing in automatically scaling services and infrastructure to minimize the amount of wasted money and resources.
In this talk, Nathan will describe how Yelp is using PaaSTA, a PaaS built on top of open source tools including Docker, Mesos, Marathon, and Chronos, to automatically and gracefully scale services and the underlying cluster. He will go into detail about how this functionality was implemented and the design designs that were made while architecting the system. He will also provide a brief comparison of how this approach differs from existing solutions.
Introducing TiDB Operator
This document discusses PingCAP's Kubernetes operator for TiDB, an open source distributed SQL database. It provides a brief history of PingCAP and the TiDB community. It then gives a technical overview of TiDB's architecture before explaining how the TiDB operator works. The operator allows users to deploy and manage TiDB clusters on Kubernetes through custom resources that are controlled by custom controllers. This provides capabilities like automated scaling, updates, and failover for stateful applications running on Kubernetes. The operator is open source and TiDB is also available as a managed service on GCP Marketplace.
Real Time Data Streaming using Kafka & Storm
This presentation describes 3 real use case of Real-Time Data Streaming and how they were implemented in LivePerson using Kafka and Storm
The architecture of SkySQL
SkySQL implements a groundbreaking, state-of-the-art architecture based on Kubernetes and ServiceNow, and with a strong emphasis on cloud security – using compartmentalization and indirect access to secure and protect customer databases.
In this session, we’ll walk through the architecture of SkySQL and discuss how MariaDB leverages an advanced Kubernetes operator and powerful ServiceNow configuration/workflow management to deploy and manage databases on cloud infrastructure.
Infrastructure Considerations : Design : "webops"
This document provides recommendations for preparing infrastructure for the future. It suggests choosing a long-term supported Linux distribution like RHEL, CentOS, or Ubuntu LTS. Infrastructure should be designed to be scalable, robust, manageable, resilient, and cost-effective. Services should be split across redundant servers and high availability/failover implemented. Automation, configuration management, and continuous integration/deployment are also recommended. Comprehensive monitoring of infrastructure and applications is important for capacity planning and issue detection. The document provides additional tips regarding caching, security, change management, backups, and ongoing re-architecting.
Monitoring the Dynamic Resource Usage of Scala and Python Spark Jobs in Yarn:...
We all dread “Lost task” and “Container killed by YARN for exceeding memory limits” messages in our scaled-up spark yarn applications. Even answering the question “How much memory did my application use?” is surprisingly tricky in the distributed yarn environment. Sqrrl has developed a testing framework for observing vital statistics of spark jobs including executor-by-executor memory and CPU usage over time for both the JDK and python portions of pyspark yarn containers. This talk will detail the methods we use to collect, store, and report spark yarn resource usage. This information has proved to be invaluable for performance and regression testing of the spark jobs in Sqrrl Enterprise.
(BDT404) Large-Scale ETL Data Flows w/AWS Data Pipeline & Dataduct
"As data volumes grow, managing and scaling data pipelines for ETL and batch processing can be daunting. With more than 13.5 million learners worldwide, hundreds of courses, and thousands of instructors, Coursera manages over a hundred data pipelines for ETL, batch processing, and new product development.
In this session, we dive deep into AWS Data Pipeline and Dataduct, an open source framework built at Coursera to manage pipelines and create reusable patterns to expedite developer productivity. We share the lessons learned during our journey: from basic ETL processes, such as loading data from Amazon RDS to Amazon Redshift, to more sophisticated pipelines to power recommendation engines and search services.
Attendees learn:
Do's and don’ts of Data Pipeline
Using Dataduct to streamline your data pipelines
How to use Data Pipeline to power other data products, such as recommendation systems
What’s next for Dataduct"
Blackray @ SAPO CodeBits 2009
The document provides an overview and agenda for a presentation on the BlackRay database. It summarizes BlackRay's history and capabilities, positions it relative to other projects, and outlines the team and roadmap. The presentation covers BlackRay's architecture, APIs, management features, clustering support, and roadmap for future improvements.
Similar to ActiveSupport::Notifications inside out (20)
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
Big data Argentina meetup 2020-09: Intro to presto on docker
Big data Argentina meetup 2020-09: Intro to presto on docker
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
DevOpsDays Taipei 2019 - Mastering IaC the DevOps Way
Monal Daxini - Beaming Flink to the Cloud @ Netflix
Monal Daxini - Beaming Flink to the Cloud @ Netflix
Beaming flink to the cloud @ netflix ff 2016-monal-daxini
Beaming flink to the cloud @ netflix ff 2016-monal-daxini
A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)
A Big Data Lake Based on Spark for BBVA Bank-(Oscar Mendez, STRATIO)
Writing and deploying serverless python applications
Writing and deploying serverless python applications
Monitoring the Dynamic Resource Usage of Scala and Python Spark Jobs in Yarn:...
Monitoring the Dynamic Resource Usage of Scala and Python Spark Jobs in Yarn:...
(BDT404) Large-Scale ETL Data Flows w/AWS Data Pipeline & Dataduct
(BDT404) Large-Scale ETL Data Flows w/AWS Data Pipeline & Dataduct
Recently uploaded
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
Charla impartida en el evento de "KuberTENes Birthday Bash Guadalajara" para celebrar el 10mo. aniversario de Kubernetes #kuberTENes #celebr8k8s #k8s
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers. This video you can watch from my youtube channel at https://youtu.be/m-F-mZA3MkU
E-commerce Application Development Company.pdf
Your business can reach new heights with our assistance as we design solutions that are specifically appropriate for your goals and vision. Our eCommerce application solutions can digitally coordinate all retail operations processes to meet the demands of the marketplace while maintaining business continuity.
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat is industry-trusted event management software, built by event professionals for event professionals. Founded in 2017, we have three key products tailored for the live event industry.
Crescat Event for concert promoters and event agencies. Crescat Venue for music venues, conference centers, wedding venues, concert halls and more. And Crescat Festival for festivals, conferences and complex events.
With a wide range of popular features such as event scheduling, shift management, volunteer and crew coordination, artist booking and much more, Crescat is designed for customisation and ease-of-use.
Over 125,000 events have been planned in Crescat and with hundreds of customers of all shapes and sizes, from boutique event agencies through to international concert promoters, Crescat is rigged for success. What's more, we highly value feedback from our users and we are constantly improving our software with updates, new features and improvements.
If you plan events, run a venue or produce festivals and you're looking for ways to make your life easier, then we have a solution for you. Try our software for free or schedule a no-obligation demo with one of our product specialists today at crescat.io
Hand Rolled Applicative User Validation Code Kata
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
https://2024.springio.net/sessions/automated-software-refactoring-with-openrewrite-and-generative-ai/
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
Regarding mobile operating systems, two major players dominate our thoughts: Android and iPhone. With Android leading the market, software development companies are focused on delivering apps compatible with this OS. Ensuring an app's functionality across various Android devices, OS versions, and hardware specifications is critical, making Android app testing essential.
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Vitthal Shirke Java Microservices Resume.pdf
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
Artificia Intellicence and XPath Extension Functions
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Presentation from Ghazal Aakel and Eric Jochum at the GSD Community Stage Meetup on June 6th, 2024
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar's Aviation Industry Q1 Incident Report is out now!
The aviation industry has always been a prime target for cybercriminals due to its critical infrastructure and high stakes. In the first quarter of 2024, the sector faced an alarming surge in cybersecurity threats, revealing its vulnerabilities and the relentless sophistication of cyber attackers.
SOCRadar’s Aviation Industry, Quarterly Incident Report, provides an in-depth analysis of these threats, detected and examined through our extensive monitoring of hacker forums, Telegram channels, and dark web platforms.
Webinar On-Demand: Using Flutter for Embedded
Flutter is a popular open source, cross-platform framework developed by Google. In this webinar we'll explore Flutter and its architecture, delve into the Flutter Embedder and Flutter’s Dart language, discover how to leverage Flutter for embedded device development, learn about Automotive Grade Linux (AGL) and its consortium and understand the rationale behind AGL's choice of Flutter for next-gen IVI systems. Don’t miss this opportunity to discover whether Flutter is right for your project.
OpenMetadata Community Meeting - 5th June 2024
The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features.
* How to run your own data quality framework
* What is the performance impact of running data quality frameworks
* How to run the test cases in your own ETL pipelines
* How the Incident Manager is integrated
* Get notified with alerts when test cases fail
Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Dr. Jesús Barrasa, Head of Solutions Architecture for EMEA, Neo4j
Découvrez les dernières innovations de Neo4j, et notamment les dernières intégrations cloud et les améliorations produits qui font de Neo4j un choix essentiel pour les développeurs qui créent des applications avec des données interconnectées et de l’IA générative.
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版一模一样【微信：741003700 】【美国纽约州立大学奥尔巴尼分校毕业证学位证书】【微信：741003700 】学位证，留信认证（真实可查，永久存档）offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原海外各大学 Bachelor Diploma degree, Master Degree Diploma
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
WhatsApp offers simple, reliable, and private messaging and calling services for free worldwide. With end-to-end encryption, your personal messages and calls are secure, ensuring only you and the recipient can access them. Enjoy voice and video calls to stay connected with loved ones or colleagues. Express yourself using stickers, GIFs, or by sharing moments on Status. WhatsApp Business enables global customer outreach, facilitating sales growth and relationship building through showcasing products and services. Stay connected effortlessly with group chats for planning outings with friends or staying updated on family conversations.
Recently uploaded (20)
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
ActiveSupport::Notifications inside out
- 2. Context ★ Runtime data collection management ★ Introduce Rails instrumentation ★ Instrumentation in practice ★ AS::Notification implementation ★ Maximise instrumentation power
- 3. Runtime data collection management
- 4. Problems ★ Logging, profiling modules etc tightly coupled with functional modules ★ Modules mixed together (SRP violation) ★ Different types of runtime data require different handlers
- 5. Instrumentation to rescue ★ Data logging ★ Event tracking ★ Performance tracking ★ Debugging and exception handling ★ Profiling
- 6. Introduce Rails instrumentation ★ Provide and used by Rails ★ Hook with Rails events or BYO ★ Pub/ Sub pattern ★ Highly customisable modules ★ Thread safe
- 7. How Rails instrument - ActiveRecord
- 8. How Rails instrument - ActionView
- 10. Instrumentation in practice (cont)
- 11. Instrumentation in practice (cont)
- 12. Instrumentation in practice (cont)
- 14. ActiveSupport::PerThreadRegistry Namesapced global varibles per thread
- 15. Maximise instrumentation power ★ Logging ★ New Relic ★ Email notification ★ Event tracking - SiteCatalyst ★ Stats collection - statsD ★ Trigger Background jobs ★ Web hooks to trigger other web services
- 16. Maximise instrumentation power ★ Independent from Rails, plug in to any Ruby projects ★ Good Abstraction ○ replace notifier (Fanout) with your own queue ○ pass in your own listeners to customise start/ publish/ finish hooks, instead of plain blocks ★ Send slow subscriber works to background jobs or message queues