This document summarizes a presentation about distributed caching technologies from key-value stores to in-memory data grids. It discusses the memory hierarchy and how software caches can improve performance by reducing data access latency and offloading storage. Different caching patterns like cache-aside, read-through, write-through and write-behind are explained. Popular caching products including Memcached, Redis, Cassandra and data grids are overviewed. Advanced concepts covered include data distribution, replication, consistency protocols and use cases.
Communication between Microservices is inherently unreliable. These integration points may produce cascading failures, slow responses, service outages. We will walk through stability patterns like timeouts, circuit breaker, bulkheads and discuss how they improve stability of Microservices.
From cache to in-memory data grid. Introduction to Hazelcast.Taras Matyashovsky
This presentation:
* covers basics of caching and popular cache types
* explains evolution from simple cache to distributed, and from distributed to IMDG
* not describes usage of NoSQL solutions for caching
* is not intended for products comparison or for promotion of Hazelcast as the best solution
DNS is critical network infrastructure and securing it against attacks like DDoS, NXDOMAIN, hijacking and Malware/APT is very important to protecting any business.
This presentation shortly describes key features of Apache Cassandra. It was held at the Apache Cassandra Meetup in Vienna in January 2014. You can access the meetup here: http://www.meetup.com/Vienna-Cassandra-Users/
Communication between Microservices is inherently unreliable. These integration points may produce cascading failures, slow responses, service outages. We will walk through stability patterns like timeouts, circuit breaker, bulkheads and discuss how they improve stability of Microservices.
From cache to in-memory data grid. Introduction to Hazelcast.Taras Matyashovsky
This presentation:
* covers basics of caching and popular cache types
* explains evolution from simple cache to distributed, and from distributed to IMDG
* not describes usage of NoSQL solutions for caching
* is not intended for products comparison or for promotion of Hazelcast as the best solution
DNS is critical network infrastructure and securing it against attacks like DDoS, NXDOMAIN, hijacking and Malware/APT is very important to protecting any business.
This presentation shortly describes key features of Apache Cassandra. It was held at the Apache Cassandra Meetup in Vienna in January 2014. You can access the meetup here: http://www.meetup.com/Vienna-Cassandra-Users/
Introduction to memcached, a caching service designed for optimizing performance and scaling in the web stack, seen from perspective of MySQL/PHP users. Given for 2nd year students of professional bachelor in ICT at Kaho St. Lieven, Gent.
This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013mumrah
Apache Kafka is a new breed of messaging system built for the "big data" world. Coming out of LinkedIn (and donated to Apache), it is a distributed pub/sub system built in Scala. It has been an Apache TLP now for several months with the first Apache release imminent. Built for speed, scalability, and robustness, Kafka should definitely be one of the data tools you consider when designing distributed data-oriented applications.
The talk will cover a general overview of the project and technology, with some use cases, and a demo.
Apache Flink is a popular stream computing framework for real-time stream computing. Many stream compute algorithms require trailing data in order to compute the intended result. One example is computing the number of user logins in the last 7 days. This creates a dilemma where the results of the stream program are incomplete until the runtime of the program exceeds 7 days. The alternative is to bootstrap the program using historic data to seed the state before shifting to use real-time data.
This talk will discuss alternatives to bootstrap programs in Flink. Some alternatives rely on technologies exogenous to the stream program, such as enhancements to the pub/sub layer, that are more generally applicable to other stream compute engines. Other alternatives include enhancements to Flink source implementations. Lyft is exploring another alternative using orchestration of multiple Flink programs. The talk will cover why Lyft pursued this alternative and future directions to further enhance bootstrapping support in Flink.
Speaker
Gregory Fee, Principal Engineer, Lyft
Thrift vs Protocol Buffers vs Avro - Biased ComparisonIgor Anishchenko
Igor Anishchenko
Odessa Java TechTalks
Lohika - May, 2012
Let's take a step back and compare data serialization formats, of which there are plenty. What are the key differences between Apache Thrift, Google Protocol Buffers and Apache Avro. Which is "The Best"? Truth of the matter is, they are all very good and each has its own strong points. Hence, the answer is as much of a personal choice, as well as understanding of the historical context for each, and correctly identifying your own, individual requirements.
Under The Hood Of A Shard-Per-Core Database ArchitectureScyllaDB
Most databases are based on architectures that pre-date advances to modern hardware. This results in performance issues, the need to overprovision, and a high total cost of ownership. In this webinar, we will discuss the advances to modern server technology and take a deep dive into ScyllaDB’s shard-per-core architecture and our asynchronous engine, the Seastar framework.
Join us to learn how Seastar (and ScyllaDB):
- Avoid locks and contention on the CPU level
- Bypass kernel bottlenecks
- Implement its per-core shared-nothing autosharding mechanism
- Utilize modern storage hardware
- Leverage NUMA to get the best RAM performance
- Balance your data across CPUs and nodes for the best and smoothest performance
Plus we’ll cover the advantages of unlocking vertical scalability.
From: DataWorks Summit 2017 - Munich - 20170406
HBase hast established itself as the backend for many operational and interactive use-cases, powering well-known services that support millions of users and thousands of concurrent requests. In terms of features HBase has come a long way, overing advanced options such as multi-level caching on- and off-heap, pluggable request handling, fast recovery options such as region replicas, table snapshots for data governance, tuneable write-ahead logging and so on. This talk is based on the research for the an upcoming second release of the speakers HBase book, correlated with the practical experience in medium to large HBase projects around the world. You will learn how to plan for HBase, starting with the selection of the matching use-cases, to determining the number of servers needed, leading into performance tuning options. There is no reason to be afraid of using HBase, but knowing its basic premises and technical choices will make using it much more successful. You will also learn about many of the new features of HBase up to version 1.3, and where they are applicable.
Cluster computing frameworks such as Hadoop or Spark are tremendously beneficial in processing and deriving insights from data. However, long query latencies make these frameworks sub-optimal choices to power interactive applications. Organizations frequently rely on dedicated query layers, such as relational databases and key/value stores, for faster query latencies, but these technologies suffer many drawbacks for analytic use cases. In this session, we discuss using Druid for analytics and why the architecture is well suited to power analytic applications.
User-facing applications are replacing traditional reporting interfaces as the preferred means for organizations to derive value from their datasets. In order to provide an interactive user experience, user interactions with analytic applications must complete in an order of milliseconds. To meet these needs, organizations often struggle with selecting a proper serving layer. Many serving layers are selected because of their general popularity without understanding the possible architecture limitations.
Druid is an analytics data store designed for analytic (OLAP) queries on event data. It draws inspiration from Google’s Dremel, Google’s PowerDrill, and search infrastructure. Many enterprises are switching to Druid for analytics, and we will cover why the technology is a good fit for its intended use cases.
Speaker
Nishant Bangarwa, Software Engineer, Hortonworks
We will show the advantages of having a geo-distributed database cluster and how to create one using Galera Cluster for MySQL. We will also discuss the configuration and status variables that are involved and how to deal with typical situations on the WAN such as slow, untrusted or unreliable links, latency and packet loss. We will demonstrate a multi-region cluster on Amazon EC2 and perform some throughput and latency measurements in real-time (video http://galeracluster.com/videos/using-galera-replication-to-create-geo-distributed-clusters-on-the-wan-webinar-video-3/)
Capacity Planning For Your Growing MongoDB ClusterMongoDB
Your MongoDB deployment is growing, but are you prepared for that growth? Capacity planning is an essential practice when deploying any database system. You need to understand your usage patterns and determine the appropriate hardware based on your application's needs. Scaling reads and scaling writes will require different types of resources. With the proper tools in place, you can understand your working set, gain visibility into when it's time to add resources or start sharding and avoid performance issues. In this session, you'll learn how to use MongoDB Management Service and other tools to identify patterns and predict growth, ensuring your success with MongoDB.
Cassandra Data Modeling - Practical Considerations @ Netflixnkorla1share
Cassandra community has consistently requested that we cover C* schema design concepts. This presentation goes in depth on the following topics:
- Schema design
- Best Practices
- Capacity Planning
- Real World Examples
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Lindsey Aitchison
Having tested and validated Oracle Grid reference configurations, Dell Global Solutions engineers share their insight of how best to set up and implement this computing resource to enable networked computers to share on-demand resource pools.
Introduction to memcached, a caching service designed for optimizing performance and scaling in the web stack, seen from perspective of MySQL/PHP users. Given for 2nd year students of professional bachelor in ICT at Kaho St. Lieven, Gent.
This is the presentation I made on JavaDay Kiev 2015 regarding the architecture of Apache Spark. It covers the memory model, the shuffle implementations, data frames and some other high-level staff and can be used as an introduction to Apache Spark
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013mumrah
Apache Kafka is a new breed of messaging system built for the "big data" world. Coming out of LinkedIn (and donated to Apache), it is a distributed pub/sub system built in Scala. It has been an Apache TLP now for several months with the first Apache release imminent. Built for speed, scalability, and robustness, Kafka should definitely be one of the data tools you consider when designing distributed data-oriented applications.
The talk will cover a general overview of the project and technology, with some use cases, and a demo.
Apache Flink is a popular stream computing framework for real-time stream computing. Many stream compute algorithms require trailing data in order to compute the intended result. One example is computing the number of user logins in the last 7 days. This creates a dilemma where the results of the stream program are incomplete until the runtime of the program exceeds 7 days. The alternative is to bootstrap the program using historic data to seed the state before shifting to use real-time data.
This talk will discuss alternatives to bootstrap programs in Flink. Some alternatives rely on technologies exogenous to the stream program, such as enhancements to the pub/sub layer, that are more generally applicable to other stream compute engines. Other alternatives include enhancements to Flink source implementations. Lyft is exploring another alternative using orchestration of multiple Flink programs. The talk will cover why Lyft pursued this alternative and future directions to further enhance bootstrapping support in Flink.
Speaker
Gregory Fee, Principal Engineer, Lyft
Thrift vs Protocol Buffers vs Avro - Biased ComparisonIgor Anishchenko
Igor Anishchenko
Odessa Java TechTalks
Lohika - May, 2012
Let's take a step back and compare data serialization formats, of which there are plenty. What are the key differences between Apache Thrift, Google Protocol Buffers and Apache Avro. Which is "The Best"? Truth of the matter is, they are all very good and each has its own strong points. Hence, the answer is as much of a personal choice, as well as understanding of the historical context for each, and correctly identifying your own, individual requirements.
Under The Hood Of A Shard-Per-Core Database ArchitectureScyllaDB
Most databases are based on architectures that pre-date advances to modern hardware. This results in performance issues, the need to overprovision, and a high total cost of ownership. In this webinar, we will discuss the advances to modern server technology and take a deep dive into ScyllaDB’s shard-per-core architecture and our asynchronous engine, the Seastar framework.
Join us to learn how Seastar (and ScyllaDB):
- Avoid locks and contention on the CPU level
- Bypass kernel bottlenecks
- Implement its per-core shared-nothing autosharding mechanism
- Utilize modern storage hardware
- Leverage NUMA to get the best RAM performance
- Balance your data across CPUs and nodes for the best and smoothest performance
Plus we’ll cover the advantages of unlocking vertical scalability.
From: DataWorks Summit 2017 - Munich - 20170406
HBase hast established itself as the backend for many operational and interactive use-cases, powering well-known services that support millions of users and thousands of concurrent requests. In terms of features HBase has come a long way, overing advanced options such as multi-level caching on- and off-heap, pluggable request handling, fast recovery options such as region replicas, table snapshots for data governance, tuneable write-ahead logging and so on. This talk is based on the research for the an upcoming second release of the speakers HBase book, correlated with the practical experience in medium to large HBase projects around the world. You will learn how to plan for HBase, starting with the selection of the matching use-cases, to determining the number of servers needed, leading into performance tuning options. There is no reason to be afraid of using HBase, but knowing its basic premises and technical choices will make using it much more successful. You will also learn about many of the new features of HBase up to version 1.3, and where they are applicable.
Cluster computing frameworks such as Hadoop or Spark are tremendously beneficial in processing and deriving insights from data. However, long query latencies make these frameworks sub-optimal choices to power interactive applications. Organizations frequently rely on dedicated query layers, such as relational databases and key/value stores, for faster query latencies, but these technologies suffer many drawbacks for analytic use cases. In this session, we discuss using Druid for analytics and why the architecture is well suited to power analytic applications.
User-facing applications are replacing traditional reporting interfaces as the preferred means for organizations to derive value from their datasets. In order to provide an interactive user experience, user interactions with analytic applications must complete in an order of milliseconds. To meet these needs, organizations often struggle with selecting a proper serving layer. Many serving layers are selected because of their general popularity without understanding the possible architecture limitations.
Druid is an analytics data store designed for analytic (OLAP) queries on event data. It draws inspiration from Google’s Dremel, Google’s PowerDrill, and search infrastructure. Many enterprises are switching to Druid for analytics, and we will cover why the technology is a good fit for its intended use cases.
Speaker
Nishant Bangarwa, Software Engineer, Hortonworks
We will show the advantages of having a geo-distributed database cluster and how to create one using Galera Cluster for MySQL. We will also discuss the configuration and status variables that are involved and how to deal with typical situations on the WAN such as slow, untrusted or unreliable links, latency and packet loss. We will demonstrate a multi-region cluster on Amazon EC2 and perform some throughput and latency measurements in real-time (video http://galeracluster.com/videos/using-galera-replication-to-create-geo-distributed-clusters-on-the-wan-webinar-video-3/)
Capacity Planning For Your Growing MongoDB ClusterMongoDB
Your MongoDB deployment is growing, but are you prepared for that growth? Capacity planning is an essential practice when deploying any database system. You need to understand your usage patterns and determine the appropriate hardware based on your application's needs. Scaling reads and scaling writes will require different types of resources. With the proper tools in place, you can understand your working set, gain visibility into when it's time to add resources or start sharding and avoid performance issues. In this session, you'll learn how to use MongoDB Management Service and other tools to identify patterns and predict growth, ensuring your success with MongoDB.
Cassandra Data Modeling - Practical Considerations @ Netflixnkorla1share
Cassandra community has consistently requested that we cover C* schema design concepts. This presentation goes in depth on the following topics:
- Schema design
- Best Practices
- Capacity Planning
- Real World Examples
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Lindsey Aitchison
Having tested and validated Oracle Grid reference configurations, Dell Global Solutions engineers share their insight of how best to set up and implement this computing resource to enable networked computers to share on-demand resource pools.
NoSQL is not a buzzword anymore. The array of non- relational technologies have found wide-scale adoption even in non-Internet scale focus areas. With the advent of the Cloud...the churn has increased even more yet there is no crystal clear guidance on adoption techniques and architectural choices surrounding the plethora of options available. This session initiates you into the whys & wherefores, architectural patterns, caveats and techniques that will augment your decision making process & boost your perception of architecting scalable, fault-tolerant & distributed solutions.
In this session, we'll discuss architectural, design and tuning best practices for building rock solid and scalable Alfresco Solutions. We'll cover the typical use cases for highly scalable Alfresco solutions, like massive injection and high concurrency, also introducing 3.3 and 3.4 Transfer / Replication services for building complex high availability enterprise architectures.
Introduction to NoSQL | Big Data Hadoop Spark Tutorial | CloudxLabCloudxLab
Big Data with Hadoop & Spark Training: http://bit.ly/2kyP2Ct
This CloudxLab Introduction to NoSQL tutorial helps you to understand NoSQL in detail. Below are the topics covered in this slide:
1) Introduction to NoSQL
2) Scaling Out vs Scaling Up
3) ACID - Properties of DB Transactions
4) RDBMS - Story
5) What is NoSQL?
6) Types Of NoSQL Stores
7) CAP Theorem
8) Serialization
9) Column Oriented Database
10) Column Family Oriented DataStore
Beyond The Data Grid: Coherence, Normalisation, Joins and Linear ScalabilityBen Stopford
In 2009 RBS set out to build a single store of trade and risk data that all applications in the bank could use. This talk discusses a number of novel techniques that were developed as part of this work. Based on Oracle Coherence the ODC departs from the trend set by most caching solutions by holding its data in a normalised form making it both memory efficient and easy to change. However it does this in a novel way that supports most arbitrary queries without the usual problems associated with distributed joins. We'll be discussing these patterns as well as others that allow linear scalability, fault tolerance and millisecond latencies.
Using Distributed In-Memory Computing for Fast Data AnalysisScaleOut Software
This is an overview of how distributed data grids can enable sharing across web servers and virtual cloud environments to enable scalability and high availability. It also covers how distributed data grids are highly useful for running MapReduce analysis across large data sets.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
2. Memory Hierarchy
R
<1ns
L1
~4 cycles, ~1ns Cost
L2
~10 cycles, ~3ns
L3
~42 cycles, ~15ns
DRAM
>65ns
Flash / SSD / USB
Storage
term
HDD
Tapes, Remote systems, etc
2 Max A. Alexejev
3. Software caches
Improve response times by reducing data access latency
Offload persistent storages
Only work for IO-bound applications!
3 Max A. Alexejev
4. Caches and data location
Consistency
protocol
Shared
Local Remote
Distributed
Hierarchical Distribution
algorithm
4 Max A. Alexejev
5. Ok, so how do we grow beyond one node?
Data replication
5 Max A. Alexejev
6. Pro’s and Con’s of replication
Pro
• Best read performance (for local replicated caches)
• Fault tolerant cache (both local and remote)
• Can be smart: replicate only part of CRUD cycle
Con
• Poor writes performance
• Additional network load
• Can scale only vertically: limited by single machine size
• In case of master-master replication, requires complex consistency protocol
6 Max A. Alexejev
7. Ok, so how do we grow beyond one node?
Data distribution
7 Max A. Alexejev
8. Pro’s and Con’s of data distribution
Pro
• Can scale horizontally beyond single machine size
• Reads and writes performance scales horizontally
Con
• No fault tolerance for cached data
• Increased latency of reads (due to network round-trip and
serialization expenses)
8 Max A. Alexejev
9. What do high-load applications need
from cache?
Linear Distributed
Low
horizontal
latency cache
scalability
9 Max A. Alexejev
10. Cache access patterns: Client
Cache Aside
For reading data: For writing data
1. Application asks 1. Application writes
for some data for a some new data or
given key updates existing.
Cache
2. Check the cache 2. Write it to the
3. If data is in the cache
cache return it to 3. Write it to the DB.
the user
4. If data is not in the Overall:
cache fetch it from
the DB, put it in • Increases reads
the cache, return it performance
to the user. • Offloads DB reads
DB
• Introduces race
conditions for
writes
10 Max A. Alexejev
11. Cache access patterns: Client
Read Through
For reading data:
1. Application asks for some data for a given key
2. Check the cache
3. If data is in the cache return it to the user Cache
4. If data is not in the cache – cache will invoke fetching
it from the DB by himself, saving retrieved value and
returning it to the user.
Overall:
• Reduces reads latency
• Offloads read load from underlying storage
• May have blocking behavior, thus helping with dog-pile DB
effect
• Requires “smarter” cache nodes
11 Max A. Alexejev
12. Cache access patterns: Client
Write Through
For writing data
1. Application writes some new data or
updates existing. Cache
2. Write it to the cache
3. Cache will then synchronously write it
to the DB.
Overall:
• Slightly increases writes latency
DB
• Provides natural invalidation
• Removes race conditions on writes
12 Max A. Alexejev
13. Cache access patterns: Client
Write Behind
For writing data
1. Application writes some new data or updates existing.
2. Write it to the cache
Cache adds writes request to its internal queue.
3.
Cache
4. Later, cache asynchronously flushes queue to DB on a
periodic basis and/or when queue size reaches certain
limit.
Overall:
• Dramatically reduces writes latency by a price of
inconsistency window
• Provides writes batching
• May provide updates deduplication DB
13 Max A. Alexejev
14. A variety of products on the market…
Memcached
Hazelcast
Cassandra
GigaSpaces
Redis
Terracotta
Oracle
Coherence Infinispan
MongoDB
Riak
EhCache …
14 Max A. Alexejev
15. KV caches NoSQL Data Grids
Oracle
Memcached Redis
Coherence
Ehcache Cassandra GemFire
Lets sort em out! … MongoDB GigaSpaces
Some products are really hard to
sort – like Terracotta in both DSO … GridGain
and Express modes.
Hazelcast
Infinispan
15 Max A. Alexejev
16. Why don’t we have any distributed
in-memory RDBMS?
Master – MultiSlaves configuration
• Is, if fact, an example of replication
• Helps with reads distribution, but does not help with writes
• Does not scale beyond single master
Horizontal partitioning (sharding)
• Helps with reads and writes for datasets with good data affinity
• Does not work nicely with joins semantics (i.e., there are no
distributed joins)
16 Max A. Alexejev
17. Key-Value caches
• Memcached and EHCache are good examples to look at
• Keys and values are arbitrary binary (serializable) entities
• Basic operations are put(K,V), get(K), replace(K,V), remove(K)
• May provide group operations like getAll(…) and putAll(…)
• Some operations provide atomicity guarantees (CAS,
inc/dec)
17 Max A. Alexejev
18. Memcached
• Developed for LiveJournal in 2003
• Has client libraries in PHP, Java,
Ruby, Python and many others
• Nodes are independent and don’t
communicate with each other
18 Max A. Alexejev
19. EHCache
• Initially named “Easy Hibernate Cache”
• Java-centric, mature product with open-
source and commercial editions
• Open-source version provides only
replication capabilities, distributed
caching requires commercial license for
both EHCache and Terracotta TSA
19 Max A. Alexejev
20. NoSQL Systems
A whole bunch of different products with both persistent and
non-persistent storage options. Lets call them caches and
storages, accordingly.
Built to provide good horizontal scalability
Try to fill the feature gap between pure KV and full-blown
RDBMS
20 Max A. Alexejev
21. Written in C, supported by
VMWare
Client libraries for C, C#, Java,
Scala, PHP, Erlang, etc
Single-threaded async impl
Has configurable persistence
Case study: Redis
Works with K-V pairs, where K is a
string and V may be either number,
hset users:goku powerlevel 9000 string or Object (JSON)
hget users:goku powerlevel Provides 5 interfaces for: strings,
hashes, sorted lists, sets, sorted
sets
Supports transactions
21 Max A. Alexejev
22. Use cases: Redis
Good for fixed lists, tagging, ratings, counters, analytics and
queues (pub-sub messaging)
Has Master – MultiSlave replication support. Master node is
currently a SPOF.
Distributed Redis was named “Redis Cluster” and is currently
under development
22 Max A. Alexejev
23. • Written in Java, developed in
Facebook.
• Inspired by Amazon Dynamo
replication mechanics, but
uses column-based data
model.
Case study: Cassandra • Good for logs processing,
index storage, voting, jobs
storage etc.
• Bad for transactional
processing.
• Want to know more? Ask
Alexey!
23 Max A. Alexejev
24. In-Memory Data Grids
New generation of caching products, trying to combine benefits of replicated and
distributed schemes.
24 Max A. Alexejev
25. IMDG: Evolution
Data Grids Computational
• Reliable storage and Grids
live data balancing • Reliable jobs
among grid nodes execution, scheduling
and load balancing
Modern
IMDG
25 Max A. Alexejev
26. IMDG: Caching concepts
• Implements KV cache interface • Live data redistribution when nodes are
going up or down – no data loss, no
• Provides indexed search by values clients termination
• Provides reliable distributed locks • Supports RT, WT, WB caching patterns
interface and hierarchical caches (near caching)
• Caching scheme – partitioned or • Supports atomic computations on grid
distributed, may be specified per cache nodes
or cache service
• Provides events subscription for entries
(change notifications)
• Configurable fault tolerance for
distributed schemes (HA)
• Equal data (and read/write load) 26 Max A. Alexejev
distribution among grid nodes
27. IMDG: Under the hood
• All data is split in a number of sections,
called partitions.
• Partition, rather then entry, is an atomic
unit of data migration when grid
rebalances. Number of partitions is fixed
for cluster lifetime.
• Indexes are distributed among grid nodes.
• Clients may or may not be part of the grid
cluster.
27 Max A. Alexejev
28. IMDG Under the hood:
Requests routing
For get() and put() requests:
1. Cluster member, that makes a request, calculates key hash
code.
2. Partition number is calculated using this hash code.
3. Node is identified by partition number.
4. Request is then routed to identified node, executed, and
results are sent back to the client member who initiated
request.
For filter queries:
1. Cluster member initiating requests sends it to all storage
enabled nodes in the cluster.
2. Query is executed on every node using distributed indexes
and partial results are sent to the requesting member.
3. Requesting member merges partial results locally.
4. Final result set is returned from filter method.
28 Max A. Alexejev
29. IMDG: Advanced use-cases
Messaging
Map-Reduce calculations
Cluster-wide singleton
And more…
29 Max A. Alexejev
30. GC tuning for large grid nodes
An easy way to go: rolling restarts or storage-enabled cluster
nodes. Can not be used in any project.
A complex way to go: fine-tune CMS collector to ensure that
it will always keep up cleaning garbage concurrently under
normal production workload.
An expensive way to go: use OffHeap storages provided by
some vendors (Oracle, Terracotta) and use direct memory
buffers available to JVM.
30 Max A. Alexejev
31. IMDG: Market players
Oracle Coherence: commercial, free for evaluation use.
GigaSpaces: commercial.
GridGain: commercial.
Hazelcast: open-source.
Infinispan: open-source.
31 Max A. Alexejev
32. Terracotta
A company behind EHCache, Quartz and Terracotta Server Array.
Acquired by Software AG.
32 Max A. Alexejev
33. Terracotta Server Array
All data is split in a number of sections, called stripes.
Stripes consist of 2 or more Terracotta nodes. One of them is Active node, others have Passive status.
All data is distributed among stripes and replicated inside stripes.
Open Source limitation: only one stripe. Such setup will support HA, but will not distribute cache data. I.e., it is not horizontally scalable.
33 Max A. Alexejev