The document describes how Infinispan was used to improve the performance and scalability of two different systems - an existing production system (Part 1) and a new greenfield project (Part 2).
For the existing system, Infinispan replaced EhCache to distribute caching across multiple nodes and reduce heap usage. Benchmarking was used to validate the solution. Issues encountered with Hotrod were addressed by using embedded caching instead.
For the new project, distributed caching did not work due to high network traffic. Distributed execution was used instead, where tasks are run on nodes containing the data to minimize traffic. This reduced latency from 2 seconds to 200ms.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
Josh Berkus
You've heard that PostgreSQL is the highest-performance transactional open source database, but you're not seeing it on YOUR server. In fact, your PostgreSQL application is kind of poky. What should you do? While doing advanced performance engineering for really high-end systems takes years to learn, you can learn the basics to solve performance issues for 80% of PostgreSQL installations in less than an hour. In this session, you will learn: -- The parts of database application performance -- The performance setup procedure -- Basic troubleshooting tools -- The 13 postgresql.conf settings you need to know -- Where to look for more information.
My experience with embedding PostgreSQLJignesh Shah
At my current company, we embed PostgreSQL based technologies in various applications shipped as shrink-wrapped software. In this session we talk about the experience of embedding PostgreSQL where it is not directly exposed to end-user and the issues encountered on how they were resolved.
We will talk about business reasons,technical architecture of deployments, upgrades, security processes on how to work with embedded PostgreSQL databases.
Josh Berkus
You've heard that PostgreSQL is the highest-performance transactional open source database, but you're not seeing it on YOUR server. In fact, your PostgreSQL application is kind of poky. What should you do? While doing advanced performance engineering for really high-end systems takes years to learn, you can learn the basics to solve performance issues for 80% of PostgreSQL installations in less than an hour. In this session, you will learn: -- The parts of database application performance -- The performance setup procedure -- Basic troubleshooting tools -- The 13 postgresql.conf settings you need to know -- Where to look for more information.
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
Training Slides: Basics 102: Introduction to Tungsten ClusteringContinuent
This 30 minutes training session provides an introduction to how Tungsten Clustering for MySQL / MariaDB / Percona Server works, its basic principles, understanding Tungsten Clustering topologies, failover, rolling maintenance and related tools.
AGENDA
- Review the key benefits offered by Tungsten Clustering
- Examine the Tungsten Clustering architecture
- Tungsten Cluster Topologies for MySQL High Availability and Disaster Recovery
- Composite vs Multi-Site/Multi-Master
- Review automatic and manual failover
- Explore the concepts of a rolling maintenance procedure
- Study key resources to monitor and manage the cluster
Exchange 2013 Haute disponibilité et tolérance aux sinistres (Session 1/2 pre...Microsoft Technet France
Attention, Session en Anglais. Attention Session en 2 parties. Ceci est la première partie. Cette session sera animée par Scott Schnoll, Senior Content Developer chez Microsoft Corp et veritable Gourou Exchange. La messagerie est un élément ultra critique du système d'information : Elle ne DOIT PAS tomber. Pour cela, Exchange 2013 intègre les toutes dernières technologies en terme de tolérance de panne et de haute disponibilité. Scott Schnoll vous expliquera la mécanique de l'intérieur ! Cette session vous donne accès à l'état de l'art sur Exchange. C'est LA session à suivre pour découvrir la mécanique de haute disponibilité d'Exchange 2013.
Speaker : Scott Schnoll (Microsoft)
Exchange Server 2013 : les mécanismes de haute disponibilité et la redondance...Microsoft Technet France
La nouvelle version d'Exchange Server 2013 intègre une foule de nouveautés lui permettant d'être aujourd'hui le serveur de messagerie le plus sécurisé et le plus fiable sur le marché. L'expérience acquise par la gestion des solutions de messagerie Cloud par les équipes Microsoft a été directement intégrée dans cette nouvelle version du produit ce qui va vous permettre la mise en place d'un système de messagerie ultra résilient. Scott Schnoll, Principal Technical Writer dans l'équipe Exchange à Microsoft Corp va vous expliquer de manière didactique l'ensemble des mécanismes de haute disponibilité et les solutions de resilience inter sites dans les plus petits détails. Venez apprendre directement par l'expert qui a travaillé sur ces sujets chez Microsoft ! Attention, session très technique, en anglais.
Learn strategies to maintain your database's high availability even during peak use periods. MariaDB's Field CTO Max Mether offers best practices for high availability, disaster recovery and more.
An introduction into the Garbage First (G1) garbage collector for the JVM. The session covers general GC concepts, the fundamentals of G1 and how to setup and tune the JVM for G1.
Topics covered in this presentation:
1. The difference between traditional (e.g. MySQL) replication and Galera Cluster
2. General Galera Cluster principles
VMworld 2013: Performance and Capacity Management of DRS Clusters VMworld
VMworld 2013
Anne Holler, VMware
Ganesha Shanmuganathan, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Topics covered in this presentation, which was used for user group meetings, conferences & webinars:
1. Galera Cluster for MySQL - overview
2. Release 3 New Features:
* WAN Replication
* 5.6 Global Transaction ID (GTID) Support
* MySQL Replication Support
* and more features
3. The Galera Cluster Project
Abstract:
Reactive applications need to be able to respond to demand, be elastic and ready to scale up, down, in and out—taking full advantage of mobile, multi-core and cloud computing architectures—in real time.
In this talk we will discuss the guiding principles making this possible through the use of share-nothing and non-blocking designs, applied all the way down the stack. We will learn how to deliver systems that provide reactive supply to changing demand.
I gave this talk at React Conf 2014 in London. Recording available here: https://www.youtube.com/watch?v=mBFdj7w4aFA
Deep Dive into RDS PostgreSQL UniverseJignesh Shah
In this session we will deep dive into the exciting features of Amazon RDS for PostgreSQL, including new versions of PostgreSQL releases, new extensions, larger instances. We will also show benchmarks of new RDS instance types, and their value proposition. We will also look at how high availability and read scaling works on RDS PostgreSQL. We will also explore lessons we have learned managing a large fleet of PostgreSQL instances, including important tunables.
This presentation answer a lot of your questions about PostgreSQL and the Red Hat Cluster Suite.
It reviews how you can create failover/standby capabilities with the following activities:
General PostgreSQL clustering options
Overview of Red Hat Cluster Service
Identification of candidate databases for clustering
Identification of hardware for clustering
Analysis of uptime requirements and data latency
Implementation of clustering
Testing of clustering
PostgreSQL installation tips for RHCS
Training Slides: Basics 102: Introduction to Tungsten ClusteringContinuent
This 30 minutes training session provides an introduction to how Tungsten Clustering for MySQL / MariaDB / Percona Server works, its basic principles, understanding Tungsten Clustering topologies, failover, rolling maintenance and related tools.
AGENDA
- Review the key benefits offered by Tungsten Clustering
- Examine the Tungsten Clustering architecture
- Tungsten Cluster Topologies for MySQL High Availability and Disaster Recovery
- Composite vs Multi-Site/Multi-Master
- Review automatic and manual failover
- Explore the concepts of a rolling maintenance procedure
- Study key resources to monitor and manage the cluster
Exchange 2013 Haute disponibilité et tolérance aux sinistres (Session 1/2 pre...Microsoft Technet France
Attention, Session en Anglais. Attention Session en 2 parties. Ceci est la première partie. Cette session sera animée par Scott Schnoll, Senior Content Developer chez Microsoft Corp et veritable Gourou Exchange. La messagerie est un élément ultra critique du système d'information : Elle ne DOIT PAS tomber. Pour cela, Exchange 2013 intègre les toutes dernières technologies en terme de tolérance de panne et de haute disponibilité. Scott Schnoll vous expliquera la mécanique de l'intérieur ! Cette session vous donne accès à l'état de l'art sur Exchange. C'est LA session à suivre pour découvrir la mécanique de haute disponibilité d'Exchange 2013.
Speaker : Scott Schnoll (Microsoft)
Exchange Server 2013 : les mécanismes de haute disponibilité et la redondance...Microsoft Technet France
La nouvelle version d'Exchange Server 2013 intègre une foule de nouveautés lui permettant d'être aujourd'hui le serveur de messagerie le plus sécurisé et le plus fiable sur le marché. L'expérience acquise par la gestion des solutions de messagerie Cloud par les équipes Microsoft a été directement intégrée dans cette nouvelle version du produit ce qui va vous permettre la mise en place d'un système de messagerie ultra résilient. Scott Schnoll, Principal Technical Writer dans l'équipe Exchange à Microsoft Corp va vous expliquer de manière didactique l'ensemble des mécanismes de haute disponibilité et les solutions de resilience inter sites dans les plus petits détails. Venez apprendre directement par l'expert qui a travaillé sur ces sujets chez Microsoft ! Attention, session très technique, en anglais.
Learn strategies to maintain your database's high availability even during peak use periods. MariaDB's Field CTO Max Mether offers best practices for high availability, disaster recovery and more.
An introduction into the Garbage First (G1) garbage collector for the JVM. The session covers general GC concepts, the fundamentals of G1 and how to setup and tune the JVM for G1.
Topics covered in this presentation:
1. The difference between traditional (e.g. MySQL) replication and Galera Cluster
2. General Galera Cluster principles
VMworld 2013: Performance and Capacity Management of DRS Clusters VMworld
VMworld 2013
Anne Holler, VMware
Ganesha Shanmuganathan, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
Topics covered in this presentation, which was used for user group meetings, conferences & webinars:
1. Galera Cluster for MySQL - overview
2. Release 3 New Features:
* WAN Replication
* 5.6 Global Transaction ID (GTID) Support
* MySQL Replication Support
* and more features
3. The Galera Cluster Project
Abstract:
Reactive applications need to be able to respond to demand, be elastic and ready to scale up, down, in and out—taking full advantage of mobile, multi-core and cloud computing architectures—in real time.
In this talk we will discuss the guiding principles making this possible through the use of share-nothing and non-blocking designs, applied all the way down the stack. We will learn how to deliver systems that provide reactive supply to changing demand.
I gave this talk at React Conf 2014 in London. Recording available here: https://www.youtube.com/watch?v=mBFdj7w4aFA
Flink Forward SF 2017: Feng Wang & Zhijiang Wang - Runtime Improvements in Bl...Flink Forward
In 2016, we introduced Alibaba’s compute engine Blink which was based on our private branch of flink. It enalbed many large scale applications in Alibaba’s core business, such as search, recommendation and ads. With the deep and close colaboration with the flink community, we are finally close to contribute our improvements back to the flink community. In this talk, we will present our key contributions to flink runtime recently, such as the new YARN cluster mode for Flip-6, fine-grained failover for Flip-1, async i/o for Flip-12, incremental checkpoint, and the further improvements plan from Alibaba in the near future. Moreover, we will show some production use cases to illustrate how flink works in Alibaba’s large scale online applications, which includes real-time ETL as well as online machine learning. This talk is presented by Alibaba.
Diagnosing Problems in Production - CassandraJon Haddad
This presentation covers diagnosing and solving common problems encountered in production, using performance profiling tools. We’ll also give a crash course to basic JVM garbage collection tuning. Readers will leave with a better understanding of what they should look for when they encounter problems with their in-production Cassandra cluster. This presentation is intended for people with a general understanding of Cassandra, but it not required to have experience running it in production.
'www.NoSQL.com' by Mark Addy
The talk, split into two sections and drawing from a real implementation covers potential NoSQL architectures and the features they offer in the quest to reach the Holy Grail AKA linear scalability. First we examine an existing traditional replicated cache running at full capacity and how replacement with a distributed solution allowed the application to scale dramatically with a performance improvement to boot. Secondly we look at one of the pitfalls of distributed caching and how, using an essential tool from the Data Grid functionality armory, namely grid-execution we can provide massive scale out and low latencies. Both parts describe a real-world implementation delivered for a global on-line travel agency using Infinispan. Level: All levels, the talk is aimed at developers, architects and middleware specialists interested in caching solutions. Focus: Use-cases including technical detail.
London JBUG April 2015 - Performance Tuning Apps with WildFly Application ServerJBUG London
by Jeremy Whiting; London JBoss User Group, 15th April 2015.
Find out more: http://www.c2b2.co.uk/london_jbug_april_2015
London JBoss User Group (London JBUG) is organised and sponsored by C2B2 Consulting - The Leading Independent Middleware Experts.
C2B2 JBoss Support & Consultancy: http://www.c2b2.co.uk/jboss
London JBoss UG: http://www.c2b2.co.uk/jbug
Lightning talk by Navin Surtani (Consultant, C2B2 ) presented at the London JBoss User Group event on the 14th of January 2015.
For more information see Navin's blog post 'Clustering Websockets on WildFly' here: http://blog.c2b2.co.uk/2014/12/clustering-websockets-on-wildfly.html
Presentation delivered by Darran Lofthouse, Principal Software Engineer, Red Hat & Kabir Khan, Principal Software Engineer, Red Hat, during London JBoss User Group event on the 21st of May 2014.
Presentation by Peter Skopek (JBoss by Red Hat) delivered at the London JBoss User Group event on the 30th of April 2014.
Presentation
Introductory talk to PicketLink from Federation through to Identity Management.
What is PicketLink?
PicketLink is an umbrella project for security and identity management for Java Applications. PicketLink is an important project under the security offerings from JBoss.
A Picket Fence is a secure system of pickets joined together via some type of links. Basically, the Pickets by themselves do not offer any security. But when they are brought together by linking them, they provide the necessary security.
This project is that link for other security systems or systems to bring together or join, to finally provide the necessary secure system.
For more information visit http://picketlink.org/
Presentation by Tomaz Cerar (Red Hat), delivered at the London JBoss User Group event on the 12th of February 2014.
Watch the video here: http://www.youtube.com/watch?v=eu9K5NLUKBI
Join London JBUG: http://www.c2b2.co.uk/jbug
Presentation delivered by Mircea Markus (JBoss) at the London JBoss User Group Event on Wednesday, the 4th of December 2013.
In this talk, Mircea Markus (Infinispan Lead) covers the major features added in the 6.0 release to the Infinispan ecosystem:
- querying in client/server mode
- better integration with the persistent storage
- multi-master cross-site replication
- support for heterogeneous clusters
Participants will take home a better understanding of Infinispan capabilities and the use cases in which it can be put at work. Ideally the attendees should have a basic knowledge of the in-memory data grids.
Compensating Transactions: When ACID is too muchJBUG London
The talk was presented by Dr Paul Robinson (Red Hat) at the London JBoss User Group event on the 25th of September 2013.
Video available soon
ACID transactions are a useful tool for application developers and can provide very strong guarantees, even in the presence of failures. However, ACID transactions are not always appropriate for every situation.
ACID transactions are achieved by holding locks on resources and require the ability to rollback changes. In some situations, the blocking nature of the protocol can be too limiting to performance, especially if the transaction is distributed. Also, some actions can’t simply be rolled back; for example, sending an email.
A common strategy for applications that can’t use ACID, is to throw out transactions altogether. However, with this approach you are missing out on many of the benefits that transactions can provide. There are alternative transaction models that relax some of the ACID properties, while still retaining many of the strong guarantees essential for building robust enterprise applications. These should be considered before deciding not to use transactions at all.
In this talk, I’ll present one such alternative to ACID transactions: compensation-based transactions. I’ll provide an overview of ACID and it’s limitations and describe some use-cases where ACID is not appropriate. I’ll then present our new API for compensation-based transactions and demonstrate how it can be used to address these problems. Finally, I’ll present our future plans to improve and expand our support for compensation-based transactions.
London JBUG - Connecting Applications Everywhere with JBoss A-MQJBUG London
Presentation by Rob Davies (Red Hat) delivered at the London JBoss USer Group event on the 10th of July 2013.
See the video here: http://youtu.be/VrJyKLAXVTg
As part of the FuseSource acquisition, Red Hat now supports Apache ActiveMQ as the recently released Red Hat JBoss A-MQ product.
ActiveMQ is the most widely used message-oriented middleware that uses messaging to connect remote applications written in Java, C/C++, Python, Perl, Ruby, and many other languages. ActiveMQ is standards based and supports messaging protocols such as AMQP 1.0, WebSockets, Stomp, OpenWire, and MQTT.
In this session, Rob Davies will discuss the product’s features and functionality, and demonstrate connectivity from a microprocessor to web sockets using JBoss A-MQ.
Easy Integration with Apache Camel and Fuse IDEJBUG London
The talk presented by James Strachan (Red Hat) on the 6th of March 2013 at the London JBoss USer Group event.
Visit London JBUG on Meetup http://www.meetup.com/JBoss-User-Group/
Talk by Mauricio Salatino and Esteban Aliverti presented at the JBUG London Event on the 12th of December 2012.
To watch the video please visit http://www.youtube.com/watch?v=9M1eaH3v4qg
Arquillian - Integration Testing Made EasyJBUG London
Slides by Davide D'Alto presented at JBUG London event on the 24th of October 2012.
You can watch the presentation video here http://youtu.be/MuqAx9SuIOk
Presented by Matt Brasier on 15th of June 2011 at the JBUG is Back! June Event - JBoss 7
More info: http://www.meetup.com/JBoss-User-Group/events/19943531/
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
3. Who am I?
Mark Addy, Senior Consultant
Fast, Reliable, Secure, Manageable
4. Agenda
Part 1
• An existing production system unable to scale
Part 2
• A green-field project unable to meet SLA’s
5. About the Customer
• Global on-line travel & accommodation
provider
– 50 million searches per day
• Our relationship
– Troubleshooting
– Workshops
6. Part 1
Part 1 – Existing Application
Connectivity Engine
• Supplements site content with data from
third parties (Content Providers)
– Tomcat
– Spring
– EhCache
– MySQL
– Apache load-balancer / mod_jk
8. Part 1
Content Provider Challenges
• Unreliable third party systems
• Distant network communications
• Critical for generating local site content
• Response time
• Choice & low response time == more
profit
9. Part 1
Existing Cache
• NOT Hibernate 2LC
• Spring Interceptors wrap calls to content providers
<bean id="searchService" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="proxyInterfaces" value=“ISearchServiceTargetBean"/>
<property name="target" ref="searchServiceTargetBean"/>
<property name="interceptorNames">
<list>
<value>cacheInterceptor</value>
</list>
</property>
</bean>
<bean id="searchServiceTargetBean“ class=“SearchServiceTargetBean">
...
</bean>
10. Part 1
Extreme Redundancy
800,000 elements
10 nodes = 10 copies of data
11. Part 1
The Price
• 10G JVM Heap
– 10-12 second pauses for major GC
– Over 8G of heap is cache
• Eviction before Expiry
– More trips to content providers
• EhCache expiry / eviction piggybacks
client cache access
13. Part 1
Objectives
• Reduce JVM Heap Size
– 10 second pauses are too long
• Increase cache capacity
• Remove Eviction
– Cache entries should expire naturally
• Improve Response Times
– Latency decreases if eviction, GC pauses
and frequency are reduced
15. Part 1
Why Infinispan?
• Open source advocates
• Cutting edge technology
16. Part 1
Benchmarking
• Must be reproducible
• Must reflect accurately the production
load and data
– 50 million searches / day == 600 / sec
• Must be able to imitate the content
providers
17. Part 1
Solution
• Replica load-test environment
• Port mirror production traffic
– Capture incoming requests
– Capture content provider responses
• Custom JMeter script
• Mock application Spring Beans
25. Part 1
Distributed Mode
hash(key) determines owners
26. Part 1
Distribution Features
• Configurable redundancy
– numOwners
• Dynamic scaling
– Automatic rebalancing for distribution and
recovery of redundancy
• Replication (distribution) overhead does
not increase as more nodes are added
27. Part 1
Hotrod
• Client – Server architecture
– Java client
– Connection pooling
– Dynamic scaling
– Smart routing
• Separate application and cache
memory requirements
28. Part 1
Application – Cache Separation
Application
• CPU intensive
• High infant mortality
Cache
• Low CPU requirement
• Mortality linked to
expiry / eviction
30. Part 1
Remember this is cutting edge
• Latest final release was 4.2.1
• Lets get cracking...
– Distributed mode
– Hotrod client
– What issues did we encounter...
31. Part 1
Topology Aware Consistent Hash
• Ensure back-ups are held preferentially on
separate machine, rack and site
• https://community.jboss.org/thread/168236
• We need to upgrade to the latest 5.0.0.CR
33. Part 1
Virtual Nodes
• Improves data distribution
• But didn’t work at the time for Hotrod
• https://issues.jboss.org/browse/ISPN-1217
34. Part 1
Hotrod Concurrent Start-up
• Dynamic scaling
– Replicated ___hotRodTopologyCache holds current cluster topology
– New starters must lock and update this cache to add themselves to
the current view
– Deadlock!
– https://issues.jboss.org/browse/ISPN-1182
• Stagger start-up
35. Part 1
Hotrod Client Failure Detection
Unable to recover from cluster splits
36. Part 1
Hotrod Client Failure Detection
• New servers only added to
___hotRodTopologyCache on start-up
• Restart required to re-establish client topology view
38. Part 1
Hotrod Server Cluster Meltdown
• Clients can’t start without an available
server
• Static Configuration is only read once
• To restart client-server communications
either
– Restart last “known” server
– Restart the client
39. Part 1
Change of tack
• Hotrod abandoned, for now
– Data distribution
– Concurrent start up
– Failure detection
– Unacceptable for this customer
• Enter the classic embedded approach
• How did we get this to work...
40. Part 1
Dynamic Scaling
• Unpredictable under heavy load, writers blocked
– Unacceptable waits for this system
<hash numOwners=“2” rehashEnabled=“false” />
– Accept some data loss during a leave / join
• Chunked rehashing / state transfer (5.1)
– https://issues.jboss.org/browse/ISPN-284
• Non-blocking state transfer
– https://issues.jboss.org/browse/ISPN-1424
• Manual rehashing
– https://issues.jboss.org/browse/ISPN-1394
41. Part 1
Cache Entry Size
• Average cache entry ~6K
– 1 million entries = 6GB
– Hotrod stores serialized entries by default
• JBoss Marshalling
– Default Infinispan mechanism
– Get reference from ComponentRegistry
• JBoss Serialization
– Quick, easy to implement
42. Part 1
Compression Considerations
• Trade
– Capacity in JVM vs Serialization Overhead
• Suitability
– Assess on a cache by cache basis
– Very high access is probably too expensive
• Average 6K reduced to 1K
43. Part 1
Advanced Cache Tuning
cache.getAdvancedCache.withFlags(Flag... flags)
• Flag.SKIP_REMOTE_LOOKUP
– Prevents remote gets being run for an update
put(K key, V value)
DistributionInterceptor.remoteGetBeforeWrite()
DistributionInterceptor.handleWriteCommand()
DistributionInterceptor.visitPutKeyValueCommand()
– We don’t need to return the previous cache entry
value
44. Part 1
JGroups
• UDP out-performed TCP (for us)
• Discovery
– For a cold, full cluster start-up avoid split
brain / merge scenarios
<PING timeout="3000" num_initial_members="10"/>
• Heartbeat
– Ensure failure detection is configured
appropriately
<FD_ALL interval="3000" timeout="10000"/>
46. Part 1
Current Production System
• Over 20 nodes
– 8 Request facing, remainder storage only
• Over 15 million entries
– 7.5 million unique
– 20GB cached data
– Nothing is evicted before natural expiration
• 5GB JVM Heap, 3-4 second GC pauses
• 30% reduction in response times
47. Part 1
Summary
• Don’t compromise on the benchmarking
– Understand your cached data profile
– Functional testing is NOT sufficient
– Monitoring and Analysis is essential
• Tune Virtual Nodes for best distribution
• Mitigate memory usage of embedded cache
– Consider compressing embedded cache entries
– Non request facing storage nodes
• Distributed Infinispan out performs EhCache
• Don’t rule Hotrod out
– Not acceptable for this customer
– Many improvements and bug fixes
48. Part 2
Part 2 – Green Field SLA’s
New Pricing Engine
– Tomcat
– Spring & Grails
– Infinispan
– Oracle RAC
– Apache load-balancer / mod_jk
Historical Pricing Engine
– EhCache
– MySQL
– 2 second full Paris Query
49. Part 2
Logical View
• New Pricing Engine
– Side by side rollout
– Controller determines
where to send requests and
aggregates results
– NOT Hibernate 2LC
– Spring Interceptors
containing logic to check /
update cache wrap calls to
DB that extract and
generate cache entries
50. Part 2
Proposed Caching
• Everything distributed
– It worked before so we just turn in on, right?
51. Part 2
The Pain
• Distributed Mode
– Network saturation on 1Gb switch
(125MB/second) under load
– Contention in org.jgroups
• Performance SLA’s
– Caching data in Local mode required 14G heap &
20 second GC pauses
• Aggressive rollout strategy
– Struggling at low user load
52. Part 2
Cache Analysis
• Eclipse Memory Analyzer
– Identify cache profile
– Small subset of elements account for
almost all the space
– Paris “Rates” sizes 20K – 1.6MB
– Paris search (500 rates records) ==
50MB total
– 1Gb switch max throughput =
125MB/second
53. Part 2
Revised Caching
• Local caching for numerous “small” elements
• Distributed for “large” expensive elements
54. Part 2
Distributed Issue
• Here’s why normal distributed doesn’t work
– One Paris request requires 500 rates records (50MB)
– 10 nodes distributed cluster = 1 in 5 chance data is local
– 80% remote Gets == 40MB network traffic
55. Part 2
Options
• Rewrite the application caching logic
– Significantly reduce the element size
• Run Local caching with oversized heap
– Daily restart, eliminate full GC pauses
– Large memory investment and careful management
• Sacrifice caching and hit the DB
– Hits response times and hammer the database
• Distributed Execution?
– Send a task to the data and extract just what you need
56. Part 2
Change in Psychology...
If the mountain will not come to
Muhammad, then Muhammad must go
to the mountain
57. Part 2
Distributed Execution
• DefaultExecutorService
– http://docs.jboss.org/infinispan/5.1/apidocs/org/infinispan/distexec
/DefaultExecutorService.html
• Create the Distributed Execution Service to run on the cache
node specified
public DefaultExecutorService(Cache masterCacheNode)
• Run task on primary owner of Key input
public Future<T> submit(Callable<T> task, K... input)
– Resolve primary owner of Key then either
• Run locally
• Issue a remote command and run on the owning node
58. Part 2
Pricing Controller
• Callable task
– Contains code to
• Grab reference to local Spring Context
• Load required beans
• Spring interceptor checks cache at the owning
Existing node (local get)
Code • If not found then goto database, retrieve and
update cache
• Extract pricing based on request criteria
• Return results
59. Part 2
Pricing Controller
• Create a new DefaultExecutorService
– Create callable tasks required to satisfy request
– Issue callable tasks concurrently
while (moreKeys) {
Callable<T> callable = new MyCallable<T>(...);
Future<T> future = distributedExecutorService.submit(callable, key);
...
}
– Collate results and assemble response
while (moreFutures) {
T result = future.get();
}
60. Part 2
Distributed Execution
• Only the relevant information from the cache
entry is returned
61. Part 2
Results
• Latency – Paris search
– Historic Engine 2 seconds
– Dist-Exec 200ms
• JVM
– 5GB Heap
– 3-4 second pauses
62. Part 2
Limitations
• Failover
– Task sent to primary owner only
– https://community.jboss.org/wiki/Infinispan60-
DistributedExecutionEnhancements
– Handle failures yourself
• Hotrod not supported
– This would be fantastic!
– https://issues.jboss.org/browse/ISPN-1094
• Both in 6.0?
63. Part 2
Summary
• Analysis and re-design of cached data
• Accessing large data sets requires an
alternative access pattern
• Dramatically reduced latency
– Parallel execution
– Fraction of data transferred across the wire
• Execution failures must be handled by
application code, at the moment...