Many businesses are faced with some new messaging challenges for modern applications, such as horizontal scalability of the messaging tier, heterogeneous messaging systems and access methods, and extreme transaction processing. This presentation/demo will cover how businesses can overcome these messaging challenges with the use of Spring and RabbitMQ technologies. Tom will build a case for AMQP, explain how SpringSource is providing AMQP support via Spring AMQP and Spring Integration, explain how RabbitMQ is a modern messaging solution that offers a reliable, highly available, scalable and portable messaging system with predictable and consistent throughput and latency, and demonstrate how Spring Integration and RabbitMQ can be progressively introduced into a standard Spring web application deployed on Cloud Foundry.
The document discusses messaging for modern applications. It describes how modern applications are trending towards being data intensive, deployed on virtual infrastructure, and written with frameworks. This is putting pressure on traditional middleware and databases that are not keeping pace. The document advocates for using modern messaging approaches like asynchronous messaging to make applications more scalable and resilient. It introduces Spring Integration as a way to incorporate messaging into Spring applications and describes common messaging patterns it supports like events, fragments, routing, batching, publishing and modularity. The document also provides an overview of the Advanced Message Queuing Protocol (AMQP) and why it is useful as an open, ubiquitous protocol for reliable asynchronous messaging.
IBM MQ CONNAUTH/CHLAUTH Doesn't Work Like You Think it Does (and if you aren'...T.Rob Wyatt
This document summarizes the findings from research into security behaviors when using IBM MQ's password authentication (CONNAUTH) feature. It identifies five distinct behaviors exhibited by the interaction between CONNAUTH and CHLAUTH access control rules. The document provides recommendations for mandatory and avoid configurations when using CONNAUTH. It also warns that applying fix packs can cause failures or silently over-authorize users. The summary concludes by thanking two people for their contributions to testing and improving the tools and research presented.
HHM 6887 Managing Your Scalable Applications in an MQ Hybrid Cloud Worldmatthew1001
This document provides an overview of IBM MQ sessions at the InterConnect 2017 conference, including:
- A schedule of MQ sessions occurring each day of the conference across topics like the MQ Appliance, hybrid cloud, and security.
- A brief description of each session including titles and speakers.
- A notice that the information provided is subject to change and limitations on warranties and liability.
This chapter describes how to configure dynamic VLAN membership using VMPS. It discusses how the VMPS works by assigning ports to VLANs based on the connected device's MAC address. When a host moves to another port, the new port is dynamically assigned to the proper VLAN. The document provides guidelines for configuring VMPS and dynamic port VLAN membership. It also describes entering the IP address of the VMPS and default VMPS configuration.
This document describes how to configure VTP and VLANs on Cisco switches. It discusses:
- How VTP works to synchronize VLAN configurations across switches in a VTP domain.
- Configuring a switch as a VTP server to propagate VLAN changes to other switches, or as a VTP client which receives but does not change VLAN configurations.
- Default VTP settings and guidelines for implementing VTP in a network.
Interconnect 2017: 6893 Keep out the bad guys by securing your MQ messaging e...Robert Parker
Presentation delivered at Interconnect 2017. Session ID 6893: Keep out the bad guys by securing your MQ messaging environment.
With today's focus on security, ensuring you utilize all of the options available to maximize your systems security is a high priority for many businesses. In this session, we will work through a step-by-step case study that details how you can enhance the security of your Queue Managers using the different features available in IBM MQ.
The document discusses messaging for modern applications. It describes how modern applications are trending towards being data intensive, deployed on virtual infrastructure, and written with frameworks. This is putting pressure on traditional middleware and databases that are not keeping pace. The document advocates for using modern messaging approaches like asynchronous messaging to make applications more scalable and resilient. It introduces Spring Integration as a way to incorporate messaging into Spring applications and describes common messaging patterns it supports like events, fragments, routing, batching, publishing and modularity. The document also provides an overview of the Advanced Message Queuing Protocol (AMQP) and why it is useful as an open, ubiquitous protocol for reliable asynchronous messaging.
IBM MQ CONNAUTH/CHLAUTH Doesn't Work Like You Think it Does (and if you aren'...T.Rob Wyatt
This document summarizes the findings from research into security behaviors when using IBM MQ's password authentication (CONNAUTH) feature. It identifies five distinct behaviors exhibited by the interaction between CONNAUTH and CHLAUTH access control rules. The document provides recommendations for mandatory and avoid configurations when using CONNAUTH. It also warns that applying fix packs can cause failures or silently over-authorize users. The summary concludes by thanking two people for their contributions to testing and improving the tools and research presented.
HHM 6887 Managing Your Scalable Applications in an MQ Hybrid Cloud Worldmatthew1001
This document provides an overview of IBM MQ sessions at the InterConnect 2017 conference, including:
- A schedule of MQ sessions occurring each day of the conference across topics like the MQ Appliance, hybrid cloud, and security.
- A brief description of each session including titles and speakers.
- A notice that the information provided is subject to change and limitations on warranties and liability.
This chapter describes how to configure dynamic VLAN membership using VMPS. It discusses how the VMPS works by assigning ports to VLANs based on the connected device's MAC address. When a host moves to another port, the new port is dynamically assigned to the proper VLAN. The document provides guidelines for configuring VMPS and dynamic port VLAN membership. It also describes entering the IP address of the VMPS and default VMPS configuration.
This document describes how to configure VTP and VLANs on Cisco switches. It discusses:
- How VTP works to synchronize VLAN configurations across switches in a VTP domain.
- Configuring a switch as a VTP server to propagate VLAN changes to other switches, or as a VTP client which receives but does not change VLAN configurations.
- Default VTP settings and guidelines for implementing VTP in a network.
Interconnect 2017: 6893 Keep out the bad guys by securing your MQ messaging e...Robert Parker
Presentation delivered at Interconnect 2017. Session ID 6893: Keep out the bad guys by securing your MQ messaging environment.
With today's focus on security, ensuring you utilize all of the options available to maximize your systems security is a high priority for many businesses. In this session, we will work through a step-by-step case study that details how you can enhance the security of your Queue Managers using the different features available in IBM MQ.
IBM MQ V8 introduced a number of new security features. This session will take you through the two major features, Multiple Certificates and Connection Authentication. In IBM MQ V8 you are no longer restricted to only using one certificate for you queue manager with an IBM enforced label. Now you can have your own certificate labels and can allocated a different certificate for any specific channel. How about authentication? Finding that digital certificates are more security than your need? Want some authentication without having to write a security exit. IBM MQ V8 gives you built-in user ID and password validation. Other security features related to the MQ CHLAUTH rules are covered in a separate session
Slides presented to OpenStack developer summit during the "Quantum Overview" session (note: these are not the slides presented during the conference, these slides are more technical, and less polished)
This document discusses networking virtual machines in VMware environments. It covers virtual switch connections, port group policies, and networking IP storage using iSCSI and NAS. Specifically, it describes how to configure virtual switches, define different connection types (service console ports, VMkernel ports, virtual machine port groups), apply network policies like VLANs and security, and configure IP storage access through iSCSI software initiators and NFS.
HHM-2833: Where is My Message?: Using IBM MQ Tools to Work Out What Applicati...Matt Leming
Every MQ infrastructure team member has been asked this question, and most developers who have worked with MQ have asked it:
"Where is my message?" In this session, we look into the tools that MQ provides to find your messages.
We demonstrate how to analyze the MQ recovery log on distributed platforms to find out what happened to your persistent messages,
with the assistance of a new tool. We also look at how to trace the route messages take through your MQ infrastructure, and how to generate
and analyze activity reports showing the behavior of MQ applications.
As more businesses explore the benefits of cloud computing, network managers will have to increasingly meet the challenges of redesigning their networks for the cloud. This talk will introduce the basic concepts of Open vSwitch & Openflow and show how these technologies can help satisfy these needs. We will also explain how Open vSwitch fits into XCP and XenServer and routing of dataflows.
First part of talk discussing the networking challenges that cloud implementers face.
- Networking challeges
- Data isolation
Introducing Open vSwitch
- What it is, and its features
- Why it is important in a virtualized environment
Openflow
- Basics of Openflow
- How flows are routed in XenServer & XCP
Where is My Message?: Use MQ Tools to Work Out What Applications Have DoneMorag Hughson
Every MQ infrastructure team member has been asked this question, and most developers who have worked with MQ have asked it: "Where is my message?" In this session, we look into the tools that MQ provides to find your messages. We show how to analyze the MQ recovery log on distributed platforms to find out what happened to your persistent messages, with the assistance of a new tool. We also look at how to trace the route messages take through your MQ infrastructure, and how to generate and analyze activity reports showing the behavior of MQ applications.
Presentation from Robin Wiley.
DataPower can connect to MQ as a client, sending and receiving messages to and from queues. Unlike traditional MQ client programming, the DataPower client interface has certain predefined functions that limit the scope of activities that can be performed. This session will explain what you can and can't do when integrating DataPower into your MQ infrastructure. Topics covered will include syncpoint processing, message grouping, message properties, JMS considerations, connec-tion options, and many more.
Toward a practical “HPC Cloud”: Performance tuning of a virtualized HPC clusterRyousei Takano
1) Performance tuning methods for HPC Cloud include PCI passthrough, NUMA affinity, and reducing VMM noise to improve performance and close the gap with bare metal machines.
2) Evaluation of MPI and HPC applications on a 16-node cluster showed PCI passthrough improved MPI bandwidth close to bare metal, and NUMA affinity improved performance up to 2%.
3) Parallel efficiency of coarse-grained applications was comparable to bare metal, but fine-grained applications saw up to 22% degradation due to communication overhead and virtualization.
This document discusses virtualization technologies including server virtualization using Hyper-V, desktop virtualization, application virtualization, and presentation virtualization. It covers key features of Hyper-V like live migration, failover clustering, thin provisioning, and improvements in Windows Server 2008 R2. Management techniques for virtualized environments are also addressed.
Secure Messages with IBM WebSphere MQ Advanced Message SecurityMorag Hughson
In some scenarios, securing access to a messaging infrastructure is not enough - teams must also secure access to message content. Come to this session to learn how to provide end-to-end message protection where message contents are secure from the point they are sent to the point they are received, including while at rest on queues. This session starts by describing the theory and capabilities of the product. Then CSX provides a real-world customer example in which it presents its experiences and recommendations for securing messages across distributed and z/OS platforms. Topics covered include an overview of message level security, when it is appropriate to deploy this level of protection, how the message protection is applied, how it can be administered, and the new features available in the latest version of IBM WebSphere MQ.
This document discusses addressing messaging challenges using Spring and RabbitMQ. It begins by describing modern application trends towards asynchronous and distributed architectures. It then discusses how to modularize applications with Spring Integration and distribute messaging using AMQP (Advanced Message Queuing Protocol) and RabbitMQ. Finally, it demonstrates distributing the Spring Integration Cafe sample application using AMQP.
Am 04 track1--salvatore orlando--openstack-apac-2012-finalOpenCity Community
1) Quantum provides network services for OpenStack and allows tenants to control virtual networking in their cloud.
2) Quantum uses a plugin architecture that supports different networking technologies through plugins like Nicira NVP.
3) The Nicira NVP plugin uses network virtualization and overlay tunneling to decouple logical and physical network topology and provide flexible network designs.
Monitoring and problem determination of your mq distributed systemsmatthew1001
The document discusses various tools for monitoring MQ distributed systems over time, including accounting and statistics messages, and system topics. Accounting and statistics messages provide overlapping information and are written to dedicated queues when enabled. System topics do not require enabling and provide more granular data on MQ resources as well as broader system metrics, making them better for multiple concurrent consumers.
Software Defined Networking is seeing a lot of momentum these days. With server virtualization solving the virtual machines problem, and large scale object storage solving the distributed storage challenge, SDN is seen as key in virtual networking.
In this talk we don't try to define SDN but rather dive straight into what in our opinion is the core enabled of SDN: the virtual switch OVS.
OVS can help manage VLAN for guest network isolation, it can re-route any traffic at L2-L4 by keeping forwarding tables controlled by a remote controller (Openfow controller). We show these few OVS capabilities and highlight how they are used in CloudStack and Xen.
Xen Summit presentation of CloudStack and Software Defined Networks. OpenVswitch is the default bridge in Xen and supported in XenServer and Xen Cloud Platform
IBM MQ (formerly known as MQSeries) is a middleware messaging product that allows applications on different platforms to communicate asynchronously by sending and receiving messages. It guarantees message delivery and supports advanced features like triggering actions on message receipt. MQ provides a common API for applications to connect to message queues, publish/consume messages, and ensures delivery across heterogeneous systems. It is widely used to integrate legacy mainframe systems with modern platforms.
Keeping track of your messaging infrastructure is essential to be able to rapidly respond to your changing business demands whilst complying with your constantly changing audit demands. This session will provide illustrations and insight on the techniques and capabilities available in IBM MQ to help you meet these monitoring and auditing demands.
Quantum for Cloud Operators - Folsom Conference Dan Wendlandt
Quantum is a networking service for OpenStack that provides tenant control over virtual networks. It uses a plugin architecture that supports different networking technologies and allows for advanced network services like routers and firewalls. The current project status is that Quantum provides basic L2 networking capabilities and some initial advanced services. Future priorities include enabling full tenant control of networks and expanding support for advanced network services.
Building scalable flexible messaging systems using qpidJack Gibson
PayPal had built a messaging system using Qpid to handle high volumes of payments asynchronously at near real-time latency. They evolved their system to include distinct tiers of brokers partitioned by function and user type to improve scalability. Key lessons included optimizing for message size over latency, externalizing addressing, and using Qpid's management framework to monitor broker health and performance.
As more OpenStack clouds move into production, the limits of scale and performance of the cloud need to be known as a pre-requisite to building a predictable operations plan. PLUMgrid ONS is based on a fully distributed architecture that is built for scale. Since forwarding decisions are distributed and made at each individual server, every new server added to the cloud increases the cloud’s forwarding capacity. This unique distributed architecture allows any OpenStack cloud built using the PLUMgrid Open Networking Suite to scale to tens of thousands of workloads across multiple racks. This joint PLUMgrid and Ixia session between will highlight the latest scale and performance numbers for PLUMgrid ONS. In addition, it will cover the various scale targets that were achieved, the testing methodology plus the Ixia IxChariot product used to measure them.
IBM MQ V8 introduced a number of new security features. This session will take you through the two major features, Multiple Certificates and Connection Authentication. In IBM MQ V8 you are no longer restricted to only using one certificate for you queue manager with an IBM enforced label. Now you can have your own certificate labels and can allocated a different certificate for any specific channel. How about authentication? Finding that digital certificates are more security than your need? Want some authentication without having to write a security exit. IBM MQ V8 gives you built-in user ID and password validation. Other security features related to the MQ CHLAUTH rules are covered in a separate session
Slides presented to OpenStack developer summit during the "Quantum Overview" session (note: these are not the slides presented during the conference, these slides are more technical, and less polished)
This document discusses networking virtual machines in VMware environments. It covers virtual switch connections, port group policies, and networking IP storage using iSCSI and NAS. Specifically, it describes how to configure virtual switches, define different connection types (service console ports, VMkernel ports, virtual machine port groups), apply network policies like VLANs and security, and configure IP storage access through iSCSI software initiators and NFS.
HHM-2833: Where is My Message?: Using IBM MQ Tools to Work Out What Applicati...Matt Leming
Every MQ infrastructure team member has been asked this question, and most developers who have worked with MQ have asked it:
"Where is my message?" In this session, we look into the tools that MQ provides to find your messages.
We demonstrate how to analyze the MQ recovery log on distributed platforms to find out what happened to your persistent messages,
with the assistance of a new tool. We also look at how to trace the route messages take through your MQ infrastructure, and how to generate
and analyze activity reports showing the behavior of MQ applications.
As more businesses explore the benefits of cloud computing, network managers will have to increasingly meet the challenges of redesigning their networks for the cloud. This talk will introduce the basic concepts of Open vSwitch & Openflow and show how these technologies can help satisfy these needs. We will also explain how Open vSwitch fits into XCP and XenServer and routing of dataflows.
First part of talk discussing the networking challenges that cloud implementers face.
- Networking challeges
- Data isolation
Introducing Open vSwitch
- What it is, and its features
- Why it is important in a virtualized environment
Openflow
- Basics of Openflow
- How flows are routed in XenServer & XCP
Where is My Message?: Use MQ Tools to Work Out What Applications Have DoneMorag Hughson
Every MQ infrastructure team member has been asked this question, and most developers who have worked with MQ have asked it: "Where is my message?" In this session, we look into the tools that MQ provides to find your messages. We show how to analyze the MQ recovery log on distributed platforms to find out what happened to your persistent messages, with the assistance of a new tool. We also look at how to trace the route messages take through your MQ infrastructure, and how to generate and analyze activity reports showing the behavior of MQ applications.
Presentation from Robin Wiley.
DataPower can connect to MQ as a client, sending and receiving messages to and from queues. Unlike traditional MQ client programming, the DataPower client interface has certain predefined functions that limit the scope of activities that can be performed. This session will explain what you can and can't do when integrating DataPower into your MQ infrastructure. Topics covered will include syncpoint processing, message grouping, message properties, JMS considerations, connec-tion options, and many more.
Toward a practical “HPC Cloud”: Performance tuning of a virtualized HPC clusterRyousei Takano
1) Performance tuning methods for HPC Cloud include PCI passthrough, NUMA affinity, and reducing VMM noise to improve performance and close the gap with bare metal machines.
2) Evaluation of MPI and HPC applications on a 16-node cluster showed PCI passthrough improved MPI bandwidth close to bare metal, and NUMA affinity improved performance up to 2%.
3) Parallel efficiency of coarse-grained applications was comparable to bare metal, but fine-grained applications saw up to 22% degradation due to communication overhead and virtualization.
This document discusses virtualization technologies including server virtualization using Hyper-V, desktop virtualization, application virtualization, and presentation virtualization. It covers key features of Hyper-V like live migration, failover clustering, thin provisioning, and improvements in Windows Server 2008 R2. Management techniques for virtualized environments are also addressed.
Secure Messages with IBM WebSphere MQ Advanced Message SecurityMorag Hughson
In some scenarios, securing access to a messaging infrastructure is not enough - teams must also secure access to message content. Come to this session to learn how to provide end-to-end message protection where message contents are secure from the point they are sent to the point they are received, including while at rest on queues. This session starts by describing the theory and capabilities of the product. Then CSX provides a real-world customer example in which it presents its experiences and recommendations for securing messages across distributed and z/OS platforms. Topics covered include an overview of message level security, when it is appropriate to deploy this level of protection, how the message protection is applied, how it can be administered, and the new features available in the latest version of IBM WebSphere MQ.
This document discusses addressing messaging challenges using Spring and RabbitMQ. It begins by describing modern application trends towards asynchronous and distributed architectures. It then discusses how to modularize applications with Spring Integration and distribute messaging using AMQP (Advanced Message Queuing Protocol) and RabbitMQ. Finally, it demonstrates distributing the Spring Integration Cafe sample application using AMQP.
Am 04 track1--salvatore orlando--openstack-apac-2012-finalOpenCity Community
1) Quantum provides network services for OpenStack and allows tenants to control virtual networking in their cloud.
2) Quantum uses a plugin architecture that supports different networking technologies through plugins like Nicira NVP.
3) The Nicira NVP plugin uses network virtualization and overlay tunneling to decouple logical and physical network topology and provide flexible network designs.
Monitoring and problem determination of your mq distributed systemsmatthew1001
The document discusses various tools for monitoring MQ distributed systems over time, including accounting and statistics messages, and system topics. Accounting and statistics messages provide overlapping information and are written to dedicated queues when enabled. System topics do not require enabling and provide more granular data on MQ resources as well as broader system metrics, making them better for multiple concurrent consumers.
Software Defined Networking is seeing a lot of momentum these days. With server virtualization solving the virtual machines problem, and large scale object storage solving the distributed storage challenge, SDN is seen as key in virtual networking.
In this talk we don't try to define SDN but rather dive straight into what in our opinion is the core enabled of SDN: the virtual switch OVS.
OVS can help manage VLAN for guest network isolation, it can re-route any traffic at L2-L4 by keeping forwarding tables controlled by a remote controller (Openfow controller). We show these few OVS capabilities and highlight how they are used in CloudStack and Xen.
Xen Summit presentation of CloudStack and Software Defined Networks. OpenVswitch is the default bridge in Xen and supported in XenServer and Xen Cloud Platform
IBM MQ (formerly known as MQSeries) is a middleware messaging product that allows applications on different platforms to communicate asynchronously by sending and receiving messages. It guarantees message delivery and supports advanced features like triggering actions on message receipt. MQ provides a common API for applications to connect to message queues, publish/consume messages, and ensures delivery across heterogeneous systems. It is widely used to integrate legacy mainframe systems with modern platforms.
Keeping track of your messaging infrastructure is essential to be able to rapidly respond to your changing business demands whilst complying with your constantly changing audit demands. This session will provide illustrations and insight on the techniques and capabilities available in IBM MQ to help you meet these monitoring and auditing demands.
Quantum for Cloud Operators - Folsom Conference Dan Wendlandt
Quantum is a networking service for OpenStack that provides tenant control over virtual networks. It uses a plugin architecture that supports different networking technologies and allows for advanced network services like routers and firewalls. The current project status is that Quantum provides basic L2 networking capabilities and some initial advanced services. Future priorities include enabling full tenant control of networks and expanding support for advanced network services.
Building scalable flexible messaging systems using qpidJack Gibson
PayPal had built a messaging system using Qpid to handle high volumes of payments asynchronously at near real-time latency. They evolved their system to include distinct tiers of brokers partitioned by function and user type to improve scalability. Key lessons included optimizing for message size over latency, externalizing addressing, and using Qpid's management framework to monitor broker health and performance.
As more OpenStack clouds move into production, the limits of scale and performance of the cloud need to be known as a pre-requisite to building a predictable operations plan. PLUMgrid ONS is based on a fully distributed architecture that is built for scale. Since forwarding decisions are distributed and made at each individual server, every new server added to the cloud increases the cloud’s forwarding capacity. This unique distributed architecture allows any OpenStack cloud built using the PLUMgrid Open Networking Suite to scale to tens of thousands of workloads across multiple racks. This joint PLUMgrid and Ixia session between will highlight the latest scale and performance numbers for PLUMgrid ONS. In addition, it will cover the various scale targets that were achieved, the testing methodology plus the Ixia IxChariot product used to measure them.
The Future of Messaging: RabbitMQ and AMQP Eberhard Wolff
RabbitMQ and AMQP provide a standardized messaging protocol that improves scalability and reliability over traditional RPC approaches. The document discusses the benefits of messaging including decoupling and asynchronous communication. It provides an overview of core AMQP concepts like exchanges, queues, bindings and routing. Spring AMQP and Spring's RabbitTemplate simplify working with RabbitMQ in Java by providing a common API and configuration support.
Apache Kafka's rise in popularity as a streaming platform has demanded a revisit of its traditional at-least-once message delivery semantics.
In this talk, we present the recent additions to Kafka to achieve exactly-once semantics (EoS) including support for idempotence and transactions in the Kafka clients. The main focus will be the specific semantics that Kafka distributed transactions enable and the underlying mechanics which allow them to scale efficiently.
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffJAX London
2011-11-02 | 05:45 PM - 06:35 PM
The JMS standard is 9 years old - but outside the Java community innovation is happening. The AMQP standard with implementations like RabbitMQ is gaining more and more traction. This session explains the standard and its advantages. It will also show how an AMQP application can be implemented using Java.
Apache Kafka is a distributed publish-subscribe messaging system that was originally created by LinkedIn and contributed to the Apache Software Foundation. It is written in Scala and provides a multi-language API to publish and consume streams of records. Kafka is useful for both log aggregation and real-time messaging due to its high performance, scalability, and ability to serve as both a distributed messaging system and log storage system with a single unified architecture. To use Kafka, one runs Zookeeper for coordination, Kafka brokers to form a cluster, and then publishes and consumes messages with a producer API and consumer API.
Rabbit MQ is a message broker software that implements the Advanced Message Queuing Protocol (AMQP) standard. It allows for push-style communication where the server initiates transactions by sending messages to clients. Rabbit MQ supports publish/subscribe and queuing of messages across platforms. It provides reliability, security, and scalability for message delivery. Common uses of Rabbit MQ include notifications, messaging, and communication between services.
The document compares the performance of Apache Kafka and RabbitMQ for streaming data. It finds that without fault tolerance, both brokers have similar latency, but with fault tolerance enabled, Kafka has slightly higher latency than RabbitMQ. Latency increases with message size and is improved after an initial warmup period. Overall, RabbitMQ demonstrated the lowest latency for both configurations. The document also describes how each system is deployed and configured for the performance tests.
The document discusses reliability guarantees in Apache Kafka. It explains that Kafka provides reliability through replication of data across multiple brokers. As long as the minimum number of in-sync replicas (ISRs) is maintained, messages will not be lost even if individual brokers fail. It also discusses best practices for producers and consumers to ensure data is not lost such as using acks=all for producers, disabling unclean leader election, committing offsets only after processing is complete, and monitoring for errors, lag and reconciliation of message counts.
This document discusses messaging with AMQP and RabbitMQ. It provides an overview of why messaging is useful for decoupling systems and making them asynchronous and reliable. It then discusses why AMQP and RabbitMQ are good choices for messaging and covers some basic AMQP concepts like exchanges, queues, bindings and routing. It also discusses how Spring supports AMQP and RabbitMQ through its AMQP libraries and templates. Finally, it briefly covers how web messaging can be done through STOMP over websockets.
Building Cloud-Native App Series - Part 2 of 11
Microservices Architecture Series
Event Sourcing & CQRS,
Kafka, Rabbit MQ
Case Studies (E-Commerce App, Movie Streaming, Ticket Booking, Restaurant, Hospital Management)
RabbitMQ is an open source message broker that implements the AMQP protocol. It provides various messaging patterns using different exchange types and supports clustering for scalability and high availability. Administration of RabbitMQ includes managing queues, exchanges, bindings and other components. Integrations exist for protocols like STOMP, MQTT and frameworks like Spring, while security features include authentication, authorization, and SSL/TLS encryption.
Enterprise Messaging With ActiveMQ and Spring JMSBruce Snyder
The document discusses ActiveMQ, an open source message broker. It provides an overview of installing and configuring ActiveMQ, and describes how to use Spring JMS with ActiveMQ for both synchronous and asynchronous messaging. Key ActiveMQ features like persistence, clustering, security, messaging patterns, and consumer options are also summarized.
Network and Service Virtualization tutorial at ONUG Spring 2015SDN Hub
Tutorial at ONUG Spring 2015 on Network and Service Virtualization. The tutorial covers three converging trends 1) Network virtualization, 2) Service virtualization, 3) overlay networking for Docker and OpenStack. The talk concludes with pointers to the hands-on portion of the tutorial that uses LorisPack, and the operational lessons learned.
Orchestration Patterns for Microservices with Messaging by RabbitMQVMware Tanzu
Companies looking to speed up their software development are adopting microservices architectures (MSA). Building applications as groups of smaller components with fewer dependencies helps companies such as Comcast, Capital One, Uber, and Netflix deliver more frequent releases and thus innovate faster.
An important consideration in adopting an MSA is deciding how individual services should communicate between each other. Adding a message queue such as RabbitMQ to handle interservice messages can improve communication by:
- Simplifying our services so they only need to know how to talk to the messenger service.
- Abstracting communication by having the messenger service handle sophisticated orchestration patterns.
- Scaling message throughput by increasing the cluster size of the messenger service.
In this webinar we'll discuss:
- Requirements for communicating between microservices
- Typical messaging patterns in microservice architectures
- Use cases where RabbitMQ shines
- How to use the RabbitMQ service for Pivotal Cloud Foundry to deploy and run your applications
We’ll also demonstrate how to deploy RabbitMQ in Pivotal Cloud Foundry, and how to incorporate it in microservices-based applications.
Presenters: Greg Chase, Pivotal and Dan Baskette, Pivotal
IBM IMPACT 2014 - AMC-1882 Building a Scalable & Continuously Available IBM M...Peter Broadhurst
This document provides an overview of designing a scalable and highly available IBM MQ infrastructure. Key points include:
- Using a client/server architecture with MQ deployed separately from applications provides flexibility and allows MQ to be treated as critical infrastructure similar to a database.
- Each sender should connect to two queue managers and each receiver should have two listeners concurrently attached to provide redundancy and no single point of failure.
- Other topics covered include synchronous request/response, publish/subscribe messaging, limitations for ordered messages, and integrating with IBM Integration Bus.
The document emphasizes an active/active design philosophy with minimum two queue managers and discusses workload management strategies for sending and receiving messages across multiple queue managers.
The 100% open source WSO2 Message Broker is a lightweight, easy-to-use, distributed message-brokering server. It features high availability (HA) support with a complete hot-to-hot continuous availability mode, the ability to scale up to several servers in a cluster, and no single point of failure. It is designed to manage persistent messaging and large numbers of queues, subscribers and messages.
World of Tanks Experience of Using KafkaLevon Avakyan
In this paper I speak about BigWorld technology, WoT server, Apache Kafka and how we started to use it together. What difficulties we had and how we had solved them.
Event Driven Architecture and Apache Kafka were discussed. Key points:
- Event driven systems allow for asynchronous and decoupled communication between services using message queues.
- Apache Kafka is a distributed streaming platform that allows for publishing and subscribing to streams of records across a cluster of servers. It provides reliability through replication and allows for horizontal scaling.
- Kafka provides advantages over traditional queues like decoupling, scalability, and fault tolerance. It also allows for publishing of data and consumption of data independently, unlike traditional APIs.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Project Management Semester Long Project - Acuityjpupo2018
Acuity is an innovative learning app designed to transform the way you engage with knowledge. Powered by AI technology, Acuity takes complex topics and distills them into concise, interactive summaries that are easy to read & understand. Whether you're exploring the depths of quantum mechanics or seeking insight into historical events, Acuity provides the key information you need without the burden of lengthy texts.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
3. Session Agenda
• Modern Applications
– Trends and Impact on Messaging
– Impact on Application Architecture
• Spring Integration
– Messaging Use Cases
– Messaging DSL for Spring Applications
• AMQP
– Why AMQP?
– AMQP Architecture
• RabbitMQ
– Why RabbitMQ?
– RabbitMQ Architecture
4. Modern Application Trends
• Written with frameworks
– Developer productivity and innovation
• New Application Types
– Mobile, SaaS, Social
– Apps released early and often
• Data intensive
– Emerging requirements: elasticity, multi-cloud
– Web orientation drives exponential data
volumes
• Deployed on virtual and cloud infrastructure
– Trending toward hybrid (public and private)
infrastructure
5. Modern Applications need Modern Messaging
Traditional approaches to messaging
imply predictable, static interaction
between well-known end-points
Modern approaches to messaging
embrace the unpredictable, dynamic,
and transient nature of the web
6. Synchronous Architecture is Brittle
An exception
occurred, Manage
please resend Inventory
Place Fulfill
Order Order
X
X
X X X
X X
X
X
Schedule
Delivery
If any component is unavailable within this synchronous flow, all
in-flight messages will be affected, regardless of what
component they are currently in -- and since state is maintained
across the components, the system can be complex to reset
7. Asynchronous Architecture Rules the Web
The status of
your order is I can fulfill
… part of
that Order
I have an
Order
I can fulfill
part of
Your order will that Order
be delivered
next Tuesday
I’ll schedule
delivery of
that order
In this asynchronous flow:
• all state is kept in the simple messages being passed
• each stateless component interacts only with the broker cloud
• if a component is lost, only the in-flight messages within that
component must be re-delivered
• components can be quickly provisioned on-demand
8. How is Messaging Used within Modern Apps?
§ Event
I need to know when to do something
§ Fragment
I need to only give you a piece of this data
§ Route
Control who gets which message, without changing sender or receiver
§ Batch
Producer and consumer can run at independent times
§ Publish
Tell everyone who wants to know about this
§ Modularity
Deploy, Scale, and Version a distributed system
8
9. Event
I need to know when to do something …
<service-activator
input-channel="hotDrinks”
ref="barista"
method="prepareHotDrink"
output-channel="preparedDrinks” />
An Event-Driven Consumer is one that is automatically handed messages as
they’re delivered on the channel. The receiver acts like the message delivery
is an event that triggers the receiver into action.
9
10. Fragment
I need to only give you this piece of data …
<int:splitter
input-channel="orders”
expression="payload.items”
output-channel="drinks” />
Use a Splitter to break out the composite message into a series of individual
messages, each containing data related to one item.
10
11. Route
Control who gets what message
<int:router
input-channel="drinks”
expression="payload.iced ? 'coldDrinks' : 'hotDrinks'” />
A Router consumes a Message from one Channel and republishes it to a different
Channel channel depending on a set of conditions.
11
12. Batch
Producer and Consumer can run at independent times
<int:channel id="preparedDrinks"/>
<int:aggregator
input-channel="preparedDrinks"
method="prepareDelivery"
output-channel="deliveries” >
<bean class="org.sf.integration.samples.cafe.xml.Waiter"/>
</int:aggregator>
Use a stateful filter, an Aggregator, to collect and store individual messages until
a complete set of related messages has been received. Then, the Aggregator
publishes a single message distilled from the individual messages.
12
13. Publish
Tell everyone who wants to know about this
<int-event:outbound-channel-adapter
channel="eventChannel"/>
All messages sent to the 'eventChannel' channel will be published as
ApplicationEvents to any relevant ApplicationListener instances that are
registered within the same Spring ApplicationContext.
13
14. Modularity
Deploy, Scale, Version a distributed system
Local Distributed
<service-activator <int-amqp:outbound-gateway
input-channel="coldDrinks” id="coldDrinksBarista"
ref="barista” request-channel="coldDrinks”
method="prepareColdDrink” reply-channel="preparedDrinks”
output-channel="preparedDrinks” /> routing-key="ordered.drinks.cold” />
<service-activator <int-amqp:outbound-gateway
input-channel="hotDrinks” id="hotDrinksBarista"
ref="barista" request-channel="hotDrinks”
method="prepareHotDrink" reply-channel="preparedDrinks”
output-channel="preparedDrinks” /> routing-key="ordered.drinks.hot” />
Spring Integration doesn’t force you to finalize decisions about deployment before
testing begins. Individual modules can be versioned, deployed, and scaled
independently at any time through configuration, not code.
14
16. Advanced Message Queuing Protocol
“Like
email,
but
you
can
send
money
over
it”
Open,
Ubiquitous,
and
Adaptable
async SMTP AMQP
sync HTTP IIOP
unreliable reliable
16
17. Why AMQP?
A
Protocol,
not
an
API
• A defined set of
messaging capabilities
called the AMQ model
• A network wire-level
protocol, AMQP
On
commodity
hardware
• 10-‐25
thousand
messages
per
second
is
rou>ne
*
• The
NIC
is
usually
the
boDleneck
*
Non-‐persistent
messages
17
18. Why AMQP?
AMQP
security
• Brokers
support
separate
virtual
hosts
• Three
levels
of
permissions
• AMQP
over
SSL
is
supported
Designed
to
scale
horizontally
• Dozens
of
clustered
brokers
are
common
• JPMorgan
sends
1
billion
AMQP
messages
per
day
18
22. Demo – Scaling the Café Demo
with AMQP in Cloud Foundry
22
23. Legacy Middleware not suited for Modern Apps
Apps &
Middleware Existing middleware impacts agility by tightly
coupling applications to underlying application
server and other middleware components
Costly to Scale Up Complex to Scale Out Licensed by Peak Physical Servers
23
24. RabbitMQ – Messaging that Just Works
design goal: no bloat - “it just works”
mission: “be everywhere”
AMQP LEADER
25. Incredible Community Lowers Your Cost of Use
!"##$%(0$.%-$#G%,0(#K(C4.%(@$/G9(%KE,(E8"i4-C.
!"##$%&'`j<>("/0(&$:-4.4R(2g7(E"-%/,-.+$E
A,(4^,-(E":T"?,.(F4-(C"/K(4%+,-(=Q(E8"i4-C.
;84G0("/0(WG$80H!G/H&"/"?,(*$88(0-$3,(C4-,(."8,.
28. Enterprise Features, such as Fault Tolerance
ESX1
VM1 VM2
RMQ1 RMQ2
hot-drinks hot-drinks
cold-drinks cold-drinks
master
slave
29. Our New Mission: “It’s Just There”
Java App Private Cloud Hybrid Cloud
vFabric Cloud Foundry vFabric Cloud Foundry
Ruby App vSphere vSphere
Downloadable
Micro Cloud
1. Build your messaging application
vFabric Cloud Foundry
2. Deploy it, Scale it using Messaging Service Service
Provider
3. Move on or off premise as needed
31. Demo – Deploying the Café Demo
with RabbitMQ in a Hybrid Cloud
31
32. Summary
• Modern Applications
– Use frameworks, less emphasis on servers
– The explosion of connected mobile devices
requires a new solution for messaging
• Spring Integration
– A DSL for introducing messaging abstractions
into your spring application
– Externalizes messaging concepts from code
• AMQP
– A TCP/IP protocol not an API
– Fast, Reliable, Open, Secure, Scalable
• RabbitMQ
– AMQP Leader
– Backed by VMware