The document discusses systems integration in the cloud era using Apache Camel. It introduces Apache Camel as an open source framework that implements enterprise integration patterns and supports integration with various cloud platforms and services. It provides examples of using Apache Camel to integrate with IaaS platforms like Amazon Web Services, PaaS platforms like Google App Engine, and SaaS services like Salesforce. The key messages are that cloud integration is already possible, the cloud needs to be integrated, and Apache Camel helps significantly with cloud integration.
You have built a IaaS cloud and can deliver compute, storage and bandwidth to your customers. Is that enough to hold on to your customer base, or do you wish to deliver value added services such as platform services and databases? Join us our session and we will demonstrate how you could deploy higher margin services such as Database-as-a-Service, Caching-as-a-Service, Load Balancing-as-a-Service, MongoDB-as-a-Service, and many more - instantly without high capital investments. In this session, we will also cover different services that your customers will like and how you can make them available on your cloud and integrate them with your metering, billing and other business processes.
You have built a IaaS cloud and can deliver compute, storage and bandwidth to your customers. Is that enough to hold on to your customer base, or do you wish to deliver value added services such as platform services and databases? Join us our session and we will demonstrate how you could deploy higher margin services such as Database-as-a-Service, Caching-as-a-Service, Load Balancing-as-a-Service, MongoDB-as-a-Service, and many more - instantly without high capital investments. In this session, we will also cover different services that your customers will like and how you can make them available on your cloud and integrate them with your metering, billing and other business processes.
An Overview of Designing Microservices Based Applications on AWS - March 2017...Amazon Web Services
Microservices are an architectural approach to decompose complex applications into smaller, independent services. AWS customers benefit from increased agility, simplified scalability, resiliency, and faster deployments by migrating from monoliths to microservices based architecture.
In this session, we will provide an overview of the benefits and challenges of microservices, and share best practices for architecting and deploying microservices on AWS. We will dive into different approaches you can take to run microservices applications at scale and explore how services like Amazon ECS, AWS Lambda, and AWS X-Ray make it simpler to design and maintain these applications.
Learning Objectives:
1. Understand the fundamentals of the microservices architectural approach
2. Learn best practices for designing microservices on AWS
3. Learn the basics of Amazon EC2 Container Service, AWS Lambda, and AWS X-Ray
Secure and Govern Integration between the Enterprise & the CloudCA API Management
Secure, govern and mediate integrations between enterprise applications and Cloud services
Overview
For Best Buy, the public Cloud provides a strategic way to dynamically scale consumer and partner-facing Web and API assets. The Cloud lets Best Buy accommodate peaks in demand without overbuilding, while isolating sensitive data from the public.
Best Buy also needs a consistent way to control what information is shared with applications in the Cloud, while simultaneously insulating development teams from the vagaries of security, management and mediation challenges that arise when implementing a hybrid Cloud solution.
This Webinar, presented by Best Buy, Amazon Web Services and Layer 7 Technologies, looks at a specific example of the Best Buy API Developer Portal and share best practices for security, governance and mediation of enterprise services with applications in the Cloud.
This talks shows different approaches to PaaS for Java developers. It covers Google App Engine, Amazon Beanstalk and Cloud Foundry in detail. I did the talk with Bernd Kolb at JAX 2011, Germany.
Presented at MPLS 2010 Oct 24-27 Washington D.C.
Monique Morrow, Cisco Distinguished Consulting Engineer, discussed the role of the network in developing as a base for cloud computing in developing XaaS models across a private backbone vs offering Cloud-based services over the Internet.
The presenter further presented the potential evolution of Cloud Computing in the form of Private, Hybrid and Inter-Cloud.
Service Level Management and Security are also highlighted themes in this presentation.
An overview of the various standards organizations and forums that may be specific to cloud computing and emerging inter-cloud was also provided
Discussion and takeaway - the value of these models to your business.
This lecture was given by Professor June Sung Park at Korea Advanced Institute of Science and Technology in the 2012 Cloud Conference held in August 2012 by Electronic Times in Korea.
Come learn how to extend your existing Web applications and get them to live and breathe within Live Mesh. See how Mesh-enabled Web applications can be accessed from anywhere through a Web browser as well as run locally (and offline) on a user's desktop. Also see how Web applications can take full advantage of value-add Mesh services such as a dedicated sandbox, online and offline synchronized storage, automatic application updates, identity, application catalogue, social computing, and more.
2011 State of the Cloud: A Year's Worth of Innovation in 30 Minutes - Jinesh...Amazon Web Services
A Year's Worth of Innovation in 30 Minutes -
In this Keynote talk, Jinesh Varia discuss all the new features and services that AWS released in 2011 and discusses AWS growth and innovation along with customers and partners.
The speaker notes contain the links to the blog posts of announcements.
Cloud Integration for Hybrid IT: Balancing Business Self-Service and IT ControlAshwin V.
The success of SaaS applications has led to the era of hybrid IT - an era where both on-premise and SaaS applications coexist. To make this coexistence fruitful, it is essential to have a cloud integration strategy for any hybrid IT environment. Read to find out more.
What happens when a SOA success meets the cloud for the first time? And how to avoid the inevitable...
A presentation I did at radars i2i (Integrate 2 Innovate) conference in Stockholm.
An Overview of Designing Microservices Based Applications on AWS - March 2017...Amazon Web Services
Microservices are an architectural approach to decompose complex applications into smaller, independent services. AWS customers benefit from increased agility, simplified scalability, resiliency, and faster deployments by migrating from monoliths to microservices based architecture.
In this session, we will provide an overview of the benefits and challenges of microservices, and share best practices for architecting and deploying microservices on AWS. We will dive into different approaches you can take to run microservices applications at scale and explore how services like Amazon ECS, AWS Lambda, and AWS X-Ray make it simpler to design and maintain these applications.
Learning Objectives:
1. Understand the fundamentals of the microservices architectural approach
2. Learn best practices for designing microservices on AWS
3. Learn the basics of Amazon EC2 Container Service, AWS Lambda, and AWS X-Ray
Secure and Govern Integration between the Enterprise & the CloudCA API Management
Secure, govern and mediate integrations between enterprise applications and Cloud services
Overview
For Best Buy, the public Cloud provides a strategic way to dynamically scale consumer and partner-facing Web and API assets. The Cloud lets Best Buy accommodate peaks in demand without overbuilding, while isolating sensitive data from the public.
Best Buy also needs a consistent way to control what information is shared with applications in the Cloud, while simultaneously insulating development teams from the vagaries of security, management and mediation challenges that arise when implementing a hybrid Cloud solution.
This Webinar, presented by Best Buy, Amazon Web Services and Layer 7 Technologies, looks at a specific example of the Best Buy API Developer Portal and share best practices for security, governance and mediation of enterprise services with applications in the Cloud.
This talks shows different approaches to PaaS for Java developers. It covers Google App Engine, Amazon Beanstalk and Cloud Foundry in detail. I did the talk with Bernd Kolb at JAX 2011, Germany.
Presented at MPLS 2010 Oct 24-27 Washington D.C.
Monique Morrow, Cisco Distinguished Consulting Engineer, discussed the role of the network in developing as a base for cloud computing in developing XaaS models across a private backbone vs offering Cloud-based services over the Internet.
The presenter further presented the potential evolution of Cloud Computing in the form of Private, Hybrid and Inter-Cloud.
Service Level Management and Security are also highlighted themes in this presentation.
An overview of the various standards organizations and forums that may be specific to cloud computing and emerging inter-cloud was also provided
Discussion and takeaway - the value of these models to your business.
This lecture was given by Professor June Sung Park at Korea Advanced Institute of Science and Technology in the 2012 Cloud Conference held in August 2012 by Electronic Times in Korea.
Come learn how to extend your existing Web applications and get them to live and breathe within Live Mesh. See how Mesh-enabled Web applications can be accessed from anywhere through a Web browser as well as run locally (and offline) on a user's desktop. Also see how Web applications can take full advantage of value-add Mesh services such as a dedicated sandbox, online and offline synchronized storage, automatic application updates, identity, application catalogue, social computing, and more.
2011 State of the Cloud: A Year's Worth of Innovation in 30 Minutes - Jinesh...Amazon Web Services
A Year's Worth of Innovation in 30 Minutes -
In this Keynote talk, Jinesh Varia discuss all the new features and services that AWS released in 2011 and discusses AWS growth and innovation along with customers and partners.
The speaker notes contain the links to the blog posts of announcements.
Cloud Integration for Hybrid IT: Balancing Business Self-Service and IT ControlAshwin V.
The success of SaaS applications has led to the era of hybrid IT - an era where both on-premise and SaaS applications coexist. To make this coexistence fruitful, it is essential to have a cloud integration strategy for any hybrid IT environment. Read to find out more.
What happens when a SOA success meets the cloud for the first time? And how to avoid the inevitable...
A presentation I did at radars i2i (Integrate 2 Innovate) conference in Stockholm.
Business Applications Integration In The CloudAnna Brzezińska
Filip Rogaczewski - Atlassian Connect Team Lead.
Presentation from Gdansk University of Technology about integration business application in the cloud i.e. how to integrate 50 000+ servers together.
Join the cloud and big data integration experts from SnapLogic to find out about our Summer 2015 release, which improves support for self-service integration in the enterprise with new component reuse and governance capabilities.
This recorded webinar provides an overview and demonstration of new capabilities, including: simplified component reuse, enhance governance and usr management for administrators, improvements in big data and cloud analytics and additional updated Snaps.
To learn more, visit: www.snaplogic.com/summer2015
*MuleSoft’s Architecture Themes
- Mule ESB – Enterprise Service Bus
- Event Based Model
- API Centered Design
- Anypoint Studio – Visual Design
*Languages Used
*Mule as an Open Source project
*Mule Flows
Application Architecture: The Next Wave | MuleSoftBui Kiet
The explosion of APIs, SaaS applications, and mobile devices has created a massive integration wave. The resulting shift in the way we connect is forcing an IT mega change unlike anything we've seen before. As the development model moves from writing lots of code to composing APIs together, a new generation of middle tier application architecture is being born.
In the cloud, new businesses can emerge overnight. What are you doing to differentiate your application? How are you innovating ahead of your competition, while meeting your Executive KPI's? At a recent Executive Roundtable Ann Winblad, Co-Founder and Managing Director of Hummer Winblad Venture Partners, discussed these SaaS best practices with a group of 45 CEOs, Sales and Services Executives. View this presentation to learn how Executives from leading SaaS businesses in the Bay Area are creating a competitive advantage, and meeting their investor's key performance indicators.
What you will learn:
* 6 SaaS metrics that matter to your investors, and should matter to you
* How to increase recurring revenue without additional resources
* How to minimize customer turnover with a unique competitive advantage
We start with an introduction to what Apache Camel is, and how you can use Camel to make integration much easier. Allowing you to focus on your business logic, rather than low level messaging protocols, and transports. You will also hear what other features Camel provides out of the box, which can make integration much easier for you.
We look into web console tooling that allows you to get insight into your running Apache Camel applications, which has among others visual route diagrams with tracing/debugging and profiling capabilities. In addition to the web tooling we will also show you other tools in the making.
The explosion of APIs, SaaS applications, and mobile devices has created a massive integration wave. The resulting shift in the way we connect is forcing an IT mega change unlike anything we've seen before. As the development model moves from writing lots of code to composing APIs together, a new generation of middle tier application architecture is being born.
Businesses of every shape and size are using cloud to gain rapid access to world class IT capabilities (resource pooling, rapid elasticity,measured service, cost predictability, agility, rapid implementation, etc). Integrating a cloud solution with existing systems can be one of the most complex, costly and time-consuming aspects of cloud adoption.
(Diapositivas de presentación son en inglés.)
Las arquitecturas serverless permiten construir y ejecutar aplicaciones y servicios sin necesidad de manejar una infraestructura. En esta sesión veremos la filosofía de AWS y las mejores prácticas para construir aplicaciones basadas en microservicios, cómo servicios de AWS como Lambda y API Gateway benefician a los desarrolladores y cómo nuestros clientes están usando estos servicios y el resto de servicios de AWS para construir sus aplicaciones.
In this presentation, Jeff Barr introduces AWS, with a focus on EC2, and then shows how to use AWS Elastic Beanstalk with Git-based deployment of a PHP application.
AWS' philosophy and recommended best practices for building microservices applications, how AWS services like Lambda and API gateway benefit developers building microservices apps, and how customers are using these two and other AWS services to deliver their microservices apps
Systems Integration in the Cloud Era, Kai Wähner MaibornWolffCloudOps Summit
CloudOps Summit 2012, Frankfurt, 20.9.2012
Lightning Talk by Kai Wähner, IT Consultant at MaibornWolff et al.
http://cloudops.de/sprecher/#kaiwaehner
Find the video of this talk at http://youtu.be/-8K6yfbzBIk
You have heard how containers are great for running microservices, but running and managing large scale applications with microservices architectures is hard and often requires operating complex container management infrastructure. So what exactly is needed to get microservices to run in production at scale?
In this session, we will explore the reasoning and concepts behind microservices and how containers simplify building microservices based applications, and we will walk through a number of patterns used by our customers to run their microservices platforms. We will also dive deep into some of the challenges of running microservices, such as load balancing, service discovery, and secrets management, and we’ll see how Amazon EC2 Container Service (ECS) can help address them. We will also demo how you can easily deploy complex microservices applications using Amazon ECS.
SQLUG event: An evening in the cloud: the old, the new and the big Mike Martin
Belgian SQL UG talk On Windows Azure SQL Database and CO: The last couple of months have been very exciting times for Microsoft fans. With the release of the central flagship, SQL Server 2012, and the new and improved Cloud offering on Windows Azure, Microsoft managed to change the IT Landscape dramatically. During this talk we want to take you through all the SQL Server features in the Windows Azure Environment in either the full cloud spectrum or hybrid scenario’s. We’ll cover subjects like SQL Databases, IaaS, provisioning, integration, migration and touch some of the more computing topics in the Windows Azure when it comes to data. Going from the pure basics of all SQL related to the more advanced stuff you can do with this magnificent cloud platform.
Systems Integration in the Cloud Era with Apache Camel @ ApacheCon Europe 2012Kai Wähner
Shows the elegance of Apache Camel to integrate different cloud providers such as Amazon Web Services (IaaS), Google App Engine (PaaS), or Salesforce (SaaS).
O'Reilly Webcast: Architecting Applications For The CloudO'Reilly Media
This presentation analyzes aspects of the Amazon EC2 IaaS cloud environment that differ from a traditional data center and introduces general best practices for ensuring data privacy, storage persistence, and reliable DBMS backup. Presented by Jorge Noa, CTO of Hyperstratus
Apache Kafka as Data Hub for Crypto, NFT, Metaverse (Beyond the Buzz!)Kai Wähner
Decentralized finance with crypto and NFTs is a huge topic these days. It becomes a powerful combination with the coming metaverse platforms across industries. This session explores the relationship between crypto technologies and modern enterprise architecture.
I discuss how data streaming and Apache Kafka help build innovation and scalable real-time applications of a future metaverse. Let's skip the buzz (and NFT bubble) and instead review existing real-world deployments in the crypto and blockchain world powered by Kafka and its ecosystem.
Apache Kafka is the de facto standard for data streaming to process data in motion. With its significant adoption growth across all industries, I get a very valid question every week: When NOT to use Apache Kafka? What limitations does the event streaming platform have? When does Kafka simply not provide the needed capabilities? How to qualify Kafka out as it is not the right tool for the job?
This session explores the DOs and DONTs. Separate sections explain when to use Kafka, when NOT to use Kafka, and when to MAYBE use Kafka.
No matter if you think about open source Apache Kafka, a cloud service like Confluent Cloud, or another technology using the Kafka protocol like Redpanda or Pulsar, check out this slide deck.
A detailed article about this topic:
https://www.kai-waehner.de/blog/2022/01/04/when-not-to-use-apache-kafka/
Kafka for Live Commerce to Transform the Retail and Shopping MetaverseKai Wähner
Live commerce combines instant purchasing of a featured product and audience participation.
This talk explores the need for real-time data streaming with Apache Kafka between applications to enable live commerce across online stores and brick & mortar stores across regions, countries, and continents in any retail business.
The discussion covers several building blocks of a live commerce enterprise architecture, including transactional data processing, omnichannel, natural language processing, augmented reality, edge computing, and more.
The Heart of the Data Mesh Beats in Real-Time with Apache KafkaKai Wähner
If there were a buzzword of the hour, it would certainly be "data mesh"! This new architectural paradigm unlocks analytic data at scale and enables rapid access to an ever-growing number of distributed domain datasets for various usage scenarios.
As such, the data mesh addresses the most common weaknesses of the traditional centralized data lake or data platform architecture. And the heart of a data mesh infrastructure must be real-time, decoupled, reliable, and scalable.
This presentation explores how Apache Kafka, as an open and scalable decentralized real-time platform, can be the basis of a data mesh infrastructure and - complemented by many other data platforms like a data warehouse, data lake, and lakehouse - solve real business problems.
There is no silver bullet or single technology/product/cloud service for implementing a data mesh. The key outcome of a data mesh architecture is the ability to build data products; with the right tool for the job.
A good data mesh combines data streaming technology like Apache Kafka or Confluent Cloud with cloud-native data warehouse and data lake architectures from Snowflake, Databricks, Google BigQuery, et al.
Apache Kafka vs. Cloud-native iPaaS Integration Platform MiddlewareKai Wähner
Enterprise integration is more challenging than ever before. The IT evolution requires the integration of more and more technologies. Applications are deployed across the edge, hybrid, and multi-cloud architectures. Traditional middleware such as MQ, ETL, ESB does not scale well enough or only processes data in batch instead of real-time.
This presentation explores why Apache Kafka is the new black for integration projects, how Kafka fits into the discussion around cloud-native iPaaS (Integration Platform as a Service) solutions, and why event streaming is a new software category.
A concrete real-world example shows the difference between event streaming and traditional integration platforms respectively cloud-native iPaaS.
Video Recording of this presentation:
https://www.youtube.com/watch?v=I8yZwKg_IJc&t=2842s
Blog post about this topic:
https://www.kai-waehner.de/blog/2021/11/03/apache-kafka-cloud-native-ipaas-versus-mq-etl-esb-middleware/
Data Warehouse vs. Data Lake vs. Data Streaming – Friends, Enemies, Frenemies?Kai Wähner
The concepts and architectures of a data warehouse, a data lake, and data streaming are complementary to solving business problems.
Unfortunately, the underlying technologies are often misunderstood, overused for monolithic and inflexible architectures, and pitched for wrong use cases by vendors. Let’s explore this dilemma in a presentation.
The slides cover technologies such as Apache Kafka, Apache Spark, Confluent, Databricks, Snowflake, Elasticsearch, AWS Redshift, GCP with Google Bigquery, and Azure Synapse.
Serverless Kafka and Spark in a Multi-Cloud Lakehouse ArchitectureKai Wähner
Apache Kafka in conjunction with Apache Spark became the de facto standard for processing and analyzing data. Both frameworks are open, flexible, and scalable.
Unfortunately, the latter makes operations a challenge for many teams. Ideally, teams can use serverless SaaS offerings to focus on business logic. However, hybrid and multi-cloud scenarios require a cloud-native platform that provides automated and elastic tooling to reduce the operations burden.
This session explores different architectures to build serverless Apache Kafka and Apache Spark multi-cloud architectures across regions and continents.
We start from the analytics perspective of a data lake and explore its relation to a fully integrated data streaming layer with Kafka to build a modern data Data Lakehouse.
Real-world use cases show the joint value and explore the benefit of the "delta lake" integration.
Resilient Real-time Data Streaming across the Edge and Hybrid Cloud with Apac...Kai Wähner
Hybrid cloud architectures are the new black for most companies. A cloud-first strategy is evident for many new enterprise architectures, but some use cases require resiliency across edge sites and multiple cloud regions. Data streaming with the Apache Kafka ecosystem is a perfect technology for building resilient and hybrid real-time applications at any scale. This talk explores different architectures and their trade-offs for transactional and analytical workloads. Real-world examples include financial services, retail, and the automotive industry.
Video recording:
https://qconlondon.com/london2022/presentation/resilient-real-time-data-streaming-across-the-edge-and-hybrid-cloud
Data Streaming with Apache Kafka in the Defence and Cybersecurity IndustryKai Wähner
Agenda:
1) Defence, Modern Warfare, and Cybersecurity in 202X
2) Data in Motion with Apache Kafka as Defence Backbone
3) Situational Awareness
4) Threat Intelligence
5) Forensics and AI / Machine Learning
6) Air-Gapped and Zero Trust Environments
7) SIEM / SOAR Modernization
Technologies discussed in the presentation include Apache Kafka, Kafka Streams, kqlDB, Kafka Connect, Elasticsearch, Splunk, IBM QRadar, Zeek, Netflow, PCAP, TensorFlow, AWS, Azure, GCP, Sigma, Confluent Cloud,
Real-World Deployments of Data Streaming with Apache Kafka across the Healthcare Value Chain using open source and cloud-native technologies and serverless SaaS:
1) Legacy Modernization and Hybrid Cloud: Optum (UnitedHealth Group, Centene, Bayer)
2) Streaming ETL (Bayer, Babylon Health)
3) Real-time Analytics (Cerner, Celmatix, CDC/Centers for Disease Control and Prevention)
4) Machine Learning and Data Science (Recursion, Humana)
5) Open API and Omnichannel (Care.com, Invitae)
The Rise of Data in Motion in the Healthcare Industry - Use Cases, Architectures and Examples powered by Apache Kafka.
Use Cases for Data in Motion in the Healthcare Industry:
- Know Your Patient (= “Customer 360”)
- Operations (Healthcare 4.0 including Drug R&D, Patient Care, etc.)
- IT Perspective (Cybersecurity, Mainframe Offload, Hybrid Cloud, Streaming ETL, etc)
Real-world examples include Covid-19 Electronic Lab Reporting, Cerner, Optum, Centene, Humana, Invitae, Bayer, Celmatix, Care.com.
Apache Kafka for Real-time Supply Chainin the Food and Retail IndustryKai Wähner
Use Cases, Architectures, and Real-World Examples for data in motion and real-time event streaming powered by Apache Kafka across the supply chain and logistics. Case studies and deployments include Baader, Walmart, Migros, Albertsons, Domino's Pizza, Instacart, Grab, Royal Caribbean, and more.
Kafka for Real-Time Replication between Edge and Hybrid CloudKai Wähner
Not all workloads allow cloud computing. Low latency, cybersecurity, and cost-efficiency require a suitable combination of edge computing and cloud integration.
This session explores architectures and design patterns for software and hardware considerations to deploy hybrid data streaming with Apache Kafka anywhere. A live demo shows data synchronization from the edge to the public cloud across continents with Kafka on Hivecell and Confluent Cloud.
Apache Kafka for Predictive Maintenance in Industrial IoT / Industry 4.0Kai Wähner
The manufacturing industry is moving away from just selling machinery, devices, and other hardware. Software and services increase revenue and margins. Equipment-as-a-Service (EaaS) even outsources the maintenance to the vendor.
This paradigm shift is only possible with reliable and scalable real-time data processing leveraging an event streaming platform such as Apache Kafka. This talk explores how Kafka-native Condition Monitoring and Predictive Maintenance help with this innovation.
More details:
https://www.kai-waehner.de/blog/2021/10/25/apache-kafka-condition-monitoring-predictive-maintenance-industrial-iot-digital-twin/
Video recording:
https://youtu.be/tfOuN5KeI9w
Apache Kafka Landscape for Automotive and ManufacturingKai Wähner
Today, in 2022, Apache Kafka is the central nervous system of many applications in various areas related to the automotive and manufacturing industry for processing analytical and transactional data in motion across edge, hybrid, and multi-cloud deployments.
This presentation explores the automotive event streaming landscape, including connected vehicles, smart manufacturing, supply chain optimization, aftersales, mobility services, and innovative new business models.
Afterwards, many real-world examples are shown from companies such as Audi, BMW, Porsche, Tesla, Uber, Grab, and FREENOW.
More detail in the blog post:
https://www.kai-waehner.de/blog/2022/01/12/apache-kafka-landscape-for-automotive-and-manufacturing/
Kappa vs Lambda Architectures and Technology ComparisonKai Wähner
Real-time data beats slow data. That’s true for almost every use case. Nevertheless, enterprise architects build new infrastructures with the Lambda architecture that includes separate batch and real-time layers.
This video explores why a single real-time pipeline, called Kappa architecture, is the better fit for many enterprise architectures. Real-world examples from companies such as Disney, Shopify, Uber, and Twitter explore the benefits of Kappa but also show how batch processing fits into this discussion positively without the need for a Lambda architecture.
The main focus of the discussion is on Apache Kafka (and its ecosystem) as the de facto standard for event streaming to process data in motion (the key concept of Kappa), but the video also compares various technologies and vendors such as Confluent, Cloudera, IBM Red Hat, Apache Flink, Apache Pulsar, AWS Kinesis, Amazon MSK, Azure Event Hubs, Google Pub Sub, and more.
Video recording of this presentation:
https://youtu.be/j7D29eyysDw
Further reading:
https://www.kai-waehner.de/blog/2021/09/23/real-time-kappa-architecture-mainstream-replacing-batch-lambda/
https://www.kai-waehner.de/blog/2021/04/20/comparison-open-source-apache-kafka-vs-confluent-cloudera-red-hat-amazon-msk-cloud/
https://www.kai-waehner.de/blog/2021/05/09/kafka-api-de-facto-standard-event-streaming-like-amazon-s3-object-storage/
The Top 5 Apache Kafka Use Cases and Architectures in 2022Kai Wähner
I see the following topics coming up more regularly in conversations with customers, prospects, and the broader Kafka community across the globe:
Kappa Architecture: Kappa goes mainstream to replace Lambda and Batch pipelines (that does not mean that there is no batch processing anymore). Examples: Kafka-powered Kappa architectures from Uber, Disney, Shopify, and Twitter.
Hyper-personalized Omnichannel: Retail and customer communication across online and offline channels becomes the new black, including context-specific upselling, recommendations, and location-based services. Examples: Omnichannel Retail and Customer 360 in Real-Time with Apache Kafka.
Multi-Cloud Deployments: Business units and IT infrastructures span across regions, continents, and cloud providers. Linking clusters for bi-directional replication of data in real-time becomes crucial for many business models. Examples: Global Kafka deployments.
Edge Analytics: Low latency requirements, cost efficiency, or security requirements enforce the deployment of (some) event streaming use cases at the far edge (i.e., outside a data center), for instance, for predictive maintenance and quality assurance on the shop floor level in smart factories. Examples: Edge analytics with Kafka.
Real-time Cybersecurity: Situational awareness and threat intelligence need to process massive data in real-time to defend against cyberattacks successfully. The many successful ransomware attacks across the globe in 2021 were a warning for most CIOs. Examples: Cybersecurity for situational awareness and threat intelligence in real-time.
Event Streaming CTO Roundtable for Cloud-native Kafka ArchitecturesKai Wähner
Technical thought leadership presentation to discuss how leading organizations move to real-time architecture to support business growth and enhance customer experience. This is a forum to discuss use cases with your peers to understand how other digital-native companies are utilizing data in motion to drive competitive advantage.
Agenda:
- Data in Motion with Event Streaming and Apache Kafka
- Streaming ETL Pipelines
- IT Modernisation and Hybrid Multi-Cloud
- Customer Experience and Customer 360
- IoT and Big Data Processing
- Machine Learning and Analytics
Apache Kafka in the Public Sector (Government, National Security, Citizen Ser...Kai Wähner
The Rise of Data in Motion in the Public Sector powered by event streaming with Apache Kafka.
Citizen Services:
- Health services, e.g. hospital modernization, track & trace - Covid distance control
- Public administration - reduce bureaucracy, data democratization across government departments
- eGovernment - Efficient and digital citizen engagement, e.g. personal ID application process
Smart City
- Smart driving, parking, buildings, environment
Waste management
- Open exchange – e.g. mobility services (1st and 3rd party)
Energy
- Smart grid and utilities infrastructure (energy distribution, smart home, smart meters, smart water, etc.)
- National Security
Law enforcement, surveillance, police/interior security data exchange
- Defense and military (border control, intelligent solider)
Cybersecurity for situational awareness and threat intelligence
Telco 4.0 - Payment and FinServ Integration for Data in Motion with 5G and Ap...Kai Wähner
The Era of Telco 4.0: Embracing Digital Transformation with Data in Motion. Learn about Payment and FinServ Integration for Data in Motion with 5G and Apache Kafka.
1) The rise of Telco 4.0 and the future forward
2) Data in Motion in the Telco industry
3) Real-world Fintech and Payment examples powered by Data in Motion
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Jazoon 2012 - Systems Integration in the Cloud Era with Apache Camel
1. Systems Integration
in the Cloud Era
with Apache Camel
Kai Wähner (Twitter: @KaiWaehner)
MaibornWolff et al GmbH
2. Kai Wähner (MaibornWolff et al GmbH)
Main Tasks
Evaluation of Technologies and Products
Requirements Engineering
Enterprise Architecture Management
Business Process Management
Architecture and Development of Applications
Planning and Introduction of SOA
Integration of Legacy Applications
Cloud Computing
Consulting Contact
Developing Email: kai.waehner@mwea.de
Speaking Blog: www.kai-waehner.de/blog
Coaching Twitter: @KaiWaehner
Writing Social Networks: Xing, LinkedIn
3. What is the Problem?
Growth
• Applications
• Interfaces
• Technologies
• Products
14. Cloud Computing
Infrastructure Platform Software
On Premise
On Premise as a Service as a Service as a Service
hosted
(IaaS) (PaaS) (SaaS)
App App App App App
VM VM VM VM VM
Server Server Server Server Server
Storage Storage Storage Storage Storage
Network Network Network Network Network
Organisation Control Vendor
has control is shared has control
17. Alternatives for Systems Integration
Enterprise Service Bus
Integration (ESB)
No Tool
Framework
Complexity
of Integration
Low High
18. Alternatives for Systems Integration
Enterprise Service Bus
Integration (ESB)
No Tool
Framework
Complexity
of Integration
Low High
Apache Camel vs. Spring Integration vs. Mule ESB
http://www.kai-waehner.de/blog/2012/01/10/spoilt-for-choice-which-integration-framework-to-use-spring-integration-mule-esb-or-apache-camel/
29. Cloud Computing
Infrastructure Platform Software
On Premise
On Premise as a Service as a Service as a Service
hosted
(IaaS) (PaaS) (SaaS)
App App App App App
VM VM VM VM VM
Server Server Server Server Server
Storage Storage Storage Storage Storage
Network Network Network Network Network
Organisation Control Vendor
has control is shared has control
36. Cloud Computing
Infrastructure Platform Software
On Premise
On Premise as a Service as a Service as a Service
hosted
(IaaS) (PaaS) (SaaS)
App App App App App
VM VM VM VM VM
Server Server Server Server Server
Storage Storage Storage Storage Storage
Network Network Network Network Network
Organisation Control Vendor
has control is shared has control
38. PaaS Concepts
Application Deployment
Easy Deployment
Automatic Scaling
Development Restrictions
JRE Class White List
Workarounds for Frameworks
No „naked“ Domains
No „write once run everywhere“
Quotas and Limits
Services
Push Queue
Pull Queue
URL Fetch
Accounts
Mail
Memcache
XMPP
Images
Datastore
Cloud Storage
Cloud SQL
39. Hint
Google App Engine
is a complex scenario for Apache
Camel due to its many restrictions!
Other „more open“ PaaS solutions
such as OpenShift or Heroku
are easier to use ...
40. Code Example - Google App Engine (Java API)
public class GAEJCreateTaskServlet extends HttpServlet {
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
String strCallResult = "";
resp.setContentType("text/plain");
try {
String strEmailId = req.getParameter("emailid");
if (strEmailId == null) throw new Exception ("Email Id field cannot be empty.");
strEmailId = strEmailId.trim();
if (strEmailId.length() == 0) throw new Exception("Email Id field cannot be empty.");
Queue queue = QueueFactory.getQueue("subscription-queue");
queue.add(TaskOptions.Builder.url(“signupsubscriber“)
.param("emailid",strEmailId));
strCallResult = "Successfully created a Task in the Queue";
resp.getWriter().println(strCallResult);
}
catch (Exception ex) {
strCallResult = "Fail: " + ex.getMessage();
resp.getWriter().println(strCallResult);
} }}
45. Cloud Computing
Infrastructure Platform Software
On Premise
On Premise as a Service as a Service as a Service
hosted
(IaaS) (PaaS) (SaaS)
App App App App App
VM VM VM VM VM
Server Server Server Server Server
Storage Storage Storage Storage Storage
Network Network Network Network Network
Organisation Control Vendor
has control is shared has control
47. SaaS Concepts
Software (CRM)
Sales
Service
Social
Data.com
AppExchange
... more ...
Development
Online-Development
(even the Compiler is in the Cloud!)
Own Addons Force.com (PaaS)
Apex Integration of Interfaces
Visualforce
REST
SOAP
Client APIs (Java, etc.)
48. Code Example – Salesforce (Java API)
ConnectorConfig config = new ConnectorConfig();
config.setUsername(userId);
config.setPassword(passwd);
config.setAuthEndpoint(authEndPoint);
connection = new EnterpriseConnection(config);
GetUserInfoResult userInfo = connection.getUserInfo();
System.out.println("User Full Name: " + userInfo.getUserFullName());
QueryResult qResult = null;
String soqlQuery = "SELECT FirstName, LastName FROM Contact"; // SOQL
qResult = connection.query(soqlQuery);
boolean done = false;
if (qResult.getSize() > 0) {
System.out.println("Logged-in user can see " +
qResult.getRecords().length +
" contact records."
);
while (! done) {
SObject[] records = qResult.getRecords();
for ( int i = 0; i < records.length; ++i ) {
Contact con = (Contact) records[i];
String fName = con.getFirstName();
String lName = con.getLastName();
// ... more stuff here ...