Munich MuleSoft Meetup Recap

28th-OCT-2020
Munich MuleSoft Meetup

3
The information in this presentation is confidential and proprietary to MuleSoft and may not be disclosed
without the permission of MuleSoft. This presentation is not subject to your license agreement or any other
service or subscription agreement with MuleSoft. MuleSoft has no obligation to pursue any course of
business outlined in this document or any related presentation, or to develop or release any functionality
mentioned therein. This document, or any related presentation and MuleSoft's strategy and possible future
developments, products and or platforms directions and functionality are all subject to change and may be
changed by MuleSoft at any time for any reason without notice. The information on this document is not a
commitment, promise or legal obligation to deliver any material, code or functionality. This document is
provided without a warranty of any kind, either express or implied, including but not limited to, the implied
warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is for
informational purposes and may not be incorporated into a contract. MuleSoft assumes no responsibility for
errors or omissions in this document, except if such damages were caused by MuleSoft intentionally or
grossly negligent.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results
to differ materially from expectations. Readers are cautioned not to place undue reliance on these
forward-looking statements, which speak only as of their dates, and they should not be relied upon in
making purchasing decisions.
Safe Harbor statement

4
● Keep your microphones on mute at all times unless you are addressing a question
● Slow network…No problem. Just disconnect and join back
● Questions in handy for the Q&A AFTER each topic.
○ Use the Q&A section in your Chat tab.
○ Address using @[Name] if you are addressing someone speciﬁcally
● Turn On your camera for an interactive session
● Troubleshooting in Bevy virtual
https://help.bevylabs.com/article/499-troubleshooting-in-bevy-virtual
● The meetup is recorded
Mic Check...

5
● Introductions
● Integration journey: Lessons Learnt
● Managing RTF: Hybrid Deployment Model at BP
● Interactive Quiz
● MuleSoft Hackathon Details
● Picture Time
● Wrap up
Agenda

6
● About the organizer:
○ Chinmaya Sahu
○ Ganesh Kumar
○ Sakthi
● About the speakers:
○ Andreas Grimm
○ Demir Akarsu
Introductions

- Andreas Grimm
Integration Journey: Lessons Learnt

All contents © MuleSoft, LLC
Andreas Grimm
Senior Solution Architect / Practice Lead Professional Services
Running Integration Projects:
Lessons Learned
October 2020

This presentation is a representation of my personal experiences with
Integration Projects. They are NOT providing advice or guidance for
projects completed, in progress, or planned. Nor shall they be taken as
MuleSoft’s advice to any existing or planned project. Every integration
project is different and needs to be seen in it’s content. Common places
made in this presentation are made to explain the context. No example in
the project is a referring to a real client engagements, but are generated
oversimplified reflects on experiences...

Who am I
I am
… an architect working on integration projects
… working with MuleSoft’s Professional
Services team for 5 years
…. doing integration projects since 1999
(calling it 20 years makes me feel old)
… working in IT since 1990 (I had the fun of
seeing a lot of new technologies arrive - and I
still ﬁnd this exciting)
… having opinions
10

Why am I talking about this here?
Introduction
OK - so you are leading a new integration
project. Either as an architect - or as a
developer - or as a project manager.
And your main worry is:
My project can fail.
Let’s have a look whether your worry is
justiﬁed...
11

Why am I talking about this here?
Introduction
So - I looked into the statistics, and the
statistics says:
The project will fail.
Unless I do something immediately when the
project starts.
But to do something - I need to understand
the main threads to my project.
12

Out of the “Grimm’s” Storybook...
Does not have to be like this...
● OK - No project fails without a reason.
● And failure does have a good side
when we learn from it.
● I have been asked to ﬁre-ﬁght
integration projects in the last 20
years.
● The next 5 sections are lessons I
learned from projects that were in
jeopardy or failed.
13

Lesson 1: As a Project Manager - I
understand how to run projects - I do
not need to understand Integrations

Ideal vs. Reality
#1: Integration Projects are just like other Projects
15
Ideal world: Clear view and no obstacle Reality: I cannot even see the next 50 meters

This is how your peers and managers see you in
the project:
As the integration lead you are causing issues:
- Integrations are a necessary evil
- Integration projects are too “technical”
- Integration projects are interfering with
already completed parts of the project
Integration projects are coming in on the
11th hour…
- And they costs too much
- 16
but only because the value of integration is misunderstood
so let’s take the focus away from the technology
as you start with them too late
as you start with them too late
they only cost more than expected because...

Actually - they are not just like other projects.
- Integration projects are highly interactive, more than normal projects - most integration
projects I have seen fail due to communication deﬁcits…
- Integration project are highly volatile - you will be very much challenged with changes at a
way higher rate than normal projects. So adopting a project approach that reﬂects this is
crucial
- Integration projects are technical diverse - you need to interact with all the other
technologies in the environment. You might not need all the skills in your team, but you
need to identify the right people working with you.
17

Lesson 2: We start the Integration
Project when the connecting systems are
ready...

Misconception:
- We are doing Scrum
“We need to have the other systems ready before you start…”
#2: Timing
20
Requirements Design Implementation MaintenanceVerification
Most common Project model:
Implementation
- Based on the estimates, integration can do it if it starts here...

This is NOT Scrum or Agile - this is
Waterfall
Treat it like this...
#2: Timing
21

#2: Timing
22
Requirements Design Implementation MaintenanceVerification
So - what is the problem here?
ﬁxed
planned
Design Implementation
Verification Maintenance reality
Real start of the integration effort
Planned start of the integration effort
The integration eﬀort is too slow - it puts the project into
jeopardy…

#2: Timing
23
Requirements Design MaintenanceVerification
First - changing dependencies...
ﬁxed
mitigated
Implementation
Planned start of the integration effort
OK - we have seen this before - and we think that does not work…?

#2: Timing
24
OK - we have seen this before - and we think that does not work…?
- We know which systems we want to integrate,
but we do not understand the data going from
one system to another yet…
- The developers of the connecting systems are
not ready yet…
- The integrations are going to change…
- We do not have the infrastructure ready yet…
- We have problems with the business
functionality, which we want to put into the
integration...
- Using a layered architecture and versionable
APIs allows you to build the main flows which is
not covering the final functionality.
- System Layer APIs should expose the full
business entity, not just the data required for the
flow.
- A main part of the work is the build or the initial
integration and the automated unit test.
Changes afterwards can be done with limited
effort…
- If Cloudhub is used, the set-up can be done in
an early stage of the project. We want this
connectivity and security discussions out of the
way as soon as somehow possible...

#2: Timing
25
Lessons learned: Every day you start earlier saves two or more days at the end...

Lesson 3: None of the systems provides
the function? We can make it part of the
Integration

“We do not know where it lives - so why not in the integration…”
#3: Moving Business Functionality into
the Integration…
27
You have not seen me screaming about this section...
- We know which systems we want to integrate,
but we do not understand the data going from
one system to another yet…
- The developers of the connecting systems are
not ready yet…
- The integrations are going to change…
- We do not have the infrastructure ready yet…
- We have problems with the business
functionality, which we want to put into the
integration...
- This anti-pattern has been recognized 20 years
ago -> a business system should not extended by
moving functionality in the integration layer.
- But we still see it during architecture reviews
- Commonly explained by
- The business application is immutable (we
cannot change the code)
- The MuleSoft team can do this sooo much
faster
- Why not - the System Layer is part of the
application anyway...

28
Application owned business logic does not belong in the integration layer...
the Integration…
Ever!!!

29
the Integration…

Lesson 4: Fire and Forget is a valid
Integration Paradigm...

“And old ‘Friend’ - coming back frequently…”
#4: Fire and Forget - and Integration Paradigm
31
An old architecture friend (not one of ours):
The bad one:
The integration consists of a source system, sending
a transaction to the integration environment. It then
marks the business process completed.
The not so bad one:
Return codes are not required, failed transactions are
covered by a database in the integration layer…
The really really bad one:
An operator looks at the errors, ﬁxes the
transactions, and resubmits them...

32
So why is this sooo bad?
System Inconsistency:
As soon as the transaction fails the business process fails - and the
data in the target system deviates from the sender. Transactional
integrity (and if the transaction is ﬁxed in the integration layer: at least
audit trail and auditability) are broken. This will not pass a good
auditor.
Moving the business logic:
See my last section
Stability:
And the integration gains something you do not want to have: Data At
Rest (including sizing of the data stores, backup and recovery
processes, etc.)

33
But we implemented an event-driven architecture - what our
legacy ERP system sends are just that - events… and those are
ﬁre-and-forget
Seriously?
When building an event-driven architecture - or the publish / subscribe paradigm - it is best
practice to always implement a feedback loop to the Controller (the source of the event). The use
of an asynchronous transaction request does not imply that we are giving transactional integrity
up...

Lesson 5: We are building Batch
Transfers - we do not need reusable
components

#5: Batch Flows Do Not Need API’s
Why building reusable components into Batch ﬂows?
So - this is how the concept of a Batch Flow in MuleSoft looks like:
35
Or - more conceptual:

In reality - they are often build like this:
36
Is this not the most beautiful point-to-point connection you have seen?
No Re-use of Back-end Connectivity… No Scaling when needed… No Abstraction of
Back-end Systems… etc. etc.

Let’s think about API’s:
37
Experience LayerBatch Load Lead
Process Layer
System Layer
SL SFDC
PL Lead
get post
post
Experience Layer

- Demir Akarsu
Managing RTF: Hybrid Deployment Model

Managing RTF in a Hybrid Deployment Model
Presented by Demir Akarsu
Software Engineering Specialist || BP

• Hybrid Deployment Model
• Runtime Fabric Overview
• CI/CD Pipeline Integration
• Developer Monitoring and Alerting Tools
• Automating Upgrades & Security Patches
• Managing Utilization

Cloudhub
- Customers tend to adopt Cloudhub as the
default deployment platform
- External only
- Difficult route to internal systems
- MuleSoft managed ecosystem
On-prem
- As platform matures in an org on-prem
options are sought out
- Internal only
- Usually managed by other internal teams
- Different control plane
- Learning curve for mule developers

Overview
Runtime Fabric orchestrates and automates the deployment of
Mule runtimes into containers in any cloud or on-premises.
Benefits
- Deploy consistently across any cloud (Azure & AWS) or data
center
- Run multiple runtime versions in the same Runtime Fabric
- Isolate apps, scale horizontally, redeploy w/ zero-downtime
- Connect to the control plane hosted by MuleSoft
4343
CUSTOMER’S NETWORK
Control
plane
Server Server Server
Mule
App
Mule
App
Mule
App
Mule
App
Mule
App
Mule
App
Mule
App
Mule
App
Mule
App
Runtime FabricDocker K8s

Responsibilities
- MuleSoft manage the Runtime Fabric appliance
and is responsible for the delivered components:
- Runtime Fabric appliance
- Runtime Fabric agent
- Mule runtime engine
- other dependencies for Mule applications
- Customer is responsible for provisioning,
configuring, and managing the infrastructure
required for Runtime Fabric including VM
resources, operating systems, load balancing and
kernel patching
4444

Monitoring & Logs in your own infrastructure
- Grafana Dashboards for real-time POD and OS level
metrics (RTF & On-prem)
- Log Appenders for historical and real-time log
forwarding
API Statistics, Consumption & Performance
- Splunk Dashboards
- API Resource Quota
- Response Times
- Top Consumers & Endpoints
- API Health
Alerting Tools
- SMTP to Slack and Mailboxes
- Data from logs, POD metrics & platform APIs

RTF Upgrades
- Cluster specific infrastructure provisioning (automate
controllers and workers builds and joins)
- Replica scaling for production
- Azure DevOps pipelines
API Statistics, Consumption & Performance
- Splunk Dashboards for tracking compliance and
patching versions
- Azure DevOps pipelines to apply security patches

API Resource Quota
- Managed through MuleOps deploy and release pipeline
Platform License Allocation
- Splunk Dashboards
- Hybrid Infrastructure that requires a single view
- Data sourced from Cloudhub APIs, python scripts and
on-prem HTTP Event Collector Tokens (HEC)
- Total Licenses Purchased and Spare
- Breakdown of individual platforms
- Peak trackers for on-prem compliance

Thank you for listening
Presented by Demir Akarsu
Software Engineering Specialist || BP

52
● Go to www.menti.com
● Joining Code - 8557274
Interactive Quiz

53
● First-ever MuleSoft Virtual Hackathon
https://www.mulesoft.com/hackathon2020
● Oﬃcial Rules and Prizes
https://www.mulesoft.com/events/mulesoft-hackathon-oﬃcial-rules
● Contest runs from Oct 22, 2020 at 9:00 am PT and ends on Nov 29, 2020 at
11:59pm PT.
● Register your spot now and All the best!!!
MuleSoft Hackathon

54
● Share:
○ Tweet your pictures with the hashtag #MuleMeetup
○ Invite your network to join: https://meetups.mulesoft.com/munich/
● Feedback:
○ Contact your organizers
● MeetUp Recordings:
○ Youtube Channel: MuleSoft Meetup Munich
https://www.youtube.com/channel/UCGp3GH-t2NPXG_YSWbnHgjw
● Next Topics:
○ Error Handling and Propagation in API-Led, Event-Driven and Batch Arch
○ Azure DevOps with MuleSoft
○ Reach out to Organizers to be a Speaker
What’s Next

55
● Go to www.menti.com
● Joining Code - 6563101
Feedback and Topics for Next Meetup

Munich MuleSoft Meetup Recap

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Munich MuleSoft Meetup Recap

Similar to Munich MuleSoft Meetup Recap (20)

Recently uploaded

Recently uploaded (20)

Munich MuleSoft Meetup Recap