In this session, you will learn how MuleSoft customers can establish a pragmatic C4E to accelerate delivery, but then leverage platform insights to drive continuous quality into your API ecosystem and your organization
Speaker: Steve Clarke
Facilitator: Angel Alberici
5:42 Introduction
11:18 Part 1 – A pragmatic way of C4E delivery
1. Quick refresh on what a C4E is and its role in API delivery
2. Core capabilities to focus on in C4E Launch
3. Key outcomes you can look for at launch and beyond
33:36 Part 2 – Metrics Insight
1. Planning your delivery of a Metrics solution
2. Identifying key KPI’s to measure
3. How those KPI’s tie back to C4E and API Delivery Maturity
4. Delivering your solution
5. Monitoring, Measuring, Feedback
46:50 Part 3 – Bringing it together
57:18 Summary
59:40 Q & A
📝 Slides and recordings 🎥 : https://meetups.mulesoft.com/events/details/mulesoft-online-group-english-presents-pko-driving-value-into-your-api-delivery-through-c4e-and-platform-metrics/
👤 Watch all meetups here: https://meetups.mulesoft.com/online-group-english/
📝 Read all slide decks here: https://www.slideshare.net/AngelAlberici
2. 2
Intros
Agenda
Part 1
Pragmatic C4E
Part 2
Metrics Insight
Part 3
Bringing
It
Together
● Quick C4E Refresh
● Core Capabilities at
Launch
● Key outcomes to look for
● Thinking beyond MVP
● Practical Tips for Success
● Planning delivery of your
Metrics Solution
● Identifying key KPI’s
● Tying KPI’s back to C4E
and API Delivery Maturity
● Delivering your solution
● Monitoring, Measuring,
Feedback
● Making it work in practice
● Creating a virtuous circle
4. 4
v Australian born, dual Oz/UK national
v 25 years total experience
v Worked globally across Asia/Pac, EMEA, US, Africa and places in-between
such as a few pacific islands that are barely visible on a map!
v 22 years in systems integration across every tech stack imaginable – Oracle,
IBM, APIGEE – and MuleSoft
v Been using MuleSoft for nearly 6 years – dropped into it at the deep end a
client wanting a POC ! Went from there J
v 6 x MuleSoft Certified (MCD / MCPA / MCIA)
v MuleSoft Champion and Mentor
v Enjoy riding the elevator from C-Suite to Engine Room
v Always hands on with the technology
About the Speaker
Steve Clarke
https://www.linkedin.com/in/steverclarke/
steve.clarke@slalom.com
Practice Director – Integration | APIs | MuleSoft – Slalom UK/EMEA
5. 5
Slalomisamodernconsulting firm
focusedonstrategy, technology,
andbusinesstransformation.
S T R A T E G Y
Redefine what’s possible
Go beyond the expected. We help you move
confidently through ambiguity and risk,
focus on your customer needs, and deliver
sustainable business value.
We analyze, architect,
and co-create
Learn from your data, create incredible
digital experiences, or make the most of
new tech. We blend design, engineering,
and analytics expertise to help you build
the future.
T E C H N O L O G Y
Enact lasting change
New technologies. Shifting customers.
Industry disruption. Business moves fast,
and we specialize in working through
change with you.
T R A N S F O R M A T I O N
6. 6
Personal
connection at
global scale
THRIVING WITH OUR NEIGHBORS
In 42 markets with 9,000+ consultants
across the US, Canada, Europe, and
Asia Pacific, we live and work side by
side with our customers in our own
communities while having the depth and
resources of a global company.
Office + Salesforce Hub Office
We’re proud to be Salesforce’s #3 partner
in North America and #4 globally.
7. 7
Expertise
that powers
innovation
• Marketing Cloud
• B2B Commerce
• Customer 360 platform
• Integration/Mulesoft
• Public Sector
• Revenue Cloud
• Sales Cloud
• Service Cloud
• Analytics (Tableau and
Einstein)
• B2B Commerce
• B2C Commerce
• Community Cloud
• Customer 360 Platform
• Field Service Lightning
• Heroku
T O P 1 0
C O N S U L T I N G
P A R T N E R
• B2B Commerce
• B2C Commerce
• Consulting Ecosystem
• Consumer Goods
Cloud
• Field Service Lightning
• Financial Services
P A R T N E R
A D V I S O R Y
B O A R D
• Health Cloud
• Marketing Cloud
• Pardot
• Sales Cloud
• Services Cloud
• Tableau
N A V I G A T O R
E X P E R T
As a leader in every area of
the Salesforce ecosystem
we help our clients connect
the dots to achieve the
extraordinary.
• MuleSoft
• Maps
• Marketing Cloud
• Public Sector
• Revenue Cloud
• Sales Cloud
• Service Cloud
• Vlocity
service cloud
sales cloud marketing cloud community cloud platform
financial services cloud
health cloud einstein B2B commerce cloud
C360 audiences
B2C commerce cloud
maps
voice
quip
loyalty cloud
revenue cloud consumer goods cloud
communications cloud
experience cloud
CRM
enterprise architecture
field service lightning
datorama
interaction studio
chatbots
apps
CLOUD LEADERSHIP
8. 8
Capability Highlights
Integration and API Strategy
Working with you to understand and frame the Integration and Connectivity
challenges within your organization and shaping an API led strategy via
MuleSoft that will lead to rapid success.
MuleSoft Accelerators
Establishing the foundational People, Process and Technology capabilities
needed to support and accelerate your API ecosystem with MuleSoft at the core
MuleSoft Platform Delivery
Providing Architectural, Build and Delivery services to rapidly accelerate your API
journey with MuleSoft and unlock value across your enterprise.
API and Platform Assurance
Capabilities and services designed to enable your MuleSoft API lifecycle and ensure
your are continually delivering the highest quality API’s for your business, customers
and partners.
API Insights and Optimization
Gaining insight into the operational effectiveness of your platform investment and
leveraging core MuleSoft and Third Party solutions to unlock and optimize value in
your API ecosystem.
Slalom is Focused
on MuleSoft
We define integration strategies, deliver solutions and are considered thought
leaders in the spaces of Enterprise Integration, Application Programming
Interfaces, Platform-as-a-Service and Microservices. Slalom has implemented
MuleSoft across an array of diverse customer use cases ranging from
ERP/CRM Integration, Legacy Modernization, Cloud Adoption and Data
Synchronization through to Customer 360 initiatives.
We take considerable pride in not only launching new customers on their
MuleSoft journey, but also accelerating existing MuleSoft customers derive the
most value from their platform investment to support their business.
82+
MuleSoft projects
completed
66+
MuleSoft
certifications
60+
MuleSoft
practitioners
30+
Slalom offices
worldwide
#5
MuleSoft Partner in
North America
10. 10
Nutshell: What is a C4E ?
Best Practices, Tools,
Methods for
MuleSoft API
delivery in your
Organisation
Continuous Quality
Improvement
+
Agility / Time to
Market
API delivery
mindset evolution
+
Evangelize
+
Capability
development
A Centre for Enablement (C4E) plays a key role in evolving, assuring, guiding and governing the
evolution of your API landscape using MuleSoft. It comprises the people, process and technology
enablers that allow your organisation to evolve a thriving API ecosystem.
11. 11
A C4E offers real value – if you can measure that value
…but it takes time
12. 12
API Delivery End Game – Vicious vs Virtuous thinking
API’s can be equated to a ‘currency’ in an organisation. To make that currency of value, one needs
to provide a structure that can pivot your organisation toward a ‘virtuous’ delivery mode, rather
than ‘vicious’.
A virtuous delivery circle can continually add to your API ecosystem value so the adoption of a
C4E that is self improving is what you want to aim for.
13. 13
Launching your C4E (think MVP)
When starting out, adopting a centralized – API Factory – type
model can offer a lot of benefits as you move to future state
(especially if MuleSoft is new to the business) :-
ü Define your Vision, Mission, Charter and Roadmap
ü Shape your foundational team
ü Identify skills, strengths, weaknesses, opportunities
ü People – buddy up new with experienced hands
ü Develop your foundational standards, baselines, ways of
working
ü Establish your Governance artefacts – Integration Catalogue,
API Registry, NFR’s, Templates, etc.
ü Validate through several sprints, refine and revise, build speed
ü Think about next step evolution…..
Scrum Master
API Factory Lead
API Lead
API Developers API Designers API QAs
14. 14
Think C4E Structure beyond your MVP
C4E Lead API Lead
C4E
API Developers
API Designers
API QAs
API Developers
API Designers
API QAs
API Developers
API Designers
API QAs
API Developers
API Designers
API QAs
API Architect Consultant API Developers
API delivery capability is embedded within the Spoke teams, working under the direction of the relevant Product Owner.
Consultant API Developers from within C4E to provide specialist technical support as required
LOB LOB LOB LOB
15. 15
C4E – Establish your Vision and Charter – Not all C4E’s are alike !
• Define Standards, Processes, Procedures, Templates, Integration
Patterns, Governance Framework & Ways of Working
• Govern MuleSoft releases ensuring compliance with the specified
processes, standards and QA strategy
• Control, manage and review API Roadmap, Assets in Exchange, etc.
• Advise. Provide consultancy and technical expertise into the Spoke
Teams
• Maintain and monitor the platform e.g. vCore capacity, Re-usability
• Coach and develop spoke team skills in the use of the MuleSoft platform
C4E
Team
Control
Govern
Define
Advise
Maintain
Coach
Target State
16. 16
Develop your roadmap….
•Initial MuleSoft engagement
•Limited platform knowledge
•Simple business use case selected
for pilot
•Limited resource
•Basic staff training
•Basic infrastructure implementation
Catalyst Launch
•Integration principles
•Integration patterns
•REST Standards
•MuleSoft knowledge acquisition
•Rest Standards definition
•Template creation
•Process definition
•Formal training
•Defined roles and responsibilities
•Additional delivery resource on-boarding
•Professional service engagement
•CD / CI pipeline development
•Application Network Development
•IT Service Transition & Support Model
•Formal delivery pipeline
•KPIs
•NFRs
API Factory
•De-centralised delivery model
•Customer MuleSoft specialists working
directly within spoke teams
•MuleSoft delivery capability embedded
within each delivery spoke
•Centralised governance
•Organisational processes, templates
and standards
•API discover and reusability
•API-led connectivity
•Minimum level of technical
competence.
C4E
Stage 1
Months 1 - 3
Stage 2
Months 4-6
Stage 3
Months 6+
Establishing a realistic roadmap is key to ensuring you can hit the ground running – but
deliver value and evolve your capability at the same time.
Its important that this is well thought out before attempting to devolve across the
business
17. 17
Identify
Integrations
Identify API's
Design
Prototype
Validate
Develop
Test
Deploy
Operationalise
Publish
Feedback
Identify business processes requiring
integrations. Verify if existing in Integration
Catalogue or add as new.
Break down integrations into
API’s. Check API Registry for
re-use or add new.
Design API technical contract or
extend existing API.
Mock the API and test the mock
Validate the API design with consumers /
peers for good design and functionality / data
needs.
Develop the API implementation
or extend existing API
Develop mUnit Tests
(>70%) to cover Functional
/ Non-Functional scenarios
Deploy API to Mulesoft using
Runtime Manager or DevOps
pipeline.
Publish API’s to Exchange for
reuse/discover and also specific
portals for specific communities
(e.g. External Partners)
Operationalise API by applying
policies, configuration etc.
Support operations through analytics
from runtime and other sources to
inform usage, quality of service,
improvements, etc.
Socialise through API Exchange
or your Partner / Dev Portal.
API delivery should be seen as a continuous
cycle. API’s may be iterated over numerous times
during their lifetime, extended or updated as their
usage evolves.
Ensuring governance throughout this lifecycle will
optimize reuse and also allow alignment between
API’s and the Integrations (business processes)
they fulfil.
Within each phase, it is possible to iterate several
times until ready to progress to the next.
Cloudhub
API
Lifecycle
Mapping Project
Delivery to MuleSoft
Capability
18. 18
Managing Scope, Guiding Governance through good artefacts
Design
Build
Operation
Discovering
Retirement
Design (decisions
about data
format, security
Integration,
Implementation
and
Deployment
Operation,
Configuration,
monitoring,
analytics
Enabling API’s to be
discoverable, accessible,
reusable, etc. by end user
applications
Deprecation and
Version uplift
API Registry
Integration Catalogue
+ Mud Map
Examples included at the end of this presentation
19. 19
API Governance – Winhearts andminds
through clearly defined explanations
An essential success factor in taking product
owners, architects, developers and other
stakeholders on the API led journey is
winning hearts and minds around the vision
of your API ecosystem.
To achieve this, consider taking them through
how end to end integrations are broken down
into the API’ that enable them.
Often, API’s are understood as to what they
are , but often what’s not explained is how
one gets there in a structured, predictable,
reusable way.
By taking people through that journey, your
C4E can become the enabler that helps you
achieve that virtuous lifecycle you need it to
be.
API-001
API-002
API-003 API-004
Consumer
Application
#1
System of
Record #1
System of
Record #2
Consumer
Application
#2
API-005
INT-002
INT-001
20. 20
Definition of Ready / Definition of Done – Design and Build
Build Architect / Design
In order to create “effective habits” in delivery, the following constructs were developed to coach and govern
deliverables.
These largely revolved around defining the required artefacts or deliverables around:
- Design - Definition of Ready (DoR) and Definition of Done (DoD)
- Build - Definition of Ready (DoR) and Definition of Done
21. 21
Build out Templates and Accelerators (build in predictability)
Templates and Accelerators consisted of:
ü Tips and Tricks
ü Configuration Guides
ü Getting Started Guides for Anypoint Studio
ü Mule Templates – Published to Exchange
ü Principles – Overall Integration and API Led Principles
ü Patterns – EAI + API Centric
ü Standards – REST, Dataweave, Flows
ü API Sizing and Story Pointing Metrics / Methodology
ü Platform NFR’s and Candidate Solutions or Patterns
(aligned to Enterprise NFR’s)
ü JIRA templates
ü Logging Frameworks (log4j)
ü CI/CD Frameworks
Pro Tip: Align your templated backlog tasks to DoR / DoD for Design and Build. Clone
these ahead of or during Sprint Planning to accelerate readiness.
23. 23
What are the Benefits ?
• MVP and Strategic KPI's which provide insight into
the operational effectiveness of the platform
• Rich metric data is readily available from the MuleSoft
platform
• Provision of a set of configurable dashboards which
will enable regional and (eventually) global insight into
platform operations and usage
• Configurable approach, patterns and templates
needed to add further sources and KPI’s across a
single pane of glass
• Insight for senior stakeholders and product owners
into the effectiveness of their API products on the
platform
• End Game – a single pane of glass
24. 24
HighOrder Bit - What C4E KPI’s doyouwant tomeasure, from where andwhy ?
25. 25
What MuleSoft Platform data might we capture?
Production
Monitoring
Sandbox
Events, Runtime Statistics
Logging
Basic Alerts
● For Applications and Servers
from Anypoint Monitoring
API Alerts
● Policy Violation
● Request Count
● Response Code
● Response Time
● Contract alerts
Runtime Alerts
● CPU Usage
● Deployment Failed
● Worker not Responding
● Memory Usage
● Custom Application
Notifications
● (Error notifications &
Operational Milestone alerts)
Functional API
Events
Notification
Ticketing
Elastic Stack for API Governance, Platform
Monitoring (Events) and Product Owner
Insights
API Functional Monitoring
● Functional Monitoring of API endpoints.
(Health check & SLA)
● Monitor public & private end points
● Integration with ServiceNow ticketing
system
Elastic Stack
● Single pane of glsss
● Routine monitoring of apps & apis (via
Runtime Manager API’s)
● Infrastructure Monitoring ( JVM, Heap,
Memory Utilization )
● Performance Monitoring of apps
● Governance, Assurance and Insights into
Usage
● Leverages the Platform API’s
Governance KPI’s
● API Policies Enforced
● Exchange Content
Completed
● Tags, Domains, Categories
assigned
● Degree of Reuse
● Domain Usage
● Deployment Velocity
26. 26
What’s the best tooling to adopt ?
ü MuleSoft Metrics Accelerator
• Based on Open Source
• Pulls all key metric data via the Platform API’s
• Supports a number of commonly used analytic platforms including Splunk and Elastic
Search
• Runs on MuleSoft – easily deployable
• Easy to extend to other target analytic solutions
• Doesn’t support data from other sources – but can be easily extended (think API’s !!!)
• Allows you to achieve a ‘single pane of glass’ analytic outcome
• How to - https://blogs.mulesoft.com/api-integration/strategy/measure-the-value-of-your-
anypoint-platform-implementation/
ü Anypoint Monitoring
• Needs specific licensing
• Out of the box
• Single pane of glass
• Hard to push external metrics in
Regardless of which solution you pick, a goal should be to achieve a ‘single pane of glass’ solution.
This will ensure your C4E customers have ease of access to the insights they need across the
platform. Value will be achieved quicker.
27. 27
Consider the designof your target solution
• Consider your data sources (MuleSoft, ServiceNow, JIRA, etc)
• How do they relate to each other ?
• How can you tie them together through search, KPI’s and Visuals ?
• Configurable to add new data sources from MuleSoft or elsewhere
• Support an cloud/on-premise MuleSoft solution
• Deal with global requirements
• Source mappings to KPIs
• Reliability – think DR and rebuild
28. 28
Key Delivery Challenges youmay encounter
• Mapping of KPI’s to data sources
• Need to form MVP of core searchable data
• Target KPI Stack application design knowledge
• Platforms access - Roles and Permissions
• Source Data Structures
• Implementation decisions
29. 29
Candidate Architecture (think Metrics Accelerator but extended)
Metrics Accelerator
Validate / Enrich / Transform / Structure Data
Event
Queue
Poll
Data
Source
ELK Rest Connector ELK Log Connector
Process Flow
Elastic Search
REST API File Beat
Poll
Data
Source
Poll
Data
Source
Poll
Data
Source
Exchange
API
Runtime
API
Platform
API
Poll
Data
Source
Mule App
Events
Poll
Data
Source
31. 31
Example #1: MeasuringUsage of the Platform
31
• The What ?
• Organisations and Business Groups – who
are your consumers ?
• Number of reusable assets on Exchange
• Total number of reusable API’s
• Registered or active users of your platform by
type, role, business group, etc.
• The Why ?
• Understand the usage of your platform,
where its high, where its needing C4E focus
• Monitor asset growth over time
• Monitor and assist usage over time – who are
your ‘customers’ ?
33. 33
Example #2 : API Usage by Organization, TopUsedAPIs
33
• The What ?
• Usage by different parts of the business
• Degree of reuse by API or Category
• The Why ?
• Low reuse can give insight into how
discoverable, flexible an API
• Low reuse can help identify where consumers
are seeing gaps in your API ecosystem
• Reuse monitoring can help identify
opportunities for further business enablement
34. 34
Example #3: API by Business Domain
34
• The What ?
• Breakdown consumption by domain, type and
rating
• The Why ?
• Provides insight into usage across domains
• Insight into reusability across the enterprise vs
domain specific assets
• Insight into consumer feedback to drive further
enablement
35. 35
Example #4: UsingKPI’s for Governance andQuality Improvement
35
• The What ?
• Identify gaps in Design, Non-Functionals,
Testing, etc.
• Monitor effectiveness of standards, principles,
patterns and their implementation
• Ensure platform configurations which enforce
NFR’s are being applied (e.g. Policies)
• Ensure Exchange documentation is up to
date
• The Why ?
• Gaps in quality can affect delivery
• Gaps in Non-Functionals can expose API’s to issues
around Security, Reliability, Scalability
• Non-Categorisation of assets can affect
discoverability, reusability
37. 37
Example #6: PlatformCapacity Monitoring
37
• The What ?
• Understand platform capacity usage and
consumption
• The Why ?
• Insight into platform consumption ensures effective
use of license
• It can also give insight into potential reuse
challenges or gaps in overall design strategy of the
API ecosystem (e.g. DDD)
38. 38
Example #7: Defect Management
38
• The What ?
• Cross align API insights with incident
management insights
• Reliant on integration with your incident
management platform (e.g. ServiceNow)
• The Why ?
• Insight into Build, Test and Runtime
defects across API’s, Severity, etc.
39. 39
Nutshell - Platform Insight + C4E can improve quality,
reduce cost and accelerate delivery
API delivery velocity can be
accelerated quickly with the
adoption of practical
templates, tools and ways
of working to aid architects
and developers.
But….
If you can’t measure how
effective your delivery is,
you can never be so sure
0
20
40
60
80
100
120
140
1 2 3 4 5 6
Story Points Delivered
40. 40
As an example, a recent project where a strong C4E and Platform Insights were combined
over time achieved the following outcomes:
40
44% Governance insights helped improve tagging,
categorization, documentation, etc., making
API’s easier to find, understand and consume
Improved discovery
60% Insights helped to significantly reduce build
and deployment errors and improve the
quality of DevOps over time
Reduce delivery
errors, harden
DevOps
30% Higher quality assets plus ease of discovery
increased reuse of API assets, especially System
API’s which were domain centric
Increased reusability
How good can it really be ?
65% Quality, Re-Use, Governance – all helped to
massively accelerate projects and increase
trust in the API ecosystem
Accelerate delivery