CloudHub Networking Stategies & DLB setup and configuration demo

1. Thursday 22/04/2021
Rome MuleSoft Meetup Group
Networking Strategies on CloudHub

2. 2
Muleys, Meetup Leaders and… speakers:
Contacts:
● amartino@mulesoft.com
● sbenfari@mulesoft.com
Introductions

3. 3
● CloudHub Physical Architecture (50 mins)
● Demo time: Application traffic routing through Dedicated Load Balancer (30 mins)
● Q&A (15 mins)
● Trivia quiz (15 mins)
Agenda

4. CloudHub Physical Architecture

5. MuleSoft’s Anypoint Platform
Control Plane
Runtime Plane
Unified single solution for iPaaS and full lifecycle API
Control Plane:
MuleSoft Managed : 2 Control plane US (N Virginia)
& EU (Frankfurt or Dublin)
Customer Managed : Private Cloud Edition - PCE
Runtime plane:
This is where the APIs & Integrations are hosted.
CloudHub: MuleSoft Managed Runtime plane
Standalone Mule/RTF/PCE : Customer managed.

6. 6
CloudHub
(Commercial/GovCloud)
Hybrid
(Standalone/RTF)
On-premise
(Private Cloud Edition)
Control Plane
Runtime Plane
Managed by MuleSoft
Managed by the Customer
(Data center)
Managed by the Customer
(3rd party Cloud)
MuleSoft’s Anypoint Platform
Deployment Model

7. Runtime Plane
Worker Worker
Mule
App
Mule
App
Worker
Mule
App
Worker
Mule
App
Worker
Mule
App
Worker
Mule
App
Anypoint CloudHub
Anypoint Platform
Control Plane
Internal & External APIs
RUNTIME
MANAGER
ACCESS
MANAGEMENT
MANAGEMENT
CENTER
VISUALIZER
ADVANCE
MONITORING
EXCHANGE
PARTNER
MANAGER`
API DESIGNER
DESIGN
CENTER
API
ANALYTICS
CLOUDHUB

8. Worker
Mule
App
Worker
Mule
App
Worker
Mule
App
Anypoint CloudHub Architecture
SHARED/DEDICATED
LOAD BALANCER
RUNTIME PLANE
HTTP/S
mTLS
mTLS
mTLS
mTLS
RUNTIME
MANAGER
ADVANCED
MONITORING
MANAGEMENT
CENTER
API
MANAGER
CUSTOMER VPC
CONTROL PLANE

9. Multi-tenancy Support and Logical Segregation
Business Groups & Environments

10. ● Business Groups provide a mechanism for delegating
management and administration of Anypoint Platform to
users within different business units or functions
● Business Groups provide complete isolation of resources
allowing for multitenant use cases within your Anypoint
Platform account
● Centralized administrators can create a multi-level
hierarchical structure of Business Groups and then
delegate users from the specific groups to be administrators
at BG level
Multi-tenancy Implementation on CH
Business Groups
Reference Documentation: https://docs.mulesoft.com/access-management/business-groups

11. 11
● Business Groups reside all within Master Org (root), as
part of Customer unique subscription
● Flexible model - it enables delegation at different level
○ Certain Core Functions such as SSO and common
artifacts (i.e. libraries) are setup at the root org
level and shared across Business Groups.
○ Infrastructure Components can be defined centrally
or specific to a Business Group.
○ Operations and Deployments are local to a
Business Group.
● Business Group level environments are logical
boundaries and map to shared/centralized
infrastructure without need for replication.
Multi-tenancy Implementation on CH
Business Groups
Reference Documentation: https://docs.mulesoft.com/access-management/business-groups

12. 12
Root Org
LoB 1 (BG)
LoB 2 (BG)
LoB 3 (BG)
VPC Sandbox VPC PROD
Dev Test UAT PROD
Side note - APIs in different VPCs can still communicate each other (depending on networking rules implemented)
Dev Test UAT
Dev Test UAT
PROD
PROD
Multi-tenancy Implementation on CH
Business Groups & Environment Segregation - Typical Setup

13. Anypoint Resources (i.e VPC or redistributable entitlements like vCores or Static IPs) can be created at any level of the
Anypoint Organization structure but can only be shared vertically down the Anypoint Organization structure, not up or across
Example 2: VPC created in Business Group B
Example 1: VPC created in Business Group A
Example 3: VPC created in Master Organization
Multi-tenancy Implementation on CH
Resources Inheritance in a Hierarchical Org Structure

14. CloudHub Virtual Private Cloud

15. 15
Isolated network segment specific to a customer hosted in
our AWS account and managed by MuleSoft
● Workers are deployed into this network segment and assigned an
internal IP address within the address space determined by the
customer (CIDR block)
● Can be connected to customer’s data center or one of their private AWS
VPC (via peering)
● The base Anypoint VPC subscription includes two Anypoint VPCs
● Each Anypoint VPC can be associated with multiple environments
(typically 1 VPC for PROD and 1 for SANDBOX)
● It allows you to configure firewall rules to apply to your workers
● Regional service (it must be binded to a specific Region)
CloudHub Networking
Virtual Private Cloud (VPC)
Reference Documentation: https://docs.mulesoft.com/runtime-manager/virtual-private-cloud
Availability Zone
1
Availability Zone
2
Availability Zone
4
Each VPC has its
own firewall to
gate access
Region
Mule Worker Mule Worker Mule Worker
Mule Worker Mule Worker Mule Worker
A VPC is
created within
an AWS
Region
A VPC can span up to 4
Availability Zones
A VPC can have a
maximum of 64K IP
addresses
Non-Production VPC -
[10.1.0.0/16]
Production VPC -
[10.1.1.0/24]
A VPC must have minimum
of 256 IP addresses

16. 16
CloudHub Networking
Virtual Private Cloud (VPC) Sizing
Proper sizing has to be performed upfront (no possibility to change it later)
Supported CIDR blocks /24 - /16
No costs associated to over provisioning
For each worker deployed to CloudHub, the following IP assignation takes place:
● A few IP addresses are reserved for infrastructure
● At least two IP addresses per worker to perform at zero-downtime
General Rule of Thumb - 10 IPs per Mule Application (interface)

17. High Availability & Disaster Recovery

18. CloudHub offers several features for redundancy and reliability, which
includes reliability across data centers within a region
● Multiple workers for the same application are distributed across
two or more data centers.
● Apps are automatically restarted whenever they fail.
● The load balancer directs traffic to other workers if a worker is
down
If an application uses a single worker, when the availability zone is
unavailable, CloudHub automatically restarts the application in a
different availability zone. In this case, the application might
experience downtime.
Important: define upfront the Business Critical apps which can’t
tolerate service disruption and allocate them multiple Workers!
High Availability
CloudHub Deployment Model & Self-healing Mechanisms

19. Disaster Recovery
19
CloudHub Global Infrastructure - DR across Regions
Global Infrastructure relying on AWS Regions
Automatic disaster recovery across regions is not offered by CloudHub
If a Region goes down, the applications within the region are unavailable and not
automatically replicated in other regions.

20. Disaster Recovery
20
CloudHub Global Infrastructure - DR across Regions
Mitigation Strategy
Multi-Region deploy (active-active, active-passive, pilot fire)
It requires
● External Load Balancer provisioned by Customer (i.e. AWS ELB)
● Setup of VPC and VPN connectivity across the different regions
w/ local DC
● Fully stateless Apps to avoid message loss (AMQ, ObjectStore,
VM Queues are regional services)

21. Connectivity Options with On-prem

22. CloudHub Networking
Connectivity Options

23. CloudHub Networking
Virtual Private Network
Connecting to your Anypoint VPC via VPN extends your corporate intranet and allows CloudHub workers to access
resources behind your corporate firewall.
● Anypoint VPN supports site-to-site Internet Protocol security (IPsec) connections. A physical or software
appliance, called a VPN endpoint, is the terminator on your side of the connection. The MuleSoft side of the
connection is an implementation of a virtual private gateway (VGW)
● MuleSoft VGW can support up to 10 VPN connections per VPC
● Supports Dynamic (BGP protocol) and Static Routing - BGP is the preferred option
● Anypoint VPN acts only as a responder (you must initiate traffic to open tunnel - see KB how to generate
interesting traffic)
● Each Anypoint VPN connection consists of two tunnels for HA
● MuleSoft VGW implementation supports a maximum throughput of 1.25 Gbps (shared across all the VPNs
connected to the VPC)
● Anypoint Management Console allows to download device specific configuration
● VPN and Direct Connect can’t coexist on same VPC
● VPC CIDR block can’t overlap to Data Center IP range
VPN Requirements: https://docs.mulesoft.com/runtime-manager/vpn-about
KB: https://help.mulesoft.com/s/article/Anypoint-VPN-Knowledge-Articles
Test / troubleshoot connectivity (KB) here

24. Load Balancing Strategies

25. Load Balancing
25
CloudHub Shared Load Balancer (default)
• This type of load balancer sits outside of the Customer’s VPC and it is shared between all the
CloudHub customers
• There is one SLB clustered instance in each CloudHub region that serves all the CloudHub customers
in that AWS region (limited throughput respect DLB)
• SLB can only be used to load balance calls for the external-facing APIs (internal LB through HTTP
Connector or Messaging patterns)
• It supports HTTPS, but it is not possible to use custom TLS certificates
• Impossible to setup vanity domains

26. Shared Load Balancer (default)
Region AWS Region
CloudHub Shared VPC
Mule Worker
appEthel
[us-e2]
[cloudhub.io] CloudHub Domain Name
Shared Load Balancer
[us-e2.cloudhub.io]
SLB Domain Name
<app_name>.<region>.cloudhub.io
appEthel.us-e2.cloudhub.io
Standard FQDN template
1
HTTP/S Client
http://3.23.92.132:8081
http://appLucy.us-
e2.cloudhub.io
4
http://mule-worker-appLucy.us-
e2.cloudhub.io:8081
5
Default mapping rules for the
Shared Load Balancer
3
appLucy.us-e2.cloudhub.io:80 mule-worker-appLucy.us-e2.cloudhub.io:8081
appEthel.us-e2.cloudhub.io:443 mule-worker-appEthel.us-e2.cloudhub.io:8082
When an application is deployed to CloudHub, the application
gets several public DNS records.
2
Mule Worker
[3.23.92.132]
appLucy
Mule Worker
[3.12.83.228]
appLucy
A CNAME record(alias) to the shared load balancer of the region where the app is deployed to.
A records for all public IP addresses of the shared load balancer
A records for all public IP addresses of the CH workers running the app. An app can be directly
accessed (i.e., the SLB can be bypassed) by pre-pending "mule-worker-" to the FQDN of the app.
A records for all private IP addresses of the CH workers running the app. An app can be directly
accessed from other apps within a customer VPC by pre-pending "mule-worker-internal-" to the
FQDN of the app.
appLucy.us-e2.cloudhub.io. 60 IN CNAME us-e2.cloudhub.io.
us-e2.cloudhub.io. 60 60 IN A 3.130.220.225
us-e2.cloudhub.io. 60 60 IN A 3.22.15.255
us-e2.cloudhub.io. 60 60 IN A 18.221.57.233
mule-worker-appLucy.us-e2.cloudhub.io. 30 IN A 3.23.92.132
mule-worker-appLucy.us-e2.cloudhub.io. 30 IN A 3.12.83.228
mule-worker-internal-appLucy.us-e2.cloudhub.io. 60 IN A 172.25.6.221
mule-worker-internal-appLucy.us-e2.cloudhub.io. 60 IN A 172.25.92.140

27. Load Balancing
27
CloudHub Dedicated Load Balancer (Subscription req.)
• The Dedicated Load Balancer sits inside the Customer’s VPC
• It is possible to configure more than one DLB in a VPC
• Unlike SLB, a DLB instance can be used for load balancing internal traffic across the workers within a
VPC.
• It supports Custom TLS Certificates (including 2-Way TLS Authentication)
• It allows to define proxy rules - Vanity Domain Names
• It allows to enforce IP Whitelist/Blacklist

28. ● A DLB is an optional add-on entitlement
● A DLB typically has higher throughput capabilities than a Shared Load Balancer
● A DLB supports custom SSL certificates and two-way SSL (Mutual SSL)
● A DLB supports proxy rules for mapping applications to custom domains, so, for example, everything can
be hosted under a single vanity domain
● A DLB is deployed inside of a particular Anypoint VPC
■ A VPC can have more than one DLB
■ A DLB cannot span multiple VPC
● A DLB is comprised of one or more load balancer units but can be scaled
■ A single load balancer unit is comprised of two workers
■ You can assign a maximum to 4 load balancer units to a DLB
■ A fully maxed out DLB will have 8 workers
Load Balancing
CloudHub Dedicated Load Balancer (Subscription req.)

29. Using a Single DLB
● Provides network isolation of Mule applications within a VPC
● Governs which internal endpoints can be accessed via customizable mapping
rules
● Supports IP Whitelisting to govern which networks can access the DLB
● Performs load balancing of internal traffic across workers within a VPC
● Offloads TLS processing by terminating TLS connections at the DLB

30. Dedicated Load Balancer Architecture
http://appLucy.us-
e2.cloudhub.io
7
[cloudhub.io] CloudHub Domain Name
Region AWS Region
[us-e2]
Customer VPC [10.100.1.0/24]
DNS records for the
Dedicated Load Balancer
3
HTTP/S
Client
https://my-
dlb.lb.anypointdns.net/appLucy
6
Mule Worker
appLucy
Mule Worker
appLucy
Mule Worker
appEthel
● FQDN for the apps the same as in SLB architecture.
● DNS entries are similar as in SLB architecture, except
internal addresses will be from the 10.100.1.0/24 pool.
Default mapping rules for the
Dedicated Load Balancer
4
Default mapping rules for the Shared Load Balancer
5
Shared Load Balancer
[us-e2.cloudhub.io]
<lb_name>.lb.anypointdns.net
my-dlb.lb.anypointdns.net
Standard FQDN template
for the DLB
2
Dedicated Load Balancer
[my-dlb.lb.anypointdns.net]
Default firewall rules for the customer VPC
1
Source CIDR Dest. Port
10.100.1.0/24 8091
10.100.1.0/24 8092
0.0.0.0/0 8081
0.0.0.0/0 8082
my-dlb.lb.anypointdns.net. 60 IN A 3.130.220.225
my-dlb.lb.anypointdns.net. 60 IN A 3.22.15.255
internal-my-dlb.lb.anypointdns.net. 30 IN A 10.100.1.21
internal-my-dlb.lb.anypointdns.net. 30 IN A 10.100.1.105
appLucy.us-e2.cloudhub.io:80 mule-worker-appLucy.us-e2.cloudhub.io:8081
appEthel.us-e2.cloudhub.io:443 mule-worker-appEthel.us-e2.cloudhub.io:8082
my-dlb.lb.anypointdns.net:8080/appLucy mule-worker-internal-appLucy.us-e2.cloudhub.io:8091
my-dlb.lb.anypointdns.net:443/appEthel mule-worker-internal-appEthel.us-e2.cloudhub.io:8092

31. Using Multiple DLBs
● Provides for even greater network isolation of Mule applications within a VPC
● Provides for a flexible governance and load balancing architecture
○ Use one DLB to expose and load balance external facing APIs (e.g., Experience
APIs)
○ Use a second DLB to hide internal APIs (e.g., Process and System APIs) but still
allow internal APIs to be load balanced.
○ Naming convention and patterns allow for dynamic routing and protection of internal
APIs
KB: https://help.mulesoft.com/s/article/How-to-dynamically-restrict-external-access-to-specific-APIs-using-Dedicated-Load-Balancers

32. [cloudhub.io]
Region
[us-e2]
Customer VPC [10.100.1.0/24]
Process
requests from
anywhere
4
Multiple Dedicated Load Balancers Architecture
Mule Worker
Exp. API
Mule Worker
Exp. API
Mule Worker
Exp. API
Mule Worker
Process API
Mule Worker
Process API
Mule Worker
Process API
Mule Worker
System API
Mule Worker
System API
Mule Worker
System API
5
Process requests only
from VPC addresses
Default mapping rules for each DLB
3
● All applications within the VPC should be exposed on either port 8091 for HTTP or 8092 for HTTPS
● All applications have internal addresses from the 10.100.1.0/24 pool.
1
/{app} mule-worker-internal-{app}.us-e2.cloudhub.io:8091
Internal/Private DLB
[int-dlb.lb.anypointdns.net]
External/Public DLB
[ext-dlb.lb.anypointdns.net]
Firewall rules for the
customer VPC to allow traffic
from only within the VPC
2
Source CIDR Dest. Port
10.100.1.0/24 8091
10.100.1.0/24 8092
Whitelisted CIDR
0.0.0.0/0
Whitelisted CIDR
10.100.1.0/24
Shared DLB
[us-e1.cloudhub.io]

33. Demo time: Application traffic routing
through DLB

34. Prerequisite - Permissions Required
Runtime Manager> Read applications + Create Applications + CloudHub Network Admin

35. Prerequisite - Org Entitlements
The Anypoint Organisation must included at least following entitlements:
● 1 VPC
● 1 Dedicated Load Balancer
● At least 0.1 vCores allocated

36. Prerequisite - Development Tools
A certificate generation tool e.g. Open SSL
A REST client e.g. Postman or Advanced REST Client
Anypoint Studio

37. Creating a VPC
1. Provide a name
2. Choose the AWS deployment region
3. Provide a CIDR (must not clash with any
internal network ranges if relevant)
Choose environments to be included in the
VPC
1. Optionally share with sub BGs
Documentation Reference
CIDR Size Reference

38. Configure Firewall Rules
In order to block public inbound traffic remove
the Anywhere 0.0.0.0/0 rules
Note that all internal traffic will use port 8091
for HTTP and port 8092 for HTTPS
Documentation Reference

39. Creating a DLB - Prerequisites
A VPC must already exist
A certificate key pair in PEM format
How to create a key pair using SSL

40. Creating a DLB
1. Provide a name
2. Choose the VPC to associate with
2 workers is default - each DLB license allows
for 2 workers
1. Whitelist the IPs to accept traffic from in CIDR
notation. Anywhere is default

41. Creating a DLB
Choose how HTTP requests should be handled
Additional optional configurations (leave defaults)

42. Creating an SSL Endpoint
Upload the public and private keys
If mutual TLS was required a client cert can be added

43. API Naming and URL Mapping Rules
A common use case customers have is to make only certain APIs publicly accessible. For example,
we may want to make our Experience APIs available to the public internet but secure our Process
and System APIs within the VPC.
This can be achieved by defining a naming convention for ‘public’ and ‘private’ applications, and
then applying mapping rules based on this convention.

44. Configuring URL Mapping Rules
In the example below, we map all incoming requests to applications with the naming convention exp-
{app}
This means only Experience APIs will be accessible via the DLB
Default Mapping
Rules
Custom
Mapping Rules

45. Creating a DLB
Once all configuration options are complete create the DLB and wait for it to start

46. Testing a VPC and DLB Configuration
Create a simple Mule app which listens for HTTP requests
Ensure the listener uses port 8091

47. Testing a VPC and DLB Configuration
1. Deploy the app to CloudHub
2. Ensure the region matches the VPC region and the environment is included in the VPC
3. Ensure the HTTP port is 8091 (8092 for HTTPS listeners)

48. Testing a VPC and DLB Configuration
In order to send a request, we must call the DLB endpoint/app-name, keeping in mind any mapping
rules e.g. mapping rule adds exp- prefix, to the request must not include this prefix
*If a Chrome security alert appears, typing “this is unsafe” while on the page will bypass the warning

49. Q&A

50. What’s next

51. 51
● Share:
○ Tweet using the hashtag #MuleSoftMeetups
○ Invite your network to join: https://meetups.mulesoft.com/rome/
● Feedback:
○ Fill out the survey feedback and suggest topics for upcoming events
○ Contact MuleSoft at meetups@mulesoft.com for ways to improve the program
What’s next?

52. Thank you