In this two-part series, Sumanth Donthi, Associate IO at AIG explains how AIG changed software delivery process with API Governance and how they improved operational efficiencies with GitOps.

1. Improving Agile Software Delivery and Operational efficiencies at scale.
API Governance and GitOps in Hybrid
Integration Platform (MuleSoft)
June 17, 2020
June 17, 2020

2. Sumanth Donthi
IT Tech AIO @ AIG
General Insurance IT
Mobile: 281-409-1108
Email: sumanth.donthi@aig.com
About Me
LinkedIn: https://www.linkedin.com/in/sumanth-donthi-924b5710/
Over 15 years of experience in Software Engineering
Leading Global Platform strategy, API Governance, Platform Engineering
and Operations for Hybrid Integration Platform (HIP) at AIG.
Specialties: Software Architecture, Program Management, Digital Transformation,
Platform Engineering, Scalable & Distributed systems, and DevOps at scale.

3. 1. API Governance
Efficient software delivery

4. 4 | Process Improvements
API Governance Outline
2 | Problems & Objectives
3 | API Metadata context
1 | API Governance
5 | Benefits

5. Problems
Wealth of Organization’s knowledge on APIs is scattered and
maintained in silos.
Incoherent Data
Unable to assess impact due to lack of understanding of dependencies, data
flow and owners leading to increased effort in troubleshooting and estimates.
Missing Context
Classic Systems & Processes
Inconsistent Standards
Teams using different tools and processes to document metadata leading to
inconsistent data and relationships of assets.
Inconsistent deliverables such as code standards from multiple vendors,
naming conventions of artifacts, design patterns.
Improve Operational Efficiency
Shared Repository
Easy access to contextual data
Ensure consistent deliverables
Objectives
Ensure Data consistency, integrity and supervise the collection and usage
of metadata. Standard deliverables, coding practices, naming conventions,
etc.
Facilitate the usage of contextual data within the Organization.
Enable Metadata Repository Management of technical, operational and
business assets.
Enable API Governance in a transparent, supportive and
collaborative manner.

6. o Set vision & strategy
o API discovery
o Business value
o API Lifecycle
Product Owner/Stakeholder
o API-led design
o Standardize API build
o API versioning
o Reusable assets
Designer/Developer
o Technology Enablement &
Foundational Services, Guidelines
o Policies, Procedures, Security
Standards
o Consistent deliverables
o Automation
o Auditing
Platform Owner
API Governance context
Different context across cross-functional groups
Operations
o API dependencies
o API owners
o API logging, tracing
o API monitoring
o Troubleshooting
o Auditing

7. API Metadata Context
Technical Operational Business
API Basic Info:
o Name
o Description
o Portal/provider's URL
o Lifecycle
o Architectural Style
o Technology used
o Scope
API Detailed Info:
o Authentication Scheme
o Transport Security Scheme
o Request/Response formats
o Specification URL
o Documentation URL
Assets & Metadata:
o Deployment Info
o Owner Info
o Release Info
o Monitoring Info
Relationships:
Assets relationship to
operational metadata
Assets & Metadata:
o Program Info
o Owner Info
o Geography/Locatio
n
o Business Function
o Category
Relationships:
Assets relationship to
business metadata
Assets & Metadata:
o API
o API Consumer
o Queues
o Queue exchanges
o Data Sources (DB)
Relationships:
o Consumer to API
o API to API
o API to Queue
o API to DB
Common understanding of metadata

8. Create SpringCloudConfig
Repository
Discovery & Assessment Provisioning Production Operations
DevOps create
service pipelines
Create Asset in
Anypoint Exchange
API Delivery Process - before
Inefficient software delivery
Discovery in silos and review
with Stakeholders
Catalog services
and email
Create API in
API Manager
SpringCloudConfig
Mule properties
Email1
2
Add code to new
Mule API Git
Repository
Platform Lead
manual work
Identify dependencies in
Silos and deal with
Inconsistent deliverables
1
2
3
4
9
6
5
8
7
1
0

9. o Add template code to new
Mule API Git Repository
o Create SpringCloudConfig
Repository
Discovery & Assessment Provisioning Pre-Release
Platform Engineer
Bootstraps an API
Create Asset in
Anypoint Exchange
Ensure API data quality and
consistency in Collibra
Compliance check for data
consistency in Collibra
API Governance Process - After
Efficient Software Delivery
Discovery & detailed assessment
review with Stakeholders
Open JIRA & enter API
Basic Info in Collibra
Create API in
API Manager
SpringCloudConfig
Mule properties
Add comments
to JIRA
Create
Pipelines
1
2
3
4
9
6
5
8
7
1
0

10. Information aggregated centrally resulted in common understanding.
Effective collaboration
Easier to find, faster to resolve issues and providing consistent data
driven estimates.
Easier to find contextual data
Enable service discovery & owners
Easy to Identify services and owners eliminating the need to create a
duplicate service which on an average takes about 2 weeks.
Simplified knowledge transfer between Software Delivery
and Support teams
Unified employee training
Improve service delivery to DEV
Enforce standard practices
Ensure consistent deliverables & Audit
API Governance Benefits
Ensure consistent Git repository naming, configurations, pipelines, data
integrity and supervise the collection and usage of metadata.
Enforce API Standards, coding practices, security policies,
logging, tracing, error handing practices.
Average time taken to setup service in DEV is 2 hours.
Developer productivity significantly improved as the focus shifted to writing
core functions instead of worrying of configurations and setup.
Self-service -- Bootstrapping API

11. 2. GitOps in Hybrid Integration Platform (MuleSoft)
Improving operational efficiencies at scale

12. 4 | Process Improvements
GitOps in HIP Outline
2 | Problems & Objectives
3 | Deployment footprint
1 | GitOps in HIP
5 | Benefits

13. Hybrid Integration Platform (MuleSoft)

14. Problems
Average cluster setup effort is 4 days.
Inconsistent cluster setup
On an average server configurations and rollout of patch
installs effort is 4 weeks.
Inefficient patch updates
Configuration drift
Increased operational cost
Manually updating configuration on the server.
Slower software delivery increased the overall operational cost.
Reduce operating cost
Faster Cluster provisioning
Faster Software patching at scale
Audit all configuration updates
Objectives
Complete visibility into all configuration updates.
Efficient software patch rollouts by removing redundant and repeatable tasks.
Reduce cluster provisioning time to hours.
Reduce cost maintain existing clusters and onboard new work loads.

15. 600+
2K+
6X
Mule applications across all
business functions
Mule Application Inventory
6X
Mule Application deployment
pipelines across all environments
Mule Application pipelines
6X
o Standalone-cluster setup
o MRE wrapper config updates
o MRE patch installs
o MRE Log4j updates
o Application log4j updates
Mule Configuration pipelines
4X
Different Mule runtime versions
on-premise and cloud.
Mule Runtime Inventory
400+
2K+
Deployment footprint
Q1 2018 Q1 2020

16. Register cluster to
Anypoint Runtime Manager
Assessment Provisioning Production Operations
Operations engineer
initiates request
Retrieve Mule certs
and install scripts
Mule cluster provisioning - before
Inefficient Mule setup
Environment availability
assessment between Developer
and Platform owner
Platform engineer
opens ServiceNow
attaching wrapper.conf
Create & configure
Mule cluster
Email
1
2
Manually retrieve Mule
binaries and License
Cross-functional groups
does not have visibility to
wrapper.conf updates
made on the server
Mule Cluster
1
2
3
4
9
6
5
8
7

17. Register cluster to
Anypoint Runtime Manager
Assessment Provisioning Production Operations
Operations
engineer initiates
pipeline
Retrieve Mule certs
and install scripts
Mule Cluster Provisioning- After
Mule configurations in Git
Environment availability
assessment between Developer
and Platform owner
Platform engineer creates
wrapper.conf in GIT
Create & configure
Mule cluster
Retrieve Mule binaries
Subsequent updates to
wrapper.conf is a GIT pull
request submitted to
Platform engineer
Mule Cluster
Retrieve Mule License and
Anypoint API credentials
Add comments
to JIRA
1
2
3
4
9
6
5
8
7
1
0

18. Assessment Provisioning Production Operations
Retrieve Mule certs and
patch install scripts
Mule Patch Installs - before
Inefficient Mule patch installs
Environment availability
assessment between Developer
and Platform owner
Platform engineer opens
ServiceNow attaching
Patch-Install instructions Apply patch manually
on Mule cluster
Email
Manually retrieve Mule
Patch binaries
No visibility into patches
applied on the server at
various stages
Mule Cluster
1
2
3
4
6
5
7
Operations engineer
initiates request4

19. Apply changes to cluster
Assessment Provisioning Production Operations
Operations engineer
initiates pipeline
Retrieve Mule certs
and install scripts
Mule Patch Install- After
Mule configurations in Git
Platform engineer opens JIRA for
internal tracking.
Platform engineer creates
patch-install scripts in GIT
Retrieve Mule patch
binaries
Subsequent updates to
patch install script is a GIT
pull request submitted to
Platform engineer
Mule Cluster
Retrieve Anypoint API
credentials
Add comments
to JIRA
1
2
3
4
9
6
5
8
7

20. Common understanding of configurations
Effective collaboration
Easier to find configuration history
Easy to identify configurations changes made in the past.
Faster cluster provisioning
Standard process for Configuration Auditing environment changes
Operational Benefits
Ensure consistent configurations and traceabilityServer configurations are version controlled in Git.
Average cluster setup time is 2 hours.
Average time taken to deploy patches such as security patches, wrapper
config updates, log4j2 updates, etc. across 400+ MREs is 2 days.
Automation -- Patch install scripts

21. Do you have any questions?
Thank you for your time!