1.
Curated "Cloud Design
Patterns" for Opensips-based
Platforms
Alejandro Rios, Livevox Inc.
Opensips Summit – May 1st 2018, Amsterdam

2.
© LiveVox 2018
Agenda
• Why? a story born in the clouds
• Challenges in cloud-based contact center solutions
• 5 Key Cloud Principles
• 12 Livevox's Curated Patterns
• Demo
4/30/20182

3.
© LiveVox 2018
LiveVox Overview
4/30/20183
➢ Founded Jan. 2000 by Louis Summe and Larry Siegel, executive staff include former leaders from
Cisco, Genesys (Soundbite), Aspect, and Avaya
➢ Provides fully integrated customer engagement solutions across a multichannel environment
from a PCI-DSS certified pure cloud platform with multi-carrier interoperability and multi-tenant
scalability
➢ Servicing 200+ contact center clients with a focus of 500+ agents across multiple industries;
industry leading NPS scores and end-to-end SLAs
➢ Supported by 400+ employees and rapidly growing; San Francisco HQ; offices in Atlanta,
Bangalore and Medellin
➢ 15+ yrs in the financial services industry developing cutting-edge compliance tools (e.g. TCPA,
CFPB, PCI, and more) for highly regulated industries

4.
© LiveVox 20184/30/20184
LiveVox manages
9+ Billion
interactions/year
across outbound,
inbound, self-service
voice, SMS and email

5.
© LiveVox 2018
Adoption of LiveVox continues to grow across industries
4/30/201823

6.
PROPRIETARY AND CONFIDENTIAL© LiveVox 2018
A story born in the Clouds
Friday, November 2nd 2012, 9am...
4/30/20185

7.
© LiveVox 20184/30/20186

8.
© LiveVox 2018
Challenges
in cloud-based contact center solutions
• Reliability
To minimize downtime and comply with customers SLA, the system must be highly available and
resilient.
• Flexibility
Loosely coupling of components to simplify administration, development and reusability.
• Scalability
Handling variable workloads and peaks in activity without impact on performance.
• Security
Prevent malicious or accidental actions outside of the designed usage, and to prevent disclosure
or loss of information.
4/30/20187
Azure CloudPatterns.MicrosoftAzure.Nov2017 [https://docs.microsoft.com/en-us/azure/architecture/patterns]

9.
5/1/2018© LiveVox 2017 19
Active Redundancy
Hot/StandBy Failover

10.
© LiveVox 20184/30/2018
5 Key Cloud
Principles
CDA: Cattle Driven
Architecture, make
things replaceable!
DRY: don't repeat
yourself
SOA: (micro)Service
Oriented
Architecture
CI: Continuous
Integration
Configuration as
Code, code as
service
8

11.
© LiveVox 2018
(1/5) CDA: Cattle Driven Architecture
Make things replaceable!
In a cattle vs pets approach to development
and operations:
• Pets: are servers that we heal and patch
when they fail.
• Cattle: are replaceable, like cattle,
configuration is never “adjusted” after
deployment.
4/30/20189
RandyBias,Petsvs.Cattle:The ElasticCloudStory.Feb2014. http://cloudscaling.com/blog/cloud-computing/pets-
vs-cattle-the-elastic-cloud-story/
If it fails, just terminate
it and start a
new instance.

12.
© LiveVox 2018
(2/5) DRY
Don't repeat yourself
Also known as DIE - Duplication is Evil:
• Numerous implementations of the same
classes, libraries etc.
• In case of a patch, it has to be retested
everywhere it has been repeated.
Instead we can write code once and use it
everywhere.
4/30/201810
FormulatedbyAndyHuntandDave Thomas in“The PragmaticProgrammer”
Have a single source
of truth for both
application and
configuration data.

13.
© LiveVox 2018
(3/5) SOA
(micro)Service Oriented Architecture
As application complexity increases, it can be
broken into smaller, loosely coupled components:
• Microservices are “Small”, independent,
composable services
• Remove the vendor, remove technology, boil
down to the essential service
• Access behavior though a defined interface:
libraries or APIs.
4/30/201811
Architectingforthe Cloud:AWSBestPractices.AmazonWebServices.February2016
The service
is implemented once,
then used by others

14.
© LiveVox 2018
(4/5) Configuration as Code
Code as Service
• Configuration needs to be managed (versioned,
deployed, validated) and can be buggy, just as
code
• There are different types of configuration data, and
each of them should be managed
differently: System, Static, Dynamic
• Make your whole infrastructure reusable,
maintainable, extensible, and testable.
4/30/201812
Architectingforthe Cloud:AWSBestPractices.AmazonWebServices.February2016

15.
© LiveVox 2018
(5/5) CI
Continuous Integration
• Members of a team integrate their work
frequently: multiple integrations per day
• Each integration is verified by an automated
build (including test) to detect integration errors
as quickly as possible
• Make it easy for anyone to get the latest
executable and see what’s happening
4/30/201813
Fowler,Martin.“ContinuousIntegration.”MartinFowler,01 May 2006,
https://www.martinfowler.com/articles/continuousIntegration.html

16.
© LiveVox 2018
12 Livevox's Curated
Cloud Design Patterns (CDP)
4/30/201814

17.
PROPRIETARY AND CONFIDENTIAL© LiveVox 2018
What is a Design Pattern?
• Is an optimized, reusable solution
• Is not vendor/technology specific
• It is a template that has to be implemented in
the correct situation in order to prove useful
and effective
• Each pattern has a context of the problem, the
solution, implementation examples and
reference patterns
4/30/201815
Most cloud-architects have
encountered the same
challenges before, and have
used common solutions to
remedy them.

18.
© LiveVox 2018
(1/12) Bootstrapping
Context/Problem:
How to quickly create server
instances for executing
installation, startup, and setup of
system data
Solution:
Start up the virtual server, either
from scratchor from a template
image, and set up the basic
system data
4/30/201816
Implementation examples:
• Puppet recipes for setting up Opensips module configuration
parameters, i.e. TLS certificates, hostname, IP addresses,
security profiles (firewall, remote access), etc.
Reference:
• http://en.clouddesignpattern.org/index.php/CDP:Bootstrap_Pattern
• http://en.clouddesignpattern.org/index.php/CDP:Stack_Deployment_P
attern

19.
© LiveVox 2018
(2/12) Configuration Sidekick
Implementation examples:
• Use a separate version control system for code and static
configuration.
• Package custom scripts and properties as a separate .rpm or .deb
file (domains, ports, dynamic data endpoints), specifying the upstream
OpenSIPS RPMs as dependencies of our RPM.
• Use a service registration and discovery method to allow retrieval of the
endpoint IP addresses and to pull dynamic configuration from the
central Component Manager
Reference:
• https://docs.microsoft.com/en-us/azure/architecture/patterns/sidecar
• https://www.opensips.org/Documentation/Generating-Configs-2-4
4/30/201817
Context/Problem:
Managing upstream software
provider versions independent
from static application
configuration changes.
Provide clean source to
auditors and partners.
Solution:
Keep code/binaries separate
from static configuration

20.
© LiveVox 2018
(3/12) Component Manager
Implementation examples:
• Move dynamic configuration out of the deployment package to a
centralized location.
• Keep a central service for provisioning of routing rules and pool
management, then use the opensips MI command interface to order a
reload of data to all instances.
Reference:
• https://docs.microsoft.com/en-us/azure/architecture/patterns/external-
configuration-store
4/30/201818
Context/Problem:
It's challenging to manage
changes to local configurations
across multiple running
instances of the application
components
Solution:
A single place to describe and
provision all components and
resources.

21.
© LiveVox 2018
(4/12) Load Balancing Pool
Implementation examples:
• Opensips “dispatcher” module allows the definition of weights for the
destination,
• The “load_balancer” module can be preconfigured with the maximum load
accepted by each of the destination pools.
• Also both modules probe the health of the destination members to disable or
enable them on recovery.
• If a pool member keeps failing, instead of repairing it, take it out of the pool,
and replace it by a fresh new instance.
Reference:
• http://en.clouddesignpattern.org/index.php/CDP:Multi-Server_Pattern
• http://en.clouddesignpattern.org/index.php/CDP:Scale_Out_Pattern
4/30/201819
Context/Problem:
Single points of failure can be
removed by introducing
redundancy, which is having
multiple resources for the same
task.
Solution:
Since Call center calls are mostly
composed of stateless interactions,
resiliency is more important than
HA: so redundancy can be active
instead of standby.

22.
© LiveVox 2018
(5/12) Scale Out
Implementation examples:
• Opensips provides detailed memory and dialog stats through the MI interface.
• T-Shirt sizes of each component's type in a vendor/cloud agnostic way:
small, medium, large, etc.
• If the traffic volumes follow a pattern, you can turn off instances when traffic
decreases. i.e. we have more intensive outbound dialing campaigns by end of
month.
• Don't abuse Auto-scaling. Do a semi-manual or human verified
escalation instead.
Reference:
•http://en.clouddesignpattern.org/index.php/CDP:Scale_Out_Pattern
•http://en.clouddesignpattern.org/index.php/CDP:Scheduled_Scale_Out_Pattern
4/30/201820
Context/Problem:
To process high traffic volumes,
you need high-specification sip and
media servers, but Scaling Up in
processing power can increase
cost and reduce availability in case
of failure.
Solution:
Benchmark and select an instance
type that suits each component
workload’s requirements.

23.
© LiveVox 2018
(6/12) Domain Sharding(6/12) Domain Sharding
Context/Problem:
A large number of requests originating from
one client may exhaust available resources in
the service. Other consumers are no longer
able to consume the service, causing a
cascading failure effect.
Solution:
Add a fault-isolating mechanism to horizontal
scaling, instead of spreading traffic from all
customers across every node, you can group
the instances into “shards” or SIP domains.
Context/Problem:
A large number of requests originating from
one client may exhaust available resources in
the service. Other consumers are no longer
able to consume the service, causing a
cascading failure effect.
Solution:
Add a fault-isolating mechanism to horizontal
scaling, instead of spreading traffic from all
customers across every node, you can group
the instances into “shards” or SIP domains.
4/30/18
Implementation examples:
●
Use Opensips TLS domain/port separation to serve different domains,
and different certificate settings, for customers with different security
requirements.
●
Create separate specialized pools of servers: i.e. a pool of media
servers with higher CPU specs to handle heavier transcoding.
●
Reference:
●
https://docs.microsoft.com/en-us/azure/architecture/patterns/sharding
●
https://docs.microsoft.com/en-us/azure/architecture/patterns/bulkhead
●
http://en.clouddesignpattern.org/index.php/CDP:Multi_Load_Balancer_Pattern

24.
© LiveVox 2018
(7/12) Transitioned migration(7/12) Transitioned migration
Context/Problem:
Completely replacing a complex system or
transferring the system as a whole can be
a huge undertaking, every time a feature
or service is migrated, clients need to be
updated to point to the new location.
Solution:
Incrementally migrate a legacy system by
gradually replacing specific pieces of
functionality with new applications and
services.
Context/Problem:
Completely replacing a complex system or
transferring the system as a whole can be
a huge undertaking, every time a feature
or service is migrated, clients need to be
updated to point to the new location.
Solution:
Incrementally migrate a legacy system by
gradually replacing specific pieces of
functionality with new applications and
services.
4/30/18
Implementation examples:
●
Keep a range of IPs that can be used to be whitelisted by customers,
and give space for new servers.
●
Add/remove members to the load balancing pool with a low weight (for
example, 1%), then increase over the next few days and monitor.
●
Embed Opensips load balancers into the Media Servers for greater
granularity of pool routing.
●
Reference:
●
https://docs.microsoft.com/en-us/azure/architecture/patterns/strangler
●
http://en.clouddesignpattern.org/index.php/CDP:Weighted_Transition_Pattern

25.
© LiveVox 2018
(8/12) Cache DataSet(8/12) Cache DataSet
Context/Problem:
How to improve application performance by
avoiding reading sets of data from the disk
every time they are used
Solution:
Store previously calculated data that is read
frequently in a memory cache. Make sure
that the data in the cache is as up-to-date as
possible. Refresh data if cache has become
stale
Context/Problem:
How to improve application performance by
avoiding reading sets of data from the disk
every time they are used
Solution:
Store previously calculated data that is read
frequently in a memory cache. Make sure
that the data in the cache is as up-to-date as
possible. Refresh data if cache has become
stale
4/30/18
Implementation examples:
●
Use the cachedb_redis opensips module, so a restart both on the
OpenSIPS and the Redis server will not cause any data loss.
●
Update the cache by pushing, not pulling data.
●
Keep different redis sets in memory, after all the reload is done, update
the set index.
●
Reference:
●
https://docs.microsoft.com/en-us/azure/architecture/patterns/cache-aside
●
http://en.clouddesignpattern.org/index.php/CDP:Inmemory_DB_Cache_Pattern
●
https://www.opensips.org/Documentation/Tutorials-KeyValueInterface

26.
© LiveVox 2018
(9/12) Dynamic routing rules(9/12) Dynamic routing rules
Context/Problem:
Opensips has a very flexible custom scripting
language, but the hard-coded routing rules can
get very complex and the script can get very
big and hard to maintain
Solution:
Keep the Opensips script small, and keep
routing rules and parameters as external data.
Then you can update and load the routing rules
to an in-memory cache dynamically, without
having to restart the server.
Context/Problem:
Opensips has a very flexible custom scripting
language, but the hard-coded routing rules can
get very complex and the script can get very
big and hard to maintain
Solution:
Keep the Opensips script small, and keep
routing rules and parameters as external data.
Then you can update and load the routing rules
to an in-memory cache dynamically, without
having to restart the server.
4/30/18
Implementation examples:
●
The “drouting”, “lcr”, and “carrierroute” Opensips modules already do
this for destination matching rules.
●
You can use OpenSIPS with Redis cache integration to provide
extended dynamic routing capabilities: i.e. destination and
transformation rules based on other SIP message parameteres
●
Reference:
●
http://en.clouddesignpattern.org/index.php/CDP:Inmemory_DB_Cache_Pattern
●
https://www.opensips.org/Documentation/Tutorials-KeyValueInterface

27.
© LiveVox 2018
(10/12) Auth Delegation(10/12) Auth Delegation
Context/Problem:
How to isolate the cost sensitive areas of
the service and provide additional layers
of security while minimizing administration
and facilitating user experience
Solution:
Don't rely on SIP authentication for
outbound call fraud prevention, add extra
layers of authentication for each call leg
type
Context/Problem:
How to isolate the cost sensitive areas of
the service and provide additional layers
of security while minimizing administration
and facilitating user experience
Solution:
Don't rely on SIP authentication for
outbound call fraud prevention, add extra
layers of authentication for each call leg
type
4/30/18
Implementation examples:
●
keep separate call legs for users and outbound calls, and use high level
business logic to bridge both calls using conferences on the media
servers.
●
If using webRTC with SIP over WebSockets, delegate authentication to
the Web server application, and generate a token of the valid session, to
use it as credentials on the SIP message. Internally, the SIP server
validates the token with the webserver using opensips rest_get or redis
cache modules.
●
Reference:
●
https://docs.microsoft.com/en-us/azure/architecture/patterns/gatekeeper
●
https://docs.microsoft.com/en-us/azure/architecture/patterns/federated-identity

28.
© LiveVox 2018
(11/12) Client-Side Load Balancing(11/12) Client-Side Load Balancing
Context/Problem:
There are scenarios where a load balancer
does not meet your requirements For
example: Agent leg audio requires client
devices to maintain a connection to a specific
server for prolonged periods of time.
Solution:
if you control the code that runs on the client,
use client-side load balancing. This adds
extra complexity but can be useful for Multi-
Data Center Resilience.
Context/Problem:
There are scenarios where a load balancer
does not meet your requirements For
example: Agent leg audio requires client
devices to maintain a connection to a specific
server for prolonged periods of time.
Solution:
if you control the code that runs on the client,
use client-side load balancing. This adds
extra complexity but can be useful for Multi-
Data Center Resilience.
4/30/18
Implementation examples:
●
Keep the endpoints to a limited subset of client programs that can be
controlled for provisioning and upgrades. Use client auto provisioning
tools to push changes to SIP user-agents.
●
AnyCast IP for Geographic
●
DNS-SRV records for UDP/TLS
●
HTTP load balancing for WSS
●
Use a short TTL and DNS flush to force updates on the client side
●
Reference:
●
http://en.clouddesignpattern.org/index.php/CDP:Multi-Datacenter_Pattern

29.
© LiveVox 2018
(12/12) Automated Testing and Monitoring
Implementation examples:
• Define at least 3 levels of testing: unit, integration, load
• Should be designed to run cleanly and unattended in a repeatable,
parameterizable way
• Use sipp scenarios for unit/component testing
• Browser or desktop apps automation tools for user-agent testing
• Make the cloud platform to test itself end-to-end: i.e. generate lookback
load tests with auto-answer IVRs
Reference:
• https://docs.microsoft.com/en-us/azure/architecture/patterns/health-endpoint-
monitoring
• http://en.clouddesignpattern.org/index.php/CDP:Monitoring_Integration_Patter
n
4/30/201821
Context/Problem:
A big challenge when constantly
updating and deploying component
versions and instances in a
cloud platform is to achieve speed
without sacrificing quality.
Solution:
This patter helps define test levels,
track test case coverage and using
automated tools as part of the
continuous integration process.

30.
© LiveVox 20187
LiveVox
provides a
comprehensive
approach to
business
continuity &
disaster
recovery
LiveVox
provides a
comprehensive
approach to
business
continuity &
disaster
recovery
4/30/18
Outage
Causes
LiveVox Cloud
Mitigation
Customers
HW and SW Failures/Bugs Human Error Force Majeure
Resilient Data Center, Agent & Carrier Interconnect
Architectures
QA, Phased Rollouts & Continuous Integrated System
Training
Change Control
Geographic
Data Center
Fallover
Service Level Agreements
Service SLA of
99.99%
to Platinum
customers
Service Level SLA
Account for end to end
impact on customer
business
Support SLA
Guaranteed
Response Times —
24/7
Carrier, customer, internal, and
vendor hardware & software
Carrier, customer &
internal personnel
Weather &
terrorism

31.
Thank YouThank You
https://github.com/alerios/cloud-design-patternshttps://github.com/alerios/cloud-design-patterns