An overview of the much needed vicissitude in the architectural thought transformation from monolithic to microservices architecture

1. ARCHITECTING EXTREMELY
LARGE SCALE WEB
APPLICATIONS
A MUST read for every architect
[ #Prashanth B Panduranga ]

2. An overview of the much needed
vicissitude in the architectural
thought transformation from
monolithic to microservices
architecture

3. New York times auto scaled to 500,000 users
HipChat has about 1.2 billion messages/documents stored
Sales force deals with 1,300,000,000 daily transactions with over 24,000 database
transactions per second and over 22PB of raw SAN storage capacity
CinchCast has over 50 million page views a month
Pinterest has over 18 million visitors with a 10X growth rate
Amazon has over 55 million active customer accounts
Flickr has over 4 billion queries per day
Netflix has 48 million members with over 50,000 requests per second and the list goes
on. .
Thanks to HighScalability, where the above statistics are derived from, you get a
picture of the websites and the scale that I am referring to

8. I have elaborated on SAAS requirements and general architectural requirements in my previous blog.
https://prashanthpanduranga.wordpress.com/2015/03/25/inevitability-of-multi-tenancy-saas-in-product-
engineering/
The requirements which apply the most to large scale web applications:
Performance: There are lot of statistics published relating to performance implications of web applications. One
such statistic, Users abandon website even if there is a 2 second delay during a transaction. Large scale web
applications obviously have very high performance requirements. A very large percentage of applications gets
redesigned primarily for a better user experience. Average Page Load Time as a fraction of Server and client time,
Network time, Page views, Bounce Rate, Percentage Exit, Average Redirection Time, Average Domain Lookup Time,
Average Server Connection Time, Average Server Response Time, Average Page Download Time, Average content
load time, Average session time, DNS resolution time, TCP connection time, Time to first byte, Full page object load
time, Requests per second, error rates, Peak response time, Uptime, CPU utilization, Memory Utilization are just a
few metrics to look for

9. Availability:
Business continuity is of utmost importance. Various availability techniques can be applied on every layer.
AlwaysOn Failover Cluster Instances, AlwaysOn Availability Groups, Database mirroring, Log shipping,
Redundancy models - Active-Active, Active-Passive, Redundancy Methods - Hot Standby, Warm Standby, Cold
Standby, and the measurement of the same expressed as mean time to failure, mean time to repair, Eliminating
single points of failure, Accelerating fault detection, isolation and resolution, hot spares, warm spares, cold
spares, clustering, RAID, redundancy, Heart beats, watermarking resources, check pointing, Watch dogs and
more

10. Monitoring and Diagnostics:
26 front end proxy serves. Double that in backend app servers [Atlassian HipChat]
15,000+ hardware systems – [Salesforce]
100 hardware nodes in production – [CinchCast]
180 Web Engines + 240 API Engines, 88 MySQL DBs (cc2.8xlarge) + 1 slave each, 110 Redis Instances, 200
Memcache Instances, 4 Redis Task Manager + 80 Task Processors, Sharded Solr – [Pinterest]
1000+ supported devices – [Netflix]
Imagine maintaining those, when there are innumerable servers involved, failure of system components is
common, needless to say, to take appropriate timely actions monitoring and diagnostics plays a very important
role.

11. Scalability:
Capability of supporting and optimizing resource utilization on increasing workloads on various dimensions such
as memory, cores, data structures, throughout and more. Goes without doubt that the application need to scale,
in order for the application to perform well, and without automating it, the application cannot stay inexpensive.
Automation:
Identifying failures and automating re-provisioning of those components/servers is extremely important

12. Architecture has significantly emerged from a monolithic architecture to microservices architecture
APPLICATION
DATA
PRESENTATION
SERVICE/BUSINESS LOGIC
DATA

13. PRESENTATION
SERVICE/BUSINESS LOGIC
DATA
SECURITY
ANALYTICS
MONITORING&DIAGNOSTICS
STORAGE
EVENTS
NOTIFICATION
VIRTUALIZATION
NETWORKCOMPUTE
HARDWARE LAYER
CLOUD
CLOUD ADAPTER
AUTOMATION/BATCH
INTEGRATION
PROCESSING ENGINE
CACHE
MANAGEMENT
AD-HOCLAYER
AUDITING
CONFIGURATION
DISTRIBUTED PROCESSING
INGESTION
DATA MANAGEMENT DATA RULES META DATA DATA QUALITY
CLICKSTREAM DATA
SOURCES
CHAT
SENSOR
SOCIAL
LOGS
CRM
ERP
APPLICATION
DATA
CHANNELS
EDW
METADATA MANAGEMENT MULTI TENANT PROCESSING PARALLEL COMPUTE
PARALLEL OPERATIONS COMPLEX EVENT PROCESSING GOVERNANCE
WORKFLOW
REPORTING
STREAM PROCESSING
IN-MEMORY PROCESSING
MESSAGE TRANSPORTSMESSAGE QUEUESMESSAGE BROKERS
INTEGRATION FRAMEWORKS
ENTERPRISE SERVICE BUS
INTEGRATION SUITES
OPERATING SYSTEM

14. Tools/software Usage/Description TOP few similar tools/software to consider Comments
RabbitMQ Message broker
system
Message oriented
middleware
Queuing software
ESB
ActiveMQ, Amazon SQS, HornetQ, HiveMQ,
JMS, Kafka, ZeroMQ, MSMQ, NServiceBus, Azure
Service Bus, OpenMQ, Redis, Storm, Akka,
Apache Camel and Spring, OFM, Fuse ESB,
WebsphereMQ, Windows Service Bus, BizTalk,
WSO2, Mule, Talend ESB, Gearman, JBoss,
ServiceMix, OpenESB, Apache QPID
Look out for AMQP
compliance
Some of the tools
referred aren’t
Message brokers but
are used in
conjunction to perform
the same
Other AMQP
PIKA
Shovel
Kinesis Real time data
processing
Kafka, Storm
Tornado Web Server
HTTP Server
Nginx, Apache, IIS, lighttpd, haproxy, varnish,
glassfish, Jetty, Geronimo, Tomcat,
Some are even used
as reverse proxy,
proxies,

15. Cassandra Distributed database
management system
Mongodb, aerospike, accumulo, azure table storage,
bigtable, couchbase, couchdb, dynamodb, datastax,
ElasticSearch, Greenplum, Vertica, HBase, InfiniDb,
InnoDB, MariaDB, neo4j, Netezza, TeraData, RedShift,
Riak, RavenDB, Solr, Spark, VoltDB,
With over 200 dbs, it’s
difficult to list all: Checkout
the below link
https://prashanthpanduran
ga.wordpress.com/2013/12
/23/why-nosql-ok-but-why-
so-many/
Some of them are
dataware housing
Solutions, While some are
data processing engines
Hana In-Memory database GemFire, Hekaton, Aerospike, BigMemory, DataBlitz,
EhCache, eXtremeDB, FuelDB, HazelCast, MonetDB,
Coherence, VoltDB
Any Key value store can be
used for the same, some of
the enterprises have
experimented using NoSQL
store used as a cache and
unstructured db solution
Linux Operating System SUSE, FreeBSD, Solaris, Debian/Ubuntu, Windows Server,
Mac OS X, RHEL
Only Server OS included in
the list

16. SockJS Web Socket like
object
Web socket, socket.io, atmosphere, SignalR,
Alchemy, Fleck
Couple of them listed
are open source
Libev Event Loop LibEvent, Asio, Nginx, epoll
Fabrik Visual programming
IDE
Also known for
Content construction
Kits (CCK)
IDE: Visual Studio, Eclipse, NetBeans, Aptana
CCKS: Seblod, K2, chronoform, Zoo,
Breezingforms, Cobalt, FlexiContent
Java Programming
language
C, C++, Python, C#, PHP, Javascript, Ruby, R,
Matlab, Objective-C, Visual Basic, Perl, Swift,
Scala, Shell, GO, LISP, SAS, F#, Groovy, Lua
Some of them listed
are web programming
languages adiitionals:
HTML, SQL, Haskell

17. Twisted Event driven network
programming
framework
Tornado, Django, Asyncio,
AWS Cloud provider Azure, Rackspace, CenturyLink, Salesforce,
Engineyard, Google, OpenStack, SAP,
CloudBees, CumuLogic, Eucalyptus, Gigaspaces,
Mulesoft, Parallels, Pivotal, puppet Labs, Ravello,
Rightscale, SoftwareAG, Xively, AT & T, Cisco,
Comcast, EMC, GoGrid, CSC, HP, IBM
smartcloud, Joyent,
The list includes
infrastructure, platform,
storage and security
cloud providers
Lucene Test search Engine
Library
Azure Search, Autonomy, Solr, GSA, Attivio,
DTSearch, elasticSearch, endeca, FAST,
MarkLogic, Nutch, Sphinx, Sketchy, Scumblr
A few NOSQL
databases have been
used for the same, This
list does not include all
the NOSQL databases
that could be used
Adobe Air Cross-platform
runtime
Cordova(Phonegap), Appcelerator, Qt, Sencha,
cocos2d-x, Xamarin, ionic, Kony, mono, xcode
The ones listed here are
cross platform as well as
mobile development
platforms.

18. Sensu Monitoring
Framework
Zabbix, Nagios, icinga, monit, Riemann, statsd,
graphite, zenoss, collectd, munin, cacti, new
Relic, ganglia, splunk, sentry, dynatrace,
datadog, skylight, zenoss, observium, spiceworks,
solarwinds, fiddler, wireshark, httpwatch, firebug,
soapUI, OpManager
The list includes some of
the: Infrastructure
monitoring
Searching, monitoring
and analysing, Network
monitoring
Scalable distributed
monitoring system
PagerDuty
NeoLoad
Incident
management system
and performance
testing and
monitoring
OpsGenie, VictorOps, xmatters, pingdom,
Gomez, webpagetest, monitis, uptrends,
keynote, OpsView, Apache JMeter, LoadRunner,
WebLOAD, Appvance, NeoLoad, LoadUI, WAPT,
Loadster, LoadImpact, Soasta, Rational
Performance Tester, Testing Anywhere,OpenSTA,
QEngine (ManageEngine), Loadstorm,
CloudTest, Httperf, SilkPerformer, BlazeMeter,
Visual Studio Test Suite,
Also includes web site
monitoring
Cloud based quality
testing
Performance
monitoring
Chef IT Automation Puppet, ansible, salt, docker, Jenkins,
Capistrano, saltstack
Configuration
management, SCCM
memCache Distributed memory
object caching
Apc, memcached, dynacache, ehcache,
xcache,
key value based
NOSQL databases are
also used

19. Razor Physical and virtual
hardware provisioning
solution
Axemblr, Cobbler, JuJU, SaltCloud, Dell Crowbar,
Ansible, CFEngine, Chef
Perforce Version Management
and Content
collaboration
Git, SVN, TFS, bitbucket, ClearCase, Subversion
Pytheas ITIL assets management
software
Remedy (BMC), Assyst (Axios), FrontRange, EasyVista,
Hornbill, HP Service Manager, SmartCloud Control Desk
(IBM), ServiceNow
IT incident management, IT
problem management, IT
change management, IT
release governance, IT user
self-service, IT request
management, IT
knowledge management,
IT service support analytics
and reporting, IT SLA
management
Ref: Gartner
ZUUL Service that provides
dynamic routing,
monitoring, resiliency
and security
Nginx, lightpd, Netscaler, HAProxy, Radware,
CoyotePoint, Barracuda, Kemp, Varnish, Avast, Norton,
Kaspersky, Mcafee, AVG, Avast, Bitdefender, F5,
PaloAlto, Cisco ASA, Cisco ACE, Foundary, Juniper SSG,
MS TMG
Can be firewall, router, web
load balancing server,
proxy Server etc.

20. Feign Java http client binder Retrofit, JAX-RS, web socket,
Jersey, CXF, Apache HC
Includes transport libraries
Hive Querying and managing
large datasets residing in
distributed storage
Impala, BigSQL, HAWQ
AWS ELB Elastic Load balancing Nginx, HAProxy, Route53,
Azure Traffic Manager, F5
Port-bound servers, sticky sessions, TCP session
reassignment, automatic unfail, slow start, SynGuard,
dynamic feedback protocol, NAT, maximum
connection, Round Robin, Least Connections,
Weighted Round Robin, Weighted Least Connections,
Fastest Response
Layer 4 and Layer 7 load balancing
Cloud Load balancing features: Dedicated (static) IP
address,SSL termination, Multiple protocols, Advanced
access control, Connection logging, Advanced
algorithmic routing, Session persistence, Connection
throttling, Node management, High availability
Content caching, Persistent connections, Gzip
compression, Regionalized load balancers

21. gZip Application used for
file compression and
decompression
httpZip,deflate, 7zip, bzip2, zlib
Akamai Content delivery
network
Azure CDN, Cloudfront, Torbit, Incapsula,
Cotendo, Fastly
HTML 5 frameworks
Javascript
Frameworks
https://www.facebook.com/notes/prashanth-panduranga/frameworks/10152107517972934
OpenStack Open Source Cloud
computing platform
OpenStack currently has the following features:
Compute(Nova), Object Storate (Swift), Block
Storage (Cinder), Networking (Neutron),
Dashboard (Horizon), Identity Service (Keystone),
Image Service (Glance), Telemetry (Ceilometer),
Orchestration (Heat), Database (Trove), Bare
Metal Provisioning (Ironic), Multiple tenant cloud
messaging (Zaqar), Elastic Map Reduce
(Sahara)

22. Hadoop Distributed storage and distributed processing of very large data sets on computer clusters
Aegisthus Bulk Data Pipeline out of Cassandra
Eureka Eureka is a REST (Representational State Transfer) based service that is primarily used in the AWS
cloud for locating services for the purpose of load balancing and failover of middle-tier servers
Genie Federated Job Execution Engine
Clojure Dynamic programming language that targets the Java Virtual Machine
PigPen Map-Reduce for Clojure
Governator Governator is a library of extensions and utilities that enhance Google Guice to provide:
classpath scanning and automatic binding, lifecycle management, configuration to field
mapping, field validation and parallelized object warmup
Inviso Visualize Hadoop performance
Ribbon Ribbon is a Inter Process Communication (remote procedure calls) library with built in software
load balancers
Hystrix Hystrix is a latency and fault tolerance library designed to isolate points of access to remote
systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex
distributed systems where failure is inevitable

23. Suro Distributed data pipeline
Aminator A tool for creating EBS AMIs
Lipstick Pig Visualization framework
Zeno In-Memory Data Propagation Framework
Blesk Lightweight client for pushing notifications to web based applications/sites
Turbine Turbine is a tool for aggregating streams of Server-Sent Event (SSE) JSON data into a
single stream. The targeted use case is metrics streams from instances in an SOA
being aggregated for dashboards
Priam Co-Process for backup/recovery, Token Management, and Centralized
Configuration management for Cassandra
Workflowable Workflowable is a Ruby gem that allows adding flexible workflow functionality to
Ruby on Rails Applications
s3mper S3mper is a library that provides an additional layer of consistency checking on top
of Amazon's S3 index through use of a consistent, secondary index

24. Astyanax Java Client for Apache Cassandra
Denominator Denominator is a portable Java library for manipulating DNS clouds. Denominator
has pluggable back-ends, including AWS Route53, Neustar Ultra, DynECT, Rackspace
Cloud DNS, OpenStack Designate, and a mock for testing
GCViz Garbage Collector Visualization framework
Curator The Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It
adds many features that build on ZooKeeper and handles the complexity of
managing connections to the ZooKeeper cluster and retrying operations
Staash A language-agnostic as well as storage-agnostic web interface for storing data into
persistent storage systems, the metadata layer abstracts a lot of storage details and
the pattern automation APIs take care of automating common data access patterns
Edda Edda is a Service to track changes in cloud deployments

25. Brutal An asyc centered chat bot framework for python programmers written using the twisted
framework
CassJMeter JMeter plugin to run cassandra tests
Glisten Groovy library for building JVM applications with Amazon Simple Workflow (SWF)
Pig Platform for analyzing large data sets
Spark Engine for big data processing, with built-in modules for streaming, SQL, machine learning and
graph processing
Karyon Framework and a library for a cloud ready web service. Blueprint for the services. It contains
Bootstrapping, Libraries and Lifecycle Management, Runtime Insights and Diagnostics,
Pluggable Web Resources, Cloud-Ready hooks
EBS Elastic Block store, persistent block level storage volume
Curler A Gearman worker which cURLs to do work
archaius, , Library for configuration management API
ZooKeeper ZooKeeper is a centralized service for maintaining configuration information, naming, providing
distributed synchronization, and providing group services

26. Parallel processing - Explicit and Implicit parallelism, batch parallelism, asynchronous programming, segregating
layers, distributing workloads, Load balancing, multi- tenancy, scaling out on all layers, sharding, partitioning,
CAP preference, reads, writes, statelessness, logging and telemetry, automating, SOA adoption, caching,
throttling, distributing requests across multiple zones, effective usage of CDNs, Auto provisioning, Auto scaling,
compression, queuing, workload distribution, batch processing, designing system with fault tolerance,
redundancy, Consistency, Availability, Partition Tolerance, event processing, web sockets, cloud computing, fog
computing, Grid Computing, Client side workload distribution, In-Memory processing, Proxies, No single points
of failure. Resilience to failure, Graceful degradation, Recoverability from failure, design for failure, Database
Transactions, Client side transactions, two-phase commit, Auto-commit, Partition Everything, DB operations
ordering, Considerations for Eventual consistency, Functional Segmentation, Application Pools, Prevention of
session state, Async Everywhere, Index, Structured Indexes, text indexes, entity indexes, Fuzzy match indexes,
pre-aggregated indexes, pre-calculated indexes, embedded value indexes, join indexes, link indexes, De-
Normalized Indexes (all kinds) are all important considerations for a highly successful and scalable website.
Rest assured if you have considered all the above factors in your architecture you are on your way to create a
scalable one. Do let me know if you have questions regarding any particular subject and I will be glad to write
up on the same.

27. Reference
www.HighScalability.com
has some awesome architecture blogs, I have derived the technology stack from multitude of articles
hosted by highscalability and a few from Netflix blogs

28. PRASHANTH B PANDURANGA
https://prashanthpanduranga.wordpress.com/
Telephone (India): 7259767006, 7760824680
Prashanth.panduranga@gmail.com