© P K Mallik
Architecture
Concepts
Principles, Types,
Views,
Components, Agile
© P K Mallik
Agenda
Principles
Types
Views
Components
Architecture to Design
Agile Architecture
© P K Mallik
Architecture
Principles
Separation of concerns
Points of failure
Scalability
Access control
Architecture Styles (types of architecture,
event driven, process centric)
Segregation
© P K Mallik
Example – Shopping Site
© P K Mallik
Separation of Concerns
• Each Feature must be addressed
by a single architectural
component
• One component may address
multiple features a single feature
should not be managed by
multiple components.
• If multiple components manage a
feature they should be bundled
together as a ‘package’ and
treated as a single component
(note if you end up with only 1
component in your system it is no
longer architecture)
© P K Mallik
Local Log
Points Of Failure
• Number of components that need to
fail for the entire system to shut down
• Larger the points of failure the more
stable the overall architecture
• Usually has a trade off with
performance
• Use Asynchronous interfaces to
reduce failure points
• Use Hardware redundancy to provide
for switchover
• Clustering (hot switchover)
• DR Site (cold switchover)
• Protect ‘sensitive’ components from
overloading
Order
Booking
Payment
Audit
Logging
Payment
Gateway (3rd
Party)
Order
Booking Payment
Audit
Logging
Payment
Gateway (3rd
Party)
Local Log
Drip Feed
© P K Mallik
Workload/Usage Patterns
• Define peak/off peak
patterns and expected
‘surges’
• Project expected growth
over period of time
• Provide for capacity to
absorb peak usages
• Provide economically viable
solution
• Pattern could be Users,
Transactions, Data Loading,
Processing
• Scale Out when
• Peaks are for short periods of
time
• One time surge (such as start-
up)
• Budgets prevent large
investments in hardware and
licenses
• Scale Up when
• There is a predictable growth
in volumes
• Growth patterns do not have
significant peaks and troughs
• Budgets support initial
overcapacity
© P K Mallik
Segregation
• Need to decouple component
• High resource usage
• Scale out a high usage component
• Availability issues
• Security Concerns
• Example
• Reporting/Data warehousing
• External Access
• Workflow
Time
Volume
Data Feed
© P K Mallik
Architecture
Types
Three Tier
Service Orientation
Cloud Computing
Event Driven
High Throughput
No SQL
Map Reduce
© P K Mallik
Three Tier
• Default type and most commonly use
• Applies the Model View Controller Pattern
• Model
• Encapsulates Application State
• Responds to Queries
• Exposes Functionality
• Notifies Changes
• View
• Renders models
• Requests for updates
• Sends user actions to controller
• Allows selection by controller
• Controller
• Defines application behaviour
• Maps user action to model
updates
• Selects view for response
• Separate for each functionality
© P K Mallik
Service Orientation
• Provide a set of services rather than interfaces to external
applications
• Ability to monitor and manage the resource usage by services
• Ability to allocate system resources based upon simple rules
• Service has a contract – key elements of that contract. Same
service can have multiple contracts
• Who can call
• For that caller, what it need to pass
• For that caller – what will be response
• When caller can expect response
• What will be communication channel
• Operates on a publish-subscribe model
• May be aligned with commercial models which require specific
usage data to be maintained
© P K Mallik
Event Driven
• Used where the system is part of
a partially or fully automated
operation or a real time loop
• Operates on a throw—catch
model
• Usually built in a single tier with
interfaces with other systems for
Human Machine Interface (such
as mobile alerts, emails)
• Decouples application and
component interfaces
• Internal interfaces through
components raising and
processing events
• Typical example would be system
monitoring and management
© P K Mallik
Cloud Computing
• Hosted environment provided by vendors for
• Web Sites
• Databases
• File Services
• Images and Videos
• Removes need for administrating site and managing scaling
• Different architectural components may be deployed on different
clouds
• Provides elasticity in capacity building and management
• Commercials depend upon nature of site and type (public/private) of
usage (Rate Controls)
• Private clouds help reduce capital budgets with shared services
• Typical Usages
• Mobile Applications
• Customer Services
• Business Intelligence
© P K Mallik
No SQL Databases
• No SQL databases support unstructured data by replacing RDBMS concept of row with
‘Document’(Mostly JSON)
• Schema is not enforced by database, consuming applications are expected to deal with structure
• Data is mostly distributed across some kind of network file system for resilience
Key Value Databases for high
performance and scalability
Document Databases for high
flexibility required for storing
content
Wide Column Stores for storing
data in columns for high
performance queries and widely
used in Big Data processing
Graph Stores to store
relationships between data nodes
without the restriction of a strict
schema
© P K Mallik
Map Reduce
Architecture
• Map-Reduce architecture is built
for handling terabyte – petabyte
data load using cluster of cheap
commodity nodes
• Algorithms are expressed as two
functions. Map function travels
over network where data resides
and applies on data
• Final results are combined back
using reduce function
• Our own IP – Data Extraction
Platform (DEP) is built using Map-
Reduce which provides grid scale
capability to handle massive
data loads
© P K Mallik
Architecture
Views
Overview
Business
Application
Technical
Information
Deployment
© P K Mallik
Enterprise Architecture
© P K Mallik
Business Architecture
CEO
Operations Finance Vendor Mgmt Sales
Members
Support
Editing
Receipts
Banking
investments
Accounting
Acquisition
Delivery
Payments
Payments
Promotions
Events
Loyalty Mgmt
Products Data Analysis
© P K Mallik
Application Components
Website
Content
Management
Workflow
CRM Payment
Gateway
BPM
Vendors
© P K Mallik
Layering
Data Layer
Business Layer
Db Server No SQL
Content
Server
User Interface Layer
Web Client App 3rd Party
CRM Content Transactions
Interface
BPM
© P K Mallik
Information Architecture
Main Database
(Entities and
Relations)
DR Site
Configuration
Regional
Databases
Content Server
Content BackupPurchase Payment Pricing Delivery
Customer C R U R U R U
Product R C R U R U
© P K Mallik
Technical Architecture
O/S
App Server
Web Site
Content Server
Workflow
LDAP
O/S
Database Server
O/S
BPM
External
Interfaces
O/S
Mail Server
© P K Mallik
Deployment Architecture
Members
Partners
Employees
VPN
Scale Up
Scale Out
Server Farm
© P K Mallik
Architecture Components
Independently maintained software
Addresses one or more functional and non-functional
requirements
Selection from a class of similar
software on the basis of
Compatibility with proposed stack
Cost
Fitment with client requirements
Common Components
Portal
Workflow
BPM
ETL
Document Management
Rules Engine
Security
Systems Management
© P K Mallik
Common Architecture Components
• Products and tools regularly used in systems
• List of most popular components
• Integration & Workflow
• Document Management
• Rules Engine
• Security
• System Management
• Business Intelligence
© P K Mallik
Workflow
• Products which manage
processes within an organisation
• Generates tasks on completion of
an event
• Has capability to determine which
task to initiate depending upon
configurable parameter
• May be bundled with CRM and
Document Management products
• Primarily meant to handle manual
tasks and does not scale well to
large transaction volumes
© P K Mallik
Integration
• Define and manage external interfaces
• Manage processes across multiple systems or organisations
• Process integration defined through standard language
(such as BPEL or BPMN 2)
• Provide an Orchestration service to compose business
views from multiple sources
• Provide a highly configurable platform when dealing with
multiple partners or systems
• Can be implemented in conjunction with workflow engines
• Not to be used for high volume data processing (use ETL
tools instead) for integration or data loading
© P K Mallik
Document Management
• Repository based tool to create, modify and publish documents
• Manages access through Access Control Lists or third party Role
Based Access Control systems
• Maintains history of changes and can produce specific version
• May include multiple channels including RSS feeds
• Significantly improves productivity
• Can extend to maintain content, specially for web based
applications
• Manage feeds from external devices (scanners, cameras etc)
• Meant for stable process flows, has problems when document
management processes change frequently
• Not very cost effective for small volumes
© P K Mallik
Rules Engine
• Define and manage complex business rules with high
transaction volumes
• Ideally suited for STP type of requirements
• Can be combined with Workflow or BPM engine to
determine process direction
• May be used to make complex rule driven calculations
• Expensive and process hungry
© P K Mallik
Security
• Manage access to data and functions
• Three aspects
• Identity Management – Issue/update password, smart card,
biometrics
• Authentication – Verify identity of user
• Access Control – Determine access to functions and data
• Each aspect may be addressed by a different product
• Regulatory requirements regarding privacy at local and
country level
• Additional measures would include firewalls, certification
and anti-virus solutions
© P K Mallik
System Management
• System Monitoring and Management
• Instrumentation for escalating key events
• Performance monitoring and diagnostics
• Compatibility with application and product
instrumentation
• Optimizes hardware and software resources
• Expensive to deploy and maintain
© P K Mallik
Common Components
• Can be a library or published services
• Nature of services or methods provided
• Usage rules must be defined
• Typically fall into these categories
• Interface Patterns or Facades
• System Service Providers (Security, Access Control, Audit)
• File Management
• Instrumentation
© P K Mallik
Architecture
to Design
COMMON
COMPONENTS
CONSTRAINTS
RISKS ASSUMPTION -
IMPACT
© P K Mallik
Agile
Architecture
Solution Modelling
Evolutionary Design
NFR Management
© P K Mallik
Solution
Modelling
Three Dimensional Models
Business/Product/Service
for which the software is
used
Features and Functionality
which are provided by the
software
Technology and
Configuration used to run
the software
Release objectives and
scope set on one
dimension
Teams defined on another
Product Backlog on the
third
© P K Mallik
F5
Traditional– Technology Driven
Model
F1
F2
F3
F4
T1 T2 T3 T4 T5 T6 T7
Technologies
© P K Mallik
F5
Agile– Business Driven Model
F1
F2
F3
F4
P1 P2 P3 P4 P5 P6 P7
Businesses/Products/Services
© P K Mallik
Agile Architecture - Themes
Data Layer
Business Layer
Db Server No SQL
Content
Server
User Interface Layer
Usability Themes
User Story 1 User Story 2
Maintainability Themes
Performance Themes
Task1Task2
Task1Task2Task3
Security ThemesEpic
User Story 3
Availability
Themes
QueryMemory/CPURendering
AuthenticationAccessControlDataFilter
© P K Mallik
NFR Management
Features User Story Security Performance Usability Others
RBAC Data Filter Capacity Latency UI Preferences Menu
Feature 1 User Story 1 X X X X X X
User Story 2 X
User Story 3 X X X X X
Feature 2 User Story 4 X X
Feature 3 User Story 5 X X X
User Story 6 X X X
User Story 7 X X X X
Feature 4 User Story 8 X X
User Story 9 X X X
User Story 10 X X X X
Feature 5 User Story 11
User Story 12 X X X X
Implementation approach Template Component Service Optimise Db Template Template Template
Implemented by Story Story Story Activity Story Story Story
© P K Mallik
Object
Model
Design Framework
User Interface Business Rules Data Management External Interfaces
Forms
Grids
Workflow
Validations
Stored Procedures/Triggers
Database
Content
Configuration
Protocols
Transport
Content/Payload
GenericProduct
Flex
Flash Browser YAWL
DROOLS
ANSI (Oracle/SQL Server)
File Services
Web Services REST/SOAP
Blaze
Server Interface
Model
Application
Objects
Logical Model
Group Model
Data
Model
Interface
Objects
Implement
Pagination
Actions
Process Flow
Work Flow
Rules
Transaction Manager
Persistence Manager
Interface Manager
File Manager
Eclipse for Modelling
BIRT over RDB
Custom Workflow YAWL TemplatesJBEAM
Calculations
Calculations
© P K Mallik

Architecture Concepts

  • 1.
    © P KMallik Architecture Concepts Principles, Types, Views, Components, Agile
  • 2.
    © P KMallik Agenda Principles Types Views Components Architecture to Design Agile Architecture
  • 3.
    © P KMallik Architecture Principles Separation of concerns Points of failure Scalability Access control Architecture Styles (types of architecture, event driven, process centric) Segregation
  • 4.
    © P KMallik Example – Shopping Site
  • 5.
    © P KMallik Separation of Concerns • Each Feature must be addressed by a single architectural component • One component may address multiple features a single feature should not be managed by multiple components. • If multiple components manage a feature they should be bundled together as a ‘package’ and treated as a single component (note if you end up with only 1 component in your system it is no longer architecture)
  • 6.
    © P KMallik Local Log Points Of Failure • Number of components that need to fail for the entire system to shut down • Larger the points of failure the more stable the overall architecture • Usually has a trade off with performance • Use Asynchronous interfaces to reduce failure points • Use Hardware redundancy to provide for switchover • Clustering (hot switchover) • DR Site (cold switchover) • Protect ‘sensitive’ components from overloading Order Booking Payment Audit Logging Payment Gateway (3rd Party) Order Booking Payment Audit Logging Payment Gateway (3rd Party) Local Log Drip Feed
  • 7.
    © P KMallik Workload/Usage Patterns • Define peak/off peak patterns and expected ‘surges’ • Project expected growth over period of time • Provide for capacity to absorb peak usages • Provide economically viable solution • Pattern could be Users, Transactions, Data Loading, Processing • Scale Out when • Peaks are for short periods of time • One time surge (such as start- up) • Budgets prevent large investments in hardware and licenses • Scale Up when • There is a predictable growth in volumes • Growth patterns do not have significant peaks and troughs • Budgets support initial overcapacity
  • 8.
    © P KMallik Segregation • Need to decouple component • High resource usage • Scale out a high usage component • Availability issues • Security Concerns • Example • Reporting/Data warehousing • External Access • Workflow Time Volume Data Feed
  • 9.
    © P KMallik Architecture Types Three Tier Service Orientation Cloud Computing Event Driven High Throughput No SQL Map Reduce
  • 10.
    © P KMallik Three Tier • Default type and most commonly use • Applies the Model View Controller Pattern • Model • Encapsulates Application State • Responds to Queries • Exposes Functionality • Notifies Changes • View • Renders models • Requests for updates • Sends user actions to controller • Allows selection by controller • Controller • Defines application behaviour • Maps user action to model updates • Selects view for response • Separate for each functionality
  • 11.
    © P KMallik Service Orientation • Provide a set of services rather than interfaces to external applications • Ability to monitor and manage the resource usage by services • Ability to allocate system resources based upon simple rules • Service has a contract – key elements of that contract. Same service can have multiple contracts • Who can call • For that caller, what it need to pass • For that caller – what will be response • When caller can expect response • What will be communication channel • Operates on a publish-subscribe model • May be aligned with commercial models which require specific usage data to be maintained
  • 12.
    © P KMallik Event Driven • Used where the system is part of a partially or fully automated operation or a real time loop • Operates on a throw—catch model • Usually built in a single tier with interfaces with other systems for Human Machine Interface (such as mobile alerts, emails) • Decouples application and component interfaces • Internal interfaces through components raising and processing events • Typical example would be system monitoring and management
  • 13.
    © P KMallik Cloud Computing • Hosted environment provided by vendors for • Web Sites • Databases • File Services • Images and Videos • Removes need for administrating site and managing scaling • Different architectural components may be deployed on different clouds • Provides elasticity in capacity building and management • Commercials depend upon nature of site and type (public/private) of usage (Rate Controls) • Private clouds help reduce capital budgets with shared services • Typical Usages • Mobile Applications • Customer Services • Business Intelligence
  • 14.
    © P KMallik No SQL Databases • No SQL databases support unstructured data by replacing RDBMS concept of row with ‘Document’(Mostly JSON) • Schema is not enforced by database, consuming applications are expected to deal with structure • Data is mostly distributed across some kind of network file system for resilience Key Value Databases for high performance and scalability Document Databases for high flexibility required for storing content Wide Column Stores for storing data in columns for high performance queries and widely used in Big Data processing Graph Stores to store relationships between data nodes without the restriction of a strict schema
  • 15.
    © P KMallik Map Reduce Architecture • Map-Reduce architecture is built for handling terabyte – petabyte data load using cluster of cheap commodity nodes • Algorithms are expressed as two functions. Map function travels over network where data resides and applies on data • Final results are combined back using reduce function • Our own IP – Data Extraction Platform (DEP) is built using Map- Reduce which provides grid scale capability to handle massive data loads
  • 16.
    © P KMallik Architecture Views Overview Business Application Technical Information Deployment
  • 17.
    © P KMallik Enterprise Architecture
  • 18.
    © P KMallik Business Architecture CEO Operations Finance Vendor Mgmt Sales Members Support Editing Receipts Banking investments Accounting Acquisition Delivery Payments Payments Promotions Events Loyalty Mgmt Products Data Analysis
  • 19.
    © P KMallik Application Components Website Content Management Workflow CRM Payment Gateway BPM Vendors
  • 20.
    © P KMallik Layering Data Layer Business Layer Db Server No SQL Content Server User Interface Layer Web Client App 3rd Party CRM Content Transactions Interface BPM
  • 21.
    © P KMallik Information Architecture Main Database (Entities and Relations) DR Site Configuration Regional Databases Content Server Content BackupPurchase Payment Pricing Delivery Customer C R U R U R U Product R C R U R U
  • 22.
    © P KMallik Technical Architecture O/S App Server Web Site Content Server Workflow LDAP O/S Database Server O/S BPM External Interfaces O/S Mail Server
  • 23.
    © P KMallik Deployment Architecture Members Partners Employees VPN Scale Up Scale Out Server Farm
  • 24.
    © P KMallik Architecture Components Independently maintained software Addresses one or more functional and non-functional requirements Selection from a class of similar software on the basis of Compatibility with proposed stack Cost Fitment with client requirements Common Components Portal Workflow BPM ETL Document Management Rules Engine Security Systems Management
  • 25.
    © P KMallik Common Architecture Components • Products and tools regularly used in systems • List of most popular components • Integration & Workflow • Document Management • Rules Engine • Security • System Management • Business Intelligence
  • 26.
    © P KMallik Workflow • Products which manage processes within an organisation • Generates tasks on completion of an event • Has capability to determine which task to initiate depending upon configurable parameter • May be bundled with CRM and Document Management products • Primarily meant to handle manual tasks and does not scale well to large transaction volumes
  • 27.
    © P KMallik Integration • Define and manage external interfaces • Manage processes across multiple systems or organisations • Process integration defined through standard language (such as BPEL or BPMN 2) • Provide an Orchestration service to compose business views from multiple sources • Provide a highly configurable platform when dealing with multiple partners or systems • Can be implemented in conjunction with workflow engines • Not to be used for high volume data processing (use ETL tools instead) for integration or data loading
  • 28.
    © P KMallik Document Management • Repository based tool to create, modify and publish documents • Manages access through Access Control Lists or third party Role Based Access Control systems • Maintains history of changes and can produce specific version • May include multiple channels including RSS feeds • Significantly improves productivity • Can extend to maintain content, specially for web based applications • Manage feeds from external devices (scanners, cameras etc) • Meant for stable process flows, has problems when document management processes change frequently • Not very cost effective for small volumes
  • 29.
    © P KMallik Rules Engine • Define and manage complex business rules with high transaction volumes • Ideally suited for STP type of requirements • Can be combined with Workflow or BPM engine to determine process direction • May be used to make complex rule driven calculations • Expensive and process hungry
  • 30.
    © P KMallik Security • Manage access to data and functions • Three aspects • Identity Management – Issue/update password, smart card, biometrics • Authentication – Verify identity of user • Access Control – Determine access to functions and data • Each aspect may be addressed by a different product • Regulatory requirements regarding privacy at local and country level • Additional measures would include firewalls, certification and anti-virus solutions
  • 31.
    © P KMallik System Management • System Monitoring and Management • Instrumentation for escalating key events • Performance monitoring and diagnostics • Compatibility with application and product instrumentation • Optimizes hardware and software resources • Expensive to deploy and maintain
  • 32.
    © P KMallik Common Components • Can be a library or published services • Nature of services or methods provided • Usage rules must be defined • Typically fall into these categories • Interface Patterns or Facades • System Service Providers (Security, Access Control, Audit) • File Management • Instrumentation
  • 33.
    © P KMallik Architecture to Design COMMON COMPONENTS CONSTRAINTS RISKS ASSUMPTION - IMPACT
  • 34.
    © P KMallik Agile Architecture Solution Modelling Evolutionary Design NFR Management
  • 35.
    © P KMallik Solution Modelling Three Dimensional Models Business/Product/Service for which the software is used Features and Functionality which are provided by the software Technology and Configuration used to run the software Release objectives and scope set on one dimension Teams defined on another Product Backlog on the third
  • 36.
    © P KMallik F5 Traditional– Technology Driven Model F1 F2 F3 F4 T1 T2 T3 T4 T5 T6 T7 Technologies
  • 37.
    © P KMallik F5 Agile– Business Driven Model F1 F2 F3 F4 P1 P2 P3 P4 P5 P6 P7 Businesses/Products/Services
  • 38.
    © P KMallik Agile Architecture - Themes Data Layer Business Layer Db Server No SQL Content Server User Interface Layer Usability Themes User Story 1 User Story 2 Maintainability Themes Performance Themes Task1Task2 Task1Task2Task3 Security ThemesEpic User Story 3 Availability Themes QueryMemory/CPURendering AuthenticationAccessControlDataFilter
  • 39.
    © P KMallik NFR Management Features User Story Security Performance Usability Others RBAC Data Filter Capacity Latency UI Preferences Menu Feature 1 User Story 1 X X X X X X User Story 2 X User Story 3 X X X X X Feature 2 User Story 4 X X Feature 3 User Story 5 X X X User Story 6 X X X User Story 7 X X X X Feature 4 User Story 8 X X User Story 9 X X X User Story 10 X X X X Feature 5 User Story 11 User Story 12 X X X X Implementation approach Template Component Service Optimise Db Template Template Template Implemented by Story Story Story Activity Story Story Story
  • 40.
    © P KMallik Object Model Design Framework User Interface Business Rules Data Management External Interfaces Forms Grids Workflow Validations Stored Procedures/Triggers Database Content Configuration Protocols Transport Content/Payload GenericProduct Flex Flash Browser YAWL DROOLS ANSI (Oracle/SQL Server) File Services Web Services REST/SOAP Blaze Server Interface Model Application Objects Logical Model Group Model Data Model Interface Objects Implement Pagination Actions Process Flow Work Flow Rules Transaction Manager Persistence Manager Interface Manager File Manager Eclipse for Modelling BIRT over RDB Custom Workflow YAWL TemplatesJBEAM Calculations Calculations
  • 41.
    © P KMallik

Editor's Notes

  • #41 Large Object model Performance issues Interface Objects can come from Object of Data Model Sandeep to explain