2. Mr. White has fifteen years of experience designing and managing the
deployment of Systems Monitoring and Event Management software. Prior
to joining IBM, Mr. White held various positions including the leader of the
Monitoring and Event Management organization of a Fortune 100 company
and developing solutions as a consultant for a wide variety of organizations,
including the Mexican Secretaría de Hacienda y Crédito Público, Telmex,
Wal-Mart of Mexico, JP Morgan Chase, Nationwide Insurance and the US
Navy Facilities and Engineering Command.
Andrew White
Cloud and Smarter Infrastructure Solution Specialist
IBM Corporation
4. Ground rules for this
session…
• If you can’t tell if I am trying to be funny…
–
GO AHEAD AND LAUGH!
• Feel free to text, tweet, yammer, or whatever.
Use
• If you have a question, no need to wait until
the end. Just interrupt me. Seriously… I
don’t mind.
5. Why are we here?
I am going to share some of what I have learned about
Software Defined,
Continuous Integration,
& Process Management
7. According to Enterprise
Management Associates…
“Software-Defined” serves to “abstract app/service design and
delivery away from the details of the hosting/delivery technologies.
This is delivered by making use of technical enablers
including virtualization and programmability (API’s)
This approach drives Service-Aligned IT and
allows for more flexible applications
Preference is given to open solutions that shift control from
hardware to software and leave single purpose appliances for
flexible capabilities managed from a central location
8. The 4 key principles for
AT&T:
Domain
2.0
Open
–
APIs
are
the
perfect
tool
Simple
–
More
common
infrastructure
Scalable
–
Supports
growth
Secure
–
Protect
the
Control
Plane
Source: John Donovan, Senior Executive VP AT&T, Keynote Presenter at Open Networking Summit 2014, 4 MAR 2014
9. What is driving this move?
Three Trends
Vitrualization
Utilization and operation
Cloud Computing
Building blocks (compute, network, and
storage) with economies of scale
Internet of Things
Home Cinema, Connected Car
Two Industry Initiatives
Software Defined Environments
New architectural approach, Open Network
Foundation - OpenFlow/OpenDaylight, Open
Data Center Alliance
Network Function Virtualization
New architectural approach, leaving dedicated
hardware for VMs, ETSI
Three Implications
Network Cloud
Lower cost, simplified operations, flexibility
Use Cases
CDN, video on demand, home automation
Industry Status
Wide support for ONF and NVF
10. Increasing
Complexity
§ Heterogeneous environments
§ Organizational silos
§ Skill gaps
Massive
Scale
§ Users, transactions, data
§ Rapid demand cycles
§ Unpredictable
Rapid
Pace
§ Evolving ecosystem
§ Minimize time to value
§ Accelerating business needs
Today’s IT infrastructures are too complex, provide poor scalability, and are
slow to keep up with today’s rapid rate of change
A new set of challenges
V1 V2 V3 V4 V5 V5 ... …. Vn
C
C
W1 W2 W3 W4
R1 R2 R3
Traditional
(Systems of Record)
Emerging
(Systems of Interaction)
Workload View
11. Future
§ Rapidly changing workloads,
dynamic patterns
§ Dynamic automatic composition
of heterogeneous system
§ Autonomic and proactive
management
Current
§ Diverse workload, limited
patterns
§ Homogeneous resource
pooling
§ Expert configuration and
mapping of workload
Traditional
§ Few, stable, and well known
workloads
§ Fixed System hardware,
manual scaling
§ Hardwired workload, minimal
configuration
W1 W2 W3 W4
R1 R2 R3
Volatile workload characteristics result from changing business requirements
V1 V2 V3 V4 V5 … Vn
V1 V2 V3 V4 V5 V5 ... …. Vn
C
C
Workloads are volatile
12. SDE is an Enabler
Software Defined Environment
Cloud Environment
Traditional
Environment
Social and
Mobile
Big Data and
Analytics
Other
Business
Applications
Workload
Service
Delivery
IT Infrastructure
Programmable, open standards-based infrastructure foundation
to enable cloud, mobile and other dynamic enterprise solutions
SDE is the infrastructure
approach to provide the
most efficient and scalable
cloud solutions
SDE improves agility of business applications and
accelerates the application lifecycle through rapid change
13. So@ware
Defined
Environments
provides
abstracEons
of
workloads,
services
and
infrastructure
and
an
end-‐to-‐end
mappings
.
Workload Abstraction
Based on pattern and
functional and non-functional requirements
Resource Abstraction
Semantically rich abstractions of heterogeneous
resource capabilities and system components
Mapping to resource
Map requirements to potential system
architectures. Proactively orchestrate
infrastructure and workload
Continuous Optimization
Autonomously construct available system
architecture to optimize workload outcome
Agility
EfficiencyConsumability
IMG
IMG
IMG
Agile Workload
Development Services
Workload Abstraction
SSD HDD
Tape
PowerVM
x86 KVM
PowerVM
x86 KVM
RDMA
Ethernet
Software Defined Compute, Network and Storage
Agility, Consumability, Efficiency (ACE)
Resource Abstraction
Software Defined Environments
Continuous, Autonomous Mapping
J2EE/OLTP
Transactional
Map/Reduce
Analytics
Web 2.0 Pattern
Web
14. Increasing Automation
SDE fully integrates IT infrastructure across resource domains to maximize
utilization, ensure compliance and decrease administration costs
BEFORE AFTER
StorageNetwork
Compute Continuous Optimization
+
+
+
+
+
+
+
+
+
+
+
+
Compute
§ IT silos and costly specialization
§ Slow and manual
§ Reactive administration
§ Fully integrated management
§ Rapid, repeatable and automated
§ Proactive administration
Policy
Policy
Policy
Policy
Software Defined Environment
Application
Aware
Policy
SDE in Action
15. Software Defined Networking (SDN) moves the network control
plane away from the switch to the software – for improved
programmability, efficiency and extensibility
Virtualized Network
OS
OS
OS
OS
SDN API
Open Flow
Open Flow
Open Flow
Software Defined Control Plane
SDN Controller
& Analytics
Routing API Traffic Engineering API Flow Insertion API Firewall API
routing
VPN
…
monitoring
Direct Access to Physical Network
Traditional Switches
Console Based HW Configuration
routing
VPN
IPS
monitoring
OS
routing
VPN
IPS
monitoring
OS
routing
VPN
IPS
monitoring
OS
routing
VPN
IPS
monitoring
OS
Network Services
Traditional switch and router vendors being disrupted by the emerging SDN
16. What is the promise
of Software-Defined
Everything?
18. What the CIO is hoping
• Economies of Scale from End-to-End Virtualization
– Develop a truly shared infrastructure
– Eliminate “vendor lock-in”
– Compete on cost with 3rd party IT providers
• Break Down the Silos
– Align with services and not technologies (silos of virtualization
are still silos)
– Improve time to value
– Reduce the number of IT specialties in the workforce
• Empower the business
– Enable the self service consumption of IT services
– Simplify the services being offered
– Enable continuous improvement
19. What the architects are hoping
• Cleanly separate the environment into four layers (planes):
Management, Services, Control, and Forwarding - providing
the architectural underpinning to optimize each plane within
the network.
• Centralize the appropriate aspects of the Management,
Services and Control planes to simplify the design and lower
operating costs.
• Use the Cloud for elastic scale and flexible deployment,
enabling usage-based pricing to reduce time to service and
correlate cost based on value.
• Create a platform for network applications, services, and
integration into management systems, enabling new
business solutions.
• Standardize protocols for interoperable, heterogeneous
support across vendors, providing choice and lowering cost.
20. Reality Sets in
The current environment is not ready for change
The staff is overworked, staffing levels are dropping, open
req’s go unfilled to an inability to find adequate talent.
The business won’t abandon “legacy tools”
Increasing amounts of governance are established to manage the chaos
Technical debt and security risks cause
incidents that distract from deployments
21. Architecture by Accident
The Humble Start…
Meeting Demand…
The First Bottleneck…
The Second Bottleneck…
Becoming Mission
Critical…
Enabling SOA…
The Fun Begins…
How Did We Get Here?
22. Game changers
1. Increased demands for high availability and
low latency
2. The visibility gap grows
3. Market forces drive increased velocity and
volume of changes
4. Productivity losses and customer satisfaction
decreases impact the business
23. Broken Promises
The ultimate result in the exact opposite of what
the CIO initially hoped for:
• Communication failures
• Security incidents
• Poor performance
• Compliance failure
• Higher costs
25. Software delivery is
critical to success
86%
of
companies
believe
so/ware
delivery
is
important
or
cri5cal
25%
leverage software delivery effectively today
But only…
Source: “The Software Edge: How effective software development drives competitive advantage,” IBM Institute of Business Value, March 2013
69%
outperform
those who don’t
of those who
leverage software
delivery today
28. Feedback Loops
Unfortunately feedback has taken on both positive and negative
indications. In reality, positive feedback is not “praise” and
negative feedback is not “criticism.” Positive feedback
reinforces while negative feedback balances.
Profits
Productivity
Cost Cutting
Reinforcing
Balancing
31. Be Careful of Good
Intentions
Availability
Change
Frequency
Change
Size
Change
Capability Change
Risk
(-)
(+)
(+)
(-) (-)
Business
Value
Business
Demand
Change
Backlog (+)
(+)
(+)
(+)
(-)
(+)
Change
Process
Release
Process
(+)
(+)
(-)
(+)
(+)
Adapted From: http://www.lean4it.com/2013/05/devops-cld-part-2.html
32. Be Careful of Good
Intentions
Availability
Change
Frequency
Change
Size
Change
Capability Change
Risk
(-)
(+)
(+)
(-) (-)
Business
Value
Business
Demand
Change
Backlog (+)
(+)
(+)
(+)
(-)
(+)
Change
Process
Release
Process
(+)
(+)
(-)
(+)
(+)
Change
Automation
Adapted From: http://www.lean4it.com/2013/05/devops-cld-part-2.html
(+) (-)(-)
33.
34. Organizations don’t fail
because they take the
wrong path, they fail
because they can’t
imagine a better path
than the one they are on.
-- Marty Neumeier
39. Turning Services Into Solutions
Service
Interface
Automa5on
Orchestra5on
Choreography
Business
Service
Offering
Billing
Customer
Management
Add
Customer
Order
Management
Assign
Service
to
Customer
Order
Fulfillment
Provisioning
Deploy
Device
Configure
Device
40. Palette of library
assets enable easy
workflow composition
through drag and drop
Access to rich libraries
(toolkits) of reusable
automation assets that
enable to speed
automation creation
Rich set of actions types,
flow control, data handling
primitives that simplify
creation of complex
automations
Easy workflow action editing
for managing: data mapping,
error recovery options,
implementation details , etc.
Graphical editor for
composing and
connecting
workflows
Rich tooling
functions to edit,
version, debug,
optimize workflows
Automating Processes
42. OrŸchesŸtraŸtion [AWR-kuh-strey-shun]
• A central process controls everything and coordinates the
execution of different operations involved in the operation
• The services do not "know” that they are involved in a
composite process
• Only the central coordinator of the orchestration is aware of
the desired outcome,
• The orchestration leverages explicit process definitions to
operate the services in the correct order of invocation
1. the act of arranging a piece of music
2. the planning or execution of events in order to achieve a desired effect
3. The technique of arranging or manipulating, especially by means of
clever or thorough planning or maneuvering
44. ChoŸreŸogŸraŸphy [kawr-ee-OG-ruh-fee]
1. the art of composing ballets and other dances
2. the method of representing the various movements in dancing by a
system of notation
3. The arrangement or manipulation of actions leading up to an event
• Choreography does not rely on a central coordinator.
• Each service knows exactly who and when to execute
• Focuses on the exchange of messages and information
• All services need to be aware of the business process,
operations to execute, messages to exchange, timing, etc.
46. Choreography vs. Orchestration
• From the perspective of composing services to
execute business processes, orchestration is a more
flexible paradigm and has the following advantages
over choreography:
– The coordination of component processes is centrally
managed by a known coordinator.
– Web services can be incorporated without their being
aware that they are taking part in a larger business
process.
– Alternative scenarios can be put in place in case faults
occur.
Page
46
47. Orchestration Requirements
• Event-based processing
• Coordinate asynchronously between services
• Correlate messages being exchanged
• Provide for parallel processing
• Allow for transaction roll-back
• Manipulate and transform data between messaging
partners
• Be able to manage long running business
transactions and activities
• Have a robust mechanism for fault and error
handling
48. Why use an event-based
orchestration engine?
to have the ability to receive
real-time feedback to assist its
decision making processes
50. Environments
QA
PROD
Banking Application
Banking Application
Banking Application
DEV
IBM UrbanCode Deploy
OpenStack Heat
IBM Platform Resource Scheduler
Server
Storage
Network
Application "
Lifecycle
Applications
Heat Orchestration Template (HOT)
Heat Orchestration Template (HOT)
OpenStack Heat
IBM Platform Resource Scheduler
Server
Storage
Network
TEST
IBM Cloud Orchestrator
Architecture on Purpose
Public
Dedicated
Private
Traditional
IT
Application
template
Infrastructure
template
Hardware
51.
52. Completing the journey
Define
• Review the existing architecture
• Review the business outcomes
• Define the end state
Prioritize
• Consolidations
• Technologies to virtualize
• Business processes to model and workflows to automate
Execute
• Look for early wins
• Evolve incrementally
• Organize the teams effectively
53. You have to be realistic about how fast
you can mature. Iterating helps form a
cultural of continuous improvements
Iterative development
