IIA3: Better ways of developing
software - or coding like a unicorn
Michael Coté | @cote
Better ways of developing software
or coding like a unicorn
@cote
July 2016
2 Slides: http://cote.io/unicorn-coding/
• @cote – Director, Marketing at
Pivotal
• Former industry analyst at 451
Research and RedMonk
• Corporate strategy and M&A at
Dell
• Former software developer
• More: cote.io/podcasts,
http://cote.io, cote@pivotal.io, The
Register DevOps column.
Hello!
3
4
Meanwhile, in business-land …
5
“Silicon Valley is
coming …
we are going to
work hard to make
our services as
seamless and
competitive as
theirs.”
-Jamie Dimon, CEO@JPMC,
2015 letter to shareholders
“If advantages eventually disappear, it
only makes sense to have a process
for filling your pipeline with new ones.
This in turn means that rather than
being an on-again, off-again
mishmash of projects, your innovation
process needs to be carefully
orchestrated.”
- Prof. Rita McGrath, “Transient Advantage,” HBR, June 2013
“Cloud Native” is IT’s answer to transient advantage
6
Since 2000,
52% of the F500
are no longer on
the list
Sources: “Transient Advantage,” above; F500 figure from 2014 Constellation Research Summary, R. “Ray” Wang, Feb. 2014.
Cloud Native Business Theory
(1.) In an era of transient advantage, the
inherit agility of software is one of the best
strategic tools.
(2.) However, software in large organizations
does not currently “work like that.”
(3.) The fix is changing how we think of the
process of software, supported by cloud
native technologies and practices.
7
8
It’s not much of an IT department, but I’m
sorta attached to it …
9
Source: Gartner press release, Aug 2015; “The Agile Advantage,” MeriTalk/Accenture, May 2015; Cutter Consortium, July 2015.
IT is failing to meet the transient advantage needs
of the business
What is your IT
organization's role
in business
innovation?75% of application
development
supporting digital
business will be
built not bought by
2020
10
A user-centric, small batch approach creates a
software defined business
11 Source: Pivotal Labs, March 2016. See also “Agile Transformation in the Age of the 3rd Platform,” Al Hilwa , IDC, Feb 2016 (unpublished)
Exploration Discovery
Feedback Loop
Continuous Delivery
Release once every 6 months
More bugs in production
Release early and often
Higher quality code
DevOps
Not my problem
Separate tools, varied incentives,
opaque process
Shared responsibility
Common incentives, tools, process,
and culture
Microservices
Tightly coupled components
Slow deployment cycles waiting on
integrated tests teams
Loosely coupled components
Automated deploy without waiting on
individual components
Platform, Standard Practices and Tools, Cloud Native Culture
TraditionalITCloudNative
The Process
13
“We spent a lot of time talking about technology, but
technology really isn't the meat of the conversation that
needs to occur.” Matt Curry, Allstate
14
Source: Matt Curry quote from Feb 2016 TheNewStack interview; “Agile Transformation in the Age of the 3rd Platform,” Al Hilwa ,
IDC, March 2016.
Benchmark your agile behaviors, plan catching up as
needed
Source: “Town Hall: Agile in the Enterprise,” Mike West, Nathan Wilson, Thomas Murphy, Dec 2015, Gartner AADI US conference.15
Discovering and delivering the
software your customer actually
wants
• Most organizations
emphasize building to plan
• In reality, the “plan”
constantly evolves
• Learning organizations
explore and experiment
How most
businesses
think they’re
doing
How smart
businesses
know they’re
doing
16
Use small batch thinking at all levels
• Weekly, even daily
deploys to production
• Deploy to production
each time, avoid
waterscrumfall
• Relies on:
- Product approach
- Small batch oriented
pipeline and process
- Cloud native automation
Inception
Hypothesize
Validate
w/real use(r)s
Did it work?
Sources: “Good Software is a Series of Little Failures,” Coté, April 2016; The Lean Startup, Eric Ries, 2011. The Lean Enterprise, Barry
O'Reilly, Jez Humble, and Joanne Molesky. See also overview of this approach at the IRS from Dec 2015.17
Small batches reduce risk and increase controls
18
Five risks that small batches reduce
1. Bug swarms – less software leads to faster
fixing
2. Useless software – don’t wait 6+ months to
find out, when you can find out every 24
hours
3. Stymied innovation – daily opportunities to
learn and innovate
4. Budget overruns – check in daily on ROI
and projections. Shut it down early and re-
try if needed
5. Schedule elongation – if you ship every
day, guaranteed, how can you ever be late?
Source: "Dear Developers, Small Batch Releases Are Your Friend," Coté, Feb 2016, also in The Register.
"There have obviously been culture
shocks. What is more interesting
though is that the teams that tend to
have the worst culture shock are not
those typical teams that you might think
of, audit or compliance. In fact, if
you’re able to successfully
communicate to them what you’re
doing, DevOps and all of the
associated practices seem like
common sense. [Auditors] say, 'Why
weren’t we doing this before?’”
"DevOps Enterprise Adoption at E*TRADE,”
InfoQ/DOES, Jan 2016.
DevOps, the word defines itself
19
(1.)
DevOps is the practice of operations
and development engineers
participating together in the entire
service lifecycle, from design through
the development process to production
support.
(2.)
DevOps is also characterized by
operations staff making use of many of
the same techniques as developers for
their systems work.
Source: “What Is DevOps?,” The Agile Admin, Dec 2011. For more: Effective DevOps, Jennifer Davis, Katherine Daniels, May, 2016; see
The Practice of Cloud System Administration, chapter 8 for an in-depth definition and discussion. Also, see Damon Edward's 2012 piece
"Use DevOps to Turn IT into a Strategic Weapon.” Also, The Phoenix Project for a story of DevOps in the style of The Goal.
Defining DevOps by practices and goals, part 1
20
Defining DevOps by practices and goals, part 2
21
Go big with small batches
Sources: "Ford teams with Pivotal, bets on Cloud Foundry," Larry Dignan, ZDNet, Dec 2015;”Application Modernization, Service By
Microservice," Forrester, Dec 2015. See also BMC Software case study in The Concise Executive Guide to Agile and Cutter Executive
Report, Vol. 9, No. 9, 2008.
"We did an analysis of hundreds
of projects over a multi-year
period. The ones that delivered
in less than a quarter
succeeded about 80% of the
time, while the ones that lasted
more than a year failed at
about the same rate. We're
simply not very good at large
efforts.”
-Dev leader, large financial22
“… sped up software
development times from
months to weeks using
agile techniques.”
The Tools
23
24
Sources: "The New Developer Landscape: Understanding the Modern Software Developer," Al Hilwa, March 2016. See also: "Application
Modernization, Service By Microservice," Kurt Bittner and Randy Heffner, Dec 2015, Forrester. See the“The RedMonk Programming
Language Rankings: January 2016” for popular programming languages.
A microservices approach gives you organizational
agility, at the expense of architectural complexity
25
Benefits
• Decoupling service
releases, teams
• Faster innovation cycles
• Composite applications
vs. monolithic apps
Problems
• The one about the
murder mystery
• Incompatible APIs
• You still need other
people to do work
It’s worth solving the problems if you want the benefits
…and it’s possible!
The application framework promises
Codebase
Dependencie
s Config
Backing
Services
Build,
Release,
Run
Processes
Port
Binding
Concurrency
Disposability
Dev/Prod
Parity
Logs
Admin
Processes
Source: “The Twelve-Factor App.”
• A set of best practices for
developing and deploying
cloud-native software.
• Practices translate into
platform features and
workflow requirements.
26
A deep look at the cloud platform value line
27
Source: Pivotal; "Technical Dive into Cloud Native Application Platforms," Brian Gracely, Wikibon, Sep 2015. See also "The cloud-native
future," Casey West, O'Reilly Radar, Aug 2015
Transforming:
28
Transformation starts with “management”
• Leading change
management
• Setting,
communicating,
tracking goals
• Dramatic organization
change, gradually
Sources: Leading the Transformation, 2015; “Management’s Job is orchestrating the ‘why,’” 2015; The Concise Executive Guide to Agile,
2010.29
“At the end of the day, we are just looking to start
by changing one behavior, which can get the
momentum spinning to drive a much larger
cultural change.” -Matt Curry, Allstate
Staffing: There is no talent shortage
F100 CTO:
“But Netflix has a superstar dev team, we don’t!”
@adrianco:
“We hired them from you.”
Source: @blueboxjesse, Oct 2014. See also “There is no talent shortage.”30
To change, you must actually change
Or, how t-shirts jump-start digital transformation
“I can’t tell you what having a leader
stand up in front of an organization
with a hoodie and t-shirt does to
cultural change. It all of the sudden
makes it OK for everyone within that
organization to participate in
change.”
-Matt Curry, Allstate
31 Source: “How Allstate Upped Productivity By Killing Meetings & Wearing T-Shirts,” Coté, June 2016.
Dealing with legacy: the stuff that makes all the
money
Quarantine
• Low-impact
applications
• Lacks good
testing, risky to
change
• Change-resistant
priorities
Migration Strategies
• Virtualize and re-platform
to control costs and
maximize management
• API gateways combined
with the strangler pattern
• Portfolio management
(cf. The Three Horizons)
Sources: “DevOps isn't just about the new: It's about cleaning up the old, too,” Coté, April 2016; The Cloud Native Journey, Coté, 2016;
Escape Velocity, Geoffrey Moore, Sep 2011 and "To Succeed in the Long Term, Focus on the Middle Term," Geoffrey Moore, August
2007.
32
Cloud-native approaches speed up the whole process,
more efficiently than previously possible
Source: “Deployments We Can Believe In,” Diego Lapiduz, 18F, June 2015; see also “Barriers to DevOps in Government,” Coté, Oct 2015;
Humana keynote, CF Summit 2015; Gregg Otto at CF Summit NA 2016.
18F/cloud.gov
ATO reduction from
9-14 months to 2-3 days
• Humana Cue Apple Watch app:
“Two people built the app and got it
into the AppStore in five weeks.”
• Large financial institution on PCF:
145 apps with 2 ops people
• “We get features in days, not
weeks, and scale takes minutes,
not months.” –Gregg Otto, Comcast
33
For more: transforming to cloud native
(1.)
Greenfield
Projects
(2.)
Dealing with
Legacy
(3.)
Corporate
Transformation
Get the free booklet at http://cote.io/pivotal34
Thanks!@cote | cote@pivotal.io
Slides: http://cote.io/unicorn-coding/
“We are uncovering better
ways of developing software
by doing it and helping others
do it.”
- The Agile Manifesto, 2001
35
General Electric reserves the right to make changes in specifications and features, or discontinue the product or service described at any time, without notice or obligation. These materials do not constitute a
representation, warranty or documentation regarding the product or service featured. Illustrations are provided for informational purposes, and your configuration may differ. This information does not
constitute legal, financial, coding, or regulatory advice in connection with your use of the product or service. Please consult your professional advisors for any such advice. GE, Predix and the GE Monogram are
trademarks of General Electric Company. ©2016 General Electric Company – All rights reserved.

IIA3: Coding Like a Unicorn (Predix Transform 2016)

  • 1.
    IIA3: Better waysof developing software - or coding like a unicorn Michael Coté | @cote
  • 2.
    Better ways ofdeveloping software or coding like a unicorn @cote July 2016 2 Slides: http://cote.io/unicorn-coding/
  • 3.
    • @cote –Director, Marketing at Pivotal • Former industry analyst at 451 Research and RedMonk • Corporate strategy and M&A at Dell • Former software developer • More: cote.io/podcasts, http://cote.io, cote@pivotal.io, The Register DevOps column. Hello! 3
  • 4.
  • 5.
    5 “Silicon Valley is coming… we are going to work hard to make our services as seamless and competitive as theirs.” -Jamie Dimon, CEO@JPMC, 2015 letter to shareholders
  • 6.
    “If advantages eventuallydisappear, it only makes sense to have a process for filling your pipeline with new ones. This in turn means that rather than being an on-again, off-again mishmash of projects, your innovation process needs to be carefully orchestrated.” - Prof. Rita McGrath, “Transient Advantage,” HBR, June 2013 “Cloud Native” is IT’s answer to transient advantage 6 Since 2000, 52% of the F500 are no longer on the list Sources: “Transient Advantage,” above; F500 figure from 2014 Constellation Research Summary, R. “Ray” Wang, Feb. 2014.
  • 7.
    Cloud Native BusinessTheory (1.) In an era of transient advantage, the inherit agility of software is one of the best strategic tools. (2.) However, software in large organizations does not currently “work like that.” (3.) The fix is changing how we think of the process of software, supported by cloud native technologies and practices. 7
  • 8.
  • 9.
    It’s not muchof an IT department, but I’m sorta attached to it … 9
  • 10.
    Source: Gartner pressrelease, Aug 2015; “The Agile Advantage,” MeriTalk/Accenture, May 2015; Cutter Consortium, July 2015. IT is failing to meet the transient advantage needs of the business What is your IT organization's role in business innovation?75% of application development supporting digital business will be built not bought by 2020 10
  • 11.
    A user-centric, smallbatch approach creates a software defined business 11 Source: Pivotal Labs, March 2016. See also “Agile Transformation in the Age of the 3rd Platform,” Al Hilwa , IDC, Feb 2016 (unpublished) Exploration Discovery Feedback Loop
  • 12.
    Continuous Delivery Release onceevery 6 months More bugs in production Release early and often Higher quality code DevOps Not my problem Separate tools, varied incentives, opaque process Shared responsibility Common incentives, tools, process, and culture Microservices Tightly coupled components Slow deployment cycles waiting on integrated tests teams Loosely coupled components Automated deploy without waiting on individual components Platform, Standard Practices and Tools, Cloud Native Culture TraditionalITCloudNative
  • 13.
  • 14.
    “We spent alot of time talking about technology, but technology really isn't the meat of the conversation that needs to occur.” Matt Curry, Allstate 14 Source: Matt Curry quote from Feb 2016 TheNewStack interview; “Agile Transformation in the Age of the 3rd Platform,” Al Hilwa , IDC, March 2016.
  • 15.
    Benchmark your agilebehaviors, plan catching up as needed Source: “Town Hall: Agile in the Enterprise,” Mike West, Nathan Wilson, Thomas Murphy, Dec 2015, Gartner AADI US conference.15
  • 16.
    Discovering and deliveringthe software your customer actually wants • Most organizations emphasize building to plan • In reality, the “plan” constantly evolves • Learning organizations explore and experiment How most businesses think they’re doing How smart businesses know they’re doing 16
  • 17.
    Use small batchthinking at all levels • Weekly, even daily deploys to production • Deploy to production each time, avoid waterscrumfall • Relies on: - Product approach - Small batch oriented pipeline and process - Cloud native automation Inception Hypothesize Validate w/real use(r)s Did it work? Sources: “Good Software is a Series of Little Failures,” Coté, April 2016; The Lean Startup, Eric Ries, 2011. The Lean Enterprise, Barry O'Reilly, Jez Humble, and Joanne Molesky. See also overview of this approach at the IRS from Dec 2015.17
  • 18.
    Small batches reducerisk and increase controls 18 Five risks that small batches reduce 1. Bug swarms – less software leads to faster fixing 2. Useless software – don’t wait 6+ months to find out, when you can find out every 24 hours 3. Stymied innovation – daily opportunities to learn and innovate 4. Budget overruns – check in daily on ROI and projections. Shut it down early and re- try if needed 5. Schedule elongation – if you ship every day, guaranteed, how can you ever be late? Source: "Dear Developers, Small Batch Releases Are Your Friend," Coté, Feb 2016, also in The Register. "There have obviously been culture shocks. What is more interesting though is that the teams that tend to have the worst culture shock are not those typical teams that you might think of, audit or compliance. In fact, if you’re able to successfully communicate to them what you’re doing, DevOps and all of the associated practices seem like common sense. [Auditors] say, 'Why weren’t we doing this before?’” "DevOps Enterprise Adoption at E*TRADE,” InfoQ/DOES, Jan 2016.
  • 19.
    DevOps, the worddefines itself 19 (1.) DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support. (2.) DevOps is also characterized by operations staff making use of many of the same techniques as developers for their systems work. Source: “What Is DevOps?,” The Agile Admin, Dec 2011. For more: Effective DevOps, Jennifer Davis, Katherine Daniels, May, 2016; see The Practice of Cloud System Administration, chapter 8 for an in-depth definition and discussion. Also, see Damon Edward's 2012 piece "Use DevOps to Turn IT into a Strategic Weapon.” Also, The Phoenix Project for a story of DevOps in the style of The Goal.
  • 20.
    Defining DevOps bypractices and goals, part 1 20
  • 21.
    Defining DevOps bypractices and goals, part 2 21
  • 22.
    Go big withsmall batches Sources: "Ford teams with Pivotal, bets on Cloud Foundry," Larry Dignan, ZDNet, Dec 2015;”Application Modernization, Service By Microservice," Forrester, Dec 2015. See also BMC Software case study in The Concise Executive Guide to Agile and Cutter Executive Report, Vol. 9, No. 9, 2008. "We did an analysis of hundreds of projects over a multi-year period. The ones that delivered in less than a quarter succeeded about 80% of the time, while the ones that lasted more than a year failed at about the same rate. We're simply not very good at large efforts.” -Dev leader, large financial22 “… sped up software development times from months to weeks using agile techniques.”
  • 23.
  • 24.
    24 Sources: "The NewDeveloper Landscape: Understanding the Modern Software Developer," Al Hilwa, March 2016. See also: "Application Modernization, Service By Microservice," Kurt Bittner and Randy Heffner, Dec 2015, Forrester. See the“The RedMonk Programming Language Rankings: January 2016” for popular programming languages.
  • 25.
    A microservices approachgives you organizational agility, at the expense of architectural complexity 25 Benefits • Decoupling service releases, teams • Faster innovation cycles • Composite applications vs. monolithic apps Problems • The one about the murder mystery • Incompatible APIs • You still need other people to do work It’s worth solving the problems if you want the benefits …and it’s possible!
  • 26.
    The application frameworkpromises Codebase Dependencie s Config Backing Services Build, Release, Run Processes Port Binding Concurrency Disposability Dev/Prod Parity Logs Admin Processes Source: “The Twelve-Factor App.” • A set of best practices for developing and deploying cloud-native software. • Practices translate into platform features and workflow requirements. 26
  • 27.
    A deep lookat the cloud platform value line 27 Source: Pivotal; "Technical Dive into Cloud Native Application Platforms," Brian Gracely, Wikibon, Sep 2015. See also "The cloud-native future," Casey West, O'Reilly Radar, Aug 2015
  • 28.
  • 29.
    Transformation starts with“management” • Leading change management • Setting, communicating, tracking goals • Dramatic organization change, gradually Sources: Leading the Transformation, 2015; “Management’s Job is orchestrating the ‘why,’” 2015; The Concise Executive Guide to Agile, 2010.29 “At the end of the day, we are just looking to start by changing one behavior, which can get the momentum spinning to drive a much larger cultural change.” -Matt Curry, Allstate
  • 30.
    Staffing: There isno talent shortage F100 CTO: “But Netflix has a superstar dev team, we don’t!” @adrianco: “We hired them from you.” Source: @blueboxjesse, Oct 2014. See also “There is no talent shortage.”30
  • 31.
    To change, youmust actually change Or, how t-shirts jump-start digital transformation “I can’t tell you what having a leader stand up in front of an organization with a hoodie and t-shirt does to cultural change. It all of the sudden makes it OK for everyone within that organization to participate in change.” -Matt Curry, Allstate 31 Source: “How Allstate Upped Productivity By Killing Meetings & Wearing T-Shirts,” Coté, June 2016.
  • 32.
    Dealing with legacy:the stuff that makes all the money Quarantine • Low-impact applications • Lacks good testing, risky to change • Change-resistant priorities Migration Strategies • Virtualize and re-platform to control costs and maximize management • API gateways combined with the strangler pattern • Portfolio management (cf. The Three Horizons) Sources: “DevOps isn't just about the new: It's about cleaning up the old, too,” Coté, April 2016; The Cloud Native Journey, Coté, 2016; Escape Velocity, Geoffrey Moore, Sep 2011 and "To Succeed in the Long Term, Focus on the Middle Term," Geoffrey Moore, August 2007. 32
  • 33.
    Cloud-native approaches speedup the whole process, more efficiently than previously possible Source: “Deployments We Can Believe In,” Diego Lapiduz, 18F, June 2015; see also “Barriers to DevOps in Government,” Coté, Oct 2015; Humana keynote, CF Summit 2015; Gregg Otto at CF Summit NA 2016. 18F/cloud.gov ATO reduction from 9-14 months to 2-3 days • Humana Cue Apple Watch app: “Two people built the app and got it into the AppStore in five weeks.” • Large financial institution on PCF: 145 apps with 2 ops people • “We get features in days, not weeks, and scale takes minutes, not months.” –Gregg Otto, Comcast 33
  • 34.
    For more: transformingto cloud native (1.) Greenfield Projects (2.) Dealing with Legacy (3.) Corporate Transformation Get the free booklet at http://cote.io/pivotal34
  • 35.
    Thanks!@cote | cote@pivotal.io Slides:http://cote.io/unicorn-coding/ “We are uncovering better ways of developing software by doing it and helping others do it.” - The Agile Manifesto, 2001 35
  • 36.
    General Electric reservesthe right to make changes in specifications and features, or discontinue the product or service described at any time, without notice or obligation. These materials do not constitute a representation, warranty or documentation regarding the product or service featured. Illustrations are provided for informational purposes, and your configuration may differ. This information does not constitute legal, financial, coding, or regulatory advice in connection with your use of the product or service. Please consult your professional advisors for any such advice. GE, Predix and the GE Monogram are trademarks of General Electric Company. ©2016 General Electric Company – All rights reserved.