The hitchhiker’s guide for the confused developer
Making sense of an increasingly mad IT

Uwe Friedrichsen – codecentric AG – 2012-2018
Uwe Friedrichsen

IT traveller.
Dot Connector.
Cartographer of uncharted territory.
Keeper of timeless wisdom.
CTO and Fellow at codecentric.

https://www.slideshare.net/ufried
https://medium.com/@ufried
 @ufried
Have you ever felt like (almost) everyone
and everything in IT is going mad?
Like everything is getting more and more
complex every week?
And you already feel like you hardly can
hold together all the bits and pieces?
But you are getting less and less time
to pick up the plethora of new stuff?
And all that “Agile” just feels like a big hoax
which makes things worse than better?
Do you sometimes feel like you might
miss what is really important?
And instead waste your time
in a crazy rat race to nowhere?
Will AI replace your job in a few years?
And what about IoT, Blockchain
and all those other “disruptions”?
Do you sometimes ask yourself
if you are still on the right track?
Or that you might focus
on the wrong things?
But what are the “right” things?
Do you sometimes feel like this?
Or like this?
Then it might be time for a little
hitchhiker’s guide to the IT of today
But first of all ...
Understanding IT

A complex structure observed from multiple viewpoints
Viewpoint #1

Market evolution
Formal part of
value creation
Solution:
machine
Dynamic part
of value
creation
Solution: man
sluggishness/low dynamic high dynamichigh dynamic
The historical course of market dynamics
and the recent rise of highly dynamic and complex markets
The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.
t1970/80 today
Age of
crafts manu-
facturing
Age of
tayloristic
industry
Age of
global
markets
1850/1900
Spacious markets,
little competition
Local markets,
high customi-
zation
Outperformers exercise
market pressure over
conventional companies
We call the graph shown here the “Taylor Bathtub”.
The “bathtub” curve
Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
Formal part of
value creation
Solution:
machine
Dynamic part
of value
creation
Solution: man
sluggishness/low dynamic high dynamichigh dynamic
The historical course of market dynamics
and the recent rise of highly dynamic and complex markets
The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.
t1970/80 today
Age of
crafts manu-
facturing
Age of
tayloristic
industry
Age of
global
markets
1850/1900
Spacious markets,
little competition
Local markets,
high customi-
zation
Outperformers exercise
market pressure over
conventional companies
We call the graph shown here the “Taylor Bathtub”.
Pre-industrial era
Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
Tailor-made
solutions
Mastery
is key to success
Formal part of
value creation
Solution:
machine
Dynamic part
of value
creation
Solution: man
sluggishness/low dynamic high dynamichigh dynamic
The historical course of market dynamics
and the recent rise of highly dynamic and complex markets
The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.
t1970/80 today
Age of
crafts manu-
facturing
Age of
tayloristic
industry
Age of
global
markets
1850/1900
Spacious markets,
little competition
Local markets,
high customi-
zation
Outperformers exercise
market pressure over
conventional companies
We call the graph shown here the “Taylor Bathtub”.
Industrial era
Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
Cost-efficiently
scale production
Getting more done with less people
is key to success
Formal part of
value creation
Solution:
machine
Dynamic part
of value
creation
Solution: man
sluggishness/low dynamic high dynamichigh dynamic
The historical course of market dynamics
and the recent rise of highly dynamic and complex markets
The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.
t1970/80 today
Age of
crafts manu-
facturing
Age of
tayloristic
industry
Age of
global
markets
1850/1900
Spacious markets,
little competition
Local markets,
high customi-
zation
Outperformers exercise
market pressure over
conventional companies
We call the graph shown here the “Taylor Bathtub”.
Post-industrial era
Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
Continuously respond
to changing demands
Continuous market adaption
is key to success
Key drivers




Pre-industrial era
•  No clear driver





Industrial era
•  Cost-efficiency
•  Scalability
•  Repeatability
•  Stability
•  Efficiency & scale





Post-industrial era
•  Cycle times
•  Adaptability
•  Flexibility
•  Resilience
•  Effectiveness & speed
Viewpoint #2

IT evolution
1960
 1970
 1980
 1990
 2000
 2010
 2020
Complicated

(Business functions)
Complex

(Business processes)
Highly complex

(Business nervous system)
Software crisis
Software engineering
PC
LAN
Internet
Business
Support
of IT
Selective
Holistic
Complicated
Complex
“Moore’s law”
Mobile
IoT
IT has changed a lot over the decades ...
1960
 1970
 1980
 1990
 2000
 2010
 2020
Complicated

(Business functions)
Complex

(Business processes)
Highly complex

(Business nervous system)
Software crisis
PC
LAN
Internet
Business
Support
of IT
Selective
Holistic
Complicated
Complex
“Moore’s law”
Mobile
IoT
Software engineering
... but still we strive to
control our IT of today ...
... based on the concepts
we developed for an IT
almost 50 years ago
Formal part of
value creation
Solution:
machine
Dynamic part
of value
creation
Solution: man
sluggishness/low dynamic high dynamichigh dynamic
The historical course of market dynamics
and the recent rise of highly dynamic and complex markets
The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact.
t1970/80 today
Age of
crafts manu-
facturing
Age of
tayloristic
industry
Age of
global
markets
1850/1900
Spacious markets,
little competition
Local markets,
high customi-
zation
Outperformers exercise
market pressure over
conventional companies
We call the graph shown here the “Taylor Bathtub”.
Remember the “bathtub” curve?
Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
1960
 1970
 1980
 1990
 2000
 2010
 2020
Complicated

(Business functions)
Complex

(Business processes)
Highly complex

(Business nervous system)
Software crisis
Software engineering
PC
LAN
Internet
Business
Support
of IT
Selective
Holistic
Complicated
Complex
“Moore’s law”
Mobile
IoT
Also the business we
support with IT today ...
... is very different from
the business we
supported back then
What is the role of IT today?
IT today is ...


•  ... the nervous system of the business
•  ... an enabler of (disruptive) new business models
•  ... an integral part of the business model (“digitization”)
•  ... the medium for the continuous customer communication
Viewpoint #3

Technology evolution
Disruptive technologies
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
UNIX
Server
Personal
Computer
Notebook
Disruptive technologies
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
Personal
Computer
Standard
Software
OSS
Notebook
Disruptive technologies
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Mobile
networks
(3G+)
Wi-Fi
Modem

9600+ Baud
Disruptive technologies
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Smartphone
Mobile
networks
(3G+)
Tablet
Wi-Fi
Modem

9600+ Baud
Disruptive technologies
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Smartphone
Cloud
Mobile
networks
(3G+)
Big Data
Tablet
Wi-Fi
SaaS
Serverless

FaaS +
Managed
services
Modem

9600+ Baud
Disruptive technologies
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Smartphone
Cloud
Mobile
networks
(3G+)
Big Data
Tablet
Wi-Fi
SaaS
Serverless

FaaS +
Managed
services
PLC
Industrial
PC
 OPC UA
 IoT
Modem

9600+ Baud
Disruptive technologies
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Smartphone
Cloud
Mobile
networks
(3G+)
Big Data
Tablet
Wi-Fi
SaaS
Serverless

FaaS +
Managed
services
PLC
Industrial
PC
 OPC UA
 IoT
Modem

9600+ Baud
Viewpoint #4

Programming model evolution
Programming models
DIY

Do It yourself
Web applications
Enterprise Frameworks

CORBA, SOA, JEE, .net, ...
IoT
Serverless

FaaS +
Managed
services
Cloud native
Microservices
?
Libraries &
Frameworks
Embedded computing
Standard software
customization
HPC

High Performance Computing
Local computing paradigm
Distributed computing paradigm
Complementing viewpoints
System theory - Cynefin
Source: By Snowded - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=33783436
The simple things

•  Do them ad hoc
•  Use check lists
•  KISS!
•  ...
The domain of planning

•  Waterfall, V-Model, ...
•  Iterative & incremental
•  SAFe
•  ...
The domain of uncertainty

•  Inspect & adapt
•  Agile (XP, Kanban, ...)
•  Focus on outcome
•  ...
Never be here in an IT
project! Never, ever!
Source: http://blog.gardeviance.org/2015/02/an-introduction-to-wardley-value-chain.html
WardleyMaps
Source: http://blog.gardeviance.org/2015/02/an-introduction-to-wardley-value-chain.html
Source: http://blog.gardeviance.org/2015/02/an-introduction-to-wardley-value-chain.html
Source: http://blog.gardeviance.org/2015/02/an-introduction-to-wardley-value-chain.html
Pioneer
Settler
Town
Planner
More complementing viewpoints



•  Evolution of “Make or buy”
•  Double Helix
•  Local computing vs. distributed computing
Understanding current IT trends

Complementing the picture
Meta-Trend #1

Digitization (whatever that means ...)
Digitization (Definition)


•  Blurry and misleading term – yet an important change driver
•  Response to market demands and technology evolution
•  Core: IT becomes integral part of business offerings
•  Consequence: Cross-domain boundaries start to dissolve
and leverage – currently still unknown – business models
Digitization (Effect)

•  Uncertainty regarding business model viability
•  Effect: Short iterations, pervasive metrics (incl. measuring
outcome), A/B testing, smart rollout strategies, ...
•  Dissolving domain boundaries
•  Effect: API, platforms, ...
•  Adaption of persistent requirements to a new context
•  Effect: Poor connectivity as a constraint, re-thinking
accessibility, “developer experience” on the API level, ...
Meta-Trend #2

Customer expectations
Customer expectations

•  Great user experience (UX)
•  Fast response times
•  Multi-device capabilities
•  Zero downtime
•  Strong Security (regarding data privacy)
•  Adapts to customer’s needs and demands
Resulting IT trends
Moving fast

•  Minimize cycle times to accelerate the feedback loop
•  Response to business model uncertainty
•  Goal is to minimize idle and value-reducing performances
•  Massive paradigm shift (“Re-thinking IT”)

That is what DevOps actually is about!
Focus on frontends


•  The battle for customers gets decided in the frontend
•  Backends just need to work and must not get in the way
•  UX and UI design become key differentiators
•  Backend becomes commodity
•  Bear all types of frontends in mind (incl. IoT, API, ...)
Cloud-native and serverless

•  Reduce vertical integration depth
•  Focus on value-creating IT performance to move faster
•  Newest incarnation of “make or buy”
•  Managed services as successor of standard software
•  FaaS to orchestrate managed services
•  Managed container and batch jobs for deep business logic
Ambient computing

•  Shift towards human-centered user interfaces
•  Complemented by context-aware computing & UIs
•  “Mobile first” is just a transitional step
•  New types of human-machine interaction
•  Voice, gestures, tactile, ...
•  Augmented by AR & VR interface capabilities
InfoSec

•  Indispensable supplement for most needed technologies
•  Cloud
•  Wireless
•  IoT
•  Mobile
•  ...
•  Must be an integral part of all stages of the IT value chain
IT enabled trends
IT enabled trends

Artificial intelligence / deep learning
Internet of Things (including industry 4.0)
Maker movement (including 3D printing)
Blockchain
•  Currently hyped topics with disruptive potential
•  Actual future impact not yet clearly foreseeable
Evaluating your position

Finding out where you are and where you want to go
Step #1

Create a framework
Framework dimensions



•  Work model evolution
•  Technology evolution
•  Programming model evolution
Dimension 1: Work model evolution
Remember this model?




Pre-industrial era
•  No clear driver





Industrial era
•  Cost-efficiency
•  Scalability
•  Repeatability
•  Stability
•  Efficiency & scale





Post-industrial era
•  Cycle times
•  Adaptability
•  Flexibility
•  Resilience
•  Effectiveness & speed
This model can also be applied to IT
Properties
 Pre-industrial
 Industrial
 Post-industrial
Market
 Emerging market
 Wide & slow market
 Narrow & fast market
Strategic goal
 None (“Just do it!”)
 Cost-efficiency
 Short cycle times
System theory
 Simple
 Complicated
 Complex
Work model
 Ad hoc
 Processes
 Collaboration
Organization model
 Peer to peer
 Hierarchies
 Autonomous teams
Practices
 “Whatever goes”
 ITIL, V-Modell, SAFe
 DevOps, Kanban, XP, Agile
Individuals
 Experts (Cube-shaped)
 Specialists (I-shaped)
 Generalists (T-Shaped)
Automation
 Low
 Medium-high
 Very high
Manual work
 Ad hoc
 Defined and controlled
 Avoided
Programming model
 Do it yourself
Standard Software,
Enterprise Frameworks
Cloud-native, Serverless,
Managed Services
Another, related point of view
Taking this “magic triangle” ...
Good
Fast
 Cheap
Optimizing for quality and cycle times
will result in higher costs
Optimizing for quality and costs
will result in long cycle times
Optimizing for cycle times and costs
will result in reduced quality
... will lead you to this model
You may pick
two
Good
Fast
 Cheap
Industrial IT

Deliver large batches at minimized
costs towards slow markets
Post-industrial IT

Quickly adapt to ever-changing needs
of dynamic, fast-moving markets
Startup IT

Test hypotheses and pivot as fast as
possible to discover a product-market fit
You may pick
two
Good
Fast
 Cheap
Industrial IT

Deliver large batches at minimized
costs towards slow markets
Post-industrial IT

Quickly adapt to ever-changing needs
of dynamic, fast-moving markets
Startup IT

Test hypotheses and pivot as fast as
possible to discover a product-market fit
This is not a pre-industrial IT model,
as it comes with a clear strategic
goal (“Discover product-market fit
as fast as possible”), still it usually
shares many of the other properties
You may pick
two
Good
Fast
 Cheap
Industrial IT

Deliver large batches at minimized
costs towards slow markets
Post-industrial IT

Quickly adapt to ever-changing needs
of dynamic, fast-moving markets
Startup IT

Test hypotheses and pivot as fast as
possible to discover a product-market fit
1
2 3
Product evolution cycle
Work model evolution


•  You can reason about the work model at different levels
•  Company (Market)
•  IT department
•  Product/Project
•  The work model for a given product will change over time
•  You will need to figure out your preferred work model
Dimension 2: Technology evolution
Let us start with the disruptive technologies ...
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Smartphone
Cloud
Mobile
networks
(3G+)
Big Data
Tablet
Wi-Fi
SaaS
Serverless

FaaS +
Managed
services
PLC
Industrial
PC
 OPC UA
 IoT
Modem

9600+ Baud
... aggregate them in groups ...
Assembly
language
1950
 1960
 1970
 1980
 1990
 2000
 2010
 2020
Mainframe
Transistor
Multitasking
High-level
language

Algol
Lisp
Simula
Cobol
UNIX
Server
LAN
Personal
Computer
Standard
Software
OSS
WWW
Notebook
Smartphone
Cloud
Mobile
networks
(3G+)
Big Data
Tablet
Wi-Fi
SaaS
Serverless

FaaS +
Managed
services
PLC
Industrial
PC
 OPC UA
 IoT
Modem

9600+ Baud
Traditional
software
development
Industrial computing / IoT
Web
centered
Cloud native
Mobile
centered
Mobile centered
(Big & Fast) Data centered
... add the trends ...

•  Ambient computing
•  Focus on frontends
•  InfoSec
•  Artificial intelligence / deep learning
•  Maker movement (including 3D printing)
•  Blockchain
... and distill the major directions
Frontends as differentiators
•  Mobile centered à Ambient computing
Backends as commodity
•  Driven by cloud native (especially managed services)
Smart data
•  Big/fast data & machine learning à deep learning à AI
Smart devices
•  Industrial computing & IoT ß Maker movement
InfoSec
•  Needed everywhere ß Blockchain
Traditional
•  including web & mobile centered
Technology evolution


•  All directions are interconnected
•  Still, you will most likely need to focus on one direction
•  As mastering all of them is too much for one person
•  As architect, you will need to get a grasp of all directions
•  Required to help your stakeholders making the best decisions
•  Still, you should deep dive at least into one of the directions
Dimension 3: Programming model evolution
Let us take the programming models ...
DIY

Do It yourself
Web applications
Enterprise Frameworks

CORBA, SOA, JEE, .net, ...
IoT
Serverless

FaaS +
Managed
services
Cloud native
Microservices
?
Libraries &
Frameworks
Embedded computing
Standard software
customization
HPC

High Performance Computing
Local computing paradigm
Distributed computing paradigm
... and aggregate them in groups
DIY

Do It yourself
Web applications
Enterprise Frameworks

CORBA, SOA, JEE, .net, ...
IoT
Serverless

FaaS +
Managed
services
Cloud native
Microservices
?
Libraries &
Frameworks
Embedded computing
Standard software
customization
HPC

High Performance Computing
Local computing paradigm
Distributed computing paradigm
No
paradigm

(random outcomes)
Distributed
computing
paradigm
Local
computing
paradigm
Embedded computing paradigm
... and aggregate them in groups
DIY

Do It yourself
Web applications
Enterprise Frameworks

CORBA, SOA, JEE, .net, ...
IoT
Serverless

FaaS +
Managed
services
Cloud native
Microservices
?
Libraries &
Frameworks
Embedded computing
Standard software
customization
HPC

High Performance Computing
Local computing paradigm
Distributed computing paradigm
No
paradigm

(random outcomes)
Distributed
computing
paradigm
Local
computing
paradigm
Embedded computing paradigm
Transitioning to one of those
paradigms basically means
mostly re-learning how to
develop good software
Most people just do that
in a disguise if they claim
they do “microservices”
Smart device management
including edge computing
usually requires both
paradigms
Putting the framework together
Pre-industrial
Programming
model
Technology
Work model
Industrial
 Startup
 Post-industrial
Traditional
 Backend
 Frontend
 Device
 Data / AI
 InfoSec
Ambient
AR/VR
Cloud native
Serverless
Industrial
IoT/Maker
Big/fast data
ML/DL
Everywhere
(Blockchain)
Random
 Local computing
 Distributed computing
 Embedded computing
Known territory
 New territory
C/S, Web
Mobile
Step #2

Locate your current position
Pre-industrial
Programming
model
Technology
Work model
Industrial
 Startup
 Post-industrial
Traditional
 Backend
 Frontend
 Device
 Data / AI
 InfoSec
Ambient
AR/VR
Cloud native
Serverless
Industrial
IoT/Maker
Big/fast data
ML/DL
Everywhere
(Blockchain)
Random
 Local computing
 Distributed computing
 Embedded computing
Known territory
 New territory
C/S, Web
Mobile
Step #3

Find your desired position
Pre-industrial
Programming
model
Technology
Work model
Industrial
 Startup
 Post-industrial
Traditional
 Backend
 Frontend
 Device
 Data / AI
 InfoSec
Ambient
AR/VR
Cloud native
Serverless
Industrial
IoT/Maker
Big/fast data
ML/DL
Everywhere
(Blockchain)
Random
 Local computing
 Distributed computing
 Embedded computing
Known territory
 New territory
C/S, Web
Mobile
Step #4

Define your personal evolution plan
Define your personal evolution plan

•  Defining the plan is the easy part
•  Read the literature, do the trainings, train your skills, ...
•  Still, it will require time and discipline
•  It’s a combination of personal and company game
•  You cannot expect your company to cover all the efforts
•  You will usually need to invest some of your private time
•  And if your environment does not support you
•  “Love it, change it, or leave it”
•  Trade-offs are fine, but in the end they are your decision!
Staying ahead of the curve

Some general recommendations augmenting your path ahead
Understand post-industrial markets
•  Speed trumps perfection – Rethink everything!
•  Really, really understand the implications of uncertainty!
•  Understand the difference between output and outcome
•  Understand the ideas of bets and options
•  Dismiss cargo-cult agility – learn what “Agile” really means
•  It is a revolution in your head!
•  But it should be an evolution regarding implementation
•  Otherwise you will certainly overstrain your organization
Cross-functional teams
(organized by business capabilities)
Autonomy
(incl. E2E responsibility)
Decentralized control
Microservices
Continuous Delivery
Heterogeneity
 Cloud and Containers
Resilience
Operations automation
Craftsmanship & mastery
Outcome-driven
Beyond budgeting
Feature flow
Lean EAM
Continuous improvement
T-Shaped people
(being empathic)
DevOps
Quick feedback loops
Curiosity
Master the timeless wisdom

•  Especially master the foundations of good design
•  “Loose coupling, high cohesion” & “separation of concerns”
•  “Information hiding” especially with respect to API
•  Domain-driven design can be a useful starting point
•  Good design skills are more relevant than ever
•  Affect understandability, changeability and extensibility
•  Affect usability, stability and acceptance at API level
•  Affect robustness, availability and scalability at runtime
“I would advise students to pay more attention to the
fundamental ideas rather than the latest technology.

The technology will be out-of-date before they graduate.
Fundamental ideas never get out of date.”



-- David L. Parnas (http://www.sigsoft.org/SEN/parnas.html)
Become T-shaped

•  Generalists trump specialists
•  Pure subject matter experts are an industrial concept
•  Collaboration requires understanding your collaborators
•  Leave your comfort zone once in a while
•  Still, have your area of expertise
•  Deep knowledge is still needed – but in a inclusive way
•  Understand the business domain
•  This is what turns you into a “ten times developer”
Learn frontend development ...

•  Learn JavaScript – really!
•  Embrace the concepts of UX and UI design
•  Understand CSS at least a bit
•  Start picking up the concepts of ambient computing
•  Voice-based UI, Video-based UI, AR, VR, etc.
... or embrace cloud native ...

•  Understand and learn what “cloud” means today!
•  Understand the concepts of “serverless”
•  Know the relevant managed service offerings (BaaS/SaaS)
•  Automate everything using managed services
•  Understand operation needs (especially monitoring) ...
•  ... and what existing solutions (managed services) offer
•  Abandon DIY and NIH
•  Actively reduce vertical integration depth
... or try a different direction

•  Smart Data
•  Will heavily shape future decision making processes
•  Be prepared to refresh your math
•  Smart Devices
•  Fast and wild evolution at the moment
•  Lots of exciting challenges – if you like it
•  InfoSec
•  Needed everywhere – yet treated often like an unloved chore
•  Must become an integral part of development and operations
Embrace distributed systems

“(Almost) every system is a distributed system” -- Chas Emerick
“Everything fails, all the time” -- Werner Vogels

•  Understand the non-determinism of distributed systems
•  “Memory, guesses and apologies” & promise theory
•  Understand that infrastructure cannot guarantee robustness
Wrap-up
Wrap-up

•  Moving from an industrial to a post-industrial domain
•  Technology and trends revealing new directions
•  Programming goes distributed and embedded
•  Generalists trump specialists
•  Timeless wisdom as a game changer
•  Assess where you are and where you want to be
•  Love it, change it, or leave it
Uwe Friedrichsen

IT traveller.
Connecting the dots.
Attracted by uncharted territory.
CTO at codecentric.

https://www.slideshare.net/ufried
https://medium.com/@ufried
 @ufried

The hitchhiker's guide for the confused developer

  • 1.
    The hitchhiker’s guidefor the confused developer Making sense of an increasingly mad IT Uwe Friedrichsen – codecentric AG – 2012-2018
  • 2.
    Uwe Friedrichsen IT traveller. DotConnector. Cartographer of uncharted territory. Keeper of timeless wisdom. CTO and Fellow at codecentric. https://www.slideshare.net/ufried https://medium.com/@ufried @ufried
  • 3.
    Have you everfelt like (almost) everyone and everything in IT is going mad?
  • 4.
    Like everything isgetting more and more complex every week?
  • 5.
    And you alreadyfeel like you hardly can hold together all the bits and pieces?
  • 6.
    But you aregetting less and less time to pick up the plethora of new stuff?
  • 7.
    And all that“Agile” just feels like a big hoax which makes things worse than better?
  • 8.
    Do you sometimesfeel like you might miss what is really important?
  • 9.
    And instead wasteyour time in a crazy rat race to nowhere?
  • 10.
    Will AI replaceyour job in a few years?
  • 11.
    And what aboutIoT, Blockchain and all those other “disruptions”?
  • 12.
    Do you sometimesask yourself if you are still on the right track?
  • 13.
    Or that youmight focus on the wrong things?
  • 14.
    But what arethe “right” things?
  • 15.
    Do you sometimesfeel like this?
  • 16.
  • 17.
    Then it mightbe time for a little hitchhiker’s guide to the IT of today
  • 18.
    But first ofall ...
  • 20.
    Understanding IT A complexstructure observed from multiple viewpoints
  • 21.
  • 22.
    Formal part of valuecreation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamichigh dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. The “bathtub” curve Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
  • 23.
    Formal part of valuecreation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamichigh dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Pre-industrial era Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13 Tailor-made solutions Mastery is key to success
  • 24.
    Formal part of valuecreation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamichigh dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Industrial era Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13 Cost-efficiently scale production Getting more done with less people is key to success
  • 25.
    Formal part of valuecreation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamichigh dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Post-industrial era Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13 Continuously respond to changing demands Continuous market adaption is key to success
  • 26.
    Key drivers Pre-industrial era • No clear driver Industrial era •  Cost-efficiency •  Scalability •  Repeatability •  Stability •  Efficiency & scale Post-industrial era •  Cycle times •  Adaptability •  Flexibility •  Resilience •  Effectiveness & speed
  • 27.
  • 28.
    1960 1970 1980 1990 2000 2010 2020 Complicated (Business functions) Complex (Business processes) Highly complex (Business nervous system) Software crisis Software engineering PC LAN Internet Business Support of IT Selective Holistic Complicated Complex “Moore’s law” Mobile IoT
  • 29.
    IT has changeda lot over the decades ...
  • 30.
    1960 1970 1980 1990 2000 2010 2020 Complicated (Business functions) Complex (Business processes) Highly complex (Business nervous system) Software crisis PC LAN Internet Business Support of IT Selective Holistic Complicated Complex “Moore’s law” Mobile IoT Software engineering ... but still we strive to control our IT of today ... ... based on the concepts we developed for an IT almost 50 years ago
  • 31.
    Formal part of valuecreation Solution: machine Dynamic part of value creation Solution: man sluggishness/low dynamic high dynamichigh dynamic The historical course of market dynamics and the recent rise of highly dynamic and complex markets The dominance of high dynamics and complexity is neither good nor bad. It‘s a historical fact. t1970/80 today Age of crafts manu- facturing Age of tayloristic industry Age of global markets 1850/1900 Spacious markets, little competition Local markets, high customi- zation Outperformers exercise market pressure over conventional companies We call the graph shown here the “Taylor Bathtub”. Remember the “bathtub” curve? Source: BetaCodex Network Associates, “Organize for complexity”, BetaCodex Network White Paper 12 & 13
  • 32.
    1960 1970 1980 1990 2000 2010 2020 Complicated (Business functions) Complex (Business processes) Highly complex (Business nervous system) Software crisis Software engineering PC LAN Internet Business Support of IT Selective Holistic Complicated Complex “Moore’s law” Mobile IoT Also the business we support with IT today ... ... is very different from the business we supported back then
  • 33.
    What is therole of IT today?
  • 34.
    IT today is... •  ... the nervous system of the business •  ... an enabler of (disruptive) new business models •  ... an integral part of the business model (“digitization”) •  ... the medium for the continuous customer communication
  • 35.
  • 36.
    Disruptive technologies 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking UNIX Server Personal Computer Notebook
  • 37.
    Disruptive technologies Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server Personal Computer Standard Software OSS Notebook
  • 38.
    Disruptive technologies Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Mobile networks (3G+) Wi-Fi Modem 9600+ Baud
  • 39.
    Disruptive technologies Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Smartphone Mobile networks (3G+) Tablet Wi-Fi Modem 9600+ Baud
  • 40.
    Disruptive technologies Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Smartphone Cloud Mobile networks (3G+) Big Data Tablet Wi-Fi SaaS Serverless FaaS + Managed services Modem 9600+ Baud
  • 41.
    Disruptive technologies Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Smartphone Cloud Mobile networks (3G+) Big Data Tablet Wi-Fi SaaS Serverless FaaS + Managed services PLC Industrial PC OPC UA IoT Modem 9600+ Baud
  • 42.
    Disruptive technologies Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Smartphone Cloud Mobile networks (3G+) Big Data Tablet Wi-Fi SaaS Serverless FaaS + Managed services PLC Industrial PC OPC UA IoT Modem 9600+ Baud
  • 43.
  • 44.
    Programming models DIY Do Ityourself Web applications Enterprise Frameworks CORBA, SOA, JEE, .net, ... IoT Serverless FaaS + Managed services Cloud native Microservices ? Libraries & Frameworks Embedded computing Standard software customization HPC High Performance Computing Local computing paradigm Distributed computing paradigm
  • 45.
  • 46.
    System theory -Cynefin Source: By Snowded - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=33783436 The simple things •  Do them ad hoc •  Use check lists •  KISS! •  ... The domain of planning •  Waterfall, V-Model, ... •  Iterative & incremental •  SAFe •  ... The domain of uncertainty •  Inspect & adapt •  Agile (XP, Kanban, ...) •  Focus on outcome •  ... Never be here in an IT project! Never, ever!
  • 47.
  • 48.
  • 49.
  • 50.
  • 51.
    More complementing viewpoints • Evolution of “Make or buy” •  Double Helix •  Local computing vs. distributed computing
  • 52.
    Understanding current ITtrends Complementing the picture
  • 53.
  • 54.
    Digitization (Definition) •  Blurryand misleading term – yet an important change driver •  Response to market demands and technology evolution •  Core: IT becomes integral part of business offerings •  Consequence: Cross-domain boundaries start to dissolve and leverage – currently still unknown – business models
  • 55.
    Digitization (Effect) •  Uncertaintyregarding business model viability •  Effect: Short iterations, pervasive metrics (incl. measuring outcome), A/B testing, smart rollout strategies, ... •  Dissolving domain boundaries •  Effect: API, platforms, ... •  Adaption of persistent requirements to a new context •  Effect: Poor connectivity as a constraint, re-thinking accessibility, “developer experience” on the API level, ...
  • 56.
  • 57.
    Customer expectations •  Greatuser experience (UX) •  Fast response times •  Multi-device capabilities •  Zero downtime •  Strong Security (regarding data privacy) •  Adapts to customer’s needs and demands
  • 58.
  • 59.
    Moving fast •  Minimizecycle times to accelerate the feedback loop •  Response to business model uncertainty •  Goal is to minimize idle and value-reducing performances •  Massive paradigm shift (“Re-thinking IT”) That is what DevOps actually is about!
  • 60.
    Focus on frontends • The battle for customers gets decided in the frontend •  Backends just need to work and must not get in the way •  UX and UI design become key differentiators •  Backend becomes commodity •  Bear all types of frontends in mind (incl. IoT, API, ...)
  • 61.
    Cloud-native and serverless • Reduce vertical integration depth •  Focus on value-creating IT performance to move faster •  Newest incarnation of “make or buy” •  Managed services as successor of standard software •  FaaS to orchestrate managed services •  Managed container and batch jobs for deep business logic
  • 62.
    Ambient computing •  Shifttowards human-centered user interfaces •  Complemented by context-aware computing & UIs •  “Mobile first” is just a transitional step •  New types of human-machine interaction •  Voice, gestures, tactile, ... •  Augmented by AR & VR interface capabilities
  • 63.
    InfoSec •  Indispensable supplementfor most needed technologies •  Cloud •  Wireless •  IoT •  Mobile •  ... •  Must be an integral part of all stages of the IT value chain
  • 64.
  • 65.
    IT enabled trends Artificialintelligence / deep learning Internet of Things (including industry 4.0) Maker movement (including 3D printing) Blockchain •  Currently hyped topics with disruptive potential •  Actual future impact not yet clearly foreseeable
  • 66.
    Evaluating your position Findingout where you are and where you want to go
  • 67.
  • 68.
    Framework dimensions •  Workmodel evolution •  Technology evolution •  Programming model evolution
  • 69.
    Dimension 1: Workmodel evolution
  • 70.
    Remember this model? Pre-industrialera •  No clear driver Industrial era •  Cost-efficiency •  Scalability •  Repeatability •  Stability •  Efficiency & scale Post-industrial era •  Cycle times •  Adaptability •  Flexibility •  Resilience •  Effectiveness & speed
  • 71.
    This model canalso be applied to IT
  • 72.
    Properties Pre-industrial Industrial Post-industrial Market Emerging market Wide & slow market Narrow & fast market Strategic goal None (“Just do it!”) Cost-efficiency Short cycle times System theory Simple Complicated Complex Work model Ad hoc Processes Collaboration Organization model Peer to peer Hierarchies Autonomous teams Practices “Whatever goes” ITIL, V-Modell, SAFe DevOps, Kanban, XP, Agile Individuals Experts (Cube-shaped) Specialists (I-shaped) Generalists (T-Shaped) Automation Low Medium-high Very high Manual work Ad hoc Defined and controlled Avoided Programming model Do it yourself Standard Software, Enterprise Frameworks Cloud-native, Serverless, Managed Services
  • 73.
  • 74.
    Taking this “magictriangle” ...
  • 75.
    Good Fast Cheap Optimizing forquality and cycle times will result in higher costs Optimizing for quality and costs will result in long cycle times Optimizing for cycle times and costs will result in reduced quality
  • 76.
    ... will leadyou to this model
  • 77.
    You may pick two Good Fast Cheap Industrial IT Deliver large batches at minimized costs towards slow markets Post-industrial IT Quickly adapt to ever-changing needs of dynamic, fast-moving markets Startup IT Test hypotheses and pivot as fast as possible to discover a product-market fit
  • 78.
    You may pick two Good Fast Cheap Industrial IT Deliver large batches at minimized costs towards slow markets Post-industrial IT Quickly adapt to ever-changing needs of dynamic, fast-moving markets Startup IT Test hypotheses and pivot as fast as possible to discover a product-market fit This is not a pre-industrial IT model, as it comes with a clear strategic goal (“Discover product-market fit as fast as possible”), still it usually shares many of the other properties
  • 79.
    You may pick two Good Fast Cheap Industrial IT Deliver large batches at minimized costs towards slow markets Post-industrial IT Quickly adapt to ever-changing needs of dynamic, fast-moving markets Startup IT Test hypotheses and pivot as fast as possible to discover a product-market fit 1 2 3 Product evolution cycle
  • 80.
    Work model evolution • You can reason about the work model at different levels •  Company (Market) •  IT department •  Product/Project •  The work model for a given product will change over time •  You will need to figure out your preferred work model
  • 81.
  • 82.
    Let us startwith the disruptive technologies ... Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Smartphone Cloud Mobile networks (3G+) Big Data Tablet Wi-Fi SaaS Serverless FaaS + Managed services PLC Industrial PC OPC UA IoT Modem 9600+ Baud
  • 83.
    ... aggregate themin groups ... Assembly language 1950 1960 1970 1980 1990 2000 2010 2020 Mainframe Transistor Multitasking High-level language Algol Lisp Simula Cobol UNIX Server LAN Personal Computer Standard Software OSS WWW Notebook Smartphone Cloud Mobile networks (3G+) Big Data Tablet Wi-Fi SaaS Serverless FaaS + Managed services PLC Industrial PC OPC UA IoT Modem 9600+ Baud Traditional software development Industrial computing / IoT Web centered Cloud native Mobile centered Mobile centered (Big & Fast) Data centered
  • 84.
    ... add thetrends ... •  Ambient computing •  Focus on frontends •  InfoSec •  Artificial intelligence / deep learning •  Maker movement (including 3D printing) •  Blockchain
  • 85.
    ... and distillthe major directions
  • 86.
    Frontends as differentiators • Mobile centered à Ambient computing Backends as commodity •  Driven by cloud native (especially managed services) Smart data •  Big/fast data & machine learning à deep learning à AI Smart devices •  Industrial computing & IoT ß Maker movement InfoSec •  Needed everywhere ß Blockchain Traditional •  including web & mobile centered
  • 87.
    Technology evolution •  Alldirections are interconnected •  Still, you will most likely need to focus on one direction •  As mastering all of them is too much for one person •  As architect, you will need to get a grasp of all directions •  Required to help your stakeholders making the best decisions •  Still, you should deep dive at least into one of the directions
  • 88.
    Dimension 3: Programmingmodel evolution
  • 89.
    Let us takethe programming models ... DIY Do It yourself Web applications Enterprise Frameworks CORBA, SOA, JEE, .net, ... IoT Serverless FaaS + Managed services Cloud native Microservices ? Libraries & Frameworks Embedded computing Standard software customization HPC High Performance Computing Local computing paradigm Distributed computing paradigm
  • 90.
    ... and aggregatethem in groups DIY Do It yourself Web applications Enterprise Frameworks CORBA, SOA, JEE, .net, ... IoT Serverless FaaS + Managed services Cloud native Microservices ? Libraries & Frameworks Embedded computing Standard software customization HPC High Performance Computing Local computing paradigm Distributed computing paradigm No paradigm (random outcomes) Distributed computing paradigm Local computing paradigm Embedded computing paradigm
  • 91.
    ... and aggregatethem in groups DIY Do It yourself Web applications Enterprise Frameworks CORBA, SOA, JEE, .net, ... IoT Serverless FaaS + Managed services Cloud native Microservices ? Libraries & Frameworks Embedded computing Standard software customization HPC High Performance Computing Local computing paradigm Distributed computing paradigm No paradigm (random outcomes) Distributed computing paradigm Local computing paradigm Embedded computing paradigm Transitioning to one of those paradigms basically means mostly re-learning how to develop good software Most people just do that in a disguise if they claim they do “microservices” Smart device management including edge computing usually requires both paradigms
  • 92.
  • 93.
    Pre-industrial Programming model Technology Work model Industrial Startup Post-industrial Traditional Backend Frontend Device Data / AI InfoSec Ambient AR/VR Cloud native Serverless Industrial IoT/Maker Big/fast data ML/DL Everywhere (Blockchain) Random Local computing Distributed computing Embedded computing Known territory New territory C/S, Web Mobile
  • 94.
    Step #2 Locate yourcurrent position
  • 95.
    Pre-industrial Programming model Technology Work model Industrial Startup Post-industrial Traditional Backend Frontend Device Data / AI InfoSec Ambient AR/VR Cloud native Serverless Industrial IoT/Maker Big/fast data ML/DL Everywhere (Blockchain) Random Local computing Distributed computing Embedded computing Known territory New territory C/S, Web Mobile
  • 96.
    Step #3 Find yourdesired position
  • 97.
    Pre-industrial Programming model Technology Work model Industrial Startup Post-industrial Traditional Backend Frontend Device Data / AI InfoSec Ambient AR/VR Cloud native Serverless Industrial IoT/Maker Big/fast data ML/DL Everywhere (Blockchain) Random Local computing Distributed computing Embedded computing Known territory New territory C/S, Web Mobile
  • 98.
    Step #4 Define yourpersonal evolution plan
  • 99.
    Define your personalevolution plan •  Defining the plan is the easy part •  Read the literature, do the trainings, train your skills, ... •  Still, it will require time and discipline •  It’s a combination of personal and company game •  You cannot expect your company to cover all the efforts •  You will usually need to invest some of your private time •  And if your environment does not support you •  “Love it, change it, or leave it” •  Trade-offs are fine, but in the end they are your decision!
  • 100.
    Staying ahead ofthe curve Some general recommendations augmenting your path ahead
  • 101.
    Understand post-industrial markets • Speed trumps perfection – Rethink everything! •  Really, really understand the implications of uncertainty! •  Understand the difference between output and outcome •  Understand the ideas of bets and options •  Dismiss cargo-cult agility – learn what “Agile” really means •  It is a revolution in your head! •  But it should be an evolution regarding implementation •  Otherwise you will certainly overstrain your organization
  • 102.
    Cross-functional teams (organized bybusiness capabilities) Autonomy (incl. E2E responsibility) Decentralized control Microservices Continuous Delivery Heterogeneity Cloud and Containers Resilience Operations automation Craftsmanship & mastery Outcome-driven Beyond budgeting Feature flow Lean EAM Continuous improvement T-Shaped people (being empathic) DevOps Quick feedback loops Curiosity
  • 103.
    Master the timelesswisdom •  Especially master the foundations of good design •  “Loose coupling, high cohesion” & “separation of concerns” •  “Information hiding” especially with respect to API •  Domain-driven design can be a useful starting point •  Good design skills are more relevant than ever •  Affect understandability, changeability and extensibility •  Affect usability, stability and acceptance at API level •  Affect robustness, availability and scalability at runtime
  • 104.
    “I would advisestudents to pay more attention to the fundamental ideas rather than the latest technology. The technology will be out-of-date before they graduate. Fundamental ideas never get out of date.” -- David L. Parnas (http://www.sigsoft.org/SEN/parnas.html)
  • 105.
    Become T-shaped •  Generaliststrump specialists •  Pure subject matter experts are an industrial concept •  Collaboration requires understanding your collaborators •  Leave your comfort zone once in a while •  Still, have your area of expertise •  Deep knowledge is still needed – but in a inclusive way •  Understand the business domain •  This is what turns you into a “ten times developer”
  • 106.
    Learn frontend development... •  Learn JavaScript – really! •  Embrace the concepts of UX and UI design •  Understand CSS at least a bit •  Start picking up the concepts of ambient computing •  Voice-based UI, Video-based UI, AR, VR, etc.
  • 107.
    ... or embracecloud native ... •  Understand and learn what “cloud” means today! •  Understand the concepts of “serverless” •  Know the relevant managed service offerings (BaaS/SaaS) •  Automate everything using managed services •  Understand operation needs (especially monitoring) ... •  ... and what existing solutions (managed services) offer •  Abandon DIY and NIH •  Actively reduce vertical integration depth
  • 109.
    ... or trya different direction •  Smart Data •  Will heavily shape future decision making processes •  Be prepared to refresh your math •  Smart Devices •  Fast and wild evolution at the moment •  Lots of exciting challenges – if you like it •  InfoSec •  Needed everywhere – yet treated often like an unloved chore •  Must become an integral part of development and operations
  • 110.
    Embrace distributed systems “(Almost)every system is a distributed system” -- Chas Emerick “Everything fails, all the time” -- Werner Vogels •  Understand the non-determinism of distributed systems •  “Memory, guesses and apologies” & promise theory •  Understand that infrastructure cannot guarantee robustness
  • 111.
  • 112.
    Wrap-up •  Moving froman industrial to a post-industrial domain •  Technology and trends revealing new directions •  Programming goes distributed and embedded •  Generalists trump specialists •  Timeless wisdom as a game changer •  Assess where you are and where you want to be •  Love it, change it, or leave it
  • 114.
    Uwe Friedrichsen IT traveller. Connectingthe dots. Attracted by uncharted territory. CTO at codecentric. https://www.slideshare.net/ufried https://medium.com/@ufried @ufried