Green shoots in the brownest field
Mat Wall (@matwall)
Government Digital Service
https://www.gov.uk
Being a startup in government
Friday, 8 March 13
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/UK-gov-gds-tools-techniques
Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
GDS
Government development skills largely outsourced in 80s
Government IT vendor driven, procurement focussed
People who can write code (not COBOL) very rare
Friday, 8 March 13
GDS set up about 18 months ago
Bring digital capability & vision into government
Stimulate economy of smaller vendors
A startup within government
Friday, 8 March 13
Initially build alpha of GOV.UK
Now fully released into production
Migrating government department sites
Beginning to work on transactions
Friday, 8 March 13
Design Principles
1. Start with user needs
2. Do less
3. Design with data
4. Do the hard work to make it simple
5. Iterate wildly
6. Build for inclusion
7. Understand context
8. Build services, not websites
9. Be consistent, not uniform
10. Make things open: it makes things better
https://www.gov.uk/designprinciples
Friday, 8 March 13
reboot the government
Friday, 8 March 13
GOV.UK
Friday, 8 March 13
Quick answers
Friday, 8 March 13
Quick answers
Friday, 8 March 13
Quick answers
Friday, 8 March 13
Detailed guides
Friday, 8 March 13
Detailed guides
Friday, 8 March 13
Transactions
Friday, 8 March 13
Transactions
Friday, 8 March 13
Department sites
Friday, 8 March 13
Department sites
Friday, 8 March 13
Light weight architecture & development
Focussed on user needs
Good example of “How to do it” across
government
Friday, 8 March 13
Strategic architecture
Friday, 8 March 13
Very common in government IT
Small number of in house architects
Often not hands on
Almost all development outsourced
Friday, 8 March 13
Lay down ground rules
Select technologies up front
Often vendor focussed / led
Friday, 8 March 13
Attempt to:
reduce risk
minimise cost
produce predictable results
Friday, 8 March 13
Insert strategic architecture here
Friday, 8 March 13
Friday, 8 March 13
ESB
Friday, 8 March 13
ESB
ESB
Friday, 8 March 13
ESB
ESB
Friday, 8 March 13
ESB
ESB
OPA
Friday, 8 March 13
ESB
ESB
OPAEDM
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Now, what shall we make?
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Imagine being a developer here
Most decisions made for you
Little room for innovation
Friday, 8 March 13
reduce risk = no innovation, create lock in
minimise cost = ha ha
predictable results = slow delivery
Friday, 8 March 13
Platforms
Friday, 8 March 13
Platforms first = bottom up architecture
We want to focus on user needs first
Friday, 8 March 13
Most of our apps start out as a rough idea
on someones laptop, not as a platform
Friday, 8 March 13
Platforms emerge by constant delivery
Friday, 8 March 13
Space
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
Web routing
Provisioning
Configuration management
Services / libraries
Persistence
Monitoring
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
Ruby / rails / sinatra/ jekyll
Scala / play 2.0
Java / dropwizard
MongoDB, MySql, ElasticSearch, Redis
nginx, varnish, Akami
puppet
statsd / graphite / nagios / ganglia
funkload
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
Space
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend
inside gov
smart
answers
licensingsearch
Content API
Publisher
Panopticon MongoDB MySQL
CMS tools
Licensing
admin
Sign onneed-o-tron
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Content model split across apps
Complex interaction between apps
Brittle, what if interaction fails?
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13
Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API /
Panopticon
Publisher
MongoDB
need-o-tron
routing
Event broadcast
Friday, 8 March 13
Space
Help developers create a safe space
Create small, simple apps in that space
Provide lightweight components & processes, and ability to create more
Don’t worry about trying to solve all of the problems up front
Allow for innovation & learning
Friday, 8 March 13
Momentum
Friday, 8 March 13
Momentum
We far prefer momentum to strategy
Make decisions as late as possible
Short iterations mean you can’t go to far wrong
Easy to put things back on track
Allows for innovation, experiment & learning
Friday, 8 March 13
New system: apply to register to vote (ERTP)
Initially seemed quite simple
Surely just a form and a database?
Real world is more complex
Friday, 8 March 13
Local authority 1 Local authority 2 Local authority 387....
Citizens
DWP
Friday, 8 March 13
Citizen API
Local auth API
MatchingAPI
Local authority 1 Local authority 2 Local authority 387....
DWP
Web space
LA space
DWP space
Friday, 8 March 13
Citizen API
Local auth API
MatchingAPI
Local authority 1 Local authority 2 Local authority 387....
DWPMongoDB
Frontend web
Feeds Monitoring
Friday, 8 March 13
Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Friday, 8 March 13
Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Friday, 8 March 13
Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Friday, 8 March 13
Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Anything goes to keep momentum
Friday, 8 March 13
Legacy / complex systems
Friday, 8 March 13
So far most of our systems have been very simple
Easy to be small, beautiful & agile
But that will change with transactions
Friday, 8 March 13
Government IT estate complex, old
Pre-web technologies
Oldest system I have found so far is 40 years!
How can we work in this environment?
How can we retain our momentum?
Friday, 8 March 13
ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Friday, 8 March 13
Integration strategies:
Could rebuild old system.
Could export data from old system into new.
Use what we have learned to create our own space, with our own momentum
Friday, 8 March 13
Shiny application
Domain API Domain API
Create prototype application backed of simple REST domain APIS
Each API encapsulates a single domain concept
(ie: User profile, transaction history, application service)
Domain APIs filled with mock data
HTTP integration test suite produced
Domain APIDomain API
Friday, 8 March 13
Shiny application
Domain API Domain API
Integration tests form spec that can be shared between old & new world
Api built by new team, using old APIs / integrations
APIs can be built in existing system, by existing team
Or as a joint effort, using a facade
Domain APIDomain API
Friday, 8 March 13
Shiny application
Domain API
New space Old space
Friday, 8 March 13
Shiny application Domain API
New space Old space
http, JSON
GET /user/341353/transactions
select * from transactions where
userId=431353
Friday, 8 March 13
Shiny application
Domain API
New space Old space
database
Data feed
Friday, 8 March 13
Give developers space
then get out of the way
Momentum is king
Friday, 8 March 13
http://github.com/alphagov
http://digital.cabinetoffice.gov.uk/
https://www.gov.uk
Friday, 8 March 13

Green Shoots in the Brownest Field: Being a Startup in Government