As user and business needs evolve, so too should our systems and services. But in practice we tend not to design or build them that way so change is often expensive and painful. In this presentation, I explore some of the reasons for this and make recommendations for creating a maintenance-by-design culture that engenders more resilient and adaptable services, with data playing a key role. I gave this talk at the October 2019 Government Design Meetup titled: Repair, Renovation and Maintenance.

1. 1
​Renovating and maintaining
digital services and data
​Lessons from traditional infrastructure
​Ade Adewunmi
​@adewunmi
​October 24th
2018

2. 2
Hi

3. 3
Great power, great responsibility

4. 4
I’ve been thinking a lot about technological
utopianism, agile delivery processes and the
focus on the ‘happy path’ and how these things
shape the way we think, work and the things we
build.

5. 5
H0
: Technologists are no more optimistic than
the wider population.
H1
: Technologists are more optimistic than the
rest of the population.
I’ve also been thinking about whether there’s a
type of person who’s drawn to work in data and
digital.

6. 6
I haven’t quite worked out a suitably rigorous way
of testing the null hypothesis (so I don’t have the
evidence to reject it).

7. 7
I haven’t quite worked out a suitably rigorous way
of testing the null hypothesis (so I don’t have the
evidence to reject it).
But the alternative hypothesis is one of my working
assumptions (based on my personal experience).

8. 8
Some other working hypotheses/assumptions:

9. 9
Some other working hypotheses and assumptions:
A technocratic focus often creates blind spots.
Technologists are often blind to the economic,
social and power dynamics at play, in human
interactions.

10. 10
OK, so there’s a lot to trip us up. So what?

11. 11
OK, so there’s a lot to trip us up. So what?
So, exploring structural obstacles to good
renovation and maintenance culture by exploring
failures in infrastructure building is more useful
than studying successes.

12. 12
A purely technocratic approach to building data
infrastructure that ignores the drivers shaping
the environments in which our infrastructure is
deployed and built, results in brittle
infrastructure.

13. 13
This isn’t the
cheeriest start but
stick with me, I’m
going for balanced
and hopeful.
(honestly)

14. 14
We share many traits with other technocrats
including builders and designers of traditional
infrastructure.
We’re likely to make similar mistakes which means
we can also learn from these builders and
designers.

15. 15
Lessons from traditional
infrastructure
©2018 Teradata

16. 16
Factors that shape our attitude to renovating
and maintaining infrastructure:
• economic

17. 17
Factors that shape our attitude to renovating and
maintaining infrastructure:
• economic
• social

18. 18
Factors that shape our attitude to renovating and
maintaining infrastructure
• economic
• social
• power dynamics

19. 19
In the slides that follow, I reference the NYC metro
system (MTA) as a way of illustrating some points I want
to make.
Disclaimer: I’m not an expert on the NYC MTA and the
CityLab article was written in April of this year so things
might have changed since then.
CityLab article:

20. 20
December 16, 1940: date
the last new subway line
was opened, aside from a
handful of small extensions
and connections.
Source: CityLab article.

21. 21
December 16, 1940.
3 contributing factors:
1. Lure of the suburbs

22. 22
December 16, 1940.
3 contributing factors:
1. Lure of the suburbs
2. Delivery partner
challenges

23. 23
December 16, 1940.
3 contributing factors:
1. Lure of the suburbs
2. Delivery partner
challenges
3. Growing cost of
technical debt

24. 24
The lure of the suburbs, provides a brilliant lens for
viewing (and understanding) the defunding of the
‘infrastructure commons’.
Geographic distance means people have less in
common - and that has economic and social
implications.

25. 25
Funding choices shapes the type of infrastructure we get.
For example, the City of Chicago’s decision to sell off the
management and revenue streams from its parking
metres led to a big spike in metre fees and a decline in
quality of service.
Closer to home, HMT’s funding process affects the shape
of the UK government’s digital infrastructure too.

26. 26
Code reuse in government is still not widely
practiced and data reuse is even further behind.
“Reusable code has to be specifically designed
for a generalised purpose and it is unlikely to
appear spontaneously as a natural by-product of
development projects.”Ben Morris blog.

27. 27
Culture shapes infrastructure; infrastructure shapes
culture: does your infrastructure support reuse?
Data scientists working in
government departments,
making models
Professionals within a data
federation/ecosystem using
trained model
Exploration
•Data Wrangling
•DS Lab
•Model scripting (untrained
models)
•Testing, Training, Model
Evaluation
•Version Control
•Dependency Management
Automate
• Software unit tests
• Model Training
• Storage of trained models
• Model Evaluation
• Model Business
Approval/Report Creation
• Comparison vs current
Live model
(Champion/Challenger)
Consume
•Real-time model scoring
engines
•Automatic deployment of
trained model artefacts
•Dashboards and forecasts
updated using new models
•Model performance
monitoring
•Model output logging
Involving: Analysts, Data Scientists, Engineers, Dev Ops, Business Stakeholders
© 2018 Teradata

28. 28
Culture shapes infrastructure; infrastructure shapes
culture
Data Scientist
develops model
❑ Models developed by data scientist using preferred, locally installed analytic
tools
Data Scientist
prepares
power point
❑ A data scientist prepares a Powerpoint presentation on the benefits of a new
model
❑ Manually creating standard model performance metrics of model
Model approval
committee
❑ Model business approval committee meeting to:
See presentation; Discuss benefits/risks; Approve/Reject model
Model manually
transferred to
relevant production
environment
❑ Once approved, model exported to an Excel file of weights
❑ Excel manually transcribed into format suitable for the Production environment
running on organisation-specific systems
Model
goes live
❑ New model goes live in production under close supervision
© 2018 Teradata

29. 29
Culture shapes infrastructure; infrastructure shapes
culture
Data Scientist
develops model
❑ Models developed in standardised Python or R
Automated
tests
❑ Model is tested using a validation data set
❑ Report with model performance metrics is automatically generated
Peer
approval
❑ Model performance report is available in a UI for approval
❑ Approving in the UI will trigger automatically the bundling of models into
business use case
Business
approval
❑ The business impact of the new bundle of models is estimated and displayed
in a UI for the business team to approve the bundle
Model
goes live
❑ New model gets deployed automatically with A/B testing capability to validate
business impact
❑ The performance of the models in production are continuously monitored
© 2018 Teradata

30. 30
Traditional productivity
tools assume highly
individualised, localised
ways of working.
Culture shapes infrastructure; infrastructure shapes
culture: do your tools support re-use?

31. 31
How we balance investment in paying down
technical debt against building new infrastructure
is not a neutral decision.
It directly contributes to whether or not we develop a
healthy maintenance culture.
How much extra work are we willing to do, to reuse
and create reusable data?

32. 32
Room for improvement

33. 33
• Build stronger links between design and policy, using
data
Culture shapes infrastructure, infrastructure
shapes culture

34. 34
Policy is, still, too often
made in an evidence
free vacuum.
Good data is one of
the ways we help
shine a light on policy
gaps.

35. 35
• Build stronger links between design and policy, using
data
• Apply serious effort to incorporating responsible design
of the data lifecycle - data discovery, data collection,
metadata generation - into design sprints
Culture shapes infrastructure, infrastructure
shapes culture

36. 36
We have to be aware of the
challenges and risks of
indiscriminate data gathering.
And acknowledge the power
dynamics at play and inherent
biases in data collection.
Thinking beyond the ‘happy
path’

37. 37

38. 38
• Build stronger links between design and policy, using
data
• Apply serious effort to incorporating responsible design
of the data lifecycle - data discovery, data collection,
metadata generation - into design sprints
• Standardise and reward data reuse (both supply and
demand)
Culture shapes infrastructure, infrastructure
shapes culture (part I)

39. 39
• Think more critically about whether our tools and
working environment (e.g. Google Analytics)
broaden our view of the data ecosystem or narrows it
• Raise the threshold of ‘good enough’ data quality
(defining it would be a good start!)
• Build a strong data commons by building a wide
community of users
Culture shapes infrastructure, infrastructure
shapes culture (part II)

40. 40
• Let’s think about the stories we tell - what are we
emphasising and celebrating?
Culture shapes infrastructure, infrastructure
shapes culture (part III)

41. 41
Reasons for hope
©2018 Teradata

42. 42
• We’re waking up to the brittleness of our infrastructure.
• We’ve been here before. There’s infrastructure (social
and software) from the digital transformation era we can
leverage.
• Efforts to build community and maintain a digital
commons persist.
Why I remain hopeful about the future of
government infrastructure

43. 43
Thank you.
©2018 Teradata
Thank you.
©2018 Teradata