Introduction to data exploitation. Why data is now central to operations and how data can be exploited in organisations. Covering the growth of data, algorithms, and business models.

1. Big data primer
Paul Edmunds

2. What’s in this
for you?
This primer is a short introduction to data which explains:
1. why there is a trend in big data,
2. how data is exploited,
3. what organisational attributes are needed,
to capitalise on data opportunities.
The explosion of data creates increased workloads and
opportunities to work at scale. For example ANZ Bank
realised a 20% increase in productivity in 2018 due to data
automation.
Leaders need to understand these trends so they can seize
the opportunities presented by data and deal with
increased workloads.
An MIT survey of CEOs finds this is the most important
quality for leaders today:
“The single most important skill for...leaders….is the ability
to recognise the potential of digital and data”

3. PART 1
Why is there a
trend towards
data?
Data is becoming central to leading organisations:
Historical trends show data centric companies are
becoming dominant.
Exponential growth in data makes manual approaches
obsolete for organisations that deal in data.
Data rich organisations now need algorithms to operate.

4. Historical trends show the growing
dominance of data centric companies.
We are now in the “age of data”.
Personal computer apps are being
replaced by data apps, in the same way
clerks with calculators were replaced in
the 1970’s.
Historical trends
towards an “age of
data”

5. Data has grown so much it is now measured
in Yotta bytes. Yotta is the biggest unit in the
metric system.
As a comparison, the observable universe is
830 Yotta metres across. Printed in a line, we
will soon have enough data to stretch the
length of the observable universe. We
cannot deal with this scale manually.
Manual approaches are becoming obsolete
because there is now so much data
organisations cannot handle it.
Huge amounts of
data are being created

6. Algorithms are the
only way to handle big
data
Algorithms are tools to handle large
amounts of data. They automate
previously manual cognitive tasks.
Data processing becomes central
because with the growth of data, the
intelligence problem shifts from targeted
collection, to “finding sense in noise”.*
(* source Congress report on National
Security 2017).

7. PART 2
How is big data
exploited by
organisations?
Organisations process big data with algorithms:
Data problems are addressed with “Computational
Thinking”.
Algorithms automate component parts of human cognitive
work in 4 simple ways.
Machine Learning and Artificial Intelligence (AI) combine
these algorithms to automate all steps in computational
thinking.

8. Most data problems can be broken down for
solving. A four step process called
“Computational thinking” is used:
• Decomposition of the problem.
• Organise data and recognise patterns.
• Pattern generalisation and abstraction.
• Algorithms to output analysis.*
(* Source: BBC bitesize GCSE revision guide)
“Computational
thinking”

9. 4 things algorithms do *:
• Prioritisation: making an ordered list
• Classification: assigning a category
• Association: finding links
• Filtering: pattern recognition.
Algorithms can follow rules or learn by
example (machine learning)
How algorithms work
Algorithms are used in data science
to automate human cognitive work.
Four basic cognitive tasks can be
automated.
*source Professor Hannah Fry “Hello World”

10. 1. Filtering: Isolate the features which indicate likeliness of
relevance, by recognising patterns
2. Classification: Classify groups of actors by indicators, through
combining these features
3. Prioritisation: Prioritise actors by giving them a score,
outputting analysis for decision
Machine learning
automates
“computational thinking”
Machine learning enables machines to
do computational thinking. AI identifies
patterns and abstract these details for
analysis and decisions.
Simple algorithm tasks combine to step
through the computational thinking
process and solve a real world problem.
Top Features for prediction
1
2
3
4
Ag
e
Max number of
interactions
Sum of
transactions
Earliest
engagement
Precision
% accuracy of identifying those who will transact 53%
Recall
% identifed out of the total group 3.2%
Most likely to transact with incentives
% with a more than 50% likelihood of taking up incentive 8.6%

11. PART 3
What
organisational
changes are
needed for big
data workloads?
Effective processing of big data workloads requires:
New skills for the workforce.
New organisational approaches.
New focus for governance.

12. Data Science requires operational knowledge,
plus statistics and programming skills. Teams
that use data need all of these:
Working with data needs
multi-skilled teams
The Venn diagram* shows the 3 skills needed
for data science
(* care of Data Science Association, diagram by Drew Conway)

13. New organisational
approaches
McKinsey smartphone analogy for
new organisations
To lead on data, organisations need to
break down silos with digital
architectures so multi-skilled teams can
build automated data processes at
pace.
Like smartphones, organisations
provide flexible service architectures so
that multi-functional teams are able to
make specific ‘apps’ for different
purposes.

14. New focus for governance
Processing data at scale by machine requires new governance to
address potential issues:
• Algorithms may have a negative impact on data subjects, and may
violate their rights or privacy.
• Algorithms can be biased based on data or perspective. For example,
risk scoring means that certain groups are disadvantaged.
• Auditing is difficult to explain how outcomes are reached, a learning
machine cannot be a black box.
• Human oversight is often insufficient, machines cannot divest
responsibility
• Fairness in addition to necessity and proportionality is emerging as a
key concern
• The EU has produced 7 rules for machines to follow.

15. Summary
We have learned:
We are now in the “age of data”, where data centric organisations dominate.
Exponential growth in data makes manual approaches obsolete for organisations
that deal in data.
Data rich organisations now need algorithms to operate.
Data problems are addressed with “Computational Thinking”.
Algorithms automate component parts of human cognitive work in a small
number of ways.
Machine Learning and Artificial Intelligence (AI) combine these algorithms to
automate all steps in computational thinking.
Data organisations demonstrate:
• New skills for the workforce.
• New organisational approaches.
• New focus for governance.