The document discusses the effects of real-time data on data teams and what management needs to do to support them. It notes that moving to real-time data can greatly increase complexity and that teams may need preparation first. It also states that teams require dedicated operations and data engineering functions with the right skills to be successful with real-time data, and that management must communicate the value of these teams.

1. Managing Real-time
Data Teams
1 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

2. Real-time's Effect on Data Teams
Operations Issues
Data Engineering Issues
What to Do?
Managing Real-time Data Teams
2 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

3. Data teams are affected by the
move to real-time
Real-time is more
than just a
technology
choice
Complexity levels can increase
dramatically
3 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

4. Is the team ready?
Teams may need
extensive
preparation
before starting a
real-time project
Teams that aren't ready can
stagnate
4 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

5. Real-time's Effect on Data Teams
Operations Issues
Data Engineering Issues
What to Do?
Managing Real-time Data Teams
5 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

6. Missing Operations Team
A missing operations team means no real-time operational excellence
Data Science
Data Engineering OperationsOperations
6 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

7. Real-time SLAs
Operations teams that can
barely maintain batch SLAs
(hours/days) won't be able
to handle real-time SLAs
(minutes/hours).
7 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

8. Business SLAs
Ineffective operations
teams that don't keep SLAs
mean the business can't rely
on it for mission-critical use
cases.
8 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

9. Real-time's Effect on Data Teams
Operations Issues
Data Engineering Issues
What to Do?
Managing Real-time Data Teams
9 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

10. Missing Data Engineering Team
A missing data engineering team means architecture guesses and wrong
tools for the job
Data Science
Data EngineeringData Engineering Operations
10 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

11. Batch/Real-time
If the data engineering team
can barely handle batch,
real-time will be impossible.
11 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

12. Data Products
Exposing real-time data
products isn't like an API or
REST call. Data products
must have a schema and
remove silos.
12 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

13. Connections
Data engineering must have
high bandwidth connections
to the rest of the
organization to ensure
adoption, especially with
the data science team.
13 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

14. Right Ratios
Management needs to
evaluate there are the right
ratios of data scientist to
data engineer. Incorrect
ratios lead to technical
debt.
14 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

15. Real-time's Effect on Data Teams
Operations Issues
Data Engineering Issues
What to Do?
Managing Real-time Data Teams
15 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

16. We're not a team looking for
problems to solve. We're a
team that solves business
problems (in real-time).
16 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

17. Real-time data teams will forever
be compared to the productivity
of other technical fields
Management
needs to
communicate the
value provided
Will take education to show the
difference in complexity
17 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

18. Management is responsible for
getting back on track
Management is
responsible for
the early success
and failures of
real-time data
teams
18 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

19. New Book and Survey
https://www.datateams.io
https://tiny.bdi.io/datateamssurvey
19 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9

20. Thank You
bigdatainstitute.io
20 / 20Copyright © 2020 Smoking Hand LLC. All rights Reserved. Version: 82982da9