The document discusses how Rise Science, a young company focused on sleep improvement and user enjoyment, embraced a monolithic architecture to allow their small teams to iterate quickly without breaking things. It outlines some of the obstacles they faced with separate data silos and tooling, and how centralizing their data in a single database and using Django as their framework helped break down those silos and enabled self-serve analytics. This approach created a positive feedback loop where non-technical users could access data whenever needed, freeing up the data team to focus on more strategic problems.

1. Doubling down on python to move fast without
breaking things.
Embracing the Monolith in Small Teams
Leon Sasson
@leonsasson
PyData Chicago 2016

2. Rise Science

3. Rise Science

4. Product Goals
• Sleep Improvement
• User Enjoyment

5. Iterate Fast
Young company, timeline of weeks and days.

6. Data is core to the product
No data = 😩

7. Development Cycle
Hypothesis
Exploration
Experiment
Productizing
Evaluate & Analyze

8. Easy, right?

9. 😓

10. Obstacles

11. • Data Silos

12. • Data Silos
• Different Tooling

13. • Data Silos
• Different Tooling
• People

14. • Data Silos
• Moving from phase requires different tools
• People
• "It works on staging"
• Testing data products is hard
• Garbage in → Garbage out
• Capacity problems

15. • Data Silos
• Moving from phase requires different tools
• People
• "It works on staging"
Extended Product Cycles

16. How do we start?
Descriptive, visuals, basic summaries

17. Step Back

18. What the organization needs.
Understand problem before getting into solutions

19. Solution First
Focus is on tech trade-off

20. Solution First
Focus is on tech trade-off
Problem First
Focus is on making progress for the org
vs.

21. Solution First
Focus is on tech trade-off
Problem First
Focus is on making progress for the org
vs.

22. Solution First
Focus is on tech trade-off
Problem First
Focus is on making progress for the org
vs.

23. Business
Optimality
Technical
Optimality

25. What's the least I can do to solve the problem?

26. What's the least I can do to solve the problem?
You need an architecture compatible with this mindset

27. Monolithic Architecture

28. © Martin Fowler: http://martinfowler.com/articles/microservices.html
A monolithic application
puts all its functionality
into a singles process..
... and scales by
replicating the monolith
on multiple servers

29. © Martin Fowler: http://martinfowler.com/articles/microservices.html
A microservices
architecture puts each
element of functionality
into a separate service..
... and scales by distributing
these services across
servers, replicating as
needed

30. © Martin Fowler: http://martinfowler.com/articles/microservices.html© Martin Fowler: http://martinfowler.com/articles/microservices.html

31. Django. The Good Things

32. Reuse Libraries

33. IPython Notebooks

34. Reuse your ORM when accessing data.

35. Pandas, django-pandas

36. Instrumentation

37. People

38. The Problem of Toil
".. manual, repetitive, automatable, tactical, devoid of enduring value,
and scales linearly as a service grows.."

39. Toil-induced negative data culture

40. Self-Serve Analytics

41. Breaking Data Silos

42. Why do Data Silos Happen?
person id date duration
1 2016-08-01 450
2 2016-08-01 426
1 2016-08-02 438
Row
person id date duration
1
2
1
2016-08-01
2016-08-01
2016-08-02
450
426
438
Columnar

43. Centralizing Data
Segment.com
Backend DB
RedshiftETL

44. Redshift is fast for aggregations

45. Out-of-the-box compatible with Postgres
(Mostly..)

46. Bring data to the people

49. Positive Feedback Loop on Data Culture
Non tech can access data whenever
Data team can focus on bigger problems and act as enablers

50. Be scrappy.

51. Thanks!