1. BIG DATA

3. "datasets that grow so
large that they become
difficult to work with
using relational
databases and within a
tolerable elapsed time"

4. BIG DATA IS BIG

5. LIKE, REALLY BIG

6. FACEBOOK: 140 BILLION
PHOTOS
HUMAN GENOME: 3 BILLION
BASE PAIRS
GOOGLE: 50 BILLION
WEB PAGES
WORLDCAT: 1.5 BILLION
ITEM RECORDS

7. NOT REALLY

8. EUROPEANA: 20 MILLION
(715K / COUNTRY)
LIBRARY OF 1.9 MILLION
CONGRESS:
CANADIANA: 1 MILLION
LIBRARY AND 3.5 MILLION
ARCHIVES CANADA: (ARCHIVAL
DESCRIPTIONS)

9. BIG DATA
IS COMPLICATED

10. 1966

11. 1976

12. ≠

13. ≠

14. NOT REALLY

15. ಠ_ಠ

18. SCALABILITY

19. ● ICA-AtoM (LAMP)
● BENCHMARK 3.5M RECORDS
● 100% OPEN SOURCE SOFTWARE
● COMMODITY HARDWARE

21. CAN WE DO IT?

22. WRITE SPEED

23. READ SPEED

24. WRITE MEMORY

25. READ MEMORY

26. NOSQL vs. SQL
(a.k.a. ODM vs. ORM)
● 4x - 10x FASTER
● 50% - 90% LESS MEMORY

29. RELATIONAL IF YOUR DATA
DATABASES IS NOT
SCALE WELL HIERARCHICAL
SOLR IF YOU HAVE
SCALES WELL INFINITE RAM
BEWARE THE NOSQL IS A
DOGMA OF SQL VIABLE OPTION
THINK SIDEWAYS SCALE OUT →

31. THE CLOUD IS A LIE

32. “big data is less about
size, and more about
freedom”
open source tools
+ distributed design
= new opportunities