This presentation discusses the advantages of column-oriented databases and data storage. It summarizes different database technologies like HBase and Cassandra, and how column-oriented storage is better suited for certain tasks like analytics. It also talks about future innovations for column databases, including deeper column hierarchies and more distributed and flexible schemas.

1. Breadth
or Depth
What's in a column-store?
February 23, 2013 Jeff Smith

2. This presentation
Is not Is
marketing persuasive
technical for the technical
arbitrary precise
polite opinionated
training educational

3. Srsouly

4. Bio
{ past :[startups, biotech, data_management],
school : [research, HKU, uncertain_data],
work : [AI, finance, prediction] }

5. This guy
Daniel Abadi

6. Back to the future
● 1 database to rule them all
● A scrappy band of rebels
● A brave new idea

7. The big question
Why grab this?
id thing attr1 attr2 attr3 attr4 attr5 attr6 attr7 attr8
123 doodad abc def ghi jkl mno pqr stu vwx
When all you want is this?
id thing
123 doodad

8. You're chopping it wrong.

9. Relations in pieces
id pet weight poops_per_day
1 dog 40 3
2 cat 15 2
3 bird 5 4
4 snake 78 0.25

10. Horizontal Partitions
id pet weight poops_per_day
1 dog 40 3
2 cat 15 2
3 bird 5 4
4 snake 78 0.25

11. You gotta get yourself some marble columns.

12. Vertical Partitions
id pet weight poops_per_day
1 dog 40 3
2 cat 15 2
3 bird 5 4
4 snake 78 0.25

13. We're gonna need a bigger table.

14. BigTable
NoSQL starts
Empire crumbles
Nomenclature obfuscates

15. I know that song!

16. Column...families?!
Pets Cars
row_id best_pet worst_pet illegal_pet row_id make model
123 bulldog turtle rhino 123 Smart Fortwo

17. Modest Map
Year of the snake => Year of Python
4G => LTE
NoSQL => Non-relational
Beard => Face-mane
Column-stores => {column-store |
column-family-store}

18. Does it smell as sweet?

19. C-Store rocks*
...at column-oriented tasks.
* Contrary to popular belief, after years
of effort, Cleveland still does not rock.

20. Move, b*tch.
Get out the vote.
age
23
32
45
67
56
49
43
50
63
34

21. The catch

22. Attack of the clones

23. The contenders
HBase*
Cassandra*
Hypertable
Accumulo
* The ones that matter

24. HBase
Hadoop stack
Java everywhere
Components,
extensions, variables,
headaches...

25. Tastes like SQL
SELECT sensorid, (20-down)/(up-down) AS
probability
FROM hive_sensors WHERE down>=10 AND
up>=20 and down <=20
UNION ALL
SELECT sensorid, (up-10)/(up-down) AS
probability
FROM hive_sensors WHERE up>=10 AND up<=20
and down <=10
UNION ALL
SELECT sensorid, 1 AS probability
FROM hive_sensors WHERE up<=20 and down
>=10
UNION ALL
SELECT sensorid, (20-10)/(up-down) AS
probability
FROM hive_sensors WHERE down<=10 AND
up>=20;

26. Cassandra
CQL interface
Peer to peer
Better, but...

27. Anything you can do, I can do better.

28. Sparseness
id attr1 attr2 attr3 attr4
1 1
2 1
3 1
4 1
5
6 1
7
8 1
9 1
10
11

29. Dynamic Schemas
Pets Cars
row_id best_pet worst_pet illegal_pet robot_pet row_id make model
123 bulldog turtle rhino aibo 123 Smart Fortwo
456 shi tzu gecko koala 456 VW Golf

30. Stronger in the broken places

31. Innovation
Truly distributed systems
Columns as metadata
Arbitrarily deep column hierarchies*
Community database development
* Someday soon, I hope

32. Pig & friends
data = load 'hbase://table_name' using
org.apache.pig.backend.hadoop.hbase.
HBaseStorage( 'cf1:*', '-loadKey true' )
AS (id:chararray, stats: map[int]);
@outputSchema ("values:bag{t:tuple(key,
value)}")
def bag_of_tuples (map_dict):
return map_dict.items()
register 'udfs.py' using jython as py
data = load 'hbase://table_name' using
org.apache.pig.backend.hadoop.hbase.
HBaseStorage( 'cf1:*', '-loadKey true' )
AS (id:chararray, stats: map[int]);
databag = foreach data generate id,
FLATTEN(py.bag_of_tuples(stats));
from Chase Seibert

33. No dog in this fight

34. Hey I just met you
And this is crazy
But here's my email
Mail me maybe
Work Play
jeff@aidyia.com jeffreyksmithjr@gmail.com

35. All images used in this presentation were stolen
from the internet in a daring midnight raid that
left 3 dead and 8 wounded. No license was
obtained for their use and no license is implied
by their misappropriation.
Yarrr. BarrrCamp.
Please don't sue me. I have nothing. Just a
dog. Don't take my dog.
Disclaimer