This document provides an overview of Tachyon, an open-source memory-centric distributed storage system. It discusses Tachyon's motivation of leveraging faster memory storage, architecture that manages data across tiers of storage, and status as an Apache project used in production at over 50 companies. The document outlines Tachyon's ability to improve data sharing and fault tolerance across different compute frameworks. It shares several customer use cases and previews new features around lineage and tiered storage. The overall goal of Tachyon is to better assist other components with distributed memory-centric storage.

1. Haoyuan Li, Tachyon Nexus
haoyuan@tachyonnexus.com
September 22, 2015 @ SDC 2015
A Reliable Memory-Centric
Distributed Storage System

2. • Team consists of Tachyon creators, top contributors,
people from UC Berkeley, Google, CMU, VMware, Stanford,
Facebook, etc.
• $7.5 million Series A from Andreessen Horowitz
• Committed to Tachyon Open Source
2

3. 3

4. Outline
• Overview
– Motivation
– Tachyon Architecture
– Using Tachyon
• Open Source
– Status
– Production Use Cases
• Roadmap
4

5. Outline
• Overview
– Motivation
– Tachyon Architecture
– Using Tachyon
• Open Source
– Status
– Production Use Cases
• Roadmap
5

6. Tachyon: Born in UC Berkeley AMPLab
6
Cluster manager
Parallel computation
framework
Reliable, distributed memory-centric storage system

7. 7
Why Tachyon?

8. Memory is Fast
• RAM throughput
increasing exponentially
• Disk throughput
increasing slowly
8
Memory-locality key to interactive response times

9. Memory is Cheaper
source:
jcmit.com
9

10. Realized by many…
10

11. 11
Is the
Problem Solved?

12. 12
Missing a Solution
for the Storage Layer

13. An Example: -
• Fast, in-memory data processing framework
– Keep one in-memory copy inside JVM
– Track lineage of operations used to derive data
– Upon failure, use lineage to recompute data
map
filter
map
join
reduce
Lineage Tracking
13

14. Issue 1
14
Data Sharing is the bottleneck in
analytics pipeline:
Slow writes to disk
Spark Job1
Spark mem
block manager
block 1
block 3
Spark Job2
Spark mem
block manager
block 3
block 1
HDFS / Amazon S3
block 1
block 3
block 2
block 4
storage engine &
execution engine
same process
(slow writes)

15. Issue 1
15
Spark Job
Spark mem
block manager
block 1
block 3
Hadoop MR Job
YARN
HDFS / Amazon S3
block 1
block 3
block 2
block 4
Data Sharing is the bottleneck in
analytics pipeline:
Slow writes to disk
storage engine &
execution engine
same process
(slow writes)

16. Issue 2
16
Spark Task
Spark memory
block manager
block 1
block 3
HDFS / Amazon S3
block 1
block 3
block 2
block 4
execution engine &
storage engine
same process
Cache loss when process
crashes

17. Issue 2
17
crash
Spark memory
block manager
block 1
block 3
HDFS / Amazon S3
block 1
block 3
block 2
block 4
execution engine &
storage engine
same process
Cache loss when process
crashes

18. HDFS / Amazon S3
Issue 2
18
block 1
block 3
block 2
block 4
execution engine &
storage engine
same process
crash
Cache loss when process
crashes

19. HDFS / Amazon S3
Issue 3
19
In-memory Data Duplication &
Java Garbage Collection
Spark Task1
Spark mem
block manager
block 1
block 3
Spark Task2
Spark mem
block manager
block 3
block 1
block 1
block 3
block 2
block 4
execution engine &
storage engine
same process
(duplication & GC)

20. Tachyon
Reliable data sharing at
memory-speed within and across
cluster frameworks/jobs
20

21. Technical Overview
Ideas
• A memory-centric storage architecture
• Push lineage down to storage layer
• Manage tiered storage
Facts
• One data copy in memory
• Re-computation for fault-tolerance
21

22. Eco-System
22

23. Tachyon Memory-Centric
Architecture
23

24. Tachyon Memory-Centric
Architecture
24

25. Lineage in Tachyon
25

26. Issue 1 revisited
26
Memory-speed data sharing
among jobs in different
frameworks
execution engine &
storage engine
same process
(fast writes)
Spark Job
Spark mem
Hadoop MR Job
YARN
HDFS / Amazon S3
block 1
block 3
block 2
block 4
HDFS
disk
block
1
block
3
block
2
block
4
Tachyon"
in-memory
block 1
block 3
block 4

27. HDFS / Amazon S3
block 1
block 3
block 2
block 4
Tachyon"
in-memory
block 1
block 3
block 4
Issue 2 revisited
27
Spark Task
Spark memory
block manager
execution engine &
storage engine
same process
Keep in-memory data safe,
even when a job crashes.

28. Issue 2 revisited
28
HDFS
disk
block
1
block
3
block
2
block
4
execution engine &
storage engine
same process
Tachyon"
in-memory
block 1
block 3
block 4
crash
HDFS / Amazon S3
block 1
block 3
block 2
block 4
Keep in-memory data safe,
even when a job crashes.

29. Issue 3 revisited
29
No in-memory data duplication,
much less GC
Spark Task
Spark mem
Spark Task
Spark mem
HDFS / Amazon S3
block 1
block 3
block 2
block 4
execution engine &
storage engine
same process
(no duplication & GC)
HDFS
disk
block
1
block
3
block
2
block
4
Tachyon"
in-memory
block 1
block 3
block 4

30. Comparison with In-Memory HDFS
30

31. Outline
• Overview
– Motivation
– Tachyon Architecture
– Using Tachyon
• Open Source
– Status
– Production Use Cases
• Roadmap
31

32. Open Source Status
• Started at UC Berkeley AMPLab in Summer 2012
• Apache License 2.0, Version 0.7.1 (August 2015)
• Deployed at > 50 companies (July 2014)
• 30+ Companies Contributing
32

33. Contributors Growth
33
v0.4"
Feb ‘14
v0.3"
Oct ‘13
v0.2
Apr ‘13
v0.1
Dec ‘12
v0.6"
Mar ‘15
v0.5"
Jul ‘14
v0.7"
Jul ‘15
1
3
15
30
46
70
111

34. Codebase Growth
34
v0.4"
Feb ‘14
v0.3"
Oct ‘13
v0.2
Apr ‘13
v0.6"
Mar ‘15
v0.5"
Jul ‘14
v0.7"
Jul ‘15
465
commits
696
commits
1080
commits
1610
commits
2884
commits
5021
commits

35. Thanks to Our Contributors!
35

36. Reported Tachyon Usage
36

37. Under Filesystem Choices
(Big Data, Cloud, HPC, Enterprise)
37

38. Use Case: Baidu
• Framework: SparkSQL
• Under Storage: Baidu’s File System
• Storage Media: MEM + HDD
• 100+ nodes deployment
• 1PB+ managed space
• 30x Performance Improvement
38

39. Use Case: a SAAS Company
• Framework: Impala
• Under Storage: S3
• Storage Media: MEM + SSD
• 15x Performance Improvement
39

40. Use Case: an Oil Company
• Framework: Spark
• Under Storage: GlusterFS
• Storage Media: MEM only
• Analyzing data in traditional storage
40

41. Use Case: a SAAS Company
• Framework: Spark
• Under Storage: S3
• Storage Media: SSD only
• Elastic Tachyon deployment
41

42. Outline
• Overview
– Motivation
– Tachyon Architecture
– Using Tachyon
• Open Source
– Status
– Production Use Cases
• Roadmap
42

43. New Features
• Lineage in Storage (alpha)
• Tiered Storage (alpha)
43

44. New Features
• Lineage in Storage (alpha)
• Tiered Storage (alpha)
• Data Serving
• Support for New Hardware
• …
• Your New Feature!
44

45. 45
Tachyon’s Goal?

46. Distributed Memory-Centric Storage:
Better Assist Other Components
Welcome Collaboration!
46
JIRA New Contributor Tasks

47. • Website: http://tachyon-project.org
• Github: https://github.com/amplab/tachyon
• Meetup: http://www.meetup.com/Tachyon
• News Letter Subscription: http://goo.gl/mwB2sX
• Email: haoyuan@tachyonnexus.com
47