Duy Hai Doan presented information on SASI (SSTable Attached Secondary Index), a new secondary indexing technique for Apache Cassandra. SASI indexes follow the life cycle of SSTables, using new data structures like radix trees. It allows full text search without dependency on Lucene. The presentation covered how SASI indexes are distributed, its query planning and optimizations, and benchmarks showing its performance. While not intended for complex search or analytics, SASI can address 80% of typical search use cases within Cassandra.

1. @doanduyhai
SASI, Cassandra on the full text search ride
DuyHai DOAN
Apache Cassandra Evangelist
AMSTERDAM 11-12 MAY 2016

2. @doanduyhai
Who Am I ?
Duy Hai DOAN
Apache Cassandra Evangelist
• talks, meetups, confs
• open-source devs (Achilles, Apache Zeppelin…)
• OSS Cassandra point of contact
☞ duy_hai.doan@datastax.com
☞ @doanduyhai
2

3. @doanduyhai
Datastax
• Founded in April 2010
• We contribute a lot to Apache Cassandra™
• 400+ customers (25 of the Fortune 100), 450+ employees
• Headquarter in San Francisco Bay area
• EU headquarter in London, offices in France and Germany
• Datastax Enterprise = OSS Cassandra + extra features
3

4. SASI Index
• What is SASI ?
• Distributed Index
• Life-cycle
• Query Planner

5. What is SASI ?

6. @doanduyhai
Who ?
• Open source contribution by an engineers team
6

7. @doanduyhai
How ?
7
New secondary index re-designed from scratch
• follow SSTable life-cycle (flush, compaction)
• new data-strutures
• full text search options
• no dependency on Apache Lucene
SASI = SSTable-Attached Secondary Index

8. SASI Demo

9. SASI Demo
9

10. Distributed Index

11. @doanduyhai
Index on user country
11
H
A
E
D
B C
G F
FR user1 user102 … user493
US user54 user483 … user938
FR user87 user176 … user987
FR user17 user409 … user787

12. @doanduyhai
Distributed search query handling
12
H
A
E
D
B C
G F
coordinator
1st round
Concurrency factor = 1

13. @doanduyhai
Distributed search query handling
13
H
A
E
D
B C
G F
coordinator
Not enough
results ?

14. @doanduyhai
Distributed search query handling
14
H
A
E
D
B C
G F
coordinator
2nd round
Concurrency factor = 2

15. @doanduyhai
Distributed search query handling
15
H
A
E
D
B C
G F
coordinator
Still not enough
results ?

16. @doanduyhai
Distributed search query handling
16
H
A
E
D
B C
G F
coordinator
3rd round
Concurrency factor = 4

17. @doanduyhai
Caveat 1: query with non-restrictive filters
17
H
A
E
D
B C
G F
coordinator
Hit all
nodes
L

18. @doanduyhai
Caveat 1 solution: always use LIMIT
18
H
A
E
D
B C
G F
coordinator
SELECT *
FROM …
WHERE ...
LIMIT 1000

19. @doanduyhai
Caveat 2: 1-to-1 index (user_email)
19
H
A
E
D
B C
G F
coordinator
Not found WHERE user_email
LIKE '%xxx%'

20. @doanduyhai
Caveat 2: 1-to-1 index (user_email)
20
H
A
E
D
B C
G F
coordinator
Still no result
WHERE user_email
LIKE '%xxx%'

21. @doanduyhai
Caveat 2: 1-to-1 index (user_email)
21
H
A
E
D
B C
G F
coordinator
At best 1 user found
At worst 0 user found
WHERE user_email
LIKE '%xxx%'

22. @doanduyhai
Caveat 2 solution: use materalized views
22
For 1-to-1 index/relationship, use materialized views instead
CREATE MATERIALIZED VIEW user_by_email AS
SELECT * FROM users
WHERE user_id IS NOT NULL and user_email IS NOT NULL
PRIMARY KEY (user_email, user_id)

23. @doanduyhai
Caveat 3: fetch all rows for analytics use-case
23
H
A
E
D
B C
G F
coordinator
Hit all
nodes
L

24. @doanduyhai
Caveat 3 solution: use co-located Apache Spark
24
H
A
E
D
B C
G F
Local index filtering in Cassandra
Aggregation in Spark
Local index query

25. 25
Q & A
! "

26. SASI Life-cycle

27. @doanduyhai
SASI Life-cycle: in-memory
27
Commit log1
. . .
1
Commit log2
Commit logn
Memory
. . .
MemTable
Table1
MemTable
Table2
MemTable
TableN
2
Index
MemTable1
Index
MemTable2
. . .
Index
MemTableN
3
ACK the client

28. @doanduyhai
IndexMemtable
28
Index mode, data type Data structure Usage
PREFIX, text Guava ConcurrentRadixTree name LIKE 'John%'
CONTAINS, text Guava ConcurrentSuffixTree
name LIKE ’%John%'
name LIKE ’%ny’
PREFIX, other JDK ConcurrentSkipListSet
age = 20
age >= 20 AND age <= 30
SPARSE, other JDK ConcurrentSkipListSet
age = 20
age >= 20 AND age <= 30

29. @doanduyhai
SASI Life-cycle: flush to SSTable
29
Commit log1
. . .
1
Commit log2
Commit logn
Memory
Table1
SStable1
Table2 Table3
SStable2 SStable3
4
OnDiskIndex1
OnDiskIndex2
OnDiskIndex3

30. @doanduyhai
SASI Life-cycle: compaction
30
SSTable1 SSTable2 SSTable3
New SSTable
OnDiskIndex1 OnDiskIndex2 OnDiskIndex3
New OnDiskIndex

31. @doanduyhai
OnDiskIndex Files
31
SStable1
SStable2
user_id4 FR user_id1 US user_id5 FR
user_id3 UK user_id2 DE
OnDiskIndex1
FR US
OnDiskIndex2
UK DE

32. @doanduyhai
OnDiskIndex Files
32
SStable1
SStable2
user_id4 FR user_id1 US user_id5 FR
user_id3 UK user_id2 DE
OnDiskIndex1
FR US
OnDiskIndex2
UK DE
Suffix Tree
Data structures

33. 33
Q & A
! "

34. Query Planner

35. @doanduyhai
Integrated query planner
35
Perform optimizations on predicates
1. build predicates tree
2. predicates push-down & re-ordering
3. predicate fusions for != operator

36. @doanduyhai
Query optimization example
36
WHERE age < 100 AND fname = 'p*' AND first_name != 'pa*' AND age > 21

37. @doanduyhai
Query optimization example
37
AND is associative
and commutative

38. @doanduyhai
Query optimization example
38
!= transformed to
exclusion on range scan

39. @doanduyhai
Query optimization example
39
AND is associative
and commutative

40. 40
Q & A
! "

41. Some Benchmarks

42. @doanduyhai
Hardware specs
42
13 bare-metal machines
• 6 CPU HT (12 vcores)
• 64Gb RAM
• 4 SSDs in RAID0 for a total of 1.5Tb
Data set
• 13 billions of rows
• 1 numerical index with 36 distinct values
• 2 text index with 7 distinct values
• 1 text index with 3 distinct values

43. @doanduyhai
Benchmark results
43

44. @doanduyhai
Benchmark results
44

45. @doanduyhai
Benchmark results
45

46. @doanduyhai
Benchmark results
46

47. @doanduyhai
Benchmark results
47
Full scan using server-side paging
Predicate count Fetched rows Query time in sec
1 36 109 986 609
2 2 781 492 330
3 1 044 547 372
4 360 334 116

48. Take Away

49. @doanduyhai
Conclusion
49
Is it available ?
• yes in Cassandra 3.5
Future enhancement ?
• index on collections (List, Set & Map) !
• OR clause (WHERE (xxx OR yyy) AND zzz )
• != operator

50. @doanduyhai
Conclusion
50
SASI vs Solr/ElasticSearch ?
• Cassandra is not a search engine !!! (database = durability)
• always slower because 2 passes (SASI index read + original Cassandra data)
• no scoring
• no ordering (ORDER BY)
• no grouping (GROUP BY) à Apache Spark for analytics
Still, SASI covers 80% of search use-cases and people are happy !

51. 51
@doanduyhai
duy_hai.doan@datastax.com
https://academy.datastax.com/
Thank You