Cassandra UDF and Materialized Views

1. User-Defined Functions & Materialized Views
DuyHai DOAN

2. User Define Functions
• Why ?
• Detailed Impl
• UDAs
• Gotchas
© 2015 DataStax, All Rights Reserved.
2

3. Rationale
• Push computation server-side
• save network bandwidth (1000 nodes!)
• simplify client-side code
• provide standard & useful function (sum, avg …)
• accelerate analytics use-case (pre-aggregation for Spark)
© 2015 DataStax, All Rights Reserved.
3

4. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
4
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALL ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘;

5. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
5
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘;
CREATE OR REPLACE FUNCTION IF NOT EXISTS

6. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
6
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘;
An UDF is keyspace-wide

7. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
7
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘;
Param name to refer to in the code
Type = CQL3 type

8. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
8
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language // j
AS ‘
// source code here
‘;
Always called
Null-check mandatory in code

9. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
9
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language // jav
AS ‘
// source code here
‘;
If any input is null, code block is
skipped and return null

10. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
10
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘;
CQL types
• primitives (boolean, int, …)
• collections (list, set, map)
• tuples
• UDT

11. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
11
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘; JVM supported languages
• Java, Scala
• Javascript (slow)
• Groovy, Jython, JRuby
• Clojure ( JSR 223 impl issue)

12. How to create an UDF ?
© 2015 DataStax, All Rights Reserved.
12
CREATE [OR REPLACE] FUNCTION [IF NOT EXISTS]
[keyspace.]functionName (param1 type1, param2 type2, …)
CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT
RETURN returnType
LANGUAGE language
AS ‘
// source code here
‘;
Avoid simple quotes ( ‘ ) or escape them

13. © 2015 DataStax, All Rights Reserved.
13
Simple UDF Demo
max()

14. UDF creation for C* 2.2 and 3.0
• UDF definition sent to coordinator
• Check for user permissions (ALTER, DROP, EXECUTE)
• Coordinator compiles it using ECJ compiler (not Javac because of licence
issue).
• If language != Java, use other compilers (you should ship the jars on C*
servers)
• Announces SchemaChange = CREATED, Target = FUNCTION to all nodes
• Code loaded into child classloader of current classloader
© 2015 DataStax, All Rights Reserved.
14

15. UDF creation in C* 3.0
• Compiled byte code is verified
• static and instance code block
• declaring methods/constructor à no anonymous class creation
• declaring inner class
• use of synchronized
• accessing forbidden classes, ex: java.net.NetworkInterface
• accessing forbidden methods, ex: Class.forName()
• …
• Code loaded into seperated classloader
• verify loaded classes against whitelist/backlist
© 2015 DataStax, All Rights Reserved.
15

16. UDF execution in C* 2.2
• No safeguard, arbitrary code execution server side …
• Flag “enable_user_defined_functions” in cassandra.yaml turned OFF by
default
• Keep your UDFs pure (stateless)
• Avoid expensive computation (for loop …)
© 2015 DataStax, All Rights Reserved.
16

17. UDF execution in C* 3.0
• If "enable_user_defined_functions_threads" turned ON (default)
• execute UDF in separated ThreadPool
• if duration longer than "user_defined_function_warn_timeout", retry with new
timeout = fail_timeout – warn_timeout
• if duration longer than "user_defined_function_fail_timeout", exception
• Else
• execute in same thread
© 2015 DataStax, All Rights Reserved.
17

18. UDF creation best practices
© 2015 DataStax, All Rights Reserved.
18
• Keep your UDFs pure (stateless)
• No expensive computation (huge for loop)
• Prefer Java language for perf

19. UDA
© 2015 DataStax, All Rights Reserved.
19
• Real use-case for UDF
• Aggregation server-side à huge network bandwidth saving
• Provide similar behavior for Group By, Sum, Avg etc …

20. How to create an UDA ?
© 2015 DataStax, All Rights Reserved.
20
CREATE [OR REPLACE] AGGREGATE [IF NOT EXISTS]
[keyspace.]aggregateName(type1, type2, …)
SFUNC accumulatorFunction
STYPE stateType
[FINALFUNC finalFunction]
INITCOND initCond;
Only type, no param name
State type
Initial state type

21. How to create an UDA ?
© 2015 DataStax, All Rights Reserved.
21
CREATE [OR REPLACE] AGGREGATE [IF NOT EXISTS]
[keyspace.]aggregateName(type1, type2, …)
SFUNC accumulatorFunction
STYPE stateType
[FINALFUNC finalFunction]
INITCOND initCond;
Accumulator function. Signature:
accumulatorFunction(stateType, type1, type2, …)
RETURNS stateType

22. How to create an UDA ?
© 2015 DataStax, All Rights Reserved.
22
CREATE [OR REPLACE] AGGREGATE [IF NOT EXISTS]
[keyspace.]aggregateName(type1, type2, …)
SFUNC accumulatorFunction
STYPE stateType
[FINALFUNC finalFunction]
INITCOND initCond;
Optional final function. Signature:
finalFunction(stateType)

23. How to create an UDA ?
© 2015 DataStax, All Rights Reserved.
23
CREATE [OR REPLACE] AGGREGATE [IF NOT EXISTS]
[keyspace.]aggregateName(type1, type2, …)
SFUNC accumulatorFunction
STYPE stateType
[FINALFUNC finalFunction]
INITCOND initCond;
UDA return type ?
If finalFunction
• return type of finalFunction
Else
• return stateType

24. How to create an UDA ?
© 2015 DataStax, All Rights Reserved.
24
CREATE [OR REPLACE] AGGREGATE [IF NOT EXISTS]
[keyspace.]aggregateName(type1, type2, …)
SFUNC accumulatorFunction
STYPE stateType
[FINALFUNC finalFunction]
INITCOND initCond;
If accumulatorFunction RETURNS NULL ON NULL INPUT
• initCond should be defined
Else
• initCond can be null (default value)

25. UDA execution
© 2015 DataStax, All Rights Reserved.
25
• Start with state = initCond
• For each row
• call accumulator function with previous state + params
• update state with accumulator function output
• If no finalFunction
• return last state
• Else
• apply finalFunction and returns its output

26. © 2015 DataStax, All Rights Reserved.
26
UDA Demo
Group By

27. Gotchas
© 2015 DataStax, All Rights Reserved.
27
C* C*
C*
C*
UDF
①
②
③
④ UDF execution⑤

28. Gotchas
© 2015 DataStax, All Rights Reserved.
28
C* C*
C*
C*
UDF
①
②
③
④ UDF execution⑤
Why do not apply UDF on replica node ?

29. Gotchas
© 2015 DataStax, All Rights Reserved.
29
C* C*
C*
C*
UDA
①
② & ③
④
• apply accumulatorFunction
• apply finalFunction
⑤
② & ③
② & ③
1. Cannot apply
accumulatorFunction
on replicas
2. UDA applied AFTER
read-repair

30. Gotchas
© 2015 DataStax, All Rights Reserved.
30
C* C*
C*
C*
UDA
①
② & ③
④
• apply accumulatorFunction
• apply finalFunction
⑤
② & ③
② & ③
Avoid UDA on large number of rows
• WHERE partitionKey IN (…)
• WHERE clustering >= AND clusterting <=

31. Future applications of UDFs & UDAs
• Functional index (CASSANDRA-7458)
• Partial index (CASSANDRA-7391)
• WHERE clause filtering with UDF
• …
© 2015 DataStax, All Rights Reserved.
31

32. © 2015 DataStax, All Rights Reserved.
32
Q & A
! "

33. Materialized Views
• Why ?
• Detailed Impl
• Gotchas
© 2015 DataStax, All Rights Reserved.
33

34. Why Materialized Views ?
• Relieve the pain of manual denormalization
© 2015 DataStax, All Rights Reserved.
34
CREATE TABLE user(
id int PRIMARY KEY,
country text,
…
);
CREATE TABLE user_by_country(
country text,
id int,
…,
PRIMARY KEY(country, id)
);

35. Materialzed View In Action
© 2015 DataStax, All Rights Reserved.
35
CREATE MATERIALIZED VIEW user_by_country
AS SELECT country, id, firstname, lastname
FROM user
WHERE country IS NOT NULL AND id IS NOT NULL
PRIMARY KEY(country, id)
CREATE TABLE user_by_country (
country text,
id int,
firstname text,
lastname text,
PRIMARY KEY(country, id));

36. Materialzed View Syntax
© 2015 DataStax, All Rights Reserved.
36
CREATE MATERIALIZED VIEW [IF NOT EXISTS]
keyspace_name.view_name
AS SELECT column1, column2, ...
FROM keyspace_name.table_name
WHERE column1 IS NOT NULL AND column2 IS NOT NULL ...
PRIMARY KEY(column1, column2, ...)
Must select all primary key columns of base table
• IS NOT NULL condition for now
• more complex conditions in future
• at least all primary key columns of base table
(ordering can be different)
• maximum 1 column NOT pk from base table
• 1:1 relationship between base row & MV row

37. © 2015 DataStax, All Rights Reserved.
37
Materialzed View Demo
user_by_country

38. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
38
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
①
create BatchLog
for update
OPTIONAL STEP
activated by
–Dcassandra.mv_enable_coordinator_batchlog=true

39. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
39
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
②
• send mutation to all replicas
• waiting for ack(s) with CL

40. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
40
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
③
Acquire local lock on
base table partition

41. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
41
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
④
Local read to fetch current values
SELECT * FROM user WHERE id=1

42. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
42
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
⑤
Create BatchLog with
• DELETE FROM user_by_country
WHERE country = ‘old_value’
• INSERT INTO
user_by_country(country, id, …)
VALUES(‘FR’, 1, ...)

43. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
43
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
⑥
Execute async BatchLog
to paired view replica
with CL = ONE

44. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
44
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
⑦
Apply base table updade locally
SET COUNTRY=‘FR’

45. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
45
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
⑧
Release local lock

46. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
46
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
⑨
Return ack to
coordinator

47. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
47
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
①⓪
If CL ack(s)
received, ack client

48. Materialized View Impl
© 2015 DataStax, All Rights Reserved.
48
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
①①
If QUORUM ack(s)
received, remove
BatchLog
OPTIONAL STEP
activated by
–Dcassandra.mv_enable_coordinator_batchlog=true

49. Materialzed Views impl explained
What is paired replica ?
• Base primary replica for id=1 is paired with MV primary replica for
country=‘FR’
• Base secondary replica for id=1 is paired with MV secondary replica for
country=‘FR’
• etc …
© 2015 DataStax, All Rights Reserved.
49

50. Materialzed Views impl explained
Why local lock on base replica ?
• to provide serializability on concurrent updates
Why BatchLog on base replica ?
• necessary for eventual durability
• replay the MV delete + insert until successful
Why BatchLog on base replica uses CL = ONE ?
• each base replica is responsible for update of its paired MV replica
• CL > ONE will create un-necessary duplicated mutations
© 2015 DataStax, All Rights Reserved.
50

51. Materialzed Views impl explained
Why BatchLog on coordinator with QUORUM ?
• necessary to cover some edge-cases
© 2015 DataStax, All Rights Reserved.
51

52. MV Failure Cases: concurrent updates
© 2015 DataStax, All Rights Reserved.
52
Read base row (country=‘UK’)
• DELETE FROM mv WHERE
country=‘UK’
• INSERT INTO mv …(country)
VALUES(‘US’)
• Send async BatchLog
• Apply update country=‘US’
1) UPDATE … SET country=‘US’ 2) UPDATE … SET country=‘FR’
Read base row (country=‘UK’)
• DELETE FROM mv WHERE
country=‘UK’
• INSERT INTO mv …(country)
VALUES(‘FR’)
• Send async BatchLog
• Apply update country=‘FR’
t0
t1
t2
Without local lock

53. MV Failure Cases: concurrent updates
© 2015 DataStax, All Rights Reserved.
53
Read base row (country=‘UK’)
• DELETE FROM mv WHERE
country=‘UK’
• INSERT INTO mv …(country)
VALUES(‘US’)
• Send async BatchLog
• Apply update country=‘US’
1) UPDATE … SET country=‘US’ 2) UPDATE … SET country=‘FR’
Read base row (country=‘US’)
• DELETE FROM mv WHERE
country=‘US’
• INSERT INTO mv …(country)
VALUES(‘FR’)
• Send async BatchLog
• Apply update country=‘FR’
With local lock
🔒
🔓 🔒
🔓

54. MV Failure Cases: failed updates to MV
© 2015 DataStax, All Rights Reserved.
54
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
⑥
Execute async BatchLog
to paired view replica
with CL = ONE
✘
MV replica down

55. MV Failure Cases: failed updates to MV
© 2015 DataStax, All Rights Reserved.
55
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
BatchLog
replay
MV replica up

56. MV Failure Cases: base replica(s) down
© 2015 DataStax, All Rights Reserved.
56
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
②
• send mutation to all replicas
• waiting for ack(s) with CL
✘
✘
⁇⁇
✔︎
Base replicas down

57. MV Failure Cases: base replica(s) down
© 2015 DataStax, All Rights Reserved.
57
C*
C*
C*
C*
C* C*
C* C*
UPDATE user
SET country=‘FR’
WHERE id=1
✔✔
✔︎
Base replicas up
BatchLog
replay

58. Materialized Views Gotchas
• Write performance
• local lock
• local read-before-write for MV (most of perf hits here)
• local batchlog for MV
• coordinator batchlog (OPTIONAL)
• ☞ you only pay this price once whatever number of MV
• Consistency level
• CL honoured for base table, ONE for MV + local batchlog
• QUORUM for coordinator batchlog (OPTIONAL)
© 2015 DataStax, All Rights Reserved.
58

59. Materialized Views Gotchas
• Beware of hot spots !!!
• MV user_by_gender 😱
• Repair, read-repair, index rebuild, decomission …
• repair on base replica à update on MV paired replica
• repair on MV replica possible
• read-repair on MV behaves as normal read-repair
© 2015 DataStax, All Rights Reserved.
59

60. Materialized Views Gotchas
• Schema migration
• cannot drop column from base table used by MV
• can add column to base table, initial value = null from MV
• cannot drop base table, drop all MVs first
© 2015 DataStax, All Rights Reserved.
60

61. © 2015 DataStax, All Rights Reserved.
61
Q & A
! "

62. © 2015 DataStax, All Rights Reserved.
62
@doanduyhai
duy_hai.doan@datastax.com
https://academy.datastax.com/
Thank You