Elixir / Erlang SF Meetup - 2017 / June

1. Handling Large StateHandling Large State
on BEAMon BEAM
Yoshihiro Tanaka
Erlang & Elixir SF Meetup 2017-June-29
1 . 1

2. TopicsTopics
Problem statement & Use-case / experience
Process Structure
Data Copy in Erlang VM
Constant Pool & Demo
1 . 2

3. Problem:Problem:
How to deal withHow to deal with
Large number/sizeLarge number/size
of State ?of State ?
1 . 3

4. Our Story: BackendOur Story: Backend
component Requirementcomponent Requirement
Matching incoming data to the local inventories
4-5k rules(inventory) to apply for each incoming request
Return qualified inventories to caller
Matching
Incoming rules,
properties
Qualified inventries
Caller
Each request
Set of Rules
1 . 4

5. Design 1: A process keeps one ruleDesign 1: A process keeps one rule
Forward the request to all
processes, gather results
1 . 5Rule1 Rule2 Rule3 Rule4
ETS

6. Design 1: A rule is kept as fun object andDesign 1: A rule is kept as fun object and
evaluatedevaluated
1> {ok, F} = formulerl_fun:compile(
"if (x > 23) then
{ x^5 + 2*x^3 + 5*x^2 + 48 }
else { x^3 + 2*x^4 + 5*x + 48 }"
).
{ok,#Fun<formulerl_fun.6.29233594>}
2> F(dict:store("x", 2, dict:new())).
98.0
3> erlang:size(term_to_binary(F)).
2961
Simple proof of concept:
This 'Fun' object can be stored/read from/to ETS
https://github.com/hirotnk/formulerl
1 . 6

7. Problem of Design1: A processProblem of Design1: A process
keeps one rulekeeps one rule
1 request = messages to&from each
process(rule) ex. 2k * 2=~4k messages
Too many processes(ex. 2k) in run queue
1 . 7

8. ETS
Design 2: Process Pool and One workerDesign 2: Process Pool and One worker
keeps All ruleskeeps All rules
Pool Worker
processes
1 . 8

9. Problem of Design2: MessageProblem of Design2: Message
costs & Memory usagecosts & Memory usage
This is the common pattern, but...
One worker process keeps all rules(data)
=> high memory usage
Cost of message passing: Need to
forward a request to a worker process
1 . 9

10. Cost of MessageCost of Message
Passing?Passing?
Wait, isn't it fast/cheap on Erlang VM ?
1 . 10

11. Process StructureProcess Structure
PCB Stack
Heap
Mail box
Free
Old Heap
Processes don't share state
m-buf m-buf m-buf
1 . 11

12. Cost of Message PassingCost of Message Passing
Sending process has to:
1. Calculate the size of the message
2. Allocate memory for the size (if needed off heap copy)
3. Copy the message to allocated memory
4. Allocate defined structure for message
5. Link the message to receiving-process' mailbox
Receiving-process copies data from its mailbox to
process heap
These steps involve lock/unlock, GC, memory allocation
So, the cost of message passing is
something, rather than nothing. 1 . 12

13. Constant
Pool
1 . 13
Design 3: Keep data in Constant poolDesign 3: Keep data in Constant pool
All processing happens in one process,
the result is returned from it.
No messages involved.

14. So, what is Constant Pool ?So, what is Constant Pool ?
“ Constant Erlang terms (also called literals)
are now kept in constant pools; each loaded
module has its own pool.
From 8.2 Process Messages:
http://erlang.org/doc/efficiency_guide/processes.html
1 . 14

15. Benefit of using Constant PoolBenefit of using Constant Pool
Global Access without Data Copy
1 . 15

16. Constant Pool DemoConstant Pool Demo
Compile with: erlc -'S' cp.erl
1 -module(cp).
2 -export([
3 get/0,
4 get_shared/0
5 ]).
6
7 get() ->
8 lists:duplicate(10000, $a).
9
10 get_shared() ->
11 L = "aaaa....aa", % omitted
12 10000 = length(L),
13 L.
14
1 . 16

17. 1 {module, cp}. %% version = 0
2
3 {exports, [{get,0},
{get_shared,0},
{module_info,0},{module_info,1},{vsn,0}]}.
4
5 {attributes, []}.
6
7 {labels, 11}.
8
9
10 {function, get, 0, 2}.
11 {label,1}.
12 {line,[{location,"cp.erl",10}]}.
13 {func_info,{atom,cp},{atom,get},0}.
14 {label,2}.
15 {move,{integer,97},{x,1}}.
16 {move,{integer,10000},{x,0}}.
17 {line,[{location,"cp.erl",11}]}.
18 {call_ext_only,2,{extfunc,lists,duplicate,2}}.
19
20
21 {function, get_shared, 0, 4}.
22 {label,3}.
23 {line,[{location,"cp.erl",13}]}.
24 {func_info,{atom,cp},{atom,get_shared},0}.
25 {label,4}.
26 {move,{literal,"aaa...aaa"}, % Omitted actual "aaa.."
27 {x,0}}.
28 return.
29... 1 . 17

18. 4> spawn(fun() -> L = cp:get_shared(), io:format("-->~p~n", [process_info(self(), [heap_size])])
end).
-->[{heap_size,233}]
<0.41.0>
5> spawn(fun() -> L = cp:get(), io:format("-->~p~n", [process_info(self(), [heap_size])]) end).
-->[{heap_size,1598}]
<0.43.0>
Constant Pool DemoConstant Pool Demo
1 . 18

19. Caveat about Constant PoolCaveat about Constant Pool
When the code is unloaded, the constants are copied
to the heap of the processes that refer to them
(Efficiency guide 8.2)
Current Current Old
v1 v1v2
Current Old
v2v3
1 . 19
CP CP
Reference to CP

20. Benefit of Design 3: No Data CopyBenefit of Design 3: No Data Copy
Reading from Constant Pool !=
Reading from ETS
Reading from Constant Pool => no copy
Reading from ETS => data is copied to the process
No message needed for each incoming
requests
1 . 20

21. Design 3: But wait !Design 3: But wait !
You can not.
Instead, we turned rules into module using .Merl
How can you turnHow can you turn funfun objects intoobjects into
Constant Pool ?Constant Pool ?
1> formulerl_beam:compile(
calc_example_mod,
"if (x > 23) then
{ x^5 + 2*x^3 + 5*x^2 + 48 }
else
{ x^3 + 2*x^4 + 5*x + 48 }"
).
ok
2> calc_example_mod:calc(dict:store("x", 2, dict:new())).
98.0
Simple proof of concept:
data == code + constant pool
https://github.com/hirotnk/formulerl
1 . 21

22. Revisit Design 1: A rule is kept as funRevisit Design 1: A rule is kept as fun
object and evaluatedobject and evaluated
1> {ok, F} = formulerl_fun:compile(
"if (x > 23) then
{ x^5 + 2*x^3 + 5*x^2 + 48 }
else { x^3 + 2*x^4 + 5*x + 48 }"
).
{ok,#Fun<formulerl_fun.6.29233594>}
2> F(dict:store("x", 2, dict:new())).
98.0
3> erlang:size(term_to_binary(F)).
2961
Simple proof of concept:
This 'Fun' object can be stored/read from/to ETS
https://github.com/hirotnk/formulerl
1 . 22

23. Cool, all sounds good now.
BUT...Capacity increase was only ~100%
1 . 23

24. Further investigation: Cost of Data CopyFurther investigation: Cost of Data Copy
It was reading data from ETS in the code
Data was supposed to be small...except
some entries...
A couple of entries contained large data
We fixed it by turning those into binary (refc
binary)
1 . 24

25. The result: ~800% capacity increase in totalThe result: ~800% capacity increase in total
Latency (ms)
1 . 25

26. When data copy happens ?When data copy happens ?
Message passing
Process creation
Read/Write from ETS/DTS/Mnesia
1 . 26

27. When data copy does NOTWhen data copy does NOT
happen ?happen ?
Passing data around inside process(ex. function
call)
Binary > 64 bytes are not copied
Read from Constant Pool
Literals are not copied between processes for
messages or spawn (new behavior ~20.0rc-1)
1 . 27

28. 20.0-rc120.0-rc1
OTP­13529 Application(s): erts
Erlang literals are no longer
copied during process to process messaging.
1 . 28

29. Summary of our use case:Summary of our use case:
How to avoid data copyHow to avoid data copy
1. Turn thousands of URLs into trie tree, save it to
constant pool using mochiglobal. Then processes
can access to that tree data in parallel, without any
copy
2. When we have relatively large data, try to turn it
into binary
3. Turn DSL rules(4-5K) into modules using
merl/syntax_tools (this works fine with Erlang VM)
1 . 29

30. Constant Pool ToolsConstant Pool Tools
parse transform
mochiglobal
merl
Macro ?
???
fastglobal
Erlang Elixir
1 . 30

31. ReferenceReference
https://github.com/happi/theBeamBook
https://www.erlang-solutions.com/blog/erlang-19-0-garbage-
collector.html
1 . 31

32. AcknowledgmentsAcknowledgments
This work was done with following my colleagues at OpenX, and I'd
like to express my appreciation for their insights and helps:
Kenan Gillet
David Hull
1 . 32

33. Thank you !Thank you !
Q & AQ & A
1 . 33