The document discusses the design and development of a new memcached client called libmc, addressing issues with existing libraries like libmemcached. It includes technical details on the ASCII protocol for data operations, module diagrams, and lessons learned during the development process. The author shares tips for efficient coding practices and performance optimization.

1. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Reinventing the wheel: libmc
panmiaocai@douban.com
Jan 23, 2015

2. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
The memcached family
▶ memcached (the memcached server-side software)
▶ memcache (typo and nickname)
▶ mc (nickname)
▶ libmemcached ( C/C++ client library)
▶ libmemcache (yet another C client library, DEAD since 2006)
▶ python-memcached (pure python client library)
▶ python-libmemcached (Cython client, library based on
libmemcached)
▶ cmemcached (alias of python-libmemcached)
▶ pylibmc (C-Extensions for Python by hand, based on
libmemcached)

5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Bad things of libmemcached
▶ Divergence of libmemcached between upstream and the
branch we use. (the “large split” feature)
▶ Complex
▶ Buggy
▶ (lib)memcached is a weird creature.
http://hoborglabs.com/en/blog/2013/memcached-php

7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Retrieval Commands
Request and Response
get <key>*rn
VALUE <key> <flags> <bytes>rn
<data block>rn
Example
get foo bar bazrn
VALUE foo 0 3rn
abcrn
VALUE baz 0 5rn
simp.rn
ENDrn

9. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Storage Commands
Request and Response
<key> <flags> <exptime> <bytes>rn
<data block>rn
STOREDrn | NOT STOREDrn
Example
set dou 0 0 10rn
0123456789rn
set ban 0 0 4rn
aoeirn
NOT STOREDrn
STOREDrn

10. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

11. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Design of libmc
PyClient
|
Client
| (1:1)
ConnectionPool
| (1:N)
Connection
/ | (1:1)
BufferReader Parser BufferWriter
| (1:N)
DataBlock

12. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

13. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BufferWriter
s s i z e t sendmsg ( i n t socket , const s t r u c t msghdr ∗
message , i n t f l a g s ) ;
s t r u c t i o v e c {
void ∗ i o v b a s e ;
s i z e t i o v l e n ;
};
s t r u c t msghdr {
. . .
s t r u c t i o v e c ∗ msg iov ;
s i z e t msg iovlen ;
. . .
};
▶ UIO MAXIOV
▶ commitRead

14. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

15. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BufferReader
+ Fixed + Flexible
+------------+------------+-----------------------+---------------
| 8192 recved| 8192 recved| more than 8192 recved | DataBlock ...
+----+----+-------+-------+---------------------------------------
|XXXX|OOOO|BB|BBBB|CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC| DataBlockSlice
|XXXX|OOOO|BB|BBBB|CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC| ...
+------------+------+------------+------------+-------------------
|VALU|foo | 0 0 20| value of foo value of foo valu| Token
|E | | | e of foo .....................| ...
|----+----+-------+-------------------------------+---------------
▶ peek / readUntil / skipUtil / readUnsigned / readBytes /
skipBytes / (no rewind)
▶ token.size() in 0, 1, 2 / SmallVector
▶ predictive DataBlock size

16. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

17. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Other parts
▶ Parser as a finite-state machine (FSM)
▶ poll in ConnectionPool

18. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

19. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.

20. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.

21. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.

22. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.

23. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.

24. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error

25. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing

26. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.

27. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.

28. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.

29. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.

30. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.
▶ write the right benchmark. bench order? dependency?

31. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.
▶ write the right benchmark. bench order? dependency?
▶ md5 is slow. bugfree fnv1 is hard to implement.

32. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Tips
▶ Prefer “switch statements” to if-else condition.
▶ “Exception” and “heap allocation” are slow in C++.
▶ Avoid memory copy.
▶ SmallVector optimization.
▶ Undocumented “UIO MAXIOV” limit in iovec.
▶ clean buffer on error
▶ Server may refuse if too much data is “send”(sent) without
“recv”-ing
▶ Lazy connecting.
▶ GIL issue.
▶ gevent issue.
▶ man 2 recv: For TCP sockets, the return value 0 means the
peer has closed its half side of the connection.
▶ write the right benchmark. bench order? dependency?
▶ md5 is slow. bugfree fnv1 is hard to implement.
▶ vim: Rip-Rip/clang complete + cpplint.py

33. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
Wordplay
The memcached family
Why reinventing the new wheel
Bad things of libmemcached
Design a new memcached client
The ASCII protocol(get/set part)
Module Diagram
BufferWriter
BufferReader
Other parts
Lessons I learned
Tips
Great Toolchain

34. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Great Toolchain
▶ CMake
▶ Valgrind
▶ Callgrind(valgrind –tool=callgrind) +
kcachegrind/qcachegrind
▶ googletest

35. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Links I
libmc: https://github.com/douban/libmc
benchmark of libmc: https:
//gist.github.com/mckelvin/8ded053f6523931765d4
Improving performance of std::vector:
http://fahrenheit2539.blogspot.com/2012/06/
introduction-in-depths-look-at.html
How to profile C++ application with Callgrind / KCacheGrind:
http://baptiste-wicht.com/posts/2011/09/
profile-c-application-with-callgrind-kcachegrind.
html