WiredTiger Backend for OpenLDAP

1. WiredTiger Backend for
OpenLDAP
Open Source Solution Technology Corporation
Tsukasa Hamano <hamano@osstech.co.jp>
LDAPCon 2015 Edinburgh November 2015
Open Source Solution Technology Corporation 1

2. WiredTiger Backend for OpenLDAP
About OSSTech
• ID Management leading company in
Japan.
• Storage Solution
• Open Source Contribution
Open Source Solution Technology Corporation 2

3. WiredTiger Backend for OpenLDAP
What’s back-wt
• New OpenLDAP Backend
• WiredTiger Database
Open Source Solution Technology Corporation 3

4. WiredTiger Backend for OpenLDAP
About WiredTiger
• Embedded database
• High performance
• High scalability
Open Source Solution Technology Corporation 4

5. WiredTiger Backend for OpenLDAP
Lock Free
• Hazard pointer
• Optimistic concurrency control
Open Source Solution Technology Corporation 5

6. WiredTiger Backend for OpenLDAP
Data Structure
Open Source Solution Technology Corporation 6

7. WiredTiger Backend for OpenLDAP
bdb_next_id()
§ ¤
int bdb_next_id( BackendDB *be, ID *out )
{
struct bdb_info *bdb=(struct bdb_info*)be->be_private;
ldap_pvt_thread_mutex_lock(&bdb->bi_lastid_mutex);
*out = ++bdb->bi_lastid;
ldap_pvt_thread_mutex_unlock(&bdb->bi_lastid_mutex);
return 0;
}
¦ ¥
Open Source Solution Technology Corporation 7

8. WiredTiger Backend for OpenLDAP
wt_next_id()
§ ¤
int wt_next_id(BackendDB *be, ID *out){
struct wt_info *wi = (struct wt_info *)be->be_private;
*out = __sync_add_and_fetch(&wi->wi_lastid, 1);
return 0;
}
¦ ¥
Open Source Solution Technology Corporation 8

9. WiredTiger Backend for OpenLDAP
fsync(2) is slow
Open Source Solution Technology Corporation 9

10. WiredTiger Backend for OpenLDAP
Durability Levels
1. in-memory txn log – fastest but no
durability
2. write txn log file, no sync
3. write txn log file, sync per every
commit
Open Source Solution Technology Corporation 10

11. WiredTiger Backend for OpenLDAP
New Benchmark Tool - lb
• SLAMD is dead
• Command line interface
• Written in Go
Open Source Solution Technology Corporation 11

12. WiredTiger Backend for OpenLDAP
Installation of lb
$ export GOPATH=~/go
$ go get github.com/hamano/lb
Open Source Solution Technology Corporation 12

13. WiredTiger Backend for OpenLDAP
Benchmark Environment
• 12 Core CPU
• No RAID Card
• SAS Disk
Open Source Solution Technology Corporation 13

14. WiredTiger Backend for OpenLDAP
ADD Benchmark Script
§ ¤
for c in 1 2 4 8 16 32 64 128 256 512; do
lb add -c $c -n 10000 --uuid ldap://targethost/
done
¦ ¥
Open Source Solution Technology Corporation 14

15. WiredTiger Backend for OpenLDAP
ADD (nosync) Benchmarks
0
5000
10000
15000
20000
25000
30000
1 2 4 8 16 32 64 128 256 512
Requestspersecond
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 15

16. WiredTiger Backend for OpenLDAP
ADD (sync) Benchmarks
0
1000
2000
3000
4000
5000
6000
1 2 4 8 16 32 64 128 256 512
Requestspersecond
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 16

17. WiredTiger Backend for OpenLDAP
BIND Benchmark Script
§ ¤
for c in 1 2 4 8 16 32 64 128 256 512; do
lb bind -c $c -n 100000
-D "cn=user%d,dc=example,dc=com" -w secret
--last 10000 ldap://targethost/
done
¦ ¥
Open Source Solution Technology Corporation 17

18. WiredTiger Backend for OpenLDAP
BIND Benchmark Result
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
1 2 4 8 16 32 64 128 256 512
Requestspersecond
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 18

19. WiredTiger Backend for OpenLDAP
SEARCH Benchmark Script
§ ¤
for c in 1 2 4 8 16 32 64 128 256 512; do
lb search -c $c -n 100000
-a "(cn=user%d)"
--last 10000 ldap://targethost/
done
¦ ¥
Open Source Solution Technology Corporation 19

20. WiredTiger Backend for OpenLDAP
SEARCH Benchmark Result
0
5000
10000
15000
20000
25000
30000
1 2 4 8 16 32 64 128 256 512
Requestspersecond
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 20

21. WiredTiger Backend for OpenLDAP
Tests
$ make -C tests wt
54
65 Succeed
Open Source Solution Technology Corporation 21

22. WiredTiger Backend for OpenLDAP
Tasks
• Hot-backup
• alias and glue entry
Open Source Solution Technology Corporation 22

23. WiredTiger Backend for OpenLDAP
Thank You!
Open Source Solution Technology Corporation 23