1. Building an Object
Storage for the Nordics
Oslo Riak Meetup, March 19, 2014

2. Redpill Linpro
180+ employees
Linpro founded 1995, Redpill 2003
Offices in Oslo, Stockholm, Karlstad,
Gothenburg, Stavanger and Copenhagen
Leading provider of professional Open Source
services and products in the Nordic region

3. In the beginning...
5.4 million
images
1.5 TiB
NFS on the
backend
Served via HTTP
to end users
Image store

4. key/value
access
Scalability
HTTP­based
Reliability
Cost
Why Object Storage?
Storage size

5. Multi­site
Multi tenancy
S3 compatible
Open Source
Professional Support Cost
What do we need?

6. Swift & HDFS
Missing support for multi­site removed these
quickly from our list
OpenStack Swift announced multi­site support
later that year...
(HDFS)

7. Atmos & Cloudian
EMC Atmos
Impressing hardware
Support
60 TB entry level
Cost is accordingly
And: black box
Extensive S3
compatibility
Free up to 100TB
Based on Cassandra
and Tomcat

8. Ceph
Looks promising from the outset
A closer look reveals quite a few rough edges,
making us expect some work to get started
License is a plus

9. Riak CS
Good impression after first lab test
Got a lot of support from Basho during
evaluation
And Basho released Riak CS as Open Source
during our eval period... :)

10. Let's buy some servers...
HP DL380e 12LFF, each with
SmartArray Controller
2x 4TB disk
16 GB RAM
Gigabit NIC

11. ...and set them up

12. Riak configuration
Remember to set ring_creation_size to "a few
times the amount of nodes in the cluster", so
you don't have to worry about adding nodes
later
Each node owns ring_creation_size / <number
of nodes> partitions

13. riak­admin
Handy tool – especially during initial
configuration and tuning
This is how it looks like today :­)
$ riak­admin diag
15:34:52.736 [warning] Riak crashed at Wed, 07 Dec 2011 21:47:50 GMT, leaving
crash dump in /srv/riak/log/erl_crash.dump. Please inspect or remove the file.
15:34:52.736 [notice] Data directory /srv/riak/data/bitcask is not mounted with
'noatime'. Please remount its disk with the 'noatime' flag to improve performance.
$
[root@riak01­osl3 ~]# riak­admin diag
[root@riak01­osl3 ~]#

14. Riak CS config
{fold_objects_for_list_keys, true}
Gives significant performance increase
(Riak 1.4+)
{connection_pools, [ {request_pool, {N,N} },
{bucket_list_pool, {N,N} } ]}
Increase the size of your connection pools, so
they don't fill up!

15. Riak CS config
Only one of the Riak CS­nodes in each cluster
need to do Garbage Collection, so you can set
{gc_interval, 0} on all other nodes.

16. Upgrading
Stop Riak on one node
Backup configuration
Upgrade Riak
Start Riak
Wait for synchronization...
Move on to next node, repeat from step 1

17. NOK 0.165
per GiB*
* per GiB per month gross storage used
Situla

18. Questions?
@mnemecky
michael.nemecky@redpill­linpro.com