Redis @ 6Wunderkinder
    - more than just a cache -
         Sebastian Kreutzberger, CTO
About 6Wunderkinder
•   Startup based in Berlin, 18 employees

•   VC-backed

•   ā€žWunderlistā€œ - free, multi-platform task app

    -   >600k registered users, >1,6 million downloads

    -   Android, iPhone, iPad, Web, Mac & Windows

    -   App of the Week in 69 Apple App Stores

•   ā€žWunderkitā€œ - productivity platform, currently
    in development, ā€žnext big thingā€œ
Redis @ Wunderlist
• Wunderlist backend based on LAMP
• No cache was used in the beginning
• After incredible growth in userbase the
  marketing & BI needed statistics
• Solution: Redis!
• Why? Persistent, fast & super-simple
Statistics with Redis
• Store 2 types of statistical data:
 - simple daily increments to count
     syncs, hits & registrations, etc.
 -   daily Sets of user-ids who are active on
     that day, important for marketing to see
     how many people are using Wunderlist
     for how many days
Example
<?php

     $cache = new Redis();
     /* .... connection stuff .... */

     // increment daily sync counter
     $key = "sync: " . date("Ymd"); // day-based key
     $cache->incr($key);


     // store current user in today’s active-users Set
     $key = "active: " . date("Ymd");
     $cache->sAdd($key, $user_id);


     // get all user-ids who were active yesterday
     $yesterday = strtotime("-1 days 00:00");
     $key 
        = "active: " . date("Ymd", $time);
     $users        = $cache->sMembers($key);

?>
More details ...
• PHPRedis as module for PHP is very fast!
• Wrote cache abstraction layer for possible
  replacement of Redis with Memcached
• Currently just do caching on user objects
• Tasks & lists change too often for caching
• Put a complete object into cache, not just a
  database row
Redis @ Wunderkit

• Wunderkit uses MongoDB + async jobs
• Redis is fundamental part of architecture
• Even more important than database
• Heavy usage of Redis Lists (a.k.a. queues)
Usage at Wunderkit

• Caching of nearly all objects
• Statistics
• Logging
• Job Queue
Logging with Redis
• Wrote own PHP Error Handler
• Extended it for own logging
• Store PHP errors & log in Redis List
• Keys are based upon day & log level
• Very simple retrieval
• High-performance & scalable
Job Queue I

• Queue is a simple Redis List
• Wrote a helper class for easy queue input
• Every app server can ā€žoutsourceā€œ
  processing by putting a job onto the queue
• Each job contains the following data:
  a URL to a worker server + parameters
  => very simple, light-weight & scalable!
Job Queue II

• Dozens of parallel PHP scripts running as
  daemon(!) listen for new jobs on the queue
• brPop($key, $time) listens for new jobs
• No concurrencies, Redis is single-threaded
• Job URL is optimistically called, no waiting
  for response, ā€žļ¬re & forgetā€œ
  => many jobs per second!
Example Daemon
<?php

     /* ... do crazy PHP daemon stuff here ... */

     $cache = new Redis();
     /* .... connection stuff .... */

     while (1) // infinite loop
     {
       $ret = $cache->brPop("queue", 30); // listen for 30 seconds
       if ($ret != false) // something new on the queue
       {
       
 fireAsyncRequest($ret["url"], $ret["params"]);
       }
     }

?>
Todos
• Replacing PHP daemon with Erlang
• Adding of message queue incl. callbacks
• Diving into Redis Sharding
• Putting even more stuff into Redis. Why?
  Because we love Redis!
Thanks!
• Contact:
  Twitter: @skreutzb
  E-Mail: sebastiank@6wunderkinder.com
  Web:   www.6wunderkinder.com


• P.S.: We are hiring!

Redis At 6Wunderkinder

  • 1.
    Redis @ 6Wunderkinder - more than just a cache - Sebastian Kreutzberger, CTO
  • 2.
    About 6Wunderkinder • Startup based in Berlin, 18 employees • VC-backed • ā€žWunderlistā€œ - free, multi-platform task app - >600k registered users, >1,6 million downloads - Android, iPhone, iPad, Web, Mac & Windows - App of the Week in 69 Apple App Stores • ā€žWunderkitā€œ - productivity platform, currently in development, ā€žnext big thingā€œ
  • 3.
    Redis @ Wunderlist •Wunderlist backend based on LAMP • No cache was used in the beginning • After incredible growth in userbase the marketing & BI needed statistics • Solution: Redis! • Why? Persistent, fast & super-simple
  • 4.
    Statistics with Redis •Store 2 types of statistical data: - simple daily increments to count syncs, hits & registrations, etc. - daily Sets of user-ids who are active on that day, important for marketing to see how many people are using Wunderlist for how many days
  • 5.
    Example <?php $cache = new Redis(); /* .... connection stuff .... */ // increment daily sync counter $key = "sync: " . date("Ymd"); // day-based key $cache->incr($key); // store current user in today’s active-users Set $key = "active: " . date("Ymd"); $cache->sAdd($key, $user_id); // get all user-ids who were active yesterday $yesterday = strtotime("-1 days 00:00"); $key = "active: " . date("Ymd", $time); $users = $cache->sMembers($key); ?>
  • 6.
    More details ... •PHPRedis as module for PHP is very fast! • Wrote cache abstraction layer for possible replacement of Redis with Memcached • Currently just do caching on user objects • Tasks & lists change too often for caching • Put a complete object into cache, not just a database row
  • 7.
    Redis @ Wunderkit •Wunderkit uses MongoDB + async jobs • Redis is fundamental part of architecture • Even more important than database • Heavy usage of Redis Lists (a.k.a. queues)
  • 8.
    Usage at Wunderkit •Caching of nearly all objects • Statistics • Logging • Job Queue
  • 9.
    Logging with Redis •Wrote own PHP Error Handler • Extended it for own logging • Store PHP errors & log in Redis List • Keys are based upon day & log level • Very simple retrieval • High-performance & scalable
  • 10.
    Job Queue I •Queue is a simple Redis List • Wrote a helper class for easy queue input • Every app server can ā€žoutsourceā€œ processing by putting a job onto the queue • Each job contains the following data: a URL to a worker server + parameters => very simple, light-weight & scalable!
  • 11.
    Job Queue II •Dozens of parallel PHP scripts running as daemon(!) listen for new jobs on the queue • brPop($key, $time) listens for new jobs • No concurrencies, Redis is single-threaded • Job URL is optimistically called, no waiting for response, ā€žļ¬re & forgetā€œ => many jobs per second!
  • 12.
    Example Daemon <?php /* ... do crazy PHP daemon stuff here ... */ $cache = new Redis(); /* .... connection stuff .... */ while (1) // infinite loop { $ret = $cache->brPop("queue", 30); // listen for 30 seconds if ($ret != false) // something new on the queue { fireAsyncRequest($ret["url"], $ret["params"]); } } ?>
  • 13.
    Todos • Replacing PHPdaemon with Erlang • Adding of message queue incl. callbacks • Diving into Redis Sharding • Putting even more stuff into Redis. Why? Because we love Redis!
  • 14.
    Thanks! • Contact: Twitter: @skreutzb E-Mail: sebastiank@6wunderkinder.com Web: www.6wunderkinder.com • P.S.: We are hiring!