Tarantool/Box: a use case withserving 2 billion queries a day        November 1st, 2012, Tallinn, Estonia            Konst...
AgendaTarantool/Box: features and architectureCase study 1: advanced memcacheCase study 2: a message queue serverCase stud...
Feature overview●    Flexible data model, primary and secondary keys●    Fully cached: 100% of data is cached in RAM●    P...
Data model●    A game of fields, tuples and spaces●    HASH, TREE, BITMAP, partial, indexes●    Secondary keys●    Single-...
Server architecture
Lua API                  Redis                 Tarantool (Lua)redis.set(key, value)       box.insert(space, key, value)red...
Performance overviewIntel I5 , 4G RAM, 7200 RPM SATA10 threads, 200-300 bytes per tupleTarantool 1.4.6: 170k writes,260k r...
Memory footprint
Raw GET/SET performance
Use case 1: flexible memcache●    You can create your own fibers●    box.fiber.create(), box.fiber.yield()●    A backgroun...
Use case 2: message queue● Reliable queues are a vital ingredient for buildingscalable applications●  In Web Apps queues a...
Message queues: how●    Rich Lua stored procedure environment      ―   box.fiber, box.space, box.cfg●    Specialized inter...
Use case 3: reliable database●    User profile: 500 bytes of key/value pairs●    Math: 200M users * 500 = 100GB●    At lea...
Example: a FIFOfunction fifo_push(name, val)    fifo = find_or_create_fifo(name)    top = box.unpack(i, fifo[1])    bottom...
Conclusion●    Tarantool/Box – a high performance data engine● Tarantool/Lua – a building block for your heavily loadedweb...
Whats cooking in 1.5●    Disk-based backend●    Synchronous master-master replication●    New data types (array, date, cur...
Thank you!
Linkshttp://github.com/mailru/tarantool - source codehttp://github.com/mailru/tntlua - open source stored proceduresreposi...
Upcoming SlideShare
Loading in...5
×

My talk at Topconf.com conference, Tallinn, 1st of November 2012

254

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
254
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

My talk at Topconf.com conference, Tallinn, 1st of November 2012

  1. 1. Tarantool/Box: a use case withserving 2 billion queries a day November 1st, 2012, Tallinn, Estonia Konstantin Osipov, Mail.Ru kostja@tarantool.org
  2. 2. AgendaTarantool/Box: features and architectureCase study 1: advanced memcacheCase study 2: a message queue serverCase study 3: a reliable in-memory databaseProject future
  3. 3. Feature overview● Flexible data model, primary and secondary keys● Fully cached: 100% of data is cached in RAM● Persistent using a Write Ahead Log● Log shipping replication and online backup● extensible with Lua
  4. 4. Data model● A game of fields, tuples and spaces● HASH, TREE, BITMAP, partial, indexes● Secondary keys● Single-part, multi-part● STRING, NUM and NUM64 data types
  5. 5. Server architecture
  6. 6. Lua API Redis Tarantool (Lua)redis.set(key, value) box.insert(space, key, value)redis.get(key) box.select(space, 0, key, value)redis.getset(key, newkey) box.update(space, key, =p, 0, newkey)redis.incr(key) box.update(space, key, +p, 1, 1)redis.lpush(key, value) box.update(space, key, !p, 1, value)redis.rpush(key, value) You guess it...
  7. 7. Performance overviewIntel I5 , 4G RAM, 7200 RPM SATA10 threads, 200-300 bytes per tupleTarantool 1.4.6: 170k writes,260k reads
  8. 8. Memory footprint
  9. 9. Raw GET/SET performance
  10. 10. Use case 1: flexible memcache● You can create your own fibers● box.fiber.create(), box.fiber.yield()● A background fiber performs a customized expire● Session store: 20M online users, 200M monthly users,4 2-CPU units, 96GB RAM each● 60K RPS, CPU usage is below 20%
  11. 11. Use case 2: message queue● Reliable queues are a vital ingredient for buildingscalable applications● In Web Apps queues are used for delayed processing,load balancing, e-mail notifications● Our use case: prefetching avatars
  12. 12. Message queues: how● Rich Lua stored procedure environment ― box.fiber, box.space, box.cfg● Specialized inter-procedure communication API: ― box.ipc● Specialized data structures ― Bitmaps and partial keys
  13. 13. Use case 3: reliable database● User profile: 500 bytes of key/value pairs● Math: 200M users * 500 = 100GB● At least 2x smaller memory footprint than in a RDBMS● Win: predictable response time, significantly higherperformance
  14. 14. Example: a FIFOfunction fifo_push(name, val) fifo = find_or_create_fifo(name) top = box.unpack(i, fifo[1]) bottom = box.unpack(i, fifo[2]) if top == fifomax+2 then -- % size top = 3 … end return box.update(0, name, =p=p=p, 1, top, 2, bottom, top, val)end
  15. 15. Conclusion● Tarantool/Box – a high performance data engine● Tarantool/Lua – a building block for your heavily loadedweb applications● Tarantool – our approach to an easy to use databasefor highly volatile data
  16. 16. Whats cooking in 1.5● Disk-based backend● Synchronous master-master replication● New data types (array, date, currency, json)● authentication
  17. 17. Thank you!
  18. 18. Linkshttp://github.com/mailru/tarantool - source codehttp://github.com/mailru/tntlua - open source stored proceduresrepositoryhttp://groups.google.com/group/tarantool - mailing listhttp://tarantool.org/dist/ - always fresh .tar.gz and .rpm
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×