Large Data
Largest Living creation:
1 California Redwood tree named the General Sherman
PROBLEM
Highly Interactive
2 Starlings forming fascinating formations over Tøndermarsken,
south-west Jutland, Denmark
PROBLEM
Huge user base
5 World’s largest gathering: Kumbh Mela 2007
PROBLEM
3
4
6 Brad Fitzpatrick, LiveJournal
SOUNDS FAMILIAR!
WHO ELSE HAS FACED THIS BEFORE?
Secondary Storage: Low access speed, high capacity
Processor: Small size, high speed
Network: Blazing fast
Primary Memory:...
Pre-Generate Static Pages
• Fast Read
• Dynamic context aware content
• Popularity prediction in advance
Web Server Farm
•...
MEMCACHED
Distributed object caching
Global distributed two layered object caching
MEMCACHED
Distributed object caching
Right granularity at right Layer
hashta...
Servers
1
2
4 3
6
5
10
11
7
8
Clients on application Servers Data Stores
12
9
Cross-Platform
Application namespace isolati...
Associative Array: Meta information
Runtime Cache: Single node applications
Database Cache: No large data, rarely updated,...
REFERENCES
1. http://www.mnn.com/sites/default/files/imagecache/node-gallery-display/general%20sherman.jpg
2. http://en.wi...
High scalability | Memcached - Distributed object caching
Upcoming SlideShare
Loading in …5
×

High scalability | Memcached - Distributed object caching

2,366 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,366
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • DB Replication Image Reference : http://technet.microsoft.com/en-us/library/ms152567.aspx
  • Storage keys evenly spread across servers, application uses Hash Table to determine which server to go to

    Two layers:
    1: Server where key is stored
    2: Actual Serialized object

    Since Caching: Collisions result in loss or wrong data being read : Application responsibility to avoid collision

    Blocking vs Non-Blocking
    Thread invoking an I/O function, like read value, does not have to wait on any previous operation before executing
    Objects have multiple versions and are reference counted

  • Server Instance
    Listens on a specified IP address and Port
    Multiple instances on same machine, where server total memory greater than the amount that the kernel makes available to a single process
    Client Instance
    Read: Hash(Object Key) if Memcached has return else fetch from DB and Put
    Write: No transactions, Pull from DB -> Update Object -> Save to DB -> Save to Cache


    Interdependence: Data lost but rest of the machines in farm function normally, subsequent requests can be routed accordingly
    Expiration: LRU

  • High scalability | Memcached - Distributed object caching

    1. 1. Large Data Largest Living creation: 1 California Redwood tree named the General Sherman PROBLEM
    2. 2. Highly Interactive 2 Starlings forming fascinating formations over Tøndermarsken, south-west Jutland, Denmark PROBLEM
    3. 3. Huge user base 5 World’s largest gathering: Kumbh Mela 2007 PROBLEM 3 4
    4. 4. 6 Brad Fitzpatrick, LiveJournal SOUNDS FAMILIAR! WHO ELSE HAS FACED THIS BEFORE?
    5. 5. Secondary Storage: Low access speed, high capacity Processor: Small size, high speed Network: Blazing fast Primary Memory: Fast, cheap Machine: Low reliability COMPUTING HARDWARE
    6. 6. Pre-Generate Static Pages • Fast Read • Dynamic context aware content • Popularity prediction in advance Web Server Farm • More concurrent web requests served • Dynamic Content delivery DB Replication • Data Replicas for Load Balancing • Fast Read • Slow Writes for ACID conformance ALTERNATIVES DB Sharding • Fast Read due to Horizontal partitioning • Limited scalability • Reliance on file system performance
    7. 7. MEMCACHED Distributed object caching
    8. 8. Global distributed two layered object caching MEMCACHED Distributed object caching Right granularity at right Layer hashtableon RAM Cross-Platform Application namespace isolation Network independence Caches Objects, not raw data Dynamic server cluster reconfiguration Compression No authentication Highly volatile
    9. 9. Servers 1 2 4 3 6 5 10 11 7 8 Clients on application Servers Data Stores 12 9 Cross-Platform Application namespace isolation Network independence Caches Objects, not raw data Dynamic server cluster reconfiguration Compression No authentication Highly volatile MEMCACHED Distributed object caching
    10. 10. Associative Array: Meta information Runtime Cache: Single node applications Database Cache: No large data, rarely updated, similar requests, object creation not costly Memcached: Large scale distributed applications with frequently updated data and involving complex objects File Cache: Long term, Large Objects DB Read: Small scale needs 8 Cheetah, fastest land animal: 120km/h 7,9 Slug, slowest animal: 0.03mph 8 9 P E R F O R M A N C E MEMCACHED Distributed object caching
    11. 11. REFERENCES 1. http://www.mnn.com/sites/default/files/imagecache/node-gallery-display/general%20sherman.jpg 2. http://en.wikipedia.org/wiki/File:Sort_sol_pdfnet.jpg 3. http://odeworld.wordpress.com/2007/05/04/size-matters/ 4. http://www.kumbhamela.net/ 5. http://en.wikipedia.org/wiki/List_of_largest_peaceful_gatherings_in_history 6. http://picasaweb.google.com/dolboeb/BradFitzInSF#5176116242455706722 7. http://lilomag.com/2010/07/29/the-7-worlds-slowest-animal/ 8. http://en.wikipedia.org/wiki/Cheetah 9. http://en.wikipedia.org/wiki/File:Slugs_1896.png Memcached: http://benrobb.com/wp-content/uploads/2009/01/memcached.pdf By: Jeremy Leishman, Ben Robison, Josh Taylor

    ×