Your SlideShare is downloading. ×
0
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Memcache basics on google app engine
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Memcache basics on google app engine

4,117

Published on

In this talk we will cover: …

In this talk we will cover:
1. What is Memcache?
2. Why do we need Memcache?
3. How to Use Memcache on Google App Engine?
4. Caveats And Solutions

Published in: Technology, News & Politics
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,117
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
39
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. App Engine Memcache Basics
  • 2. App Engine: Memcache Basics Who? Why? Ido Green Solutions Architect plus.google.com/greenido greenido.wordpress.com
  • 3. App Engine: Memcache Basics Topics We Cover in This Lesson ● What is Memcache? ● Why do we need Memcache? ● How to Use Memcache? ● Caveats And Solutions
  • 4. App Engine: Memcache Basics What Is Memcache?
  • 5. What is Memcache? What is Memcache? ● Memcache is an in-memory Key-Value Pairs data store ○ Put a value with a key ○ Get a value with a key value key "user001" : "John Doe" "user002" : "Larry Page" ● key or value can be anything that is serializable ● Memcache is a shared service accessed via App Engine APIs.
  • 6. What Is Memcache What Do We Use Memcache For? ● ● ● Caching In Front of Datastore ○ Cache entities for low-latency reads ○ Integrated into most ORM frameworks (ndb, Obectify ..) Caching for Read heavy operations ○ User authentication token and session data ○ APIs call or other computation results Semi-durable Shared state Cross App Instances ○ Sessions ○ Counters / Metrics ○ Application Configurations
  • 7. App Engine: Memcache Basics Why Do We Need Memcache?
  • 8. Why do we need Memcache? Why Do We Use Memcache? ● Improve Application Performance ● Reduce Application Cost
  • 9. Why do we need Memcache? How Fast Can Memcache be? And How Much? Datastore Query Latency Memcache Read Latency
  • 10. How to Use Memcache
  • 11. How to Use Memcache Memcache APIs ● Java ○ JCache APIs ○ GAE Low-Level Memcache APIs ○ Objectify for Datastore ● Python ○ google.appengine.api.memcache module ○ ndb for Datastore
  • 12. How to Use Memcache General Memcache Usage Pattern Coordinate data read with Datastore: ● Check if Memcache value exists ○ if it does, display/use cached value directly; otherwise ○ fetch the value from Datastore and write the value to Memcache Coordinate data write with Datastore: ● Update Memcache value ○ to handle race condition, leverage put if untouched/compare and set to detect race conditions ● Write the value to Datastore ○ optionally, leverage the task queue for background writes
  • 13. How to Use Memcache Using GAE Client Library (Java) import com.google.appengine.api.memcache; ... MemcacheService syncCache = MemcacheServiceFactory. getMemcacheService(); // Act on error as cache miss syncCache.setErrorHandler(ErrorHandlers. getConsistentLogAndContinue(Level.INFO)); value = (byte[]) syncCache.get(key); // read from cache if (value == null) { value = getDataFromDb(key); // fetch value from datastore syncCache.put(key, value); // write to cache // (key and value must be serializable) } ...
  • 14. How to Use Memcache Using GAE Client Library (Python) from google.appengine.api import memcache ... value = memcache.get(key) if value is None: value = get_value_from_db(key) if not memcache.add(key, value): logging.error('Memcache add failed.') ...
  • 15. How to Use Memcache Batch Operations ● getAll(), putAll(), deleteAll() A single read or write operation for multiple memcache entries Note ● Further improve Memcache performance by reducing network traffics ● Batch size < 32 MB
  • 16. How to Use Memcache Atomic Operations ● increment(key, delta), incrementAll(...), Provide atomic increment of numeric value(s) ● getIdentifiable(), putIfUntouched() in Java, or cas() in Python A mechanism to update a value consistently by concurrent requests Note ● Helps managing memcache data consistency in multi-instances/concurrent environment
  • 17. How to Use Memcache Some Other Features ● Asynchronous calls Provides a mechanism to make a non-blocking call for memcache operations ● Namespace Logically separates data layers for different application purposes (like multi tenancy) across many GAE services such as Datastore, Memcache, Task Queue etc...
  • 18. Caveats And Solutions
  • 19. Caveats And Solutions Caveat: Memcache Is Volatile Entries can be evicted anytime by various reasons: ● ● ● entry reaches expiration entry is evicted because memcache memory is full memcache server fails Tip ● It's important to handle cache-miss gracefully! ● Implements write-through logic by backing memcache with datastore in your application! ○ Psst... Objectify and ndb do it for you.
  • 20. Caveats And Solutions Caveat: Memcache Is Not Transactional Instance 1 reads $100 $100 $100 Instance 2 reads $100 $100 $100 Instance 2 deducts $20 $80 $80 Instance 1 deducts $30 $70 $70 Tip Using getIdentifiable() and putIfUntouched(...) for optimistic locking
  • 21. Caveats And Solutions Caveat: Memcache Is A Limited Resource My Application Does NOT Have Enough Memcache! Tips ● Only need to cache what is useful and necessary! ● Your application should function without memcache! ● Dedicated Memcache
  • 22. Caveats And Solutions Dedicated Memcache Enabling dedicated memcache from the admin console 1. Provides fixed cache capacity that is exclusive to your application 2. Billed by the GB per hour of cache size. Charged in 15 minute increments 3. Gives you additional control over cache size. More predictable performance Note: Whether shared or dedicated, Memcache is not a durable storage. Plan for your application to function without Memcache
  • 23. Discussion And Questions Key Takeaways 1. Memcache is supported natively in GAE. Take advantage of it to improve your GAE application performance. 2. Memcache supports open standard JCache API. Many advanced features are available by GAE Memcache APIs to suit your application's need i.e. Batch, Atomic, Asynchronous operations 3. Seamless integration with GAE Datastore in a few libraries like Python ndb and Java Objectify. 4. Read-frequently and write-rarely data is most suitable in combining with Memcache 5. Handle Memcache's volatility in your application 6. Use Memcache wisely, it is not an unlimited resource
  • 24. App Engine: Memcache Basics Thank you! Questions?

×