In today’s systems , the time it takes to bring data to the end-user can be very long, especially under heavy load. An application can often increase performance by using an appropriate caching system. There are many caching level that you can use in our application today : CDN, In-Memory/Local Cache, Distributed Cache, Outut Cache, Browser Cache, Html Cache
4. Caching
introduction
A cache transparently stores data “somewhere” so that
future requests can be served faster
If requested data is in the cache (cache hit), this
request can be served by simply reading the cache,
which is comparatively faster.
Otherwise (cache miss), the data has to be recomputed
or fetched from its original storage location, which is
comparatively slower
4
5. Caching
why ? how ? when ? who ?
• Why ?
–
–
–
–
Better performance
Better scalability
Better robustness
Reduce costs
system loads faster/better responsiveness
limit bottlenecks in a system
can support more load
reduce round trips, servers and hardware
• How ?
– Many APIs/Frameworks available to meet all of your needs
– Many types of caching
always easy to use
“think cache” !
• When ?
– Not too late
“Designed with cache” is better than “Designed for being cached”
• Who ?
– You (The Devs)
you will never have a business request to use caching
Do you remember « Quality of Service » ? Caching is typically implicit and
you will have to put it in your projects by yourself
5
6. Caching
short quiz
• Does Caching always improve performance ?
No : even if performance is the target, an unhealthy cache will hurt performance.
« Cache everything » is also counterproductive.
• Is Caching small data items useless ?
No : ideal caching candidates are items frequently accessed, long to compute or
that change infrequently.
• Is Caching mandatory for web apps ?
Yes : because of high traffic, dynamic page content, … and because our business
involves frequently updates (lives, odds, …)
• Is Caching only for data access layer ?
No : you can cache everything. Caching raw data is only a small part of the job.
• Caching == indexes ?
No : Indexes are a way to get data faster ; cache is a copy of data (stale)
6
7. Caching
terminology
•
•
•
•
•
•
•
•
•
•
•
•
Cache Hit : when requested data is contained in the cache
Cache Miss : when requested data in not in the cache.Has to be recomputed or fetched
from its original storage
Cache Key : unique identifier for a data item in the cache
Expiration
– Absolute : item expires at a specific date, regardless of how often it is accessed
– Sliding : specifies how long after an item was last accessed that it expires
Backing store : persist cached data on disk
Cache Scavenging : deleting items from the cache when memory is scarce
MRU/LRU/LFU : algorithms for removing objects from the cache when scavenging
Memoization : avoid repeating the calculation of results for previously processed inputs
Cache Dependancy : item’s lifetime in the cache to be dependent on other application
elements such as files or databases
Local Cache : caching data on clients rather than on servers
Distributed Cache : extension of the traditional concept of cache that may span multiple
servers
…
7
8. Caching in Web Applications
in the next slides
• CDN (forward proxy)
– Serve content to end-users with high availability and high
performance (Images, Scripts, Medias, …)
• User-scoped caching (Session)
– allows us to store data that persists between multiple
requests on a per-user basis
•
•
•
•
Memory cache
Browser cache
HTML5 cache features
ASP.NET caching techniques
8
9. System.Runtime.Caching.MemoryCache
who does not know this cache, does not know very well BetClic
•
•
•
•
Previously called asp.net cache or Web Cache
Built-in, supported and recommended by Microsoft
Basically a key-value store (dictionary) that lives
throughout the lifetime of the application but
ASP.NET automatically manages removal of cached
items (Expiration, Eviction, Dependancy, …)
Be aware of double-checked locking (see below)
9
10. Browser Cache
part of HTTP protocol
Resources that are cached locally by the browser are controlled by
three basic mechanisms, defined by HTTP Headers.
• Freshness allows a response to be used without re-checking it on the
origin server, and can be controlled by both the server and the client.
–
–
•
Validation is used to check whether a cached response is still good after it
becomes stale.
–
–
•
“Expires” response header gives a date when the document becomes stale
“Cache-Control: max-age” tells the cache how many seconds the response is
fresh for.
For example, if the response has a ”Last-Modified” header, a cache can make a
conditional request using the ”If-Modified-Since” header to see if it has changed.
The ”Etag” mechanism allows for both strong validation rather than the IfModified-Since header which is often referred as weak validation.
Invalidation is usually a side effect of another request that passes through
the cache. For example, if URL associated with a cached response
subsequently gets a POST, PUT or DELETE request, the cached response will
be invalidated.
What’s the difference between a response 200 (from cache) and 304 (Not Modified) ?
10
11. Output Cache
an efficient cache for dynamic content
• Caching the HTML that is generated as a result of a request
• Simple add an attribute to a Controller or an action
• Able to cache multiple versions of the same controller action based on the
request parameters used to call the action
• By default content is cached in three locations: the web server, any proxy
servers, and the user’s web browser, but you can have fine-grained control
Warning : do not add output cache to something which is bound to user
like session
11
12. Output Cache
Donut Caching Vs Donut Hole Caching
• Donut caching
Server-side caching technique in which the entire page gets cached except for
a small portions which remain dynamic. It’s not a native feature of MVC (why
?) but there are Nuget Packages.
• Donut Hole Caching
Donut hole caching is where you cache one or more parts of a page, but not
the entire page. It is handled by using the built-in OutputCache attribute on
one or more child actions (called using Html.Action or Html.RenderAction from
the parent view)
12
13. HTML 5 Web Storage
Local Storage /Session Storage
• A "super cookie“ that allow us to persist
data (up to 5 MB) in the browser
(shared between tabs and keep after
restart)
• Only strings (thanks to JSON)
• JS API to store, get, remove, clear, get
remaining space, …
• Many frameworks available such as
http://www.jstorage.info/ ( Caching &
TTL Support for all browsers)
• Used at betClic for Live/Multiplex (Keep
static translations)
13
14. HTML Cache Manifest
for offline web applications
•
HTML5 feature to access a web application even without a
network connection
–
–
•
Cache Manifiest File : Allows a dev to specify which files the
browser should cache and make available to offline users.
–
–
–
•
Avoid the classic « 404 Not Found » Page, but not only …
In the Top 5 supported HTML5 features in mobiles devices
CACHE files will be explicitly cached after they're downloaded
NETWORK files are white-listed resources that require a
connection ; resources bypass the cache,
FALLBACK fallback file if a resource is inaccessible
Also some APIs to check the cache state and switch to online
too (work offline)
14
15. Caching for Information Systems and Web Farms
On the Road to another type of cache
All previous techniques are good but suffer from majors inconvenients :
– Cached Data reside in the same process/server
If we have 30 front end servers, they will have to set up their own cache.
– No High availability
If the web site is restarted/the pool recycled, all cached items are lost.
– Cached Data is limited to the server capacity
If all cached data become important, we need additional memory on the server
– Does not fit very well in WebFarms/Information Systems
imagine a company with databases in Gib and frontend server in Paris …
Couldn't we find a way to solve all these problems?
Distributed Cache
15
16. Distributed Cache
introduction
Quite recent in web applications because
•
•
Memory is now very cheap and network cards are very fast
Works well on lower cost machines usually employed for web servers as opposed
to database servers which require expensive hardware
Currently two main approaches
•
•
Use a real distributed cache
Use a NoSql Database
Memcached
16
17. Typical Web Architecture
without distributed cache
Users
• Need to route users to same
machine (i.e. sticky sessions)
Web Tier
• Each machine round trips for data
• Some data might be expensive to
retrieve
• Cached data is typically stored in
the memory of one (each) server
Database
• CPU and disk can get saturated due
with an increase traffic
17
18. Scalable Web Architecture
with distributed cache
Users
Web Tier
Caching
Tier
Database
• No stick load balancing needed – all
servers have copy of cached data
• Easy access to cache cluster
• Multiple machines means scale and
potential for high-availability
• More machines == more memory for
cache objects
• Reduces load on database
18
19. Windows Server AppFabric
an example of Distributed Caching
•
•
•
A Distributed in-memory cache for “data”
.Net Client Api (Nuget Package)
Two Patterns :
– Cache Aside
– Read-Through and Write-Behind
•
Main Features
–
–
–
–
–
–
–
–
–
–
Logical Containers (Cache/Region)
Local Cache
Expiration/Eviction
Notifications
Secure API
High Availability
Concurrency Model
Tags
Monitoring API
…
//create DataCacheFactory based on config file
var dcf = new DataCacheFactory();
//get the cache named "TestCache"
var cache = dcf.GetCache(“MyCache");
//Add an item called "Test" - throws if exists
cache.Add("Test", new Data { TheData = "Test" });
//Get "Test" - add if not in cache (cache-aside)
var test = cache.Get("Test") as Data;
if (test == null)
{
test = new Data {TheData = "Test" };
cache.Add("Test", test);
}
19
20. Windows Service AppFabric Caching
Bonuses
•
Additional ASPNET Providers:
– Session State Provider
– Output Cache Provider
20
21. Caching challenges
remember this !
• Adding caching to a system is always easy
• Items in the cache may can become out-of-date or stale
– Find to best expiration duration
• Be careful of multiple cache level
– Db Cache + Service Cache + API Cache + Browser Cache = ???
• Be careful of Cache Health
– Always monitor your cache
• Be careful of Cache Context
– Do not include User-Specific data
• Find the best granularity for your usage
– Large list Vs many small items ?
• …
21
25. About Betclic
•
•
•
Betclic Everest Group, one of the world leaders in online gaming, has a unique portfolio
comprising various complementary international brands: Betclic, Everest Gaming, bet-athome.com, Expekt…
Active in 100 countries with more than 12 million customers worldwide, the Group is
committed to promoting secure and responsible gaming and is a member of several
international professional associations including the EGBA (European Gaming and Betting
Association) and the ESSA (European Sports Security Association).
Through our brands, Betclic Everest Group places expertise, technological know-how and
security at the heart of our strategy to deliver an on-line gaming offer attuned to the passion
of our players.