TurboGears is a full-stack Python web framework that can be slowed down by its many features. The document provides tips to optimize TurboGears applications, such as only using necessary features, avoiding static file serving, leveraging caching strategically, and offloading work asynchronously. It also stresses that understanding how users interact with an application is key to optimization.
From nothing to a video under 2 seconds / Mikhail Sychev (YouTube)Ontico
What does it take to achieve sub two seconds video playback latency on the 3rd largest website in the world?
We will peek under the hood of the Watch page and explore what common problems are being solved by
YouTube's Desktop team and what interesting solutions had to be implemented to achieve this goal.
We will discuss how page loads are classified and what specific treatment is required for every type, what tools and technologies are used in the stack, how being one of the largest image serving websites affects our approach to thumbnails and how we maintain and monitor our latency goals.
Large websites with large customer bases should have fast page loads no matter where your customers are coming from. In this day and age speed is expected. Getting there requires engineers to both have data and the ability to analyze and find problems.
This talk will address page load speed in two parts. A "cold" load where a user first comes to your site and a "warm" load which deals with intra-site page load speed. We will dive into the details of each page load and what is really going on. From network optimization to browser render performance, all things matter when it comes to optimizing the load of your web page. Furthermore, we will look into some tools that can be used to analyze and help developers discover and address problems.
The 5 most common reasons for a slow WordPress site and how to fix them – ext...Otto Kekäläinen
Presentation given in WP Meetup in October 2019.
Includes fresh new tips from summer/fall 2019!
A Must read for all WordPress site owners and developers.
From nothing to a video under 2 seconds / Mikhail Sychev (YouTube)Ontico
What does it take to achieve sub two seconds video playback latency on the 3rd largest website in the world?
We will peek under the hood of the Watch page and explore what common problems are being solved by
YouTube's Desktop team and what interesting solutions had to be implemented to achieve this goal.
We will discuss how page loads are classified and what specific treatment is required for every type, what tools and technologies are used in the stack, how being one of the largest image serving websites affects our approach to thumbnails and how we maintain and monitor our latency goals.
Large websites with large customer bases should have fast page loads no matter where your customers are coming from. In this day and age speed is expected. Getting there requires engineers to both have data and the ability to analyze and find problems.
This talk will address page load speed in two parts. A "cold" load where a user first comes to your site and a "warm" load which deals with intra-site page load speed. We will dive into the details of each page load and what is really going on. From network optimization to browser render performance, all things matter when it comes to optimizing the load of your web page. Furthermore, we will look into some tools that can be used to analyze and help developers discover and address problems.
The 5 most common reasons for a slow WordPress site and how to fix them – ext...Otto Kekäläinen
Presentation given in WP Meetup in October 2019.
Includes fresh new tips from summer/fall 2019!
A Must read for all WordPress site owners and developers.
Presentation from webinar held on August 19th, 2020, with Xdebug developer Derick Rethans and Seravo CEO Otto Kekäläinen.
This presentation shows you how to use Xdebug (which is very easy with the Seravo WordPress development environment!) to make a record of what the WordPress PHP code does during a website page load, and how to use that information to optimize the performance of your WordPress site.
For a video recording and Dericks presentation see https://seravo.com/blog/webinar-xdebug-profile-php/
It's no denying that rich Javascript applications (sometimes called One Page Applications) are a big thing, but what if you want to leverage Drupal on the backend, or have an existing site? Tools like Angular.JS and EmberJS are great when you have an API, but Drupal 7 doesn't really have an API layer. I'll explore the parts of a one page application and how to integrate it into either an existing or a new Drupal site, and the pitfalls that one must watch out for.
This session was organized today at Zebpay office in Mumbai along with team joining from Ahmedabad.
I discussed the basic core principals of js and slowly moved towards the advance concepts, code examples of closures and NodeJS.
We also looked at JS runtime visualization using Loupe.
The team was very excited and interactive. They loved it and supported for future such discussions.
Enjoyed the evening today with the team!
How to investigate and recover from a security breach in WordPressOtto Kekäläinen
Talk given at the first ever WordCamp Nordic on March 8th, 2019.
How to investigate and recover from a security breach – real-life experiences with WordPress
Sometimes the bad guys get in, despite all the protections and precautions. If that happens, there are many techniques that can be used to stop further damage, track down what the intruder did and how they got in. Finally the site needs to be cleaned up and re-opened for visitors. In this talk the most important techniques are presented along with real-life examples when they were used.
In recent years, a number of features have appeared on the web platform that allow us to provide better user experiences, largely through doing things more efficiently rather than inventing completely new patterns. In this talk, Mozilla’s Chris Mills explores a few of these features — such as Streams, Service workers and PWAs — and why they are worth knowing about as we move towards the future.
Lets look at an example of what a performant website can look like. This discuss what concepts should we be considering when looking at website performance. Next we will go over two areas pertaining to website performance: 1) website performance tweaks that you as a web developer can directly make 2) website performance tweaks that you may have to work with your hosting provider or IT department to achieve
Ad109 - XPages Performance and Scalabilityddrschiw
Understanding the XPages architecture is key to building performant scalable enterprise-ready Lotus Domino web applications. We'll show how to go under the hood to discover functional features that help your application perform and scale well. You'll learn about design patterns and techniques that ensure your applications are optimally tuned for your business requirements, and we'll show how to integrate existing business logic -- without increasing performance cost.
Presentation from webinar held on August 19th, 2020, with Xdebug developer Derick Rethans and Seravo CEO Otto Kekäläinen.
This presentation shows you how to use Xdebug (which is very easy with the Seravo WordPress development environment!) to make a record of what the WordPress PHP code does during a website page load, and how to use that information to optimize the performance of your WordPress site.
For a video recording and Dericks presentation see https://seravo.com/blog/webinar-xdebug-profile-php/
It's no denying that rich Javascript applications (sometimes called One Page Applications) are a big thing, but what if you want to leverage Drupal on the backend, or have an existing site? Tools like Angular.JS and EmberJS are great when you have an API, but Drupal 7 doesn't really have an API layer. I'll explore the parts of a one page application and how to integrate it into either an existing or a new Drupal site, and the pitfalls that one must watch out for.
This session was organized today at Zebpay office in Mumbai along with team joining from Ahmedabad.
I discussed the basic core principals of js and slowly moved towards the advance concepts, code examples of closures and NodeJS.
We also looked at JS runtime visualization using Loupe.
The team was very excited and interactive. They loved it and supported for future such discussions.
Enjoyed the evening today with the team!
How to investigate and recover from a security breach in WordPressOtto Kekäläinen
Talk given at the first ever WordCamp Nordic on March 8th, 2019.
How to investigate and recover from a security breach – real-life experiences with WordPress
Sometimes the bad guys get in, despite all the protections and precautions. If that happens, there are many techniques that can be used to stop further damage, track down what the intruder did and how they got in. Finally the site needs to be cleaned up and re-opened for visitors. In this talk the most important techniques are presented along with real-life examples when they were used.
In recent years, a number of features have appeared on the web platform that allow us to provide better user experiences, largely through doing things more efficiently rather than inventing completely new patterns. In this talk, Mozilla’s Chris Mills explores a few of these features — such as Streams, Service workers and PWAs — and why they are worth knowing about as we move towards the future.
Lets look at an example of what a performant website can look like. This discuss what concepts should we be considering when looking at website performance. Next we will go over two areas pertaining to website performance: 1) website performance tweaks that you as a web developer can directly make 2) website performance tweaks that you may have to work with your hosting provider or IT department to achieve
Ad109 - XPages Performance and Scalabilityddrschiw
Understanding the XPages architecture is key to building performant scalable enterprise-ready Lotus Domino web applications. We'll show how to go under the hood to discover functional features that help your application perform and scale well. You'll learn about design patterns and techniques that ensure your applications are optimally tuned for your business requirements, and we'll show how to integrate existing business logic -- without increasing performance cost.
Basics of Web App Systems Architecture
General Web Software Optimization Strategies
Defining a Goal for Performance
Performance Metrics, tools
Performance Debugging Techniques
What Can You Control?
What Is Caching?
Drupal Performance modules
Optimizing Drupal
Developing high performance and responsive web apps using web workerSuresh Patidar
A brief talk on web application performance and responsiveness. Why it is important and how web worker can help achieve it. It also covers a simple example implementing web workers.
PyConUK 2014 - PostMortem Debugging and Web Development UpdatedAlessandro Molina
Developers tend to ignore that users can be more creative than them. Use their debugging skills for your own benefit: post-mortem debugging is one of the most important features your web framework can provide.
This talk will cover some of the simplest practices and available tools for debugging on production environments and to immediately improve quality of your web applications.
JavascriptMVC: Another choice of web frameworkAlive Kuo
JavascriptMVC is another client side web MVC framework based on jQuery. It has totally solution to build a web application. This slide will introduce basic features of JavascriptMVC3.2
In this session we will present an overview from the point of view 'system that implementative on how to get the best performance from your drupal application.
We will also show examples of use cases for drupal scalable infrastructure.
AD113 Speed Up Your Applications w/ Nginx and PageSpeededm00se
My slide deck from my session, AD113: Speed Up Your Applications with Nginx + PageSpeed, at MWLUG 2015 in Atlanta, GA at the Ritz-Carlton.
For more, see:
- https://edm00se.io/self-promotion/mwlug-ad113-success
- https://github.com/edm00se/AD113-Speed-Up-Your-Apps-with-Nginx-and-PageSpeed
As cloud adoption has grown more rapidly in the last decade , how DBA's a can add more value to system and bring in more scalability to the DB server. This talk was presented at Open Source India 2018 conference by Kabilesh and Manosh of Mydbops. They share a few experience and value addition made to customers during their consulting process.
Apache Arrow has set itself as the standard data interchange format between processes and systems dealing with high amounts of data in the data engineering field. PyArrow is the Python implementation of Apache Arrow and it provides access to all Apache Arrow features.
Arrow has evolved as a on-disk storage format, an in-memory format, a format to exchange data over network and a compute engine. This allows it to be used as a full-stack solution in the data engineering world, and we will see how we can combine those capabilities to build a fully functional data engineering solution that is incredibly effective and fast.
ROOM
PyconIE 2016 - Kajiki, the fast and validated template engine your were looki...Alessandro Molina
Kajiki si an XML/HTML template engine born as a replacement for Genshi with specific improvements for speed and an easier inheritance system based on blocks.
EP2016 - Moving Away From Nodejs To A Pure Python Solution For AssetsAlessandro Molina
Presents the DukPY project ( https://github.com/amol-/dukpy ) which aims at making possible to run Javascript code and perform common assets transformation steps through the WebAssets framework
EuroPython 2015 - Storing files for the web is not as straightforward as you ...Alessandro Molina
DEPOT ( http://depot.readthedocs.org/en/latest/ ) is a file storage framework born from the experience on a project that saved a lot of files on disk, until the day it went online and the customer system engineering team decided to switch to Heroku, which doesn’t support storing files on disk.
The talk will cover the facets of a feature “saving files” which has always been considered straightforward but that can become complex in the era of cloud deployment and when infrastructure migration happens.
After exposing the major drawbacks and issues that big projects might face on short and long terms with file storage the talk will introduce DEPOT and how it tried to solve most of the issues while providing a super-easy-to-use interface for developers. We will see how to use DEPOT to provide attachments on SQLAlchemy or MongoDB and how to handle problems like migration to a different storage backend and long term evolution.
Like SQLAlchemy makes possible to switch your storage on the fly without touching code, DEPOT aims at making so possible for files and even use multiple different storages together.
Beaker is probably the most widespread cross-framework solution to manage sessions and caching in the python web ecosystem.
Born in 2005 from Pylons author has been historically maintained together with the Pylons framework. Since Pylons has been deprecated in favour of Pyramid, the Pylons Project team decided to write a custom session management solution for Pyramid and let the user handle more advanced backends.
Since 2015 beaker maintenance has been passed to the TurboGears project who ported it to a fully native Python3 solution and it’s stille the de-facto standard for Sessions and Caching on Bottle.
The talk will cover the advantages and drawbacks of the architectural decisions behind the 10 years of history of Beaker, the drawbacks we discovered while porting it to Python3 and which have been the “wrong choices” and how we plan to solve them and make it shine again as the best whole-around solution for Sessions and Caching on WSGI.
PyConFR 2014 - DEPOT, Story of a file.write() gone wrongAlessandro Molina
DEPOT ( http://depot.readthedocs.org/en/latest/ ) is a file storage framework born from the experience on a project that saved a lot of files on disk, until the day it went online and the customer system engineering team diceded to switch to Heroku, which doesn't support storing files on disk.
Like SQLAlchemy makes possible to switch your storage on the fly without touching code, DEPOT aims at making so possible for files and even use multiple different storages together.
Developers tend to ignore that users can be more creative than them. Use their debugging skills for your own benefit: post-mortem debugging is one of the most important features your web framework can provide.
This talk will cover some of the simplest practices and available tools for debugging on production environments and to immediately improve quality of your web applications.
MongoTorino 2013 - BSON Mad Science for fun and profitAlessandro Molina
The talk will cover how to use BSON directly as an exchange protocol to gain speed and advanced types.
BSON is the underlying serialization protocol used by MongoDB to store and represent data.
Whenever we retrieve data from MongoDB we get it as BSON, then our drivers decode it just so that our web service can encode it back in JSON.
We will see how to take advantage of BSON for fun and speed skipping this double step by directly fetching BSON and decoding it at client side.
PyConUK2013 - Validated documents on MongoDB with MingAlessandro Molina
Ming is a SQLAlchemy-inspired object-document mapper (ODM) for MongoDB developed at SourceForge which is also used by the TurboGears2 web framework to provide mongodb support.
After a short introduction to the basic Ming layer we will cover the Ming Object Document Mapper layer to show how to take advantage of its Unit Of Work to avoid performing incomplete changes and achieve relations between collections.
The last part of the talk will show how to use Ming to perform lazy migration of data when your schema changes and how to drop below the ODM layer to achieve maximum speed.
EuroPython 2013 - FAST, DOCUMENTED AND RELIABLE JSON BASED WEBSERVICES WITH P...Alessandro Molina
Web development is more and more moving to rich JavaScript and mobile applications, building a fast and reliable API server has become a core foundation in such situations.
The new 2.3 version of TurboGears2 is aiming at providing a great toolkit on both Python3 and Python2 for such cases thanks to builtin support for:
New minimal mode for small single file applications.
Gevent based deploy on Mozilla Circus using the new GearBox toolkit
The TGJSonAutodoc sphinx extension for automatically documenting JSON based API
Out of the box support for MongoDB using the Ming ODM
The DebugBar profiler and query analyzer for MongoDB
Those tools can make really easy to quickly prototype a fully working and documented web service, greatly improving a developer life and quality of the services. Even deployment can become a single line command thanks to the GearBox toolkit integration with Mozilla Circus while the jsoncall sphinx extension can really speed up service documentation thanks to its tgjsonautodoc directive.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
2. About me
● Python Developer since 2001
● Work @ on Python and iOS
● development team member since 2.1
3. What's this talk about?
● Some general rules which apply to any framework
● Some quick-wins for TurboGears
● Some real cases reported
● My personal experiences and preferences, feel free
to disagree!
4. Raw Speed
● People seem obsessed with webservers
● The truth is that it doesn't matter so much
○ You are not going to serve "Hello World"s
(If you are asking my personal stack is
nginx+mod_wsgi or nginx+Circus-Chaussette-gevent)
● Avoid the great idea of serving mostly empty pages
performing hundreds of ajax requests
○ Browser have limited concurrency
○ HTTP has overhead
○ You will actually slow down things
● Learn your framework for real
5. About TurboGears
● Framework for rapid development encouraging
flexibility
● Created in 2005, 2.0 was a major rewrite in 2009 to
embrace the WSGI standard.
● Object Dispatch based. Regular expressions can get
messy, write them only when you must.
● By default an XML template engine with error detection
● Declarative Models with transactional unit of work
● Built in Validation, Authentication, Authorization,
Caching, Sessions, Migrations, MongoDB Support and
many more.
6. Looking at the code
class RootController(BaseController):
@expose('myproj.templates.movie')
@expose('json')
@validate({'movie':SQLAEntityConverter(model.Movie)}
def movie(self, movie, **kw):
return dict(movie=movie, user=request.identity and request.identity['user'])
Serving /movie/3 as a webpage and /movie/3.json as a
json encoded response
8. Features vs Speed
● TurboGears is a full-stack framework. That makes it
quite slow by default!
● The team invested effort to constantly speed it up since
2.1 release.
● Still keeping all the
features around has
its price
● To cope with this
minimal mode got
introduced
9. Use only what you need
● Only use what you really need. Disabling some features
can make a big difference:
○ full featured -> ~900 req/sec
○ browser language detection -> ~1000 req/sec
○ widgets support -> ~1200 req/sec
○ sessions -> ~1300 req/sec
○ transaction manager -> ~1400 req/sec
○ minimal mode -> ~2100 req/sec
Measures are on wsgiref, purpose is only to show delta
10. Avoid serving statics
Cascading files serving is a common pattern:
static_app = StaticURLParser('statics')
app = Cascade([static_app, app])
What it is really happening is a lot:
○ path gets parsed to avoid ../../../etc/passwd
○ path gets checked on file system
○ a 404 response is generated
○ The 404 response is caught by the Cascade
middleware that forwards the requests to your app
11. Using Caching
● Caching means preorganizing your data the way you are
going to use it, if you already did that during design
phase you are already half done. Let the template drive
your data, not the opposite.
● Frameworks usually provide various type of caching.
TurboGears specifically provides:
○ @cached_property
○ tg.cache object for custom caching
○ @beaker_cache for controllers caching
○ Template Caching
○ Entity Caching
12. Use HTML5 & JS
● If only small portions of your page change, cache the
page and use JS to perform minor changes.
○ Invalidating your whole cache to say: "Welcome back
Mister X" is not a great idea.
● If you are using Varnish, nginx or any other frontend
cache consider using JS+localstorage instead of cookies
for trivial customizations. Cookies will skip frontend
caching
13. Template Caching
● Template caching means prerendering your
template based on controller computation results.
● It's common for template to access related entities,
those will be cached for you.
● If correctly organized it's the caching behavior with
best trade-off
○ Simple to implement
○ Guarantees correctly updates results
● An updated_at field on models is often all you need
14. WikiPage Caching
● WikiPage caching is the standard template caching
example in TurboGears documentation
@expose('wikir.templates.page')
@validate({'page':SQLAEntityConverter(model.WikiPage, slugified=True)},
error_handler=fail_with(404))
def _default(self, page, *args, **kw):
cache_key = '%s-%s' % (page.uid,
page.updated_at.strftime('%Y%m%d%H%M%S'))
return dict(wikipage=page, tg_cache={'key':cache_key,
'expire':24*3600,
'type':'memory'})
15. Caching Partials
● Case study: Notifications
○ Page delivered in 2.16s
○ Query took only 2ms
○ Most of the work was
actually in rendering
each notification
● Caching was useless as
notifications happened
often, constantly changing
content.
16. Entity Caching
Map each object to a partial: @entitycached decorator
makes easy to cache each notification by itself.
from tgext.datahelpers.caching import entitycached
@entitycached('notification')
def render_post(notification):
return Markup(notification.as_html)
● Page with cached notifications is now delivered in 158ms
● A single notification can be cached up for days, it will
never change.
17. Caching can be harmful
If you content changes
too often, caching on
first reuqest can
actually be harmful.
If you have multiple
processes and a lot of
requests you can end
up having a race
condition on cache
update.
18. Cache Stampede
● During football matches there were thousands of users
constantly pressing "refresh" button to reload page.
● Content constantly changed due to match being reported
on real time.
● After each update, all the running processes decided that
the cache was not valid anymore at the same time,
starting to regenerate the cache.
● Sometimes the content changed even while processes were
still updating cache for previous update.
19. Proactive Update
● To solve cache stampede the cache generation has been
bound to an hook on the article update so it happened only
once.
● Easy to achieve using Template caching in together with
tg.template_render on article id as cache key
● SQLALchemy @event.listens_for supports even notifications
on relationships, so it's reasy to update page cache even
when related comments, tags, and so on change.
20. A real solution
● The source of the issue were users pressing "reload" button
like there's no tomorrow.
● Solutions has been to push updates to the users through a
box that updates in real-time.
○ No more insane reloads
○ Users were actually more satisfied
○ Was a lot easier to maintain
○ Not only solved match article issues but also reduced
the load on other parts of the website
22. Think Different
● If you are struggling too much at improving performances,
you are probabling doing something your application is not
meant to do.
● Lesson learnt?
○ Soccer fans are eager for updates (no... for real!)
○ There is only one thing that gets more visits than a
football match: Rumors on football players trading
23. Offload Work
● The only person that know that something changes is the
author of the change itself.
○ Only update the core cache to provide author with an
immediate feedback
○ Don't be afraid of updating related caches
asynchronously. Author usually understands that it
might take some time before his changes propagate
and other users don't know that a change happened
yet.
● You can often provide an answer to user with little instant
computation, messages and notifications are a typical
example.
24. Maste-Slave replication is easy
● SQLAlchemy unit of work pattern makes easy for
frameworks to do the right thing 90% of the time
○ Read from slaves unless we are flushing the session
● Won't require changes to your code for most common cases
● Exceptions are as easy as @with_engine('master')
● As easy as
sqlalchemy.master.url = mysql://masterhost/dbname
sqlalchemy.slaves.slave1.url = mysql://slavehost/dbname
sqlalchemy.slaves.slave2.url = mysql://slavehost/dbname
25. Fast enough
● Speed should not be your primary focus, but it makes
sense to care a lot about it, users tend to get frustrated
by slow responses.
● New Relic App Speed Index reports an average of 5.0
seconds of response time for accepted experience.
● That is End-User time, not request time, to achieve 5
seconds you have to aim a lot lower
● Mean Opinion Score degrades quickly when surpassing
200ms. Less than 200ms is perceived as "right now".
http://newrelic.com/ASI
26. Development Tools
● It's easy to introduce changes with heavy impacts on
performances without noticing. Development tools can
help keeping under control impact of changes
● The DebugBar provides core utilities to track your
application speed while developing:
○ Controller Profiling
○ Template & Partials Timing
○ Query Reporting
○ Query Logging for AJAX/JSON requests
29. Check even after release
● Users use your application more widely than you
might have expected
● Fast now, doesn't mean fast forever. Like test units
avoid breaking things, rely on a speed feedback to
keep acceptable speed.
● Keep your Apdex T index updated, user
expectations evolve!
30. There is no silver bullet
● Sorry, there is no silver bullet.
● Every application is a separate case, general and
framework optimizations can usually provide little
benefit when compared to domain specific
optimizations
● Understanding how users interact with your application
is the golden rule of optimization
● Don't understimate how its easy to do something really
slow unconsciously: development tools can help
catching those.