SlideShare a Scribd company logo
1 of 77
Download to read offline
TURBOCHARGING
    YOUR
  EXTENSION
  © Klas Berlič & BzZzZ 2010
HOW IT ALL CAME ABOUT
    // How I got involved with cache //
Quite an amazing story
          of a willpower
and turning a failure into a sucess
        with contributions to common good
              as a final consequence
In 2007 approached by a client to build travel portal for
a client.

No clearly defined business strategy, exact site scope was
unknown




        Joint venture starting with visibility study
NOT AT ALL AN EASY TASK!
 especially if you look unimportant in the eyes of big players
2
Took 2 years
Only after then




development finally began.
Vacations, Hotels, Travel and Airtickets
                              booking management system                                                                 html integration

                                                                                                                   Already made airticket
                                                                                                                   booking engine




                                                                                                                      Website search box
                             XML search

                                                                     CORE
                                          XML results
                                                                     Routing searches
suppliers
German




              Traffics
                                                                     Combining results                                Rendering results
               Tibet
                               temporary booking                     Routing booking requests
                XML                over XML




                                                        static XML                                                    Booking request form



                                                                          XML
                                                                          Other SLO suppliers

              Traffics
               Tibet                                                                                 ADMINISTRATION
              Cosmo
                                                        Palma




                                                                                                Own inventory management


            Manual booking                                                                      Bookings management
And then..
Development subcontractor left with
only 10% done




                                           actual   plan




..when we should have been at beta stage
If your partners fail, you have to do the job yourself.




                                  // Problem was I knew php only briefly //
3
                                    // compare that to Joomla 1.6. :))




Learned php & OOP & brought it to
ß level in 3 months.
End result
Probably one of the most complex and technologically advanced
sites ever built on Joomla (it is simple..but in a Google way)
Combined power of Joomla php framework

and ExtJS javascript framework used on a presentation layer
Results from own database merged with results from multiple
different on-demand XML data sources




                                                                       Descriptions & images 1




                                                                                                 Descriptions & images 2
                                                                                                                           Website search box
                                      XML search

                                                                 CORE
      suppliers
      German




                           Traffics                XML results
                                                                 Routing searches
                            Tibet                                Combining results
                             XML                                                                                           Rendering results
                                        temporary booking
                                                                 Routing booking requests
                                            over XML




                  Own database                                                                                             Booking requests
Initially it took on average 45-65 seconds to perform a search &
render results.



Site badly needed a speed boost.
Speed?
Speed?




         // No, nothing illegal :)
Found an an answer in




  CACHING
JOOMLA! CACHE
    // explained //
How simple solutions
can speed up things
     10 times
      or more
CACHE
Temporary store an unit of information
to be reused later
// a kind of digital recycling //
Multiple layers of web caching:

Network (content delivery networks)
Ÿ
Server (proxy servers, load distribution..)
Ÿ
Daemons and interpreters – Apache mod_cache, Php –
Ÿ
opcode caching, Mysql query cache
Ÿ application (Joomla cache)
Web
Customer (browser cache) - used by Joomla Page cache
Ÿ
plugin
Web application caching

creating temporary static copies of otherwise dynamic
pages (or smaller data units).
Typical web site is displaying the same content over and
over again

No cache:
everything has to be generated for each and every page
view

Caching:
Ÿ some or all of the information your code generates
 store
 in a cache object
Ÿ it when next user requests the same page or
 serve
 particular piece of information.
JOOMLA! CACHING TYPOLOGY
       & when to use each of them
template



component



modules



functions



smaller
units

            page cache   component    module   function/callback   output cache
                         view cache   cache          cache          raw cache
PAGE CACHE
Takes snapshots of entire pages including everything -
component, modules, plugins and a template.

The widest and the least flexible approach of all caching
options.
// enabled by core system plugin -> site administrators choice //
MODULE & COMPONENT
    VIEW CACHE
A group as they they both create a static copy of complete
output of a component or a module

J1.5.
Records based on the calling URL            // UNSAFE, DOS //

J1.6.
Changes                                         // more later //
Important difference                  // J1.5., changed in J1.6! //


module caching can only be set on/off for all instances,
can’t be controlled from within the module
Most widespread cache type, sometimes equaled with J
caching in general.


       Performs well in the speed terms


       Disables any
       user<->extension<->framework
       interaction until a cached copy has expired

Not suitable for any components or modules that react to
user actions or render frequently changing content.
Side effect
Cached copy includes only modules or components own
output -> any external file that is called by using
methods like $document->addStyleSheet() won't get included

Workarounds performed to come around this limitations

Catch22
Workarounds require computing time -> diminish the
effect
FUNCTION / CALLBACK
       CACHE
The first of flexible caching types that enable us to
differentiate between various parts of the extension and
cache only those parts that are cacheable, while keeping
dynamic parts uncached.
Caches results of function calls

Records (cacheID) based on the arguments passed to the
function
Often useful to cache model methods and keep views
uncached.

Example use
Model performing expensive queries or similar operations
is run only once to create a larger pool of data which is
then further manipulated inside the view (sorting,
pagination, calculations etc.)
OUTPUT CACHE
Caches output of some part od the script

Records based on id passed
Basically output buffering with caching
RAW CACHE
Caches any data units

Records based on id passed
Fully controlled by the coder – what, when to store, how
to classify stored units (cache id).

Highly useful when we are dealing with finite number of
reusable data units
Example use
High number of possible combinations of relatively small
number of units – e.g. products in online store. No point
to cache multiple parameter searches, cache each product
separately.


Other examples
Expensive queries, remote XML, thumbnails, reusable
texts or any reusable data set.
Also useful to pass large amounts of data between
different parts of application (e.g. steps in click flow)

Used in Joomla core: list of components, list of modules,
menu tree, available languages, user groups, html
renderers etc.
CASE STUDY
 travel booking engine
PROBLEM

It took on average 45 to 65 seconds to display a
search result
DIAGNOSIS

Largest portion of time spent on remote XML requests,
waiting for and receiving replies.

Other two performance hogs were resizing images and
internal database queries.

2nd and other pages of paginated results displayed
instantly
Descriptions & images 1




                                                                                           Descriptions & images 2
    Each result from badly designed
    primary XML paired with description
    and image that come from separate
    XML sources (one by one)


                                                                                                                     Website search box
                                XML search

                                                           CORE
suppliers
German




                     Traffics                XML results
                                                           Routing searches
                      Tibet                                Combining results
                       XML                                                                                           Rendering results
                                  temporary booking
                                                           Routing booking requests
                                      over XML




            Own database                                                                                             Booking requests
Descriptions & images 1




                                                                                           Descriptions & images 2
       SLOW
                                                                                                                     Website search box
                                XML search

                                                           CORE
suppliers
German




                     Traffics                XML results
                                                           Routing searches
                      Tibet                                Combining results
                       XML                                                                                           Rendering results
                                  temporary booking
                                                           Routing booking requests
                                      over XML




            Own database                                                                                             Booking requests
SOLUTION
Cache data on multiple levels:

Ÿ pages that were fully rendered - a lot of users
Cache
 search with default parameters or click lastminute
 links, most just check one or two pages of paginated
 results

Ÿ XML queries (with longer lifetime). Pass data
Cache
 from short results to detail pages

Long-term caching of images and descriptions -> most
Ÿ
 important one
ALSO..

Add more indexes to Mysql tables

Query and render images and descriptions per page:
primary source queried once, secondary separately for
each subpage (pagination).
RESULT

Site runs 10 times faster
Average first page loads in 5-6 seconds
Subpages (paginated) load in 0.5 - 1 seconds

Those timings further drop dramaticaly (50% or more) on
peak hours, when most popular searches are returned
from the cache.
The bigger number of users on a site, the faster it goes.
// within the hosting limits //
JOOMLA! 1.6. CACHE CHANGES
          // firsthand //
A very long story made short:
      "It finally works"
FRAMEWORK LEVEL

Almost everyhing was redone under the hood.

Cache library has been completely refactored.
Most important framework changes

Cache handlers are now known as cache controllers (page,
view, output, callback)

Parent JCacheController was added - among other things
it controls raw get and store calls.
New cachelite and wincache storage handlers (drivers).
File cache handler heavily optimised
All other handlers fixed with missing functions (gc, clean)
added, their code cleaned and tested and should now be
working properly.

Semaphore locking was added for reliability and
improved performance
Changed component view caching (replaced unsafe
cacheID creation method)

Module cache completely reconceptualized
CMS FRAMEWORK LEVEL CHANGES

Caching implemented in all components and modules that
can potentially gain from using it

Caching added to some most expensive and frequent
framework calls:
JComponentHelper::_load(), JModuleHelper::_load(), JMenuSite::load(),
JDocumentHTML::getBuffer()..
USER LEVEL FUNCTIONAL CHANGES

Cache administration (Clean cache, Purge cache) now
works with all storage handlers

New standalone garbage collect script to be run from a
crontab added
libraries/joomla/utilities/garbagecron.php
API changes
// and other changes important for developers //
COMPONENT VIEW CACHE

Takes an array of url parameters and their types to create
Cacheid.

A replacement for a previous unsafe way which took the
whole URL and so opened the doors for DOS attacks via
random parameters/values added to request
// Old cacheid created from URL retained for backwards compatibility if there are no
$safeurlparams (to be removed in 1.7) //
Com_contact controler usage example:
$safeurlparams = array('id'=>'INT', 'catid'=>'INT', 'limit'=>'INT',
'limitstart'=>'INT', 'filter_order'=>'CMD', 'filter_order_Dir'=>'CMD’,
'lang'=>'CMD');

parent::display($cachable,$safeurlparams);
MODULE CACHE

5 different modes of operation, 3 of them are to be set
from module XML file, while 2 are meant to be used from
within the module itself.

Default is backwards compatible oldstatic mode that
requires no changes to a module.
MODES TO BE SET IN XML

Static - one cache file for all pages with the same module
parameters

Oldstatic - 1.5. definition of module caching, one cache
file for all pages with the same module id and user aid.
Default for backwards compatibility

Itemid - changes on itemid change
In addition to cache field that was required in 1.5 there is
now new hidden field called cachemode that sets any of
the above modes.
<field name="cachemode" type="hidden" default="static">
<option value="static"></option>
MODES TO BE CALLED FROM INSIDE THE MODULE:

Safeuri - id is created from URL params array, as in
component view cache

Id - module sets own cache id's

To use this modes rename 'cache' field in xml to
'owncache' field and call JModuleHelper::ModuleCache from within


// actually a shortcut to cache callback to avoid code duplication in every module //
An example that uses safeuri mode and replaces uncached
$list = modRelatedItemsHelper::getList($params) :


$cacheparams->modeparams = array('id'=>'int','Itemid'=>'int');
$cacheparams->methodparams = $params;
$cacheparams->method = 'getList';
$cacheparams->class = 'modRelatedItemsHelper';
$cacheparams->cachemode = 'safeuri';
$cacheparams = new stdClass;

$list = JModuleHelper::ModuleCache ($module, $params, $cacheparams);
RAW CACHE

Raw cache get and store are easily accesed by passing ''
(empty string) as cache controller to JFactory::getCache
Data is auto serialized / deserialized

Locking & unlocking are performed automaticaly
AND HERE THE STORY ENDS
       // ok..a little more //
If you still remember how it all started..

I hope you like where it ended.
               // tried to do my best //

More Related Content

Similar to Turbocharging your extension // Joomla //

Apache Camel: The Swiss Army Knife of Open Source Integration
Apache Camel: The Swiss Army Knife of Open Source IntegrationApache Camel: The Swiss Army Knife of Open Source Integration
Apache Camel: The Swiss Army Knife of Open Source Integrationprajods
 
Presentation of Dominos future plans
Presentation of Dominos future plansPresentation of Dominos future plans
Presentation of Dominos future plansdominion
 
Techdays 2013 the road to end user self service with service manager 2012
Techdays 2013   the road to end user self service with service manager 2012Techdays 2013   the road to end user self service with service manager 2012
Techdays 2013 the road to end user self service with service manager 2012CompuTrain. De IT opleider.
 
Techdays 2013 the road to end user self service with service manager 2012 SP1
Techdays 2013 the road to end user self service with service manager 2012 SP1Techdays 2013 the road to end user self service with service manager 2012 SP1
Techdays 2013 the road to end user self service with service manager 2012 SP1wwwally
 
Keeping Your Digital Office Clean Wim Putzeys Panoptic
Keeping Your Digital Office Clean Wim Putzeys PanopticKeeping Your Digital Office Clean Wim Putzeys Panoptic
Keeping Your Digital Office Clean Wim Putzeys PanopticWim Putzeys
 
Do and Don'ts of BPM - The Full Stack
Do and Don'ts of BPM - The Full StackDo and Don'ts of BPM - The Full Stack
Do and Don'ts of BPM - The Full StackJoram Barrez
 
03.egovFrame Runtime Environment Training Book
03.egovFrame Runtime Environment Training Book03.egovFrame Runtime Environment Training Book
03.egovFrame Runtime Environment Training BookChuong Nguyen
 
SD Forum 1999 XML Lessons Learned
SD Forum 1999 XML Lessons LearnedSD Forum 1999 XML Lessons Learned
SD Forum 1999 XML Lessons LearnedTed Leung
 
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...mfrancis
 
Mozilla In Malaysia
Mozilla In MalaysiaMozilla In Malaysia
Mozilla In MalaysiaGen Kanai
 
Good Data: Collaborative Analytics On Demand
Good Data: Collaborative Analytics On DemandGood Data: Collaborative Analytics On Demand
Good Data: Collaborative Analytics On Demandzsvoboda
 
OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011
OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011
OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011Mark Rittman
 
Integration SharePoint 2010 with CRM 2010 by Mai Omar Desouki
Integration SharePoint 2010 with CRM 2010 by Mai Omar DesoukiIntegration SharePoint 2010 with CRM 2010 by Mai Omar Desouki
Integration SharePoint 2010 with CRM 2010 by Mai Omar DesoukiMai Omar Desouki
 
Utilized JAXB to generate POJOs automatically
Utilized JAXB to generate POJOs automaticallyUtilized JAXB to generate POJOs automatically
Utilized JAXB to generate POJOs automaticallyGuo Albert
 
Amazon Webservice & Cloud Computing
Amazon Webservice & Cloud ComputingAmazon Webservice & Cloud Computing
Amazon Webservice & Cloud ComputingJack Smith
 
Plugin-able POS Solutions by Javascript @HDM9 Taiwan
Plugin-able POS Solutions by Javascript @HDM9 TaiwanPlugin-able POS Solutions by Javascript @HDM9 Taiwan
Plugin-able POS Solutions by Javascript @HDM9 TaiwanRack Lin
 
2. FOMS _ FeedHenry_ Mícheál Ó Foghlú
2. FOMS _ FeedHenry_ Mícheál Ó Foghlú2. FOMS _ FeedHenry_ Mícheál Ó Foghlú
2. FOMS _ FeedHenry_ Mícheál Ó FoghlúFOMS011
 

Similar to Turbocharging your extension // Joomla // (20)

Apache Camel: The Swiss Army Knife of Open Source Integration
Apache Camel: The Swiss Army Knife of Open Source IntegrationApache Camel: The Swiss Army Knife of Open Source Integration
Apache Camel: The Swiss Army Knife of Open Source Integration
 
Obiee 11 g
Obiee 11 gObiee 11 g
Obiee 11 g
 
Presentation of Dominos future plans
Presentation of Dominos future plansPresentation of Dominos future plans
Presentation of Dominos future plans
 
Techdays 2013 the road to end user self service with service manager 2012
Techdays 2013   the road to end user self service with service manager 2012Techdays 2013   the road to end user self service with service manager 2012
Techdays 2013 the road to end user self service with service manager 2012
 
Techdays 2013 the road to end user self service with service manager 2012 SP1
Techdays 2013 the road to end user self service with service manager 2012 SP1Techdays 2013 the road to end user self service with service manager 2012 SP1
Techdays 2013 the road to end user self service with service manager 2012 SP1
 
Keeping Your Digital Office Clean Wim Putzeys Panoptic
Keeping Your Digital Office Clean Wim Putzeys PanopticKeeping Your Digital Office Clean Wim Putzeys Panoptic
Keeping Your Digital Office Clean Wim Putzeys Panoptic
 
Do and Don'ts of BPM - The Full Stack
Do and Don'ts of BPM - The Full StackDo and Don'ts of BPM - The Full Stack
Do and Don'ts of BPM - The Full Stack
 
03.egovFrame Runtime Environment Training Book
03.egovFrame Runtime Environment Training Book03.egovFrame Runtime Environment Training Book
03.egovFrame Runtime Environment Training Book
 
SD Forum 1999 XML Lessons Learned
SD Forum 1999 XML Lessons LearnedSD Forum 1999 XML Lessons Learned
SD Forum 1999 XML Lessons Learned
 
IT Governance Portals
IT Governance   PortalsIT Governance   Portals
IT Governance Portals
 
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...
Balconies, Patios, Terraces, and Bridges. Architectural approaches for moving...
 
Mozilla In Malaysia
Mozilla In MalaysiaMozilla In Malaysia
Mozilla In Malaysia
 
Good Data: Collaborative Analytics On Demand
Good Data: Collaborative Analytics On DemandGood Data: Collaborative Analytics On Demand
Good Data: Collaborative Analytics On Demand
 
OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011
OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011
OBIEE11g Architecture & Internals : Collaborate'11, Orlando 2011
 
Kaazing
KaazingKaazing
Kaazing
 
Integration SharePoint 2010 with CRM 2010 by Mai Omar Desouki
Integration SharePoint 2010 with CRM 2010 by Mai Omar DesoukiIntegration SharePoint 2010 with CRM 2010 by Mai Omar Desouki
Integration SharePoint 2010 with CRM 2010 by Mai Omar Desouki
 
Utilized JAXB to generate POJOs automatically
Utilized JAXB to generate POJOs automaticallyUtilized JAXB to generate POJOs automatically
Utilized JAXB to generate POJOs automatically
 
Amazon Webservice & Cloud Computing
Amazon Webservice & Cloud ComputingAmazon Webservice & Cloud Computing
Amazon Webservice & Cloud Computing
 
Plugin-able POS Solutions by Javascript @HDM9 Taiwan
Plugin-able POS Solutions by Javascript @HDM9 TaiwanPlugin-able POS Solutions by Javascript @HDM9 Taiwan
Plugin-able POS Solutions by Javascript @HDM9 Taiwan
 
2. FOMS _ FeedHenry_ Mícheál Ó Foghlú
2. FOMS _ FeedHenry_ Mícheál Ó Foghlú2. FOMS _ FeedHenry_ Mícheál Ó Foghlú
2. FOMS _ FeedHenry_ Mícheál Ó Foghlú
 

Recently uploaded

JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard37
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdfMuhammad Subhan
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...marcuskenyatta275
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxFIDO Alliance
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuidePixlogix Infotech
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!Memoori
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهMohamed Sweelam
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)Wonjun Hwang
 

Recently uploaded (20)

JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 

Turbocharging your extension // Joomla //

  • 1. TURBOCHARGING YOUR EXTENSION © Klas Berlič & BzZzZ 2010
  • 2.
  • 3. HOW IT ALL CAME ABOUT // How I got involved with cache //
  • 4. Quite an amazing story of a willpower and turning a failure into a sucess with contributions to common good as a final consequence
  • 5. In 2007 approached by a client to build travel portal for a client. No clearly defined business strategy, exact site scope was unknown Joint venture starting with visibility study
  • 6. NOT AT ALL AN EASY TASK! especially if you look unimportant in the eyes of big players
  • 7.
  • 10. Vacations, Hotels, Travel and Airtickets booking management system html integration Already made airticket booking engine Website search box XML search CORE XML results Routing searches suppliers German Traffics Combining results Rendering results Tibet temporary booking Routing booking requests XML over XML static XML Booking request form XML Other SLO suppliers Traffics Tibet ADMINISTRATION Cosmo Palma Own inventory management Manual booking Bookings management
  • 12. Development subcontractor left with only 10% done actual plan ..when we should have been at beta stage
  • 13. If your partners fail, you have to do the job yourself. // Problem was I knew php only briefly //
  • 14. 3 // compare that to Joomla 1.6. :)) Learned php & OOP & brought it to ß level in 3 months.
  • 16. Probably one of the most complex and technologically advanced sites ever built on Joomla (it is simple..but in a Google way)
  • 17. Combined power of Joomla php framework and ExtJS javascript framework used on a presentation layer
  • 18. Results from own database merged with results from multiple different on-demand XML data sources Descriptions & images 1 Descriptions & images 2 Website search box XML search CORE suppliers German Traffics XML results Routing searches Tibet Combining results XML Rendering results temporary booking Routing booking requests over XML Own database Booking requests
  • 19. Initially it took on average 45-65 seconds to perform a search & render results. Site badly needed a speed boost.
  • 21. Speed? // No, nothing illegal :)
  • 22.
  • 23. Found an an answer in CACHING
  • 24. JOOMLA! CACHE // explained //
  • 25. How simple solutions can speed up things 10 times or more
  • 26. CACHE Temporary store an unit of information to be reused later // a kind of digital recycling //
  • 27. Multiple layers of web caching: Network (content delivery networks) Ÿ Server (proxy servers, load distribution..) Ÿ Daemons and interpreters – Apache mod_cache, Php – Ÿ opcode caching, Mysql query cache Ÿ application (Joomla cache) Web Customer (browser cache) - used by Joomla Page cache Ÿ plugin
  • 28. Web application caching creating temporary static copies of otherwise dynamic pages (or smaller data units).
  • 29. Typical web site is displaying the same content over and over again No cache: everything has to be generated for each and every page view Caching: Ÿ some or all of the information your code generates store in a cache object Ÿ it when next user requests the same page or serve particular piece of information.
  • 30. JOOMLA! CACHING TYPOLOGY & when to use each of them
  • 31. template component modules functions smaller units page cache component module function/callback output cache view cache cache cache raw cache
  • 33. Takes snapshots of entire pages including everything - component, modules, plugins and a template. The widest and the least flexible approach of all caching options. // enabled by core system plugin -> site administrators choice //
  • 34. MODULE & COMPONENT VIEW CACHE
  • 35. A group as they they both create a static copy of complete output of a component or a module J1.5. Records based on the calling URL // UNSAFE, DOS // J1.6. Changes // more later //
  • 36. Important difference // J1.5., changed in J1.6! // module caching can only be set on/off for all instances, can’t be controlled from within the module
  • 37. Most widespread cache type, sometimes equaled with J caching in general. Performs well in the speed terms Disables any user<->extension<->framework interaction until a cached copy has expired Not suitable for any components or modules that react to user actions or render frequently changing content.
  • 38. Side effect Cached copy includes only modules or components own output -> any external file that is called by using methods like $document->addStyleSheet() won't get included Workarounds performed to come around this limitations Catch22 Workarounds require computing time -> diminish the effect
  • 40. The first of flexible caching types that enable us to differentiate between various parts of the extension and cache only those parts that are cacheable, while keeping dynamic parts uncached.
  • 41. Caches results of function calls Records (cacheID) based on the arguments passed to the function
  • 42. Often useful to cache model methods and keep views uncached. Example use Model performing expensive queries or similar operations is run only once to create a larger pool of data which is then further manipulated inside the view (sorting, pagination, calculations etc.)
  • 44. Caches output of some part od the script Records based on id passed
  • 47. Caches any data units Records based on id passed
  • 48. Fully controlled by the coder – what, when to store, how to classify stored units (cache id). Highly useful when we are dealing with finite number of reusable data units
  • 49. Example use High number of possible combinations of relatively small number of units – e.g. products in online store. No point to cache multiple parameter searches, cache each product separately. Other examples Expensive queries, remote XML, thumbnails, reusable texts or any reusable data set.
  • 50. Also useful to pass large amounts of data between different parts of application (e.g. steps in click flow) Used in Joomla core: list of components, list of modules, menu tree, available languages, user groups, html renderers etc.
  • 51. CASE STUDY travel booking engine
  • 52. PROBLEM It took on average 45 to 65 seconds to display a search result
  • 53. DIAGNOSIS Largest portion of time spent on remote XML requests, waiting for and receiving replies. Other two performance hogs were resizing images and internal database queries. 2nd and other pages of paginated results displayed instantly
  • 54. Descriptions & images 1 Descriptions & images 2 Each result from badly designed primary XML paired with description and image that come from separate XML sources (one by one) Website search box XML search CORE suppliers German Traffics XML results Routing searches Tibet Combining results XML Rendering results temporary booking Routing booking requests over XML Own database Booking requests
  • 55. Descriptions & images 1 Descriptions & images 2 SLOW Website search box XML search CORE suppliers German Traffics XML results Routing searches Tibet Combining results XML Rendering results temporary booking Routing booking requests over XML Own database Booking requests
  • 56. SOLUTION Cache data on multiple levels: Ÿ pages that were fully rendered - a lot of users Cache search with default parameters or click lastminute links, most just check one or two pages of paginated results Ÿ XML queries (with longer lifetime). Pass data Cache from short results to detail pages Long-term caching of images and descriptions -> most Ÿ important one
  • 57. ALSO.. Add more indexes to Mysql tables Query and render images and descriptions per page: primary source queried once, secondary separately for each subpage (pagination).
  • 58. RESULT Site runs 10 times faster Average first page loads in 5-6 seconds Subpages (paginated) load in 0.5 - 1 seconds Those timings further drop dramaticaly (50% or more) on peak hours, when most popular searches are returned from the cache. The bigger number of users on a site, the faster it goes. // within the hosting limits //
  • 59. JOOMLA! 1.6. CACHE CHANGES // firsthand //
  • 60. A very long story made short: "It finally works"
  • 61. FRAMEWORK LEVEL Almost everyhing was redone under the hood. Cache library has been completely refactored.
  • 62. Most important framework changes Cache handlers are now known as cache controllers (page, view, output, callback) Parent JCacheController was added - among other things it controls raw get and store calls.
  • 63. New cachelite and wincache storage handlers (drivers). File cache handler heavily optimised All other handlers fixed with missing functions (gc, clean) added, their code cleaned and tested and should now be working properly. Semaphore locking was added for reliability and improved performance
  • 64. Changed component view caching (replaced unsafe cacheID creation method) Module cache completely reconceptualized
  • 65. CMS FRAMEWORK LEVEL CHANGES Caching implemented in all components and modules that can potentially gain from using it Caching added to some most expensive and frequent framework calls: JComponentHelper::_load(), JModuleHelper::_load(), JMenuSite::load(), JDocumentHTML::getBuffer()..
  • 66. USER LEVEL FUNCTIONAL CHANGES Cache administration (Clean cache, Purge cache) now works with all storage handlers New standalone garbage collect script to be run from a crontab added libraries/joomla/utilities/garbagecron.php
  • 67. API changes // and other changes important for developers //
  • 68. COMPONENT VIEW CACHE Takes an array of url parameters and their types to create Cacheid. A replacement for a previous unsafe way which took the whole URL and so opened the doors for DOS attacks via random parameters/values added to request // Old cacheid created from URL retained for backwards compatibility if there are no $safeurlparams (to be removed in 1.7) //
  • 69. Com_contact controler usage example: $safeurlparams = array('id'=>'INT', 'catid'=>'INT', 'limit'=>'INT', 'limitstart'=>'INT', 'filter_order'=>'CMD', 'filter_order_Dir'=>'CMD’, 'lang'=>'CMD'); parent::display($cachable,$safeurlparams);
  • 70. MODULE CACHE 5 different modes of operation, 3 of them are to be set from module XML file, while 2 are meant to be used from within the module itself. Default is backwards compatible oldstatic mode that requires no changes to a module.
  • 71. MODES TO BE SET IN XML Static - one cache file for all pages with the same module parameters Oldstatic - 1.5. definition of module caching, one cache file for all pages with the same module id and user aid. Default for backwards compatibility Itemid - changes on itemid change
  • 72. In addition to cache field that was required in 1.5 there is now new hidden field called cachemode that sets any of the above modes. <field name="cachemode" type="hidden" default="static"> <option value="static"></option>
  • 73. MODES TO BE CALLED FROM INSIDE THE MODULE: Safeuri - id is created from URL params array, as in component view cache Id - module sets own cache id's To use this modes rename 'cache' field in xml to 'owncache' field and call JModuleHelper::ModuleCache from within // actually a shortcut to cache callback to avoid code duplication in every module //
  • 74. An example that uses safeuri mode and replaces uncached $list = modRelatedItemsHelper::getList($params) : $cacheparams->modeparams = array('id'=>'int','Itemid'=>'int'); $cacheparams->methodparams = $params; $cacheparams->method = 'getList'; $cacheparams->class = 'modRelatedItemsHelper'; $cacheparams->cachemode = 'safeuri'; $cacheparams = new stdClass; $list = JModuleHelper::ModuleCache ($module, $params, $cacheparams);
  • 75. RAW CACHE Raw cache get and store are easily accesed by passing '' (empty string) as cache controller to JFactory::getCache Data is auto serialized / deserialized Locking & unlocking are performed automaticaly
  • 76. AND HERE THE STORY ENDS // ok..a little more //
  • 77. If you still remember how it all started.. I hope you like where it ended. // tried to do my best //