eZ Summer Camp. Setp 2012



eZ Publish Caching
   Mechanisms
              // WORKSHOP


  Gilles Guirand - CTO Kaliop / eZ Publish board member @gandbox
  Gilles Ballini – Lead engineer Kaliop
  Stephane Sobecki - Lead engineer Kaliop
  Jerome Lecocq - Lead engineer Kaliop
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA                            (1)

   TEMPLATE COMPILATION
      Concept / exploring the VAR
      Ex : HOWTO compile a set of template from a TXT file list


   INI CACHE
      Concept / exploring the VAR
      Ex : HOWTO use dynamic settings with shared compiled templates

   VIEWCACHE
      Concept / exploring the VAR
      Ex : HOWTO log the viewcache activity




                                                                       2 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA ( 2 )
   CACHE-BLOCK
       Concept / exploring the VAR
       Ex : HOWTO expire a named cache-block, related to the homepage
   expiry, or a contentclass, or whatever else
       Ex : HOWTO expire a set of cache-block using a shell script
       Ex : HOWTO build the perfect pagelayout

   EXPIRY.PHP FILE
       Concept / exploring the VAR

   EZCACHE.PHP SCRIPT
       Concept / exploring the VAR
       Ex : HOWTO create a custom ezcache.php --id=myid

   PHP CACHE
       Ex : HOWTO generate a PHP based cache file
                                                                        3 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CONTEXT
   Common questions :
    ➔Is it possible to never expire the ENTIRE viewcache ?
   … and which actions actually expire the ENTIRE viewcache ?

    ➔   Is it possible to reduce the I/O (template-blocks) ?

    ➔   Is it possible to expire a specific set ot template-block files ?

    ➔   Is it possible to develop a better pagelayout.tpl ?

    ➔   Is something to improve on high-trafic websites ? Or Websites
        factory ( 100 siteaccess... )

                                                                        4 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CONTEXT
   Final questions :
    ➔ How all these cache mechanisms really work ?
   … I mean in details... algorithms, I/O, triggers, exceptions,
   undocumented features & settings, unknown bugs, unknown
   fixes... and ezfs, ezfs2, ezdfs, ezdb...

   Final solutions :
    ➔   READ THE KERNEL / TEST THE KERNEL... ( be brave )




                                                                   5 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VM PREPARATION
   Copy cache & cache2 extension folders
   Activate extension (site.ini.append.php)
   Generate autoloads
   Clear the cache-all

   Add user / login policy for anomymous ( site
   access cache & cache 2 )


                                                     6 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE
   COMPILATION
      Concept / exploring the VAR
      Ex : HOWTO compile a set of template from a TXT file list

   INI CACHE
   VIEWCACHE
   CACHE-BLOCK
   EXPIRY.PHP FILE
   EZCACHE.PHP SCRIPT
   PHP CACHE
                                                                  7 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   TPL COMPILATION
   What for ?
      ➔ Compile your .tpl files to .php files
      ➔ Reduce the RAM / CPU (use the stored compiled version)




   How to set ? site.ini [templatesettings]
      ➔   TemplateCompile=enabled
          ➔   Store compiled version {varDir}/cache/template/compiled/

      ➔   NodeTreeCaching=enabled ( disabled by default )
          ➔   Store pre-compiled version {varDir}/cache/template/tree/


   Common mistake
      ➔   TemplateCache=enabled → template-block

                                                                         8 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   TPL COMPILATION
    DESIGN                  {varDir}/cache/template/tree   {varDir}/cache/template/compiled

    page_mainarea.tpl         {hash}-page_mainarea.php          page_mainarea-{hash1}.php



                        NodeTreeCaching=enabled
                                                                page_mainarea-{hash2}.php
                            ( disabled by default )



                                                                page_mainarea-{hash3}.php




   Why eZ generates several
   compiled files for the same .tpl?

                                                                                              9 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   TPL COMPILATION
   How eZ stores the final compiled files ?

   {VarDir}/cache/template/compiled/, not clusterized

   Filenames : {templatename}-{hash}.php
   page_head-49f4458d0b68aec3cf2de63a6918fd61.php
   page_head-ec3cf2de63a6918fd6149f4458d0b68a.php




                                                        10 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   TPL COMPILATION
   How eZ calculates the hash ?
   $sharedTemplates = false

   {VarDir}/cache/template/compiled/

   Filenames : {templatename}-{hash}.php
   pagelayout-49f4458d0b68aec3cf2de63a6918fd61.php

     Md5( filepath ) / internalCharset / language / useFullUrlText /
      accessText / pageLayoutVariable / indexFile / extraName


                                           /layout/set/...
                                                                   11 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   TPL COMPILATION
   How eZ calculate the hash ?
   $sharedTemplates = true

   {VarDir}/cache/template/compiled/

   Filenames : {templatename}-{hash}.php
   pagelayout-aec3cf2de63a6918fd6149f4458d0b68.php

                     Md5( filepath ) / language

                 No more siteaccess inside the key...
                Ezini operator → 'dynamic' parameter
                                                        12 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   How to pre-compile a set of template files
   (to avoid : on the fly generating) ?
    ➔   Force the compilation of 2 template files
        php bin/php/eztc.php -s mysiteaccess --force
        extension/ezdemo/design/ezdemo/templates/page_head.tpl
        extension/ezdemo/design/ezdemo/templates/pagelayout.tpl

    ➔   Force the compilation of all template files starting with « page » :
        find extension/ezdemo/ -name "page_*.tpl" | xargs php bin/php/eztc.php
        -s eng --force

    ➔   Force the compilation of all template files from a TXT file :
        cat templates.txt | xargs php bin/php/eztc.php -s eng --force


                                                                                 13 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   TPL COMPILATION
   The **** layout module
    ➔   By default ( $shareTemplates = false ), the 'layout' value is used by the
        final compiled filename
    ➔   So if you are using this module with 3 other layouts, you'll produce 4
        compiled files (instead of 1)
    ➔   eztc.php script does not manage the layout parameter

   Your 3 additional compiled files ( for each layout ) will
   be procude « on the fly », and cannot be pre-compiled
   with eztc.php
    ➔   Suggestion : improve eztc.php, add the –layout=... parameter



                                                                                    14 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE COMPILATION

   INI CACHE
      Concept / exploring the VAR
      Ex : HOWTO use dynamic settings with shared compiled templates



   VIEWCACHE
   CACHE-BLOCK
   EXPIRY.PHP FILE
   EZCACHE.PHP SCRIPT
   PHP CACHE

                                                                       15 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   INI CACHE
   When eZ generates the INI cache ?
     ➔   On the fly (if the cache file does not exist) – from an API call
         (ezini PHP class), a template call (ezini operator)

     ➔   By default, eZ automatically check if an INI file has been
         changed or not, depending of this setting (config.php)

   //define( 'EZP_INI_FILEMTIME_CHECK', false ) ;

   /* Set EZP_INI_FILEMTIME_CHECK constant to false to improve performance
   ( 10% ~ 15% on I/O ) by not checking modified time on ini files. You can also set it to
   a string, the name of a ini file you still want to check modified time on, best example
   would be to set it to 'site.ini' to make the system still check that but not the rest. */


                                                                                          16 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   INI CACHE
   How eZ stores the INI cache ?
    ➔   var/cache/ini ( and not /{varDir}/ini ), not clusterized

    ➔ {inifilename}-{hash}.php
   design-677247a49e428aa0837411b52777b920.php
   design-704216b4d0e3ea68c09742504bb366c8.php
   design-7333039e7e19aa411f8dd01836555861.php
                                    md5

         FileName / RootDir / DirectAccess / overrideDirs /
                         internalCharset



                                                                   17 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   INI CACHE
   How to expire the INI cache ?
   ➔php bin/php/ezcache.php --clear-id=global_ini
    ➔ Remove var/cache/ini




   ➔php bin/php/ezcache.php --clear-tag=ini
    ➔ Remove var/cache/ini &


      var/cache/active_extensions_{hash}.php

   ➔php bin/php/ezcache.php --clear-id=ini
    ➔ Nothing (bug) : try to recursively delete INI files in


      {varDir}/ini


                                                               18 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   INI CACHE
   How to not compile the INI value ?
   lib/ezutils/classes/ezini.php

   function eZINI( $fileName = 'site.ini', $rootDir = '', $useTextCodec = null, $useCache
   = null, $useLocalOverrides = null, $directAccess = false, $addArrayDefinition = false,
   $load = true )
   {
       ...
   }
 Dynamically load the INI values inside your compiled templates :

    site.ini [eZINISettings] / DynamicTemplateMode=enabled
                                 OR
                Ezini operator → 'dynamic' parameter

                                                                                        19 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   HOWTO use dynamic settings with shared compiled templates




                                                               20 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE COMPILATION
   INI CACHE

   VIEWCACHE
      Concept / exploring the VAR
      Ex : HOWTO log the viewcache activity


   CACHE-BLOCK
   EXPIRY.PHP FILE
   EZCACHE.PHP SCRIPT
   PHP CACHE


                                                     21 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
                         What for ?
                           ➔   Viewcache = content cache, content
                               viewcache, …

                           ➔   Viewcache is an HTML (or other) cache of the
                               $module_result.content outpout, for a node
                               view call ( content/view/... )



                         ← Illustration : Łukasz Serwatka
                         http://share.ez.no/learn/ez-publish/ez-publish-performance-optimization-
                         part-3-of-3-practical-cache-and-template-solutions/(page)/2




                                                                                         22 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   How eZ stores the viewcache ?
   {VarDir}/cache/content/{siteaccess}/{number}/{number}/{number}

   {VarDir}/cache/content/{siteaccess}/
   2-ce182234bd70236c6a009578d2a34632.cache
   20-ce182234bd70236c6a009578d2a34632.cache

   {VarDir}/cache/content/{siteaccess}/1
   101-ce182234bd70236c6a009578d2a34632.cache
   105-ce182234bd70236c6a009578d2a34632.cache

   {VarDir}/cache/content/{siteaccess}/5/2/1
   52101-ce182234bd70236c6a009578d2a34632.cache
   52144-ce182234bd70236c6a009578d2a34632.cache


                                                                    23 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   How eZ stores the viewcache ?
   {nodeid}-{hash}.php
   2-ce182234bd70236c6a009578d2a34632.cache
                                md5
  required

       nodeID / viewMode / layout / language (view.p.) / offset
                       (view.p.) / indexFile
  optionnal                            +
   roIeIDList / limitValueList / discountList / cacheNameExtra /
           pr_user / viewParameters / userPreferences
  ViewCacheTweaks[<node_id>]=<setting>[;<setting2>]
  ViewCacheTweaks[global]=<setting>[;<setting2>]

                                                                   24 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   How to know the exact matching between a
   viewcache file & my URLs ?
   {nodeid}-{hash}.php → deduce the nodeID

   At the end of file :
   "view_parameters";a:6:
   {s:6:"offset";b:0;s:4:"year";b:0;s:5:"month";b:0;s:3:"day";b:0;s:10:"na
   mefilter";b:0;s:4:"test";s:1:"2";}...

   Should be URL : /(test)/2



                                                                      25 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   How eZ expires a set of viewcache ?
   Alteration of cache states - Depending of the filehandler ( file.ini )

     ➔   Ezfsfilehandler : UNLINK file
     ➔   Ezfs2filehandler : TOUCH file (25/25/1977) ← Star Wars day
     ➔   Ezdbfilehandler : "ezdbfile" (UPDATE expired=1)
     ➔   Ezdfsfilehandler : "ezdfsfile" (UPDATE expired=1)

   Comparison of cache states - Depending of the filehandler ( file.ini )

     ➔   Ezfsfilehandler : file does not exist, or is older than the content-view-cache
         timestamp
     ➔   Ezfs2filehandler : file is older than the content-view-cache timestamp
     ➔   Ezdbfilehandler : "ezdbfile" (expired=1)
     ➔   Ezdfsfilehandler : "ezdfsfile" (expired=1)

                                                                                    26 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   How eZ expires the ENTIRE viewcache ?
       ➔   eZ does not recursively update / delete the files (too many files... unsafe)

      eZ set a global timestamp expiry value :
       ➔


    {VarDir}/cache/expiry.php ← 'content-view-cache' ( current timestamp)


   Each time the expiry.php is updated, the ENTIRE viewcache will
   expire.
   ➔ Ok, but my ENTIRE viewcache expires all the
   time... i did nothing special ! Don't know why !


                                                                                      27 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   Which event / trigger expires a set of
   cache ?
   By default : on object publication (or back-end feature)
    ➔ the related nodes & parent node




   Willingly, using a script :
    ➔ bin/php/ezcontentcache.php –clear-node=2

    ➔ bin/php/ezcontentcache.php –clear-node=2,46,63

    ➔ bin/php/ezcontentcache.php –clear-node=/company/about

    ➔ bin/php/ezcontentcache.php –clear-subtree=/company/about

    ➔ bin/php/ezcontentcache.php –clear-subtree=/company/about,/news




                                                                       28 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   Smart viewcache & cacheThreshold : a set
   of cache → the ENTIRE view cache
   Automatically, if using the smart view cache / viewcache.ini :

    ➔   The smart view cache populates a list of related « node_id » viewcache files
        to expire, depending of your settings. If this node_id list is larger than the
        CacheThreshold limit value (250 by default) → eZ expires the ENTIRE
        viewcache !

    ➔   Some settings could increase the node_id list count & excede the
        CacheThreshold limit : ALL, siblings, KeywordNodesCacheClearLimit

    ➔   Advice : Set KeywordNodesCacheClearLimit = 0, never use ALL, avoid
        siblings, log out the CacheThreshold (ezpEvent)

                                                                                  29 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   How to log the viewcache activity ? &
   prevent the CacheThreshold limit...

     Add an ezpEvent : site.ini.append.php

     [Event]
     Listeners[]=content/cache@log::logviewcache

     # log = your PHP class name
     # logviewcache = the static method



                                                     30 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   Log the viewcache node_id list to expire,
   prevent the CacheThreshold limit...
        ➔    Create a PHP class / generate autoloads
   // extension/extension-name/classes/log.php

   <?php
   class log
   {
       static public function logviewcache( $nodeList )
      {
                 $uri = $GLOBALS['_SERVER']['REQUEST_URI'];
                 eZLog::write('node_id count : '. count( $nodeList ). ' / node_id list : ' . implode(', ', $nodeList ) . ' / URI : '.$uri,
   'cachethresold.log');

                 return $nodeList;
        }}
   ?>


        ➔    Generate autoloads : php bin/php/ezpgenerateautolaods.php

                                                                                                                                             31 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   Log the viewcache node_id list to expire,
   prevent the CacheThreshold limit...
    ●   Edit / publish contents in the administration backend

    ➔   Log output : var/log/cachethreshold.log
        [ Sep 03 2012 17:37:33 ] node_id count : 14 / node_id list : 74, 73, 75, 76,
        79, 80, 81, 84, 85, 78, 77, 82, 72, 100 / URI :
        /administration/content/edit/72/2/eng-GB

        [ Sep 05 2012 13:44:54 ] node_id count : 7 / node_id list : 89, 88, 90, 91, 2,
        1, 87 / URI : /administration/content/edit/87/2/eng-GB

        [ Sep 05 2012 13:45:34 ] node_id count : 2 / node_id list : 2, 1 / URI :
        /administration/content/edit/57/2/eng-GB

                                                                                   32 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   Log the viewcache node_id list to expire,
   prevent the CacheThreshold limit...
    ●   Edit / publish contents in the administration backend

    ➔   Log output : var/log/cachethreshold.log
        [ Sep 03 2012 17:37:33 ] node_id count : 14 / node_id list : 74, 73, 75, 76,
        79, 80, 81, 84, 85, 78, 77, 82, 72, 100 / URI :
        /administration/content/edit/72/2/eng-GB

        [ Sep 05 2012 13:44:54 ] node_id count : 7 / node_id list : 89, 88, 90, 91, 2,
        1, 87 / URI : /administration/content/edit/87/2/eng-GB

        [ Sep 05 2012 13:45:34 ] node_id count : 2 / node_id list : 2, 1 / URI :
        /administration/content/edit/57/2/eng-GB

                                                                                   33 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   VIEWCACHE
   My ENTIRE viewcache still expire...
   Maybe you invoked one of these features (back-end, API)  :

    ➔   Create a new content class ( BUG ? )
    ➔   Update / delete an existing content class ( or attributes )
    ➔   Update / delete and existing role ( & policies ) OR role assignement
    ➔   Section assignement
    ➔   Update an ezshop rule ( discount rules, currency... )




                                                                               34 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS




                                                     35 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS




                                                     36 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE COMPILATION
   INI CACHE
   VIEWCACHE

   CACHE-BLOCK
       Concept / exploring the VAR
       Ex : HOWTO expire a named cache-block, related to the homepage
   expiry, or a contentclass, or whatever else
       Ex : HOWTO expire a set of cache-block using a shell script
       Ex : HOWTO build the perfect pagelayout

   EXPIRY.PHP FILE
   EZCACHE.PHP SCRIPT
   PHP CACHE
                                                                        37 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   ...                   What for ?
                           ➔   Template-block cache = cache files related to
                               cache-block, also called « template cache »
                               ( common mistake with the template
                               compilation )

                           ➔   Template-block cache is an HTML (or other)
                               cache of a template part ( cache-block )



                         ← Illustration : Łukasz Serwatka
                         http://share.ez.no/learn/ez-publish/ez-publish-performance-optimization-
                         part-3-of-3-practical-cache-and-template-solutions/(page)/2




                                                                                         38 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   Impact of common parameters
   The « keys » parameter :

   Please AVOID :
   {cache-block keys=$uri_string}     1 template-block file for
   ...                                each URI !!!
   {/cache-block}

   Use instead :
   {cache-block keys=$my_custom_limited_range_key}
   ...
   {/cache-block}
                        Only possible states of your content
                        inside your block
                                                                  39 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   Impact of common parameters
   The « URI key » parameter : stupid but signifiant simulation

    ➔   A classic Web site, with 1000 nodes, news folder ( year, month archives ),
        products catalog ( filters ), calendar, forum...
    ➔   Could generate 10 000 « indexable » several URI ( google bot )
    ➔   A cache-block ( only 1 URI key ) to cache the global menu, who is
        generating 300 SQL queries
    ➔   No « expiry » or « ignore_content_expiry » parameters : so the cache-block
        expires every 2 hours (or on content publication)

    ➔   So... 300 SQL queries * 12 expiration per day * 10 000 URI = 36 millions of
        SQL queries per day...



                                                                                40 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   Impact of common parameters
   The « expiry » parameter :

   If possible, always set a custom expiry value & ignore content
   expiry :
   {cache-block ignore_content_expiry expiry=86400}
   ...
   {/cache-block}




                                                                    41 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   Impact of common parameters
   The « sutree_expiry » parameter : no more expiry /
   ignore_content_expiry needed

   {cache-block subtree_expiry='news/'}
   ...
   {/cache-block}


   {cache-block subtree_expiry=142}
   ...
   {/cache-block}


                                                        42 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   Impact of common parameters
   The « sutree_expiry » parameter : tip of the « magic » node

    ➔   To expire a cache-block, depending of a custom rule ( per content class, per
        PHP algorithm ... )

    ➔   Create a {cache-block subtree_expiry=$magic_node_id}

    ➔   Create a node somewhere ( safe location, without FULL view )

    ➔   Create a workflow or customedithandler to include your custom rules on
        publication, and update the content object related to the magic node by the
        code



                                                                                43 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   Cache-block imbrication
   {cache-block expiry=3600}
   ...
       {cache-block ignore_content_expiry expiry=86400}

            ... {* you save … SQL Queries each hours *}

         {/cache-block}
   ...
         {cache-block subtree_expiry='news/'}
             ...
         {/cache-block}
   ...
   {/cache-block}



                                                          44 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   I need a « real time » block ( TTL=1m )
   A cache-block may not be the solution, use instead :

    ➔   ESI ( Edge Side Include ) : better for front-end loading & google
        bot indexing. But reverse-proxy required ( varnish, squid,
        akamai... )

    ➔   AJAX : easier to implement




                                                                      45 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   How eZ stores the template-block cache
   {VarDir}/cache/template-block/1/2/3/12345678.cache

   {VarDir}/cache/template-block/5/5/5/555666888.cache

   {VarDir}/cache/template-block/subtree/1/5/9/cache/1/2/3/12345678.cache


                           subtree_expiry=159                    CRC32
                                                             Complex... The
                                                          pathname doesn't help




                                                                                  46 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   How eZ stores the template-block cache
   {VarDir}/cache/template-block/1/2/3/12345678.cache

                                                      CRC32 ( 32-bit polynomial checksum )



      Cache-block position in the .tpl file / template path / keys /
                              siteaccess


          78_1_78_94_extension/ezwebin/design/ezwebin/templates/pagelayout.tpl

          78 = {cache-block... } line start
          1 = {cache-block... } column start (a simple ' ' before...)
          78 = {cache-block... } line end
          94 = {cache-block... } column end
                                                                                             47 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   How to expire the ALL of the template-
   block cache files ?
   php bin/php/ezcache.php –clear-id=template-block

    ➔   Expires all the template-block files, by updating the global
        cache-block expiry time ( global-template-block-cache ) in the
        {varDir}/expiry.php file




                                                                    48 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   How to expire a set of specific template-
   block cache files ?
    ➔   Not really possible... exept by searching a specific string inside
        each files, and deleting matching files ( ezfs / ezFS2 )

    ➔   find var/ezdemo_site/cache/template-block/ -name "*.cache" -exec grep -iHl
        "string-to-search" {} ; -delete

        {cache-block ignore_content_expiry expiry=0}
            <!-- string-to-search -->
            ...
        {/cache-block}


                                                                               49 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   CACHE-BLOCK
   PULL REQUEST IDEA :
      {cache-block id=myfolder ignore_content_expiry expiry=0}
      <!-- ID:BLOCK1 -->
          ...
      {/cache-block}


      {VarDir}/cache/template-block/myfolder/1/2/3/12345678.cache

   So... to expire your cache-block :
   mv myfolder myfolder-to-remove
   nice -n 20 rm -r myfolder-to-remove
                                                                    50 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   Ex : HOWTO expire a named cache-block, related to the
   homepage expiry, or a contentclass, or whatever else

   Ex : HOWTO expire a set of cache-block using a shell
   script

   Ex : HOWTO build the perfect pagelayout




                                                           51 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE COMPILATION
   INI CACHE
   VIEWCACHE
   CACHE-BLOCK

   EXPIRY.PHP FILE
      (not so) funny quiz

   EZCACHE.PHP SCRIPT
   PHP CACHE




                                                     52 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXPIRY.PHP FILE
   <?php
   $Timestamps = array (
                                                       What for ?
     'state-limitations' => 1321010927,
     'user-info-cache' => 1323788789,
     'content-view-cache' => 1326299075,
     'class-identifier-cache' => 1326298969,
     'global-template-block-cache' => 1325856140,
     'content-tree-menu' => 1326298969,
     'image-manager-alias' => 1323788788,
     'active-extensions-cache' => 1325668728,
     'ts-translation-cache' => 1323788789,
     'content-complex-viewmode-cache' => 1325780763,
     'template-block-cache' => 1326298969,
     'user-class-cache' => 1326298969,
     'sort-key-cache' => 1326298969,
   );




                                                                    53 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE COMPILATION
   INI CACHE
   VIEWCACHE
   CACHE-BLOCK
   EXPIRY.PHP FILE

   EZCACHE.PHP
   SCRIPT
      Concept / exploring the VAR
      Ex : HOWTO create a custom ezcache.php --id=myid


   PHP CACHE
                                                         54 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EZCACHE.PHP
   SCRIPT
   FAMOUS EZSCRIPT
    php bin/php/ezcache.php --clear-id=[ID]
    php bin/php/ezcache.php --clear-id=[ID] --purge
    php bin/php/ezcache.php --clear-tag=[TAG]
    php bin/php/ezcache.php --clear-tag=[TAG] --purge
    php bin/php/ezcache.php --clear-all
    php bin/php/ezcache.php --clear-all --purge

    NEED SOME HELP                             Does it do something
    php bin/php/ezcache.php --help                    special ?
    php bin/php/ezcache.php --list-ids        Superstitious parameter
    php bin/php/ezcache.php --list-tags
                                                                   55 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EZCACHE.PHP
   SCRIPT
   TAG    ID
   content            content / classid / sortey / urlalias / rss_cache /
                      content_tree_menu / state_limiations / template-block /
                      ezjscore-packer

   template           template / template-block / template-override /
                      texttoimage / design_base / ezjscore-packer

   ini                ini / global_ini / active_extensions

   user               user_info_cache
   I18n               translation / chartrans
   codepage           codepage
   image              imagealias
   rest               rest / rest-routes


                                                                           56 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EZCACHE.PHP
   SCRIPT
   MAIN ALGORITHM
    Hardcoded in kernel/classes/ezcache.php.
    For each ID :

    'id' => ID
    'tag' => array( TAG LIST )
    'enabled' => Cache enabled or not
    'is-clustered' => Use the current filehandler or not ?
    'path' => Path to purge, if purging (is-clustered=false)
    'expiry-key' => Key to set inside the expiry.php file if needed
    'function' => Function to apply only on clearing (not purging)
    'function-purge' => Function to apply only on purging (not clearing)

                                                                   57 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EZCACHE.PHP
   SCRIPT
   MAIN ALGORITHM
    Add your own ID for your extensions ( site.ini.append.php )

    # Cache item entry (for eZ Publish 4.3 and up)
    [Cache]
    CacheItems[]=ezjscore

    [Cache_ezjscore]
    name=eZJSCore Public Packer cache
    id=ezjscore-packer
    tags[]=content
    tags[]=template
    path=public
    isClustered=true

                                                                  58 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EXERCISE
   How to create a custom ID cache
   expiration  ?

   Use the ezbashcompletion extension :
   https://github.com/bdunogier/ezbashcompl
   etion




                                                     59 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   EZCACHE.PHP
   SCRIPT
   A BUG HELPS TO UNDERSTAND
    php bin/php/ezcache.php --clear-id=ini

    array( 'name' => ezpI18n::tr( 'kernel/cache', 'INI cache' ),
            'id' => 'ini',
            'tag' => array( 'ini' ),
            'enabled' => true,
            'path' => 'ini' ),

    Why is it not working ?



                                                                   60 / 61
:~$ eZ Publish Caching Mechanisms // SPICY DETAILS


   AGENDA
   TEMPLATE COMPILATION
   INI CACHE
   VIEWCACHE
   CACHE-BLOCK
   EXPIRY.PHP FILE
   EZCACHE.PHP SCRIPT

   PHP CACHE
      Ex : HOWTO generate a PHP based cache file




                                                     61 / 61

Workshop eZ Publish Caching Mechanisms

  • 1.
    eZ Summer Camp.Setp 2012 eZ Publish Caching Mechanisms // WORKSHOP Gilles Guirand - CTO Kaliop / eZ Publish board member @gandbox Gilles Ballini – Lead engineer Kaliop Stephane Sobecki - Lead engineer Kaliop Jerome Lecocq - Lead engineer Kaliop
  • 2.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA (1) TEMPLATE COMPILATION Concept / exploring the VAR Ex : HOWTO compile a set of template from a TXT file list INI CACHE Concept / exploring the VAR Ex : HOWTO use dynamic settings with shared compiled templates VIEWCACHE Concept / exploring the VAR Ex : HOWTO log the viewcache activity 2 / 61
  • 3.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA ( 2 ) CACHE-BLOCK Concept / exploring the VAR Ex : HOWTO expire a named cache-block, related to the homepage expiry, or a contentclass, or whatever else Ex : HOWTO expire a set of cache-block using a shell script Ex : HOWTO build the perfect pagelayout EXPIRY.PHP FILE Concept / exploring the VAR EZCACHE.PHP SCRIPT Concept / exploring the VAR Ex : HOWTO create a custom ezcache.php --id=myid PHP CACHE Ex : HOWTO generate a PHP based cache file 3 / 61
  • 4.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CONTEXT Common questions : ➔Is it possible to never expire the ENTIRE viewcache ? … and which actions actually expire the ENTIRE viewcache ? ➔ Is it possible to reduce the I/O (template-blocks) ? ➔ Is it possible to expire a specific set ot template-block files ? ➔ Is it possible to develop a better pagelayout.tpl ? ➔ Is something to improve on high-trafic websites ? Or Websites factory ( 100 siteaccess... ) 4 / 61
  • 5.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CONTEXT Final questions : ➔ How all these cache mechanisms really work ? … I mean in details... algorithms, I/O, triggers, exceptions, undocumented features & settings, unknown bugs, unknown fixes... and ezfs, ezfs2, ezdfs, ezdb... Final solutions : ➔ READ THE KERNEL / TEST THE KERNEL... ( be brave ) 5 / 61
  • 6.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VM PREPARATION Copy cache & cache2 extension folders Activate extension (site.ini.append.php) Generate autoloads Clear the cache-all Add user / login policy for anomymous ( site access cache & cache 2 ) 6 / 61
  • 7.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION Concept / exploring the VAR Ex : HOWTO compile a set of template from a TXT file list INI CACHE VIEWCACHE CACHE-BLOCK EXPIRY.PHP FILE EZCACHE.PHP SCRIPT PHP CACHE 7 / 61
  • 8.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS TPL COMPILATION What for ? ➔ Compile your .tpl files to .php files ➔ Reduce the RAM / CPU (use the stored compiled version) How to set ? site.ini [templatesettings] ➔ TemplateCompile=enabled ➔ Store compiled version {varDir}/cache/template/compiled/ ➔ NodeTreeCaching=enabled ( disabled by default ) ➔ Store pre-compiled version {varDir}/cache/template/tree/ Common mistake ➔ TemplateCache=enabled → template-block 8 / 61
  • 9.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS TPL COMPILATION DESIGN {varDir}/cache/template/tree {varDir}/cache/template/compiled page_mainarea.tpl {hash}-page_mainarea.php page_mainarea-{hash1}.php NodeTreeCaching=enabled page_mainarea-{hash2}.php ( disabled by default ) page_mainarea-{hash3}.php Why eZ generates several compiled files for the same .tpl? 9 / 61
  • 10.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS TPL COMPILATION How eZ stores the final compiled files ? {VarDir}/cache/template/compiled/, not clusterized Filenames : {templatename}-{hash}.php page_head-49f4458d0b68aec3cf2de63a6918fd61.php page_head-ec3cf2de63a6918fd6149f4458d0b68a.php 10 / 61
  • 11.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS TPL COMPILATION How eZ calculates the hash ? $sharedTemplates = false {VarDir}/cache/template/compiled/ Filenames : {templatename}-{hash}.php pagelayout-49f4458d0b68aec3cf2de63a6918fd61.php Md5( filepath ) / internalCharset / language / useFullUrlText / accessText / pageLayoutVariable / indexFile / extraName /layout/set/... 11 / 61
  • 12.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS TPL COMPILATION How eZ calculate the hash ? $sharedTemplates = true {VarDir}/cache/template/compiled/ Filenames : {templatename}-{hash}.php pagelayout-aec3cf2de63a6918fd6149f4458d0b68.php Md5( filepath ) / language No more siteaccess inside the key... Ezini operator → 'dynamic' parameter 12 / 61
  • 13.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE How to pre-compile a set of template files (to avoid : on the fly generating) ? ➔ Force the compilation of 2 template files php bin/php/eztc.php -s mysiteaccess --force extension/ezdemo/design/ezdemo/templates/page_head.tpl extension/ezdemo/design/ezdemo/templates/pagelayout.tpl ➔ Force the compilation of all template files starting with « page » : find extension/ezdemo/ -name "page_*.tpl" | xargs php bin/php/eztc.php -s eng --force ➔ Force the compilation of all template files from a TXT file : cat templates.txt | xargs php bin/php/eztc.php -s eng --force 13 / 61
  • 14.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS TPL COMPILATION The **** layout module ➔ By default ( $shareTemplates = false ), the 'layout' value is used by the final compiled filename ➔ So if you are using this module with 3 other layouts, you'll produce 4 compiled files (instead of 1) ➔ eztc.php script does not manage the layout parameter Your 3 additional compiled files ( for each layout ) will be procude « on the fly », and cannot be pre-compiled with eztc.php ➔ Suggestion : improve eztc.php, add the –layout=... parameter 14 / 61
  • 15.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION INI CACHE Concept / exploring the VAR Ex : HOWTO use dynamic settings with shared compiled templates VIEWCACHE CACHE-BLOCK EXPIRY.PHP FILE EZCACHE.PHP SCRIPT PHP CACHE 15 / 61
  • 16.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS INI CACHE When eZ generates the INI cache ? ➔ On the fly (if the cache file does not exist) – from an API call (ezini PHP class), a template call (ezini operator) ➔ By default, eZ automatically check if an INI file has been changed or not, depending of this setting (config.php) //define( 'EZP_INI_FILEMTIME_CHECK', false ) ; /* Set EZP_INI_FILEMTIME_CHECK constant to false to improve performance ( 10% ~ 15% on I/O ) by not checking modified time on ini files. You can also set it to a string, the name of a ini file you still want to check modified time on, best example would be to set it to 'site.ini' to make the system still check that but not the rest. */ 16 / 61
  • 17.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS INI CACHE How eZ stores the INI cache ? ➔ var/cache/ini ( and not /{varDir}/ini ), not clusterized ➔ {inifilename}-{hash}.php design-677247a49e428aa0837411b52777b920.php design-704216b4d0e3ea68c09742504bb366c8.php design-7333039e7e19aa411f8dd01836555861.php md5 FileName / RootDir / DirectAccess / overrideDirs / internalCharset 17 / 61
  • 18.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS INI CACHE How to expire the INI cache ? ➔php bin/php/ezcache.php --clear-id=global_ini ➔ Remove var/cache/ini ➔php bin/php/ezcache.php --clear-tag=ini ➔ Remove var/cache/ini & var/cache/active_extensions_{hash}.php ➔php bin/php/ezcache.php --clear-id=ini ➔ Nothing (bug) : try to recursively delete INI files in {varDir}/ini 18 / 61
  • 19.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS INI CACHE How to not compile the INI value ? lib/ezutils/classes/ezini.php function eZINI( $fileName = 'site.ini', $rootDir = '', $useTextCodec = null, $useCache = null, $useLocalOverrides = null, $directAccess = false, $addArrayDefinition = false, $load = true ) { ... } Dynamically load the INI values inside your compiled templates : site.ini [eZINISettings] / DynamicTemplateMode=enabled OR Ezini operator → 'dynamic' parameter 19 / 61
  • 20.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE HOWTO use dynamic settings with shared compiled templates 20 / 61
  • 21.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION INI CACHE VIEWCACHE Concept / exploring the VAR Ex : HOWTO log the viewcache activity CACHE-BLOCK EXPIRY.PHP FILE EZCACHE.PHP SCRIPT PHP CACHE 21 / 61
  • 22.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE What for ? ➔ Viewcache = content cache, content viewcache, … ➔ Viewcache is an HTML (or other) cache of the $module_result.content outpout, for a node view call ( content/view/... ) ← Illustration : Łukasz Serwatka http://share.ez.no/learn/ez-publish/ez-publish-performance-optimization- part-3-of-3-practical-cache-and-template-solutions/(page)/2 22 / 61
  • 23.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE How eZ stores the viewcache ? {VarDir}/cache/content/{siteaccess}/{number}/{number}/{number} {VarDir}/cache/content/{siteaccess}/ 2-ce182234bd70236c6a009578d2a34632.cache 20-ce182234bd70236c6a009578d2a34632.cache {VarDir}/cache/content/{siteaccess}/1 101-ce182234bd70236c6a009578d2a34632.cache 105-ce182234bd70236c6a009578d2a34632.cache {VarDir}/cache/content/{siteaccess}/5/2/1 52101-ce182234bd70236c6a009578d2a34632.cache 52144-ce182234bd70236c6a009578d2a34632.cache 23 / 61
  • 24.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE How eZ stores the viewcache ? {nodeid}-{hash}.php 2-ce182234bd70236c6a009578d2a34632.cache md5 required nodeID / viewMode / layout / language (view.p.) / offset (view.p.) / indexFile optionnal + roIeIDList / limitValueList / discountList / cacheNameExtra / pr_user / viewParameters / userPreferences ViewCacheTweaks[<node_id>]=<setting>[;<setting2>] ViewCacheTweaks[global]=<setting>[;<setting2>] 24 / 61
  • 25.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE How to know the exact matching between a viewcache file & my URLs ? {nodeid}-{hash}.php → deduce the nodeID At the end of file : "view_parameters";a:6: {s:6:"offset";b:0;s:4:"year";b:0;s:5:"month";b:0;s:3:"day";b:0;s:10:"na mefilter";b:0;s:4:"test";s:1:"2";}... Should be URL : /(test)/2 25 / 61
  • 26.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE How eZ expires a set of viewcache ? Alteration of cache states - Depending of the filehandler ( file.ini ) ➔ Ezfsfilehandler : UNLINK file ➔ Ezfs2filehandler : TOUCH file (25/25/1977) ← Star Wars day ➔ Ezdbfilehandler : "ezdbfile" (UPDATE expired=1) ➔ Ezdfsfilehandler : "ezdfsfile" (UPDATE expired=1) Comparison of cache states - Depending of the filehandler ( file.ini ) ➔ Ezfsfilehandler : file does not exist, or is older than the content-view-cache timestamp ➔ Ezfs2filehandler : file is older than the content-view-cache timestamp ➔ Ezdbfilehandler : "ezdbfile" (expired=1) ➔ Ezdfsfilehandler : "ezdfsfile" (expired=1) 26 / 61
  • 27.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE How eZ expires the ENTIRE viewcache ? ➔ eZ does not recursively update / delete the files (too many files... unsafe) eZ set a global timestamp expiry value : ➔ {VarDir}/cache/expiry.php ← 'content-view-cache' ( current timestamp) Each time the expiry.php is updated, the ENTIRE viewcache will expire. ➔ Ok, but my ENTIRE viewcache expires all the time... i did nothing special ! Don't know why ! 27 / 61
  • 28.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE Which event / trigger expires a set of cache ? By default : on object publication (or back-end feature) ➔ the related nodes & parent node Willingly, using a script : ➔ bin/php/ezcontentcache.php –clear-node=2 ➔ bin/php/ezcontentcache.php –clear-node=2,46,63 ➔ bin/php/ezcontentcache.php –clear-node=/company/about ➔ bin/php/ezcontentcache.php –clear-subtree=/company/about ➔ bin/php/ezcontentcache.php –clear-subtree=/company/about,/news 28 / 61
  • 29.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE Smart viewcache & cacheThreshold : a set of cache → the ENTIRE view cache Automatically, if using the smart view cache / viewcache.ini : ➔ The smart view cache populates a list of related « node_id » viewcache files to expire, depending of your settings. If this node_id list is larger than the CacheThreshold limit value (250 by default) → eZ expires the ENTIRE viewcache ! ➔ Some settings could increase the node_id list count & excede the CacheThreshold limit : ALL, siblings, KeywordNodesCacheClearLimit ➔ Advice : Set KeywordNodesCacheClearLimit = 0, never use ALL, avoid siblings, log out the CacheThreshold (ezpEvent) 29 / 61
  • 30.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE How to log the viewcache activity ? & prevent the CacheThreshold limit... Add an ezpEvent : site.ini.append.php [Event] Listeners[]=content/cache@log::logviewcache # log = your PHP class name # logviewcache = the static method 30 / 61
  • 31.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE Log the viewcache node_id list to expire, prevent the CacheThreshold limit... ➔ Create a PHP class / generate autoloads // extension/extension-name/classes/log.php <?php class log { static public function logviewcache( $nodeList ) { $uri = $GLOBALS['_SERVER']['REQUEST_URI']; eZLog::write('node_id count : '. count( $nodeList ). ' / node_id list : ' . implode(', ', $nodeList ) . ' / URI : '.$uri, 'cachethresold.log'); return $nodeList; }} ?> ➔ Generate autoloads : php bin/php/ezpgenerateautolaods.php 31 / 61
  • 32.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE Log the viewcache node_id list to expire, prevent the CacheThreshold limit... ● Edit / publish contents in the administration backend ➔ Log output : var/log/cachethreshold.log [ Sep 03 2012 17:37:33 ] node_id count : 14 / node_id list : 74, 73, 75, 76, 79, 80, 81, 84, 85, 78, 77, 82, 72, 100 / URI : /administration/content/edit/72/2/eng-GB [ Sep 05 2012 13:44:54 ] node_id count : 7 / node_id list : 89, 88, 90, 91, 2, 1, 87 / URI : /administration/content/edit/87/2/eng-GB [ Sep 05 2012 13:45:34 ] node_id count : 2 / node_id list : 2, 1 / URI : /administration/content/edit/57/2/eng-GB 32 / 61
  • 33.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE Log the viewcache node_id list to expire, prevent the CacheThreshold limit... ● Edit / publish contents in the administration backend ➔ Log output : var/log/cachethreshold.log [ Sep 03 2012 17:37:33 ] node_id count : 14 / node_id list : 74, 73, 75, 76, 79, 80, 81, 84, 85, 78, 77, 82, 72, 100 / URI : /administration/content/edit/72/2/eng-GB [ Sep 05 2012 13:44:54 ] node_id count : 7 / node_id list : 89, 88, 90, 91, 2, 1, 87 / URI : /administration/content/edit/87/2/eng-GB [ Sep 05 2012 13:45:34 ] node_id count : 2 / node_id list : 2, 1 / URI : /administration/content/edit/57/2/eng-GB 33 / 61
  • 34.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS VIEWCACHE My ENTIRE viewcache still expire... Maybe you invoked one of these features (back-end, API)  : ➔ Create a new content class ( BUG ? ) ➔ Update / delete an existing content class ( or attributes ) ➔ Update / delete and existing role ( & policies ) OR role assignement ➔ Section assignement ➔ Update an ezshop rule ( discount rules, currency... ) 34 / 61
  • 35.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS 35 / 61
  • 36.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS 36 / 61
  • 37.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION INI CACHE VIEWCACHE CACHE-BLOCK Concept / exploring the VAR Ex : HOWTO expire a named cache-block, related to the homepage expiry, or a contentclass, or whatever else Ex : HOWTO expire a set of cache-block using a shell script Ex : HOWTO build the perfect pagelayout EXPIRY.PHP FILE EZCACHE.PHP SCRIPT PHP CACHE 37 / 61
  • 38.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK ... What for ? ➔ Template-block cache = cache files related to cache-block, also called « template cache » ( common mistake with the template compilation ) ➔ Template-block cache is an HTML (or other) cache of a template part ( cache-block ) ← Illustration : Łukasz Serwatka http://share.ez.no/learn/ez-publish/ez-publish-performance-optimization- part-3-of-3-practical-cache-and-template-solutions/(page)/2 38 / 61
  • 39.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK Impact of common parameters The « keys » parameter : Please AVOID : {cache-block keys=$uri_string} 1 template-block file for ... each URI !!! {/cache-block} Use instead : {cache-block keys=$my_custom_limited_range_key} ... {/cache-block} Only possible states of your content inside your block 39 / 61
  • 40.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK Impact of common parameters The « URI key » parameter : stupid but signifiant simulation ➔ A classic Web site, with 1000 nodes, news folder ( year, month archives ), products catalog ( filters ), calendar, forum... ➔ Could generate 10 000 « indexable » several URI ( google bot ) ➔ A cache-block ( only 1 URI key ) to cache the global menu, who is generating 300 SQL queries ➔ No « expiry » or « ignore_content_expiry » parameters : so the cache-block expires every 2 hours (or on content publication) ➔ So... 300 SQL queries * 12 expiration per day * 10 000 URI = 36 millions of SQL queries per day... 40 / 61
  • 41.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK Impact of common parameters The « expiry » parameter : If possible, always set a custom expiry value & ignore content expiry : {cache-block ignore_content_expiry expiry=86400} ... {/cache-block} 41 / 61
  • 42.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK Impact of common parameters The « sutree_expiry » parameter : no more expiry / ignore_content_expiry needed {cache-block subtree_expiry='news/'} ... {/cache-block} {cache-block subtree_expiry=142} ... {/cache-block} 42 / 61
  • 43.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK Impact of common parameters The « sutree_expiry » parameter : tip of the « magic » node ➔ To expire a cache-block, depending of a custom rule ( per content class, per PHP algorithm ... ) ➔ Create a {cache-block subtree_expiry=$magic_node_id} ➔ Create a node somewhere ( safe location, without FULL view ) ➔ Create a workflow or customedithandler to include your custom rules on publication, and update the content object related to the magic node by the code 43 / 61
  • 44.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK Cache-block imbrication {cache-block expiry=3600} ... {cache-block ignore_content_expiry expiry=86400} ... {* you save … SQL Queries each hours *} {/cache-block} ... {cache-block subtree_expiry='news/'} ... {/cache-block} ... {/cache-block} 44 / 61
  • 45.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK I need a « real time » block ( TTL=1m ) A cache-block may not be the solution, use instead : ➔ ESI ( Edge Side Include ) : better for front-end loading & google bot indexing. But reverse-proxy required ( varnish, squid, akamai... ) ➔ AJAX : easier to implement 45 / 61
  • 46.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK How eZ stores the template-block cache {VarDir}/cache/template-block/1/2/3/12345678.cache {VarDir}/cache/template-block/5/5/5/555666888.cache {VarDir}/cache/template-block/subtree/1/5/9/cache/1/2/3/12345678.cache subtree_expiry=159 CRC32 Complex... The pathname doesn't help 46 / 61
  • 47.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK How eZ stores the template-block cache {VarDir}/cache/template-block/1/2/3/12345678.cache CRC32 ( 32-bit polynomial checksum ) Cache-block position in the .tpl file / template path / keys / siteaccess 78_1_78_94_extension/ezwebin/design/ezwebin/templates/pagelayout.tpl 78 = {cache-block... } line start 1 = {cache-block... } column start (a simple ' ' before...) 78 = {cache-block... } line end 94 = {cache-block... } column end 47 / 61
  • 48.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK How to expire the ALL of the template- block cache files ? php bin/php/ezcache.php –clear-id=template-block ➔ Expires all the template-block files, by updating the global cache-block expiry time ( global-template-block-cache ) in the {varDir}/expiry.php file 48 / 61
  • 49.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK How to expire a set of specific template- block cache files ? ➔ Not really possible... exept by searching a specific string inside each files, and deleting matching files ( ezfs / ezFS2 ) ➔ find var/ezdemo_site/cache/template-block/ -name "*.cache" -exec grep -iHl "string-to-search" {} ; -delete {cache-block ignore_content_expiry expiry=0} <!-- string-to-search --> ... {/cache-block} 49 / 61
  • 50.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS CACHE-BLOCK PULL REQUEST IDEA : {cache-block id=myfolder ignore_content_expiry expiry=0} <!-- ID:BLOCK1 --> ... {/cache-block} {VarDir}/cache/template-block/myfolder/1/2/3/12345678.cache So... to expire your cache-block : mv myfolder myfolder-to-remove nice -n 20 rm -r myfolder-to-remove 50 / 61
  • 51.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE Ex : HOWTO expire a named cache-block, related to the homepage expiry, or a contentclass, or whatever else Ex : HOWTO expire a set of cache-block using a shell script Ex : HOWTO build the perfect pagelayout 51 / 61
  • 52.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION INI CACHE VIEWCACHE CACHE-BLOCK EXPIRY.PHP FILE (not so) funny quiz EZCACHE.PHP SCRIPT PHP CACHE 52 / 61
  • 53.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXPIRY.PHP FILE <?php $Timestamps = array ( What for ? 'state-limitations' => 1321010927, 'user-info-cache' => 1323788789, 'content-view-cache' => 1326299075, 'class-identifier-cache' => 1326298969, 'global-template-block-cache' => 1325856140, 'content-tree-menu' => 1326298969, 'image-manager-alias' => 1323788788, 'active-extensions-cache' => 1325668728, 'ts-translation-cache' => 1323788789, 'content-complex-viewmode-cache' => 1325780763, 'template-block-cache' => 1326298969, 'user-class-cache' => 1326298969, 'sort-key-cache' => 1326298969, ); 53 / 61
  • 54.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION INI CACHE VIEWCACHE CACHE-BLOCK EXPIRY.PHP FILE EZCACHE.PHP SCRIPT Concept / exploring the VAR Ex : HOWTO create a custom ezcache.php --id=myid PHP CACHE 54 / 61
  • 55.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EZCACHE.PHP SCRIPT FAMOUS EZSCRIPT php bin/php/ezcache.php --clear-id=[ID] php bin/php/ezcache.php --clear-id=[ID] --purge php bin/php/ezcache.php --clear-tag=[TAG] php bin/php/ezcache.php --clear-tag=[TAG] --purge php bin/php/ezcache.php --clear-all php bin/php/ezcache.php --clear-all --purge NEED SOME HELP Does it do something php bin/php/ezcache.php --help special ? php bin/php/ezcache.php --list-ids Superstitious parameter php bin/php/ezcache.php --list-tags 55 / 61
  • 56.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EZCACHE.PHP SCRIPT TAG ID content content / classid / sortey / urlalias / rss_cache / content_tree_menu / state_limiations / template-block / ezjscore-packer template template / template-block / template-override / texttoimage / design_base / ezjscore-packer ini ini / global_ini / active_extensions user user_info_cache I18n translation / chartrans codepage codepage image imagealias rest rest / rest-routes 56 / 61
  • 57.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EZCACHE.PHP SCRIPT MAIN ALGORITHM Hardcoded in kernel/classes/ezcache.php. For each ID : 'id' => ID 'tag' => array( TAG LIST ) 'enabled' => Cache enabled or not 'is-clustered' => Use the current filehandler or not ? 'path' => Path to purge, if purging (is-clustered=false) 'expiry-key' => Key to set inside the expiry.php file if needed 'function' => Function to apply only on clearing (not purging) 'function-purge' => Function to apply only on purging (not clearing) 57 / 61
  • 58.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EZCACHE.PHP SCRIPT MAIN ALGORITHM Add your own ID for your extensions ( site.ini.append.php ) # Cache item entry (for eZ Publish 4.3 and up) [Cache] CacheItems[]=ezjscore [Cache_ezjscore] name=eZJSCore Public Packer cache id=ezjscore-packer tags[]=content tags[]=template path=public isClustered=true 58 / 61
  • 59.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EXERCISE How to create a custom ID cache expiration  ? Use the ezbashcompletion extension : https://github.com/bdunogier/ezbashcompl etion 59 / 61
  • 60.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS EZCACHE.PHP SCRIPT A BUG HELPS TO UNDERSTAND php bin/php/ezcache.php --clear-id=ini array( 'name' => ezpI18n::tr( 'kernel/cache', 'INI cache' ), 'id' => 'ini', 'tag' => array( 'ini' ), 'enabled' => true, 'path' => 'ini' ), Why is it not working ? 60 / 61
  • 61.
    :~$ eZ PublishCaching Mechanisms // SPICY DETAILS AGENDA TEMPLATE COMPILATION INI CACHE VIEWCACHE CACHE-BLOCK EXPIRY.PHP FILE EZCACHE.PHP SCRIPT PHP CACHE Ex : HOWTO generate a PHP based cache file 61 / 61