SlideShare a Scribd company logo
eZ Cluster unleashed
The return
Purpose and history

One eZ Publish on multiple servers, for:
Performance
Redundancy

Created for eZ Publish 3.8, vastly improved since then:
3.10: huge schema & feature improvements
4.2: new eZDFS cluster handler for NFS
4.2: Stalecaching mechanism for eZDB and eZDFS
4.7: Refactored binary index architecture

Matches the shared database for filesystem operations:
Inserting new data, expiring/deleting obsolete data
serving files through HTTP
Cluster handlers



           eZ DB                        eZ DFS

From eZ Publish 3.8           From eZ Publish 4.2

Stores metadata in DB         Stores metadata in DB

Stores data in DB (BLOBs)     Stores data on NFS

Easy to setup                 Requires an NFS server

Huge DB, harder to maintain   Maintenance is much easier
HTTP vs Internal calls

HTTP

Requests done through index_cluster.php
Serves files directly
A reverse proxy caches images & binary files

INTERNAL / KERNEL CALLS

Reads/writes cache items
Stores binary files
Configuration #1: file.ini

A cluster handler must be configured in a file.ini override:

1.[ClusteringSettings]
2.FileHandler=eZDFSFileHandler


1.# DFS Cluster Handler settings
2.[eZDFSClusteringSettings]
3.MountPointPath=var/nfs
4.DBBackend=eZDFSFileHandlerMySQLBackend
5.DBHost=localhost
6.DBName=labs_ezpublish_trunk_cluster
7.DBUser=root
8.DBPassword=root
Configuration #2: database


eZDB and eZDFS both require database tables

The same DB server can be used

Performance wise it is better to use another server

Schemas are available in
kernel/sql/*/cluster_*_schema.sql (mysql only, oracle's are in
the extension)
Configuration #3: clusterize


Existing local files must be moved to the cluster

This  is done using bin/php/clusterize.php

This script will copy/move images & binary files from the local
filesystem to the cluster

It must only be executed once for the whole cluster
Configuration #4: binary URLs rewriting


URL rewriting is required to "stream" clusterized binary files to
visitors:
   Images
   Binary files

They only affect known path for these types of files

To stream different files, new rules must be added

     RewriteEngine On
     Rewriterule ^/var/([^/]+/)?storage/original/.* /index_cluster.php [L]
     Rewriterule ^/var/([^/]+/)?storage/images/.*   /index_cluster.php [L]

     # Standard eZ Publish rewrite rules
     # ...
Configuration #5: binary files index


The cluster index handles binary files HTTP requests

It doesn't use the whole kernel in order to perform better
It was completely refactored in 4.7

Before 4.7                              After 4.7
Required custom file to be created      No custom files


Many files also located at root level   Documented



Duplicated logic among handlers         Common code has been refactored



Features weren't identical among        Features are (almost) perfectly
Configuration #5: binary files index, example

/**
 * Cluster configuration file
 */
define( 'CLUSTER_STORAGE_BACKEND',    'dfsmysqli'     );
define( 'CLUSTER_STORAGE_HOST',       'localhost'     );
define( 'CLUSTER_STORAGE_PORT',       3306            );
define( 'CLUSTER_STORAGE_USER',       'clusteruser'   );
define( 'CLUSTER_STORAGE_PASS',       ''              );
define( 'CLUSTER_STORAGE_DB',         'ezcluster'     );
define( 'CLUSTER_STORAGE_CHARSET',    'utf8'          );
define( 'CLUSTER_MOUNT_POINT_PATH',   '/opt/nfs/ez'   );
How cache is handled


Remote, clusterized files are copied locally

Before using a local file, eZ always checks if it is newer
than the remote one

When a cache file is generated, it is stored on the cluster,
and re-used by other nodes

expiry.php provides an intermediate method
API: reading clusterized files

1.$path = '/var/ezwebin_site/path/to/my/file.txt'


1.// read a text file, "instance" syntax
2.$contents = eZClusterFileHandler::instance( $path )->fetchContents();


1.// read a text file, "static" syntax
2.$contents = eZClusterFileHandler::instance()->fileFetchContents( $path );


1.// fetch a file (a binary one for example) to disk
2.$path = 'var/ezflow_site/storage/original/video/961d8a65efffdd93708cc23bc6398953.flv';
3.$handler = eZClusterFileHandler::instance( $path )->fetchUnique();


1.// ... use the file ... then delete it
2.$handler->deleteLocal( $uniquePath );


1.// reading metadata
2.$file = eZClusterFileHandler::instance( $someFile );
3.echo $file->size();
4.echo $file->mtime();
API: writing a file to the cluster


Again, the native PHP I/O API isn't cluster aware !
1.// stores the contents of the $contents variable to cluster
2.$path = 'var/ezwebin_site/file.txt';
3.$handler = eZClusterFileHandler::instance( $path );
4.$handler->storeContents( $contents, 'text', 'text/plain' );


1.// alternative "static" method: fileStoreContents()


1.// stores the contents of a local image as a clusterized file
2.$imagePath = 'var/ezwebin_site/path/image.png';
3.$handler = eZClusterFileHandler::instance();
4.$handler->fileStore( $imagePath, 'image', true, 'image/png' );
API: ViewCache !


ViewCache uses advanced methods of the cluster API

These method handle:
      
        cache retrieving and/or generation
      
        Concurrency
      
        stale caching

It can technically be used to manage custom cache !

Next: introducing the processCache() method
API: The processCache() method !


It can be used for custom developements too !

It will let you implement caching in your own extensions

It uses:
       
         a generate() callback that generates the dynamic
       content
       
         a retrieve() callback that retrieves valid cached
       content
API: The processCache() method, CODE !

1.$cacheFilePath = eZCachedModule::cacheFilePath( $Params );
2.$cacheFile = eZClusterFileHandler::instance( $cacheFilePath );
3.$Result = $cacheFile->processCache(
4.    array( 'eZCachedModule', 'retrieve' ),
5.    array( 'eZCachedModule', 'generate' ),
6.    null,
7.    null,
8.    compact( 'Params' ) );

1.return $Result;
API: The processCache() method, callback


Let's review an example that caches the whole output
($Result) of a module.
It has a configurable TTL, and provides a custom cache key
that can be used to invalidate cache.

1. Get the code from github:
git clone git://gist.github.com/3635993.git extension/ezcachedmodule


2. enable the extension in site.ini
ActiveExtensions[]=ezcachedmodule


3. and do not forget to regenerate the extensions autoload:
Php bin/php/ezpgenerateautoloads.php –extension
API: The processCache() method, callback


1. Enable the DebugOutput
  Come on, you know how to do it


2. Go to http://ezpublish4.admin/cachedmodule/view/abcd
  Look into the debugOutput for « eZCachedModule »


3. Let's review it together
API: The processCache() method, callback


Let's review an example that caches the whole output
($Result) of a module.
It has a configurable TTL, and provides a custom cache key
that can be used to invalidate cache.

1. Get the code from github:
git clone git://gist.github.com/3635993.git extension/ezcachedmodule


2. enable the extension in site.ini
ActiveExtensions[]=ezcachedmodule


3. and do not forget to regenerate the extensions autoload:
Php bin/php/ezpgenerateautoloads.php –extension

More Related Content

What's hot

On secure application of PHP wrappers
On secure application  of PHP wrappersOn secure application  of PHP wrappers
On secure application of PHP wrappers
Positive Hack Days
 
20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag
garrett honeycutt
 
2015 bioinformatics bio_python
2015 bioinformatics bio_python2015 bioinformatics bio_python
2015 bioinformatics bio_python
Prof. Wim Van Criekinge
 
Puppet HackDay/BarCamp New Delhi Exercises
Puppet HackDay/BarCamp New Delhi ExercisesPuppet HackDay/BarCamp New Delhi Exercises
Puppet HackDay/BarCamp New Delhi Exercises
Julie Tsai
 
HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係
Kiwamu Okabe
 
working with files
working with filesworking with files
working with files
SangeethaSasi1
 
Shark - Lab Assignment
Shark - Lab AssignmentShark - Lab Assignment
Shark - Lab Assignment
Farzad Nozarian
 
Linux configer
Linux configerLinux configer
Linux configer
MD. AL AMIN
 
Tutorial Puppet
Tutorial PuppetTutorial Puppet
Tutorial Puppet
Daniel Sobral
 
Functional Hostnames and Why they are Bad
Functional Hostnames and Why they are BadFunctional Hostnames and Why they are Bad
Functional Hostnames and Why they are Bad
Puppet
 
Linux Kernel - Virtual File System
Linux Kernel - Virtual File SystemLinux Kernel - Virtual File System
Linux Kernel - Virtual File System
Adrian Huang
 
Making Your Capistrano Recipe Book
Making Your Capistrano Recipe BookMaking Your Capistrano Recipe Book
Making Your Capistrano Recipe Book
Tim Riley
 
Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)
Felix Geisendörfer
 
PuppetCamp SEA 1 - Version Control with Puppet
PuppetCamp SEA 1 - Version Control with PuppetPuppetCamp SEA 1 - Version Control with Puppet
PuppetCamp SEA 1 - Version Control with Puppet
Walter Heck
 
Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)
Dominic Cleal
 
Page cache in Linux kernel
Page cache in Linux kernelPage cache in Linux kernel
Page cache in Linux kernel
Adrian Huang
 
ProxySQL & PXC(Query routing and Failover Test)
ProxySQL & PXC(Query routing and Failover Test)ProxySQL & PXC(Query routing and Failover Test)
ProxySQL & PXC(Query routing and Failover Test)
YoungHeon (Roy) Kim
 
занятие8
занятие8занятие8
занятие8
Oleg Parinov
 
Building File Systems with FUSE
Building File Systems with FUSEBuilding File Systems with FUSE
Building File Systems with FUSE
elliando dias
 
Lightweight DAS components in Perl
Lightweight DAS components in PerlLightweight DAS components in Perl
Lightweight DAS components in Perl
guestbab097
 

What's hot (20)

On secure application of PHP wrappers
On secure application  of PHP wrappersOn secure application  of PHP wrappers
On secure application of PHP wrappers
 
20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag20090514 Introducing Puppet To Sasag
20090514 Introducing Puppet To Sasag
 
2015 bioinformatics bio_python
2015 bioinformatics bio_python2015 bioinformatics bio_python
2015 bioinformatics bio_python
 
Puppet HackDay/BarCamp New Delhi Exercises
Puppet HackDay/BarCamp New Delhi ExercisesPuppet HackDay/BarCamp New Delhi Exercises
Puppet HackDay/BarCamp New Delhi Exercises
 
HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係HaskellとDebianの辛くて甘い関係
HaskellとDebianの辛くて甘い関係
 
working with files
working with filesworking with files
working with files
 
Shark - Lab Assignment
Shark - Lab AssignmentShark - Lab Assignment
Shark - Lab Assignment
 
Linux configer
Linux configerLinux configer
Linux configer
 
Tutorial Puppet
Tutorial PuppetTutorial Puppet
Tutorial Puppet
 
Functional Hostnames and Why they are Bad
Functional Hostnames and Why they are BadFunctional Hostnames and Why they are Bad
Functional Hostnames and Why they are Bad
 
Linux Kernel - Virtual File System
Linux Kernel - Virtual File SystemLinux Kernel - Virtual File System
Linux Kernel - Virtual File System
 
Making Your Capistrano Recipe Book
Making Your Capistrano Recipe BookMaking Your Capistrano Recipe Book
Making Your Capistrano Recipe Book
 
Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)
 
PuppetCamp SEA 1 - Version Control with Puppet
PuppetCamp SEA 1 - Version Control with PuppetPuppetCamp SEA 1 - Version Control with Puppet
PuppetCamp SEA 1 - Version Control with Puppet
 
Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)
 
Page cache in Linux kernel
Page cache in Linux kernelPage cache in Linux kernel
Page cache in Linux kernel
 
ProxySQL & PXC(Query routing and Failover Test)
ProxySQL & PXC(Query routing and Failover Test)ProxySQL & PXC(Query routing and Failover Test)
ProxySQL & PXC(Query routing and Failover Test)
 
занятие8
занятие8занятие8
занятие8
 
Building File Systems with FUSE
Building File Systems with FUSEBuilding File Systems with FUSE
Building File Systems with FUSE
 
Lightweight DAS components in Perl
Lightweight DAS components in PerlLightweight DAS components in Perl
Lightweight DAS components in Perl
 

Similar to eZ Publish cluster unleashed revisited

Ansible ex407 and EX 294
Ansible ex407 and EX 294Ansible ex407 and EX 294
Ansible ex407 and EX 294
IkiArif1
 
Symfony internals [english]
Symfony internals [english]Symfony internals [english]
Symfony internals [english]
Raul Fraile
 
Zend_Cache: how to improve the performance of PHP applications
Zend_Cache: how to improve the performance of PHP applicationsZend_Cache: how to improve the performance of PHP applications
Zend_Cache: how to improve the performance of PHP applications
Enrico Zimuel
 
Best Practices For Direct Admin Security
Best Practices For Direct Admin SecurityBest Practices For Direct Admin Security
Best Practices For Direct Admin Security
lisa Dsouza
 
Learning Puppet basic thing
Learning Puppet basic thing Learning Puppet basic thing
Learning Puppet basic thing
DaeHyung Lee
 
Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache Mesos
Joe Stein
 
Boost Your Environment With XMLDB - UKOUG 2008 - Marco Gralike
Boost Your Environment With XMLDB - UKOUG 2008 - Marco GralikeBoost Your Environment With XMLDB - UKOUG 2008 - Marco Gralike
Boost Your Environment With XMLDB - UKOUG 2008 - Marco Gralike
Marco Gralike
 
Workshop eZ Publish Caching Mechanisms
Workshop eZ Publish Caching MechanismsWorkshop eZ Publish Caching Mechanisms
Workshop eZ Publish Caching Mechanisms
Kaliop-slide
 
Linux filesystemhierarchy
Linux filesystemhierarchyLinux filesystemhierarchy
Linux filesystemhierarchy
Dr. C.V. Suresh Babu
 
Decoupled Libraries for PHP
Decoupled Libraries for PHPDecoupled Libraries for PHP
Decoupled Libraries for PHP
Paul Jones
 
DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...
DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...
DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...
DataStax
 
Docker Security Paradigm
Docker Security ParadigmDocker Security Paradigm
Docker Security Paradigm
Anis LARGUEM
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013
grim_radical
 
Softshake - Offline applications
Softshake - Offline applicationsSoftshake - Offline applications
Softshake - Offline applications
jeromevdl
 
Oracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid cloneOracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid clone
Deepti Singh
 
Oracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid cloneOracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid clone
Deepti Singh
 
Android Data Storagefinal
Android Data StoragefinalAndroid Data Storagefinal
Android Data Storagefinal
Nakka Srilakshmi
 
Drupal 8 configuration management
Drupal 8 configuration managementDrupal 8 configuration management
Drupal 8 configuration management
Alexander Tkachev
 
Rman cloning guide
Rman cloning guideRman cloning guide
Rman cloning guide
Amit87_dba
 
Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023
Mark Niebergall
 

Similar to eZ Publish cluster unleashed revisited (20)

Ansible ex407 and EX 294
Ansible ex407 and EX 294Ansible ex407 and EX 294
Ansible ex407 and EX 294
 
Symfony internals [english]
Symfony internals [english]Symfony internals [english]
Symfony internals [english]
 
Zend_Cache: how to improve the performance of PHP applications
Zend_Cache: how to improve the performance of PHP applicationsZend_Cache: how to improve the performance of PHP applications
Zend_Cache: how to improve the performance of PHP applications
 
Best Practices For Direct Admin Security
Best Practices For Direct Admin SecurityBest Practices For Direct Admin Security
Best Practices For Direct Admin Security
 
Learning Puppet basic thing
Learning Puppet basic thing Learning Puppet basic thing
Learning Puppet basic thing
 
Introduction to Apache Mesos
Introduction to Apache MesosIntroduction to Apache Mesos
Introduction to Apache Mesos
 
Boost Your Environment With XMLDB - UKOUG 2008 - Marco Gralike
Boost Your Environment With XMLDB - UKOUG 2008 - Marco GralikeBoost Your Environment With XMLDB - UKOUG 2008 - Marco Gralike
Boost Your Environment With XMLDB - UKOUG 2008 - Marco Gralike
 
Workshop eZ Publish Caching Mechanisms
Workshop eZ Publish Caching MechanismsWorkshop eZ Publish Caching Mechanisms
Workshop eZ Publish Caching Mechanisms
 
Linux filesystemhierarchy
Linux filesystemhierarchyLinux filesystemhierarchy
Linux filesystemhierarchy
 
Decoupled Libraries for PHP
Decoupled Libraries for PHPDecoupled Libraries for PHP
Decoupled Libraries for PHP
 
DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...
DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...
DataStax | Building a Spark Streaming App with DSE File System (Rocco Varela)...
 
Docker Security Paradigm
Docker Security ParadigmDocker Security Paradigm
Docker Security Paradigm
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013
 
Softshake - Offline applications
Softshake - Offline applicationsSoftshake - Offline applications
Softshake - Offline applications
 
Oracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid cloneOracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid clone
 
Oracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid cloneOracle applications 11i hot backup cloning with rapid clone
Oracle applications 11i hot backup cloning with rapid clone
 
Android Data Storagefinal
Android Data StoragefinalAndroid Data Storagefinal
Android Data Storagefinal
 
Drupal 8 configuration management
Drupal 8 configuration managementDrupal 8 configuration management
Drupal 8 configuration management
 
Rman cloning guide
Rman cloning guideRman cloning guide
Rman cloning guide
 
Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023Filesystem Management with Flysystem - php[tek] 2023
Filesystem Management with Flysystem - php[tek] 2023
 

More from Bertrand Dunogier

The eZ Platform Query Field
The eZ Platform Query FieldThe eZ Platform Query Field
The eZ Platform Query Field
Bertrand Dunogier
 
The eZ Platform view layer – eZ Conference 2016
The eZ Platform view layer – eZ Conference 2016The eZ Platform view layer – eZ Conference 2016
The eZ Platform view layer – eZ Conference 2016
Bertrand Dunogier
 
Dev Symfony2 rapide avec un framework de contenu
Dev Symfony2 rapide avec un framework de contenuDev Symfony2 rapide avec un framework de contenu
Dev Symfony2 rapide avec un framework de contenu
Bertrand Dunogier
 
Discover eZ Publish: why you have to know this product
Discover eZ Publish: why you have to know this productDiscover eZ Publish: why you have to know this product
Discover eZ Publish: why you have to know this product
Bertrand Dunogier
 
eZ Publish REST API v2
eZ Publish REST API v2eZ Publish REST API v2
eZ Publish REST API v2
Bertrand Dunogier
 
E zsc2012 rest-api-v2
E zsc2012 rest-api-v2E zsc2012 rest-api-v2
E zsc2012 rest-api-v2
Bertrand Dunogier
 
eZ Publish Asynchronous Content Publishing
eZ Publish Asynchronous Content PublishingeZ Publish Asynchronous Content Publishing
eZ Publish Asynchronous Content Publishing
Bertrand Dunogier
 

More from Bertrand Dunogier (7)

The eZ Platform Query Field
The eZ Platform Query FieldThe eZ Platform Query Field
The eZ Platform Query Field
 
The eZ Platform view layer – eZ Conference 2016
The eZ Platform view layer – eZ Conference 2016The eZ Platform view layer – eZ Conference 2016
The eZ Platform view layer – eZ Conference 2016
 
Dev Symfony2 rapide avec un framework de contenu
Dev Symfony2 rapide avec un framework de contenuDev Symfony2 rapide avec un framework de contenu
Dev Symfony2 rapide avec un framework de contenu
 
Discover eZ Publish: why you have to know this product
Discover eZ Publish: why you have to know this productDiscover eZ Publish: why you have to know this product
Discover eZ Publish: why you have to know this product
 
eZ Publish REST API v2
eZ Publish REST API v2eZ Publish REST API v2
eZ Publish REST API v2
 
E zsc2012 rest-api-v2
E zsc2012 rest-api-v2E zsc2012 rest-api-v2
E zsc2012 rest-api-v2
 
eZ Publish Asynchronous Content Publishing
eZ Publish Asynchronous Content PublishingeZ Publish Asynchronous Content Publishing
eZ Publish Asynchronous Content Publishing
 

Recently uploaded

leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
flufftailshop
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Jeffrey Haguewood
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
saastr
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 

Recently uploaded (20)

leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfNunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdf
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 

eZ Publish cluster unleashed revisited

  • 2. Purpose and history One eZ Publish on multiple servers, for: Performance Redundancy Created for eZ Publish 3.8, vastly improved since then: 3.10: huge schema & feature improvements 4.2: new eZDFS cluster handler for NFS 4.2: Stalecaching mechanism for eZDB and eZDFS 4.7: Refactored binary index architecture Matches the shared database for filesystem operations: Inserting new data, expiring/deleting obsolete data serving files through HTTP
  • 3.
  • 4. Cluster handlers eZ DB eZ DFS From eZ Publish 3.8 From eZ Publish 4.2 Stores metadata in DB Stores metadata in DB Stores data in DB (BLOBs) Stores data on NFS Easy to setup Requires an NFS server Huge DB, harder to maintain Maintenance is much easier
  • 5. HTTP vs Internal calls HTTP Requests done through index_cluster.php Serves files directly A reverse proxy caches images & binary files INTERNAL / KERNEL CALLS Reads/writes cache items Stores binary files
  • 6. Configuration #1: file.ini A cluster handler must be configured in a file.ini override: 1.[ClusteringSettings] 2.FileHandler=eZDFSFileHandler 1.# DFS Cluster Handler settings 2.[eZDFSClusteringSettings] 3.MountPointPath=var/nfs 4.DBBackend=eZDFSFileHandlerMySQLBackend 5.DBHost=localhost 6.DBName=labs_ezpublish_trunk_cluster 7.DBUser=root 8.DBPassword=root
  • 7. Configuration #2: database eZDB and eZDFS both require database tables The same DB server can be used Performance wise it is better to use another server Schemas are available in kernel/sql/*/cluster_*_schema.sql (mysql only, oracle's are in the extension)
  • 8. Configuration #3: clusterize Existing local files must be moved to the cluster This  is done using bin/php/clusterize.php This script will copy/move images & binary files from the local filesystem to the cluster It must only be executed once for the whole cluster
  • 9. Configuration #4: binary URLs rewriting URL rewriting is required to "stream" clusterized binary files to visitors: Images Binary files They only affect known path for these types of files To stream different files, new rules must be added RewriteEngine On Rewriterule ^/var/([^/]+/)?storage/original/.* /index_cluster.php [L] Rewriterule ^/var/([^/]+/)?storage/images/.* /index_cluster.php [L] # Standard eZ Publish rewrite rules # ...
  • 10. Configuration #5: binary files index The cluster index handles binary files HTTP requests It doesn't use the whole kernel in order to perform better It was completely refactored in 4.7 Before 4.7 After 4.7 Required custom file to be created No custom files Many files also located at root level Documented Duplicated logic among handlers Common code has been refactored Features weren't identical among Features are (almost) perfectly
  • 11. Configuration #5: binary files index, example /** * Cluster configuration file */ define( 'CLUSTER_STORAGE_BACKEND', 'dfsmysqli' ); define( 'CLUSTER_STORAGE_HOST', 'localhost' ); define( 'CLUSTER_STORAGE_PORT', 3306 ); define( 'CLUSTER_STORAGE_USER', 'clusteruser' ); define( 'CLUSTER_STORAGE_PASS', '' ); define( 'CLUSTER_STORAGE_DB', 'ezcluster' ); define( 'CLUSTER_STORAGE_CHARSET', 'utf8' ); define( 'CLUSTER_MOUNT_POINT_PATH', '/opt/nfs/ez' );
  • 12. How cache is handled Remote, clusterized files are copied locally Before using a local file, eZ always checks if it is newer than the remote one When a cache file is generated, it is stored on the cluster, and re-used by other nodes expiry.php provides an intermediate method
  • 13. API: reading clusterized files 1.$path = '/var/ezwebin_site/path/to/my/file.txt' 1.// read a text file, "instance" syntax 2.$contents = eZClusterFileHandler::instance( $path )->fetchContents(); 1.// read a text file, "static" syntax 2.$contents = eZClusterFileHandler::instance()->fileFetchContents( $path ); 1.// fetch a file (a binary one for example) to disk 2.$path = 'var/ezflow_site/storage/original/video/961d8a65efffdd93708cc23bc6398953.flv'; 3.$handler = eZClusterFileHandler::instance( $path )->fetchUnique(); 1.// ... use the file ... then delete it 2.$handler->deleteLocal( $uniquePath ); 1.// reading metadata 2.$file = eZClusterFileHandler::instance( $someFile ); 3.echo $file->size(); 4.echo $file->mtime();
  • 14. API: writing a file to the cluster Again, the native PHP I/O API isn't cluster aware ! 1.// stores the contents of the $contents variable to cluster 2.$path = 'var/ezwebin_site/file.txt'; 3.$handler = eZClusterFileHandler::instance( $path ); 4.$handler->storeContents( $contents, 'text', 'text/plain' ); 1.// alternative "static" method: fileStoreContents() 1.// stores the contents of a local image as a clusterized file 2.$imagePath = 'var/ezwebin_site/path/image.png'; 3.$handler = eZClusterFileHandler::instance(); 4.$handler->fileStore( $imagePath, 'image', true, 'image/png' );
  • 15. API: ViewCache ! ViewCache uses advanced methods of the cluster API These method handle:  cache retrieving and/or generation  Concurrency  stale caching It can technically be used to manage custom cache ! Next: introducing the processCache() method
  • 16. API: The processCache() method ! It can be used for custom developements too ! It will let you implement caching in your own extensions It uses:  a generate() callback that generates the dynamic content  a retrieve() callback that retrieves valid cached content
  • 17. API: The processCache() method, CODE ! 1.$cacheFilePath = eZCachedModule::cacheFilePath( $Params ); 2.$cacheFile = eZClusterFileHandler::instance( $cacheFilePath ); 3.$Result = $cacheFile->processCache( 4. array( 'eZCachedModule', 'retrieve' ), 5. array( 'eZCachedModule', 'generate' ), 6. null, 7. null, 8. compact( 'Params' ) ); 1.return $Result;
  • 18. API: The processCache() method, callback Let's review an example that caches the whole output ($Result) of a module. It has a configurable TTL, and provides a custom cache key that can be used to invalidate cache. 1. Get the code from github: git clone git://gist.github.com/3635993.git extension/ezcachedmodule 2. enable the extension in site.ini ActiveExtensions[]=ezcachedmodule 3. and do not forget to regenerate the extensions autoload: Php bin/php/ezpgenerateautoloads.php –extension
  • 19. API: The processCache() method, callback 1. Enable the DebugOutput Come on, you know how to do it 2. Go to http://ezpublish4.admin/cachedmodule/view/abcd Look into the debugOutput for « eZCachedModule » 3. Let's review it together
  • 20. API: The processCache() method, callback Let's review an example that caches the whole output ($Result) of a module. It has a configurable TTL, and provides a custom cache key that can be used to invalidate cache. 1. Get the code from github: git clone git://gist.github.com/3635993.git extension/ezcachedmodule 2. enable the extension in site.ini ActiveExtensions[]=ezcachedmodule 3. and do not forget to regenerate the extensions autoload: Php bin/php/ezpgenerateautoloads.php –extension