This document discusses optimizing eZ Publish for high-traffic, big data contexts. It emphasizes the importance of project management skills, technical expertise across multiple technologies, and an approach involving continuous monitoring, metrics collection, and code optimization. Creative solutions are needed like custom caching, metrics tools, and leveraging new technologies to solve challenges involving large volumes of data, high traffic levels, and many daily transactions.

1. 1401 Avenue du Mondial 98
Immeuble Oxygène
34000 Montpellier
04 99 13 68 60
How to deploy &
optimize eZ Publish
high-traffic / bigdata context
1 KALIOP.COM

2. Critical issues : Critical mass : Complexity to solve :
Big Data ?
Peak loads ?
Transactions ?
(import – export - REST)
User profiling ?
(auth – settings – peruser
> 5 M pages / vues / mois Replicated ?
Le
09/10/2
High traffic ? Huge volume ?
2
High-traffic ?
Exemples
> 5 M pages / vues / mois
UI)
550000KK ++ uusseerr//ddaayy
1M+ datas
+200%
100k transact./day
50%+ authenticated
(portal features)
+
+
++
++
+++
Exemples
(PRA – Worldwide cloud)
2+ ++++

3. Do not focus solely on technical
expertise !
Le
09/10/2
Keys of success
3
Web Architect /
SCRUM Master
Exemples
Project management
> 5 M pages / vues / mois
approach
Web Architect /
SCRUM Master
Technical expertise Tools

4. Project management
Role of the Web Architect : Never stop to (re)design. Arbitrate & find
workarounds for all the contradictory issues : UX, front-end, performances, dev,
hosting, delay, cost, stability, SEO, marketing operations...
Role of the project manager (or SCRUM Master) : Protect the team.
Understand, explain & make the POC possible , code refactoring, continuous
integration & devOps
Role of the project owner : Build the team & be clear on the KPIs
Exemple : possible to challenge the UX ? (VS performances)
Exemple : ready to work without a Gantt ? (VS kanban)
Le
09/10/2
Keys of success / PM approach
4
approach
Exemples
> 5 M pages / vues / mois

5. Le
09/10/2
Keys of success / PM approach
5
Tools
Monitoring : complexity to monitor all the stack
(in most cases, only on low level services, not on PHP / slow queries)
Metrics : complexity to have metrics on all the the stack
(in most cases, only on low level services, or end-users trafic)
Deployment tools : the easier it is, the more consuming it is
(in most cases, copy the files & purge the cache...)
Testing : complexity to automate unit testing / user testing
(in most cases... nothing)

6. Le
09/10/2
Keys of success / Technical expertise
6
Technical expertise
(not only eZ)
FPM
Fast
CGI
APC
And more...

7. Le
09/10/2
Keys of success / Technical expertise
Médias (copy)
7
eZ FS2 StaleCache
INI
Templates
Médias
Viewcache
Template-block
HTTP Caches
ESI
++++
Easy to host
Save ressources
Easy to setup/switch
StaleCache
Simplify the deployment process
----
High NFS activity
Require top-level ($$$) NFS
Require an high eZ quality dev
Require frequent purges
APC APC APC
Stale Cache
( no flock )

8. Le
09/10/2
Keys of success / Technical expertise
Médias (copy)
8
eZ DFS Cluster
INI
Templates
Cache (copy)
INI
Templates
Cache (copy)
INI
Templates
Cache (copy)
HTTP Caches
ESI
APC APC APC
EZ DFS
Médias
Viewcache
Template-block
...
META DATA
++++
Compatible with most of platforms
Low NFS activity
Easy to setup/switch
StaleCache
----
Waste of ressources
ClusterDB bottleneck
Complex deployment process
Require an high eZ quality dev
Require frequent purges

9. Le
09/10/2
Keys of success / Technical expertise
9
DevOps mode
Do not expect the hosting architecture (sizing) to make up
for the application issues...
Do not expect the development (hack, trick) to make up
for the hosting architecture issues...

10. Le
09/10/2
Keys of success / Technical expertise
10
DevOps mode
(expertise in the middle)
APC
apc.filters = "-cache.*.php"
(exlude compiled templates)
MySql
tmpdir → tmpfs (avoid « on disk » storage)
Add missing indexes (remote_id)
Solr
Split read/write operations
Varnish
Synchronize with eZ (x-node pruge)
APC

11. Le
09/10/2
Keys of success / Technical expertise
11
OpteiZm Pizueb yloisuhr code
(what you can deduce)
Reduce the
Cache-block activity
& volume
Optimize the
viewcache
activity
Remove
Useless SQL
Use small range keys
Set an expiry value
Use ezpagedata() instead
Replace with ESI
No more than 1K files !
Avoid too many view parameters
Optimize smartviewcache
Synchronize with Varnish
Replace with Solr
Use PHP cache
Optimize Avoid datamap load
Solr
Schedule your
background
tasks
split read / write activities
Delay « commit » (not only index)

12. Le
09/10/2
Keys of success / Technical expertise
12
OpteiZm Pizueb yloisuhr code
(read tutos)

13. Le
09/10/2
Keys of success / Technical expertise
13
OpteiZm Pizueb yloisuhr code
(Learn how it works)
Basic optimization :
DelayedIndexing = true (→ will not delay hard commits for all actions ! )
OptimizeOnCommit = disabled
CRON // ezfoptimizeindex (3 minutes)
CRON // ezfindexcontent (1 day)
Medium optimization :
OptimizeOnCommit = disabled
DisableDeleteCommits = true (+ CRON to commit)
DisableDirectCommits=true (+ CRON to commit)
CRON // bin/php/ezsolrcommit.php (3 minutes)
CRON // ezfoptimizeindex (1 day)
Other great optimizations :
- Replicate Solr, and split the Read (search..) / Write (commit, optimize), to avoid concurrencies
- Avoid hard commits (soft commits to avoid fsync ops, commitWithin to delay pending docs within Solr)
addDoc Hard Commit (rebuild indexes + fsync)
Soft Commit (rebuild indexes)
CommitWithin (commit all pending docs within X secs)

14. Le
09/10/2
Keys of success / Technical expertise
14
Optimize your code
(Get metrics)
(what you can not guess)
APC
EZ DFS
META DATA
GET
METRICS

15. Le
09/10/2
Keys of success / Technical expertise
15
Optimize your code
(Get metrics)
(what you can not guess)
Why session_start
lock ???
eZ Bug on the
checkValidity=false setting...
will always create a
session_start on the index.php

16. Le
09/10/2
Keys of success / Technical expertise
16
Optimize your code
(Get metrics)
(what you can not guess)
Why a unique table
insert needs 9,6
sec ???
Server side tuning requested
(memory, table optimize, host
the clusterDB on another
server...)

17. Le
09/10/2
Keys of success / Technical expertise
17
Optimize your code
(Get metrics)
(what you can not guess)
Why my code
provides 1988
cURL calls ???
Mmhh...
How to say...
NEXT

18. Critical issues : Critical mass : Complexity to solve :
Big Data ?
Peak loads ?
Transactions ?
(import – export - REST)
User profiling ?
(auth – settings – peruser
> 5 M pages / vues / mois Replicated ?
Le
09/10/2
Enough to resolve the following context ?
18
High-traffic ?
Exemples
> 5 M pages / vues / mois
UI)
550000KK ++ uusseerr//ddaayy
1M+ datas
+200%
100k transact./day
50%+ authentifié
(portal features)
+
+
++
++
+++
Exemples
(PRA – Worldwide cloud)
2+ ++++

19. Externalize the user database (if too big)
Build a full ESI front-end (pure HTTP cache logical)
Replace the DB cluster with Redis or memcached
Externalize medias on CDN
Use a custom PHP Cache (exemple)
Use messaging tools
Build POC with docker
Build custom metrics tools
...
Le
09/10/2
Enough to resolve the following context ?
19
NO ! Be creative !

20. Le
09/10/2
Enough to resolve the following context ?
20
Be creative !
Object.datamap → PHP array
PHP cache file (eZPHPCreator)
On publish
Store all the attributes
Pre-generate related values
On read
Read object
datamap
Fetch, filter,
facet, sort..
HTML

21. Le
09/10/2
Enough to resolve the following context ?
21
Be creative !
Custom metrics
tools

22. Le
09/10/2
Ready for eZ5 ?
22
Ready for eZP5...
Native HTTP Cache
Native Storage abstraction
Native pure MVC
Native ESI support
…
Full Symfony stack

23. Le
09/10/2
Ready for eZ5 ? HTTP Cache
Twig compiled templates
Routing
service container
translations...
23
Persistence cache
User
GET /
200 OK
Age : 100
Cache-control:
Maxage=300
X-Location-Id:2
X-Location-Id:{nodeID}
Vary: X-User-Hash
eZ
expired
HTTP Cache
PROD
DEV
Clear on
Viewcache
event

24. Le
09/10/2
Ready for eZ5 ? ESI logical
24
HEADER
SLIDER
FOOTER
LAST
NEWS
LAST
COMMENTS
USER
SELECTION
FAVORITES
TOPICS
ESI key : Shared content
Expiry ? 1 day TTL
ESI key : Shared content
Expiry ? Long TTL + Dynamic, on change
ESI key : Per user content
Expiry ? Long TTL + Dynamic, on change
ESI key : Shared content
Expiry ? 5 min TTL
ESI key : Per topics
Expiry ? Long TTL + Dynamic, on change
ESI key : Shared content
Expiry ? 1 day TTL
LAST
Tweets
AJAX

25. 1401 Avenue du Mondial 98
Immeuble Oxygène
34000 Montpellier
04 99 13 68 60
How to deploy &
optimize eZ Publish
high-traffic / bigdata context
Question ?
25KALIOP.COM