Scaling symfony apps

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Rome, 28 October 2016
Scaling Symfony apps

WHO AM I?

Matteo Moretti

CTO @
website: madisoft.it
tech blog: labs.madisoft.it

Scalability

It’s from experience.
There are no lessons.

Nuvola
● > 3M HTTP requests / day
● ~ 1000 databases
● ~ 350GB mysql data
● ~ 180M query / day
● ~ 25M of media files
● ~ ...

Scalability
Your app is scalable if it can adapt to
support an increasing amount of data
or a growing number of users.

“But… I don’t have an increasing load”
(http://www.freepik.com/free-photos-vectors/smile - Smile vector designed by Freepi...

“Scalability doesn’t matter to you.”
(http://www.freepik.com/free-photos-vectors/smile - Smile vector designed by Freepik)

“I do have an increasing load”
(http://www.freepik.com/free-photos-vectors/smile - Smile vector designed by Freepik)

Your app is growing

But… suddenly…

Ok, we need to scale

Scaling… what?
PHP code?
Database?
Sessions?
Storage?
Async tasks?

Everything?

Can Node.js scale?

Can Symfony scale?

Can PHP scale?

Scaling is about
app architecture

App architecture
How can you scale your web server if
you put everything inside?
Database, user files, sessions, ...

App architecture / Decouple
● Decouple services
● Service: do one thing and do it well

App architecture / Decouple

4 main areas
1. web server
2. sessions
3. database
4. filesystem
There are some more (http caching, frontend, queue system...

Web server
Many small webservers
(scale up vs scale out)

Web server
NGINX + php-fpm
PHP 7
Symfony 3

Web server / Cache
PHP CACHE
SYMFONY CACHE
DOCTRINE CACHE

Web server / PHP Cache
Opcache
Bytecode caching
opcache.enable = On
opcache.validate_timestamps = 0
https://tideways.io/pr...

PHP code / Symfony cache
● Put Symfony cache in ram
● Use cache warmers during deploy
releaseN/var/cache -> /var/www/proje...

PHP code / Doctrine cache
● Configure Doctrine to use cache
● Disable Doctrine logging and profiling on prod
doctrine.orm....

PHP code / Cache
DISK I/O ~ 0%

Monitor
Measure
Analyze

PHP code / Profiling
XHProf
Blackfire
New Relic

PHP code / Recap
● Easy
● No need to change your PHP code
● It’s most configuration and tuning
● You can do one by one and...

Sessions
● Think session management as a service
● Use centralized Memcached or Redis (Ec2
or ElasticCache on AWS)
● Avoid...

Session / Memcached
No bundle required
https://labs.madisoft.it/scaling-symfony-sessions-with-memcached/

Session / Redis
https://github.com/snc/SncRedisBundle
https://labs.madisoft.it/scaling-symfony-sessions-with-redis/

Session / Redis
config.yml
framework:
session:
handler_id: snc_redis.session.handler

Session / Redis
Bundle config
snc_redis:
clients:
session_client:
dsn: '%redis_dsn_session%'
logging: false # https://gith...

Session / Redis
parameters.yml
redis_db: 3
redis_dsn_session: 'redis://%redis_ip%/%redis_db%'
redis_ip: redis-cluster.my-l...

Session / Recap
● Very easy
● No need to change your PHP code
● Redis better than Memcached: it has persistence and many o...

Database
Aka “The bottleneck”

Database
Relational databases

Database
NOSQL db?

Database
If you need data integrity do
not replace your SQL db with
NOSQL to scale

Database
How to scale SQL db?

Database
When to scale?

Database
If dbsize < 10 GB
dont_worry();

Database / Big db problems
● Very slow backup. High lock time
● If mysql crashes, restart takes time
● It takes time to do...

Database / Short-term solutions
Use a managed db service like AWS RDS
● It scales for you
● It handles failover and backup...

Database / Long-term solutions
Sharding

Database / Sharding
Split a single big db into
many small dbs
(multi-tenant)

Database / Sharding
● Very fast backuo. Low lock time
● If mysql crashes, restart takes little time
● Fast to download and...

Database / Sharding
● How can Symfony deal with them?
● How to execute a cli command on one of them?
● How to apply a migr...

Database / Sharding
Doctrine DBAL & ORM

Database / Sharding
Define a DBAL connection and a ORM
entity manager for each db
https://symfony.com/doc/current/doctrine...

Database / Sharding
doctrine:
orm:
entity_managers:
global:
connection: global
shard1:
connection: shard1
shard2:
connecti...

Database / Sharding
This works for few dbs
(~ <5)

Database / Sharding
Doctrine sharding
http://docs.doctrine-project.org/projects/doctrine-dbal/en/latest/reference/sharding...

Database / Doctrine sharding
● Suited for multi-tenant applications
● Global database to store shared data (ie: user data)...

Database / Sharding
Configuration
doctrine:
dbal:
default_connection: global
connections:
default:
shard_choser_service: v...

Database / Sharding
ShardManager Interface
$shardManager = new PoolingShardManager();
$currentCustomerId = 3;
$shardManage...

Database / Sharding
● It works but it’s complex to be managed
● No documentation everywhere
● Need to manage shard configu...

Database / Recap
● NOSQL is not used to scale SQL: they have different purposes. You can
use both.
● Sharding is difficult...

Filesystem
Users upload files: documents, media, etc
How to handle them?

Filesystem
● Need of filesystem abstraction
● Use external object storage like S3
● Avoid using NAS: it’s tricky to be set...

Filesystem / Abstraction
● FlysystemBundle
● KnpGaufretteBundle
https://github.com/1up-lab/OneupFlysystemBundle

Filesystem / Abstraction
https://github.com/1up-lab/OneupFlysystemBundle
● AWS S3
● Dropbox
● FTP
● Local filesystem
● ...

Filesystem / Abstraction
Configuration
oneup_flysystem:
adapters:
s3_adapter:
awss3v3:
client: s3_client
bucket: "%s3_buck...

Filesystem / Abstraction
Configuration
prod.yml
oneup_flysystem:
filesystems:
my_filesystem:
adapter: s3_adapter
dev.yml
o...

Filesystem / Abstraction
Usage
// LeagueFlysystemFilesystemInterface
$filesystem = $container->get(‘oneup_flysystem.my_fil...

Filesystem / Recap
● Easy
● Need to change your PHP code
● Ready-made bundles
● Avoid local filesystem and NAS
Take away: ...

Scaling / Recap
● Sessions and filesystem: easy. Do it
● PHP code: not difficult. Think of it. Save money.
● Database: ver...

THANK YOU

WE ARE HIRING!
(wanna join? ask us or visit our website)

QUESTIONS?

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

Speed up your Symfony2 application ... by Ricard Clau 27706 views
SaaS con Symfony2 by Matteo Moretti 3889 views
Nuvola: a tale of migration to AWS by Matteo Moretti 1090 views
Filesystem Abstraction with Flysystem by Frank de Jonge 4913 views
How to Clear Cache in a Symfony Cro... by Mike Taylor 1115 views
Multi kernelowa aplikacja w oparciu... by Radek Baczynski 812 views

View on Slideshare

1 of 83 Ad

View on Slideshare

1 of 83 Ad