SOA with PHP and Symfony


Published on

Lesson learned during new project base on SOA architecture. Technology used in our project:
- Symfony 2.3
- PHPUnit
- SoapUI
- RabbitMQ
- MySQL (Percona)
- Elasticsearch
- Jenkins
- Memcached
- Nagios
- New Relic

Published in: Technology
  • Be the first to comment

  • Be the first to like this

SOA with PHP and Symfony

  1. 1. • Service-oriented architecture • Communication over protocol (typically network) • Reusable components • Services are a „black box” for a clients • Each component is independent and self- contained
  2. 2. • E-learning platform for primary school • Addition to books • Teaching by playing – Gamification – Team working – Personalization (dynamic avatars) – Virtual currency
  3. 3. • Producer send message to RabbitMQ server • Consumer can pick up selected messages • Consumers are run in CLI • Consumers can be run in different modes • We are running consumers in endless loop
  4. 4. Problem: We have fixed date when project will go live but team wants to work in Agile methodology. Solution: Spend few days/weeks for analysing requirements and plan all sprints to the deadline.
  5. 5. Problem: Data consistency between all independent and self-contained components Solution: - Advanced Message Queuing Protocol. We have used RabbitMQ - Communication with components using their external interface - Using cron jobs for clearing data at nights - Ignore this, it's not always important and critical for your application Improve/explore : ESB - Enterprise Service Bus
  6. 6. Problem: How to generate reports based on two or more self-contained components? Solution: RabbitMQ
  7. 7. Problem: How to test all components? What about integration tests of our orchestration level? Solution: • PHPUnit in components (mock external components) • Automated tests written by our testers in SOAP UI Improve: • More tests  • Behat • PHPSpec
  8. 8. Problem: Performance of the whole application doesn’t satisfy us Solution: • Disable unused components like: Session, Security • Disable Doctrine logging • Proper configuration of Symfony logger • Always use Composer class loading optimization • Too many listeners on HTTP requests • Use APC, Xcache or any other PHP accelerator • Use ApcClassLoader or XcacheClassLoader • HTTP cache (Varnish)
  9. 9. Problem: Processing of messages was very slow and memory leaks in consumers. Solution: Clean up after yourself • Use unset() method • Doctrine methods: free(), detach(), clear()
  10. 10. Problem: MySQL errors: • 2013: Lost connection to MySQL server • 2006: MySQL server has gone away • 2003: Can't connect to MySQL server Solution: Catch all those exceptions and try to reconnect to the database server. We have set up also some timeout between all retries.
  11. 11. Problem: Can I trust consumer scripts? What with situation when consumer will have problem with connection with database etc. Exception is thrown and message can be lost. Solution: • Creat failover files • Catch all exceptions • Log message into that file. • Create mechanism for processing failed messages
  12. 12. Problem: We are using Memcached but QPS (Query Per Second) of the MySQL server doesn't satisfy us Solution: Configure read/write split in your database and prepare separate connections in your application
  13. 13. Problem: Performance of the database layer still doesn’t satisfy us Solution: Monitor database queries during development How: • Use Symfony profiler • Log all database queries to the temporary file on your dev environment. Then use „tail –f” command to monitor all queries during requests • Use EXPLAIN on more complicated queries
  14. 14. Problem: Doctrine performance is unsatisfactory. Moreover Doctrine create many unnecessary queries. Solution: • Use DQL instead Doctrine findAll() method • Use native query for more complicated queries • Avoid entity mapping when it’s not necessary (hydration mode) • Use „extra lazy” fetching strategy for associations • For big collections fetch only necessary fields
  15. 15. Problem: How to monitor application? How detect problems before they will become a serious? Solution: Monitoring software • New Relic • Nagios Improve: Logstash + Kibana + Elasticsearch
  16. 16. • Symfony 2.3 • PHPUnit • SoapUI • RabbitMQ • MySQL (Percona) • Elasticsearch • Jenkins • Memcached • Nagios • New Relic
  17. 17. • First project on new framework • First use of Doctrine
  18. 18. • Run all Symfony2 console commands with --env=prod parameter • Write your own wrappers to all main external libraries • Use Vagrant or Docker • Run load tests on filled database • Automate your processes (ie. Ant + Jenkins) • Add created_date and modified_date to all tables (use MySQL triggers)
  19. 19. Quetions?