By John Coggeshall and Marc Urbaitel ENTERPRISE PHP: A CASE STUDY

INTRODUCTIONS Who am I: John Coggeshall Architect, North American Professional Services PHP 5 Core Contributor Author: PHP 5 Unleashed Member of Zend’s Education Advisory Board May 28, 2009 Enterprise PHP: A Case Study PAGE

Who am I:

John Coggeshall

Architect, North American Professional Services

PHP 5 Core Contributor

Author: PHP 5 Unleashed

Member of Zend’s Education Advisory Board

INTRODUCTIONS Who did I work with? Marc Urbaitel CTO & Co-Founder In Ticketing PHP Programmer Chief Architect of Existing Ticketing System May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Who did I work with?

Marc Urbaitel

CTO & Co-Founder In Ticketing

PHP Programmer

Chief Architect of Existing Ticketing System

SESSION AGENDA Introductions History Technical Details Demo Questions May 28, 2009 Enterprise PHP: A Case Study PAGE

Introductions

History

Technical Details

Demo

Questions

HISTORY In Ticketing, Inc. Founded in 1999 (Formerly InHouse Ticketing) Competitor to Ticket Master Full Service Ticketing Provider System built entirely on PHP/MySQL Platform May 28, 2009 Enterprise PHP: A Case Study PAGE

In Ticketing, Inc.

Founded in 1999 (Formerly InHouse Ticketing)

Competitor to Ticket Master

Full Service Ticketing Provider

System built entirely on PHP/MySQL Platform

THE PROBLEM Organically grown web application Hard to add features (i.e. new up-sell opportunities) Ticket Insurance "Green" up your experience Even harder to scale to meet client demand Wants to be able to process roughly 25,000 ticket transactions per minute Or … ~85 tickets per second Or … 300-400+ requests per second Must maintain current reporting / site admin facilities used by clients May 28, 2009 Enterprise PHP: A Case Study PAGE

Organically grown web application

Hard to add features (i.e. new up-sell opportunities)

Ticket Insurance

"Green" up your experience

Even harder to scale to meet client demand

Wants to be able to process roughly 25,000 ticket transactions per minute

Or … ~85 tickets per second

Or … 300-400+ requests per second

Must maintain current reporting / site admin facilities used by clients

START WITH DUCT TAPE In existing system hardware wouldn't solve the problem System itself wasn't scalable Initial Solution: Serialize the ticket process for customers Customers would be placed into a virtual queue and released in small groups to purchase tickets Resulted in online wait times to begin order process anywhere from 30 minutes to 2 hours Kept the request/second requirement low enough for system to handle It worked, but was only a temporary solution May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

In existing system hardware wouldn't solve the problem

System itself wasn't scalable

Initial Solution: Serialize the ticket process for customers

Customers would be placed into a virtual queue and released in small groups to purchase tickets

Resulted in online wait times to begin order process anywhere from 30 minutes to 2 hours

Kept the request/second requirement low enough for system to handle

It worked, but was only a temporary solution

KEY APPLICATION CONCERNS Load Pattern is Spiked Associated with Ticket Sale Start Usually associated with a single concert, etc. May 28, 2009 NAME OF THE SESSION COMES HERE PAGE Note: Simulated Data

Load Pattern is Spiked

Associated with Ticket Sale Start

Usually associated with a single concert, etc.

KEY APPLICATION CONCERNS Load is usually focused on a small subsection of clients Translation: System needs to scale down to at least the event level May 28, 2009 NAME OF THE SESSION COMES HERE PAGE Note: Simulated Data

Load is usually focused on a small subsection of clients

Translation: System needs to scale down to at least the event level

THE SOLUTION: OLT (Online Ticketing) Re-write of entire application not an option Too expensive to re-write entire online business Instead, separate out the important bits Re-write the online order engine, integrate with old system Significantly less effort and will solve most of the goal Offload integration points outside of web request Keep local copies of data on each server and sync as necessary New Architecture will allow easy improvements / ongoing maintenance May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Re-write of entire application not an option

Too expensive to re-write entire online business

Instead, separate out the important bits

Re-write the online order engine, integrate with old system

Significantly less effort and will solve most of the goal

Offload integration points outside of web request

Keep local copies of data on each server and sync as necessary

New Architecture will allow easy improvements / ongoing maintenance

PROJECT FOCUS May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

HIGH LEVEL SERVER ARCHITECTURE May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

HIGH LEVEL SERVER ARCHITECTURE May 28, 2009 NAME OF THE SESSION COMES HERE PAGE Admin is for Clients, which create events and view reporting (Ticket Metadata)

HIGH LEVEL SERVER ARCHITECTURE May 28, 2009 NAME OF THE SESSION COMES HERE PAGE Ticket Meta Data is replicated as SQLite databases on each OLT server (read-only)

HIGH LEVEL SERVER ARCHITECTURE May 28, 2009 NAME OF THE SESSION COMES HERE PAGE Online Orders come through the event server and are handed off to OLT once the transaction beings

HIGH LEVEL SERVER ARCHITECTURE May 28, 2009 NAME OF THE SESSION COMES HERE PAGE The OLT Cluster Manages Ticket Allotments, transaction, reporting back the order results to the reporting system

HIGH LEVEL SERVER ARCHITECTURE May 28, 2009 NAME OF THE SESSION COMES HERE PAGE Box office orders use stored procedures to access allotments in their respective OLT to complete orders

THE OLT CLUSTER An OLT Cluster consists of at least: One Web Server One Database Server One Job Queue Server Cluster Database Performance is Glass Ceiling Can add Job Queue, Web servers Can also add slaves, replace with MySQL cluster, etc. May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

An OLT Cluster consists of at least:

One Web Server

One Database Server

One Job Queue Server

Cluster Database Performance is Glass Ceiling

Can add Job Queue, Web servers

Can also add slaves, replace with MySQL cluster, etc.

THE OLT CLUSTER SERVERS Database: MySQL 5.0 Utilizes Stored Procedures to Encapsulate complex allotment management Web Server: PHP 5.2 ZC2, Platform, Zend Framework Job Queue: PHP 5.2 InTicketing specific Frameworks: OLT, In Built on ZF Front End: ZF MVC May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Database: MySQL 5.0

Utilizes Stored Procedures to Encapsulate complex allotment management

Web Server: PHP 5.2

ZC2, Platform, Zend Framework

Job Queue: PHP 5.2

InTicketing specific Frameworks: OLT, In

Built on ZF

Front End: ZF MVC

TRULY SCALABLE Each cluster is self-contained A single OLT cluster can scale up and down One cluster can run entire business .. Or one client .. Or one event .. Or a single ticket allotment May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Each cluster is self-contained

A single OLT cluster can scale up and down

One cluster can run entire business

.. Or one client

.. Or one event

.. Or a single ticket allotment

DATABASE SCHEMAS Previous Implementation has 183 tables in MySQL At least 7 tables directly related to a ticket order used every request Remainder related to admin / reporting / etc. OLT: 5 tables total in MySQL 3 of which are used during the web request InnoDB tables Replaced complex JOINs with composite keys (Ticket SKUs) May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Previous Implementation has 183 tables in MySQL

At least 7 tables directly related to a ticket order used every request

Remainder related to admin / reporting / etc.

OLT: 5 tables total in MySQL

3 of which are used during the web request

InnoDB tables

Replaced complex JOINs with composite keys (Ticket SKUs)

DATABASE SCHEMAS Most OLT data is read-only and stored locally on every server as a series of SQLite Databases One DB per 'Nug' (client selling tickets) Only low-traffic admin systems from legacy touch OLT MySQL Database (for allotment updates) May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Most OLT data is read-only and stored locally on every server as a series of SQLite Databases

One DB per 'Nug' (client selling tickets)

Only low-traffic admin systems from legacy touch OLT MySQL Database (for allotment updates)

DATABASE SCHEMAS May 28, 2009 NAME OF THE SESSION COMES HERE PAGE MySQL read/write DB schema

WEB APPLICATION ARCHITECTURE ~ 30,000 lines of code and ~160 classes Broken into two libraries, OLT and IN plus front-end The OLT Library Heart of OLT system Handles all major functionality of system The IN Library Support Library Provides functionality which isn't OLT-specific for code-reuse reasons Contains Transaction Engine, Custom Filters/Validators, Customer / Transaction information objects May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

~ 30,000 lines of code and ~160 classes

Broken into two libraries, OLT and IN plus front-end

The OLT Library

Heart of OLT system

Handles all major functionality of system

The IN Library

Support Library

Provides functionality which isn't OLT-specific for code-reuse reasons

Contains Transaction Engine, Custom Filters/Validators, Customer / Transaction information objects

MVC DETAILS Built using Zend Framework Model / View / Controller (MVC) 8 Controllers total manage the entire order process Orders are began by posting to the correct OLT engine the details Which tickets to buy / quantity Meta data for order OLT manages entire order process and triggers backend job to complete transaction May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Built using Zend Framework Model / View / Controller (MVC)

8 Controllers total manage the entire order process

Orders are began by posting to the correct OLT engine the details

Which tickets to buy / quantity

Meta data for order

OLT manages entire order process and triggers backend job to complete transaction

BASIC ORDER FLOW Real Time Flow Process Captcha Identify Language & Skinning of Order Temporarily Reserve Tickets During Order Gather Customer Information Perform Ticket Transaction Asynchronous Operations Perform up-sell transactions Generate PDF tickets Store Order Data Send E-mails Etc. May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Real Time Flow

Process Captcha

Identify Language & Skinning of Order

Temporarily Reserve Tickets During Order

Gather Customer Information

Perform Ticket Transaction

Asynchronous Operations

Perform up-sell transactions

Generate PDF tickets

Store Order Data

Send E-mails

Etc.

UNDER THE HOOD Order Process Order is gathered Consolidated into single object Order contains everything possibly known about the transaction at hand May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Order Process

Order is gathered

Consolidated into single object

Order contains everything possibly known about the transaction at hand

ASYNCHRONOUS BACKEND Order is then passed into the Job Queue to process numerous asynchronous tasks Primary Ticket Transaction is still real time Transactions can happen through numerous gateways May 28, 2009 #

Order is then passed into the Job Queue to process numerous asynchronous tasks

Primary Ticket Transaction is still real time

Transactions can happen through numerous gateways

WHAT DOES THIS ALL MEAN FOR INTICKETING? Scalable solution means when the next big client comes they can say YES to their business Larger volumes can now be handled just by purchasing more hardware Added Revenue Streams adds to bottom line PayPal Express Checkout Ticket Insurance Green your Experience Broader Marketplace International clients thanks to built in internationalization May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

Scalable solution means when the next big client comes they can say YES to their business

Larger volumes can now be handled just by purchasing more hardware

Added Revenue Streams adds to bottom line

PayPal Express Checkout

Ticket Insurance

Green your Experience

Broader Marketplace

International clients thanks to built in internationalization

WHAT’S NEXT FOR OLT? New event listing architecture Current bottleneck in system is the event landing pages Optimization of Cluster Improving minimum cluster overall performance lowers cost per ticket More Features New Upsells Different types of events (Cruises, etc) More Reporting, Introspection May 28, 2009 NAME OF THE SESSION COMES HERE PAGE

New event listing architecture

Current bottleneck in system is the event landing pages

Optimization of Cluster

Improving minimum cluster overall performance lowers cost per ticket

More Features

New Upsells

Different types of events (Cruises, etc)

More Reporting, Introspection

THANK YOU! Questions? Comments? Suggestions?