North east user group tour


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • We released Zend Server and Zend Server Community Edition in April this year Products written from scratch based on our very extensive experience with Platform technologies Full integrated stack, native installer, ZF and Studio integration, software updates, all new UI, … Both editions have been very well received by users (love performance boost, ease of use, deployment) Great fit to our partners – we’re working with Varien/Magento, KnowledgeTree, MCS, … Next step will round up web app server offering to support high availability and scalability – more on this in the next few months
  • North east user group tour

    1. 1. Northeast User Group Tour – April 2011<br />By Kevin Schroeder<br />Technology Evangelist<br />Zend Technologies<br />
    2. 2. About Kevin<br /> Past: Programming/Sys Admin<br /> Current: Technology Evangelist/Author/Composer<br /> @kpschrade<br />
    3. 3. What do we do?<br />
    4. 4.
    5. 5. Zend Framework<br />Zend Server<br />PHP<br />Test<br />Application<br />Performance<br />Reliability &<br />Management<br />Scale-Out<br />Enterprise Enablement<br />Acceleration<br />Monitoring<br />Clustering<br />Hot Fixes<br />Optimization<br />Root-Cause<br />Job Queue<br />Support<br />Profile<br />Caching<br />Downloads<br />Java Bridge<br />Configuration<br />Zend Studio (Eclipse-based)<br />Debug<br />Collaborate<br />Linux<br />(rpm/web repositories)<br />IBM i<br />(PTF)<br />Windows<br />(MSI)<br />
    6. 6. A quick word on something new…<br />
    7. 7.
    8. 8. Wanna try it?<br />Go to<br />… or go to<br />Email me at (this will help expedite your approval)<br />
    9. 9. Things we’re giving away<br />
    10. 10. 1 Zend Certification Test (you still have to pass it)<br />1 copy of Zend Studio<br />ElePHPants<br />Books<br />Dart Boards<br />Little Note Books (the paper kind)<br />
    11. 11. Queuing<br />
    12. 12. Could your PHP apps benefit from being able to process data or execute asynchronously?<br />Twtpoll results<br />
    13. 13. Performance<br />Execute logic apart from the main request (asynchronicity)<br />Scalability<br />The ability to handle non-immediate logic as resources are available<br />Why would you want to queue?<br />
    14. 14. People often say that performance and scalability are separate things.<br />This is inaccurate (consider Google)<br />Performance is the speed by which a request is executed<br />Scalability is the ability of that request to maintain its performance as load/infrastructure increases<br />Performance & Scalability<br />
    15. 15. DON’T!!<br />You are not Facebook<br />You probably won’t be<br />Don’t overcomplicate your problems by trying to be<br />So how do you scale to Facebook size?<br />
    16. 16. Typical anatomy of a PHP Application<br />Bad for scala-bility!<br />Presentation<br />Application Control<br />Database Access<br />Business Logic<br />Presentation<br />Application Control<br />Business Logic<br />Presentation<br />| 16<br />
    17. 17. Good for Scalability<br />Presentation<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Something.phtml<br />Application Control<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Business Logic<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Database Access<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />Something.php<br />| 17<br /> | 14-Apr-11<br />
    18. 18. Defined tasks<br />Loose coupling<br />Resource discovery<br />What helps make software scalable?<br />
    19. 19. The Golden Rule of Scalability<br />“It can probably wait”<br />
    20. 20. Pre-caching data<br />Data analysis<br />Pre-analysis (predicting where your users will go)<br />Data processing<br />Pre-calculating (preparing data for the next request)<br />Data is “out of date” once it leaves the web server<br />Immediacy is seldom necessary<br />Asynchronous execution uses<br />
    21. 21. A sledgehammer can hit a machine<br />Scalability and High Availability are yin and yang<br />A site that can’t keep running is not scalable<br />A site that can’t scale will fail (if it gets really popular)<br />Machines can be added and removed at will<br />Not “cloudy” necessarily<br />No single point of failure<br />Data exists in at least two, preferably at least three, places<br />Characteristics<br />
    22. 22. Waste disk space<br />Control usage (don’t let users do anything they want)<br />Pre-calculate as much as possible<br />Calculate and cache/store<br />Don’t scan large amounts of data<br />Keep data processing off the front end servers<br />Don’t just cache<br />Don’t let it substitute for thought<br />Cache hit rates can be meaningless if you have hundreds of cache hits for a request<br />Considerations<br />
    23. 23. <ul><li>Build a deployment mechanism with NO hardcoded values like directory or resource locations
    24. 24. Make as much as possible configurable/discoverable</li></ul>Decouple/Partition<br />Don’t tie everything (relationships and such) into the database<br />Use queues/messaging<br />Stomp interfaces are really good for PHP – Can also use Java Bridge<br />Zend_Queue has several interfaces<br />Try to use stateless interfaces<br />pollingis more scalable than idle connections; introduces lag<br />Considerations<br />
    25. 25. Use Cron /w PHP CLI (probably shouldn’t do this)<br />Use Gearman<br />Use home-grown (don’t do this)<br />Use pcntl_fork() (NEVER do this)<br />Use Zend Server Job Queue<br />Options<br />
    26. 26. Your only real options<br />Very cloud friendly<br />* I am not an expert on Gearman. Corrections will be taken in the spirit that they are given.<br />For obvious reasons, I will focus on Zend Server<br />
    27. 27. Schedule jobs in the future<br />Set recurring jobs<br />Execute immediately, as resources are available (my fav)<br />Utilize ZendJobQueue()<br />Using the Zend Server Job Queue<br />
    28. 28. Job Queue Architecture – Elastic Backend<br />Web Server /w JQ<br />Web Server<br />Users!<br />Web Server /w JQ<br />Web Server<br />Load Balancer<br />Web Server /w JQ<br />Web Server<br />Pros<br />Scale the backend as necessary<br />Default (easy) mechanism<br />Cons<br />Getting the job status requires using a DB<br />
    29. 29. Job Queue Architecture – Elastic Frontend<br />Web Server<br />Web Server /w JQ<br />Users!<br />Web Server<br />Web Server /w JQ<br />Load Balancer<br />Web Server<br />Web Server /w JQ<br />Pros<br />Easy to communicate with the Job Queue server handling the job<br />Cons<br />Requires you to build your own interface<br />
    30. 30. Create a task-handling controller<br />Create an abstract task class<br />Understands the Job Queue<br />Self contained<br />If Elastic Backend: connects to localhost<br />If Elastic Frontend: connects to load balancer (my preferred), load balanced JQ servers manage themselves<br />Execute the task, have it serialize itself and send it to send to the task handler<br />Kevin’s favorite way to implement it<br />
    31. 31. comzendjobqueueManager<br />Handles connecting to the queue and passing results back and forth<br />comzendjobqueueJobAbstract<br />Abstract class that a job would be based off of<br />comzendjobqueueResponse<br />The response from the manager when a job is queued. Contains the server name and job number<br />orgeschradejobsScandir<br />The actual job that scans the directory<br />orgeschradejobsScandirResult<br />An object that represents the data found<br />Classes involved in the demo<br />
    32. 32. Create job and set data<br />Execute job<br />Job passes itself to the queue manager<br />Manager serializes job<br />Manager uses HTTP call through a load balancer to queue the job<br />The queue on the other end returns the job id and server name<br />Job ID and server name is passed to the client<br />Client polls the manager to get a completed job<br />When the job is returned pass the serialized version of the executed job<br />Execution Flow<br />
    33. 33. Let’s write some code<br />(no I’m not copping out with slides. We’re all told to show our work in grade skool)<br />
    34. 34. Deployment<br />
    35. 35. Agenda<br />What we will be talking about <br />Development->Testing->Staging->Production <br />Rsync/SVN/PEAR/yum mechanisms<br />What we will not be talking about <br />Build tools (Phing, Maven, etc.) <br />Database versioning/deployment <br />Continuous Integration <br />MSI installs (Sorry MS folks, I really wanted to but, y’know, time) <br />Test-Driven Development<br />
    36. 36. Before we get started<br />Who here is a developer?<br />Who here is a system administrator?<br />Who here has an existing deployment strategy?<br />Who here has edited a file in production?<br />Who here has wished they didn’t?<br />
    37. 37. Application Stages<br />| 36<br />Increasing Maturity of PHP Applications<br />Development<br />Testing/QA<br />Staging<br />Production<br />
    38. 38. Maturity<br />| 37<br />Increasing Maturity of PHP Applications<br />Uptime!!<br />Testing & Validating Environment<br />Testing & Validating Application<br />Distinct Dev Environments<br />Benefit<br />Where are you?<br />Structure<br />
    39. 39. Development<br />Purpose<br />To provide developers an environment to write their code<br />Characteristics<br />Should be similar to production, though it usually isn’t<br />Often security is more open<br />Usually local to the developer<br />
    40. 40. Consider Virtualization<br />
    41. 41. Testing<br />Purpose<br />To provide a non-programming environment to test functionality<br />Characteristics<br />Continuous Integration could be included<br />Generally no development occurs, only testing<br />Developers should not do the testing, if possible<br />Restricted outbound networking<br />Use Zend Server Monitoring and Code Tracing to help reproduce errors<br />
    42. 42. Prepared for and handed off to System Administrators<br />(the stuff we’re here to talk about)<br />
    43. 43. Staging<br />Purpose<br />To test your deployment process/scripting (not your code)<br />Characteristics<br />Developers generally do not have access, unless they are also the sysadmin<br />Very restricted outbound networking<br />Mirrors production as best as possible<br />
    44. 44. Change Control<br />A formal process used to ensure that changes are introduced in a controlled and coordinated manner<br />Protect against unnecessary changes introduced to a system without forethought<br />Requires documenting the release process<br />Software version number<br />Plans for rollback (tested? y/n)<br />Expected outage time<br />Customer impact<br />Done prior to doing anything in production<br />
    45. 45. Pre-Production (Optional)<br />Purpose<br />Test the code in the production environment without impacting customers<br />Characteristics<br />Not likely to have use in the cloud or large scale deployments<br />Deployed in production immediately prior to making it live<br />Test the application with production settings without customer interaction<br />
    46. 46. Production<br /><ul><li>Purpose
    47. 47. Do whatever it is that it’s supposed to be doing
    48. 48. Characteristics
    49. 49. Developers do not have access (as they might be tempted to fix something)</li></ul>Deployment should be done without requiring developer input<br /><ul><li>Very limited inbound traffic – generally only the service that is being served. i.e. HTTP</li></li></ul><li>Application considerations<br />Build the application to be aware of different environments<br />Zend_Config_* is good at this<br />Do not set the environment type in your code<br />Environment variables can be placed in server config<br />It means that you won’t accidentally keep your development settings turned on in production<br />Make production your default environment<br />Keeps accidental data leakage to a minimum<br />
    50. 50. Deployment Option 1 – rsync<br />Benefits<br />Easy<br />Already installed on your system<br />Drawbacks<br />No deployment script can be run<br />Not integrated with either PHP or the OS<br />Rollbacks require rolling back the entire source server<br />Does not understand versioning <br />One option is to move the source tree prior to the rsync<br />Uses xinetd<br />
    51. 51. Deployment Option 2 – Source Control<br />Benefits <br />Easy <br />You should already be using source control <br />Has versioning <br />Can have pre/post install scripts <br />Easy to do automatic deployments <br />Drawbacks <br />Do you want to put details on how to access your source code on your production box? <br />Requires that your source control is accessible from production<br />Deny access to .svn directories<br />
    52. 52. Deployment Option 3 – PEAR<br />Benefits<br />Designed for PHP<br />Very scriptable<br />Natural for doing things like clearing a cache<br />Has a better understanding of PHP<br />Cross platform compatible<br />Drawbacks<br />Requires admins to be familiar with PHP code<br />Limited to PHP deployment<br />Available tooling is restrictive/build your own package.xml<br />Define standardized locations (www, library, etc.)<br />
    53. 53. Deployment Option 3 – PEAR<br />Creating your PEAR Channel<br />pear channel-discover <br />pear install pirum/Pirum-beta<br />vi /var/www/pear/pirum.xml<br /><server>  <name>localhost/pear</name>  <summary>OurOrg PEAR channel</summary>  <alias>helloworld</alias>  <url>http://localhost/pear/</url></server><br />php ./pirum build /var/www/pear/<br />
    54. 54. Deployment Option 3 - PEAR<br />Creating your PEAR package<br />php ./build-pear.php<br />pear package<br />pirum add /var/www/pear/ HelloWorld-0.1.3.tgz<br />
    55. 55. Deploying your app<br />pear channel-discover localhost/pear<br />pear config-set www_dir /var/www<br />pear install helloworld/HelloWorld<br />pear clear-cache<br />pear upgrade helloworld/HelloWorld<br />Roll Back<br />pear uninstall helloworld/HelloWorld<br />pear install helloworld/HelloWorld-0.1.3<br />
    56. 56. Deployment Option 4 - OS-based<br />Benefits<br />Easily added as part of your server deployment<br />Very cloud friendly<br />Admins already know how to use it<br />Can describe OS-requirements/dependencies<br />Downtime limited to actual install, not network transfer time<br />Drawbacks<br />Depends on environment<br />Any PHP deployment scripting needs to be deployed with the application and executed in %post hook<br />
    57. 57. Which should you use?<br />Need something simple? rsync<br />Need to deploy to multiple disparate platforms? PEAR<br />Need to deploy internally/with minimal developer input?OS (yum/apt)<br />If using the OS consider maintaining an internal “blessed” repository instead of depending on the distribution<br />
    58. 58. Continuous Deployment<br />Using automated deployment<br />Very tight end-to-end integration<br />Requires a lot of trust that the developers have a fully tested application w/ backend and frontend code<br />Probably more work to manage if you can count the servers in your production environment with your fingers<br />Could be beneficial but make sure you know what you’re doing<br />You will need to have several testing experts<br />
    59. 59. Takeaways<br />No need for copy-and-paste deployment<br />No need for (S)FTP (Kinda not good for deployment)<br />Have an easy rollback mechanism<br />Prepare for errors in deployment<br />Try to minimize the amount of scripting needed for deployment<br />Consider using VMs on anything important to keep downtime to zero<br /><br />
    60. 60. Follow us!<br />Zend Technologies<br /><br /> (me!)<br />
    61. 61. Get this information and all the examples at…<br />