The document discusses Yahoo!'s strategy for building web applications using the Symfony framework, emphasizing the benefits of open source technologies and the importance of scalability, maintainability, and performance. It highlights Yahoo!'s adoption of Symfony for various services and projects, including Yahoo! Answers, while advocating for a modular approach to development and the use of components tailored to specific needs. The document also details the integration of Yahoo!'s developer tools and APIs, such as YQL (Yahoo! Query Language), aimed at facilitating developers in building applications and accessing web data.

1. Building a platform from open sourceDustin Whittle

3. OverviewWhy symfony?symfony vs. ysymfonyWhat does scaling really mean?Social Search: Delicious and AnswersInternational: ShopGenie.co.uk, FoxyTunes, Wretch.ccYahoo! Open StrategyWhat is the Yahoo! Open Stack?Developer ToolsYUI, Design Patterns, TutorialsData & Social APIsYQL: Yahoo! Query LanguageProfiles, Connections, Updates, …Geo, Flickr, Delicious, UpcomingYOS SDK for PHP

4. Who am I?Working with symfony since open sourcesymfony Core Team MemberResponsible for the development and support of ysymfony at Yahoo!Worked with Y! Answers, Delicious, Y! Widgets, Y! Bookmarks, Yahoo! Application PlatformDeveloper advocacy for Yahoo! Developer NetworkConsultantCommercial symfony support + training (USA)

5. EXAMPLES | TUTORIALS | CODE SAMPLESdeveloper.yahoo.com

6. Freebsd | linux | apache | php | mysql | BUGZILLA | hadoop | symfonyYahoo! Is powered by open source technologies

7. W3C | MICROFORMATS | OAUTH | OPENID | OPENSOCIALYAHOO! embraces open standards

8. YUI | BROWSER PLUS | DESIGN PATTERNS | R3 | YSLOW + PERFORMANCE RULESYAHOO! gives back to open source

9. YQL | PIPES | BOSS | CONTACTS | UPDATES | MAIL | DELICIOUS | FLICKR | UPCOMING | HOTJOBS | MAPS | FIREEAGLE | GEOLOCATION | LOCAL | TRAFFIC | WEATHER | MUSIC | ANSWERS | SHOPPING | FINANCE | TRAVELYAHOO! SHARES ITS DATA THROUGH OPEN APIs AND WEB SERVICES

10. Conferences | Hack Days | HackU | Tech Talks | YDN TheaterYAHOO! ENGAGES COMMUNITIES WITH OPEN HACK EVENTS around the world

12. UsersLoadBalancersFrontendBackendLinuxysymfony/ YUIApacheCustom ModulesPHPAPC, PEAR, PECL, Custom ExtensionsUserAPIMySQL/MemcacheWeb ServicesAd API

13. Why a frontend platform?Rasmus says “frameworks are not well suited for Yahoo!”Build applications to requirementsDo exactly what you need: no more, no lessUnderstand that frameworks add a lot of overheadChoosing functional components is a better fitDespite choosing open source or building your ownEveryone uses a frameworkIf you use open source, use only the pieces you need

14. Y! needs from a frontend platformFit existing environment (RHEL/PHP5/Apache)Development Cycle – How easy to develop, test, and deploy?Clean separation between data, logic, and display (MVC) Independent model layer to fit service oriented architectureExtensible and pluggableInternationalization and localization supportDetailed documentation and active community of supportOpen source and ability to contribute back

15. Why a framework at all?Another software layer (ysymfony, yphp, yapache)Factors out common patternsCode LayoutConfigurationURL RoutingAuthentication / SecurityForm Validation / RepopulationInternationalization / LocalizationEncourages good designAbstraction > Consistency > Maintainability

16. The choice to adopt symfony?PhilosophyFull-stack framework for building complex web applicationsAdopt best ideas from anywhere, using existing code if available (Mojavi, Prado, Rails, Django) DesignClean separation between Model, View, and ControllerController using modules and actionsViews using templates in straight PHP with helpersEasy to reuse view modules to compose a pageLayouts, Components, Partials, Slots

17. The choice to use symfonyConfigurability / FlexibilityFeatures we do not want are easily disabledUse of factories for easy customizationDocumentation / Support CommunityThe Definitive Guide to symfonyAskeet, Jobeet, Cookbooks, AdventsActive community with wiki, mailing lists, forums, irc channel

18. Why ysymfony for Yahoo! teams?Eliminate common patterns by adding a layer on PHPCode layout/structure (MVC)ConfigurationInternationalizationysymfony is just a toolkitLearn one set of toolsShift between multiple projectsConsistencyLong term maintainability through platform

20. How Yahoo! uses symfonyFull StackYahoo! BookmarksYahoo! AnswersDeliciousYahoo! WidgetsYahoo! Application PlatformShopGenie.co.uk, Foxytunes, Wretch.ccInternal Dashboards + Reporting ToolsComponentsDependency InjectionFormsCaching (APC/Memcache)Routing

21. symfony MVC framework

24. A look at Yahoo! Answershttp://answers.yahoo.comYahoo! Answers is the largest collection of human knowledge on the Web with more than 135 million users and 515 million answers worldwide (Yahoo! Internal Data, March 2008).Yahoo! Answers is the 2nd ranked education & reference site on the web (comScore)Available in 26 markets and 12 languages

25. Yahoo! Answers at the beginningStarted as a small development team on PHP4 from a fork of Yahoo! Taiwan Knowledge+Launched December 2005 by December 2006 there were 60 million users and 65 million answersThe code base eventually became difficult to maintain and iterate new featuresLarge distributed development teams (US / UK)

32. The big pictureA complete platform for building web applications from frameworksPHP FrameworkJavaScript FrameworkCSS FrameworkUI Design Patterns + Best PracticesDevelopment Tools (logger, profiler, debugger, docs)Unit + Functional Testing Frameworks (LIME / YUI Test)Deployment Tools (rsync deployment system)

33. What does Yahoo! change?Minor changes to fit our environmentMost changes are easily implemented via factories/plugins Dropped the ORM and pushed down the stack (SOA)Added a parallel API Dispatcher (ysfAPIClientPlugin)Added dimensions to configurations (ysfDimensionsPlugin)Integrated R3 translation/template management (ysfR3Plugin)R3 - http://developer.yahoo.com/r3/ Created a build and deployment solution (ysfBuildPlugin)Uses internal tools for packaging/deploymentIntegrate support for Y! User Interface libraries (ysfYUIPlugin)

34. Propel or Doctrine or ???No ORM for large projectsDoctrine for medium sized projectsService Oriented ArchitecturePlatforms/Backendsas services (reusable to all)Thin Controller/Fat Model (where model == services)Use PHP as the frontend glueNo heavy lifting in PHP = Push down the stackJava/C++/Erlang + JSON/XML

35. Localizing with dimensionsCascading Configuration based on YAMLFramework -> Project -> Application -> ModuleExtending the cascade to be based on dimensionsDimensions can be anything (and can be chained together)Data Center + Environment for customizing configurationsCulture for localizing user interface + dataTheme for customizing look and feelUser info (is user on corporate intranet?)Caching

36. symfony I18n + L10nTranslation + Template ManagementIntegrated with symfony i18n + view layers<r3:trans>hello</r3:trans> vs __(‘hello’)Dictionary Management (YALA)SQLite/XLIFF BackendAdded additional tasksImporting/Exporting XLIFF dictionaryGenerating sample translations for projectsFind missing strings

37. A build and deployment systemAggregate and minify stylesheets and javascriptsRewrite templates, css, js for CDN (Akamai, S3, …)Generate translations for configurations + templatesGenerate configuration cacheAggregate core classes + remove debug statementsRun lint, unit, functional testsPackage applications as .tgzDeployment via packagesIntegrated with Yahoo! internal tools

38. Hello World PerformanceHello world benchmarks are generally not a useful measurementYou don’t use a framework to write hello worlddie(‘hello world’);Performance is relative to the features you useORMI18NOutput EscapingForms/ValidationAbility to scale != performance

39. What does it mean to scale?A system whose performance improves after adding hardware, proportionally to the capacity added, is said to be a scalable system.High Availability + Scalability + PerformanceBigger dataset, more traffic, maintainableNot about performancePHP is slow, but it is very rarely your bottleneckLanguages do not scale, architectures doPlanning to grow and planning to failCapacity PlanningBusiness Continuity Planning

40. Scaling – PlanningPlanning hardware purchases and hosting options to have as much as you need without breaking your walletPartitioning and distributing databases to support large datasets and simultaneous transactionsOffline transactions and queuingMonitoring your applications to find and clear bottlenecksProviding service APIs and using services from other providers to increase your site's reach and capabilitiesThink Minimal, Plan to grow, Plan to fail

41. Scaling – The basics in PHPPHP is rarely the bottleneck (even though it can be slow)“Most performance comes not from the language, but from application design” – RasmusShare Nothing ArchitectureIndependent, self-sufficient, no single point of contentionNo local storage = No PHP SessionsUse a database (works for distributed)Use a small signed cookie (ideal)Important data in databaseIndividual expiration on session objectsSmall data itemsUse a distributed cacheMemcacheForget about small efficienciesPremature optimization is the root of all evil.“ vs. ‘, echo vs. print

42. Scaling Databases – The basicsDrop the ORMIt’s a choice of convenience over performanceMaster/Slave ReplicationFirst stepsHelps with reads, writes are still bottleneckPartitioningSegmenting dataSharding (horizontal partitioning)Segmenting data onto different physical machinesMake problems smaller, easier to grow

43. Improving latency with CachingAlways use PHP opcode cache (APC, Xcache, etc)Use for routing and i18n cacheMemcache (distributed cache)Use for view cacheDistributed invalidation can be a painsfViewCacheManager makes this easy!Be intelligent about cache_keys (uri, user, state)There is a fine line to cachingAt what point do you spend more time managing the cache, than reading from it?

44. Tweaking PerformanceDo not use .htaccess (move to real apache config)Set a minimal include pathIncrease realpath_cache_size + realpath_cache_ttlUse apc.stat=0Don’t use features you do not needDisable in settings.yml/ factories.ymlUse core_compile to aggregate classes to reduce file i/oRemove debug statements + optimize file lookupssfOptimizerPlugin / project:optimizeUse @routeName and use caching for factories

45. Symfony (v2)Symfony 2 is a set of cohesive yet decoupled componentsThis makes it much easier to use single component to solve a single problemWhich makes it easier to build micro frameworks that solve very specific problemsYahoo! Teams generally prefer solutions that are specific to their exact problemSelling the full stack can be difficult when a team only wants a few componentsSymfony 2 is the right direction, even if it breaks backwards compatability

46. Do it yourself for cheapOpen source software = FreeApachePHPMySQLMemcache / Perlbal / MogileFS / Squid / Gearmansymfony / Doctrine / Propel / SwiftNagios / CactiAmazon Shared Infrastructure = CheapEC2 CloudS3 Storage + CloudfrontSimpleDB

47. Y! OS – The Open Stack

48. What Yahoo! offer developersYahoo! Developer Network

49. Developer Tools (YUI, etc)

50. Social APIs

51. Profiles

52. Connections

53. Updates

54. Data APIs

55. Flickr, Delicious, Search, etc

56. Yahoo! Query Language

57. Yahoo! Application Platform

58. OpenSocial

59. developer.yahoo.com

60. What is Yahoo! Developer Network?The Yahoo! Developer Network offers open source tools and open data APIs to make it easy for developers to build applications and mashups.50+ APIs / Web ServicesDeveloper Dashboard to create/manage Oauth applicationsTutorials + Code Samples on using our apisComplete API DocumentationYahoo! User Interface librariesASTRA Flash ComponentsDesign Patterns LibraryEvangelism: Conferences / Theater / Blogs / Events

62. JavaScript FrameworkUtilities - YAHOO, Dom, Event, Animation, Browser History Manager, Connection Manager, Cookie, DataSource, Drag and Drop, Element, Get, ImageLoader, JSON, Resize, Selector, LoaderControls / Widgets - AutoComplete, Button, Calendar, Charts, Color Picker, DataTable, ImageCropper, Rich Text Editor, Slider, UploaderContainer (Module, Overlay, Panel, Tooltip, Dialog), Layout Manager, Menu, TabView, TreeViewDebug – Logger, Profiler, TestY! Developer Network – YUI Javascript

63. CSS FoundationReset - Neutralizes browser CSS stylesBase - Applies consistent style foundationFonts - Foundation for typography and font-sizingGrids - Thousands of wireframe layoutsUser Interface Design Patterns LibraryProven solutions to common interfaceshttp://developer.yahoo.com/ypatterns/Grade Browser Support / Progressive EnhancementY! Developer Network – YUI CSS

64. More than 275 functional exampleshttp://developer.yahoo.com/yui/examples/YSlow + Performance Ruleshttp://developer.yahoo.com/performanceYUI Bloghttp://yuiblog.com/Mailing List @ Yahoo! Groupshttp://tech.groups.yahoo.com/group/ydn-javascript/Y! Developer Network – Documentation

65. A Query Language for the WebSELECT * FROM INTERNET

66. Thousands of web services that provide valuable dataRequire developers to read documentation and form URLs/queries.Data is isolated and can not be combinedNeeds combining, tweaking, shaping even after it gets to the developer.Before YQL

67. SQL-Like LanguageSynonymous with Data accessFamiliar to developersExpressive enough to get the right dataSelf Describing - show, desc tableAllows you to query, filter and join data across Web Services.Y! Open Stack – YQL

69. YQL – Open TablesTwitterGoogleFacebookFriendfeedWesabeWhitepagesZillowSearchWeatherFlickrUpcomingDeliciousDopplrGithubNew York TimesShopping…any web service can be as easy as SQLAvailable on github - http://github.com/spullara/yql-tables/

70. YQL – Open Tables

71. YQL – Open Tables

72. YQL – Open Tables

73. select * from social.connectionsselect * from delicious.feeds.popularselect * from flickr.photos.interestingnessselect * from friendfeed.statusselect * from github.checkinsYQL - Examples

74. OAuth Endpointhttp://query.yahooapis.com/v1/yql?q=...Public Endpoint http://query.yahooapis.com/v1/public/yql?q=YQL Console http://developer.yahoo.com/yql/consoleYQL – API End Points

75. Books worth reading

76. (I will be around for the whole event, so please say hello and ask questions)Questions?

77. enjoy the rest ofsymfonylive 2010EXAMPLES | TUTORIALS | CODE SAMPLESdeveloper.yahoo.com

78. Want to join Yahoo?We are hiring!