Your SlideShare is downloading. ×
0
Octubre, 2009
Tuenti Tech Teams Frontend, Backend, Systems and more, working together
<ul><ul><li>Web </li></ul></ul>Tuenti Tech Teams - Frontend
<ul><ul><li>Web </li></ul></ul>Tuenti Tech Teams - Frontend
<ul><ul><li>Mobile Web </li></ul></ul>Tuenti Tech Teams - Frontend
<ul><ul><li>Frameworks </li></ul></ul><ul><ul><ul><li>Javascript </li></ul></ul></ul><ul><ul><ul><ul><li>Client-side Profi...
<ul><ul><li>Product teams </li></ul></ul><ul><ul><ul><li>Developers - JS/PHP - frontend and backend </li></ul></ul></ul><u...
<ul><ul><ul><li>Make possible the &quot;web scale&quot; 20.000.000.000 page views / month 3.000.000 page views / day in tu...
<ul><ul><ul><li>DB design ...                            ... forgetting about the old theory </li></ul></ul></ul><ul><ul><...
<ul><ul><ul><li>  Managing our amazing servers: </li></ul></ul></ul>Tuenti Tech Teams - Systems
<ul><ul><ul><li>Fundamental, the base of everything!!! </li></ul></ul></ul><ul><ul><ul><li>Network design, systems archite...
A sample case Developing the Tuenti IM
<ul><ul><li>Large scale & cost-effective web-based IM service </li></ul></ul><ul><ul><ul><li>Open source + innovative idea...
<ul><ul><li>Designed in Ericsson’s Computer Science Lab </li></ul></ul><ul><ul><ul><li>Ericsson ⇒ AXD301 ATM switch </li><...
<ul><ul><li>The challenge,  </li></ul></ul><ul><ul><ul><li>Handle even more concurrent users per server </li></ul></ul></u...
<ul><ul><li>Our strategy, </li></ul></ul><ul><ul><ul><li>Benchmark </li></ul></ul></ul><ul><ul><ul><li>Optimize </li></ul>...
<ul><ul><li>Build a rich UI </li></ul></ul><ul><ul><ul><li>Increase user engagement </li></ul></ul></ul><ul><ul><ul><li>Ma...
 
<ul><ul><li>Make the chat visible at the very first page load </li></ul></ul><ul><ul><ul><li>Show the buddy list </li></ul...
   
<ul><ul><li>Discard IE6 </li></ul></ul><ul><ul><ul><li>SLOW javascript </li></ul></ul></ul><ul><ul><ul><li>Style limitatio...
<ul><ul><li>Send XMPP requests from the client to the jabber servers </li></ul></ul><ul><ul><ul><li>Pick a JS Jabber libra...
<ul><ul><li>Fault tolerant request engine </li></ul></ul><ul><ul><ul><li>Users poor connections </li></ul></ul></ul><ul><u...
gracias [email_address] http://jobs.tuenti.com
Upcoming SlideShare
Loading in...5
×

Tuenti teams - Php Conference

2,165

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,165
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • La misión del equipo de backend es posibilitar la &amp;quot;escala web&amp;quot;.  De hecho a veces nos llamamos &amp;quot;Scalability team&amp;quot;. La web ya es un producto de consumo general y con los dispositivos móviles e internet en trabajo, se consume a cualquier hora. La audiencia potencial son 1.600 millones de personas. Evidentemente no tenemos que soportar a todos, pero si diseñar una infraestrcutura que escale lo más linearmente posible con el número de servidores. Sólo así se está preparado para la escala web.
  • DB design: La escalabilidad requiere cambios en la forma de hacer las cosas. Desnormalizar, particionar, archivar... Olvidar Foreign keys Mover la lógica a la aplicación por la escalabilidad. Bugs pueden costar datos. Muchas pruebas. El siguiente punto es la caché. No se trata de cachear páginas, eso es fácil. Se trata de que la caché sea un almacen de datos más, gestionado de nuevo por la aplicación. Tuenti es prácticamente en tiempo real, pero todo se cachea. Todo esto requiere cambios en la forma de pensar. El código no es normal, hay que pensar en concurrencia, condiciones de carrera, errores... Muy difícil testear escalabilidad, y muy pocos sitios donde se pueda aprender. La única manera de probar realmente un nuevo sistema es en producción. Hablaremos luego un poco de cómo se puede hacer. Experiencia. Cada bit cuenta, ser consciente de qué hace una llamada, qué es más eficiente.
  • Sistemas es la base de todo. Este equipo se encarga de que todo funcione. No hay que pensar en sistemas como los que instalan servidores, es mucho más que eso. En sistemas se encargan del diseño de la arquitectura  de red y de sistemas. Algunos de los problemas que hay que afrontar son el tráfico interno, latencias, aislar tráfico de las granjas de servidores, entre otras cosas. Por supuesto un punto importantísimo es la gestión de los balanceadores, no utilizamos big iron, sino simples máquinas linux, bien configuradas, con software robusto. Y funciona!
  • Soft real time Lightweight processes   Fast message passing Share nothing philosophy   Efficient generational garbage collection   Robustness   Simple and consistent error recovery model   Supervision hierarchies   On-line code upgrading support
  • Transcript of "Tuenti teams - Php Conference"

    1. 1. Octubre, 2009
    2. 2. Tuenti Tech Teams Frontend, Backend, Systems and more, working together
    3. 3. <ul><ul><li>Web </li></ul></ul>Tuenti Tech Teams - Frontend
    4. 4. <ul><ul><li>Web </li></ul></ul>Tuenti Tech Teams - Frontend
    5. 5. <ul><ul><li>Mobile Web </li></ul></ul>Tuenti Tech Teams - Frontend
    6. 6. <ul><ul><li>Frameworks </li></ul></ul><ul><ul><ul><li>Javascript </li></ul></ul></ul><ul><ul><ul><ul><li>Client-side Profiling </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Sample perceived load times! This is what matters to the users! </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Presentation / UI </li></ul></ul></ul><ul><ul><ul><li>Navigation </li></ul></ul></ul>Tuenti Tech Teams - Frontend Stick  together! Have fun!
    7. 7. <ul><ul><li>Product teams </li></ul></ul><ul><ul><ul><li>Developers - JS/PHP - frontend and backend </li></ul></ul></ul><ul><ul><ul><li>Product managers </li></ul></ul></ul><ul><ul><ul><li>Designers </li></ul></ul></ul><ul><ul><ul><li>Do </li></ul></ul></ul><ul><ul><ul><ul><li>Product development </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Product design </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Software design </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Everybody's input matters - it's a team effort! Everybody has to be proud of the result! </li></ul></ul></ul></ul>Tuenti Tech Teams - Frontend
    8. 8. <ul><ul><ul><li>Make possible the &quot;web scale&quot; 20.000.000.000 page views / month 3.000.000 page views / day in tuenti mobile 2.500.000 uploaded photos / day 72 minutes per user per day 500+ servers </li></ul></ul></ul>Tuenti Tech Teams - Backend
    9. 9. <ul><ul><ul><li>DB design ...                            ... forgetting about the old theory </li></ul></ul></ul><ul><ul><ul><li>Cache, cache, cache </li></ul></ul></ul><ul><ul><ul><li>Change the way you think...                            ... concurrency, race conditions, failures </li></ul></ul></ul><ul><ul><ul><li>Hard to test scalability </li></ul></ul></ul><ul><ul><ul><li>Every bit counts </li></ul></ul></ul><ul><ul><ul><li>Know your software & work closely with systems </li></ul></ul></ul>Tuenti Tech Teams - Backend
    10. 10. <ul><ul><ul><li>  Managing our amazing servers: </li></ul></ul></ul>Tuenti Tech Teams - Systems
    11. 11. <ul><ul><ul><li>Fundamental, the base of everything!!! </li></ul></ul></ul><ul><ul><ul><li>Network design, systems architecture </li></ul></ul></ul><ul><ul><ul><li>Cope with problems: </li></ul></ul></ul><ul><ul><ul><ul><li>Internal network traffic, latencies, isolate traffic </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Load balancers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Electricity consumption </li></ul></ul></ul></ul><ul><ul><ul><ul><li>BGP, OSPF, multiple links with providers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Know how to find problems, know how everything works. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Deep linux knowledge, know how to optimize servers for each task. </li></ul></ul></ul></ul><ul><ul><ul><li>A big challenge to maintain and monitor hundreds of servers +500! </li></ul></ul></ul>Tuenti Tech Teams - Systems
    12. 12. A sample case Developing the Tuenti IM
    13. 13. <ul><ul><li>Large scale & cost-effective web-based IM service </li></ul></ul><ul><ul><ul><li>Open source + innovative ideas </li></ul></ul></ul><ul><ul><ul><li>Do not reinvent the wheel </li></ul></ul></ul><ul><ul><ul><ul><li>  Delay product launch indefinitely </li></ul></ul></ul></ul><ul><ul><ul><ul><li>  Repeat old mistakes </li></ul></ul></ul></ul><ul><ul><li>  XMPP is a mature, open, distributed & extensible middle-ware </li></ul></ul><ul><ul><ul><li>Next generation large-scale real-time web applications </li></ul></ul></ul><ul><ul><ul><li>Google Wave! </li></ul></ul></ul><ul><ul><li>1M concurrent chatting users </li></ul></ul><ul><ul><ul><li>...launching to everybody in a couple of days </li></ul></ul></ul><ul><ul><li>Get a high quality IM platform: ejabberd </li></ul></ul><ul><ul><ul><li>Extend + adapt + optimize </li></ul></ul></ul><ul><ul><ul><li>High performance, clustered & fault-tolerant </li></ul></ul></ul><ul><ul><ul><li>Open source + deployed all over the world </li></ul></ul></ul><ul><ul><ul><li>Implemented in Erlang/OTP </li></ul></ul></ul>Overview
    14. 14. <ul><ul><li>Designed in Ericsson’s Computer Science Lab </li></ul></ul><ul><ul><ul><li>Ericsson ⇒ AXD301 ATM switch </li></ul></ul></ul><ul><ul><ul><li>Nortel Networks (Alteon) ⇒ SSL accelerator </li></ul></ul></ul><ul><ul><ul><li>CouchDB, RabbitMQ, Yaws, MochiWeb, Tsung, ejabberd... </li></ul></ul></ul><ul><ul><li>  Distributed functional paradigm </li></ul></ul><ul><ul><ul><li>Simple and easy to learn </li></ul></ul></ul><ul><ul><ul><li>High level of abstraction </li></ul></ul></ul><ul><ul><ul><li>High productivity </li></ul></ul></ul><ul><ul><ul><li>Built in solid concurrency model </li></ul></ul></ul><ul><ul><ul><li>Explicit or transparent distribution </li></ul></ul></ul><ul><ul><ul><li>Asynchronous message passing </li></ul></ul></ul><ul><ul><ul><li>Soft real time </li></ul></ul></ul><ul><ul><ul><li>Robustness </li></ul></ul></ul><ul><ul><ul><li>Multi-core architectures </li></ul></ul></ul>A glimpse at Erlang/OTP
    15. 15. <ul><ul><li>The challenge, </li></ul></ul><ul><ul><ul><li>Handle even more concurrent users per server </li></ul></ul></ul><ul><ul><ul><ul><li>Optimize memory & CPU consumption </li></ul></ul></ul></ul><ul><ul><ul><li>Be ready for site/service growth </li></ul></ul></ul><ul><ul><ul><ul><li>Smart partitioning/load balancing strategies </li></ul></ul></ul></ul><ul><ul><ul><li>Integrate in existing backend </li></ul></ul></ul><ul><ul><ul><ul><li>State-less instant messaging service </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Data duplications / additional storage reqs </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>API integration </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Monitoring infrastructure </li></ul></ul></ul></ul><ul><ul><ul><li>Self-management when overloaded </li></ul></ul></ul><ul><ul><ul><li>Anti-abuse policies </li></ul></ul></ul><ul><ul><li>Controlled client implementation + not server federation </li></ul></ul><ul><ul><ul><li>Some cheating is allowed </li></ul></ul></ul>The backend/systems side (I)
    16. 16. <ul><ul><li>Our strategy, </li></ul></ul><ul><ul><ul><li>Benchmark </li></ul></ul></ul><ul><ul><ul><li>Optimize </li></ul></ul></ul><ul><ul><ul><li>Monitor </li></ul></ul></ul><ul><ul><ul><li>Dark launch </li></ul></ul></ul><ul><ul><ul><li>Optimize </li></ul></ul></ul><ul><ul><ul><ul><li>Performance bottlenecks </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Bugs </li></ul></ul></ul></ul><ul><ul><ul><li>Launch </li></ul></ul></ul><ul><ul><li>Probably largest Jabber/XMPP deployment in Spain </li></ul></ul><ul><ul><ul><li>> 100M routed messages first week on-line </li></ul></ul></ul><ul><ul><ul><li>Continuous  grown </li></ul></ul></ul><ul><ul><ul><li>Average 25M daily routed messages </li></ul></ul></ul>The backend/systems side (and II)
    17. 17. <ul><ul><li>Build a rich UI </li></ul></ul><ul><ul><ul><li>Increase user engagement </li></ul></ul></ul><ul><ul><ul><li>Make them use the chat </li></ul></ul></ul><ul><ul><li>  Browser issues </li></ul></ul><ul><ul><ul><li>Render time </li></ul></ul></ul><ul><ul><ul><li>CSS constraints </li></ul></ul></ul><ul><ul><li>Technical requirements </li></ul></ul><ul><ul><ul><li>XMPP JS library </li></ul></ul></ul><ul><ul><ul><li>Cross domain XHR </li></ul></ul></ul><ul><ul><ul><li>Fault tolerant client-side engine </li></ul></ul></ul><ul><ul><ul><li>Metrics </li></ul></ul></ul>The frontend side
    18. 19. <ul><ul><li>Make the chat visible at the very first page load </li></ul></ul><ul><ul><ul><li>Show the buddy list </li></ul></ul></ul><ul><ul><ul><li>One click - start chat </li></ul></ul></ul><ul><ul><li>Unobtrusive interface </li></ul></ul><ul><ul><ul><li>The user must be able to keep browsing the site with minimum impact </li></ul></ul></ul><ul><ul><ul><li>Deal with small screen resolutions and multiple conversations </li></ul></ul></ul><ul><ul><ul><li>Integrate with the rest of our site (i.e. video player) </li></ul></ul></ul>Building a rich UI
    19. 20.    
    20. 21. <ul><ul><li>Discard IE6 </li></ul></ul><ul><ul><ul><li>SLOW javascript </li></ul></ul></ul><ul><ul><ul><li>Style limitations (position fixed) </li></ul></ul></ul><ul><ul><ul><ul><li>Fake the behavior attaching events is expensive </li></ul></ul></ul></ul><ul><ul><ul><li>Thank god the number of IE6 users is going down </li></ul></ul></ul><ul><ul><li>IE7 render performance </li></ul></ul><ul><ul><ul><li>Save states of the rendered elements </li></ul></ul></ul><ul><ul><ul><li>Reduce DOM manipulation </li></ul></ul></ul><ul><ul><ul><ul><li>Reuse the buddy list module instead of repainting it </li></ul></ul></ul></ul><ul><ul><ul><li>Firebug is your best friend </li></ul></ul></ul><ul><ul><li>  Webkit browsers throwing generic errors from the JS library </li></ul></ul><ul><ul><ul><li>Build our own error wrapper </li></ul></ul></ul>Browser issues
    21. 22. <ul><ul><li>Send XMPP requests from the client to the jabber servers </li></ul></ul><ul><ul><ul><li>Pick a JS Jabber library </li></ul></ul></ul><ul><ul><ul><ul><li>Audit the code </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Performance tests </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Adapt and extend </li></ul></ul></ul></ul><ul><ul><li>Cross domain XHR requests </li></ul></ul><ul><ul><ul><li>Can't work with the current iframe approach </li></ul></ul></ul><ul><ul><ul><li>Approaches </li></ul></ul></ul><ul><ul><ul><ul><li>window.name  </li></ul></ul></ul></ul><ul><ul><ul><ul><li>cookie transport </li></ul></ul></ul></ul><ul><ul><ul><li>Finally </li></ul></ul></ul><ul><ul><ul><ul><li>iframe controller file </li></ul></ul></ul></ul>Technical requirements (I)
    22. 23. <ul><ul><li>Fault tolerant request engine </li></ul></ul><ul><ul><ul><li>Users poor connections </li></ul></ul></ul><ul><ul><ul><li>Multiple connections from different browsers/computers </li></ul></ul></ul><ul><ul><ul><li>Gracefully recover from server errors </li></ul></ul></ul><ul><ul><li>Metrics </li></ul></ul><ul><ul><ul><li>Chat usage patterns </li></ul></ul></ul><ul><ul><ul><li>Detect possible message delivery problems </li></ul></ul></ul><ul><ul><ul><li>Track active chat user engagement </li></ul></ul></ul><ul><ul><ul><li>Browser stats </li></ul></ul></ul>Technical requirements (II)
    23. 24. gracias [email_address] http://jobs.tuenti.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×