0
LinkedIn: A Professional Social Network Built
with Java™ Technologies and Agile Practices

Nick Dellamaggiore, Principal S...
Learn how Java™ and agile practices are employed in
building large-scale consumer internet sites like
LinkedIn, the world’...
Agenda

 LinkedIn(troduction)
 Agile engineering process
 Architecture
 Building LinkedIn with Java™
 Questions/Discussion...
2008 JavaOneSM Conference | java.com.sun/javaone |   4
LinkedIn
Your professional identity on the web
  The site                          The numbers
   • Launched in 2003      ...
Engineering Process

 Release often
  • Short, 2-4 week development cycles
 All dev tasks are broken down into small, mana...
Engineering Cards




                    2008 JavaOneSM Conference | java.com.sun/javaone |   7
Testing

 6500+ unit and integration tests
 500 HtmlUnit tests
 Large-scale Hudson deployment
  • 20+ nodes
  • Continuous...
“Celebrities” Test Network




                             2008 JavaOneSM Conference | java.com.sun/javaone |   9
Hudson




         2008 JavaOneSM Conference | java.com.sun/javaone | 10
Testing…
Lessons Learned

  Test suite too focused on integration
   • Long running time
   • Long-running setup/teardown ...
Architecture




               2008 JavaOneSM Conference | java.com.sun/javaone | 12
LinkedIn Architecture: 2003-2005




                               2008 JavaOneSM Conference | java.com.sun/javaone | 13
LinkedIn Architecture: 2006




                              2008 JavaOneSM Conference | java.com.sun/javaone | 14
LinkedIn Architecture: Today




                               2008 JavaOneSM Conference | java.com.sun/javaone | 15
LinkedIn Architecture: Today




                               2008 JavaOneSM Conference | java.com.sun/javaone | 16
Service-oriented Architecture

  Benefits                              Caveats
   • Each service can scale              • ...
Services




           2008 JavaOneSM Conference | java.com.sun/javaone | 18
The Cloud
Backend server caching the entire LinkedIn Network
  Graph cache
      • Updated via Databus
      • Persisted t...
The Cloud
The numbers

  22M nodes, 120 million edges
  12 GB JVM heap
   • In-memory caches implemented in C++, accessed ...
LinkedIn News

 Part of a new trend in LinkedIn
 Crawling the web for news
  • Distributed task management
  • Parsing tec...
News Service Architecture




                            2008 JavaOneSM Conference | java.com.sun/javaone | 22
LinkedIn: 99% Pure Java™

     Plumbing        Services Tier        Web Tier                       Tools

   Spring Remoti...
Why we love Java™ at LinkedIn

 Static typing is a lifesaver
  • Huge codebase, 1M+ lines of code
     • Refactoring with ...
LinkedIn: A Professional Social Network Built
with Java™ Technologies and Agile Practices
Nick Dellamaggiore (http://www.l...
Upcoming SlideShare
Loading in...5
×

LinkedIn - A Professional Network built with Java Technologies and Agile Practices

215,611

Published on

Published in: Business, Technology
23 Comments
260 Likes
Statistics
Notes
  • Dr. Maged Hamouda
    Commercial breeder corn
    Egypt
    Email:

    drmagedhamouda164@gmail.com
    magedhamouda70@gmail.com



    https://plus.google.com/photos/107246734046663904615/albums?banner=pwa
    https://www.facebook.com/pages/Breeding-plant-corn/404361229632443
    https://www.facebook.com/pages/Agricultural-research-in-crop-breeding/114171765319682
    https://www.facebook.com/pages/Research-company-breeding-and-production-of-field-crops/126687924062283
    https://twitter.com/ComMaged
    facbook: corn seeds
    cornseeds@yahoo.com
    . drmagedhamouda164@gmail.com
    https://twitter.com/following
    https://plus.google.com/photos/107246734046663904615/albums?banner=pwa&gpsrc=pwrd1#photos/107246734046663904615/albums?banner=pwa&gpsrc=pwrd1
    https://www.facebook.com/#!/corn.seeds/media_set?set=a.113854638683572.14627.100001771105085&type=1

    https://www.facebook.com/pages/Agricultural-research-in-crop-breeding/114171765319682?sk=photos_albums

    https://plus.google.com/photos/107246734046663904615/albums/5365384115257135137?banner=pwa
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • This answered my query of whehter java is used to make large scale social networking websites and got this slideshow while surfing. Now I am definitely going with Java for the social networking site I am going to build. : ) Yeayy...
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • good
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 不错
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Great and interesting slides. Thanks for sharing

    http://www.diyhousepainting.net/
    http://www.diyhousepainting.net/category/walls-painting/
    http://www.diyhousepainting.net/category/wood-painting/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
215,611
On Slideshare
0
From Embeds
0
Number of Embeds
51
Actions
Shares
0
Downloads
8,455
Comments
23
Likes
260
Embeds 0
No embeds

No notes for slide

Transcript of "LinkedIn - A Professional Network built with Java Technologies and Agile Practices"

  1. 1. LinkedIn: A Professional Social Network Built with Java™ Technologies and Agile Practices Nick Dellamaggiore, Principal Software Engineer Eishay Smith, Senior Software Engineer
  2. 2. Learn how Java™ and agile practices are employed in building large-scale consumer internet sites like LinkedIn, the world’s largest professional network. 2008 JavaOneSM Conference | java.com.sun/javaone | 2
  3. 3. Agenda LinkedIn(troduction) Agile engineering process Architecture Building LinkedIn with Java™ Questions/Discussion 2008 JavaOneSM Conference | java.com.sun/javaone | 3
  4. 4. 2008 JavaOneSM Conference | java.com.sun/javaone | 4
  5. 5. LinkedIn Your professional identity on the web The site The numbers • Launched in 2003 • 22 million members • 3 degree network • 4+ million unique visitors/month • Strong focus on privacy • 40 million page views/day The company • 2 million searches/day • Profitable since 2006 • 250K invitations sent/day • 250 employees • 1 million answers posted • Based in Mountain View, CA • We’re hiring! 2008 JavaOneSM Conference | java.com.sun/javaone | 5
  6. 6. Engineering Process Release often • Short, 2-4 week development cycles All dev tasks are broken down into small, manageable engineering cards Strong focus on testing Minimize meetings, status (standup meetings as needed) 2008 JavaOneSM Conference | java.com.sun/javaone | 6
  7. 7. Engineering Cards 2008 JavaOneSM Conference | java.com.sun/javaone | 7
  8. 8. Testing 6500+ unit and integration tests 500 HtmlUnit tests Large-scale Hudson deployment • 20+ nodes • Continuous integration build/smoke test on SVN commit • Nightly test run (full test suite) 2008 JavaOneSM Conference | java.com.sun/javaone | 8
  9. 9. “Celebrities” Test Network 2008 JavaOneSM Conference | java.com.sun/javaone | 9
  10. 10. Hudson 2008 JavaOneSM Conference | java.com.sun/javaone | 10
  11. 11. Testing… Lessons Learned Test suite too focused on integration • Long running time • Long-running setup/teardown routine • Exacerbated by move to SOA • Tests become redundant due to overlapping integration tests • Better: use mocks (EasyMock works great) 2008 JavaOneSM Conference | java.com.sun/javaone | 11
  12. 12. Architecture 2008 JavaOneSM Conference | java.com.sun/javaone | 12
  13. 13. LinkedIn Architecture: 2003-2005 2008 JavaOneSM Conference | java.com.sun/javaone | 13
  14. 14. LinkedIn Architecture: 2006 2008 JavaOneSM Conference | java.com.sun/javaone | 14
  15. 15. LinkedIn Architecture: Today 2008 JavaOneSM Conference | java.com.sun/javaone | 15
  16. 16. LinkedIn Architecture: Today 2008 JavaOneSM Conference | java.com.sun/javaone | 16
  17. 17. Service-oriented Architecture Benefits Caveats • Each service can scale • Diminished data integrity, independently consistency • Encourages decoupling, reduces • No cross-service transactions, code+deployment dependencies referential integrity or joins • Graceful degradation of • Backward compatibility functionality • “8 Fallacies of Distributed Computing” Takeaways • Not applicable to small or simple sites, startups • Build services with interfaces, POJOs, dependency injection • Be conscious of API design, granularity 2008 JavaOneSM Conference | java.com.sun/javaone | 17
  18. 18. Services 2008 JavaOneSM Conference | java.com.sun/javaone | 18
  19. 19. The Cloud Backend server caching the entire LinkedIn Network Graph cache • Updated via Databus • Persisted to disk on shutdown Transient network cache • Computationally intensive to build • Bound to member session • Sticky load balancing to Cloud Graph operations: • findRoute(m1, m2) • visit(visitor, deg) • visit(visitor, deg, since) 2008 JavaOneSM Conference | java.com.sun/javaone | 19
  20. 20. The Cloud The numbers 22M nodes, 120 million edges 12 GB JVM heap • In-memory caches implemented in C++, accessed via JNI 40 Cloud server instances deployed in production 2008 JavaOneSM Conference | java.com.sun/javaone | 20
  21. 21. LinkedIn News Part of a new trend in LinkedIn Crawling the web for news • Distributed task management • Parsing technologies: StAX, Rome Indexing and searching • Serving the right news to the right people Duplications Duplications Duplications • Real time problem, fast solution • 500 articles batch = 125K compares 2008 JavaOneSM Conference | java.com.sun/javaone | 21
  22. 22. News Service Architecture 2008 JavaOneSM Conference | java.com.sun/javaone | 22
  23. 23. LinkedIn: 99% Pure Java™ Plumbing Services Tier Web Tier Tools Spring Remoting LinkedIn Spring Louis (in-house fwk) Hudson (CI) ActiveMQ Eh-cache Spring MVC Eclipse+Mylyn Quartz Lucene Grails JIRA/Greenhopper HttpClient Jetty DWR JUnit, HtmlUnit 2008 JavaOneSM Conference | java.com.sun/javaone | 23
  24. 24. Why we love Java™ at LinkedIn Static typing is a lifesaver • Huge codebase, 1M+ lines of code • Refactoring with confidence • Code navigation (via IDE) • 20 active branches, compiler helps with merging • Scaling the engineering organization • Currently 50+ engineers, 8 teams… and growing • Excellent talent pool of Java™ engineers to hire from Community 2008 JavaOneSM Conference | java.com.sun/javaone | 24
  25. 25. LinkedIn: A Professional Social Network Built with Java™ Technologies and Agile Practices Nick Dellamaggiore (http://www.linkedin.com/in/nick) Eishay Smith (http://www.linkedin.com/in/eishay) We’re hiring!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×