Engineering the New LinkedIn Profile
Upcoming SlideShare
Loading in...5
×
 

Engineering the New LinkedIn Profile

on

  • 10,979 views

Overview of the new frontend architecture used for the New Profile at LinkedIn.

Overview of the new frontend architecture used for the New Profile at LinkedIn.

Statistics

Views

Total Views
10,979
Views on SlideShare
1,361
Embed Views
9,618

Actions

Likes
5
Downloads
53
Comments
5

25 Embeds 9,618

http://engineering.linkedin.com 9349
http://www.linkedin.com 54
http://www.newsblur.com 42
https://www.linkedin.com 33
http://linkedin1129.rssing.com 28
http://digg.com 27
http://cloud.feedly.com 21
http://newsblur.com 13
https://internal.autodesk360beta.com 11
http://www.feedspot.com 10
http://www.feedly.com 6
http://www.mybestcv2.co.il 5
http://translate.googleusercontent.com 3
https://twitter.com 2
http://localhost 2
http://feedly.com 2
https://www.google.com 2
http://www.tuicool.com 1
http://www.google.com 1
http://www.digg.com 1
http://www.goread.io 1
http://inappreciable15.wadukuri.com 1
http://feedspot.com 1
http://feeder.co 1
https://engineering.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • @joshclemm Thanks Josh. Another question: how do you handle the JS and CSS file dependencies that the embed templates might have? Are those just inlined as script/link tags in the embed templates, or does the mapper response programmatically return those and Fizzy handles putting them in the head or bottom of the page as appropriate?
    Are you sure you want to
    Your message goes here
    Processing…
  • Thanks ctoomey. For 1) We do use jQuery on the client but don't use any standard client MVC library. By reusing the same endpoints and same dust templates, it turned out to be fairly straightforward with a small amount of javascript. We did explore using client MVC, and as the page evolves over time and becomes more interactive, then we'll revisit that. Great observation for 2) Yes, we've already experimented with using Fizzy to render our templates server side on initial page load. Modern browsers are quite close performance wise, but this technique definitely helps perf on older browsers.
    Are you sure you want to
    Your message goes here
    Processing…
  • Cool stuff, couple questions. 1) Do you guys use a client MVC library or framework, or lower-level libraries like jQuery on the client? E.g., editing and then re-rendering the profile seems like a job ideally suited for a client MVC solution. 2) For performance reasons, have you guys explored doing the initial HTML generation on the server side with Fizzy and then subsequent interactivity/rendering on the client, even for modern/fast browsers? Along the lines of Twitter (https://blog.twitter.com/2012/improving-performance-twittercom) and Airbnb (http://nerds.airbnb.com/weve-launched-our-first-nodejs-app-to-product/). Seems that it wouldn't be too hard to do by rendering the JSON data into the templates but also including the JSON data and template URI for subsequent client manipulation.
    Are you sure you want to
    Your message goes here
    Processing…
  • Great question. I didn't go into the details for this presentation, but our Fizzy ATS plugin has the option to generate HTML using V8, making it SEO friendly. Look for a published blog post regarding Fizzy in the future!
    Are you sure you want to
    Your message goes here
    Processing…
  • by populating markup using client-side templating engine, will the page be SEO friendly? suppose i search for 'senior product manager at zoomjax' in google, will it return the profile page? As per my understanding, google bot just crawl the html but not json.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • New & interactive modules like: Connections, Following, Activity, PYMK, Similar profiles, Data insights, and improved Recommendations and Groups modules.
  • Profile 2.0 high level architectural diagram. The old page used to make 1 request to the profile web app, where a JSP with its various data was rendered, returning HTML. In the new architecture, we have a base page specifying which content we want on the page, and a new layer called Fizzy makes the requests for us. This results in parallel asynchronous calls to our Profile web app, retreiving JSON instead. Fizzy then streams back that JSON and uses Dust client templates to finally render page in the browser.
  • Convert incoming data models from mid-tier services into JSON
  • Convert incoming data models from mid-tier services into JSON

Engineering the New LinkedIn Profile Engineering the New LinkedIn Profile Presentation Transcript

  • Engineering the New ProfileJosh ClemmTech Lead for the New ProfileMarch 2013 | LinkedInDavid Fleming!Senior Product Manager at Zoomjax!San Francisco Bay Area | Software!!Previous!Education!Golden Phase, FixDex!Silicon Valley Business Academy!
  • Lets compare...
  • Really a brand new productRefreshed Look & FeelSimplified DesignSurfaced New &InteractiveModulesNew Data Insights View slide
  • New Features Improved in-lineediting experienceModules with In-linesearching &pagination View slide
  • The New Profile - Goals●  Represent your entire professional identityo  Not just your resumeo  Activity, Groups, Following, Connections,Insights about you and your network
  • The New Profile - Goals●  Represent your entire professional identityo  Not just your resumeo  Activity, Groups, Following, Connections,Insights about you and your network●  Needs to be more interactiveo  Keep users engaged on the pageo  Inline pagination, editing, searching
  • The New Profile - Goals●  Represent your entire professional identityo  Not just your resumeo  Activity, Groups, Following, Connections,Insights about you and your network●  Needs to be more interactiveo  Keep users engaged on the pageo  Inline pagination, editing, searching●  Needs to be fluid, flexible, fasto  Progressive renderingo  Maintain high performance
  • So how did we achieveall that?
  • Using a lot of Technologies
  • And in particular...
  • Lets look at our High LevelFrontend Architecture
  • GroupsContentServiceConnectionsContentServiceProfileContentServiceClient/Browser CDNLoad BalancerSCDSFizzyProfile WebAppProfileContentServiceProfile WebAppConnectionsContentServiceGroupsContentServiceDust/JS/CSSServerRetrieve DataModels fromMid-tier/profile/view?id=32top_card.tl,background.tl/profile/topcard /profile/background
  • GroupsContentServiceConnectionsContentServiceProfileContentServiceClient/Browser CDNLoad BalancerSCDSFizzyProfile WebAppProfileContentServiceProfile WebAppConnectionsContentServiceGroupsContentServiceDust/JS/CSSServerRetrieve DataModels/profile/view?id=32top_card.tl,background.tl/profile/topcard /profile/backgroundOur newarchitectureuses new techat all layers
  • GroupsContentServiceConnectionsContentServiceProfileContentServiceClient/Browser CDNLoad BalancerSCDSFizzyProfile WebAppProfileContentServiceProfile WebAppConnectionsContentServiceGroupsContentServiceDust/JS/CSSServerRetrieve DataModels/profile/view?id=32top_card.tl,background.tl/profile/topcard /profile/backgroundLets start at the bottomwith Mappers
  • Mappers - JSON endpoints●  Convert data models from mid-tier servicesinto JSON●  Each have an unique endpoint URL ( /profile/positions?id=42 )PositionsMapperJSON"positions": [{"position": {"id:{}"},]}Biz ProfileModelProfileModelRich MediaRest ModelReferencesModelProfile FlexModel
  • Mappers - an examplepublic class PictureMapper extends ProfileParametersAwareMapper!{!private PictureContentMap pictureContentMap;!private static final int ZOOMABLE_DIMENSION = 225;!!@Override!public void doService()!{!PictureContentModel picCM = getContent(PictureContentModel.class); //declare content needs!picCM.criteria().setId(getVieweeId()); //supply any input params!!assemble(); //invoke framework to retrieve declared content!!if (isResolvedWithoutErrors(picCM))!{!//all went well, create new content map to hold output (JavaBean-like objects)!pictureContentMap = ContentMap.proxyNew(PictureContentMap.class);!!Integer pictureWidth = picCM.getPictureWidth(); !pictureContentMap.setIsZoomable(pictureWidth >= ZOOMABLE_DIMENSION);!!if(pictureWidth != null && pictureWidth > 0)!{!pictureContentMap.setWidth(pictureWidth);!pictureContentMap.setHeight(picCM.getPictureHeight());!}!pictureContentMap.setPictureID(picCM.getPictureID());!}!}!// tell framework to add our map to final output (uses Jackson to process into JSON)!addOutput(pictureContentMap);!}!1!2!3!4!5!6!7!8!9!10!11!12!13!14!15!16!17!18!19!20!21!22!23!24!25!26!27!28!29!30!31!32!
  • Mappers - an examplepublic class PictureMapper extends ProfileParametersAwareMapper!{!private PictureContentMap pictureContentMap;!private static final int ZOOMABLE_DIMENSION = 225;!!@Override!public void doService()!{!PictureContentModel picCM = getContent(PictureContentModel.class); //declare content needs!picCM.criteria().setId(getVieweeId()); //supply any input params!!assemble(); //invoke framework to retrieve declared content!!if (isResolvedWithoutErrors(picCM))!{!//all went well, create new content map to hold output (JavaBean-like objects)!pictureContentMap = ContentMap.proxyNew(PictureContentMap.class);!!Integer pictureWidth = picCM.getPictureWidth(); !pictureContentMap.setIsZoomable(pictureWidth >= ZOOMABLE_DIMENSION);!!if(pictureWidth != null && pictureWidth > 0)!{!pictureContentMap.setWidth(pictureWidth);!pictureContentMap.setHeight(picCM.getPictureHeight());!}!pictureContentMap.setPictureID(picCM.getPictureID());!}!}!// tell framework to add our map to final output (uses Jackson to process into JSON)!addOutput(pictureContentMap);!}!1!2!3!4!5!6!7!8!9!10!11!12!13!14!15!16!17!18!19!20!21!22!23!24!25!26!27!28!29!30!31!32!Declare the data you needSet the data you wantcoming back as JSON
  • Mappers - features●  Modularo  A single mapper can retrieve data for a section§  Positions, Educations, Groups, etc. ●  Reusable & Combinableo  Mappers can be used more than once§  Positions Mapper is used for Positions section and thepositions part of Top Cardo  You can Aggregate Mappers under a commonroot element and a new URL endpoint
  • Profiles Many Mappers●  Each section on Profile has either a single Mapperor Aggregated Mapper (like Top Card) for its data
  • Profiles Many Mappers●  Each section on Profile has either a single Mapperor Aggregated Mapper (like Top Card) for its dataSummaryMapperPositionsMapperEducationsMapperPictureMapperTop CardMapper"TopCard": {"positions": {},"educations":{},"picture":{}}Profile Web AppConnectionsMapperJSON"Summary":{ "summary":"Im anexperienced..."}JSON
  • So we have these Mappers thatreturn JSON for each section.Who calls each one?
  • GroupsContentServiceConnectionsContentServiceProfileContentServiceClient/Browser CDNLoad BalancerSCDSFizzyProfile WebAppProfileContentServiceProfile WebAppConnectionsContentServiceGroupsContentServiceDust/JS/CSSServerRetrieve DataModels/profile/view?id=32top_card.tl,background.tl/profile/topcard /profile/backgroundFizzy - the UI aggregator
  • Fizzy●  Fizzy is an UI aggregator in 2 parts:o  Fizzy Server fetches the content youwanto  Fizzy Client renders it when ready●  Your base page defines its structure andwhich UI components it needs (called"embeds")*Fizzy Server is an Apache Traffic Server Plugin, Fizzy Client is a JS library
  • Profiles EmbedsTop Card EmbedActivity EmbedEmbedEmbedEmbedEmbedYet another embed
  • Profiles Embeds in code<html><body>...<div id=“wrapper”><div id=“profile”><script type=“embed” fs-id=“topcard” fs-uri=“/profile/topcard”/><script type=“embed” fs-id=“background” fs-uri=“/profile/background”/>...<script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/></div><div id=“insights”><script type=“embed” fs-id=“people_you_may_know” fs-uri=“/profile/pymk”/><script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/>...<script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/></div></div></body></html>12345678910111213141516171819
  • Profiles Embeds in code<html><body>...<div id=“wrapper”><div id=“profile”><script type=“embed” fs-id=“topcard” fs-uri=“/profile/topcard”/><script type=“embed” fs-id=“background” fs-uri=“/profile/background”/>...<script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/></div><div id=“insights”><script type=“embed” fs-id=“people_you_may_know” fs-uri=“/profile/pymk”/><script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/>...<script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/></div></div></body></html>12345678910111213141516171819Each embed specifies aMapper endpoint thatFizzy will fetch
  • Fizzy fetches data and sends itto the browser.Whats rendering the actualmarkup?
  • GroupsContentServiceConnectionsContentServiceProfileContentServiceClient/Browser CDNLoad BalancerSCDSFizzyProfile WebAppProfileContentServiceProfile WebAppConnectionsContentServiceGroupsContentServiceDust/JS/CSSServerRetrieve DataModels/profile/view?id=32top_card.tl,background.tl/profile/topcard /profile/backgroundYou guessed it...Dust client templates
  • {Dust} client templates●  LinkedIns latest and greatest renderinglayer●  Logic-less client template language●  Profile page made up of many* templates●  Templates + JSON = full markup!*over 400 actually
  • {Dust} - what it looks like<div id="top_card"><h4>{name}</h4><h5>{headline}</h5>{#info}<h6>{location} | {industry}</h6>{/info}</div>{"name": "Frank Stallone","headline": "Actor and Less Famous Brother","info":{"location": "Hollywood","industry": "Entertainment"}}
  • {Dust} - why its cool for profile●  Cached markup in CDNs and browserso  Members browse many profiles in a row●  Very DRY and reusableo  Improved development speedo  Templates within templates within templates...§  date range, degree badge, formatted summary field●  Super easy to refresh a section on the pageo  Re-render and replaceo  Useful in pagination, inline searching, and inlineediting
  • {Dust} on profile<hgroup>{>"tl/apps/profile/v2/embed/company_logo"/}<h4><a href="{link_title_pivot}" name=title>{title_highlight|s}</a>{?selfView}<span class="edit-tools"><a class="edit-section">{i18n_Edit}</a></span>{/selfView}</h4><h5>{?companyName}{>"tl/apps/profile/v2/embed/company_link" track_param="prof-exp"/}{/companyName}</h5></hgroup><span class="experience-date-locale">{>"tl/apps/profile/v2/partial/daterange"/}{! Location !}{@pre.fmt key="fmt_location" type="geo.region" value="{location}" render="false"/}{?fmt_location}<span class="locality">{fmt_location}</span>{:else}{?locationName}<span class="locality">{locationName}</span>{/locationName}{/fmt_location}</span>{>"tl/apps/profile/v2/partial/summary_field" _summary=summary/}{>"tl/apps/profile/v2/partial/associated_content" trkCodePrefix="exp"/}
  • {Dust} on profile<hgroup>{>"tl/apps/profile/v2/embed/company_logo"/}<h4><a href="{link_title_pivot}" name=title>{title_highlight|s}</a>{?selfView}<span class="edit-tools"><a class="edit-section">{i18n_Edit}</a></span>{/selfView}</h4><h5>{?companyName}{>"tl/apps/profile/v2/embed/company_link" track_param="prof-exp"/}{/companyName}</h5></hgroup><span class="experience-date-locale">{>"tl/apps/profile/v2/partial/daterange"/}{! Location !}{@pre.fmt key="fmt_location" type="geo.region" value="{location}" render="false"/}{?fmt_location}<span class="locality">{fmt_location}</span>{:else}{?locationName}<span class="locality">{locationName}</span>{/locationName}{/fmt_location}</span>{>"tl/apps/profile/v2/partial/summary_field" _summary=summary/}{>"tl/apps/profile/v2/partial/associated_content" trkCodePrefix="exp"/}Partial templates like this areused on almost all thebackground sections
  • We have our Mappers, Fizzy,and Dust templates...Lets bring it all together.
  • Feature: Inline Editing●  Dust + Mappers make this easy●  Dust templates for view and edit●  Mappers return just the data we need to refresh"View" template"Edit" templateSummary Section
  • Feature: Inline Editing●  Dust + Mappers make this easy●  Dust templates for view and edit●  Mappers return just the data we need to refresh"View" template"Edit" templateSummary SectionWhen edit link isclicked, render"edit" template
  • Feature: Inline Editing●  Dust + Mappers make this easy●  Dust templates for view and edit●  Mappers return just the data we need to refresh"View" template"Edit" templateSummary SectionOn submit, ourendpoint sendsback JSON, andwe re-render
  • Inline editing - ExampleEither the "view" or"edit" template isshown at a time
  • Inline editing sounds easy...●  Issue: Profile data is highly coupled with differentsections.o  Adding/editing a position needs to be reflectedon Top Card...o  Deleting a project needs to also be removedfrom any associated position...●  Solution: since we have mappers for each section,we know which to re-fetch upon a save andrefresh the respective templates
  • What about pagination, search?●  Same idea but easier (not coupled with other sections)●  Mappers can take URL offset paramsSwitch out apartial templatecontaining just listof profiles
  • Lets talk performance
  • High Performance Page●  Profile is the most trafficked page onLinkedIn●  Profile fetches 48+ different types ofcontent on each page loado  This content results in 250+ totaldownstream calls
  • High Performance Page●  Profile is the most trafficked page onLinkedIn●  Profile fetches 48+ different types ofcontent on each page loado  This content results in 250+ totaldownstream callsBottom Line: We need to be fast, butneed to consider downstream fanout
  • High Performance - Parallel RequestsFizzyProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppIn the beginning, Profile had 15embeds with 15 different endpointswhich is great for speed...
  • High Performance - Parallel RequestsFizzyProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceThats a lot of downstream calls, oftenrequesting the same data too.ContentServiceContentService
  • High Performance - Parallel RequestsFizzyProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceand those calls call more and more... until...
  • High Performance - Parallel RequestsFizzyProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppProfileAppContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceContentServiceaaaaand the sites down
  • High Performance - Parallel Requests●  Tradeoff: Speed vs Scalability●  15 parallel calls will be fast but withProfiles load will take down site
  • High Performance - Parallel Requests●  Tradeoff: Speed vs Scalability●  15 parallel calls will be fast but withProfiles load will take down siteBatched Endpoints to the rescue
  • Batching Calls to MappersOriginally, we hadseparate endpoints (URIs)for each embed<html><body>...<div id=“wrapper”><div id=“profile”><script type=“embed” fs-id=“topcard” fs-uri=“/profile/topcard”/><script type=“embed” fs-id=“background” fs-uri=“/profile/background”/>...<script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/></div><div id=“insights”><script type=“embed” fs-id=“peeps_you_may_know” fs-uri=“/profile/pymk”/><script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/>...<script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/></div></div></body></html>12345678910111213141516171819
  • Batching Calls to MappersWe now tell framework to batchthese two endpoints(Fizzy knows how to deliver theright data to the right embed)<html><body>...<div id=“wrapper”><div id=“profile”><script type=“embed” fs-id=“topcard” fs-uri=“/profile/mappers?a=topcard,background”/><script type=“embed” fs-id=“background” fs-uri=“/profile/mappers?a=topcard,background”/>...<script type=“embed” fs-id=“connections” fs-uri=“/profile/connections”/></div><div id=“insights”><script type=“embed” fs-id=“peeps_you_may_know” fs-uri=“/profile/pymk”/><script type=“embed” fs-id=“strength_meter” fs-uri=“/profile/strength”/>...<script type=“embed” fs-id=“in_common” fs-uri=“/profile/incommon”/></div></div></body></html>12345678910111213141516171819
  • Batching Calls in Profile●  Today, we use between 3-5 parallelrequests to Profile●  Its a good balance of speed vs.scalability●  Batching requests that need the samedata has the added benefit of lessdownstream calls
  • Progressive RenderingWith parallel fetch, Profile modules render whenready for improved perceived performance
  • Progressive RenderingWith parallel fetch, Profile modules render whenready for improved perceived performance
  • How else to improve page loadtimes?
  • Profiles can be long...EmbedEmbedEmbedEmbedEmbedEmbedEmbedProfile Page
  • Optimize for Above the FoldEmbedEmbedEmbedEmbedEmbedEmbedEmbedProfile PageThe foldNo needto renderNo needto renderor fetch
  • Optimize for Above the Fold●  The New Profile renders above the foldas fast as we cano  Deferring everything at the bottomo  Reduces the static content we need to initiallydownload
  • Optimize for Above the Fold●  The New Profile renders above the foldas fast as we cano  Deferring everything at the bottomo  Reduces the static content we need to initiallydownloadSorry guys, we’llget to you later
  • Optimize for Above the Fold●  The New Profile defers fetching themodules at the bottom of the pageo  Improves server side assembly timeso  Lowers payload and improves network timeo  Improves client rendering times
  • Optimize for Above the Fold●  The New Profile defers fetching themodules at the bottom of the pageo  Improves server side assembly timeso  Lowers payload and improves network timeo  Improves client rendering timesGo fetch!
  • So we covered the product,technologies, and somefeatures... lets revisit our goals.
  • Revisiting the New Profile Goals●  Needs to surface your entire professional identityo  Easily expose new data endpoints●  Needs to be more interactiveo  Inline editing, searching●  Needs to beo  Fluido  Flexibleo  Fast
  • Revisiting the New Profile Goals●  Needs to surface your entire professional identityo  Easily expose new data endpoints (Mappers)●  Needs to be more interactiveo  Inline editing, searching (Dust + Mappers)●  Needs to beo  Fluid (Fizzy progressive rendering)o  Flexible (Fizzy deferred rendering)o  Fast (Fizzy parallel calls + defer fetch, batchingMappers, Dust template caching)
  • Takeaways●  Mappers for each module makes senseo  You can combine and reuse with easeo  A must if refreshing part of page●  When structuring your page, start with manyembeds o  You can control number of requests, whenembeds are rendered, and when to fetch data●  Many partial templates are a good thingo  Allows you finer control over what to re-rendero  Improves developer speed
  • Takeaways cont.●  Make these technologies work for youo  Our Mappers are structured to guaranteeminimal downstream callso  Take advantage of decoupling rendering layerwith server side endpoints§  Engineers can build endpoints§  Web devs can start the templates withmocked JSON data
  • Questions?