SlideShare a Scribd company logo
LinkedIn: Network Updates Uncovered


           Ruslan Belkin
           Sean Dawson
Agenda

•    Quick Tour
•    Requirements (User Experience / Infrastructure)
•    Service API
•    Internal Architecture
•    Applications (e.g., Twitter Integration, Email Delivery)
•    Measuring Performance




•    Shameless self promotion
The Stack


       Environment          90% Java
                            5% Groovy
                            2% Scala
                            2% Ruby
                            1% C++
       Containers           Tomcat, Jetty
       Data Layer           Oracle, MySQL, Voldemort,
                            Lucene, Memcache
       Offline Processing   Hadoop
       Queuing              ActiveMQ
       Frameworks           Spring
The Numbers


              Updates Created   35M / week
              Update Emails     14M / week
              Service Calls     20M / day
                                230 / second
Stream View
Connection View
Profile
Groups
Mobile
Email




        NUS Email digest screenshot
HP without NUS
Expectations – User Experience

•    Multiple presentation views
•    Comments on updates
•    Aggregation of noisy updates
•    Partner Integration
•    Easy to add new updates to the system
•    Handles I18N and other dynamic contexts
•    Long data retention
Expectations - Infrastructure

•    Large number of connections, followers and groups
•    High request volume + Low Latency
•    Random distribution lists
•    Black/White lists, A/B testing, etc.
•    Tenured storage of update history
•    Tracking of click through rates, impressions
•    Supports real-time, aggregated data/statistics
•    Cost-effective to operate
Historical Note

     (homepage circa 2007)   •  Legacy “network update”
                                feature was a mixed bag of
                                detached services.
                             •  Neither consistent nor scalable
                             •  Tightly coupled to our Inbox
                             •  Migration plan
                                 •  Introduce API, unify all
                                    disparate service calls
                                 •  Add event-driven activity
                                    tracking with DB backend
                                 •  Build out the product
                                 •  Optimize!
Network Updates Service – Overview
Service API – Data Model



                           <updates>	
                            <NCON>	
                             <connection>	
                              <id>2</id>	
                              <firstName>Chris</firstName>	
                              <lastName>Yee</lastName>	
                             </connection>	
                            </NCON>	
                           </updates>
Service API – Post


  NetworkUpdatesNotificationService service =	
    getNetworkUpdatesNotificationService();	

  ProfileUpdateInfo profileUpdate = createProfileUpdate();	

  Set<NetworkUpdateDestination> destinations = 	
    Sets.newHashSet(	
      NetworkUpdateDestinations.newMemberFeedDestination(1213)	
    );	

  NetworkUpdateSource source = 	
    new NetworkUpdateMemberSource(1214);	

  Date updateDate = getClock().currentDate();	

  service.submitNetworkUpdate(source, 	
                              destinations,	
                              updateDate, 	
                              profileUpdate);
Service API – Retrieve


  NetworkUpdatesService service = getNetworkUpdatesService();	

  NetworkUpdateChannel channel =	
    NetworkUpdateChannels.newMemberChannel(1213);	

  UpdateQueryCriteria query = 	
    createDefaultQuery().	
    setRequestedTypes(NetworkUpdateType.PROFILE_UPDATE).	
    setMaxNumberOfUpdates(5).	
    setCutoffDate(ClockUtils.add(currentDate, -7));	

  NetworkUpdateContext context =	
    NetworkUpdateContextImpl.createWebappContext();	

  NetworkUpdatesSummaryResult result =	
    service.getNetworkUpdatesSummary(channel, 	
                                     query, 	
                                     context);
System at a glance
Data Collection – Challenges

•  How do we efficiently support collection in a dense social
   network
•  Requirement to retrieve the feed fast
•  But – there a lot of events from a lot of members and
   sources
•  And – there are multiplier effects
Option 1: Push Architecture (Inbox)

•  Each member has an inbox of notifications received from
   their connections/followees
•  N writes per update (where N may be very large)
•  Very fast to read
•  Difficult to scale, but useful for private or targeted
   notifications to individual users
Option 1: Push Architecture (Inbox)
Option 2: Pull Architecture

•  Each member has an “Activity Space” that contains their
   actions on LinkedIn
•  1 write per update (no broadcast)
•  Requires up to N reads to collect N streams
•  Can we optimize to minimize the number of reads?
    -  Not all N members have updates to satisfy the query
    -  Not all updates can/need to be displayed on the screen
    -  Some members are more important than others
    -  Some updates are more important than others
    -  Recent updates generally are more important than older ones
Pull Architecture – Writing Updates
Pull Architecture – Reading Updates
Storage Model

•  L1: Temporal
   •    Oracle
   •    Combined CLOB / varchar storage
   •    Optimistic locking
   •    1 read to update, 1 write (merge) to update
   •  Size bound by # number of updates and retention policy
•  L2: Tenured
   •  Accessed less frequently
   •  Simple key-value storage is sufficient (each update has a unique ID)
   •  Oracle/Voldemort
Member Filtering

•  Need to avoid fetching N feeds (too expensive)
•  Filter contains an in-memory summary of user activity
   •  Needs to be concise but representative
   •  Partitioned by member across a number of machines
•  Filter only returns false-positives, never false-negatives
•  Easy to measure heuristic; for the N members that I
   selected, how many of those members actually had good
   content
•  Tradeoff between size of summary and filtering power
Member Filtering
Commenting

•  Users can create discussions around updates
•  Discussion lives in our forum service
•  Denormalize a discussion summary onto the tenured
   update, resolve first/last comments on retrieval
•  Full discussion can be retrieved dynamically
Twitter Sync

•  Partnership with Twitter
•  Bi-directional flow of status
   updates
•  Export status updates,
   import tweets
•  Users register their twitter
   account
•  Authorize via OAuth
Twitter Sync – Overview
Email Delivery

•  Multiple concurrent email generating tasks
•  Each task has non-overlapping ID range generators to avoid
   overlap and allow parallelization
•  Controlled by task scheduler
    •  Sets delivery time
    •  Controls task execution status, suspend/resume, etc
•  Caches common content so it is not re-requested
•  Tasks deliver content to Notifier, which packages the
   content into an email via JSP engine
•  Email is then delivered to SMTP relays
Email Delivery
Email Delivery
What else?

Brute force methods for scaling:
•  Shard databases
•  Memcache everything
•  Parallelize everything
•  User-initiated write operations are asynchronous when
   possible
Know your numbers

•    Bottlenecks are often not where you think they are
•    Profile often
•    Measure actual performance regularly
•    Monitor your systems
•    Pay attention to response time vs transaction rate
•    Expect failures
Measuring Performance
Another way of measuring performance
LinkedIn is a great place to work
Questions?

  Ruslan Belkin (http://www.linkedin.com/in/rbelkin)
Sean Dawson (http://www.linkedin.com/in/seandawson)

More Related Content

What's hot

Ronald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11g
Ronald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11gRonald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11g
Ronald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11g
Getting value from IoT, Integration and Data Analytics
 
Queues, Pools, Caches
Queues, Pools, CachesQueues, Pools, Caches
Queues, Pools, Caches
Gwen (Chen) Shapira
 
Spring in the Cloud
Spring in the CloudSpring in the Cloud
Spring in the Cloud
Eberhard Wolff
 
Distributed Design and Architecture of Cloud Foundry
Distributed Design and Architecture of Cloud FoundryDistributed Design and Architecture of Cloud Foundry
Distributed Design and Architecture of Cloud Foundry
Derek Collison
 
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...
Alex Gorbachev
 
New life inside monolithic application
New life inside monolithic applicationNew life inside monolithic application
New life inside monolithic application
Taras Matyashovsky
 
The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13
The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13
The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13
Dominopoint - Italian Lotus User Group
 
Hyper-V’s Virtualization Enhancements - EPC Group
Hyper-V’s Virtualization Enhancements - EPC GroupHyper-V’s Virtualization Enhancements - EPC Group
Hyper-V’s Virtualization Enhancements - EPC Group
EPC Group
 
Copy Data Management for the DBA
Copy Data Management for the DBACopy Data Management for the DBA
Copy Data Management for the DBA
Kellyn Pot'Vin-Gorman
 
Virtualization and Containers
Virtualization and ContainersVirtualization and Containers
Virtualization and Containers
Kellyn Pot'Vin-Gorman
 
12 Things About WebLogic 12.1.3 #oow2014 #otnla15
12 Things About WebLogic 12.1.3 #oow2014 #otnla1512 Things About WebLogic 12.1.3 #oow2014 #otnla15
12 Things About WebLogic 12.1.3 #oow2014 #otnla15
Frank Munz
 
Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...
Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...
Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...
Dominopoint - Italian Lotus User Group
 
Stay productive while slicing up the monolith
Stay productive while slicing up the monolith Stay productive while slicing up the monolith
Stay productive while slicing up the monolith
Markus Eisele
 
AAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing Workloa
AAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing WorkloaAAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing Workloa
AAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing Workloa
WASdev Community
 
AAI-1445 Managing Dynamic Workloads with WebSphere ND and in the Cloud
AAI-1445 Managing Dynamic Workloads with WebSphere ND and in the CloudAAI-1445 Managing Dynamic Workloads with WebSphere ND and in the Cloud
AAI-1445 Managing Dynamic Workloads with WebSphere ND and in the Cloud
WASdev Community
 
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBossArchitectForum
 
ARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUG
ARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUGARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUG
ARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUG
Markus Eisele
 
Bridging the Developer and the Datacenter
Bridging the Developer and the DatacenterBridging the Developer and the Datacenter
Bridging the Developer and the Datacenter
lurs83
 
Zarafa SummerCamp 2012 - Steve Hardy Friday Keynote
Zarafa SummerCamp 2012 - Steve Hardy Friday KeynoteZarafa SummerCamp 2012 - Steve Hardy Friday Keynote
Zarafa SummerCamp 2012 - Steve Hardy Friday Keynote
Zarafa
 
Stay productive while slicing up the monolith
Stay productive while slicing up the monolithStay productive while slicing up the monolith
Stay productive while slicing up the monolith
Markus Eisele
 

What's hot (20)

Ronald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11g
Ronald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11gRonald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11g
Ronald van Luttikhuizen - Effective fault handling in SOA Suite and OSB 11g
 
Queues, Pools, Caches
Queues, Pools, CachesQueues, Pools, Caches
Queues, Pools, Caches
 
Spring in the Cloud
Spring in the CloudSpring in the Cloud
Spring in the Cloud
 
Distributed Design and Architecture of Cloud Foundry
Distributed Design and Architecture of Cloud FoundryDistributed Design and Architecture of Cloud Foundry
Distributed Design and Architecture of Cloud Foundry
 
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...
Bridging Oracle Database and Hadoop by Alex Gorbachev, Pythian from Oracle Op...
 
New life inside monolithic application
New life inside monolithic applicationNew life inside monolithic application
New life inside monolithic application
 
The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13
The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13
The Latest and Greatest from OpenNTF and the IBM Social Business Toolkit, #dd13
 
Hyper-V’s Virtualization Enhancements - EPC Group
Hyper-V’s Virtualization Enhancements - EPC GroupHyper-V’s Virtualization Enhancements - EPC Group
Hyper-V’s Virtualization Enhancements - EPC Group
 
Copy Data Management for the DBA
Copy Data Management for the DBACopy Data Management for the DBA
Copy Data Management for the DBA
 
Virtualization and Containers
Virtualization and ContainersVirtualization and Containers
Virtualization and Containers
 
12 Things About WebLogic 12.1.3 #oow2014 #otnla15
12 Things About WebLogic 12.1.3 #oow2014 #otnla1512 Things About WebLogic 12.1.3 #oow2014 #otnla15
12 Things About WebLogic 12.1.3 #oow2014 #otnla15
 
Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...
Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...
Back to the Future: Understand and Optimize your IBM Notes and Domino Infrast...
 
Stay productive while slicing up the monolith
Stay productive while slicing up the monolith Stay productive while slicing up the monolith
Stay productive while slicing up the monolith
 
AAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing Workloa
AAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing WorkloaAAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing Workloa
AAI-2075 Evolving an IBM WebSphere Topology to Manage a Changing Workloa
 
AAI-1445 Managing Dynamic Workloads with WebSphere ND and in the Cloud
AAI-1445 Managing Dynamic Workloads with WebSphere ND and in the CloudAAI-1445 Managing Dynamic Workloads with WebSphere ND and in the Cloud
AAI-1445 Managing Dynamic Workloads with WebSphere ND and in the Cloud
 
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
 
ARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUG
ARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUGARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUG
ARCHITECTING LARGE ENTERPRISE JAVA PROJECTS - vJUG
 
Bridging the Developer and the Datacenter
Bridging the Developer and the DatacenterBridging the Developer and the Datacenter
Bridging the Developer and the Datacenter
 
Zarafa SummerCamp 2012 - Steve Hardy Friday Keynote
Zarafa SummerCamp 2012 - Steve Hardy Friday KeynoteZarafa SummerCamp 2012 - Steve Hardy Friday Keynote
Zarafa SummerCamp 2012 - Steve Hardy Friday Keynote
 
Stay productive while slicing up the monolith
Stay productive while slicing up the monolithStay productive while slicing up the monolith
Stay productive while slicing up the monolith
 

Similar to Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered

Linkedin NUS QCon 2009 slides
Linkedin NUS QCon 2009 slidesLinkedin NUS QCon 2009 slides
Linkedin NUS QCon 2009 slides
ruslansv
 
InfoQ QCon San Francisco 2009
InfoQ QCon San Francisco 2009InfoQ QCon San Francisco 2009
InfoQ QCon San Francisco 2009
Sean Dawson
 
Do I Need A Service Mesh.pptx
Do I Need A Service Mesh.pptxDo I Need A Service Mesh.pptx
Do I Need A Service Mesh.pptx
PINGXIONG3
 
Do You Need A Service Mesh?
Do You Need A Service Mesh?Do You Need A Service Mesh?
Do You Need A Service Mesh?
NGINX, Inc.
 
.NET microservices with Azure Service Fabric
.NET microservices with Azure Service Fabric.NET microservices with Azure Service Fabric
.NET microservices with Azure Service Fabric
Davide Benvegnù
 
JavaOne Conference, 2008
JavaOne Conference, 2008JavaOne Conference, 2008
JavaOne Conference, 2008
Sean Dawson
 
Disruptive Trends in Application Development
Disruptive Trends in Application DevelopmentDisruptive Trends in Application Development
Disruptive Trends in Application Development
WaveMaker, Inc.
 
170215 msa intro
170215 msa intro170215 msa intro
170215 msa intro
Sonic leigh
 
LINE's messaging service architecture underlying more than 200 million monthl...
LINE's messaging service architecture underlying more than 200 million monthl...LINE's messaging service architecture underlying more than 200 million monthl...
LINE's messaging service architecture underlying more than 200 million monthl...
kawamuray
 
DataEngConf SF16 - Scalable and Reliable Logging at Pinterest
DataEngConf SF16 - Scalable and Reliable Logging at PinterestDataEngConf SF16 - Scalable and Reliable Logging at Pinterest
DataEngConf SF16 - Scalable and Reliable Logging at Pinterest
Hakka Labs
 
Scalable and Reliable Logging at Pinterest
Scalable and Reliable Logging at PinterestScalable and Reliable Logging at Pinterest
Scalable and Reliable Logging at Pinterest
Krishna Gade
 
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
LINE Corporation
 
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
LINE Corporation
 
Building a [micro]services platform on AWS
Building a [micro]services platform on AWSBuilding a [micro]services platform on AWS
Building a [micro]services platform on AWS
Shaun Pearce
 
Building Modern Digital Services on Scalable Private Government Infrastructur...
Building Modern Digital Services on Scalable Private Government Infrastructur...Building Modern Digital Services on Scalable Private Government Infrastructur...
Building Modern Digital Services on Scalable Private Government Infrastructur...
Andrés Colón Pérez
 
O'Reilly Velocity Conference 2008
O'Reilly Velocity Conference 2008O'Reilly Velocity Conference 2008
O'Reilly Velocity Conference 2008
Sean Dawson
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
MahmoudZidan41
 
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suroDevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
Gaurav "GP" Pal
 
Evolution of the IBM Cloud Console: From Monolith to Microservices and Beyond
Evolution of the IBM Cloud Console: From Monolith to Microservices and BeyondEvolution of the IBM Cloud Console: From Monolith to Microservices and Beyond
Evolution of the IBM Cloud Console: From Monolith to Microservices and Beyond
Tony Erwin
 
LinkedIn - A highly scalable Architecture on Java!
LinkedIn - A highly scalable Architecture on Java!LinkedIn - A highly scalable Architecture on Java!
LinkedIn - A highly scalable Architecture on Java!
manivannan57
 

Similar to Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered (20)

Linkedin NUS QCon 2009 slides
Linkedin NUS QCon 2009 slidesLinkedin NUS QCon 2009 slides
Linkedin NUS QCon 2009 slides
 
InfoQ QCon San Francisco 2009
InfoQ QCon San Francisco 2009InfoQ QCon San Francisco 2009
InfoQ QCon San Francisco 2009
 
Do I Need A Service Mesh.pptx
Do I Need A Service Mesh.pptxDo I Need A Service Mesh.pptx
Do I Need A Service Mesh.pptx
 
Do You Need A Service Mesh?
Do You Need A Service Mesh?Do You Need A Service Mesh?
Do You Need A Service Mesh?
 
.NET microservices with Azure Service Fabric
.NET microservices with Azure Service Fabric.NET microservices with Azure Service Fabric
.NET microservices with Azure Service Fabric
 
JavaOne Conference, 2008
JavaOne Conference, 2008JavaOne Conference, 2008
JavaOne Conference, 2008
 
Disruptive Trends in Application Development
Disruptive Trends in Application DevelopmentDisruptive Trends in Application Development
Disruptive Trends in Application Development
 
170215 msa intro
170215 msa intro170215 msa intro
170215 msa intro
 
LINE's messaging service architecture underlying more than 200 million monthl...
LINE's messaging service architecture underlying more than 200 million monthl...LINE's messaging service architecture underlying more than 200 million monthl...
LINE's messaging service architecture underlying more than 200 million monthl...
 
DataEngConf SF16 - Scalable and Reliable Logging at Pinterest
DataEngConf SF16 - Scalable and Reliable Logging at PinterestDataEngConf SF16 - Scalable and Reliable Logging at Pinterest
DataEngConf SF16 - Scalable and Reliable Logging at Pinterest
 
Scalable and Reliable Logging at Pinterest
Scalable and Reliable Logging at PinterestScalable and Reliable Logging at Pinterest
Scalable and Reliable Logging at Pinterest
 
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
 
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
Building a company-wide data pipeline on Apache Kafka - engineering for 150 b...
 
Building a [micro]services platform on AWS
Building a [micro]services platform on AWSBuilding a [micro]services platform on AWS
Building a [micro]services platform on AWS
 
Building Modern Digital Services on Scalable Private Government Infrastructur...
Building Modern Digital Services on Scalable Private Government Infrastructur...Building Modern Digital Services on Scalable Private Government Infrastructur...
Building Modern Digital Services on Scalable Private Government Infrastructur...
 
O'Reilly Velocity Conference 2008
O'Reilly Velocity Conference 2008O'Reilly Velocity Conference 2008
O'Reilly Velocity Conference 2008
 
Introduction to Microservices
Introduction to MicroservicesIntroduction to Microservices
Introduction to Microservices
 
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suroDevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
DevOps in the Amazon Cloud – Learn from the pioneersNetflix suro
 
Evolution of the IBM Cloud Console: From Monolith to Microservices and Beyond
Evolution of the IBM Cloud Console: From Monolith to Microservices and BeyondEvolution of the IBM Cloud Console: From Monolith to Microservices and Beyond
Evolution of the IBM Cloud Console: From Monolith to Microservices and Beyond
 
LinkedIn - A highly scalable Architecture on Java!
LinkedIn - A highly scalable Architecture on Java!LinkedIn - A highly scalable Architecture on Java!
LinkedIn - A highly scalable Architecture on Java!
 

More from LinkedIn

How LinkedIn is Transforming Businesses
How LinkedIn is Transforming BusinessesHow LinkedIn is Transforming Businesses
How LinkedIn is Transforming Businesses
LinkedIn
 
Networking on LinkedIn 101
Networking on LinkedIn 101Networking on LinkedIn 101
Networking on LinkedIn 101
LinkedIn
 
5 تحديثات على ملفك في 5 دقائق
5 تحديثات على ملفك في 5 دقائق5 تحديثات على ملفك في 5 دقائق
5 تحديثات على ملفك في 5 دقائق
LinkedIn
 
5 LinkedIn Profile Updates in 5 Minutes
5 LinkedIn Profile Updates in 5 Minutes5 LinkedIn Profile Updates in 5 Minutes
5 LinkedIn Profile Updates in 5 Minutes
LinkedIn
 
The Student's Guide to LinkedIn
The Student's Guide to LinkedInThe Student's Guide to LinkedIn
The Student's Guide to LinkedIn
LinkedIn
 
The Top Skills That Can Get You Hired in 2017
The Top Skills That Can Get You Hired in 2017The Top Skills That Can Get You Hired in 2017
The Top Skills That Can Get You Hired in 2017
LinkedIn
 
Accelerating LinkedIn’s Vision Through Innovation
Accelerating LinkedIn’s Vision Through InnovationAccelerating LinkedIn’s Vision Through Innovation
Accelerating LinkedIn’s Vision Through Innovation
LinkedIn
 
How To Tell Your #workstory
How To Tell Your #workstoryHow To Tell Your #workstory
How To Tell Your #workstory
LinkedIn
 
LinkedIn Q1 2016 Earnings Call
LinkedIn Q1 2016 Earnings CallLinkedIn Q1 2016 Earnings Call
LinkedIn Q1 2016 Earnings Call
LinkedIn
 
The 2016 LinkedIn Job Search Guide
The 2016 LinkedIn Job Search GuideThe 2016 LinkedIn Job Search Guide
The 2016 LinkedIn Job Search Guide
LinkedIn
 
LinkedIn Q4 2015 Earnings Call
LinkedIn Q4 2015 Earnings CallLinkedIn Q4 2015 Earnings Call
LinkedIn Q4 2015 Earnings Call
LinkedIn
 
Banish The Buzzwords
Banish The BuzzwordsBanish The Buzzwords
Banish The Buzzwords
LinkedIn
 
LinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career Advice
LinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career AdviceLinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career Advice
LinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career Advice
LinkedIn
 
LinkedIn Q3 2015 Earnings Call
LinkedIn Q3 2015 Earnings CallLinkedIn Q3 2015 Earnings Call
LinkedIn Q3 2015 Earnings Call
LinkedIn
 
LinkedIn Economic Graph Research: Toronto
LinkedIn Economic Graph Research: TorontoLinkedIn Economic Graph Research: Toronto
LinkedIn Economic Graph Research: Toronto
LinkedIn
 
Freelancers Are LinkedIn Power Users [Infographic]
Freelancers Are LinkedIn Power Users [Infographic]Freelancers Are LinkedIn Power Users [Infographic]
Freelancers Are LinkedIn Power Users [Infographic]
LinkedIn
 
Top Industries for Freelancers on LinkedIn [Infographic]
Top Industries for Freelancers on LinkedIn [Infographic]Top Industries for Freelancers on LinkedIn [Infographic]
Top Industries for Freelancers on LinkedIn [Infographic]
LinkedIn
 
LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...
LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...
LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...
LinkedIn
 
LinkedIn Connect to Opportunity™ -- Stories of Discovery
LinkedIn Connect to Opportunity™ -- Stories of DiscoveryLinkedIn Connect to Opportunity™ -- Stories of Discovery
LinkedIn Connect to Opportunity™ -- Stories of Discovery
LinkedIn
 
LinkedIn Q2 2015 Earnings Call
LinkedIn Q2 2015 Earnings CallLinkedIn Q2 2015 Earnings Call
LinkedIn Q2 2015 Earnings Call
LinkedIn
 

More from LinkedIn (20)

How LinkedIn is Transforming Businesses
How LinkedIn is Transforming BusinessesHow LinkedIn is Transforming Businesses
How LinkedIn is Transforming Businesses
 
Networking on LinkedIn 101
Networking on LinkedIn 101Networking on LinkedIn 101
Networking on LinkedIn 101
 
5 تحديثات على ملفك في 5 دقائق
5 تحديثات على ملفك في 5 دقائق5 تحديثات على ملفك في 5 دقائق
5 تحديثات على ملفك في 5 دقائق
 
5 LinkedIn Profile Updates in 5 Minutes
5 LinkedIn Profile Updates in 5 Minutes5 LinkedIn Profile Updates in 5 Minutes
5 LinkedIn Profile Updates in 5 Minutes
 
The Student's Guide to LinkedIn
The Student's Guide to LinkedInThe Student's Guide to LinkedIn
The Student's Guide to LinkedIn
 
The Top Skills That Can Get You Hired in 2017
The Top Skills That Can Get You Hired in 2017The Top Skills That Can Get You Hired in 2017
The Top Skills That Can Get You Hired in 2017
 
Accelerating LinkedIn’s Vision Through Innovation
Accelerating LinkedIn’s Vision Through InnovationAccelerating LinkedIn’s Vision Through Innovation
Accelerating LinkedIn’s Vision Through Innovation
 
How To Tell Your #workstory
How To Tell Your #workstoryHow To Tell Your #workstory
How To Tell Your #workstory
 
LinkedIn Q1 2016 Earnings Call
LinkedIn Q1 2016 Earnings CallLinkedIn Q1 2016 Earnings Call
LinkedIn Q1 2016 Earnings Call
 
The 2016 LinkedIn Job Search Guide
The 2016 LinkedIn Job Search GuideThe 2016 LinkedIn Job Search Guide
The 2016 LinkedIn Job Search Guide
 
LinkedIn Q4 2015 Earnings Call
LinkedIn Q4 2015 Earnings CallLinkedIn Q4 2015 Earnings Call
LinkedIn Q4 2015 Earnings Call
 
Banish The Buzzwords
Banish The BuzzwordsBanish The Buzzwords
Banish The Buzzwords
 
LinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career Advice
LinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career AdviceLinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career Advice
LinkedIn Bring In Your Parents Day 2015 - Your Parents' Best Career Advice
 
LinkedIn Q3 2015 Earnings Call
LinkedIn Q3 2015 Earnings CallLinkedIn Q3 2015 Earnings Call
LinkedIn Q3 2015 Earnings Call
 
LinkedIn Economic Graph Research: Toronto
LinkedIn Economic Graph Research: TorontoLinkedIn Economic Graph Research: Toronto
LinkedIn Economic Graph Research: Toronto
 
Freelancers Are LinkedIn Power Users [Infographic]
Freelancers Are LinkedIn Power Users [Infographic]Freelancers Are LinkedIn Power Users [Infographic]
Freelancers Are LinkedIn Power Users [Infographic]
 
Top Industries for Freelancers on LinkedIn [Infographic]
Top Industries for Freelancers on LinkedIn [Infographic]Top Industries for Freelancers on LinkedIn [Infographic]
Top Industries for Freelancers on LinkedIn [Infographic]
 
LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...
LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...
LinkedIn Quiz: Which Parent Are You When It Comes to Helping Guide Your Child...
 
LinkedIn Connect to Opportunity™ -- Stories of Discovery
LinkedIn Connect to Opportunity™ -- Stories of DiscoveryLinkedIn Connect to Opportunity™ -- Stories of Discovery
LinkedIn Connect to Opportunity™ -- Stories of Discovery
 
LinkedIn Q2 2015 Earnings Call
LinkedIn Q2 2015 Earnings CallLinkedIn Q2 2015 Earnings Call
LinkedIn Q2 2015 Earnings Call
 

Recently uploaded

The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
operationspcvita
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Fwdays
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 

Recently uploaded (20)

The Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptxThe Microsoft 365 Migration Tutorial For Beginner.pptx
The Microsoft 365 Migration Tutorial For Beginner.pptx
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 

Ruslan Belkin And Sean Dawson on LinkedIn's Network Updates Uncovered

  • 1. LinkedIn: Network Updates Uncovered Ruslan Belkin Sean Dawson
  • 2. Agenda •  Quick Tour •  Requirements (User Experience / Infrastructure) •  Service API •  Internal Architecture •  Applications (e.g., Twitter Integration, Email Delivery) •  Measuring Performance •  Shameless self promotion
  • 3.
  • 4. The Stack Environment 90% Java 5% Groovy 2% Scala 2% Ruby 1% C++ Containers Tomcat, Jetty Data Layer Oracle, MySQL, Voldemort, Lucene, Memcache Offline Processing Hadoop Queuing ActiveMQ Frameworks Spring
  • 5. The Numbers Updates Created 35M / week Update Emails 14M / week Service Calls 20M / day 230 / second
  • 6.
  • 12. Email NUS Email digest screenshot
  • 14. Expectations – User Experience •  Multiple presentation views •  Comments on updates •  Aggregation of noisy updates •  Partner Integration •  Easy to add new updates to the system •  Handles I18N and other dynamic contexts •  Long data retention
  • 15. Expectations - Infrastructure •  Large number of connections, followers and groups •  High request volume + Low Latency •  Random distribution lists •  Black/White lists, A/B testing, etc. •  Tenured storage of update history •  Tracking of click through rates, impressions •  Supports real-time, aggregated data/statistics •  Cost-effective to operate
  • 16. Historical Note (homepage circa 2007) •  Legacy “network update” feature was a mixed bag of detached services. •  Neither consistent nor scalable •  Tightly coupled to our Inbox •  Migration plan •  Introduce API, unify all disparate service calls •  Add event-driven activity tracking with DB backend •  Build out the product •  Optimize!
  • 17. Network Updates Service – Overview
  • 18. Service API – Data Model <updates> <NCON> <connection> <id>2</id> <firstName>Chris</firstName> <lastName>Yee</lastName> </connection> </NCON> </updates>
  • 19. Service API – Post NetworkUpdatesNotificationService service = getNetworkUpdatesNotificationService(); ProfileUpdateInfo profileUpdate = createProfileUpdate(); Set<NetworkUpdateDestination> destinations = Sets.newHashSet( NetworkUpdateDestinations.newMemberFeedDestination(1213) ); NetworkUpdateSource source = new NetworkUpdateMemberSource(1214); Date updateDate = getClock().currentDate(); service.submitNetworkUpdate(source, destinations, updateDate, profileUpdate);
  • 20. Service API – Retrieve NetworkUpdatesService service = getNetworkUpdatesService(); NetworkUpdateChannel channel = NetworkUpdateChannels.newMemberChannel(1213); UpdateQueryCriteria query = createDefaultQuery(). setRequestedTypes(NetworkUpdateType.PROFILE_UPDATE). setMaxNumberOfUpdates(5). setCutoffDate(ClockUtils.add(currentDate, -7)); NetworkUpdateContext context = NetworkUpdateContextImpl.createWebappContext(); NetworkUpdatesSummaryResult result = service.getNetworkUpdatesSummary(channel, query, context);
  • 21. System at a glance
  • 22. Data Collection – Challenges •  How do we efficiently support collection in a dense social network •  Requirement to retrieve the feed fast •  But – there a lot of events from a lot of members and sources •  And – there are multiplier effects
  • 23. Option 1: Push Architecture (Inbox) •  Each member has an inbox of notifications received from their connections/followees •  N writes per update (where N may be very large) •  Very fast to read •  Difficult to scale, but useful for private or targeted notifications to individual users
  • 24. Option 1: Push Architecture (Inbox)
  • 25. Option 2: Pull Architecture •  Each member has an “Activity Space” that contains their actions on LinkedIn •  1 write per update (no broadcast) •  Requires up to N reads to collect N streams •  Can we optimize to minimize the number of reads? -  Not all N members have updates to satisfy the query -  Not all updates can/need to be displayed on the screen -  Some members are more important than others -  Some updates are more important than others -  Recent updates generally are more important than older ones
  • 26. Pull Architecture – Writing Updates
  • 27. Pull Architecture – Reading Updates
  • 28. Storage Model •  L1: Temporal •  Oracle •  Combined CLOB / varchar storage •  Optimistic locking •  1 read to update, 1 write (merge) to update •  Size bound by # number of updates and retention policy •  L2: Tenured •  Accessed less frequently •  Simple key-value storage is sufficient (each update has a unique ID) •  Oracle/Voldemort
  • 29. Member Filtering •  Need to avoid fetching N feeds (too expensive) •  Filter contains an in-memory summary of user activity •  Needs to be concise but representative •  Partitioned by member across a number of machines •  Filter only returns false-positives, never false-negatives •  Easy to measure heuristic; for the N members that I selected, how many of those members actually had good content •  Tradeoff between size of summary and filtering power
  • 31. Commenting •  Users can create discussions around updates •  Discussion lives in our forum service •  Denormalize a discussion summary onto the tenured update, resolve first/last comments on retrieval •  Full discussion can be retrieved dynamically
  • 32. Twitter Sync •  Partnership with Twitter •  Bi-directional flow of status updates •  Export status updates, import tweets •  Users register their twitter account •  Authorize via OAuth
  • 33. Twitter Sync – Overview
  • 34. Email Delivery •  Multiple concurrent email generating tasks •  Each task has non-overlapping ID range generators to avoid overlap and allow parallelization •  Controlled by task scheduler •  Sets delivery time •  Controls task execution status, suspend/resume, etc •  Caches common content so it is not re-requested •  Tasks deliver content to Notifier, which packages the content into an email via JSP engine •  Email is then delivered to SMTP relays
  • 37. What else? Brute force methods for scaling: •  Shard databases •  Memcache everything •  Parallelize everything •  User-initiated write operations are asynchronous when possible
  • 38. Know your numbers •  Bottlenecks are often not where you think they are •  Profile often •  Measure actual performance regularly •  Monitor your systems •  Pay attention to response time vs transaction rate •  Expect failures
  • 40. Another way of measuring performance
  • 41. LinkedIn is a great place to work
  • 42. Questions? Ruslan Belkin (http://www.linkedin.com/in/rbelkin) Sean Dawson (http://www.linkedin.com/in/seandawson)