The document discusses how AI is used at scale to create professional opportunities. It provides an overview of how AI powers the user and customer experience on LinkedIn through search, recommendations, staying informed, and getting hired. It describes how AI uses profile and network data to improve recommendations through understanding member characteristics and connections. The document also discusses how LinkedIn's recommendation system works, including using a generalized additive mixed-effect model called GLMix for large-scale regression to provide personalized job recommendations.
This is part 1 of the tutorial Xavier and Deepak gave at Recsys 2016 this year. You can find the second part http://www.slideshare.net/xamat/recsys-2016-tutorial-lessons-learned-from-building-reallife-recommender-systems
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
Keynote for the ACM Intelligent User Interface conference in 2016 in Sonoma, CA. I start with the past by talking about the Recommender Problem, and the Netflix Prize. Then I go into the Present and the Future by talking about approaches that go beyond rating prediction and ranking and by finishing with some of the most important lessons learned over the years. Throughout my talk I put special emphasis on the relation between algorithms and the User Interface.
This is part 1 of the tutorial Xavier and Deepak gave at Recsys 2016 this year. You can find the second part http://www.slideshare.net/xamat/recsys-2016-tutorial-lessons-learned-from-building-reallife-recommender-systems
Past, present, and future of Recommender Systems: an industry perspectiveXavier Amatriain
Keynote for the ACM Intelligent User Interface conference in 2016 in Sonoma, CA. I start with the past by talking about the Recommender Problem, and the Netflix Prize. Then I go into the Present and the Future by talking about approaches that go beyond rating prediction and ranking and by finishing with some of the most important lessons learned over the years. Throughout my talk I put special emphasis on the relation between algorithms and the User Interface.
Best Practices in Recommender System ChallengesAlan Said
Recommender System Challenges such as the Netflix Prize, KDD Cup, etc. have contributed vastly to the development and adoptability of recommender systems. Each year a number of challenges or contests are organized covering different aspects of recommendation. In this tutorial and panel, we present some of the factors involved in successfully organizing a challenge, whether for reasons purely related to research, industrial challenges, or to widen the scope of recommender systems applications.
This tutorial gives an overview of how search engines and machine learning techniques can be tightly coupled to address the need for building scalable recommender or other prediction based systems. Typically, most of them architect retrieval and prediction in two phases. In Phase I, a search engine returns the top-k results based on constraints expressed as a query. In Phase II, the top-k results are re-ranked in another system according to an optimization function that uses a supervised trained model. However this approach presents several issues, such as the possibility of returning sub-optimal results due to the top-k limits during query, as well as the prescence of some inefficiencies in the system due to the decoupling of retrieval and ranking.
To address this issue the authors created ML-Scoring, an open source framework that tightly integrates machine learning models into Elasticsearch, a popular search engine. ML-Scoring replaces the default information retrieval ranking function with a custom supervised model that is trained through Spark, Weka, or R that is loaded as a plugin in Elasticsearch. This tutorial will not only review basic methods in information retrieval and machine learning, but it will also walk through practical examples from loading a dataset into Elasticsearch to training a model in Spark, Weka, or R, to creating the ML-Scoring plugin for Elasticsearch. No prior experience is required in any system listed (Elasticsearch, Spark, Weka, R), though some programming experience is recommended.
Tutorial on People Recommendations in Social Networks - ACM RecSys 2013,Hong...Anmol Bhasin
Tutorials at ACM RecSys 2013
Social Networks
Learning to Rank
Beyond Friendship
Pref. Handling
Beyond Friendship: The Art, Science and Applications of Recommending People to People in Social Networks
by Luiz Augusto Pizzato (University of Sydney, Australia)
& Anmol Bhasin (LinkedIn, USA)
While Recommender Systems are powerful drivers of engagement and transactional utility in social networks, People recommenders are a fairly involved and diverse subdomain. Consider that movies are recommended to be watched, news is recommended to be read, people however, are recommended for a plethora of reasons – such as recommendation of people to befriend, follow, partner, targets for an advertisement or service, recruiting, partnering romantically and to join thematic interest groups.
This tutorial aims to first describe the problem domain, touch upon classical approaches like link analysis and collaborative filtering and then take a rapid deep dive into the unique aspects of this problem space like Reciprocity, Intent understanding of recommender and the recomendee, Contextual people recommendations in communication flows and Social Referrals – a paradigm for delivery of recommendations using the Social Graph. These aspects will be discussed in the context of published original work developed by the authors and their collaborators and in many cases deployed in massive-scale real world applications on professional networks such as LinkedIn.
Introduction
The basics of Social Recommenders
People recommender systems
Special Topics in People Recommenders
Why reciprocal (people) recommenders are different to traditional (product) recommendations
Multi-Objective Optimization
Intent Understanding
Feature Engineering
Social Referral
Pathfinding
Concluding remarks
The pre-requisite for this tutorial is some familiarity with foundational Recommender Systems, Data Mining, Machine Learning and Social Network Analysis literature.
Date
Oct 13, 2013 (08:30 – 10:15)
Product Recommendations Enhanced with Reviewsmaranlar
Tutorial presented by Muthusamy Chelliah (Flipkart, India) and Sudeshna Sarkar (IIT Kharagpur, India) at ACM RecSys 2017 https://recsys.acm.org/recsys17/tutorials/#content-tab-1-3-tab
E-commerce websites commonly deploy recommender systems that make use of user activity (e.g., ratings, views, and purchases) or content (product descriptions). These recommender systems can benefit enormously by also exploiting the information contained in customer reviews. Reviews capture the experience of multiple customers with diverse preferences, often on the fine-grained level of specific features of products. Reviews can also identify consumers’ preferences for product features and provide helpful explanations. The usefulness of reviews is evidenced by the prevalence of their use by customers to support shopping decisions online. With the appropriate techniques, recommender systems can benefit directly from user reviews.
This tutorial will present a range of techniques that allow recommender systems in e-commerce websites to take full advantage of reviews. Topics covered include text mining methods for feature-specific sentiment analysis of products, topic models and distributed representations that bridge the vocabulary gap between user reviews and product descriptions, and recommender algorithms that use review information to address the cold-start problem.
The tutorial sessions will be interspersed with examples from an online marketplace (i.e., Flipkart) and experience with using data mining and Natural Language Processing techniques (e.g., matrix factorization, LDA, word embeddings) from Web-scale systems.
Best Practices in Recommender System ChallengesAlan Said
Recommender System Challenges such as the Netflix Prize, KDD Cup, etc. have contributed vastly to the development and adoptability of recommender systems. Each year a number of challenges or contests are organized covering different aspects of recommendation. In this tutorial and panel, we present some of the factors involved in successfully organizing a challenge, whether for reasons purely related to research, industrial challenges, or to widen the scope of recommender systems applications.
This tutorial gives an overview of how search engines and machine learning techniques can be tightly coupled to address the need for building scalable recommender or other prediction based systems. Typically, most of them architect retrieval and prediction in two phases. In Phase I, a search engine returns the top-k results based on constraints expressed as a query. In Phase II, the top-k results are re-ranked in another system according to an optimization function that uses a supervised trained model. However this approach presents several issues, such as the possibility of returning sub-optimal results due to the top-k limits during query, as well as the prescence of some inefficiencies in the system due to the decoupling of retrieval and ranking.
To address this issue the authors created ML-Scoring, an open source framework that tightly integrates machine learning models into Elasticsearch, a popular search engine. ML-Scoring replaces the default information retrieval ranking function with a custom supervised model that is trained through Spark, Weka, or R that is loaded as a plugin in Elasticsearch. This tutorial will not only review basic methods in information retrieval and machine learning, but it will also walk through practical examples from loading a dataset into Elasticsearch to training a model in Spark, Weka, or R, to creating the ML-Scoring plugin for Elasticsearch. No prior experience is required in any system listed (Elasticsearch, Spark, Weka, R), though some programming experience is recommended.
Tutorial on People Recommendations in Social Networks - ACM RecSys 2013,Hong...Anmol Bhasin
Tutorials at ACM RecSys 2013
Social Networks
Learning to Rank
Beyond Friendship
Pref. Handling
Beyond Friendship: The Art, Science and Applications of Recommending People to People in Social Networks
by Luiz Augusto Pizzato (University of Sydney, Australia)
& Anmol Bhasin (LinkedIn, USA)
While Recommender Systems are powerful drivers of engagement and transactional utility in social networks, People recommenders are a fairly involved and diverse subdomain. Consider that movies are recommended to be watched, news is recommended to be read, people however, are recommended for a plethora of reasons – such as recommendation of people to befriend, follow, partner, targets for an advertisement or service, recruiting, partnering romantically and to join thematic interest groups.
This tutorial aims to first describe the problem domain, touch upon classical approaches like link analysis and collaborative filtering and then take a rapid deep dive into the unique aspects of this problem space like Reciprocity, Intent understanding of recommender and the recomendee, Contextual people recommendations in communication flows and Social Referrals – a paradigm for delivery of recommendations using the Social Graph. These aspects will be discussed in the context of published original work developed by the authors and their collaborators and in many cases deployed in massive-scale real world applications on professional networks such as LinkedIn.
Introduction
The basics of Social Recommenders
People recommender systems
Special Topics in People Recommenders
Why reciprocal (people) recommenders are different to traditional (product) recommendations
Multi-Objective Optimization
Intent Understanding
Feature Engineering
Social Referral
Pathfinding
Concluding remarks
The pre-requisite for this tutorial is some familiarity with foundational Recommender Systems, Data Mining, Machine Learning and Social Network Analysis literature.
Date
Oct 13, 2013 (08:30 – 10:15)
Product Recommendations Enhanced with Reviewsmaranlar
Tutorial presented by Muthusamy Chelliah (Flipkart, India) and Sudeshna Sarkar (IIT Kharagpur, India) at ACM RecSys 2017 https://recsys.acm.org/recsys17/tutorials/#content-tab-1-3-tab
E-commerce websites commonly deploy recommender systems that make use of user activity (e.g., ratings, views, and purchases) or content (product descriptions). These recommender systems can benefit enormously by also exploiting the information contained in customer reviews. Reviews capture the experience of multiple customers with diverse preferences, often on the fine-grained level of specific features of products. Reviews can also identify consumers’ preferences for product features and provide helpful explanations. The usefulness of reviews is evidenced by the prevalence of their use by customers to support shopping decisions online. With the appropriate techniques, recommender systems can benefit directly from user reviews.
This tutorial will present a range of techniques that allow recommender systems in e-commerce websites to take full advantage of reviews. Topics covered include text mining methods for feature-specific sentiment analysis of products, topic models and distributed representations that bridge the vocabulary gap between user reviews and product descriptions, and recommender algorithms that use review information to address the cold-start problem.
The tutorial sessions will be interspersed with examples from an online marketplace (i.e., Flipkart) and experience with using data mining and Natural Language Processing techniques (e.g., matrix factorization, LDA, word embeddings) from Web-scale systems.
Hadoop World 2011: LeveragIng Hadoop to Transform Raw Data to Rich Features a...Cloudera, Inc.
This presentation focuses on the design and evolution of the LinkedIn recommendations platform. It currently computes more than 100 billion personalized recommendations every week, powering an ever growing assortment of products, including Jobs You May be Interested In, Groups You May Like, News Relevance, and Ad Targeting. We will describe how we leverage Hadoop to transform raw data to rich features using knowledge aggregated from LinkedIn's 100 million member base, how we use Lucene to do real-time recommendations, and how we marshal Lucene on Hadoop to bridge offline analysis with user-facing services.
We hear a lot today about “big data” and companies looking to establish data-driven recruiting in their HR organizations. LinkedIn Talent Pool Reports are a big step to accomplishing exactly that, by providing you meaningful, objective information to inform your talent acquisition strategy and allow you to engage your stakeholders. Bottom line, these reports take a lot of the guesswork out of recruiting, and give you an in-depth look at where to recruit and what candidates are looking for.
Join us for this free LinkedIn webcast on how to use Talent Pools to power your talent strategy. During this session we are going to cover:
Why build talent pools using data
Insights about talent pools across LinkedIn
Live demonstration of LinkedIn Talent Pool reports
LinkedIn Member Segmentation Platform: A Big Data ApplicationDataWorks Summit
Creating member segmentations is one of the main functions of a marketing team at any Internet company. Marketing teams are constantly creating various member segments to tailor to the needs of marketing campaigns and these needs change frequently. Therefore there is a huge need for a self-service member segmentation platform that is easy to use and scalable to support large member data set. This presentation will go into the architecture of the LinkedIn Member Segmentation platform and how it leverages Hadoop technologies like Apache Pig, Apache Hive and enterprise data warehouse system like Teradata to provide a self-service way to create and manage member segmentations. In addition, it will also cover some of the interesting challenges and lessons learned from building this platform.
The slides go through the implementation details of Google Deepmind's AlphaGo, a computer Go AI that defeated the European champion. The slides are targeted for beginners in the machine learning area.
Korean version (한국어 버젼): http://www.slideshare.net/ShaneSeungwhanMoon/ss-59226902
Large scale social recommender systems and their evaluationMitul Tiwari
This talk will give an overview of some of the large-scale recommender systems at LinkedIn such as People You May Know (PYMK) and Suggested Skills Endorsements. This talk will also address how we formulate machine learning modeling problems to build these recommender systems and evaluate our models. Modeling for these recommender systems involves careful feature engineering and incorporating user feedback - both explicit and implicit. This talk will describe how we feature engineer through an example of modeling organizational overlap between people for link prediction and community detection over social graph. Also, how we incorporate user feedback through impression discounting ignored recommended results will be described. Careful evaluation of modeling changes both offline and online (A/B testing) is inherent part of measuring effectiveness of our recommender systems. We have built a sophisticated end-to-end A/B testing and evaluation platform called XLNT at LinkedIn and this talk will also cover how we use XLNT for power analysis, A/B testing, and measuring confidence of the results.
Data-driven Approach to Launching your CareerViral Kadakia
500 Miles is the leading mobile platform for talented candidates to discover, evaluate and engage with high-growth tech employers. This talk was specially designed for Northwestern University students
الموعد الإثنين 03 يناير 2022
143
مبادرة
#تواصل_تطوير
المحاضرة ال 143 من المبادرة
المهندس / محمد الرافعي طرباي
نقيب المبرمجين بالدقهلية
بعنوان
"IT INDUSTRY"
How To Getting Into IT With Zero Experience
وذلك يوم الإثنين 03 يناير2022
السابعة مساء توقيت القاهرة
الثامنة مساء توقيت مكة المكرمة
و الحضور من تطبيق زووم
https://us02web.zoom.us/meeting/register/tZUpf-GsrD4jH9N9AxO39J013c1D4bqJNTcu
علما ان هناك بث مباشر للمحاضرة على القنوات الخاصة بجمعية المهندسين المصريين
ونأمل أن نوفق في تقديم ما ينفع المهندس ومهمة الهندسة في عالمنا العربي
والله الموفق
للتواصل مع إدارة المبادرة عبر قناة التليجرام
https://t.me/EEAKSA
ومتابعة المبادرة والبث المباشر عبر نوافذنا المختلفة
رابط اللينكدان والمكتبة الالكترونية
https://www.linkedin.com/company/eeaksa-egyptian-engineers-association/
رابط قناة التويتر
https://twitter.com/eeaksa
رابط قناة الفيسبوك
https://www.facebook.com/EEAKSA
رابط قناة اليوتيوب
https://www.youtube.com/user/EEAchannal
رابط التسجيل العام للمحاضرات
https://forms.gle/vVmw7L187tiATRPw9
ملحوظة : توجد شهادات حضور مجانية لمن يسجل فى رابط التقيم اخر المحاضرة
Multimodal Learning to Rank in Production Scale E-commerce SearchLucidworks
ACTIVATE 2019 Keynote: Search is at the heart of modern day e-commerce. In this talk, Dr. Kamelia Aryafar, Chief Customer and Algorithms Officer at Overstock.com will share how deep learning and multimodal learning to rank methods can improve the relevancy of production scale e-commerce search engines. Kamelia will also share best practices on building successful AI teams and an experiment-driven culture, including how Overstock.com approaches search and machine learning and how she sees the field evolving.
This talk provides an overview of privacy-preserving analytics and data mining systems at LinkedIn, highlighting the practical challenges/requirements, techniques, and lessons learned from deployment. The first part presents a framework to compute robust, privacy-preserving analytics, while the second part focuses on the privacy challenges/design for a large crowdsourced system (LinkedIn Salary). This presentation is an expanded version of the talk given at the Differential Privacy Deployed workshop, co-organized by Cynthia Dwork and held at Harvard / American Academy of Sciences in September, 2018.
Having 6 Years of experience and to build A global career in information technology with readiness to accept challenge & to excel through continuous persistence & hard work for growth of organization with improvement in my own computer skills.
Elena Grewal, Data Science Manager, Airbnb at MLconf SF 2016MLconf
Before the Model: How Machine Learning Products Start, with Examples from Airbnb: Often the most important part of building a machine learning product is the formulation of the problem; the most elegant model is rendered useless without the right application and model architecture. Airbnb is an online marketplace for accommodations which has found many interesting applications for machine learning products by taking a data driven approach to investment in Machine learning products. Come hear about how the Airbnb team generates and vets ideas for machine learning products and tailors the product to business problems, with some examples of success and lessons learned along the way.
A deck adapted for an interview/portfolio setting where I walk through how a product manager/designer would build a new search tool for an HR Tech platform
Salesforce Architect Group, Frederick, United States July 2023 - Generative A...NadinaLisbon1
Joined our community-led event to dive into the world of Artificial Intelligence (AI)! Whether you were just starting your AI journey or already familiar with its concepts, one thing was certain: AI was reshaping the future of work. This enablement session was your chance to level up your skills and stay ahead in that rapidly evolving landscape.
As AI news continues to dominate headlines, it's natural to have questions and concerns about its impact on our lives. Will AI take over human jobs? Will it render us obsolete? Rest assured, the outlook is far brighter than you may think. Rather than replacing humans, AI is designed to enhance our capabilities and work alongside us. It won't be replacing marketers, service representatives, or salespeople—it will be empowering them to achieve even greater results. Companies across industries recognize this potential and are embracing AI to unlock new levels of performance.
During this enablement session, you'll have the opportunity to explore how AI advancements can positively influence your professional journey and daily life. We'll debunk common misconceptions, address fears, and showcase real-world examples of how successful AI implementation leads to workforce augmentation rather than replacement. Be prepared to gain valuable insights and practical knowledge that will help you navigate the AI landscape with confidence.
ER(Entity Relationship) Diagram for online shopping - TAEHimani415946
https://bit.ly/3KACoyV
The ER diagram for the project is the foundation for the building of the database of the project. The properties, datatypes, and attributes are defined by the ER diagram.
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
This 7-second Brain Wave Ritual Attracts Money To You.!nirahealhty
Discover the power of a simple 7-second brain wave ritual that can attract wealth and abundance into your life. By tapping into specific brain frequencies, this technique helps you manifest financial success effortlessly. Ready to transform your financial future? Try this powerful ritual and start attracting money today!
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
Talk presented at Kubernetes Community Day, New York, May 2024.
Technical summary of Multi-Cluster Kubernetes Networking architectures with focus on 4 key topics.
1) Key patterns for Multi-cluster architectures
2) Architectural comparison of several OSS/ CNCF projects to address these patterns
3) Evolution trends for the APIs of these projects
4) Some design recommendations & guidelines for adopting/ deploying these solutions.
4. Emily likes AI and deep learning. Got familiar with research on Dropout
through an article shared by one of her connection. Followed through,
implemented and got great recognition at work.
5. Impact of creating professional opportunities at scale
• Closing the skills gap by leveraging supply on a global basis, reducing income
disparity, training the workforce to prepare for the future.
JOBS QUEUE in INDIA, over supply
6. Almost all user experience powered by AI
SEARCH,CONNECT,
NURTURE
Search and connect, via
recommendations, keep
in touch
STAY INFORMED
through professional
news and knowledge
GET HIRED
and build
your career
7. All customer experience too
HIRE
Searching, researching,
nurturing qualified
candidates
MARKET
Targeting right audience via native ads,
display ads
SELL
Searching and
researching decision
makers, following up
on leads
8. How is AI used to drive value ?
• Two broad classes of problems
•Recommendation :
◆Recommend items to a users to optimize one of more
objectives of interest (e.g., connections, job applies,
engagement/revenue)
•Search:
◆User asks a question through query, need to provide answer
by searching through a large set of possibilities
10. Connecting long-term objectives to proxies that can be optimized by
machines/algorithms
Long-term objectives
(return visits to site, connections,
quality job applies,,..)
Short-term proxies (CTR,
connection prob, apply prob, …)
Large scale optimization
via ML..
Experiment
Learn
Innovate
11. Recommendation Problem: Not merely supervised learning
User i
with
user features xi
(profile info,
browse history,
geo-location,
search history, …)
visits
item j with item features xj
(keywords, content categories, author,
...)
Algorithm selects
(i, j) : response yij
Interaction (Click, share, like, apply, connect,..)/no-interaction
Which item should we select?
• The one with highest predicted utility
• The one most useful for improving
the utility prediction model
Exploit
Explore
14. AI used to standardize member profile data
text ID
● language detection
● spell correction
● segmentation
● entity recognition
● entity resolutionprofile IDs in taxonomy
text mining machine learning
15. An AI-based holistic solution for taxonomy development
human-curation to
create and structure
taxonomies
slow, costly,
not scalable
deep neural network
for entity embedding
Auto-generate
● new entities
● synonyms
● entity hierarchical structure etc.
fast, cheap,
scalable
16. Proactively helping members with profile completion
Estimate the rewards of adding a new skill through peer analysis
Infer the skill reputation of LinkedIn member
through user profiles (static), user behaviors
(dynamic) and user connections (network)
Auto-generate
profile
summary
through
knowledge
base
17. Better knowledge base powers better AI
Members (profile), Jobs, Companies
etc. collectively form the LinkedIn
professional knowledge base
direct knowledge base approach
based on logical inference rules
direct features
other training data like
network, user behaviors
graph embedding
better representation
Search
Job recommendation
Profile edit
Ads targeting
Feed
Course learning
Insights
Endorsement
…...
18. Network: Connections are fundamental to drive value
• A well connected member is much more likely to be able to use the
LinkedIn eco-system for her professional development and growth
18
20. Adding nodes: Viral Loops through recommendations
C
Landing Page
✔ ✔
Contacts Upload
Abc
abc@myschool.edu
XYZ,
lXYZ@myschool.edu
✔
21. Adding nodes: Viral loops from Guest Invitation
• Members inviting others to join the network is crucial for growth of
the network
• Anderson et. al. (WWW 2015)
• LinkedIn’s cascade trees are very viral
21
The growth of trees is
almost linear in time
23. User Intent
• Why are you here ?
•Hire, get hired, stay informed, grow network, nurture connections, sell,
market,..
•Explicit (e.g., visiting jobs homepage, search query),
•Implicit like job seeker (needs to be inferred, e.g., based on activities)
24. We know about users, their intent, about items
How to Recommend items to users algorithmically ?
Framework, Infrastructure and Tools
25. Under the Hood of a Typical Recommender System at
LinkedIn
25
27. Objective: Job Applications
Predict the probability that
a user i would apply for a job j
given …
•User features
•Profile: Industry, skills, job positions, companies, education
•Network: Connection patterns
•Item (i.e., job) features
•Source: Poster, company, location
•Content: Keywords, title, skills
•Data about users’ past interactions with different types of items
•Items: Jobs, articles, updates, courses, comments
•Interactions: Apply, click, like, share, connect, follow
28. Components
Front End
Service
Ranking
Service
Item
Index
User Feature
Stores User DB
Item DB
Offline Data Pipelines
Item Feature
Pipelines
User Feature
Pipelines
Data Stream
Processing
User Activity Data Streams
Live Index
Updater
ETL ETL
Online
Offline
Model Training
Pipelines
Offline Index
Builder
User
Photon-ML
Apache
Hadoop, Pig, Scalding,
SparkSearch Index
Experimentation
Platform A/B testing
Ranking
Library
29. Model Training
Raw User Features Raw Item Features
DAG of
Transformers
DAG of
Transformers
DAG of
Transformers
Feature Vector of User i
xi
Matching Feature Vector
mij
Feature Vector of Item j
zj
(trees, similarities)
Parameter
vector for
each user i
Parameter
vector for
each item j
p(i applies for j) = f( xi, zj, mij | θ, αi, βj )
Feature
Processing
Parameter Learning
Global
parameter
vector
31. Online Ranking
User Feature
Stores User DB
User Feature
Pipelines
Data Stream
Processing
User Activity Data Stream
Ranking
Service
Item
Index
Offline Data Pipelines
ETL ETL
Online
Offline
Model Training
Pipelines
Offline Index
Builder
Front End
Service
User
User
Features &
Parameters
Item DB
Live Index
Updater
Item Feature
Pipelines
32. Online A/B Experiments
Experiment setting
- Select a user segment
- Allocate traffic to different
models
Result reporting
- Report experimental results
- Impact on a large number of
metrics
33. How we do large scale Regression ?
GAME: Generalized Additive Mixed-Effect Model
35. GAME as a Framework
• Unifies and mixes different models into a principled
additive model.
• Predicting the response of user i on item j:
• : An effect (model)
• : Expectation of response
• : Link function
36. GAME as a Library
• Basic models implemented as building block .
• Matrix factorization
• Generalized Linear Model
• …
• New models can be directly composed by mixing existing
building blocks.
37. Ex: GLM + MF
• Predicting the response of user i on item j:
• : GLM
• : MF
• : Expectation of response
• : Link function
38. GLMix¹: Fine-Grained GAME with Linear Components
1: GLMix: Generalized Linear Mixed Models For Large-Scale Response Prediction
X. Zhang et al., KDD2016
39. • Jobs homepage
• Ramped to serve 100% traffic (400 million LinkedIn members)
• significant lift in job application rate
41
GLMix @
40. • Jobs homepage
• Ramped to serve 100% traffic (400 million LinkedIn members)
• significant lift in job application rate
• Advertising, PYMK, Feed, Recruiter,…
• significant lifts in performance
42
GLMix @
41. Generalized Linear Model (GLM)
• Predicting the response of user i on item j:
• : Feature vector (profile, items, intent, cross-prod, GBDT)
• : Coefficient vector
• : Expectation of response
• : Link function
42. ●Alice and Annie are about the same age, similar majors in college…
(similar member features )
●Alice likes to take more risks with start-ups
●Annie likes more stable career just like her parents
●GLM may return similar set of jobs to both
GLM for Job Recommendation
44
43. ●Alice and Annie are about the same age, similar majors in college…
(similar member features )
●Alice likes to take more risks with start-ups
●Annie likes more stable career just like her parents
●GLM may return similar set of jobs to both
●Need more fine-grained modeling at different granularity to better
personalize the model!
GLM for Job Recommendation
45
44. GLMix: Generalized Linear Mixed Model
• Predicting the response of user i on item j:
• Model coefficients with different granularities:
• Per-user random effect coefficients
• Per-item random effect coefficients
• GLMix = GLM + per-user model + per-item model
45. GLMix for Job Recommendation
● Global fixed effect model
○ Similarity between member profile and jobs profile, e.g. do the
member skills and job skills look similar?
46. GLMix for Job Recommendation
● Global fixed effect model
○ Similarity between member profile and jobs profile, e.g. do the
member skills and job skills look similar?
● Per-member random effect model
○ E.g. If a member has applied to a job with title = “software engineer”,
we will boost “software engineer” jobs more in her results.
47. GLMix for Job Recommendation
● Global fixed effect model
○ Similarity between member profile and jobs profile, e.g. do the
member skills and job skills look similar?
● Per-member random effect model
○ E.g. If a member has applied to a job with title = “software engineer”,
we will boost “software engineer” jobs more in her results.
● Per-job random effect model
○ E.g. If a job gets an apply with a member titled “software engineer”, we
will boost this job more for members with this title.
48. Alice and Annie’s problem revisited
● Per-user random effect coefficients for Alice:
● Per-user random effect coefficients for Annie:
● Alice and Annie now may have different job recommendations
given their per-user coefficients.
50. Takeaways
• GAME unifies and mixes different models into a principled
additive model.
• MF + GLM = RLFM/Matchbox
• GLM + DNN = Wide & Deep Learning
• ...
52
51. Takeaways
• GAME unifies and mixes different models into a principled
additive model.
• MF + GLM = RLFM
• GLM + DNN = Wide & Deep Learning
• ...
• GLMix is GAME with linear component that captures signal
from different granularity
• GLMix = GLM + Per-member model + per-item model + …
53
52. Takeaways
• GAME unifies and mixes different models into a principled
additive model.
• MF + GLM = RLFM
• GLM + DNN = Wide & Deep Learning
• ...
• GLMix is GAME with linear component that captures signal
from different granularity
• GLMix = GLM + Per-member model + per-item model + …
• GAME is part of an open-source library
• Search for Photon-ML
• https://github.com/linkedin/photon-ml
54
55. 1. Cost of a Bad Recommendation
• Making ML robust when a few bad recommendations can hurt product brand
• Maximize precision without hurting performance metrics significantly
• Collect negative feedback from users, crowd; incorporate within algorithms
• Create better product focus, filter unnecessary content from inventory
• E.g., unprofessional content on Feed
• Better insights/explanations associated with recommendations help build trust
56. 2. Data Tracking
• Proper data tracking and monitoring is not always easy!
• Data literacy and understanding across organization (front-end, UI, SRE)
• Proper tooling, continuous monitoring very important to scale the process
• Our philosophy: Loose coupling between FE and BE teams!
• FE (client) emits limited events along with trackingid
• BE emits more details and joins against trackingid
• Tracking events can have significant impact
• View-port tracking (what the user actually saw) for more informative negatives
57. 3. Content Inventory
• High quality and comprehensive content inventory as important as
recommendation algorithms
•Examples: Jobs, Feed
•Supply and demand analysis, gap analysis, proactively producing more high
quality content for creating appropriate inventory
58. 4. A/B Testing with Network Interference
• Random treatment assignments (spillover effects, need to adjust)
• Treatment recommendations affect control group as well
• A like/share in treatment may create a new item when ranking in control
60