This document discusses building natural language processing (NLP) systems at scale for large businesses. It provides examples of how NLP can be used across different business domains and customer experiences to improve outcomes. Specifically, it discusses:
1) Six motivating examples of NLP use cases like search, recommendations, question answering, conversation analysis, customer support, and item understanding.
2) How NLP can have a high return on investment when applied correctly across different types of businesses.
3) Different NLP serving scenarios like online, streaming, and batch processing that require different system architectures.
4) How building effective NLP systems requires not just models but tools for data annotation, model training/deployment, testing
Building multi billion ( dollars, users, documents ) search engines on open ...Andrei Lopatenko
How to use open source technologies to build search engines for billions of users, billions of revenue, billions of documents
Keynote talk at The 16th International Conference on Open Source Systems.
Andrei Lopatenko is a Vice President of Engineering at Zillow Group who has 15 years of experience designing, building, and improving AI-driven search engines. His talk will demonstrate how AI can be useful throughout every part of a search engine, from data acquisition to ranking to the user experience. Successful AI in search requires an infrastructure that allows for continuous introduction and improvement of AI applications across the entire search stack.
Deep learning for e-commerce: current status and future prospectsRakuten Group, Inc.
Deep learning is the prime avenue for Artificial Intelligence, with spectacular accomplishments in diverse fields such as computer vision, natural language processing, and board games such as Go. Its impact on e-commerce is already significant and will continue to grow in future years. In this talk, we will review some of the successful deep learning algorithms in light of their current and expected impact on e-commerce.
Search Product Manager: Software PM vs. Enterprise PM or What does that * PM do?John T. Kane
This document discusses the roles of search product managers and provides examples of search metrics and KPIs. It outlines the speaker's background and experience as a search PM, describes different types of search use cases, and compares roles of software vs. enterprise search PMs. It also lists references and thoughts on future directions for search and product management.
How Artificial Intelligence & Machine Learning Are Transforming Modern MarketingCleverTap
Join Almitra Karnik, Head of Marketing, CleverTap, and Jessie Paul, CEO, Paul Writer share their insights on how AI and ML are fundamentally changing the way we approach marketing and how we can harness these changes to further our businesses.
Haystack- Learning to rank in an hourly job market Xun Wang
The document discusses learning to rank models for job search rankings on an hourly job marketplace platform. It describes:
1) The complexity of matching job seekers to job postings given the many factors involved and limited historical data.
2) An iterative process of developing learning to rank models, testing improvements through A/B testing, and analyzing results to further tune the models over time.
3) Key factors considered in the models include job title/description matches, employer name, location matches, distance between seeker and job, and search/user attributes. Performance is evaluated on multiple metrics like application and conversion rates.
Improving Search in Workday Products using Natural Language ProcessingDataWorks Summit
Workday is a leading provider of cloud-based enterprise software products such as Human Capital Management, Talent, Finance, Student, Planning etc. These products produce a wealth of natural language data. However, this data is unstructured and denormalized. Retrieving relevant information from such data is a challenging task. Using simple index-based search methods can only take us so far. The Data Science team at Workday is determined to apply Machine Learning and AI to make search better across Workday’s products.
In this session, we present to you, how we use word embeddings to normalize the data and add structure to it. We will also talk about using word representations to make search intelligent. The specific use cases we will discuss are adding synonyms detection and entity-recommendation.
In this talk, we will focus on the word-embeddings techniques explored, metrics used to evaluate Natural Language Processing Models, tools built, and future work as a part of improving search.
Speaker
Namrata Ghadi, Workday Inc, Software Development Engineer (Data Science)
Adam Baker, Workday Inc, Sr Software Engineer
Building multi billion ( dollars, users, documents ) search engines on open ...Andrei Lopatenko
How to use open source technologies to build search engines for billions of users, billions of revenue, billions of documents
Keynote talk at The 16th International Conference on Open Source Systems.
Andrei Lopatenko is a Vice President of Engineering at Zillow Group who has 15 years of experience designing, building, and improving AI-driven search engines. His talk will demonstrate how AI can be useful throughout every part of a search engine, from data acquisition to ranking to the user experience. Successful AI in search requires an infrastructure that allows for continuous introduction and improvement of AI applications across the entire search stack.
Deep learning for e-commerce: current status and future prospectsRakuten Group, Inc.
Deep learning is the prime avenue for Artificial Intelligence, with spectacular accomplishments in diverse fields such as computer vision, natural language processing, and board games such as Go. Its impact on e-commerce is already significant and will continue to grow in future years. In this talk, we will review some of the successful deep learning algorithms in light of their current and expected impact on e-commerce.
Search Product Manager: Software PM vs. Enterprise PM or What does that * PM do?John T. Kane
This document discusses the roles of search product managers and provides examples of search metrics and KPIs. It outlines the speaker's background and experience as a search PM, describes different types of search use cases, and compares roles of software vs. enterprise search PMs. It also lists references and thoughts on future directions for search and product management.
How Artificial Intelligence & Machine Learning Are Transforming Modern MarketingCleverTap
Join Almitra Karnik, Head of Marketing, CleverTap, and Jessie Paul, CEO, Paul Writer share their insights on how AI and ML are fundamentally changing the way we approach marketing and how we can harness these changes to further our businesses.
Haystack- Learning to rank in an hourly job market Xun Wang
The document discusses learning to rank models for job search rankings on an hourly job marketplace platform. It describes:
1) The complexity of matching job seekers to job postings given the many factors involved and limited historical data.
2) An iterative process of developing learning to rank models, testing improvements through A/B testing, and analyzing results to further tune the models over time.
3) Key factors considered in the models include job title/description matches, employer name, location matches, distance between seeker and job, and search/user attributes. Performance is evaluated on multiple metrics like application and conversion rates.
Improving Search in Workday Products using Natural Language ProcessingDataWorks Summit
Workday is a leading provider of cloud-based enterprise software products such as Human Capital Management, Talent, Finance, Student, Planning etc. These products produce a wealth of natural language data. However, this data is unstructured and denormalized. Retrieving relevant information from such data is a challenging task. Using simple index-based search methods can only take us so far. The Data Science team at Workday is determined to apply Machine Learning and AI to make search better across Workday’s products.
In this session, we present to you, how we use word embeddings to normalize the data and add structure to it. We will also talk about using word representations to make search intelligent. The specific use cases we will discuss are adding synonyms detection and entity-recommendation.
In this talk, we will focus on the word-embeddings techniques explored, metrics used to evaluate Natural Language Processing Models, tools built, and future work as a part of improving search.
Speaker
Namrata Ghadi, Workday Inc, Software Development Engineer (Data Science)
Adam Baker, Workday Inc, Sr Software Engineer
This presentation will cover how all aspects of marketing have evolved over the years. How AI will shape the landscape of marketing in the years to come and why marketers need AI to assist them for their jobs. The future lies in working towards better customer experience and especially customer retention seems to be the key.
Marketers have to be on the lookout throughout - need to keep learning and keep a continuous tab on the customer’s pulse in order to deliver the best.
Invited Talk at Modern Data Management Systems Summit on August 29-30, 2014 at Tsinghua University in Beijing, China.
http://ise.thss.tsinghua.edu.cn/MDMS/English/program.jsp
Abstract:
Modern enterprises are increasingly relying on complex analyses on large data sets to drive business decisions. Tasks such as root cause analysis from system logs and lead generation based on social media, customer retention and digital marketing are rapidly gaining importance. These applications generally consist of three major analytic phases: text analytics, semi-structured data processing (joins, group-by, aggregation), and statistical/predictive modeling. The size of the datasets in conjunction with the complexity of the analysis necessitates large-scale distributed processing of the analytical algorithms. At IBM we are building tools and technologies based on declarative languages to support each of these analytic phases. The declarative nature of the language abstracts away the need for programmer-optimization. Furthermore, the syntax of these languages is designed to appeal to the corresponding communities. As an example for statistical modeling, we expose a high-level language with syntax similar to R -- a very popular statistical processing language.
In this talk I will give an overview of some real-world big data applications we are currently working on and use that to motivate the need for declarative analytics consisting of the three major phases discussed above. I will then describe, in some detail, declarative systems for text analytics along with a discussion on speeds, feeds and comparisons.
Interleaving, Evaluation to Self-learning Search @904LabsJohn T. Kane
Presented at Open Source Connections Haystack Relevance Conference on 904Labs' "Interleaving: from Evaluation to Self-Learning". 904Labs is the first to commercialize "Online Learning to Rank" as a state-of-art for technical Self-learning Search Ranking that automatically takes into account your customers human behaviors for personalized search results.
Human in the Loop AI for Building Knowledge Bases Yunyao Li
The ability to build large-scale domain-specific knowledge bases that capture and extend the implicit knowledge of human experts is the foundation for many AI systems. We use an ontology-driven approach for the creation, representation and consumption of such domain-specific knowledge bases. This approach relies on several well-known building blocks: natural language processing, entity resolution, data transformation and fusion. I will present several human-in-the-loop work that target domain experts (rather than programmers) to extract the domain knowledge from the human expert and map it into the "right" models or algorithms. I will also share successful use cases in several domains, including Compliance, Finance, and Healthcare: by using these tools we can match the level of accuracy achieved by manual efforts, but at a significantly lower cost and much higher scale and automation.
BigInsights and Text Analytics.
As enterprises seek to gain operational efficiencies and competitive advantage through greater use of analytics, much of the new information they need to analyze is found in text documents and, increasingly, in a wide variety of social media sites and portals. A critical step in gaining insights from this information is extracting core data from huge volumes of text. That data is then available for downstream analytic, mining and machine learning tools. AQL (Annotator Query Language) is a powerful declarative, rule-based language for the extraction of information from text documents.
Enterprise Search in the Big Data Era: Recent Developments and Open ChallengesYunyao Li
This is the slides used in our 3-hour tutorial at VLDB'2014.
Yunyao Li, Ziyang Liu, Huaiyu Zhu: Enterprise Search in the Big Data Era: Recent Developments and Open Challenges. PVLDB 7(13): 1717-1718 (2014)
Abstract:
Enterprise search allows users in an enterprise to retrieve desired information through a simple search interface. It is widely viewed as an important productivity tool within an enterprise. While Internet search engines have been highly successful, enterprise search remains notoriously challenging due to a variety of unique challenges, and is being made more so by the increasing heterogeneity and volume of enterprise data. On the other hand, enterprise
search also presents opportunities to succeed in ways beyond current Internet search capabilities. This tutorial presents an organized overview of these challenges and opportunities, and reviews the state-of-the-art techniques for building a reliable and high quality enterprise search engine, in the context of the rise of big data.
Natural language understanding is a fundamental task in artificial intelligence. English understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Discovery. However, scaling existing products/services to support additional languages remain an open challenge. In this talk, we will discuss the open challenges in supporting universal natural language understanding. We will share our work in the past few years in addressing these challenges. We will also showcase how universal semantic representation of natural languages can enable cross-lingual information extraction in concrete domains (e.g. compliance) and show ongoing efforts towards seamless scaling existing NLP capabilities across languages with minimal efforts.
Real-time Recommendations for Retail: Architecture, Algorithms, and DesignJuliet Hougland
Users are constantly searching for new content and to stay competitive organizations must act immediately based on up-to-date data. Outdated recommendations decrease the likelihood of presenting the right offer and make it harder to maintain customer loyalty. In order to provide the most relevant recommendations and increase engagement, organizations must track customer interactions and re-score recommendations on the fly.
Data sources have expanded dramatically to include a wealth of historical data and a constant influx of behavior data. The key to moving from predictive models, applied in batch, to models that provide responses in real time, is to focus on the efficiency of model application. The speed that recommendations can be served is influenced by:
Architecture of the recommendation serving platform
Choice of recommendation algorithm
Datastore access patterns
In this presentation, we’ll discuss how developers can use open source components like HBase and Kiji to develop low-latency recommendation models that can be easily deployed by e-commerce companies. We will give practical advice on how to choose models and design data stores that make use of the architecture and quickly serve new recommendations.
Toolboxes are collections of built-in functions that help data scientists perform tasks efficiently. The document discusses several Python toolboxes for data science like NumPy, Pandas, SciPy, and Scikit-Learn. It also covers IDEs like Jupyter Notebook that provide interactive environments for coding and data analysis. Overall, the document presents an overview of the Python toolbox ecosystem and how it enables effective data science work.
SystemT: Declarative Information ExtractionYunyao Li
Slides used for my talk "SystemT: Declarative Information Extraction" at the event "University of Oregon Big Opportunities with Big Data Meeting" on August 8, 2014 (http://bigdata.uoregon.edu).
Activate 2018 Closing Remarks: The Future of Search & AI - Trey GraingerLucidworks
The document discusses current and upcoming trends in search and AI. It notes that learning to rank is becoming widely adopted and that ease of operations, scaling, and performance are top priorities. It also discusses the many skills required to deliver world-class search, including relevance, scaling, natural language processing, and personalization. The future of search is shifting toward assistive search through voice, images, and conversations to provide answers and enable actions.
Nikhil Sharma has a Master of Science in Data Informatics from USC and a Bachelor of Engineering in Electronics and Communication from M.S. Ramaiah Institute of Technology in India. He has work experience as a Software Engineer Intern at Salesforce where he designed applications for database performance analysis using Python. Previously he was a Senior Systems Engineer at Infosys where he implemented IT infrastructure for banks in various countries. His academic projects involve machine learning, data mining, and information retrieval using technologies like Python, Solr, and Caffe.
The Machine Learning Workflow with AzureIvo Andreev
This document provides an overview of real world machine learning using Azure. It discusses the machine learning workflow including data understanding, preprocessing, feature engineering, model selection, evaluation and tuning. It then describes various Azure machine learning tools for building, testing and deploying machine learning models including Azure ML Workbench, Studio, Experimentation Service and Model Management Service. It concludes with an upcoming demo of predictive maintenance using Azure ML Studio.
Real-time big data analytics based on product recommendations case studydeep.bi
We started as an ad network. The challenge was to recommend the best product (out of millions) to the right person in a given moment (thousands of users within a second). We have delivered 5 billion ad views since 24 months. To put it in the scale context: If we would serve 1 ad per second it will take 160 years to serve 5 billion ads.
So we needed a solution. SQL databases did not work. Popular NoSQL databases did not work. Standard data warehouse approaches (pre-aggregations, creating schemas) - did not work too.
Re-thinking all the problems with huge data streams flowing to us every second we have built a complete solution based on open-source technologies and fresh, smart ideas from our engineering team. It is called deep.bi and now we make it available to other companies.
deep.bi lets high-growth companies solve fast data problems by providing scalable, flexible and real-time data collection, enrichment and analytics.
It was built using:
- Node.js - API
- Kafka - collecting and distributing data
- Spark Streaming - ETL, data enrichments
- Druid - real-time analytics
- Cassandra - user events store
- Hadoop + Parquet + Spark - raw data store + ad-hoc queries
Automatic suggestion of query-rewrite rules for enterprise searchYunyao Li
This document describes techniques for automatically suggesting query rewrite rules for enterprise search engines. It presents an algorithm that takes a query and desired document match as input and generates candidate rewrite rules by combining n-grams from the query and high-quality fields of the document. A classifier then labels candidate rules as natural or unnatural. The document also describes formulating the problem of optimizing rule selection as an NP-hard optimization problem and proposes greedy heuristic algorithms to solve it. Experiments on real enterprise search data show the techniques can effectively suggest rules and optimize rule sets to improve search accuracy.
FrugalML: Using ML APIs More Accurately and CheaplyDatabricks
FrugalML is a technique that uses machine learning to optimize usage of machine learning prediction APIs. It trains on data annotated by different APIs to learn a strategy that selects the best sequence of APIs to call within a given budget. This can achieve up to 90% lower costs or 5% better accuracy compared to using any single API. The strategy selects an initial "base" API and then may call additional "add-on" APIs based on the predictions and quality scores from previous APIs. FrugalML is proven to efficiently learn the optimal strategy and outperforms commercial APIs on various tasks and datasets in both cost and accuracy.
Data Analytics and Artificial Intelligence in the era of Digital TransformationJan Wiegelmann
The document discusses how data analytics and artificial intelligence are transforming businesses in the era of digital transformation. It covers the history and evolution of AI from early neural networks to today's deep learning approaches enabled by massive increases in data and computing power. Examples are given of how AI is now exceeding or matching human-level performance in areas like image recognition, medical diagnosis, and speech recognition. The document advocates that businesses leverage AI, data science, and a 360-degree view of customer data to drive personalization, predict customer needs, optimize operations, and gain competitive advantages in their industries.
Enterprise Search – How Relevant Is Relevance?Sease
Enterprise search is the outlier in search applications. It has to work effectively with very large collections of un-curated content, often in multiple languages, to meet the requirements of employees who need to make business-critical decisions.
In this talk, I will outline the challenges of searching enterprise content. Recent research is revealing a unique pattern of search behaviour in which relevance is both very important and yet also irrelevant, and where recall is just as important as precision. This behaviour has implications for the use of standard metrics for search performance (especially in the case of federated search across multiple applications) and for the adoption of AI/ML techniques.
Stefan Geissler kairntech - SDC Nice Apr 2019 Stefan Geißler
Describes the Kairntech approach to real-world NLP/AI requirements, putting an emphasis on the quick and efficient creation and curation of training data sets.
The document discusses Luxiaoteng's model for analyzing hybrid data and performing predictive analytics. The model combines traditional analysis techniques with domain knowledge and advanced machine learning. It builds a three dimensional data analysis matrix and designs customized models to address business needs. Case studies are presented on how Netflix uses machine learning recommendations and how BBVA Compass uses natural language processing on social media to understand customer sentiment. The document also describes a seed program that tracks content influences and preferences to reinvent viewing measurement and define targeted markets.
This presentation will cover how all aspects of marketing have evolved over the years. How AI will shape the landscape of marketing in the years to come and why marketers need AI to assist them for their jobs. The future lies in working towards better customer experience and especially customer retention seems to be the key.
Marketers have to be on the lookout throughout - need to keep learning and keep a continuous tab on the customer’s pulse in order to deliver the best.
Invited Talk at Modern Data Management Systems Summit on August 29-30, 2014 at Tsinghua University in Beijing, China.
http://ise.thss.tsinghua.edu.cn/MDMS/English/program.jsp
Abstract:
Modern enterprises are increasingly relying on complex analyses on large data sets to drive business decisions. Tasks such as root cause analysis from system logs and lead generation based on social media, customer retention and digital marketing are rapidly gaining importance. These applications generally consist of three major analytic phases: text analytics, semi-structured data processing (joins, group-by, aggregation), and statistical/predictive modeling. The size of the datasets in conjunction with the complexity of the analysis necessitates large-scale distributed processing of the analytical algorithms. At IBM we are building tools and technologies based on declarative languages to support each of these analytic phases. The declarative nature of the language abstracts away the need for programmer-optimization. Furthermore, the syntax of these languages is designed to appeal to the corresponding communities. As an example for statistical modeling, we expose a high-level language with syntax similar to R -- a very popular statistical processing language.
In this talk I will give an overview of some real-world big data applications we are currently working on and use that to motivate the need for declarative analytics consisting of the three major phases discussed above. I will then describe, in some detail, declarative systems for text analytics along with a discussion on speeds, feeds and comparisons.
Interleaving, Evaluation to Self-learning Search @904LabsJohn T. Kane
Presented at Open Source Connections Haystack Relevance Conference on 904Labs' "Interleaving: from Evaluation to Self-Learning". 904Labs is the first to commercialize "Online Learning to Rank" as a state-of-art for technical Self-learning Search Ranking that automatically takes into account your customers human behaviors for personalized search results.
Human in the Loop AI for Building Knowledge Bases Yunyao Li
The ability to build large-scale domain-specific knowledge bases that capture and extend the implicit knowledge of human experts is the foundation for many AI systems. We use an ontology-driven approach for the creation, representation and consumption of such domain-specific knowledge bases. This approach relies on several well-known building blocks: natural language processing, entity resolution, data transformation and fusion. I will present several human-in-the-loop work that target domain experts (rather than programmers) to extract the domain knowledge from the human expert and map it into the "right" models or algorithms. I will also share successful use cases in several domains, including Compliance, Finance, and Healthcare: by using these tools we can match the level of accuracy achieved by manual efforts, but at a significantly lower cost and much higher scale and automation.
BigInsights and Text Analytics.
As enterprises seek to gain operational efficiencies and competitive advantage through greater use of analytics, much of the new information they need to analyze is found in text documents and, increasingly, in a wide variety of social media sites and portals. A critical step in gaining insights from this information is extracting core data from huge volumes of text. That data is then available for downstream analytic, mining and machine learning tools. AQL (Annotator Query Language) is a powerful declarative, rule-based language for the extraction of information from text documents.
Enterprise Search in the Big Data Era: Recent Developments and Open ChallengesYunyao Li
This is the slides used in our 3-hour tutorial at VLDB'2014.
Yunyao Li, Ziyang Liu, Huaiyu Zhu: Enterprise Search in the Big Data Era: Recent Developments and Open Challenges. PVLDB 7(13): 1717-1718 (2014)
Abstract:
Enterprise search allows users in an enterprise to retrieve desired information through a simple search interface. It is widely viewed as an important productivity tool within an enterprise. While Internet search engines have been highly successful, enterprise search remains notoriously challenging due to a variety of unique challenges, and is being made more so by the increasing heterogeneity and volume of enterprise data. On the other hand, enterprise
search also presents opportunities to succeed in ways beyond current Internet search capabilities. This tutorial presents an organized overview of these challenges and opportunities, and reviews the state-of-the-art techniques for building a reliable and high quality enterprise search engine, in the context of the rise of big data.
Natural language understanding is a fundamental task in artificial intelligence. English understanding has reached a mature state and successfully deployed in multiple IBM AI products and services, such as Watson Natural Language Understanding and Watson Discovery. However, scaling existing products/services to support additional languages remain an open challenge. In this talk, we will discuss the open challenges in supporting universal natural language understanding. We will share our work in the past few years in addressing these challenges. We will also showcase how universal semantic representation of natural languages can enable cross-lingual information extraction in concrete domains (e.g. compliance) and show ongoing efforts towards seamless scaling existing NLP capabilities across languages with minimal efforts.
Real-time Recommendations for Retail: Architecture, Algorithms, and DesignJuliet Hougland
Users are constantly searching for new content and to stay competitive organizations must act immediately based on up-to-date data. Outdated recommendations decrease the likelihood of presenting the right offer and make it harder to maintain customer loyalty. In order to provide the most relevant recommendations and increase engagement, organizations must track customer interactions and re-score recommendations on the fly.
Data sources have expanded dramatically to include a wealth of historical data and a constant influx of behavior data. The key to moving from predictive models, applied in batch, to models that provide responses in real time, is to focus on the efficiency of model application. The speed that recommendations can be served is influenced by:
Architecture of the recommendation serving platform
Choice of recommendation algorithm
Datastore access patterns
In this presentation, we’ll discuss how developers can use open source components like HBase and Kiji to develop low-latency recommendation models that can be easily deployed by e-commerce companies. We will give practical advice on how to choose models and design data stores that make use of the architecture and quickly serve new recommendations.
Toolboxes are collections of built-in functions that help data scientists perform tasks efficiently. The document discusses several Python toolboxes for data science like NumPy, Pandas, SciPy, and Scikit-Learn. It also covers IDEs like Jupyter Notebook that provide interactive environments for coding and data analysis. Overall, the document presents an overview of the Python toolbox ecosystem and how it enables effective data science work.
SystemT: Declarative Information ExtractionYunyao Li
Slides used for my talk "SystemT: Declarative Information Extraction" at the event "University of Oregon Big Opportunities with Big Data Meeting" on August 8, 2014 (http://bigdata.uoregon.edu).
Activate 2018 Closing Remarks: The Future of Search & AI - Trey GraingerLucidworks
The document discusses current and upcoming trends in search and AI. It notes that learning to rank is becoming widely adopted and that ease of operations, scaling, and performance are top priorities. It also discusses the many skills required to deliver world-class search, including relevance, scaling, natural language processing, and personalization. The future of search is shifting toward assistive search through voice, images, and conversations to provide answers and enable actions.
Nikhil Sharma has a Master of Science in Data Informatics from USC and a Bachelor of Engineering in Electronics and Communication from M.S. Ramaiah Institute of Technology in India. He has work experience as a Software Engineer Intern at Salesforce where he designed applications for database performance analysis using Python. Previously he was a Senior Systems Engineer at Infosys where he implemented IT infrastructure for banks in various countries. His academic projects involve machine learning, data mining, and information retrieval using technologies like Python, Solr, and Caffe.
The Machine Learning Workflow with AzureIvo Andreev
This document provides an overview of real world machine learning using Azure. It discusses the machine learning workflow including data understanding, preprocessing, feature engineering, model selection, evaluation and tuning. It then describes various Azure machine learning tools for building, testing and deploying machine learning models including Azure ML Workbench, Studio, Experimentation Service and Model Management Service. It concludes with an upcoming demo of predictive maintenance using Azure ML Studio.
Real-time big data analytics based on product recommendations case studydeep.bi
We started as an ad network. The challenge was to recommend the best product (out of millions) to the right person in a given moment (thousands of users within a second). We have delivered 5 billion ad views since 24 months. To put it in the scale context: If we would serve 1 ad per second it will take 160 years to serve 5 billion ads.
So we needed a solution. SQL databases did not work. Popular NoSQL databases did not work. Standard data warehouse approaches (pre-aggregations, creating schemas) - did not work too.
Re-thinking all the problems with huge data streams flowing to us every second we have built a complete solution based on open-source technologies and fresh, smart ideas from our engineering team. It is called deep.bi and now we make it available to other companies.
deep.bi lets high-growth companies solve fast data problems by providing scalable, flexible and real-time data collection, enrichment and analytics.
It was built using:
- Node.js - API
- Kafka - collecting and distributing data
- Spark Streaming - ETL, data enrichments
- Druid - real-time analytics
- Cassandra - user events store
- Hadoop + Parquet + Spark - raw data store + ad-hoc queries
Automatic suggestion of query-rewrite rules for enterprise searchYunyao Li
This document describes techniques for automatically suggesting query rewrite rules for enterprise search engines. It presents an algorithm that takes a query and desired document match as input and generates candidate rewrite rules by combining n-grams from the query and high-quality fields of the document. A classifier then labels candidate rules as natural or unnatural. The document also describes formulating the problem of optimizing rule selection as an NP-hard optimization problem and proposes greedy heuristic algorithms to solve it. Experiments on real enterprise search data show the techniques can effectively suggest rules and optimize rule sets to improve search accuracy.
FrugalML: Using ML APIs More Accurately and CheaplyDatabricks
FrugalML is a technique that uses machine learning to optimize usage of machine learning prediction APIs. It trains on data annotated by different APIs to learn a strategy that selects the best sequence of APIs to call within a given budget. This can achieve up to 90% lower costs or 5% better accuracy compared to using any single API. The strategy selects an initial "base" API and then may call additional "add-on" APIs based on the predictions and quality scores from previous APIs. FrugalML is proven to efficiently learn the optimal strategy and outperforms commercial APIs on various tasks and datasets in both cost and accuracy.
Data Analytics and Artificial Intelligence in the era of Digital TransformationJan Wiegelmann
The document discusses how data analytics and artificial intelligence are transforming businesses in the era of digital transformation. It covers the history and evolution of AI from early neural networks to today's deep learning approaches enabled by massive increases in data and computing power. Examples are given of how AI is now exceeding or matching human-level performance in areas like image recognition, medical diagnosis, and speech recognition. The document advocates that businesses leverage AI, data science, and a 360-degree view of customer data to drive personalization, predict customer needs, optimize operations, and gain competitive advantages in their industries.
Enterprise Search – How Relevant Is Relevance?Sease
Enterprise search is the outlier in search applications. It has to work effectively with very large collections of un-curated content, often in multiple languages, to meet the requirements of employees who need to make business-critical decisions.
In this talk, I will outline the challenges of searching enterprise content. Recent research is revealing a unique pattern of search behaviour in which relevance is both very important and yet also irrelevant, and where recall is just as important as precision. This behaviour has implications for the use of standard metrics for search performance (especially in the case of federated search across multiple applications) and for the adoption of AI/ML techniques.
Stefan Geissler kairntech - SDC Nice Apr 2019 Stefan Geißler
Describes the Kairntech approach to real-world NLP/AI requirements, putting an emphasis on the quick and efficient creation and curation of training data sets.
The document discusses Luxiaoteng's model for analyzing hybrid data and performing predictive analytics. The model combines traditional analysis techniques with domain knowledge and advanced machine learning. It builds a three dimensional data analysis matrix and designs customized models to address business needs. Case studies are presented on how Netflix uses machine learning recommendations and how BBVA Compass uses natural language processing on social media to understand customer sentiment. The document also describes a seed program that tracks content influences and preferences to reinvent viewing measurement and define targeted markets.
Natural Language Processing Use Cases for Business OptimizationTakayuki Yamazaki
This document discusses several use cases for natural language processing (NLP) in business optimization. It begins with an overview of NLP, describing how it recognizes and understands human language through techniques like named entity recognition, part-of-speech tagging, sentiment analysis, and text classification. The document then outlines seven NLP use cases: using NLP for epidemiological investigations, security authentication, brand and market research, customer support chatbots, competitive analysis, automated report generation, and real-time stock analysis based on news and reports.
Using the power of OpenAI with your own data: what's possible and how to start?Maxim Salnikov
This document provides an overview of a talk by Maxim Salnikov and Jon Jahren at Oslo Spektrum from November 7-9. It discusses using OpenAI with your own data and how to get started. Examples of enterprise use cases for generative AI are presented, such as chatbots, document indexing, and financial analysis. Tools for prompt engineering like LangChain and Semantic Kernel are introduced. Best practices for fine-tuning models on proprietary data are covered, including data formatting, training data size, and an iterative tuning process. Responsible AI techniques like grounding responses and maintaining a positive tone are also discussed.
DataScientist Job : Between Myths and Reality.pdfJedha Bootcamp
Swipe through the smoke and mirrors and learn about the "sexiest job of the 21st century" with Nicola, Machine Learning Scientist @ Bumble
✨ Artificial Intelligence? Business Intelligence? Data Science? What do these terms sound like when put into action at one of the world's most forefront dating platforms? Jedha is proud to host an evening with Nicola Ghio, Senior Machine Learning Scientist at Bumble, who will give us a "peek behind the curtain" into what this enviable job title looks like in practice.
😎 Nicola will share some of his experiences working at Bumble. 🎯 Hear first-hand about Bumble's harassment and toxic imaging detector as well as the real skills required to work in the industry. We also look forward to hearing about Nicola's personal story, his background and his advice for those that want to dive deeper into the world of tech.
Meet Jedha 😍 Your Data and Cyber Security Bootcamp, ranked #1 in Europe (Switch Up). Our mission is to demystify the world of tech and to make its skills accessible to anyone who desires to learn. We have courses suited to all ambitions and skill levels: From beginners who have never typed a line of code in their lives right through to skilled tech professionals who want to achieve mastery. Our methods and teachers help to unlock human potential in the unlimited world of tech.
MLflow: Infrastructure for a Complete Machine Learning Life CycleDatabricks
ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but these platforms are limited to each company’s internal infrastructure.
In this talk, we will present MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.
Accenture's report explains how natural language processing and machine learning makes extracting valuable insights from unstructured data fast. Read more. https://www.accenture.com/us-en/insights/digital/unlocking-value-unstructured-data
Future directives in erp, erp and internet, critical success and failure factorsVarun Luthra
This ppt explains Future Directives in ERP, ERP and Internet, its critical success and failure factors, Hit 'Like' button if the ppt turns out to be useful for you in any way. Enjoy :)
The document discusses digitalization through the use of domain-specific languages (DSLs). It suggests that DSLs can help accelerate development, simplify customization, and express business goals, requirements, design, and implementation in a single language. The document outlines considerations for whether an organization needs a DSL, how to structure a proof of concept, and how to ensure long term maintenance and adoption of the DSL approach.
Tom van Ees - Academic and Commercial software DevelopmentDavinci software
The document provides an overview of similarities and differences between academic and commercial software development. It discusses types of software like bespoke vs product-based and their complexities. Key factors in commercial software sales like convincing decision-makers during demos and end-users during daily use are outlined. Important aspects of making software sellable like always considering the customer, focusing on aesthetics, and frequent releases are highlighted. The role of the developer in maintaining quality, using mainstream technologies, and not becoming too specialized is also discussed.
AI for Customer Service: How to Improve Contact Center Efficiency with Machin...Skyl.ai
About the webinar
It only takes one bad interaction for a customer to abandon a service or product. Businesses are no longer just competing with other companies’ products, they’re competing with a customer’s last service experience. All contact centers worldwide are looking for new and strategic ways to increase operational performance, reduce cost, and still provide high-touch customer experiences that improve customer loyalty and highlight ways to increase revenue and productivity.
Through this webinar, we will understand how AI can augment the effort, focus and problem-solving abilities of human agents so that they can tackle more complex or creative tasks. With an abundance of data from logs, emails, chat, and voice recordings, contact centers can ingest this data to provide contextual customer service at the right time with the right way providing satisfactory customer service and retain the brand value.
What you will learn
- How organizations are building engaging interactions that deliver value to customers
- Best practices to automate AI/ML models
- Demo: How to route customer queries to the right department or professional
The document discusses emerging technological trends identified by IBM's Global Technology Outlook. It focuses on the transition to a new era of cognitive computing driven by trends like mobility, social media, cloud computing, and big data/analytics. This cognitive systems era will be characterized by systems that can understand, reason, and learn. Technologies like IBM's Watson cognitive system aim to augment human expertise by understanding natural language, generating hypotheses, and providing personalized recommendations. APIs and cloud-based platforms are also fueling a new on-demand, "as-a-service" economy where businesses can access modular business services to quickly develop solutions and create new value for customers.
This document discusses Symfony, a PHP framework. It provides an overview of Symfony, explaining that it is a PHP framework and a set of tools and development methodology. It then discusses some key aspects of Symfony, including how it provides a toolbox of prefabricated components to write less code more productively, and a methodology for structured application development. The document also summarizes how Symfony handles HTTP requests and responses through its Request and Response classes to provide an object-oriented interface for building web applications. Finally, it briefly outlines how to install Symfony distributions and configure a Symfony project.
The document describes a Driverless ML API that was created to automate machine learning workflows including feature engineering, model validation, tuning, selection, and deployment. The API uses machine learning interpretability techniques to provide visualizations and explanations of models. It aims to help scale data science efforts and enable both expert and junior data scientists to more quickly develop accurate, production-ready models. Key capabilities of the API include automated exploratory data analysis, feature selection and engineering, model selection and hyperparameter tuning using GPUs for faster training, and model interpretability visualizations.
The document describes several Watson services available on IBM's Watson Developer Cloud platform:
1) User Modeling extracts cognitive and social characteristics from user communications to help understand user preferences.
2) Question and Answer interprets questions and returns responses directly from source documents.
3) Relationship Extraction identifies entities and relationships within unstructured text.
AI & Cognitive Computing are some of the most popular business an technical words out there. It is critical to get the basic understanding of Cognitive Computing, which helps us appreciate the technical possibilities and business benefits of the technology.
Driving Customer Loyalty with Azure Machine LearningCCG
Learn how you can leverage the elastic, on-demand processing power of Microsoft Azure to create faster, more applicable analytics by viewing this informative webinar. Data Scientist and Author, Ahmed Sherif, demonstrates key analytic use cases that can be spun up quickly with minimal effort and maximum return on investment. To watch the full recording of this webinar, visit http://ccgbi.com/resources/webinars/driving-customer-loyalty-with-AML
Similar to Natural Language Processing at Scale (20)
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
State of Artificial intelligence Report 2023kuntobimo2016
Artificial intelligence (AI) is a multidisciplinary field of science and engineering whose goal is to create intelligent machines.
We believe that AI will be a force multiplier on technological progress in our increasingly digital, data-driven world. This is because everything around us today, ranging from culture to consumer products, is a product of intelligence.
The State of AI Report is now in its sixth year. Consider this report as a compilation of the most interesting things we’ve seen with a goal of triggering an informed conversation about the state of AI and its implication for the future.
We consider the following key dimensions in our report:
Research: Technology breakthroughs and their capabilities.
Industry: Areas of commercial application for AI and its business impact.
Politics: Regulation of AI, its economic implications and the evolving geopolitics of AI.
Safety: Identifying and mitigating catastrophic risks that highly-capable future AI systems could pose to us.
Predictions: What we believe will happen in the next 12 months and a 2022 performance review to keep us honest.
Natural Language Processing (NLP), RAG and its applications .pptxfkyes25
1. In the realm of Natural Language Processing (NLP), knowledge-intensive tasks such as question answering, fact verification, and open-domain dialogue generation require the integration of vast and up-to-date information. Traditional neural models, though powerful, struggle with encoding all necessary knowledge within their parameters, leading to limitations in generalization and scalability. The paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" introduces RAG (Retrieval-Augmented Generation), a novel framework that synergizes retrieval mechanisms with generative models, enhancing performance by dynamically incorporating external knowledge during inference.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
1. Natural Language
Processing At Scale
For optimizing business success and Customer
Experience
ML Ops Aspects
MLOps: Production and Engineering - Bay Area, March 2021
Andrei Lopatenko, VP Engineering, Zillow
2. What’s the focus of this talk
How to implement and use Natural Language Processing in with your organization
at scale for a large business impact, with low development and infrastructure
costs
How to solve many different business problems with NLP, improve customer
experience,
How build NLP development processes, ops serving your business
3. NLP at Scale
Building NLP systems ‘at scale’
At scale means both
1. for multiple business tasks, building systems for a wide adoption within the company for very
heterogeneous tasks related to processing of natural languages and processing online requests of
your customers/users, documents, etc
2. For high load in the number of users’ requests per day/second, the number of documents to be
processed per second, the number of documents to be processed in one batch etc
Doing it right way has the high ROIs
I would like to advocate that building company wide and deep impact NLP systems is relatively ‘easy’ now
vs 5-10 years ago: it’s doable within relatively short period of times, with small investments, low
maintenance costs but big business and customer experience impact
4. Why I am talking about it
I have been applying NLP in Google, Apple, WalmartLab, eBay, Zillow since 2006.
Core contributor to core ranking Google Search 2006, co-founder Apple Maps Search (2010), and core
contributor AppStore Search, Walmart search, led Walmart (2014) and eBay Search Science teams,
engineering of Recruit Holding AI Lab, leading Zillow Search and Conversational AI (2019-now). Startups:
Ozlo (NLP, Conversational AI startup, acquired by Facebook in 2017),
In every organization I worked for, NLP was one of key technologies driving business and customer
experience
In 2021, due to abundance of NLP tools from development to serving, building big impact NLP systems is
more available than even several years ago
I’d like to share my 15-year experience of how to build NLP systems at scale for customer and business
gains
5. Motivating examples: NLP use cases. Case 1
Customer facing online systems: Search
Example: Web, Maps, Real Estate , eCommerce, Apps, and any other big search engines
https://blog.google/products/search/search-language-understanding-bert/
BERT - new NLP models radically changes search for 10% of traffic (reported in 2018)
But ‘old’ NLP techniques such as synonym expansion, term weighting, shallow parsing, phrase
chunking, query classification and many other have been driving majority of search experience
online since early 200X
This is applicable to any search engine (ecommerce, apps, films, real estate), NLP radically
improve quality of search results leading to improvement of customer experience and revenues
through purchases
6. Motivating example. NLP use cases. Case 2
Customer facing online systems: Recommendations .
Example from Zillow. Embedding representing information about properties
extracted from full text helps with online recommendations and other downstream
application. Similar homes recommendations.
https://www.zillow.com/tech/improve-quality-listing-text/
7. Motivating examples: NLP use cases. Case 3
Business facing Question Answering / online
Example: Bloomberg Trade Order Management Solution
https://www.bloomberg.com/professional/blog/bloomberg-adds-new-nlp-capabilities-to-to
ms/
Questions such as “Who are our top 5 accounts in the tech sector?”
Natural Language based Question Answering to unstructured text information (documents, find the
paragraph with answer “what’s our return policy”) and structured information in databases (“how many
umbrellas we solt last week”) (Natural Language Interface to Databases NLIDB)
8. Motivating examples: NLP use cases. Case 4
Analysis of conversations / (near) real-time streaming
Fascinating example: school project : Prioritization of Emergency Dispatch Calls, 3 high school participants built a
system to analyze emergency phone calls and assess their priority
https://medium.com/ai4allorg/using-natural-language-processing-to-prioritize-emergency-dispatch-calls-ab830a72de
98
Recent, example, European company Corti deploys a realtime system to analyze calls and detect Cardiac arrests ,
it’s more voice analysis (area close to the NLP) rather than NLP, but typically similar MLOps and systems as for the
the NLP systems
https://www.theverge.com/2018/4/25/17278994/ai-cardiac-arrest-corti-emergency-call-response
Understanding phone calls, transcribing them, assessing quality of service, needs of customer, performance of
customer support/business to get customer insights, assess quality of business agents, quality of conversations,
extract global insights
9. Motivating examples: NLP use cases. Case 5
Customer support dialog systems (Chatbots)
Example: Amazon customer support chatbot
https://lifehacker.com/use-a-chatbot-for-faster-amazon-returns-1843927743
Reporting a problem to amazon, amazon chatbot solves many customer problems
Very fast and efficient, reduced costs on huma force
10. Motivating examples: NLP use cases. Case 6
Item understanding
Example: Amazon or Walmart marketplaces
Getting large stream of unstructured data from various providers
Converting it into structured data (rather than embedding as in ex 2),
understanding items, both for customer and business applications (extraction of
attributes of items from merchant descriptions, analysis of reviews): multiple
business and consumer facing downstream applications from online user
experience to business analytics
https://dl.acm.org/doi/abs/10.1145/3183713.3196926
11. Motivating Examples: summary
NLP is already used to improve customer experience and business in many very different
types of businesses and across different customer experiences and business application.
It has high ROI if applied correctly (right applications, right technologies, right people)
There are very different ways to apply NLP: online, streaming, batch processing, it
frequently requires different types of systems - task is to build MLOps for all of them
6 use cases - mostly random list: there are too many other NLP use cases (
autocomplete online, classification by category offline, ...) where it radically improves
customer experience and brings big business gains
12. NLP: impact on business
Most of media hype about the NLP is about chatbots
In 2020, most of business impact of NLP is in other areas (but conversational
systems are useful too)
Better search, recommendation, personalization based on NLP -> billions of
dollars
Document understanding, better classification, information extraction -> hundreds
of millions of dollars
13. NLP serving scenarios
online scenarios : customer facing applications, critical latency and throughput,
such as search query understanding, up to 100s models, 50 ms latency, 10000+
qps, business facing (question answering)
Streaming scenarios: documents, items, processing relatively large texts (10K
symbols), 100 millions per day
Batch scenarios: documents, users, (process billion documents to extract data,
latency requirements might be process batch within a day or 4 hours etc, depends
on business needs)
14. NLP systems
But applying is NLP is not just about training models, it’s about building systems
Systems which will
1. Serve models in production (various serving scenarios)
2. Train models (science workbenches)
3. Annotate data
4. Deploy models from lab to production
5. Test models, validate models, monitor models (performance, accuracy, compliance, fairness, models
and end to end systems)
6. Integrate model serving with instream of production data
7. Integrate with outstreams : consumer and business facing applications
15. NLP Systems
Majority of big business revenue and customer experience gains are not from the
most recent, the best NLP models (‘the best science’ )
But from ‘the best engineering’, high performance, reliable, robust, scalable
systems, which are integratable with multiple business and consumer
applications, monitorable, debuggable
Focus is on reliability, robustness, performness, operations excellence,
development engineering quality, openness for collaboration (across functions:
data engineering, NLP scientists, DS scientists, application developers, etc)
Once system works and brings value, state of the art models (accuracy, no bias,
fairness, performance) is the focus
16. NLP systems
Scientist workbench: access to data sets (from large corpora: web or search logs
to ‘small’ ), annotation tools, data processing and data management, metric tools,
model training, tuning, model management (sharing, storing, retrieving)
Deployment tools: model validation, deployment into various environments
(integrated with CI/CD), model management,
Inference: model workflows, monitoring, alerting, online validation, performance
measurement/’observability’, hardware allocation / scaling
Integration with instream and outstream
17. NLP systems: high level 3 ways
Cloud native: build the system using standard cloud components
From scratch: write your own from scratch ,
Hybrid: using big open source or cloud blocks for certain tasks (there are plenty of
those now), custom build systems for other tasks
18. NLP systems; Cloud native
Multiple ways to build NLP systems
high level cloud NLP and ML services, Amazon Comprehend, Sagemaker,
Sagemaker Ground Truth, Transcribe (for Speech to Text) , Text Analytics, Lex.
Textract
Pros: Very fast to develop a prototype and to make working systems, low
development costs, easy to integrate with other systems on the same cloud
(Redshift if amazon etc), low cost operations for managed solution
Cons: high cloud/compute costs, low flexibility in the types of models to develop
and opportunities to develop high accuracy models, performance is not optimized,
integration with non-cloud systems
19. NLP systems Cloud native
Advantage: Fast MLOps pipeline development
Plenty of tools: S3 for models and artifacts, CloudFormation, AWS CodePipeline
and CodeBuild (with Git) ,ECR Container Registry, SageMaker, AWS Batch, API
Gateway, Sagemaker pipelines = NLP Services, Comprehend - very fast to build
and prototype NLP systems
Another advantage: reasonably easy to adopt to your environment, Terraform
instead of cloudformation, your serving infrastructure instead of AWS
Easy to build multi environments deployment scenarios
20. NLP Systems. Built from scratch
Built from scratch or based on (rewritten if needed) open source
Abundance of open source: PyTorch serving/TF Serving, Hugging Face
Transformers, AllenNLP Lab environment, Spacy(plenty of other NLP libraries),
Docano (annotations)
Pros: more opportunities of optimization for accuracy of models and performance
of systems, customization for your company needs, owning the software
Cons: longer prototype and production development times, high operational
support costs,
21. NLP Systems Built from scratch
Might be necessary - example, your NLP models as a part of query understanding
stack, 100s models, Gb+ dictionaries, complicated dependencies, specialized
hardware - flash drive storage etc is required on many nodes, latency and
throughput critical.
There is no good available software to serve this scenario. Nevertheless, part of
this stack can be based on open source (training, model sharing, annotation,
analysis of experiments, monitoring, )
22. NLP Systems. Mix of cloud and custom built
Mix of cloud and custom built software:
Cloud solution for serving: Sagemaker, AWS Batch, Elastic Inference: different
scenarios
Or Kubeflow on AWS
Pros: quite rapid development and deployment
Cons: cloud costs higher than in the build from scratch scenario but less than in
the cloud native scenario, development costs are cheaper than in bfs, but higher
than in the cloud native,
23. NLP Systems. Mix of cloud and custom built
Multiple scenarios of deployments (managed Kubernetes vs own)
Requires support to build custom expansions (Kubeflow operators for your serving
frameworks, not all native kubeflow operators are good - require some work to
improve them )
Many high level tools are available: Kubeflow, Cortex (from CortexLabs),
Hydrosphere (managing, monitoring models), Seldon (serving), Neptune
(experiment management), MLFlow (experiment, model, data tracking,
deployment, model registry), Comet (experiment tracking, comparison)
And low level tools: Istio, Kubernetes, Prometheus
24. NLP Systems. Mix of cloud and custom build
Kubeflow , example, streaming, non latency / qps critical online
Kubeflow pipelines, (end to end orchestration)
TF.Serving, Seldon, (serving)
Jupyter, katib, modeldb, TFMA (TF Model Analysis), TF transform (training,
workbench)
Pytorch, Tensorflow, MXNet
25. NLP systems development and adoption timeline
NLP is relatively new for many businesses, there is a lot of excitement and a lot of uncertainties
in expectations
To prove value , one have to iterate very fast, build NLP systems and models rapidly, integrate
them with business systems and environments fast, with minimum development (human+
software+) costs - show the value from business and customers points of view.
Build cloud native system fast - show the value to the business, and as it scales by the number of
consumers, lines of business, data, other loads -> move to other architectures if needed to
improve performance, costs. Important to build a good evolvable design from the beginning (it’s
true for any system, the evolvability is as important as scalability etc)
26. NLP systems - tradeoff
Tradeoffs because of difference methods: classical vs deep neural
1. Inference: 10% model accuracy vs 90% latency difference (gain in customer
experience/conversion due to quality vs loss in customer experience due to
latency and op costs)
2. Training: ex: 1 billion documents, need results in 4 hours, training time
● The design must support running very different solutions inside
● Organizational structure must support taking such decision
● Analytics/ROI assessment must support proving proper data as input to make
such decision
Many other tradeoffs
27. Scalability by design
When building, important to design systems to be scalable in multiple dimensions: it’s hard to overestimate
future demands
1. The number of human languages and domain area languages
2. The load (qps online systems, the number of messages per second - streaming, the number of
documents in batch and the number of batches - batch systems )
3. The number of different models and the number of different types of models (extraction,
classification, correction, text prediction, text generation etc)
4. The number of developers and scientists working simultaneously deploying new models, new types
of data, new integrations etc
5. The number of metrics the system is monitored and the models are monitored
6. The amounts of data in training, serving
7. The number of use cases, the number of deployments (data centers, regions, nodes)
8. etc
28. NLP at Scale
Important factor: typically, there are very different serving scenarios from,
example, search online - dozen/hundreds models with multi gigabyte ‘dictionaries’,
some are in parallel, some are consequential, 50ms latency, 10^4+ qps, to
streaming - billion documents per day to batch processing. No one system will
serve all inference cases, necessary to build multiple systems
But training, verification, testing scenario are more unifiable, and it’s possible to
build one scientist workbench, lab environment. It’s beneficial to build one to share
data and models across the organization
29. Ops
Continuous retraining of models when needed
Support of frequent deployment of models as models are improved and new
models are deployed. Integration of NLP scientist workbench with production
environment. Validation of models
Scalability, how the system scale as traffic, stream, batch size increase, size of
document increase, the number of models run in parallel, other load parameters
may change
Monitoring for performance, incidents, exceptions, quality of models and
end-to-end applications based on the NLP
30. Ops
Monitoring - what may go wrong:
1. Model performance : model and end-to-end (overall, by segments: users,
categories, regions)
2. Global data changes (changes in global distributions caused by events or seasonal
shifts.. )
3. Incoming data quality issues
4. System performance, uptime
5. Biases, compliances, fairness
6. Significant changes, outliers
Monitoring, alerting, logging
31. NLP libraries
(separation is conditional, many of them are in both categories)
‘Old’ good technologies: hidden markov chains, conditional random fields, SVM for classification, PCFGs and Dirichlet processes
and software: Stanford CoreNLP, CRFSuite, CRF++, OpenNLP, MeTA, Sempre, Mallet (still useful in some scenarios) - tradeoffs are
in next slides
‘New’ technologies: Spacy, GenSim, Hugging Face Transformers (invaluable by now), FastText, AllenNLP (lab environment),
PyTorch NLP, FlairNLP, DeText, many others
A lot of academic open source code which is adaptable to industrial environments (see Papers with Code, NLP section)
High level libraries helping to build end to end solutions for some domains: Rasa (dialog systems),
Do not hesitate to get inside of open source: Stanford library performance was improved 10X by proper multithreading
implementation and it makes a big difference when you need to process a stream of large documents 40+ millions tokens per hour
32. Team
To build, support and use system successfully
Strong engineering, science, and product management is required
Modern NLP stack based on deep neural architectures -> BERT and other
Deep understanding of cloud ML infrastructure if you are on the cloud (example
AWS ML infrastructure)
Generic software engineering - building systems rather than just models
Engineering culture, Ops
33. Data Training sets
Many NLP models are re-usable for many tasks
You company operates in a certain domain such as eCommerce or real estate or
medical or transportation with its particular language. Models and knowledge
which learnt particularities of the domain language for a certain case might be
re-usable for other cases in the same domain (by various technique). Model
discovery, re-sharing simplifies adoption of the NLP across multiple lines of
business
Training and testing data resharing - accelerates model development and the
NLP adoption
34. NLP Training sets and Metrics
Training sets are important as they train your models for something important/beneficial and
metrics are important if they contribute to measurement of the final impact
What are classification tasks which will benefit your business (improve conversion or purchase
rate for search, better routing of phone calls or customer support ticket ), what are extraction
tasks which will benefit your business (what knowledge graph do you need for better search or
recommendation, which entities are important for browsing by your business agents etc ) , not
‘what’s perplexity of the language model’ but ‘how many symbols customer type in autocomplete
or what percentage of spelling errors is solved’- what will improve the end to end quality and
performance
Focus is on end to end performance rather than classical NLP level metrics only and mature
your systems by development them to impact end to end quality and performance
35. Continuous improvement circle
Almost none of real world NLP tasks can have a final ‘perfect’ solution <- needs a lot of
leadership support to promote this vision and align with business, incremental gains in system
performance and model accuracy means gains to the business (but need to build system,
measurement, and attribution framework to execute well on it)
Each NLP model can be improved in accuracy, perplexity, etc but what really important is impact
on end to end system - conversion, revenue per session, document processing time etc
Each NLP system can be improved from performance, scalability, cloud costs etc points of view.
Improvement of NLP models and NLP systems has high ROI if done correctly but to do it
correctly requires a lot of work. End to end analysis of systems rather than just model evaluation.
Attribution analysis etc. In big business, building such environment and building organization to
improve NLP pays back
36. ROI assessment. Expenses
Expenses: salaries + software costs + compute/storage costs + data annotation costs
Small team of several good experts can create an NLP system, integrate it with business
within your company and prove value of it. You do not need more than 5 people to solve
serious tasks
Software costs. Most business cases can be solved using open source software,
Hugging Face Transformers, PyTorch Serve or TF Serving etc whole infrastructure for
training and serving (and other tasks : annotation, can be built using open source)
Compute/Storage costs. Depends. AWS Comprehend etc - more expensive, less flexible,
but fast prototyping. GPU machines are needed in many cases
Data annotation. With transfer learning you do not need huge data sets.
37. ROI assessment. Returns
For some tasks, such as search/recommendation functions directly facing
consumer, the return is easily computed running online controlled experiments.
For some tasks, such as business facing functions: ex. Document classification to
make faster processing, question answering for agents, return is harder to
compute, since one needs to run new business operation for period of time to
measure impact
Some tasks, replacing humans for information extraction, or question answering to
consumer/customer support: return is computed by the number of people
replaced.
Key: build solutions rapidly, to experiment and find the maximum returns.
38. Conclusion
NLP systems bring significant gains to business and customer experience
Building them is relatively easy task. There are multiple open source libraries,
multiple cloud solutions, there are multiple alternatives how to build NLP system
for your company.
The task of building and using NLP typically has high ROI if approached correctly