MLconf NYC Josh Wills

Application performance monitoring (APM) has become the cornerstone of software engineering allowing engineering teams to quickly identify and remedy production issues. However, as the world moves to intelligent software applications that are built using machine learning, traditional APM quickly becomes insufficient to identify and remedy production issues encountered in these modern software applications. As a lead software engineer at NewRelic, my team built high-performance monitoring systems including Insights, Mobile, and SixthSense. As I transitioned to building ML Monitoring software, I found the architectural principles and design choices underlying APM to not be a good fit for this brand new world. In fact, blindly following APM designs led us down paths that would have been better left unexplored. In this talk, I draw upon my (and my team’s) experience building an ML Monitoring system from the ground up and deploying it on customer workloads running large-scale ML training with Spark as well as real-time inference systems. I will highlight how the key principles and architectural choices of APM don’t apply to ML monitoring. You’ll learn why, understand what ML Monitoring can successfully borrow from APM, and hear what is required to build a scalable, robust ML Monitoring architecture.

Group Health Cooperative Customer Presentation

To rephrase an old saying: ‘It takes a village to raise an Analyst.’ Data Analysts and Scientists are working in teams delivering insight and analysis on an ongoing basis. So how do you get the team to support experimentation and insight delivery without ending up in an IT Engineer vs Analyst vs Data Governance war? We present 5 shocking steps to get these teams of people working together with practical, doable steps that can help you achieve data agility.

Developing Highly Instrumented Applications with Minimal Effort

Tim Hobson

Nicola Pagni - Anomaly Detection in Elasticsearch

MeetupDataScienceRoma

Bradley Evans SPEDDEXES 2014

aceas13tern

Machine Learning and Analytics Breakout Session

The Critical Missing Component in the Production ML Stack

The day the ML application is deployed to production and begins facing the real world is the best and the worst day in the life of the model builder. The joy of seeing accurate predictions is quickly overshadowed by a myriad of operational challenges. Debugging, troubleshooting & monitoring takes over the majority of their day, leaving little time for model building. In DevOps, software operations are taken to a level of an art. Sophisticated tools enable engineers to quickly identify and resolve issues, continuously improving software stability and robustness. In the ML world, operations are still largely a manual process that involves Jupyter notebooks and shell scripts. One of the cornerstones of the DevOps toolchain is logging. Traces and metrics are built on top of logs enabling monitoring and feedback loops. What does logging look like in an ML system? In this talk we will demonstrate how to enable data logging for an AI application using MLflow in a matter of minutes. We will discuss how something so simple enables testing, monitoring and debugging in an AI application that handles TBs of data and runs in real-time. Attendees will leave the talk equipped with tools and best practices to supercharge MLOps in their team.

Getting Started with Splunk Enterprise Hands-On Breakout Session

Anaconda Data Science Collaboration

Boston Consulting Group

From Volume to Value - A Guide to Data Engineering

Ry Walker

Python + MPP Database = Large Scale AI/ML Projects in Production Faster

Paige_Roberts

ODSC East virtual presentation - The best machine learning, and advanced analytics projects are often stopped when it comes time to move into large scale production, preventing them from ever impacting the business in a meaningful way. Hundreds of hours of work may never get put to use. Python is rapidly becoming the language of choice for scientists and researchers of many types to build, test, train and score models. But when data science models need to go into production, challenges of performance and scale can be a huge roadblock. By combining a Python application with an underlying massively parallel (MPP) database, Python users can achieve a simplified path to production. An MPP database also allows you to do data preparation and data analysis at far greater speeds, accelerating development and testing as well as production performance. It also allows greater numbers of concurrent jobs to run, while also continuously loading data for IoT or other streaming use cases. Analyze data in the database where it sits, rather than first moving it to another framework, then analyzing it, then moving the results, taking multiple performance hits from both CPU and IO for every move and transformation. In this talk, you will learn about combination architectures that can get your work into production, shorten development time, and provide the performance and scale advantages of an MPP database with the convenience and power of Python. Use case examples use the open source Vertica-Python project created by Uber with contributions from Twitter, Palantir, Etsy, Vertica, Kayak and Gooddata.

Machine Learning and the Elastic Stack

Yann Cluchey

Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...

Dataconomy Media

A Statistician Walks into a Tech Company: R at a Rapidly Scaling Healthcare S...

Work-Bench

Fast Data Intelligence in the IoT - real-time data analytics with Spark

Bas Geerdink

Data Science at Scale - The DevOps Approach

Mihai Criveti

Secure360 May 2018 Lessons Learned from OWASP T10 Datacall

Brian Glas

Gain insight into some of the details of the OWASP Top 10 Call for Data and industry survey, and what we were attempting to learn. Hear about was learned from collecting and analyzing widely varying industry data and attempts to build a dataset for comparison and analysis. This session will provide tips and common pitfalls for structuring vulnerability data and the subsequent analysis. Learn what the data can tell us and what questions are still left unanswered. Uncover some of the differences in collecting metrics in different stages of the software lifecycle and recommendations for handling them.

Big Data Berlin v8.0 Stream Processing with Apache Apex

Apache Apex

Data Quality With or Without Apache Spark and Its Ecosystem

Few solutions exist in the open-source community either in the form of libraries or complete stand-alone platforms, which can be used to assure a certain data quality, especially when continuous imports happen. Organisations may consider picking up one of the available options – Apache Griffin, Deequ, DDQ and Great Expectations. In this presentation we’ll compare these different open-source products across different dimensions, like maturity, documentation, extensibility, features like data profiling and anomaly detection.

Building a geospatial processing pipeline using Hadoop and HBase and how Mons...

DataWorks Summit

Monsanto built a geospatial platform on Hadoop and HBase capable of managing over 120 billion polygons. As a result of the extreme data volumes and compute complexities we were forced to migrate our data processing from a more traditional RDBMS to a scale out Hadoop implementation. Data processing that took over 30 days on 8% of the data now runs in under 12 hours on the entire data set. Very little concrete material exist for how you process spatial data via MapReduce or model it in HBase. We will provide concrete and novel examples for processing and storing spatial data on Hadoop and HBase. As part of the data processing pipeline we integrated the popular open source geospatial processing library GDAL with MapReduce to convert all geospatial datasets to a common format and projection. We developed a method for splitting and processing images via MapReduce in which the boundaries of splits needed to be shared by multiple tasks due to the nature of the computation being performed on the data. Bulk writes to HBase were performed by writing HFiles directly. Finally we developed a novel method for storing geospatial data in HBase that met the needs of our access pattern.

Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016

Machine Learning with TensorFlow: TensorFlow has enabled cutting-edge machine learning research at the top AI labs in the world. At the same time it has made the technology accessible to a large audience leading to some amazing uses. TensorFlow is used for classification, recommendation, text parsing, sentiment analysis and more. This talk will go over the design that makes it fast, flexible, and easy to use, and describe how we continue to make it better.

Data Engineering: Elastic, Low-Cost Data Processing in the Cloud

Cloudera, Inc.

What's hot

AdvancedMD Customer Presentation

Open Data Science Conference Agile Data

DataKitchen

Developing Highly Instrumented Applications with Minimal Effort

Tim Hobson

Nicola Pagni - Anomaly Detection in Elasticsearch

MeetupDataScienceRoma

Bradley Evans SPEDDEXES 2014

aceas13tern

Machine Learning and Analytics Breakout Session

The Critical Missing Component in the Production ML Stack

Getting Started with Splunk Enterprise Hands-On Breakout Session

Anaconda Data Science Collaboration

Boston Consulting Group

From Volume to Value - A Guide to Data Engineering

Ry Walker

Python + MPP Database = Large Scale AI/ML Projects in Production Faster

Paige_Roberts

Machine Learning and the Elastic Stack

Yann Cluchey

Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...

Dataconomy Media

A Statistician Walks into a Tech Company: R at a Rapidly Scaling Healthcare S...

Work-Bench

Fast Data Intelligence in the IoT - real-time data analytics with Spark

Bas Geerdink

Data Science at Scale - The DevOps Approach

Mihai Criveti

Secure360 May 2018 Lessons Learned from OWASP T10 Datacall

Brian Glas

Big Data Berlin v8.0 Stream Processing with Apache Apex

Apache Apex

Data Quality With or Without Apache Spark and Its Ecosystem

What's hot (19)

AdvancedMD Customer Presentation

Open Data Science Conference Agile Data

Developing Highly Instrumented Applications with Minimal Effort

Nicola Pagni - Anomaly Detection in Elasticsearch

Bradley Evans SPEDDEXES 2014

Machine Learning and Analytics Breakout Session

The Critical Missing Component in the Production ML Stack

Getting Started with Splunk Enterprise Hands-On Breakout Session

Anaconda Data Science Collaboration

From Volume to Value - A Guide to Data Engineering

Python + MPP Database = Large Scale AI/ML Projects in Production Faster

Machine Learning and the Elastic Stack

Zsolt Várnai, Principal Software Engineer at Skyscanner - "The advantages of...

A Statistician Walks into a Tech Company: R at a Rapidly Scaling Healthcare S...

Fast Data Intelligence in the IoT - real-time data analytics with Spark

Data Science at Scale - The DevOps Approach

Secure360 May 2018 Lessons Learned from OWASP T10 Datacall

Big Data Berlin v8.0 Stream Processing with Apache Apex

Data Quality With or Without Apache Spark and Its Ecosystem

Viewers also liked

Building a geospatial processing pipeline using Hadoop and HBase and how Mons...

DataWorks Summit

Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016

Data Engineering: Elastic, Low-Cost Data Processing in the Cloud

Cloudera, Inc.

Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop

royans

Pig, Making Hadoop Easy

Nick Dimiduk

introduction to data processing using Hadoop and Pig

Ricardo Varela

Practical Problem Solving with Apache Hadoop & PigMilind Bhandarkar

HIVE: Data Warehousing & Analytics on HadoopZheng Shao

Hive Quick Start Tutorial

Carl Steinbach

Integration of Hive and HBase

Hortonworks

Apache Hive provides SQL-like access to your stored data in Apache Hadoop. Apache HBase stores tabular data in Hadoop and supports update operations. The combination of these two capabilities is often desired, however, the current integration show limitations such as performance issues. In this talk, Enis Soztutar will present an overview of Hive and HBase and discuss new updates/improvements from the community on the integration of these two projects. Various techniques used to reduce data exchange and improve efficiency will also be provided.

Hadoop, Pig, and Twitter (NoSQL East 2009)

Kevin Weil

Introduction To Map Reducerantav

Big Data Analytics with Hadoop

Philippe Julio

Viewers also liked (13)

Building a geospatial processing pipeline using Hadoop and HBase and how Mons...

Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016

Data Engineering: Elastic, Low-Cost Data Processing in the Cloud

Facebooks Petabyte Scale Data Warehouse using Hive and Hadoop

Pig, Making Hadoop Easy

introduction to data processing using Hadoop and Pig

Practical Problem Solving with Apache Hadoop & Pig

HIVE: Data Warehousing & Analytics on Hadoop

Hive Quick Start Tutorial

Integration of Hive and HBase

Hadoop, Pig, and Twitter (NoSQL East 2009)

Introduction To Map Reduce

Big Data Analytics with Hadoop

Similar to MLconf NYC Josh Wills

Production machine learning_infrastructure

joshwills

Cloudera User Group - From the Lab to the Factory

ClouderaUserGroups

How and why you need to build a big data lab

Chris Kernaghan

Big Data Rampage

Niko Vuokko

Automating the process of continuously prioritising data, updating and deploy...

Ola Spjuth

Presentation at Data Innovation Summit 2019 in Stockholm, Sweden. ABSTRACT Microscopes are capable of producing vast amounts of data, and when used in automated laboratories both the number and size of images present many challenges for storing, categorizing, analyzing, annotating, and transforming the data into actionable information that can used for decision making; either by humans or machines. In this presentation I will describe the informatics system we have established at the Department of Pharmaceutical Biosciences at Uppsala University, which consists of computational hardware (CPUs, GPUs, storage), middleware (Kubernetes), imaging database (OMERO), and workflow system (Pachyderm) to perform online prioritization of new data, as well as the continuous analytics system to automate the process from captured images to continuously updated and deployed AI models. The AI methodologies include Deep Learning models trained on image data, and conventional machine learning models trained on features extracted from images or chemical structures. Due to the microservice architecture the system is scalable and can be expanded using hybrid-architectures with cloud computing resources. The informatics system serves a robotized cell profiling setup with incubators, liquid handling and high-content microscopy. The lab is quite young and is targeting applications primarily in drug screening and toxicity assessment, with the aim to improve research using AI and intelligent design of experiments.

Turn Data Into Actionable Insights - StampedeCon 2016

StampedeCon

At Monsanto, emerging technologies such as IoT, advanced imaging and geo-spatial platforms; molecular breeding, ancestry and genomics data sets have made us rethink how we approach developing, deploying, scaling and distributing our software to accelerate predictive and prescriptive decisions. We created a Cloud based Data Science platform for the enterprise to address this need. Our primary goals were to perform analytics@scale and integrate analytics with our core product platforms. As part of this talk, we will be sharing our journey of transformation showing how we enabled: a collaborative discovery analytics environment for data science teams to perform model development, provisioning data through APIs, streams and deploying models to production through our auto-scaling big-data compute in the cloud to perform streaming, cognitive, predictive, prescriptive, historical and batch analytics@scale, integrating analytics with our core product platforms to turn data into actionable insights.

PXL Data Engineering Workshop By Selligent

Jonny Daenen

On 2020-12-09 Laurens Vijnck and Jonny Daenen gave a workshop at PXL. During this session, we collectively provisioned a streaming ingestion pipeline in mere minutes. The technology stack included Pub/Sub, Dataflow, and BigQuery. Hereafter, students had the opportunity to perform interactive queries on their own real-time data to answer a series of business questions. These questions were borrowed from real-life cases that we encountered at Selligent Marketing Cloud. Google Colab (Free Jupyter Notebooks) and Google Data Studio have proven to be excellent tools to facilitate these kinds of interactive sessions.

Data Science meets Software Development

Alexis Seigneurin

I work in a Data Innovation Lab with a horde of Data Scientists. Data Scientists gather data, clean data, apply Machine Learning algorithms and produce results, all of that with specialized tools (Dataiku, Scikit-Learn, R...). These processes run on a single machine, on data that is fixed in time, and they have no constraint on execution speed. With my fellow Developers, our goal is to bring these processes to production. Our constraints are very different: we want the code to be versioned, to be tested, to be deployed automatically and to produce logs. We also need it to run in production on distributed architectures (Spark, Hadoop), with fixed versions of languages and frameworks (Scala...), and with data that changes every day. In this talk, I will explain how we, Developers, work hand-in-hand with Data Scientists to shorten the path to running data workflows in production.

Chaos Engineering and How to Manage Data Stages With Adi Polak | Current 2022

HostedbyConfluent

Chaos Engineering and How to Manage Data Stages With Adi Polak | Current 2022 A complex data flow is a set of operations to extract information from multiple sources, copy them into multiple data targets while using extract, transformations, joins, filters, and sorts to refine the results. These are precisely the capabilities that the new open modern data stack provides us. Spark and other tools allow us to develop complex data flow on large-scale data. Chaos Engineering concepts discuss the principles of experimenting on a distributed system to build confidence in the system’s capability to withstand turbulent conditions in production. Or, how stable is your distributed system? Let's see how we combine these two worlds to build more stability and reliability into our dataops.

Customer Presentation

A Maturing Role of Workflows in the Presence of Heterogenous Computing Archit...

Ilkay Altintas, Ph.D.

cientific workflows are used by many scientific communities to capture, automate and standardize computational and data practices in science. Workflow-based automation is often achieved through a craft that combines people, process, computational and Big Data platforms, application-specific purpose and programmability, leading to provenance-aware archival and publications of the results. This talk summarizes varying and changing requirements for distributed workflows influenced by Big Data and heterogeneous computing architectures and present a methodology for workflow-driven science based on these maturing requirements.

Searching The Enterprise Data Lake With Solr - Watch Us Do It!: Presented by...

Lucidworks

Dev nexus 2017

Roy Russo

Real-time Operational Intelligence for machine data

jKool

Real time monitoring of hadoop and spark workflows

Shankar Manian

You've Got No UI?! (Agile Data Teams)

Mark Barber

It is often not intuitive to use agile and lean startup thinking in the world of big data and analytics. You can feel far removed from users and customers and your products often have to enable people to answer open-ended questions to help solve complex business problems. Often you are the engine room of very large enterprises. We will discuss techniques and practices that we can use to stay agile under these conditions. Please note, this talk focuses more on ways of working than technical aspects. Target Audience: People in data-centric teams (warehousing, BI, analytics) or anyone with an interest in agile principles being applied.

Defcon 22-wesley-mc grew-instrumenting-point-of-sale-malware

DaveEdwards12

Options for Data Prep - A Survey of the Current Market

Dremio Corporation

Data comes in many shapes and sizes, and every company struggles to find ways to transform, validate, and enrich data for multiple purposes. The problem has been around as long as data, and the market has an overwhelming number of options. In this presentation we look at the problem and key options from vendors in the market today. Dremio is a new approach that eliminates the need for stand alone data prep tools.

Using Machine Learning to Understand Kafka Runtime Behavior (Shivanath Babu, ...

confluent

Apache Kafka is now nearly ubiquitous in modern data pipelines and use cases. While the Kafka development model is elegantly simple, operating Kafka clusters in production environments is a challenge. It’s hard to troubleshoot misbehaving Kafka clusters, especially when there are potentially hundreds or thousands of topics, producers and consumers and billions of messages. The root cause of why real-time applications is lag may be due to an application problem – like poor data partitioning or load imbalance – or due to a Kafka problem – like resource exhaustion or suboptimal configuration. Therefore getting the best performance, predictability, and reliability for Kafka-based applications can be difficult. In the end, the operation of your Kafka powered analytics pipelines could themselves benefit from machine learning (ML).

Efficient & effective data management for research projects : ILRI's Data Ma...

CIARD Movement

Similar to MLconf NYC Josh Wills (20)

Production machine learning_infrastructure

Cloudera User Group - From the Lab to the Factory

How and why you need to build a big data lab

Big Data Rampage

Automating the process of continuously prioritising data, updating and deploy...

Turn Data Into Actionable Insights - StampedeCon 2016

PXL Data Engineering Workshop By Selligent

Data Science meets Software Development

Chaos Engineering and How to Manage Data Stages With Adi Polak | Current 2022

Customer Presentation

A Maturing Role of Workflows in the Presence of Heterogenous Computing Archit...

Searching The Enterprise Data Lake With Solr - Watch Us Do It!: Presented by...

Dev nexus 2017

Real-time Operational Intelligence for machine data

Real time monitoring of hadoop and spark workflows

You've Got No UI?! (Agile Data Teams)

Defcon 22-wesley-mc grew-instrumenting-point-of-sale-malware

Options for Data Prep - A Survey of the Current Market

Using Machine Learning to Understand Kafka Runtime Behavior (Shivanath Babu, ...

Efficient & effective data management for research projects : ILRI's Data Ma...

More from MLconf

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Understanding Human Impact: Social and Equity Assessments for AI Technologies Social and Equity Impact Assessments have broad applications but can be a useful tool to explore and mitigate for Machine Learning fairness issues and can be applied to product specific questions as a way to generate insights and learnings about users, as well as impacts on society broadly as a result of the deployment of new and emerging technologies. In this presentation, my goal is to advocate for and highlight the need to consult community and external stakeholder engagement to develop a new knowledge base and understanding of the human and social consequences of algorithmic decision making and to introduce principles, methods and process for these types of impact assessments.

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

The Brain’s Guide to Dealing with Context in Language Understanding Like the visual cortex, the regions of the brain involved in understanding language represent information hierarchically. But whereas the visual cortex organizes things into a spatial hierarchy, the language regions encode information into a hierarchy of timescale. This organization is key to our uniquely human ability to integrate semantic information across narratives. More and more, deep learning-based approaches to natural language understanding embrace models that incorporate contextual information at varying timescales. This has not only led to state-of-the art performance on many difficult natural language tasks, but also to breakthroughs in our understanding of brain activity. In this talk, we will discuss the important connection between language understanding and context at different timescales. We will explore how different deep learning architectures capture timescales in language and how closely their encodings mimic the brain. Along the way, we will uncover some surprising discoveries about what depth does and doesn’t buy you in deep recurrent neural networks. And we’ll describe a new, more flexible way to think about these architectures and ease design space exploration. Finally, we’ll discuss some of the exciting applications made possible by these breakthroughs.

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Applying Computer Vision to Reduce Contamination in the Recycling Stream With China’s recent refusal of most foreign recyclables, North American waste haulers are scrambling to figure out how to make on-shore recycling cost-effective in order to continue providing recycling services. Recyclables that were once being shipped to China for manual sorting are now primarily being redirected to landfills or incinerators. Without a solution, a nearly $5 billion annual recycling market could come to a halt. Purity in the recycling stream is key to this effort as contaminants in the stream can increase the cost of operations, damage equipment and reduce the ability to create pure commodities suitable for creating recycled goods. This market disruption as a result of China’s new regulations, however, provides us the chance to re-examine and improve our current disposal & collection habits with modern monitoring & artificial intelligence technology. Using images from our in-dumpster cameras, Compology has developed an ML-based process that helps identify, measure and alert for contaminants in recycling containers before they are picked-up, helping keep the recycling stream clean. Our convolutional neural network flags potential instances of contamination inside a dumpster, enabling garbage haulers to know which containers have the wrong type of material inside. This allows them to provide targeted, timely education, and when appropriate, assess fines, to improve recycling compliance at the businesses and residences they serve, helping keep recycling services financially viable. In this presentation, we will walk through our ML-based contamination measurement and scoring process by showing how Waste Management, a national waste hauler, has experienced 57% contamination reduction in nearly 2,000 containers over six months, This progress shows significant strides towards financially viable recycling services.

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Quantum Computing: a Treasure Hunt, not a Gold Rush Quantum computers promise a significant step up in computational power over conventional computers, but also suffer a number of counterintuitive limitations --- both in their computational model and in leading lab implementations. In this talk, we review how quantum computers compete with conventional computers and how conventional computers try to hold their ground. Then we outline what stands in the way of successful quantum ML applications.

Josh Wills - Data Labeling as Religious Experience

Data Labeling as Religious Experience One of the most common places to deploy a production machine learning systems is as a replacement for a legacy rules-based system that is having a hard time keeping up with new edge cases and requirements. I'll be walking through the process and tooling we used to help us design, train, and deploy a model to replace a set of static rules we had for handling invite spam at Slack, talk about what we learned, and discuss some problems to solve in order to make these migrations easier for everyone.

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Project GaitNet: Ushering in the ImageNet moment for human Gait kinematics The emergence of the upright human bipedal gait can be traced back 4 to 2.8 million years ago, to the now extinct hominin Australopithecus afarensis. Fine grained analysis of gait using the modern MEMS sensors found on all smartphones not just reveals a lot about the person’s orthopedic and neuromuscular health status, but also has enough idiosyncratic clues that it can be harnessed as a passive biometric. While there were many siloed attempts made by the machine learning community to model Bipedal Gait sensor data, these were done with small datasets oft collected in restricted academic environs. In this talk, we will introduce the ImageNet moment for human gait analysis by presenting 'Project GaitNet', the largest ever planet-sized motion sensor based human bipedal gait dataset ever curated. We’ll also present the associated state-of-the-art results in classifying humans harnessing novel deep neural architectures and the related success stories we have enjoyed in transfer-learning into disparate domains of human kinematics analysis.

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Machine Learning Methods in Detecting Alzheimer’s Disease from Speech and Language Alzheimer's disease affects millions of people worldwide, and it is important to predict the disease as early and as accurate as possible. In this talk, I will discuss development of novel ML models that help classifying healthy people from those who develop Alzheimer's, using short samples of human speech. As an input to the model, features of different modalities are extracted from speech audio samples and transcriptions: (1) syntactic measures, such as e.g. production rules extracted from syntactic parse trees, (2) lexical measures, such as e.g. features of lexical richness and complexity and lexical norms, and (3) acoustic measures, such as e.g. standard Mel-frequency cepstral coefficients. I will present the ML model that detects cognitive impairment by reaching agreement among modalities. The resulting model is able to achieve state of the art performance in both supervised and semi-supervised manner, using manual transcripts of human speech. Additionally, I will discuss potential limitations of any fully-automated speech-based Alzheimer's disease detection model, focusing mostly on the analysis of the impact of a not-so-accurate automatic speech recognition (ASR) on the classification performance. To illustrate this, I will present the experiments with controlled amounts of artificially generated ASR errors and explain how the deletion errors affect Alzheimer's detection performance the most, due to their impact on the features of syntactic and lexical complexity.

Meghana Ravikumar - Optimized Image Classification on the Cheap

Optimized Image Classification on the Cheap In this talk, we anchor on building an image classifier trained on the Stanford Cars dataset to evaluate two approaches to transfer learning -fine tuning and feature extraction- and the impact of hyperparameter optimization on these techniques. Once we define the most performant transfer learning technique for Stanford Cars, we will double the size of the dataset through image augmentation to boost the classifier’s performance. We will use Bayesian optimization to learn the hyperparameters associated with image transformations using the downstream image classifier’s performance as the guide. In conjunction with model performance, we will also focus on the features of these augmented images and the downstream implications for our image classifier. To both maximize model performance on a budget and explore the impact of optimization on these methods, we apply a particularly efficient implementation of Bayesian optimization to each of these architectures in this comparison. Our goal is to draw on a rigorous set of experimental results that can help us answer the question: how can resource-constrained teams make trade-offs between efficiency and effectiveness using pre-trained models?

Noam Finkelstein - The Importance of Modeling Data Collection

The Importance of Modeling Data Collection Data sets used in machine learning are often collected in a systematically biased way - certain data points are more likely to be collected than others. We call this "observation bias". For example, in health care, we are more likely to see lab tests when the patient is feeling unwell than otherwise. Failing to account for observation bias can, of course, result in poor predictions on new data. By contrast, properly accounting for this bias allows us to make better use of the data we do have. In this presentation, we discuss practical and theoretical approaches to dealing with observation bias. When the nature of the bias is known, there are simple adjustments we can make to nonparametric function estimation techniques, such as Gaussian Process models. We also discuss the scenario where the data collection model is unknown. In this case, there are steps we can take to estimate it from observed data. Finally, we demonstrate that having a small subset of data points that are known to be collected at random - that is, in an unbiased way - can vastly improve our ability to account for observation bias in the rest of the data set. My hope is that attendees of this presentation will be aware of the perils of observation bias in their own work, and be equipped with tools to address it.

June Andrews - The Uncanny Valley of ML

The Uncanny Valley of ML Every so often, the conundrum of the Uncanny Valley re-emerges as advanced technologies evolve from clearly experimental products to refined accepted technologies. We have seen its effects in robotics, computer graphics, and page load times. The debate of how to handle the new technology detracts from its benefits. When machine learning is added to human decision systems a similar effect can be measured in increased response time and decreased accuracy. These systems include radiology, judicial assignments, bus schedules, housing prices, power grids and a growing variety of applications. Unfortunately, the Uncanny Valley of ML can be hard to detect in these systems and can lead to degraded system performance when ML is introduced, at great expense. Here, we'll introduce key design principles for introducing ML into human decision systems to navigate around the Uncanny Valley and avoid its pitfalls.

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Deep Learning Architectures for Semantic Relation Detection Tasks Recognizing and distinguishing specific semantic relations from other types of semantic relations is an essential part of language understanding systems. Identifying expressions with similar and contrasting meanings is valuable for NLP systems which go beyond recognizing semantic relatedness and require to identify specific semantic relations. In this talk, I will first present novel techniques for creating labelled datasets required for training deep learning models for classifying semantic relations between phrases. I will further present various neural network architectures that integrate morphological features into integrated path-based and distributional relation detection algorithms and demonstrate that this model outperforms state-of-the-art models in distinguishing semantic relations and is capable of efficiently handling multi-word expressions.

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Building an Incrementally Trained, Local Taste Aware, Global Deep Learned Recommender System Model At Netflix, our main goal is to maximize our members’ enjoyment of the selected show by minimizing the amount of time it takes for them to find it. We try to achieve this goal by personalizing almost all the aspects of our product -- from what shows to recommend, to how to present these shows and construct their home-pages to what images to select per show, among many other things. Everything is recommendations for us and as an applied Machine Learning group, we spend our time building models for personalization that will eventually increase the joy and satisfaction of our members. In this talk we will primarily focus our attention on a) making a global deep learned recommender model that is regional tastes and popularity aware and b) adapting this model to changing taste preferences as well as dynamic catalog availability. We will first go through some standard recommender system models that use Matrix Factorization and Topic Models and then compare and contrast them with more powerful and higher capacity deep learning based models such as sequence models that use recurrent neural networks. We will show what it entails to build a global model that is aware of regional taste preferences and catalog availability. We will show how models that are built on simple Maximum Likelihood principle fail to do that. We will then describe one solution that we have employed in order to enable the global deep learned models to focus their attention on capturing regional taste preferences and changing catalog.In the latter half of the talk, we will discuss how we do incremental learning of deep learned recommender system models. Why do we need to do that ? Everything changes with time. Users’ tastes change with time. What’s available on Netflix and what’s popular also change over time. Therefore, updating or improving recommendation systems over time is necessary to bring more joy to users. In addition to how we apply incremental learning, we will discuss some of the challenges we face involving large-scale data preparation, infrastructure setup for incremental model training as well as pipeline scheduling. The incremental training enables us to serve fresher models trained on fresher and larger amounts of data. This helps our recommender system to nicely and quickly adapt to catalog and users’ taste changes, and improve overall performance.

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World The adoption of voice-enabled devices has seen an explosive growth in the last few years and music consumption is among the most popular use cases. Music personalization and recommendation plays a major role at Pandora in providing a daily delightful listening experience for millions of users. In turn, providing the same perfectly tailored listening experience through these novel voice interfaces brings new interesting challenges and exciting opportunities. In this talk we will describe how we apply personalization and recommendation techniques in three common voice scenarios which can be defined in terms of request types: known-item, thematic, and broad open-ended. We will describe how we use deep learning slot filling techniques and query classification to interpret the user intent and identify the main concepts in the query. We will also present the differences and challenges regarding evaluation of voice powered recommendation systems. Since pure voice interfaces do not contain visual UI elements, relevance labels need to be inferred through implicit actions such as play time, query reformulations or other types of session level information. Another difference is that while the typical recommendation task corresponds to recommending a ranked list of items, a voice play request translates into a single item play action. Thus, some considerations about closed feedback loops need to be made. In summary, improving the quality of voice interactions in music services is a relatively new challenge and many exciting opportunities for breakthroughs still remain. There are many new aspects of recommendation system interfaces to address to bring a delightful and effortless experience for voice users. We will share a few open challenges to solve for the future.

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Neel Sundaresan - Teaching a machine to code

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Soumith Chintala - Increasing the Impact of AI Through Better Software

Roy Lowrance - Predicting Bond Prices: Regime Changes