Josh Wills, MLconf 2013

•

0 likes•2,230 views

Josh Wills, Senior Director of Data Science, Cloudera: Building a Production Machine Learning Infrastructure (Quickly)

Technology Education

From
The
Lab
to
the
Factory

Building
A
Produc8on
Machine
Learning
Infrastructure

Josh
Wills,
Senior
Director
of
Data
Science

Cloudera

1

About
Me

2

Data
Science:
Another
Deﬁni8on

3

Data
Scien8sts
Build
Data
Products.

4

All*
Products
Become
Data
Products

5

Iden8fying
the
BoHlenecks

6

Oryx:
Model
Building
and
Serving

• 

Algorithms

• 
• 
• 

• 
• 
• 
7

ALS
Recommenders

K-‐Means
Parallel

RDF

Batch
model
building

via
MapReduce

Server
for
real-‐8me

scoring
and
updates

PMML
4.1
Models

Gertrude:
Evalua8on
via
Experiments

• 

Mul8variate
Tes8ng

• 

• 

Overlapping

Experiments

• 
• 

8

Deﬁne
and
explore
a

space
of
parameters

Tang
et
al.
(2010)

Runs
mul8ple

independent

experiments
on
every

request

Planning
For
The
Future

9

Thank
you!

Josh
Wills,
Director
of
Data
Science,
Cloudera

@josh_wills

More Related Content

What's hot

Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014

Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014

Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Ilkay Altintas: Kepler

Ilkay Altintas: Kepler

Ilkay Altintas: Kepler

This talk was given at H2O World 2018 NYC and can be viewed here: https://youtu.be/n9g9GxIJoT4 Description: The goal of the research was to develop an approach to predict individual medical test results based on longitudinal medical and pharma claims data without direct lab measures using data-driven techniques. Such discoveries may result in improved treatment strategies. In the presentation we demonstrate how Driverless AI was used both for estimating highly accurate model and results explanations. Speaker's Bio: Alexander is the Data Science leader at poder.IO. He is responsible for data flow architecture and insight mining, all powered by machine learning. Before joining poder.IO, Alexander made an academic career at Belarusian State University and Minsk Innovation University where he was Head of the Informatics and Mathematics Department.

Predicting Medical Test Results using Driverless AI

Predicting Medical Test Results using Driverless AI

Predicting Medical Test Results using Driverless AI

eScience: A Transformed Scientific Method

eScience: A Transformed Scientific Method

eScience: A Transformed Scientific Method

This presentation of the Endpoint 2015 conference gives an overview of a short data science project: predicting the future happiness of a person, as if he or she walks into a circus tent! First, the domain problem is analyzed. Then, the data is gathered and analyzed. Finally a linear regression model is created and the app is published in the form of a REST API. The technology that is demoed is using Apache Spark and Zeppelin, and can be found on Github: https://github.com/geerdink/FortuneTellerApi

Fortune Teller API - Doing Data Science with Apache Spark

Fortune Teller API - Doing Data Science with Apache Spark

Fortune Teller API - Doing Data Science with Apache Spark

An overview of Tensorflow, and then we'll walk through how to utilize this library within the H2O platform. Tensorflow is an open source, deep learning framework utilized by Google and Deepmind. #h2ony - Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai - To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata

H2O & Tensorflow - Fabrizio

H2O & Tensorflow - Fabrizio

H2O & Tensorflow - Fabrizio

With data as a valuable currency and the architecture of reliable, scalable Data Lakes and Lakehouses continuing to mature, it is crucial that machine learning training and deployment techniques keep up to realize value. Reproducibility, efficiency, and governance in training and production environments rest on the shoulders of both point in time snapshots of the data and a governing mechanism to regulate, track, and make best use of associated metadata. This talk will outline the challenges and importance of building and maintaining reproducible, efficient, and governed machine learning solutions as well as posing solutions built on open source technologies – namely Delta Lake for data versioning and MLflow for efficiency and governance.

Importance of ML Reproducibility & Applications with MLfLow

Importance of ML Reproducibility & Applications with MLfLow

Importance of ML Reproducibility & Applications with MLfLow

Dataiku - for Data Geek Paris@Criteo - Close the Data Circle

Dataiku - for Data Geek Paris@Criteo - Close the Data Circle

Dataiku - for Data Geek Paris@Criteo - Close the Data Circle

H2O Machine Learning and Kalman Filters for Machine Prognostics - Galvanize SF

H2O Machine Learning and Kalman Filters for Machine Prognostics - Galvanize SF

H2O Machine Learning and Kalman Filters for Machine Prognostics - Galvanize SF

MLconf NYC Josh Wills

MLconf NYC Josh Wills

MLconf NYC Josh Wills

Analyzing Data With Python

Analyzing Data With Python

Analyzing Data With Python

Using H2O AutoML for Kaggle Competitions

Using H2O AutoML for Kaggle Competitions

Using H2O AutoML for Kaggle Competitions

H2O World - Data Science w/ Big Data in a Corporate Environment - Nachum Shacham

H2O World - Data Science w/ Big Data in a Corporate Environment - Nachum Shacham

H2O World - Data Science w/ Big Data in a Corporate Environment - Nachum Shacham

As data science workloads grow, so does their need for infrastructure. But, is it fair to ask data scientists to also become infrastructure experts? If not the data scientists, then, who is responsible for spinning up and managing data science infrastructure? This talk will address the context in which ML infrastructure is emerging, walk through two examples of ML infrastructure tools for launching hyperparameter optimization jobs, and end with some thoughts for building better tools in the future. Originally given as a talk at the PyData Ann Arbor meetup (https://www.meetup.com/PyData-Ann-Arbor/events/260380989/)

Machine Learning Infrastructure

Machine Learning Infrastructure

Machine Learning Infrastructure

Agile data science with scala

Agile data science with scala

Agile data science with scala

This presentation describes our work concerning the definition of a neutral, abstract ontology and framework that supports the vision and diverse contexts of a smart community. This framework is composed of a general, core ontology that supports what many are calling the Internet of Things, a scalable number of extension ontologies to describe various application perspectives, and a mapping methodology to relate external data and/or schemas to our ontology. Finally, we show why this ontology is robust, scalable and generic enough to support a wide range of smart devices, systems and people.

An Ontology-Driven Integration Framework for Smart Communities

An Ontology-Driven Integration Framework for Smart Communities

An Ontology-Driven Integration Framework for Smart Communities

Estamos presenciando inovações tecnológicas que possibilitam utilizar ciência dos dados sem a necessidade de antecipar grandes investimentos. Este contexto facilita a adoção de práticas e valores ágeis que encorajam a antecipação de insights e aprendizado contínuo. Nesta palestra, iremos abordar temas como times multi-funcionais, práticas ágeis de engenharia de software e desenvolvimento iterativo, incremental e colaborativo no contexto de produtos e soluções de ciência dos dados.

Agile Data Science

Agile Data Science

Agile Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

The data science process seeks to transform and empower organizations by finding and exploiting market inefficiencies and potentially hidden opportunities, but this is often an expensive, tedious process. However, many steps can be automated to provide a streamlined experience for data scientists. Eduardo Arino de la Rubia explores the tools being created by the open source community to free data scientists from tedium, enabling them to work on the high-value aspects of insight creation and impact validation. The promise of the automated statistician is almost as old as statistics itself. From the creations of vast tables, which saved the labor of calculation, to modern tools which automatically mine datasets for correlations, there has been a considerable amount of advancement in this field. Eduardo compares and contrasts a number of open source tools, including TPOT and auto-sklearn for automated model generation and scikit-feature for feature generation and other aspects of the data science workflow, evaluates their results, and discusses their place in the modern data science workflow. Along the way, Eduardo outlines the pitfalls of automated data science and applications of the “no free lunch” theorem and dives into alternate approaches, such as end-to-end deep learning, which seek to leverage massive-scale computing and architectures to handle automatic generation of features and advanced models.

Leveraging Open Source Automated Data Science Tools

Leveraging Open Source Automated Data Science Tools

Leveraging Open Source Automated Data Science Tools

Domino Data Lab

What's hot (20)

Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014

Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014

Agile Data Science by Russell Jurney_ The Hive_Janruary 29 2014

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Seeing at the Speed of Thought: Empowering Others Through Data Exploration

Ilkay Altintas: Kepler

Ilkay Altintas: Kepler

Ilkay Altintas: Kepler

Predicting Medical Test Results using Driverless AI

Predicting Medical Test Results using Driverless AI

Predicting Medical Test Results using Driverless AI

eScience: A Transformed Scientific Method

eScience: A Transformed Scientific Method

eScience: A Transformed Scientific Method

Fortune Teller API - Doing Data Science with Apache Spark

Fortune Teller API - Doing Data Science with Apache Spark

Fortune Teller API - Doing Data Science with Apache Spark

H2O & Tensorflow - Fabrizio

H2O & Tensorflow - Fabrizio

H2O & Tensorflow - Fabrizio

Importance of ML Reproducibility & Applications with MLfLow

Importance of ML Reproducibility & Applications with MLfLow

Importance of ML Reproducibility & Applications with MLfLow

Dataiku - for Data Geek Paris@Criteo - Close the Data Circle

Dataiku - for Data Geek Paris@Criteo - Close the Data Circle

Dataiku - for Data Geek Paris@Criteo - Close the Data Circle

H2O Machine Learning and Kalman Filters for Machine Prognostics - Galvanize SF

H2O Machine Learning and Kalman Filters for Machine Prognostics - Galvanize SF

H2O Machine Learning and Kalman Filters for Machine Prognostics - Galvanize SF

MLconf NYC Josh Wills

MLconf NYC Josh Wills

MLconf NYC Josh Wills

Analyzing Data With Python

Analyzing Data With Python

Analyzing Data With Python

Using H2O AutoML for Kaggle Competitions

Using H2O AutoML for Kaggle Competitions

Using H2O AutoML for Kaggle Competitions

H2O World - Data Science w/ Big Data in a Corporate Environment - Nachum Shacham

H2O World - Data Science w/ Big Data in a Corporate Environment - Nachum Shacham

H2O World - Data Science w/ Big Data in a Corporate Environment - Nachum Shacham

Machine Learning Infrastructure

Machine Learning Infrastructure

Machine Learning Infrastructure

Agile data science with scala

Agile data science with scala

Agile data science with scala

An Ontology-Driven Integration Framework for Smart Communities

An Ontology-Driven Integration Framework for Smart Communities

An Ontology-Driven Integration Framework for Smart Communities

Agile Data Science

Agile Data Science

Agile Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Leveraging Open Source Automated Data Science Tools

Leveraging Open Source Automated Data Science Tools

Leveraging Open Source Automated Data Science Tools

Viewers also liked

In this talk you will discover various state-of-the-art open-source distributed streaming frameworks, their similarities and differences, implementation trade-offs, their intended use-cases, and how to choose between them. Petr will focus on the popular frameworks, including Spark Streaming, Storm, Samza and Flink. You will also explore theoretical introduction, common pitfalls, popular architectures, and much more. The demand for stream processing is increasing. Immense amounts of data has to be processed fast from a rapidly growing set of disparate data sources. This pushes the limits of traditional data processing infrastructures. These stream-based applications, include trading, social networks, the Internet of Things, and system monitoring, are becoming more and more important. A number of powerful, easy-to-use open source platforms have emerged to address this. Petr's goal is to provide a comprehensive overview of modern streaming solutions and to help fellow developers with picking the best possible solution for their particular use-case. Join this talk if you are thinking about, implementing, or have already deployed a streaming solution.

Distributed real time stream processing- why and how

Distributed real time stream processing- why and how

Distributed real time stream processing- why and how

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...

Kafka Reliability - When it absolutely, positively has to be there

Kafka Reliability - When it absolutely, positively has to be there

Kafka Reliability - When it absolutely, positively has to be there

Gwen (Chen) Shapira

This session explores best practices of creating both unit and integration tests for Spark programs as well as acceptance tests for the data produced by our Spark jobs. We will explore the difficulties with testing streaming programs, options for setting up integration testing with Spark, and also examine best practices for acceptance tests. Unit testing of Spark programs is deceptively simple. The talk will look at how unit testing of Spark itself is accomplished, as well as factor out a number of best practices into traits we can use. This includes dealing with local mode cluster creation and teardown during test suites, factoring our functions to increase testability, mock data for RDDs, and mock data for Spark SQL. Testing Spark Streaming programs has a number of interesting problems. These include handling of starting and stopping the Streaming context, and providing mock data and collecting results. As with the unit testing of Spark programs, we will factor out the common components of the tests that are useful into a trait that people can use. While acceptance tests are not always part of testing, they share a number of similarities. We will look at which counters Spark programs generate that we can use for creating acceptance tests, best practices for storing historic values, and some common counters we can easily use to track the success of our job. Relevant Spark Packages & Code: https://github.com/holdenk/spark-testing-base / http://spark-packages.org/package/holdenk/spark-testing-base https://github.com/holdenk/spark-validator

Effective testing for spark programs Strata NY 2015

Effective testing for spark programs Strata NY 2015

Effective testing for spark programs Strata NY 2015

Organizations need to perform increasingly complex analysis on their data — streaming analytics, ad-hoc querying and predictive analytics — in order to get better customer insights and actionable business intelligence. However, the growing data volume, speed, and complexity of diverse data formats make current tools inadequate or difficult to use. Apache Spark has recently emerged as the framework of choice to address these challenges. Spark is a general-purpose processing framework that follows a DAG model and also provides high-level APIs, making it more flexible and easier to use than MapReduce. Thanks to its use of in-memory datasets (RDDs), embedded libraries, fault-tolerance, and support for a variety of programming languages, Apache Spark enables developers to implement and scale far more complex big data use cases, including real-time data processing, interactive querying, graph computations and predictive analytics. In this session, we present a technical deep dive on Spark running on Amazon EMR. You learn why Spark is great for ad-hoc interactive analysis and real-time stream processing, how to deploy and tune scalable clusters running Spark on Amazon EMR, how to use EMRFS with Spark to query data directly in Amazon S3, and best practices and patterns for Spark on Amazon EMR.

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

Amazon Web Services

10 more lessons learned from building Machine Learning systems

10 more lessons learned from building Machine Learning systems

10 more lessons learned from building Machine Learning systems

Xavier Amatriain

10 more lessons learned from building Machine Learning systems - MLConf

10 more lessons learned from building Machine Learning systems - MLConf

10 more lessons learned from building Machine Learning systems - MLConf

Xavier Amatriain

Viewers also liked (7)

Distributed real time stream processing- why and how

Distributed real time stream processing- why and how

Distributed real time stream processing- why and how

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...

Strata NYC 2015: Sketching Big Data with Spark: randomized algorithms for lar...

Kafka Reliability - When it absolutely, positively has to be there

Kafka Reliability - When it absolutely, positively has to be there

Kafka Reliability - When it absolutely, positively has to be there

Effective testing for spark programs Strata NY 2015

Effective testing for spark programs Strata NY 2015

Effective testing for spark programs Strata NY 2015

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

(BDT309) Data Science & Best Practices for Apache Spark on Amazon EMR

10 more lessons learned from building Machine Learning systems

10 more lessons learned from building Machine Learning systems

10 more lessons learned from building Machine Learning systems

10 more lessons learned from building Machine Learning systems - MLConf

10 more lessons learned from building Machine Learning systems - MLConf

10 more lessons learned from building Machine Learning systems - MLConf

Similar to Josh Wills, MLconf 2013

Cloudera User Group - From the Lab to the Factory

Cloudera User Group - From the Lab to the Factory

Cloudera User Group - From the Lab to the Factory

ClouderaUserGroups

Making Data Science Scalable - 5 Lessons Learned Making Data Science and Machine Learning scalable is not easy: #1 Data Science in silos is bad #2 ML-Feature stores should be at the heart of every ML-Platform #3 Auto ML works great if you have a Feature store #4 Treat Data Science Projekts more like Software Development #5 Cloude based Infrastructure makes it easy to get started Data Science MeetUp Cologne, Germany 16. May 2019 datasolut GmbH - https://datasolut.com

Making Data Science Scalable - 5 Lessons Learned

Making Data Science Scalable - 5 Lessons Learned

Making Data Science Scalable - 5 Lessons Learned

At Lennox International, we have thousands of IoT connected devices streaming data into the Azure platform with a minute level polling interval. The challenge was to use these data sets, combine with external data sources such as weather, and predict equipment failure with high levels of accuracy along with their influencing patterns and parameters. Previously the team was using a combination of on-premise and desktop tools to run algorithms on a sample set of devices. The result was low accuracy levels (around 65%) on a process that took more than 6 hours. The team had to work through several data orchestration challenges and identify a machine learning platform which enabled them to collaborate between our engineering SME’s, Data Engineers and Data Scientists. The team decided to use Azure Databricks to build the data engineering pipelines, appropriate machine learning models and extract predictions using PySpark. To enhance the sophistication of the learning, the team worked on a variety of Spark ML models such as Gradient Boosted Trees and Random Forest. The team also implemented stacking, ensemble methods using H2O driverless AI and sparkling water on Azure Databricks clusters, which can scale up to 1000 cores. Join us in this session and see how this resulted in models that run in 40 minutes with minimal tuning and predict failures with accuracy of about 90%.

How Azure Databricks helped make IoT Analytics a Reality with Janath Manohara...

How Azure Databricks helped make IoT Analytics a Reality with Janath Manohara...

How Azure Databricks helped make IoT Analytics a Reality with Janath Manohara...

Machine learning drove massive growth at consumer internet companies over the last decade, and this was enabled by open software, datasets, and AI research. For many problems, machine learning will produce better, faster, and more repeatable decisions at scale. Unfortunately, building and maintaining these systems is still extremely difficult and expensive. As more machine learning software moves to production, many of our traditional tools and best practices in software development will change. Pete Skomoroch walks you through what you need to know as we shift from a world of deterministic programs to systems that give unpredictable results on ever-changing training data. To navigate this world powered by nondeterministic data-dependent programs, we’ll also need a new development stack to help us write, test, deploy, and monitor machine learning software. Presented at OSCON Portland July 18, 2019

Managing Machines: The New AI Dev Stack

Managing Machines: The New AI Dev Stack

Managing Machines: The New AI Dev Stack

Peter Skomoroch

How and why you need to build a big data lab

How and why you need to build a big data lab

How and why you need to build a big data lab

Chris Kernaghan

Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink

Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink

Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink

This talk discusses integrating common data science tools like Python pandas, scikit-learn, and R with MLlib, Spark’s distributed Machine Learning (ML) library. Integration is simple; migration to distributed ML can be done lazily; and scaling to big data can significantly improve accuracy. We demonstrate integration with a simple data science workflow. Data scientists often encounter scaling bottlenecks with single-machine ML tools. Yet the overhead in migrating to a distributed workflow can seem daunting. In this talk, we demonstrate such a migration, taking advantage of Spark and MLlib’s integration with common ML libraries. We begin with a small dataset which runs on a single machine. Increasing the size, we hit bottlenecks in various parts of the workflow: hyperparameter tuning, then ETL, and eventually the core learning algorithm. As we hit each bottleneck, we parallelize that part of the workflow using Spark and MLlib. As we increase the dataset and model size, we can see significant gains in accuracy. We end with results demonstrating the impressive scalability of MLlib algorithms. With accuracy comparable to traditional ML libraries, combined with state-of-the-art distributed scalability, MLlib is a valuable new tool for the modern data scientist.

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Are You Ready? Stepping Up To The Big Data Challenge In 2016 - Learn why Testing is pivotal to the success of your Big Data Strategy. According to a new report by analyst firm IDG, 70% of enterprises have either deployed or are planning to deploy big data projects and programs this year due to the increase in the amount of data they need to manage. The growing variety of new data sources is pushing organizations to look for streamlined ways to manage complexities and get the most out of their data-related investments. The companies that do this correctly are realizing the power of big data for business expansion and growth. Learn why testing your enterprise's data is pivotal for success with big data and Hadoop. Learn how to increase your testing speed, boost your testing coverage (up to 100%), and improve the level of quality within your data - all with one data testing tool.

Testing Big Data: Automated Testing of Hadoop with QuerySurge

Testing Big Data: Automated Testing of Hadoop with QuerySurge

Testing Big Data: Automated Testing of Hadoop with QuerySurge

This session is about learning how to use Microsoft Azure Machine Learning with the devices in order to detect data patterns. This session will cover an introduction to Machine Learning, and different algorithms used to detect data patterns. The algorithms discussed will be nearest neighbor, probabilistic learning, decision trees, and neural networks. It will also cover data that comes from devices like the Kinect for Windows device. The session will show basic demos and data coming from the device. The session will then drill down into how to incorporate Azure Machine Learning features into an application to detect data patterns in real time.

Using Azure Machine Learning to Detect Patterns in Data from Devices

Using Azure Machine Learning to Detect Patterns in Data from Devices

Using Azure Machine Learning to Detect Patterns in Data from Devices

At Monsanto, emerging technologies such as IoT, advanced imaging and geo-spatial platforms; molecular breeding, ancestry and genomics data sets have made us rethink how we approach developing, deploying, scaling and distributing our software to accelerate predictive and prescriptive decisions. We created a Cloud based Data Science platform for the enterprise to address this need. Our primary goals were to perform analytics@scale and integrate analytics with our core product platforms. As part of this talk, we will be sharing our journey of transformation showing how we enabled: a collaborative discovery analytics environment for data science teams to perform model development, provisioning data through APIs, streams and deploying models to production through our auto-scaling big-data compute in the cloud to perform streaming, cognitive, predictive, prescriptive, historical and batch analytics@scale, integrating analytics with our core product platforms to turn data into actionable insights.

Turn Data Into Actionable Insights - StampedeCon 2016

Turn Data Into Actionable Insights - StampedeCon 2016

Turn Data Into Actionable Insights - StampedeCon 2016

Building an Experimentation Platform in Clojure

Building an Experimentation Platform in Clojure

Building an Experimentation Platform in Clojure

Srihari Sriraman

Co-op’s Transformation from Brick and Mortar to AI with Databricks with Rob M...

Co-op’s Transformation from Brick and Mortar to AI with Databricks with Rob M...

Co-op’s Transformation from Brick and Mortar to AI with Databricks with Rob M...

Large scale computing

Large scale computing

Large scale computing

The catalyst for the success of automobiles came not through the invention of the car but rather through the establishment of an innovative assembly line. History shows us that the ability to mass produce and distribute a product is the key to driving adoption of any innovation, and machine learning is no different. MLOps is the assembly line of Machine Learning and in this presentation we will discuss the core capabilities your organization should be focused on to implement a successful MLOps system.

MLOps - The Assembly Line of ML

MLOps - The Assembly Line of ML

MLOps - The Assembly Line of ML

Jordan Birdsell

New learning technologies seem likely to transform much of science, as they are already doing for many areas of industry and society. We can expect these technologies to be used, for example, to obtain new insights from massive scientific data and to automate research processes. However, success in such endeavors will require new learning systems: scientific computing platforms, methods, and software that enable the large-scale application of learning technologies. These systems will need to enable learning from extremely large quantities of data; the management of large and complex data, models, and workflows; and the delivery of learning capabilities to many thousands of scientists. In this talk, I review these challenges and opportunities and describe systems that my colleagues and I are developing to enable the application of learning throughout the research process, from data acquisition to analysis.

Learning Systems for Science

Learning Systems for Science

Learning Systems for Science

Machine Learning in SurveyMonkey

Machine Learning in SurveyMonkey

Machine Learning in SurveyMonkey

http://www.bigdataspain.org/2014/conference/state-of-play-data-science-on-hadoop-in-2015-keynote Machine Learning is not new. Big Machine Learning is qualitatively different: More data beats algorithm improvement, scale trumps noise and sample size effects, can brute-force manual tasks. Session presented at Big Data Spain 2014 Conference 18th Nov 2014 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Slides: https://speakerdeck.com/bigdataspain/state-of-play-data-science-on-hadoop-in-2015-by-sean-owen-at-big-data-spain-2014

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

Ai & Data Analytics 2018 - Azure Databricks for data scientist

Ai & Data Analytics 2018 - Azure Databricks for data scientist

Ai & Data Analytics 2018 - Azure Databricks for data scientist

Alberto Diaz Martin

Spark at Zillow

Spark at Zillow

Spark at Zillow

Steven Hoelscher

Similar to Josh Wills, MLconf 2013 (20)

Cloudera User Group - From the Lab to the Factory

Cloudera User Group - From the Lab to the Factory

Cloudera User Group - From the Lab to the Factory

Making Data Science Scalable - 5 Lessons Learned

Making Data Science Scalable - 5 Lessons Learned

Making Data Science Scalable - 5 Lessons Learned

How Azure Databricks helped make IoT Analytics a Reality with Janath Manohara...

How Azure Databricks helped make IoT Analytics a Reality with Janath Manohara...

How Azure Databricks helped make IoT Analytics a Reality with Janath Manohara...

Managing Machines: The New AI Dev Stack

Managing Machines: The New AI Dev Stack

Managing Machines: The New AI Dev Stack

How and why you need to build a big data lab

How and why you need to build a big data lab

How and why you need to build a big data lab

Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink

Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink

Suneel Marthi – BigPetStore Flink: A Comprehensive Blueprint for Apache Flink

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Data Science as a Service: Intersection of Cloud Computing and Data Science

Testing Big Data: Automated Testing of Hadoop with QuerySurge

Testing Big Data: Automated Testing of Hadoop with QuerySurge

Testing Big Data: Automated Testing of Hadoop with QuerySurge

Using Azure Machine Learning to Detect Patterns in Data from Devices

Using Azure Machine Learning to Detect Patterns in Data from Devices

Using Azure Machine Learning to Detect Patterns in Data from Devices

Turn Data Into Actionable Insights - StampedeCon 2016

Turn Data Into Actionable Insights - StampedeCon 2016

Turn Data Into Actionable Insights - StampedeCon 2016

Building an Experimentation Platform in Clojure

Building an Experimentation Platform in Clojure

Building an Experimentation Platform in Clojure

Co-op’s Transformation from Brick and Mortar to AI with Databricks with Rob M...

Co-op’s Transformation from Brick and Mortar to AI with Databricks with Rob M...

Co-op’s Transformation from Brick and Mortar to AI with Databricks with Rob M...

Large scale computing

Large scale computing

Large scale computing

MLOps - The Assembly Line of ML

MLOps - The Assembly Line of ML

MLOps - The Assembly Line of ML

Learning Systems for Science

Learning Systems for Science

Learning Systems for Science

Machine Learning in SurveyMonkey

Machine Learning in SurveyMonkey

Machine Learning in SurveyMonkey

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

Ai & Data Analytics 2018 - Azure Databricks for data scientist

Ai & Data Analytics 2018 - Azure Databricks for data scientist

Ai & Data Analytics 2018 - Azure Databricks for data scientist

Spark at Zillow

Spark at Zillow

Spark at Zillow

More from MLconf

Understanding Human Impact: Social and Equity Assessments for AI Technologies Social and Equity Impact Assessments have broad applications but can be a useful tool to explore and mitigate for Machine Learning fairness issues and can be applied to product specific questions as a way to generate insights and learnings about users, as well as impacts on society broadly as a result of the deployment of new and emerging technologies. In this presentation, my goal is to advocate for and highlight the need to consult community and external stakeholder engagement to develop a new knowledge base and understanding of the human and social consequences of algorithmic decision making and to introduce principles, methods and process for these types of impact assessments.

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

The Brain’s Guide to Dealing with Context in Language Understanding Like the visual cortex, the regions of the brain involved in understanding language represent information hierarchically. But whereas the visual cortex organizes things into a spatial hierarchy, the language regions encode information into a hierarchy of timescale. This organization is key to our uniquely human ability to integrate semantic information across narratives. More and more, deep learning-based approaches to natural language understanding embrace models that incorporate contextual information at varying timescales. This has not only led to state-of-the art performance on many difficult natural language tasks, but also to breakthroughs in our understanding of brain activity. In this talk, we will discuss the important connection between language understanding and context at different timescales. We will explore how different deep learning architectures capture timescales in language and how closely their encodings mimic the brain. Along the way, we will uncover some surprising discoveries about what depth does and doesn’t buy you in deep recurrent neural networks. And we’ll describe a new, more flexible way to think about these architectures and ease design space exploration. Finally, we’ll discuss some of the exciting applications made possible by these breakthroughs.

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Applying Computer Vision to Reduce Contamination in the Recycling Stream With China’s recent refusal of most foreign recyclables, North American waste haulers are scrambling to figure out how to make on-shore recycling cost-effective in order to continue providing recycling services. Recyclables that were once being shipped to China for manual sorting are now primarily being redirected to landfills or incinerators. Without a solution, a nearly $5 billion annual recycling market could come to a halt. Purity in the recycling stream is key to this effort as contaminants in the stream can increase the cost of operations, damage equipment and reduce the ability to create pure commodities suitable for creating recycled goods. This market disruption as a result of China’s new regulations, however, provides us the chance to re-examine and improve our current disposal & collection habits with modern monitoring & artificial intelligence technology. Using images from our in-dumpster cameras, Compology has developed an ML-based process that helps identify, measure and alert for contaminants in recycling containers before they are picked-up, helping keep the recycling stream clean. Our convolutional neural network flags potential instances of contamination inside a dumpster, enabling garbage haulers to know which containers have the wrong type of material inside. This allows them to provide targeted, timely education, and when appropriate, assess fines, to improve recycling compliance at the businesses and residences they serve, helping keep recycling services financially viable. In this presentation, we will walk through our ML-based contamination measurement and scoring process by showing how Waste Management, a national waste hauler, has experienced 57% contamination reduction in nearly 2,000 containers over six months, This progress shows significant strides towards financially viable recycling services.

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Quantum Computing: a Treasure Hunt, not a Gold Rush Quantum computers promise a significant step up in computational power over conventional computers, but also suffer a number of counterintuitive limitations --- both in their computational model and in leading lab implementations. In this talk, we review how quantum computers compete with conventional computers and how conventional computers try to hold their ground. Then we outline what stands in the way of successful quantum ML applications.

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Data Labeling as Religious Experience One of the most common places to deploy a production machine learning systems is as a replacement for a legacy rules-based system that is having a hard time keeping up with new edge cases and requirements. I'll be walking through the process and tooling we used to help us design, train, and deploy a model to replace a set of static rules we had for handling invite spam at Slack, talk about what we learned, and discuss some problems to solve in order to make these migrations easier for everyone.

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Project GaitNet: Ushering in the ImageNet moment for human Gait kinematics The emergence of the upright human bipedal gait can be traced back 4 to 2.8 million years ago, to the now extinct hominin Australopithecus afarensis. Fine grained analysis of gait using the modern MEMS sensors found on all smartphones not just reveals a lot about the person’s orthopedic and neuromuscular health status, but also has enough idiosyncratic clues that it can be harnessed as a passive biometric. While there were many siloed attempts made by the machine learning community to model Bipedal Gait sensor data, these were done with small datasets oft collected in restricted academic environs. In this talk, we will introduce the ImageNet moment for human gait analysis by presenting 'Project GaitNet', the largest ever planet-sized motion sensor based human bipedal gait dataset ever curated. We’ll also present the associated state-of-the-art results in classifying humans harnessing novel deep neural architectures and the related success stories we have enjoyed in transfer-learning into disparate domains of human kinematics analysis.

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Machine Learning Methods in Detecting Alzheimer’s Disease from Speech and Language Alzheimer's disease affects millions of people worldwide, and it is important to predict the disease as early and as accurate as possible. In this talk, I will discuss development of novel ML models that help classifying healthy people from those who develop Alzheimer's, using short samples of human speech. As an input to the model, features of different modalities are extracted from speech audio samples and transcriptions: (1) syntactic measures, such as e.g. production rules extracted from syntactic parse trees, (2) lexical measures, such as e.g. features of lexical richness and complexity and lexical norms, and (3) acoustic measures, such as e.g. standard Mel-frequency cepstral coefficients. I will present the ML model that detects cognitive impairment by reaching agreement among modalities. The resulting model is able to achieve state of the art performance in both supervised and semi-supervised manner, using manual transcripts of human speech. Additionally, I will discuss potential limitations of any fully-automated speech-based Alzheimer's disease detection model, focusing mostly on the analysis of the impact of a not-so-accurate automatic speech recognition (ASR) on the classification performance. To illustrate this, I will present the experiments with controlled amounts of artificially generated ASR errors and explain how the deletion errors affect Alzheimer's detection performance the most, due to their impact on the features of syntactic and lexical complexity.

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Optimized Image Classification on the Cheap In this talk, we anchor on building an image classifier trained on the Stanford Cars dataset to evaluate two approaches to transfer learning -fine tuning and feature extraction- and the impact of hyperparameter optimization on these techniques. Once we define the most performant transfer learning technique for Stanford Cars, we will double the size of the dataset through image augmentation to boost the classifier’s performance. We will use Bayesian optimization to learn the hyperparameters associated with image transformations using the downstream image classifier’s performance as the guide. In conjunction with model performance, we will also focus on the features of these augmented images and the downstream implications for our image classifier. To both maximize model performance on a budget and explore the impact of optimization on these methods, we apply a particularly efficient implementation of Bayesian optimization to each of these architectures in this comparison. Our goal is to draw on a rigorous set of experimental results that can help us answer the question: how can resource-constrained teams make trade-offs between efficiency and effectiveness using pre-trained models?

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

The Importance of Modeling Data Collection Data sets used in machine learning are often collected in a systematically biased way - certain data points are more likely to be collected than others. We call this "observation bias". For example, in health care, we are more likely to see lab tests when the patient is feeling unwell than otherwise. Failing to account for observation bias can, of course, result in poor predictions on new data. By contrast, properly accounting for this bias allows us to make better use of the data we do have. In this presentation, we discuss practical and theoretical approaches to dealing with observation bias. When the nature of the bias is known, there are simple adjustments we can make to nonparametric function estimation techniques, such as Gaussian Process models. We also discuss the scenario where the data collection model is unknown. In this case, there are steps we can take to estimate it from observed data. Finally, we demonstrate that having a small subset of data points that are known to be collected at random - that is, in an unbiased way - can vastly improve our ability to account for observation bias in the rest of the data set. My hope is that attendees of this presentation will be aware of the perils of observation bias in their own work, and be equipped with tools to address it.

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

The Uncanny Valley of ML Every so often, the conundrum of the Uncanny Valley re-emerges as advanced technologies evolve from clearly experimental products to refined accepted technologies. We have seen its effects in robotics, computer graphics, and page load times. The debate of how to handle the new technology detracts from its benefits. When machine learning is added to human decision systems a similar effect can be measured in increased response time and decreased accuracy. These systems include radiology, judicial assignments, bus schedules, housing prices, power grids and a growing variety of applications. Unfortunately, the Uncanny Valley of ML can be hard to detect in these systems and can lead to degraded system performance when ML is introduced, at great expense. Here, we'll introduce key design principles for introducing ML into human decision systems to navigate around the Uncanny Valley and avoid its pitfalls.

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

Deep Learning Architectures for Semantic Relation Detection Tasks Recognizing and distinguishing specific semantic relations from other types of semantic relations is an essential part of language understanding systems. Identifying expressions with similar and contrasting meanings is valuable for NLP systems which go beyond recognizing semantic relatedness and require to identify specific semantic relations. In this talk, I will first present novel techniques for creating labelled datasets required for training deep learning models for classifying semantic relations between phrases. I will further present various neural network architectures that integrate morphological features into integrated path-based and distributional relation detection algorithms and demonstrate that this model outperforms state-of-the-art models in distinguishing semantic relations and is capable of efficiently handling multi-word expressions.

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Building an Incrementally Trained, Local Taste Aware, Global Deep Learned Recommender System Model At Netflix, our main goal is to maximize our members’ enjoyment of the selected show by minimizing the amount of time it takes for them to find it. We try to achieve this goal by personalizing almost all the aspects of our product -- from what shows to recommend, to how to present these shows and construct their home-pages to what images to select per show, among many other things. Everything is recommendations for us and as an applied Machine Learning group, we spend our time building models for personalization that will eventually increase the joy and satisfaction of our members. In this talk we will primarily focus our attention on a) making a global deep learned recommender model that is regional tastes and popularity aware and b) adapting this model to changing taste preferences as well as dynamic catalog availability. We will first go through some standard recommender system models that use Matrix Factorization and Topic Models and then compare and contrast them with more powerful and higher capacity deep learning based models such as sequence models that use recurrent neural networks. We will show what it entails to build a global model that is aware of regional taste preferences and catalog availability. We will show how models that are built on simple Maximum Likelihood principle fail to do that. We will then describe one solution that we have employed in order to enable the global deep learned models to focus their attention on capturing regional taste preferences and changing catalog.In the latter half of the talk, we will discuss how we do incremental learning of deep learned recommender system models. Why do we need to do that ? Everything changes with time. Users’ tastes change with time. What’s available on Netflix and what’s popular also change over time. Therefore, updating or improving recommendation systems over time is necessary to bring more joy to users. In addition to how we apply incremental learning, we will discuss some of the challenges we face involving large-scale data preparation, infrastructure setup for incremental model training as well as pipeline scheduling. The incremental training enables us to serve fresher models trained on fresher and larger amounts of data. This helps our recommender system to nicely and quickly adapt to catalog and users’ taste changes, and improve overall performance.

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Vito Ostuni - The Voice: New Challenges in a Zero UI World The adoption of voice-enabled devices has seen an explosive growth in the last few years and music consumption is among the most popular use cases. Music personalization and recommendation plays a major role at Pandora in providing a daily delightful listening experience for millions of users. In turn, providing the same perfectly tailored listening experience through these novel voice interfaces brings new interesting challenges and exciting opportunities. In this talk we will describe how we apply personalization and recommendation techniques in three common voice scenarios which can be defined in terms of request types: known-item, thematic, and broad open-ended. We will describe how we use deep learning slot filling techniques and query classification to interpret the user intent and identify the main concepts in the query. We will also present the differences and challenges regarding evaluation of voice powered recommendation systems. Since pure voice interfaces do not contain visual UI elements, relevance labels need to be inferred through implicit actions such as play time, query reformulations or other types of session level information. Another difference is that while the typical recommendation task corresponds to recommending a ranked list of items, a voice play request translates into a single item play action. Thus, some considerations about closed feedback loops need to be made. In summary, improving the quality of voice interactions in music services is a relatively new challenge and many exciting opportunities for breakthroughs still remain. There are many new aspects of recommendation system interfaces to address to bring a delightful and effortless experience for voice users. We will share a few open challenges to solve for the future.

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

More from MLconf (20)

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

Recently uploaded

🐬 The future of MySQL is Postgres 🐘

🐬 The future of MySQL is Postgres 🐘

🐬 The future of MySQL is Postgres 🐘

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Artificial Intelligence: Facts and Myths

Artificial Intelligence: Facts and Myths

Slack Application Development 101 Slides

Slack Application Development 101 Slides

Slack Application Development 101 Slides

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

Finology Group – Insurtech Innovation Award 2024

Finology Group – Insurtech Innovation Award 2024

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

How to convert PDF to text with Nanonets

How to convert PDF to text with Nanonets

How to convert PDF to text with Nanonets

The 7 Things I Know About Cyber Security After 25 Years | April 2024

The 7 Things I Know About Cyber Security After 25 Years | April 2024

The 7 Things I Know About Cyber Security After 25 Years | April 2024

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

A Call to Action for Generative AI in 2024

A Call to Action for Generative AI in 2024

A Call to Action for Generative AI in 2024

Microsoft's Threat Matrix for Kubernetes helps organizations understand the attack surface a Kubernetes deployment introduces to their environments. This ensures that adequate detections and mitigations are in place. By covering over 40 different attacker techniques, defenders can learn about Kubernetes-specific mitigations and controls to deploy to their environments. In this session, we will explore the MS-TA9013 Host Path Mount technique, which is commonly used by attackers to perform privilege escalation in a Kubernetes cluster. Attendees will learn how attackers and defenders can: * Escape the container's host volume mount to gain persistence on an underlying node * Move laterally from the underlying node into the customer's cloud environment * Analyze Kubernetes audit logs to detect pods deployed with a hostPath mount * Deploy an admission controller that prevents new pods from using a hostPath mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Puma Security, LLC

Automating Google Workspace (GWS) & more with Apps Script

Automating Google Workspace (GWS) & more with Apps Script

Automating Google Workspace (GWS) & more with Apps Script

Choosing the right accounts payable services provider is a strategic decision that can significantly impact your business's financial performance and operational efficiency. By considering factors such as expertise, range of services, technology infrastructure, scalability, cost, and reputation, businesses can make informed decisions and select a provider that aligns with their unique needs and objectives. Partnering with the right provider can streamline accounts payable processes, drive cost savings, and position your business for long-term success. https://katprotech.com/accounts-payable-and-purchase-order-automation/

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Katpro Technologies

CNv6 Instructor Chapter 6 Quality of Service

CNv6 Instructor Chapter 6 Quality of Service

CNv6 Instructor Chapter 6 Quality of Service

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

[2024]Digital Global Overview Report 2024 Meltwater.pdf

[2024]Digital Global Overview Report 2024 Meltwater.pdf

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Handwritten Text Recognition for manuscripts and early printed texts

Handwritten Text Recognition for manuscripts and early printed texts

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Recently uploaded (20)

🐬 The future of MySQL is Postgres 🐘

🐬 The future of MySQL is Postgres 🐘

🐬 The future of MySQL is Postgres 🐘

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Boost PC performance: How more available memory can improve productivity

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Artificial Intelligence: Facts and Myths

Artificial Intelligence: Facts and Myths

Artificial Intelligence: Facts and Myths

Slack Application Development 101 Slides

Slack Application Development 101 Slides

Slack Application Development 101 Slides

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

Finology Group – Insurtech Innovation Award 2024

Finology Group – Insurtech Innovation Award 2024

Finology Group – Insurtech Innovation Award 2024

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

How to convert PDF to text with Nanonets

How to convert PDF to text with Nanonets

How to convert PDF to text with Nanonets

The 7 Things I Know About Cyber Security After 25 Years | April 2024

The 7 Things I Know About Cyber Security After 25 Years | April 2024

The 7 Things I Know About Cyber Security After 25 Years | April 2024

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

A Call to Action for Generative AI in 2024

A Call to Action for Generative AI in 2024

A Call to Action for Generative AI in 2024

Breaking the Kubernetes Kill Chain: Host Path Mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Automating Google Workspace (GWS) & more with Apps Script

Automating Google Workspace (GWS) & more with Apps Script

Automating Google Workspace (GWS) & more with Apps Script

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

CNv6 Instructor Chapter 6 Quality of Service

CNv6 Instructor Chapter 6 Quality of Service

CNv6 Instructor Chapter 6 Quality of Service

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Data Cloud, More than a CDP by Matt Robison

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

[2024]Digital Global Overview Report 2024 Meltwater.pdf

[2024]Digital Global Overview Report 2024 Meltwater.pdf

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Handwritten Text Recognition for manuscripts and early printed texts

Handwritten Text Recognition for manuscripts and early printed texts

Handwritten Text Recognition for manuscripts and early printed texts

Josh Wills, MLconf 2013

1. From The Lab to the Factory Building A Produc8on Machine Learning Infrastructure Josh Wills, Senior Director of Data Science Cloudera 1

3. Data Science: Another Deﬁni8on 3

4. Data Scien8sts Build Data Products. 4

5. All* Products Become Data Products 5

6. Iden8fying the BoHlenecks 6

7. Oryx: Model Building and Serving •  Algorithms •  •  •  •  •  •  7 ALS Recommenders K-‐Means Parallel RDF Batch model building via MapReduce Server for real-‐8me scoring and updates PMML 4.1 Models

8. Gertrude: Evalua8on via Experiments •  Mul8variate Tes8ng •  •  Overlapping Experiments •  •  8 Deﬁne and explore a space of parameters Tang et al. (2010) Runs mul8ple independent experiments on every request

9. Planning For The Future 9

10. Thank you! Josh Wills, Director of Data Science, Cloudera @josh_wills