Demo: Predictive Modeling with BigML - by David Gerster - PAPIs Connect

•Download as PPTX, PDF•

1 like•700 views

PAPIs.io

Demo: Predictive Modeling with BigML by David Gerster

Data & Analytics

1
David Gerster
VP Data Science
gerster@bigml.com

Demo: Predictive Modeling
• Train a predictive model using 699 biopsies
• The “label” of benign or malignant is known for each one
• Since we have labels, this is supervised learning
2

What if we don’t have labels?
• Can we get insight into our data if we don’t know the labels?
• Enter anomaly detection
• Since we don’t have labels, this is unsupervised learning
3

10 lines are needed
to isolate this data point
(not anomalous)

Only 4 lines are needed
to isolate this data point
(highly anomalous)

Demo: Anomaly Detection
• Remove the labels of benign or malignant
• Train an anomaly detector on this unlabeled data
• Create a new dataset with the anomaly scores as “labels”
• Use these “labels” to train a predictive model!
6

Minority Report
• Anomaly detection works great on large unlabeled datasets,
especially if you expect to find an (adversarial) minority class
• Millions of credit card transactions, billions of network events …
• Doesn’t require you to know what you’re looking for!
9

Thanks!
10
David Gerster
VP Data Science, BigML
gerster@bigml.com

Similar to Demo: Predictive Modeling with BigML - by David Gerster - PAPIs Connect

We've been taught that "data science" is the esoteric domain of PhDs, but like anything else, it's easy once you understand it. This talk explains the basics of data science, covering concepts in supervised learning (including a detailed explanation of decision trees and random forests) as well as examples of unsupervised learning algorithms. Far from being a dry and academic topic, data science and machine learning are useful and practical analytical tools. (This talk is intended for a general audience.) Topics will include: 1) An introduction to supervised learning using the popular decision tree algorithm 2) The concepts of training and scoring, and the meaning of "real time" machine learning 3) Model validation using holdout sets 4) Model complexity and overfitting; understanding bias and variance; using ensembles to reduce variance 5) An overview of unsupervised learning models including clustering, topic modeling and anomaly detection and more!

Data Science 101

ideatoipo

BigML Education - Anomaly Detection

BigML, Inc

L15. Machine Learning - Black Art

Machine Learning Valencia

WHAT GOOD IS ANOMALY DETECTION? Static thresholds on metrics have been falling out of fashion for a while, and for good reason. Modern tooling lets you analyze and monitor a lot more data points than you used to be able to, resulting in lots more noise. The hope is that anomaly detection answers some of this, by replacing static thresholds (anomalies) with dynamic ones. But it doesn’t work as well as most people think it will. In this talk I’ll explain how anomaly detection works, so you can understand why it isn’t a good general-purpose solution, and which specific cases it’s good at.

Influx/Days 2017 San Francisco | Baron Schwartz

InfluxData

Anomaly detection (or Outlier analysis) is the identification of items, events or observations which do not conform to an expected pattern or other items in a dataset. It is used is applications such as intrusion detection, fraud detection, fault detection and monitoring processes in various domains including energy, healthcare and finance. In this workshop, we will discuss the core techniques in anomaly detection and discuss advances in Deep Learning in this field. Through case studies, we will discuss how anomaly detection techniques could be applied to various business problems. We will also demonstrate examples using R, Python, Keras and Tensorflow applications to help reinforce concepts in anomaly detection and best practices in analyzing and reviewing results. What you will learn: Anomaly Detection: An introduction Graphical and Exploratory analysis techniques Statistical techniques in Anomaly Detection Machine learning methods for Outlier analysis Evaluating performance in Anomaly detection techniques Detecting anomalies in time series data Case study 1: Anomalies in Freddie Mac mortgage data Case study 2: Auto-encoder based Anomaly Detection for Credit risk with Keras and Tensorflow

Anomaly detection Workshop slides

QuantUniversity

AI Models For Fun and Profit by Walmart Director of Artificial Intelligence

Product School

Hands-on Introduction to Machine Learning

Brittany Lasseigne, Ph.D.

Supervised Learning

FEG

L14. Anomaly Detection

Machine Learning Valencia

Outlier analysis,Chapter-12, Data Mining: Concepts and Techniques

Ashikur Rahman

Meet up september19-final

Ido Rozen

Data science for advanced dummies

Saurav Chakravorty

Supervised learning

Alia Hamwi

Kevin Swingler: Introduction to Data Mining

Library and Information Science Research Coalition

Ethics in Data Science and Machine Learning

HJ van Veen

Implementing an appropriate data processing pipeline to make good use of your indicators of compromise is a problem that has been successfully addressed over the last few years. Even with all the push of automation and orchestration, a fundamental question remains: WHICH data should I be ingesting in my detection pipelines? There is no lack of data available, shared or not, paid or not. But how to keep my CTI IR team from spinning their wheels on a pile of CTI mud? This talk will discuss statistical analysis you can do with the CTI indicators you collect and your own network telemetry to define: - FIT: How appropriate does the CTI data apply to your own traffic. CTI vendors always talk about vertical specific threats, but is that measurable and verifiable? - IMPACT: How much was your true positive detections assisted by matches and link analysis derived from those CTI feeds - COVERAGE: Is your current mix of CTI feeds providing "intelligence" on the current threats that you should actually be concerned with? Those concepts will be introduced and explained with minimal math background needed, and pseudo-code (and real-code!) will be provided to assist organizations to perform those experiments on their own environment. We hope those tools will help attendees to better evaluate the quality of the CTI feeds they ingest from their open sources, paid providers and sharing communities.

Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...

Alex Pinto

The Dangers of Machine Learning

tothepointIT

BIG DATA AND MACHINE LEARNING

Umair Shafique

Data Con LA 2020 Description With recent events putting a spotlight on anti-racism, social-justice, climate change, and mental health there's a call for increased ethics and transparency in business. Companies are, rightfully, feeling responsible for providing underrepresented employees with the same treatment and opportunities as their majority counterparts. AI can, and will, be used to help companies understand their environment, develop strategies for improvement and monitor progress. And, as AI is used to make increasingly complex and life-changing decisions, it is critical to ensure that these decisions are fair, equitable and explainable. Unfortunately, it is becoming increasingly clear that, much like humans, AI can be biased. It is therefore imperative that as we develop AI solutions, we are fully aware of the dangers of bias, understand how bias can manifest and know how to take steps to address and minimize it. In this session you will learn: *Definitions of fairness, regulated domains and protected classes *How bias can manifest in AI *How bias in AI can be measured, tracked and reduced *Best practices for ensuring that bias doesn't creep into AI/ML models over time *How explainability can be used to perform real-time checks on predictions Speakers Lawrence Spracklen, RSquared AI, Engineering Leadership Sonya Balzer, RSquared.ai, Director of AI Marketing

Using AI to Build Fair and Equitable Workplaces

Data Con LA

DutchMLSchool. Clusters and Anomalies

BigML, Inc

Similar to Demo: Predictive Modeling with BigML - by David Gerster - PAPIs Connect (20)

Data Science 101

BigML Education - Anomaly Detection

L15. Machine Learning - Black Art

Influx/Days 2017 San Francisco | Baron Schwartz

Anomaly detection Workshop slides

AI Models For Fun and Profit by Walmart Director of Artificial Intelligence

Hands-on Introduction to Machine Learning

Supervised Learning

L14. Anomaly Detection

Outlier analysis,Chapter-12, Data Mining: Concepts and Techniques

Meet up september19-final

Data science for advanced dummies

Supervised learning

Kevin Swingler: Introduction to Data Mining

Ethics in Data Science and Machine Learning

Determining the Fit and Impact of CTI Indicators on Your Monitoring Pipeline ...

The Dangers of Machine Learning

BIG DATA AND MACHINE LEARNING

Using AI to Build Fair and Equitable Workplaces

DutchMLSchool. Clusters and Anomalies

More from PAPIs.io

The daily job of a Data Scientist ranges from a variety of tasks: improving models performance or dealing with framework structure implementations. Machine learning as a service, a hot topic in the field, implies thinking about architecture to allow constant improvements in performance for our products. This presentation shows one architecture design using RESTful resources, document oriented databases and pre-trained pipelines to achieve real-time predictions of time series with high availability, scalability and freedom to Data Scientists work directly on improving the accuracy rate of our products. We fine tunned to work on time series forecasting which is a very challenging field that still needs better solutions in terms of innovative modeling. During the presentation will be shown how these decisions keep our Data Scientists focused on working with real data and thinking about improvements that can reach a large volume of time series instead of singular and localized actions.

Shortening the time from analysis to deployment with ml as-a-service — Luiz A...

PAPIs.io

Feature engineering is one of the most important, yet elusive, skills to master if you want to be a good data scientist. Machine learning competitions are hardly ever won with strong modeling techniques alone -- it is the combination of creative feature engineering and powerful modeling techniques that makes the difference. This tutorial will give the audience practical tips and tricks to improve the performance of machine learning algorithms. We will broadly look at feature engineering for applied machine learning, touching on subjects like: categorical vs. numerical variables, data cleaning, feature extraction, transformations, and imputation.

Feature engineering — HJ Van Veen (Nubank) @@PAPIs Connect — São Paulo 2017

PAPIs.io

For online businesses, recommender systems are paramount. There is an increasing need to take into account all the user information to tailor the best product offer, tailored to each new user. Part of that information is the content that the user actually sees: the visuals of the products. When it comes to products like luxury hotels, pictures of the room, the building or even the nearby beach can significantly impact users’ decision. In this talk, we will describe how we improved an online vacation retailer recommender system by using the information in images. We’ll explain how to leverage open data and pre-trained deep learning models to derive information on user taste. We will use a transfer learning approach that enables companies to use state of the art machine learning methods without needing deep learning expertise.

Extracting information from images using deep learning and transfer learning ...

PAPIs.io

Graphs are used to map relations on unstructured data. Companies’ data are most from database and mined using traditional data mining approach. However, model relational data as a graph can reveal useful insights and discovery relation among data that is ignored by traditional data mining techniques. In this work we used graphs to map physician relations using claim data as a proxy and this approach reveal interesting insights from health insurance company.

Discovering the hidden treasure of data using graph analytic — Ana Paula Appe...

PAPIs.io

Deep learning for sentiment analysis — André Barbosa (elo7) @PAPIs Connect — ...

PAPIs.io

Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...

PAPIs.io

In times of huge amounts of heterogeneous data available, processing and extracting knowledge requires more and more efforts on building complex software architectures. In this context, Apache Spark provides a powerful and efficient approach for large-scale data processing. This talk will briefly introduce a powerful machine learning library (MLlib) along with a general overview of the Spark framework, describing how to launch applications within a cluster. In this way, a demo will show how to simulate a Spark cluster in a local machine using images available on a Docker Hub public repository. In the end, another demo will show how to save time using unit tests for validating jobs before running them in a cluster.

Building machine learning applications locally with Spark — Joel Pinho Lucas ...

PAPIs.io

Battery life is critical for smart devices, but optimizing it requires cooperation from the entire software ecosystem. Wasteful software affects user perception about devices’ battery quality. Therefore, a large team within a producer of those smart devices is focused on identifying and correcting energy consumption bugs. Since the software ecosystem grows fast, that team faces a lot of suspect issues, from which only a small fraction turns out to be genuine. Our project aims to streamline energy-related bug processing in devices of the company and its partners, by automatically identifying anomalous behaviors related to battery drain using data mining and machine learning.

Battery log data mining — Ramon Oliveira (Datart) @PAPIs Connect — São Paulo ...

PAPIs.io

News recommendations are particularly challenging given the high number of new contents produced every day and the fast deterioration of its value for the users, demanding models and infrastructure able to deal with those nuances and serve a newly trained model about 100 times per day. Attending this presentation you're going to follow a detailed overview of how R&D team of Hearst's TV division is putting together Google BigQuery, Kubernetes cluster and Tensorflow to build a hybrid recommendation system combining model-based matrix factorization, content recency, and content semantics through NLP.

A tensorflow recommending system for news — Fabrício Vargas Matos (Hearst tv)...

PAPIs.io

Machine learning as a service (MLaS) is imperative to the success of many companies as many internal teams and organizations need to gain business intelligence from big data. Building a scalable MLaS in a very challenging problem. In this paper, we present the scalable MLaS we built for a company that operates globally. We focus on several scalability challenges and our technical solutions. Video at https://www.youtube.com/watch?v=MpnszJ_3Ong Couldn't attend PAPIs '16? Get access to the other presentations' slides and videos at https://gumroad.com/products/fehon/

Scaling machine learning as a service at Uber — Li Erran Li at #papis2016

PAPIs.io

This talk will offer answers to the following questions: What is data-driven decision making? What is AI? What is Business Intelligence? Why are these concepts important? What are the biggest challenges and opportunities? Daniel is the CEO of Satalia that provides AI inspired solutions to solve industries hardest problems. He’s the co-founder of the ASI that transitions scientists into data scientists. Daniel has a MSci and EngD in AI from UCL, and is Director of UCL’s Business Analytics MSc; applying AI to solve business/social problems. Daniel has many Advisory and Executive positions, holds an international Kauffman Global Entrepreneur Scholarship and actively promotes innovation across the globe.

Real-world applications of AI - Daniel Hulme @ PAPIs Connect

PAPIs.io

Possibly the most important lesson we have learned after 60 years of AI research is that what seemed to be very difficult to achieve, such as accurate medical diagnosis to playing chess at the level of a Grand Master, turned out to be relatively easy whereas what seemed easy, such as visual object recognition or deep language understanding, turned out to be extremely difficult. In my talk I will try to explain the reasons for this apparent contradiction by briefly reviewing the past and present of AI and projecting it into the near future. Ramon Lopez de Mantaras is Research Professor of the Spanish National Research Council (CSIC) and Director of the Artificial Intelligence Research Institute of the CSIC. Technical Engineer EE (Electrical Engineering) from the Technical Engineering School of Mondragón (Spain) in 1973. Master of Sciences in Automatic Control from the University of Toulouse III (France) in 1974, Ph.D. in Physics from the University of Toulouse III (France), in 1977, with a thesis in Robotics (done at LAAS, CNRS). Master of Science in Engineering (ComputerScience) from the University of California at Berkeley (USA) in 1979. Ph.D. in Computer Science, from the Technical University of Catalonia, Barcelona (Spain) in 1981.

Past, Present and Future of AI: a Fascinating Journey - Ramon Lopez de Mantar...

PAPIs.io

Everybody uses price promotions in retail. However, individual pricing is seldom used, particularly in offline retail. Marketing literature has been advocating the use of individual price discrimination for decades. Furthermore, product recommendations, ever-present in e-commerce, are also not often found in offline retail. We show the machine learning driven system behind a new promotion channel that enables retailers and manufacturers alike to target individual customers in offline retail. Lessons learned, technologies used, and machine learning approaches driving our system will be shown. Daniel Guhl has a background in economics & marketing, and got interested in data modeling during his Ph.D.. Currently, he is working as a data scientist at a Berlin based Start-up and is pursuing a postdoc at Humboldt University. He enjoys learning everyday and focuses on solving real world problems.

Revolutionizing Offline Retail Pricing & Promotions with ML - Daniel Guhl @ P...

PAPIs.io

Deep Learning (DL) is becoming a big tsunami in the Machine Learning community. This talk aims at introducing DL, its motivation and main techniques. However, part of this talk is also devoted to demystify DL. What are the main advantages but also the main drawbacks of DL?. And what are the key issues that the practitioners have to consider? Roberto Paredes is an Associate Professor at Departamento de Sistemas Informáticos y Computación DSIC of the Universidad Poliécnica de Valencia UPV. He belongs to the Pattern Recognition and Human Language Technologies Research Centre PRHLT. Roberto Paredes is the Director of the PRHLT and the President of the Spanish AERFAI Association. His main research interests are around the statistical learning, machine learning and more recently neural networks and deep learning.

Demystifying Deep Learning - Roberto Paredes Palacios @ PAPIs Connect

PAPIs.io

The best services have one thing in common: a superb customer experience. Banking services are no exception to this rule, and indeed the quest for an effortless, well informed, and personalized customer experience is one of the main goals of today's innovation in digital banking services. According to what Maslow has described in his "pyramid of needs", customers are seeking a more intimate and meaningful experience where banking services can actively assist the customer in performing and managing their financial life. Predictive APIs have a fundamental role in all this, as they enable a new set of customer journeys such as automatic categorization of transactions, detecting and alerting recurrent payments, pre-approving credit requests or provide better tools to fight fraud without limiting legitimate customer transactions. In this talk, I will focus on how to provide better banking services by using predictive APIs. I will describe the path on how to get there and the challenges of implementing predictive APIs in a strictly audited and regulated domain such as banking. Finally, I will briefly introduce a number of data science techniques to implement those customer journeys and describe how big/fast data engineering can be used to realize predictive data pipelines. Natalino is currently Enterprise Data Architect at ING in the Netherlands, where leads the strategy, definition, design and implementation of big/fast data solutions for data-driven applications, for personalized marketing, predictive analytics, and fraud/security management. All-round Software Architect, Data Technologist, Innovator, with 15+ years experience in research, development and management of distributed architectures and scalable services and applications.

Predictive APIs: What about Banking? - Natalino Busa @ PAPIs Connect

PAPIs.io

Fintech startups are taking business away from traditional institutions like banks, exchanges, and brokerages. One of the reasons that these startups are able to compete with $30B+ behemoths like Credit Suisse and Goldman Sachs is their advanced decision making capabilities. By leveraging new data sources and better predictive analytics, companies like Ferratum Bank can make more accurate decisions in a fraction of the time. This talk will cover: Types of decisions you can automate Challenges in building predictive, financial apps First-hand, real-world examples Greg Lamp is the co-Founder and CTO of Yhat. In this role, Greg leads development of Yhat's core products and infrastructure and is the principal architect of the company's cloud and on-premise enterprise software applications. Greg was previously a product manager at OnDeck, a fintech startup in New York and before that an analyst at comScore. Greg is a graduate of the University of Virginia.

Microdecision making in financial services - Greg Lamp @ PAPIs Connect

PAPIs.io

What is the future we want to create, and what can we do – starting today – to actively shape that future with general AI? This talk outlines a vision for the future of humankind once AI reaches human or superhuman levels, and leads the audience through the steps one research group is taking to get there. From the economics of smart robots and job replacement, to bionic humans exploring the universe through space travel, the talk offers a window into the work of 30 researchers focused on AI development and safety, and explains what attendees can do themselves to help make that future happen. JoEllen is the AI Safety Ambassador and Head of PR for GoodAI, a Prague-based general AI research and development company. A high school teacher by trade, she has a bachelor’s degrees in English and Philosophy from Seattle University, a master’s degree in Transatlantic Studies from Charles University in Prague, and is the recipient of Fulbright grant. JoEllen is particularly interested in how AI will affect international government and political relations.

Engineering the Future of Our Choice with General AI - JoEllen Lukavec Koeste...

PAPIs.io

Training deep networks is a time-consuming process, with networks for object recognition often requiring multiple days to train. For this reason, leveraging the resources of a cluster to speed up training is an important area of work. In this talk we'll show how to use an AWS Spark cluster to train a model quickly from a laptop at a very little cost (around 10€). Vincent Van Steenbergen is a freelance (big) data engineer who's working on a range of international projects, implementing systems able to handle terabytes of data, usually involving Spark, Scala, Kafka, Hadoop and Cassandra. His main interest right now is applying these techniques to solve machine learning problems. Vincent was previously a technical architect at Property. Works, a real estate startup in London and before that an R&D engineer at IDAaaS.

Distributed deep learning with spark on AWS - Vincent Van Steenbergen @ PAPIs...

PAPIs.io

Shopping, or as the people on the other side of the counter call it, retail has become the number one breeding ground for predictive applications in the enterprise. What started as simple recommendation engines has evolved into a complex and powerful ecosystem of predictive applications that affect core processes such as pricing, replenishment and staff planning. In this talk, Ulrich Kerzel will share impact and experiences from building and operating predictive applications for large retailers, and explain why the future of retail is as much a science as an art. Dr. Ulrich Kerzel is a Senior data scientists at Blue Yonder and renowned scientist with research experience at the University of Cambridge and CERN. Ulrich Kerzel earned his PhD under Professor Dr Feindt at the US Fermi National Laboratory and at that time made a considerable contribution to core technology of NeuroBayes. After his PhD, he went to the University of Cambridge, were he was a Senior Research Fellow at Magdelene College. His research work focused on complex statistical analyses to understand the origin of matter and antimatter using data from the LHCb experiment at the Large Hadron Collider at CERN, the world’s biggest research institute for particle physics. He continued this work as a Research Fellow at CERN before he came to Blue Yonder as a senior data scientist.

How to predict the future of shopping - Ulrich Kerzel @ PAPIs Connect

PAPIs.io

We live in a world of data, of big data. A big portion of this data has been generated by humans, and particularly through their mobile phones. In fact, there are almost as many mobile phones in the world as humans. The mobile phone is the piece of technology with the highest levels of adoption in human history. We carry them with us all through the day (and night, in many cases), leaving digital traces of our physical interactions. Mobile phones have become sensors of human activity in the large scale and also the most personal devices. In my talk, I will present some of the work that we are doing at Telefonica Research in the area of human behavior understanding from data captured with mobile phones, and particularly our work in the area of Big Data for Social Good. I will highlight opportunities but also challenges that we would need to address in order to truly leverage this opportunity. Nuria Oliver is a computer scientist and Scientific Director at Telefónica. She holds a Ph.D. from the Media Lab at MIT. She is one of the most cited female computer scientist in Spain, with her research having been cited by more than 8900 publications. She is well known for her work in computational models of human behavior, human computer-interaction, intelligent user interfaces, mobile computing and big data for social good.

The emergent opportunity of Big Data for Social Good - Nuria Oliver @ PAPIs C...

PAPIs.io

More from PAPIs.io (20)