Large scale predictive analytics for anomaly detection - Nicolas Hohn

•

4 likes•1,602 views

This presentation will focus on anomaly detection for network data streams where the aim is to predict a distribution of future values and flag unlikely situations. Challenges both in terms of data science and engineering will be discussed, such as the accuracy, robustness and scalability of the prediction API. An example of a production deployment will also be discussed.

Data & Analytics

© 2014 Guavus, Inc. All rights reserved.
Nicolas Hohn
Director of Analytics
Guavus
LARGE SCALE PREDICTIVE ANALYTICS
FOR ANOMALY DETECTION
2015

© 2015 Guavus, Inc. All rights reserved. 2
Guavus Applications Focus
Planning
Engineering
•  Network Analytics
•  Capacity Management
•  Trending & Forecasting
•  Value-based
Network Planning
•  Quality of Experience
•  Complaints Mitigation
•  Proactive Care
•  Revenue Assurance
•  Self Care Usage Portal
Network
Operations
MarketingCare
•  Service Management
•  QoS Management
•  Performance Monitoring
•  Proactive Service
Assurance
•  Subscriber Profiling
•  Personalization &
Targeting
•  CSP Data
Monetization
Service Assurance Customer Experience

© 2015 Guavus, Inc. All rights reserved. 3
Anomaly Detection
•  Anomaly: something that is unusual or unexpected
•  Detection: extraction of particular information from a larger stream of information
without specific cooperation from or synchronization with the sender
Implementation
•  Rule based: manual thresholds
•  Automated: thresholds set by machine learning
Operational Intelligence
Service
Degrading
Problem!
Service
Anomalies
Identiﬁcation!
Root-Cause
Analysis!
Problem
Resolution!
Quantify how ‘unusual’ a signal value is
Unsupervised learning to send trigger
when signal is ‘unexpected enough’
DOES NOT SCALE
spike step slope
time time time
KPI

© 2015 Guavus, Inc. All rights reserved. 4
Anomaly detection
Event Arrival Times
2014-09-16 00:00:06
2014-09-16 00:00:09
2014-09-16 00:00:40
2014-09-16 00:00:42
2014-09-16 00:00:45
2014-09-16 00:01:00
2014-09-16 00:01:09
2014-09-16 00:01:11
2014-09-16 00:01:20
2014-09-16 00:02:09
……
5
4
Define KPI and time scale
Predict conditional baseline (black line) and
probability density given historical data
KPI value (green line)
Trigger alert (red dot) if data point
significantly above baseline, i.e. outside
confidence interval (gray bands)
1 2 3
4
•  4 step process
#events
Time

© 2015 Guavus, Inc. All rights reserved. 5
KPI
time
EASIER
Challenges
•  Predict distribution of current value based on past values
–  Uni/Multi variate time series analysis
•  Unify uncertainty metric across all types of input signals to build a global ranking of alarms
•  Scale on limited hardware footprint. Real time monitoring of potentially millions of time series
•  Keep customer happy (no alarm flooding, limit false positives, rank alarms by severity)
KPI
time
HARDER
HARD
HARD
HARD

© 2015 Guavus, Inc. All rights reserved. 6
Solutions
time
Anomalyindicator
#events
•  Data Science:
–  Robust to past anomalies
•  No guarantee that ‘training’ data is anomaly free
–  Adapt to changes
•  Retrain model
–  Cannot rely on labeled data:
•  Understand customer ‘utility function’, business impact of anomalies
•  Set thresholds automatically
•  Quantify cost of false positives and false negatives
•  Engineering:
–  Intelligent caching
–  Compression
–  Scalable system

© 2015 Guavus, Inc. All rights reserved. 7
•  Monitor KPIs, such as dropped call rate on each base station in a 4G network
•  Detect anomalies
•  Infer root cause by analyzing 1000s of other KPIs available on each cell of the network
Use case: Networks analytics

© 2015 Guavus, Inc. All rights reserved. 8
Architecture of the solution
Data
fusion

aggrega/on

Compute Cluster Analytics Cluster
Intelligent
Cache

Collector
Adapter
1

Custom

Adapter
2

Columnar

Storage

Anomaly

Detec/on

UserInterface
Time
series

analy/cs

Rules/
alerts

frame-‐work

M2MInterface
withcustomer
system
Data
streams

© 2015 Guavus, Inc. All rights reserved. 9
Conclusion
Lessons learned
•  No silver bullet, but multiple methods each with their own pros/cons
•  Simple and scalable solution
•  Adapt to:
–  data changes
–  customer needs
•  API design:
–  black-box approach: hide complexity from developers

© 2015 Guavus, Inc. All rights reserved.
QA
Nicolas Hohn, Director of Analytics
nicolas.hohn@guavus.com
2015

Your IT infrastructure is the lifeblood of your company. CTG 's end-to-end monitoring service, proactively monitors the health of your systems, allowing you to address minor issues before they escalate into larger issues that impact your business. CTG gives you peace of mind knowing your critical systems are being monitored 24x7x365, without any CAP EX costs, or having to dedicate staff time to maintaining a monitoring infrastructure.

Dynatrace FreeTrial Test Drive

Jerry Tan

Operational Analytics at Credit Suisse from ThousandEyes Connect

ThousandEyes

Unified Monitoring Webinar with Dustin Whittle

AppDynamics

How to deploy AppInternals in azure

Ian Downard

Are you a startup in Healthcare? So you have already envisioned an idea or product in Healthcare. A typical top down approach for making this successful is in a clear articulation of the product functions and its business model. This should guide to shape a beautiful application architecture that can be scaled and create a sustainable business model. Choice of Technology and the development model is another but equally important pillar in the success story. As you might have witnessed, open source technologies supplement exponential growth required for healthcare startups due to their flexible, scalable and cost efficient characteristics. In this webinar, we discussed the open technology solutions in the following areas: Remote Patient Monitoring Media for Personal and Social Interactions for Healthcare Analytics and Recommendation engines for Clinical Decision Support System

Service Assurance for Modern Apps - BigPanda NA SNO - April 2015 - Dan Turchin

PeopleReign, Inc.

LogicMonitor: An Overview

James McCabe

A high-level overview of LogicMonitor; the leading SaaS-based hybrid infrastructure performance monitoring solution on the market, changing the way enterprise organisations monitor dynamic, multi-cloud environments. Giving you hosted monitoring for your entire stack - servers, networks, containers, storage, applications, virtualisation, and cloud - from a single pane of glass. No dedicated servers required! If you want to talk AIOps, automation, and where our industry is going please hit me up with a message. I want to learn about you guys and what technology means for you!

Critical online success factors with dynatraceDynatraceANZ

Backup manager

Tracy Trottier

More than any other service an MSP provides, a good data backup is priceless. There’s no better feeling than quickly restoring your customer to full operation after they have experienced a major failure. Protecting customer data is a crucial component of the IT services you provide – and now you can do it easier and more affordably with Backup Manager from N-able. Join SolarWinds N-able Sales Engineer Jason Murphy as he highlights how Backup Manager can help you: • Meet SLA Targets • Protect critical business data • Minimize business disruption

MachinePulse Products

MachinePulse

MachinePulse provides rapidly scalable, end-to-end solutions to enable a smarter industrial Internet-of-Things. Our product offerings are modular and domain-independent. We offer a complete range of products from Industrial grade data acquisition devices to a robust and scalable cloud platform that can be deployed together or into existing IoT workflows with full flexibility. Using our products and rich domain expertise in the renewable energy sector, we have built applications to provide operational intelligence for the solar photovoltaic and wind energy industries. We are also working with our clients to deploy monitoring and analytics solutions for other industries. Our philosophy is “Any Device, Any Network, Any Protocol”.

Intelligent Digital Mesh Testing

Nagarro

Algolytics company Overview 2015

Algolytics

Story of Algolytic. Assisting companies with lots of data to get insights and predictions about customer behaviour, predicting risks based on statistical data analysis. Story of how we can help, how we changed from consulting company to product copmpany. Overview of data analytics and predictive analytics solutions - such as Advanced Miner platform for complex data mining, Social Network analysis, Data quality tools. Contact us for more info http://algolytics.com/blog/ check our news at the Blog: http://algolytics.com/blog/

Algolytics company Overview 2015

Algolytics (old account)

Prism presentation

Lakshmi .

Myths of validation

Jeff Thomas

Platforming the Major Analytic Use Cases for Modern Engineering

DATAVERSITY

We’ll describe some use cases as examples of a broad range of modern use cases that need a platform. We will describe some popular valid technology stacks that enterprises use in accomplishing these modern use cases of customer churn, predictive analytics, fraud detection, and supply chain management. In many industries, to achieve top-line growth, it is imperative that companies get the most out of existing customer relationships. Customer churn use cases are about generating high levels of profitable customer satisfaction through the use of knowledge generated from corporate and external data to help drive a more positive customer experience (CX). Many organizations are turning to predictive analytics to increase their bottom line and efficiency and, therefore, competitive advantage. It can make the difference between business success or failure. Fraudulent activity detection is exponentially more effective when risk actions are taken immediately (i.e., stop the fraudulent transaction), instead of after the fact. Fast digestion of a wide network of risk exposures across the network is required in order to minimize adverse outcomes. Supply chain leaders are under constant pressure to reduce overall supply chain management (SCM) costs while maintaining a flexible and diverse supplier ecosystem. They will leverage IoT, sensors, cameras, and blockchain. Major investments in advanced analytics, warehouse relocation, and automation, both in distribution centers and stores, will be essential for survival.

DISCUSSION ON DIGITAL OILFIELD FULL-FIELD OPTIMIZATION

wle-ss

Savvius_Introduction to workshop

STelligence Company

Industrial asset optimization overview slideshare

Genpact Ltd

Performance and penetration_testing_with_a_partner_how_to_start!

QATestLab

Even for small and midsize businesses, the security and response times of their IT systems are critical. Therefore it’s important to run the systems through thorough testing before launching them into production. But the execution of performance and penetration testing can be challenging. In-house teams often lack the time, experience or knowledge of the test tools and test techniques required.

Performance and penetration_testing_with_a_partner_how_to_start!

Sasha Kolomiichuk

API Days Paris - When RESTful may be considered harmful

Ross Garrett

My presentation from API Days Paris, December 2015. There is no single design decision that answers every question, but in the question of whether RESTful maybe considered harmful – this session will explore how some design decisions may be leading to poor user experiences and negatively affecting your business. We'll discuss the principles of reactive applications and how streaming APIs can deliver significant benefits over RESTful APIs.

What's hot

The Value of Options Analytics "as-a-Service"

Hanweck

Top 5 IT challenges for 2017

ManageEngine, Zoho Corporation

DCMS AKCP Product Presentation

Fanky Christian

[CLASS 2014] Palestra Técnica - Leonardo Scudere

TI Safe

Open Technology Solutions For Healthcare Startups

Harbinger Systems - HRTech Builder of Choice

Service Assurance for Modern Apps - BigPanda NA SNO - April 2015 - Dan Turchin

PeopleReign, Inc.

LogicMonitor: An Overview

James McCabe

Critical online success factors with dynatraceDynatraceANZ

Backup manager

Tracy Trottier

MachinePulse Products

MachinePulse

What's hot (10)

The Value of Options Analytics "as-a-Service"

Top 5 IT challenges for 2017

DCMS AKCP Product Presentation

[CLASS 2014] Palestra Técnica - Leonardo Scudere

Open Technology Solutions For Healthcare Startups

Service Assurance for Modern Apps - BigPanda NA SNO - April 2015 - Dan Turchin

LogicMonitor: An Overview

Critical online success factors with dynatrace

Backup manager

MachinePulse Products

Similar to Large scale predictive analytics for anomaly detection - Nicolas Hohn

Intelligent Digital Mesh Testing

Nagarro

Algolytics company Overview 2015

Algolytics

Algolytics company Overview 2015

Algolytics (old account)

Prism presentation

Lakshmi .

Myths of validation

Jeff Thomas

Platforming the Major Analytic Use Cases for Modern Engineering

DATAVERSITY

DISCUSSION ON DIGITAL OILFIELD FULL-FIELD OPTIMIZATION

wle-ss

Savvius_Introduction to workshop

STelligence Company

Industrial asset optimization overview slideshare

Genpact Ltd

Performance and penetration_testing_with_a_partner_how_to_start!

QATestLab

Performance and penetration_testing_with_a_partner_how_to_start!

Sasha Kolomiichuk

API Days Paris - When RESTful may be considered harmful

Ross Garrett

Hyper-connected apps: Hyper-Connected Apps: Testing Peripherals and Mobile Ap...

Infostretch

Remember how apps used to be? They connected to the internet and incorporated one device at most. Sure, testers had to factor in performance under different usage conditions, but we had great testing toolsets in place to deal with that. Fast forward to the present. These days sensors enable much more diverse functionality from our connected devices or smartphones and that means mobile app testing is no longer just about testing the app. Smartphones now include many sensors that interface with the ecosystem around them. These software and hardware components can all too easily play havoc with the apps themselves. During his presentation, Sivakumar Anna will share the complexity of peripherals testing. He will demonstrate sensor testing scenarios including location, camera, TouchID and Bluetooth. Drawing on his years of experience in this testing domain, Sivakumar will describe current industry best practices, specific challenges and how to overcome them. He will illustrate these unique issues with a case study involving device peripheral automation for one the leading medical device & solution providers.

CBN Supply Vending Presentation

Phil Filkovsky

Managing End User Expectations -- The A La Carte Strategy

Solarwinds N-able

N-able is about more than just supplying the MSP industry with the best automation software around. Based on our work with MSPs of all sizes from all corners of the world, we have developed A la Carte pricing designed to help you get the most ROI from your managed services offerings. Join Channel Manager David Weeks as he walks you through our A la Carte pricing model and explains the systematic approach N-able uses to help MSPs build their businesses and increase their revenue. This interactive and engaging webinar will highlight how MSPs around the world have continued to grow their businesses by adding new services that attract new customers.

Trends in Quality Assurance areaCiklum Ukraine

The Big Picture: Learned Behaviors in Churn

Cloudera, Inc.

The Big Picture webinar series explores how industries define their strategies for understanding their consumers better using data. From issuing better healthcare to smarter product and services recommendations, data is the fueling foundation for success. Cloudera is a modern platform that gives analytic access to users that need to understand their customers across multiple touch points and multiple enterprise systems. Cloudera not only unlocks the promise of true customer 360 but it also leverages advanced capabilities for data science and machine learning. In this webinar, we take a look at how data scientists can leverage Cloudera to identify and predict a common customer loyalty use case in telecommunications. We will explore the data, design our features, and then leverage Apache Spark to help us make some predictions on the accuracy of our finding. All within a secure and collaborative environments utilizing the Cloudera Data Science Workbench.

Trends in the quality assurance area

QA Club Kiev

Trends in the quality assurance areaSvitlana Dubyk

Witekio introducing-predictive-maintenance

Witekio

Similar to Large scale predictive analytics for anomaly detection - Nicolas Hohn (20)

Intelligent Digital Mesh Testing

Algolytics company Overview 2015

Prism presentation

Myths of validation

Platforming the Major Analytic Use Cases for Modern Engineering

DISCUSSION ON DIGITAL OILFIELD FULL-FIELD OPTIMIZATION

Savvius_Introduction to workshop

Industrial asset optimization overview slideshare

Performance and penetration_testing_with_a_partner_how_to_start!

API Days Paris - When RESTful may be considered harmful

Hyper-connected apps: Hyper-Connected Apps: Testing Peripherals and Mobile Ap...

CBN Supply Vending Presentation

Managing End User Expectations -- The A La Carte Strategy

Trends in Quality Assurance area

The Big Picture: Learned Behaviors in Churn

Trends in the quality assurance area

Witekio introducing-predictive-maintenance

More from PAPIs.io

Shortening the time from analysis to deployment with ml as-a-service — Luiz A...

PAPIs.io

The daily job of a Data Scientist ranges from a variety of tasks: improving models performance or dealing with framework structure implementations. Machine learning as a service, a hot topic in the field, implies thinking about architecture to allow constant improvements in performance for our products. This presentation shows one architecture design using RESTful resources, document oriented databases and pre-trained pipelines to achieve real-time predictions of time series with high availability, scalability and freedom to Data Scientists work directly on improving the accuracy rate of our products. We fine tunned to work on time series forecasting which is a very challenging field that still needs better solutions in terms of innovative modeling. During the presentation will be shown how these decisions keep our Data Scientists focused on working with real data and thinking about improvements that can reach a large volume of time series instead of singular and localized actions.

Feature engineering — HJ Van Veen (Nubank) @@PAPIs Connect — São Paulo 2017

PAPIs.io

Feature engineering is one of the most important, yet elusive, skills to master if you want to be a good data scientist. Machine learning competitions are hardly ever won with strong modeling techniques alone -- it is the combination of creative feature engineering and powerful modeling techniques that makes the difference. This tutorial will give the audience practical tips and tricks to improve the performance of machine learning algorithms. We will broadly look at feature engineering for applied machine learning, touching on subjects like: categorical vs. numerical variables, data cleaning, feature extraction, transformations, and imputation.

Extracting information from images using deep learning and transfer learning ...

PAPIs.io

For online businesses, recommender systems are paramount. There is an increasing need to take into account all the user information to tailor the best product offer, tailored to each new user. Part of that information is the content that the user actually sees: the visuals of the products. When it comes to products like luxury hotels, pictures of the room, the building or even the nearby beach can significantly impact users’ decision. In this talk, we will describe how we improved an online vacation retailer recommender system by using the information in images. We’ll explain how to leverage open data and pre-trained deep learning models to derive information on user taste. We will use a transfer learning approach that enables companies to use state of the art machine learning methods without needing deep learning expertise.

Discovering the hidden treasure of data using graph analytic — Ana Paula Appe...

PAPIs.io

Graphs are used to map relations on unstructured data. Companies’ data are most from database and mined using traditional data mining approach. However, model relational data as a graph can reveal useful insights and discovery relation among data that is ignored by traditional data mining techniques. In this work we used graphs to map physician relations using claim data as a proxy and this approach reveal interesting insights from health insurance company.

Deep learning for sentiment analysis — André Barbosa (elo7) @PAPIs Connect — ...

PAPIs.io

Building machine learning service in your business — Eric Chen (Uber) @PAPIs ...

PAPIs.io

Building machine learning applications locally with Spark — Joel Pinho Lucas ...

PAPIs.io

In times of huge amounts of heterogeneous data available, processing and extracting knowledge requires more and more efforts on building complex software architectures. In this context, Apache Spark provides a powerful and efficient approach for large-scale data processing. This talk will briefly introduce a powerful machine learning library (MLlib) along with a general overview of the Spark framework, describing how to launch applications within a cluster. In this way, a demo will show how to simulate a Spark cluster in a local machine using images available on a Docker Hub public repository. In the end, another demo will show how to save time using unit tests for validating jobs before running them in a cluster.

Battery log data mining — Ramon Oliveira (Datart) @PAPIs Connect — São Paulo ...

PAPIs.io

Battery life is critical for smart devices, but optimizing it requires cooperation from the entire software ecosystem. Wasteful software affects user perception about devices’ battery quality. Therefore, a large team within a producer of those smart devices is focused on identifying and correcting energy consumption bugs. Since the software ecosystem grows fast, that team faces a lot of suspect issues, from which only a small fraction turns out to be genuine. Our project aims to streamline energy-related bug processing in devices of the company and its partners, by automatically identifying anomalous behaviors related to battery drain using data mining and machine learning.

A tensorflow recommending system for news — Fabrício Vargas Matos (Hearst tv)...

PAPIs.io

News recommendations are particularly challenging given the high number of new contents produced every day and the fast deterioration of its value for the users, demanding models and infrastructure able to deal with those nuances and serve a newly trained model about 100 times per day. Attending this presentation you're going to follow a detailed overview of how R&D team of Hearst's TV division is putting together Google BigQuery, Kubernetes cluster and Tensorflow to build a hybrid recommendation system combining model-based matrix factorization, content recency, and content semantics through NLP.

Scaling machine learning as a service at Uber — Li Erran Li at #papis2016

PAPIs.io

Machine learning as a service (MLaS) is imperative to the success of many companies as many internal teams and organizations need to gain business intelligence from big data. Building a scalable MLaS in a very challenging problem. In this paper, we present the scalable MLaS we built for a company that operates globally. We focus on several scalability challenges and our technical solutions. Video at https://www.youtube.com/watch?v=MpnszJ_3Ong Couldn't attend PAPIs '16? Get access to the other presentations' slides and videos at https://gumroad.com/products/fehon/

Real-world applications of AI - Daniel Hulme @ PAPIs Connect

PAPIs.io

This talk will offer answers to the following questions: What is data-driven decision making? What is AI? What is Business Intelligence? Why are these concepts important? What are the biggest challenges and opportunities? Daniel is the CEO of Satalia that provides AI inspired solutions to solve industries hardest problems. He’s the co-founder of the ASI that transitions scientists into data scientists. Daniel has a MSci and EngD in AI from UCL, and is Director of UCL’s Business Analytics MSc; applying AI to solve business/social problems. Daniel has many Advisory and Executive positions, holds an international Kauffman Global Entrepreneur Scholarship and actively promotes innovation across the globe.

Past, Present and Future of AI: a Fascinating Journey - Ramon Lopez de Mantar...

PAPIs.io

Possibly the most important lesson we have learned after 60 years of AI research is that what seemed to be very difficult to achieve, such as accurate medical diagnosis to playing chess at the level of a Grand Master, turned out to be relatively easy whereas what seemed easy, such as visual object recognition or deep language understanding, turned out to be extremely difficult. In my talk I will try to explain the reasons for this apparent contradiction by briefly reviewing the past and present of AI and projecting it into the near future. Ramon Lopez de Mantaras is Research Professor of the Spanish National Research Council (CSIC) and Director of the Artificial Intelligence Research Institute of the CSIC. Technical Engineer EE (Electrical Engineering) from the Technical Engineering School of Mondragón (Spain) in 1973. Master of Sciences in Automatic Control from the University of Toulouse III (France) in 1974, Ph.D. in Physics from the University of Toulouse III (France), in 1977, with a thesis in Robotics (done at LAAS, CNRS). Master of Science in Engineering (ComputerScience) from the University of California at Berkeley (USA) in 1979. Ph.D. in Computer Science, from the Technical University of Catalonia, Barcelona (Spain) in 1981.

Revolutionizing Offline Retail Pricing & Promotions with ML - Daniel Guhl @ P...

PAPIs.io

Everybody uses price promotions in retail. However, individual pricing is seldom used, particularly in offline retail. Marketing literature has been advocating the use of individual price discrimination for decades. Furthermore, product recommendations, ever-present in e-commerce, are also not often found in offline retail. We show the machine learning driven system behind a new promotion channel that enables retailers and manufacturers alike to target individual customers in offline retail. Lessons learned, technologies used, and machine learning approaches driving our system will be shown. Daniel Guhl has a background in economics & marketing, and got interested in data modeling during his Ph.D.. Currently, he is working as a data scientist at a Berlin based Start-up and is pursuing a postdoc at Humboldt University. He enjoys learning everyday and focuses on solving real world problems.

Demystifying Deep Learning - Roberto Paredes Palacios @ PAPIs Connect

PAPIs.io

Deep Learning (DL) is becoming a big tsunami in the Machine Learning community. This talk aims at introducing DL, its motivation and main techniques. However, part of this talk is also devoted to demystify DL. What are the main advantages but also the main drawbacks of DL?. And what are the key issues that the practitioners have to consider? Roberto Paredes is an Associate Professor at Departamento de Sistemas Informáticos y Computación DSIC of the Universidad Poliécnica de Valencia UPV. He belongs to the Pattern Recognition and Human Language Technologies Research Centre PRHLT. Roberto Paredes is the Director of the PRHLT and the President of the Spanish AERFAI Association. His main research interests are around the statistical learning, machine learning and more recently neural networks and deep learning.

Predictive APIs: What about Banking? - Natalino Busa @ PAPIs Connect

PAPIs.io

The best services have one thing in common: a superb customer experience. Banking services are no exception to this rule, and indeed the quest for an effortless, well informed, and personalized customer experience is one of the main goals of today's innovation in digital banking services. According to what Maslow has described in his "pyramid of needs", customers are seeking a more intimate and meaningful experience where banking services can actively assist the customer in performing and managing their financial life. Predictive APIs have a fundamental role in all this, as they enable a new set of customer journeys such as automatic categorization of transactions, detecting and alerting recurrent payments, pre-approving credit requests or provide better tools to fight fraud without limiting legitimate customer transactions. In this talk, I will focus on how to provide better banking services by using predictive APIs. I will describe the path on how to get there and the challenges of implementing predictive APIs in a strictly audited and regulated domain such as banking. Finally, I will briefly introduce a number of data science techniques to implement those customer journeys and describe how big/fast data engineering can be used to realize predictive data pipelines. Natalino is currently Enterprise Data Architect at ING in the Netherlands, where leads the strategy, definition, design and implementation of big/fast data solutions for data-driven applications, for personalized marketing, predictive analytics, and fraud/security management. All-round Software Architect, Data Technologist, Innovator, with 15+ years experience in research, development and management of distributed architectures and scalable services and applications.

Microdecision making in financial services - Greg Lamp @ PAPIs Connect

PAPIs.io

Fintech startups are taking business away from traditional institutions like banks, exchanges, and brokerages. One of the reasons that these startups are able to compete with $30B+ behemoths like Credit Suisse and Goldman Sachs is their advanced decision making capabilities. By leveraging new data sources and better predictive analytics, companies like Ferratum Bank can make more accurate decisions in a fraction of the time. This talk will cover: Types of decisions you can automate Challenges in building predictive, financial apps First-hand, real-world examples Greg Lamp is the co-Founder and CTO of Yhat. In this role, Greg leads development of Yhat's core products and infrastructure and is the principal architect of the company's cloud and on-premise enterprise software applications. Greg was previously a product manager at OnDeck, a fintech startup in New York and before that an analyst at comScore. Greg is a graduate of the University of Virginia.

Engineering the Future of Our Choice with General AI - JoEllen Lukavec Koeste...

PAPIs.io

What is the future we want to create, and what can we do – starting today – to actively shape that future with general AI? This talk outlines a vision for the future of humankind once AI reaches human or superhuman levels, and leads the audience through the steps one research group is taking to get there. From the economics of smart robots and job replacement, to bionic humans exploring the universe through space travel, the talk offers a window into the work of 30 researchers focused on AI development and safety, and explains what attendees can do themselves to help make that future happen. JoEllen is the AI Safety Ambassador and Head of PR for GoodAI, a Prague-based general AI research and development company. A high school teacher by trade, she has a bachelor’s degrees in English and Philosophy from Seattle University, a master’s degree in Transatlantic Studies from Charles University in Prague, and is the recipient of Fulbright grant. JoEllen is particularly interested in how AI will affect international government and political relations.

Distributed deep learning with spark on AWS - Vincent Van Steenbergen @ PAPIs...

PAPIs.io

Training deep networks is a time-consuming process, with networks for object recognition often requiring multiple days to train. For this reason, leveraging the resources of a cluster to speed up training is an important area of work. In this talk we'll show how to use an AWS Spark cluster to train a model quickly from a laptop at a very little cost (around 10€). Vincent Van Steenbergen is a freelance (big) data engineer who's working on a range of international projects, implementing systems able to handle terabytes of data, usually involving Spark, Scala, Kafka, Hadoop and Cassandra. His main interest right now is applying these techniques to solve machine learning problems. Vincent was previously a technical architect at Property. Works, a real estate startup in London and before that an R&D engineer at IDAaaS.

How to predict the future of shopping - Ulrich Kerzel @ PAPIs Connect

PAPIs.io

Shopping, or as the people on the other side of the counter call it, retail has become the number one breeding ground for predictive applications in the enterprise. What started as simple recommendation engines has evolved into a complex and powerful ecosystem of predictive applications that affect core processes such as pricing, replenishment and staff planning. In this talk, Ulrich Kerzel will share impact and experiences from building and operating predictive applications for large retailers, and explain why the future of retail is as much a science as an art. Dr. Ulrich Kerzel is a Senior data scientists at Blue Yonder and renowned scientist with research experience at the University of Cambridge and CERN. Ulrich Kerzel earned his PhD under Professor Dr Feindt at the US Fermi National Laboratory and at that time made a considerable contribution to core technology of NeuroBayes. After his PhD, he went to the University of Cambridge, were he was a Senior Research Fellow at Magdelene College. His research work focused on complex statistical analyses to understand the origin of matter and antimatter using data from the LHCb experiment at the Large Hadron Collider at CERN, the world’s biggest research institute for particle physics. He continued this work as a Research Fellow at CERN before he came to Blue Yonder as a senior data scientist.

The emergent opportunity of Big Data for Social Good - Nuria Oliver @ PAPIs C...

PAPIs.io

We live in a world of data, of big data. A big portion of this data has been generated by humans, and particularly through their mobile phones. In fact, there are almost as many mobile phones in the world as humans. The mobile phone is the piece of technology with the highest levels of adoption in human history. We carry them with us all through the day (and night, in many cases), leaving digital traces of our physical interactions. Mobile phones have become sensors of human activity in the large scale and also the most personal devices. In my talk, I will present some of the work that we are doing at Telefonica Research in the area of human behavior understanding from data captured with mobile phones, and particularly our work in the area of Big Data for Social Good. I will highlight opportunities but also challenges that we would need to address in order to truly leverage this opportunity. Nuria Oliver is a computer scientist and Scientific Director at Telefónica. She holds a Ph.D. from the Media Lab at MIT. She is one of the most cited female computer scientist in Spain, with her research having been cited by more than 8900 publications. She is well known for her work in computational models of human behavior, human computer-interaction, intelligent user interfaces, mobile computing and big data for social good.

More from PAPIs.io (20)