Horst Samulowitz of IBM Research covers his exciting work bringing cognitive to data science.

1. © 2015 IBM Corporation
IBM Research @World of Watson 2016 Evolving Education with Cognitive &Data Science
Cognitive Assistant for Data Scientists [CADS]
Automated Machine Learning and Data Science Group
MARIA BUTRICO, GREGORY BRAMBLE, ANDRE CUNHA, ELIAS KHALIL,
UDAYAN KHURANA, PETER KIRCHNER, TIM KLINGER, FATEMEH NARGESIAN,
SRINIVASAN PARTHASARATHY, CHANDRA REDDY, ANTON RIABOV,
HORST SAMULOWITZ, GERRY TESAURO, DEEPAK TURAGA

2. © 2015 IBM Corporation
Research Vision: Automate Data Science
• Step 1: Compare Data Scientist with and without computer-based augmentation
• Show that computer-augmented data science can reduce time-to-result by an order of magnitude and
improve quality of results
• Step 2: Self-learn and validate using open competitions or evaluations (e.g., Kaggle,
OpenML), IBM customer engagements, and incorporation of Data Science knowledge
• Step 3: Commoditize data science through services on the cloud
IBM Research2

3. © 2015 IBM Corporation
Executive Summary of Current CADS
• Objective: Bring automation into key areas of large-scale data analysis tasks
• Overcome “analytic decision overload” for Data Scientists
• Enable Data Scientist to:
• view and interact with decision making process in an online fashion
• obtain rapid insights from data to answer key questions
• CADS System Demonstration
• A system for automated training of models for supervised data mining tasks
• First of its kind prototype
• Learning and Planning based principled exploration of analytic choices
• Cross-platform analytic deployments on Big Data platforms  Cloud
• Showcased running system at top Machine Learning venue (e.g., AAAI 2015, NIPS 2015, IJCAI 2016)
IBM Research3

4. © 2015 IBM Corporation
Data Scientist Workflow
PreparationIngestion Selection Generation Transform Model Operations
•Retrieval
•Storage
•Formatting
•…
•Missing Values
•Smoothing
•Normalization
•…
•Data Source Selection
•Data Composition
•Data Linkage
•Concept Extraction
•Filtering
•…
•Aggregation
•Construction
•Labelling
•Data Augmentation
•…
•Feature selection
•Feature space
transformation
•…
•Regression
•Classification
•…
•(Re)-Deployment, Re-
Training, Monitor
•Explanations
•Written Report
•Best-Worst case
scenarios
•…
Noisy Sensor StreamsOil Rig Monitoring Cleaned sensor streams Model
(e.g. ConocoPhillips)
IBM Research4

5. © 2015 IBM Corporation
Challenges of a Data Scientist
PreparationIngestion Selection Generation Transform Model Operations
•Retrieval
•Storage
•Formatting
•…
•Missing Values
•Smoothing
•Normalization
•…
•Data Source Selection
•Data Composition
•Data Linkage
•Concept Extraction
•Filtering
•…
•Aggregation
•Construction
•Labelling
•Data Augmentation
•…
•Feature selection
•Feature space
transformation
•…
•Regression
•Classification
•…
•(Re)-Deployment, Re-
Training, Monitor
•Explanations
•Written Report
•Best-Worst case
scenarios
•…
• Combinatorial Explosion in choices of algorithms (and implementations/platforms),
their parameters and their compositions
• Long, complex, tedious and sometimes artful process requiring substantial time + effort
IBM Research5

6. © 2015 IBM Corporation
Example: Choices at Model Building Stage
IBM Research6
• More than 50 different algorithms: SVM, Neural Net, DecisionTrees/Forests, Naïve
Bayes, Regression, SMO, k-nearest Neighbor, Clustering, Rules, …
• Combinatorially explosive number of parameter choices per algorithm: kernel type,
pruning strategy, number of trees in a forest, learning rate, …
• Wide variation in performance across different algorithm implementations (e.g., SPSS
vs Python vsWEKA vs SPARK …)
• User-Defined algorithms
• Substantial cost in user and compute time
• User spends time on trying new combinations and parameters
• Computational cost for training a single SVM can exceed 24h
• Selection commonly based on data scientist bias
• Each additional pipeline stage increases complexity dramatically!

7. © 2015 IBM Corporation
Serious external interest in Automation!
7
MIT Data Science MachineMIT Data Science Machine

8. © 2015 IBM Corporation
Data Scientist Workflow With CADS
8
Science of
Analytics
Repository
Deployed Analytic
R, WEKA, SPARK, Python…
UserInterface(orREST-APIdirectly)
1
4
5
Deployed Analytic
62
Learning Controller
Tactical
Planner
Orchestrator
and Scheduler
Learning Controller
Analytic Monitoring
and Adaptation
Analytic
Platforms
Knowledge Acquisition
External Knowledge
about Analytics
3
HIGGS BOSON DATA
AI technology automatically determines best analytics pipeline
Data Scientist can interact with System
Cross-Platform Deployment and Evaluation
Input: Binary Classification Problem
Submit to
CADS

9. © 2015 IBM Corporation
Live Demo
CADS (5-minute AVI Video / 213 MByte Download)
https://w3-
connections.ibm.com/communities/service/html/commu
nityview?communityUuid=a9fa82ae-09fd-4913-a935-
0f6b11926fba#fullpageWidgetId=Wab5c0428d070_41b0_
83b0_8ad8aa7afbf2&file=a1c3683d-01d1-438d-9b33-
35ad147bfe2e
IBM Research9

10. © 2015 IBM Corporation
Bootstrapping the automation of Data Engineering with
automated model selection
IBM Research10

11. © 2015 IBM Corporation
Largely Automated [CADS]
Data Scientist Workflow
PreparationIngestion Selection Generation Transform Model
•Retrieval
•Storage
•Formatting
•…
•Missing Values
•Smoothing
•Normalization
•…
•Data Source Selection
•Data Composition
•Data Linkage
•Concept Extraction
•Filtering
•…
•Aggregation
•Construction
•Labelling
•Data Augmentation
•…
•Feature selection
•Feature space
transformation
•…
•Regression
•Classification
•…
• We have automated large parts of transformation and model building stage
• How can we automate the upstream stages? Bootstrap automation with existing automation
Automated Feedback
Operations
•(Re)-Deployment, Re-
Training, Monitor
•Explanations
•Written Report
•Best-Worst case
scenarios
•…
IBM Research11

12. © 2015 IBM Corporation
Largely Automated [CADS]
Towards Automation of Data Engineering through
Predictive Modeling
PreparationIngestion Selection Generation Transform Model
•Retrieval
•Storage
•Formatting
•…
•Missing Values
•Smoothing
•Normalization
•…
•Data Source Selection
•Data Composition
•Data Linkage
•Concept Extraction
•Filtering
•…
•Aggregation
•Construction
•Labelling
•Data Augmentation
•…
•Feature selection
•Feature space
transformation
•…
•Regression
•Classification
•…
• Approach: Guide ‘Selection’, ‘Preparation’ and ‘Generation’ tools by automated stages
• Possible data and analytics space is enormous and one cannot try all possible combinations of
data, tools and options in a brute force fashion
Automated Feedback
Operations
•(Re)-Deployment, Re-
Training, Monitor
•Explanations
•Written Report
•Best-Worst case
scenarios
•…
IBM Research12

13. © 2015 IBM CorporationIBM Research13

14. © 2015 IBM Corporation14
• Extend for other problems, data types, scale and user requirements (e.g., time series,
text analytics, unsupervised data exploration)
• Self-Learning and Adaptation
• Operationalization and Maintenance
• Pipeline deployment, monitoring and adaptation
• Build first-ever conversational data science system with CADS +Watson QA
• Create a course that use automated ML and Data Science for teaching
• As part of ML101: Diversity of models, learning curves, performance dependence based on properties of data, etc.
Next steps

15. © 2015 IBM Corporation
Notices and disclaimers
• Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from
IBM.
• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.
• Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial
publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS"
WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION,
INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted
according to the terms and conditions of the agreements under which they are provided.
• IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty
terms apply.”
• Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.
• Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have
used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.
• References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which
IBM operates or does business.
• Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and
discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their
specific situation.
• It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of
any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not
provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.
15 12/1/2016
World of Watson
2016

16. © 2015 IBM Corporation
Notices and disclaimers continued
• Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM
has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party
products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR
IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
• The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other
intellectual property right.
• IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™,
FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on
Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®,
PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling
Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of
International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other
companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
16 12/1/2016
World of Watson
2016