In this talk, we will walk you through the simple steps to build and deploy machine learning for sentiment analysis and YOLO object detection as part of an IoT application that starts from devices collecting sensor data and camera images with MiNiFi. This data is streamed to Apache NiFi which integrates with Cloudera Data Science Workbench for classification with models in real-time as part of the real-time event stream. We parse, filter, fork, sort, query with SQL, dissect, enrich, transform, join and aggregate data as it is ingested. The data is landed in a Cloudera data store in the cloud for batch and interactive analytics with Spark, Hive, Phoenix, HBase, Druid, Kudu and Impala. Reference: https://blog.cloudera.com/blog/2019/02/integrating-machine-learning-models-into-your-big-data-pipelines-in-real-time-with-no-coding/ https://community.hortonworks.com/content/kbentry/239858/integrating-machine-learning-models-into-your-big.html https://community.hortonworks.com/articles/239961/using-cloudera-data-science-workbench-with-apache.html Speaker Timothy Spann, Senior Solutions Engineer Cloudera

1. Edge to AI: Analytics from Edge to Cloud with
Efficient Movement of Machine Data
TIMOTHY SPANN, Sr. SE, Cloudera

2. 2 © Cloudera, Inc. All rights reserved.
DISCLAIMER
The information in this document is proprietary to Cloudera. No part of this document may be reproduced,
copied or transmitted in any form for any purpose without the express prior written permission of Cloudera.
This document is a preliminary version and not subject to your license agreement or any other agreement
with Cloudera. This document contains only intended strategies, developments and functionalities of
Cloudera products and is not intended to be binding upon Cloudera to any particular course of business,
product strategy and/or development. Please note that this document is subject to change and may be
changed by Cloudera at any time without notice.
Cloudera assumes no responsibility for errors or omissions in this document. Cloudera does not warrant
the accuracy or completeness of the information, text, graphics, links or other items contained within this
material. This document is provided without a warranty of any kind, either express or implied, including but
not limited to the implied warranties of merchantability, fitness for a particular purpose or non-infringement.
Cloudera shall have no liability for damages of any kind including without limitation direct, special, indirect
or consequential damages that may result from the use of these materials. The limitation shall not apply in
cases of gross negligence.

3. 3© Cloudera, Inc. All rights reserved.

4. 4© Cloudera, Inc. All rights reserved.
MACHINE LEARNING PHASES
Where to Connect to Apache NiFi

6. 6© Cloudera, Inc. All rights reserved.

7. 7© Cloudera, Inc. All rights reserved.

8. Speed of Data Model Training Model Scoring Use Case
Batch
Batch
Batch
Batch Reporting,
Analytics,
Applications
Online
DS Applications/
Interactive
Dashboards
Streaming
In-stream
Streaming
Applications
Incremental/Online In-stream
Streaming
Applications
Training, Scoring and Monitoring

9. 9 © Cloudera, Inc. All rights reserved.
INTRODUCING MODELS
Machine learning models as one-click microservices (REST APIs)
Model APIs made easy!
1. Choose Python/R file, e.g. score.py
2. Choose function, e.g. forecast
f = open('model.pk', 'rb')
model = pickle.load(f)
def forecast(data):
return model.predict(data)
3. Choose resources

10. 10© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCH
Select a Project, Create a Session, Load Libraries and Data
CLOUDERA DATA SCIENCE WORKBENCH

11. 11© Cloudera, Inc. All rights reserved.
Load a File and Run It
CLOUDERA DATA SCIENCE WORKBENCH

12. 12© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCH
Install Python Libraries for Python 2 or Python 3
CLOUDERA DATA SCIENCE WORKBENCH

13. 13© Cloudera, Inc. All rights reserved.
Test your function with an argument
CLOUDERA DATA SCIENCE WORKBENCH

14. 14© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCH
Create a model from that file and function
CLOUDERA DATA SCIENCE WORKBENCH

15. 15© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHList All The Models
CLOUDERA DATA SCIENCE WORKBENCH

16. 16© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHDeploy the Model
CLOUDERA DATA SCIENCE WORKBENCH

17. 17© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHCheckout The Build
CLOUDERA DATA SCIENCE WORKBENCH

18. 18© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHTest the Model
CLOUDERA DATA SCIENCE WORKBENCH

19. 19© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHValidate the Model Results
CLOUDERA DATA SCIENCE WORKBENCH

20. 20© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHMonitor The Running Models
CLOUDERA DATA SCIENCE WORKBENCH

21. 21© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHInvoke the Model From Apache NiFi In Flow
CLOUDERA DATA SCIENCE WORKBENCH

22. 22© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHQuery Results of Classification in Flow
{ "class1": "cat", "cpu": 38.3, "end": "1549672761.1262221",
"host": "gluoncv-apache-mxnet-29-50-7fb5cfc5b9-sx6dg", "memory": 14.9,
"pct1": "98.15670800000001",
"shape": "(1, 3, 566, 512)", "systemtime": "02/09/2019 00:39:21",
"te": "3.380652666091919"
}
CLOUDERA DATA-IN-MOTION (APACHE NIFI)

23. 23© Cloudera, Inc. All rights reserved.
CLOUDERA DATA SCIENCE WORKBENCHReferences
● https://blog.cloudera.com/blog/2019/02/integrating-machine-learning-models-into-your-big-data-
pipelines-in-real-time-with-no-coding/
● https://community.hortonworks.com/articles/239961/using-cloudera-data-science-workbench-with-
apache.html
● https://community.hortonworks.com/content/kbentry/239858/integrating-machine-learning-models-
into-your-big.html
● https://github.com/tspannhw/nifi-cdsw-gluoncv
CLOUDERA