Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Apache deep learning 202 Washington DC - DWS 2019


Published on

Apache Deep Learning - Apache MXNet, Apache NiFi
A quick integration for Big Data Engineers on how to use Apache MXNet with Apache NiFi in streams, at the edge, in a processor and on Linux and OSX.

Published in: Data & Analytics
  • Be the first to comment

Apache deep learning 202 Washington DC - DWS 2019

  1. 1. @PaaSDev Apache Deep Learning 202 v1.00 (For Data Engineers) Timothy Spann
  2. 2. @PaaSDev Disclaimer • This is my personal integration and use of Apache software, no companies vision. • This document may contain product features and technology directions that are under development, may be under development in the future or may ultimately not be developed. This is Tim’s ideas only. • Technical feasibility, market demand, user feedback, and the Apache Software Foundation community development process can all effect timing and final delivery. • This document’s description of these features and technology directions does not represent a contractual commitment, promise or obligation from Hortonworks to deliver these features in any generally available product. • Product features and technology directions are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. • Since this document contains an outline of general product development plans, customers should not rely upon it when making a purchase decision.
  3. 3. @PaaSDev There are some who call him... DZone Zone Leader and Big Data MVB; Princeton Future of Data Meetup
  4. 4. @PaaSDev Deep Learning for Big Data Engineers Multiple users, frameworks, languages, devices, data sources & clusters BIG DATA ENGINEER • Experience in ETL • Coding skills in Scala, Python, Java • Experience with Apache Hadoop • Knowledge of database query languages such as SQL • Knowledge of Hadoop tools such as Hive, or Pig • Expert in ETL (Eating, Ties and Laziness) • Social Media Maven • Deep SME in Buzzwords • No Coding Skills • Interest in Pig and Falcon CAT AI • Will Drive your Car • Will Fix Your Code • Will Beat You At Q-Bert • Will Not Be Discussed Today • Will Not Finish This Talk For Me, This Time
  5. 5. @PaaSDev • Cloud ready • Python, C++, Scala, R, Julia, Matlab, MXNet.js and Perl Support • Experienced team (XGBoost) • AWS, Microsoft, NVIDIA, Baidu, Intel • Apache Incubator Project • Run distributed on YARN and Spark • In my early tests, faster than TensorFlow. (Try this yourself) • Runs on Raspberry PI, NVidia Jetson Nano and other constrained devices
  6. 6. @PaaSDev • Great documentation • Crash Course • Gluon (Open API), GluonCV, GluonNLP • Keras (One API Many Runtime Options) • Great Python Interaction. Java and Scala APIs! • Open Source Model Server Available • ONNX (Open Neural Network Exchange Format) Support for AI Models • Now in Version 1.4.0! • Rich Model Zoo! • Math Kernel Library and NVidia CUDA Optimizations • TensorBoard compatible pip3.6 install -U keras-mxnet pip3.6 install --upgrade mxnet pip3.6 install gluonnlp pip3.6 install gluoncv pip3.6 install mxnet-mkl>=1.3.0 --upgrade
  7. 7. @PaaSDev Apache MXNet GluonCV Zoo • ResNet152_v2 • MobileNetV2_0.25 • VGG19_bn • SqueezeNet1.1 • DenseNet201 • Darknet53 • InceptionV3 • CIFAR_ResNeXt29_16x64 • yolo3_darknet53_voc • ssd_512_mobilenet1.0_coco • faster_rcnn_resnet101_v1d_coco • yolo3_darknet53_coco • FCN model on PASCAL VOC
  8. 8. @PaaSDev Object Detection: GluonCV YOLO v3 and Apache NiFi
  9. 9. @PaaSDev Object Detection: Faster RCNN with GluonCV net = gcv.model_zoo.get_model(faster_rcnn_resnet50_v1b_voc, pretrained=True) Faster RCNN model trained on Pascal VOC dataset with ResNet-50 backbone
  10. 10. @PaaSDev Instance Segmentation: Mask RCNN with GluonCV net = model_zoo.get_model('mask_rcnn_resnet50_v1b_coco', pretrained=True) Mask RCNN model trained on COCO dataset with ResNet-50 backbone
  11. 11. @PaaSDev Semantic Segmentation: DeepLabV3 with GluonCV model = gluoncv.model_zoo.get_model('deeplab_resnet101_ade', pretrained=True) GluonCV DeepLabV3 model on ADE20K dataset This one is a bit slower.
  12. 12. @PaaSDev Semantic Segmentation: Fully Convolutional Networks model = gluoncv.model_zoo.get_model(‘fcn_resnet101_voc ', pretrained=True) GluonCV FCN model on PASCAL VOC dataset
  13. 13. @PaaSDev Simple Pose Estimation pip3.6 install gluoncv --pre --upgrade yolo3_mobilenet1.0_coco + simple_pose_resnet18_v1b
  14. 14. @PaaSDev Apache MXNet Native Processor for Apache NiFi This is a beta, community release by me using the new beta Java API for Apache MXNet.
  15. 15. @PaaSDev Using Apache MXNet on The Edge with Sensors and Google Coral (MiNiFi)