Hands on image recognition with scala spark and deep learning4j

•

1 like•451 views

The document provides an agenda for a presentation on hands-on image recognition with Scala, Spark and DeepLearning4j. The presentation introduces Scala for data science, deep learning concepts like neural networks, Apache Spark for large-scale data processing, and DeepLearning4j for distributed deep learning on the JVM. It also demonstrates an end-to-end example of image recognition using these tools with live coding examples.

Technology

Hands-on Image
Recognition with Scala,
Spark and
DeepLearning4j
Presented by
Guglielmo Iozzia
Kiev, April 20th 2018

Something About Me
 Big Data Delivery Lead at Optum (UHG)
 Previously at IBM and FAO of the UN
 Current fields of expertise are Big Data,
ML/DL and DevOps
 Past experience in JVM languages (Java,
Groovy, Scala) development, test
automation, CI/CD

Something About Me
 Author of the upcoming book
“Hands-on Deep Learning
with Apache Spark”
 I love preparing
home-made pizza

Agenda
 Scala for Data Science
 Deep Learning
 Definition
 Multilayer Neural Networks
 Apache Spark
 Overview
 DeepLearning4j
 ETL
 CNN
 Image Recognition Example

Scala for Data Science
 Most part of the systems/tools in the Big
Data/ML space run on the JVM.
 Robustness and performance when it comes
to production system and large datasets.
 Plenty of frameworks available:

Deep Learning
 It is a subset of machine learning that can
solve particularly hard and large-scale
problems in areas such as natural language
processing or image classification.
 It is based on Multilayered Neural Networks.

Multilayer Neural Networks
 The first layer, called
the input layer, is
where features are
input.
 The last layer is
called the output
layer.
 Any layer that is not
an input or output
layer is a hidden
layer.

Multilayer Neural Networks
 Different variations.
 CNN: used in image
recognition.
 RNN: used in NLP.

Apache Spark
 It is an Open Source fast cluster-computing
platform.
 Data are loaded in distributed memory (RAM)
over a cluster of machines.
 Compared to Hadoop MapReduce, it runs
programs up to 100x faster when the data fits
in memory, or 10x faster on disk.
 It provides support for Java, Scala, Python
and R.

DeepLearning4j: Overview
 It is an Open Source distributed Deep
Learning library for the JVM.
 It is integrated with Hadoop and Spark.
 It provides support for both CPUs and GPUs.
 It allows the import of neural net models from
the most major frameworks via Keras.

DeepLearning4j: libraries
 Deeplearning4J: Neural Net Platform
 ND4J: Numpy for the JVM
 DataVec: Tool for Machine Learning ETL
Operations
 JavaCPP: The Bridge Between Java and
Native C++
 Arbiter: Evaluation Tool for Machine
Learning Algorithms
 RL4J: Deep Reinforcement Learning for the
JVM.

Useful Links
Apache Spark: http://spark.apache.org/
DeepLearning4j: https://deeplearning4j.org/
Keras: https://keras.io/

Wrap Up
Linkedin: https://ie.linkedin.com/in/giozzia
Twitter: @GuglielmoIozzia
Blog: googlielmo.blogspot.com
DZone: https://dzone.com/users/2532948/virtualramblas.html
Hands-On Deep Learning with Apache Spark:
https://www.packtpub.com/big-data-and-business-intelligence/hands-
deep-learning-apache-spark

At the recent Spark & Machine Learning Meetup in Brussels, François Garillot of Skymind delivered this lightning talk to a sold-out crowd. Specifically, François offered a tour of the DeepLearning4J architecture intermingled with applications. He went over the main blocks of this deep learning solution for the JVM that includes GPU acceleration, a custom n-dimensional array library, a parallelized data-loading swiss army tool, deep learning and reinforcement learning libraries — all with an easy-access interface. Along the way, he pointed out the strategic points of parallelization of computation across machines and gave insight on where Spark helps — and where it doesn't.

Anomaly detection in deep learning (Updated) English

Adam Gibson

Hadoop Summit 2014 - San Jose - Introduction to Deep Learning on Hadoop

Josh Patterson

As the data world undergoes its cambrian explosion phase our data tools need to become more advanced to keep pace. Deep Learning has emerged as a key tool in the non-linear arms race of machine learning. In this session we will take a look at how we parallelize Deep Belief Networks in Deep Learning on Hadoop’s next generation YARN framework with Iterative Reduce. We’ll also look at some real world examples of processing data with Deep Learning such as image classification and natural language processing.

Convolutional Neural Networks at scale in Spark MLlib

DataWorks Summit

Jeremy Nixon will focus on the engineering and applications of a new algorithm built on top of MLlib. The presentation will focus on the methods the algorithm uses to automatically generate features to capture nonlinear structure in data, as well as the process by which it’s trained. Major aspects of that are the compositional transformations over the data, convolution, and distributed backpropagation via SGD with adaptive gradients and an adaptive learning rate. Applications will look into how to use convolutional neural networks to model data in computer vision, natural language and signal processing. Details around optimal preprocessing, the type of structure that can be learned, and managing its ability to generalize will inform developers looking to apply nonlinear modeling tools to problems that they face.

Introduction to Keras

John Ramey

An introduction to Keras, a high-level neural networks library written in Python. Keras makes deep learning more accessible, is fantastic for rapid protyping, and can run on top of TensorFlow, Theano, or CNTK. These slides focus on examples, starting with logistic regression and building towards a convolutional neural network. The presentation was given at the Austin Deep Learning meetup: https://www.meetup.com/Austin-Deep-Learning/events/237661902/

Kaz Sato, Evangelist, Google at MLconf ATL 2016

MLconf

Machine Intelligence at Google Scale: Tensor Flow and Cloud Machine Learning: The biggest challenge of Deep Learning technology is the scalability. As long as using single GPU server, you have to wait for hours or days to get the result of your work. This doesn’t scale for production service, so you need a Distributed Training on the cloud eventually. Google has been building infrastructure for training the large scale neural network on the cloud for years, and now started to share the technology with external developers. In this session, we will introduce new pre-trained ML services such as Cloud Vision API and Speech API that works without any training. Also, we will look how TensorFlow and Cloud Machine Learning will accelerate custom model training for 10x – 40x with Google’s distributed training infrastructure.

Deep learning on Hadoop/Spark -NextML

Adam Gibson

This technical session provides a hands-on introduction to TensorFlow using Keras in the Python programming language. TensorFlow is Google’s scalable, distributed, GPU-powered compute graph engine that machine learning practitioners used for deep learning. Keras provides a Python-based API that makes it easy to create well-known types of neural networks in TensorFlow. Deep learning is a group of exciting new technologies for neural networks. Through a combination of advanced training techniques and neural network architectural components, it is now possible to train neural networks of much greater complexity. Deep learning allows a model to learn hierarchies of information in a way that is similar to the function of the human brain.

KERAS Python Tutorial

MahmutKAMALAK

Deep Learning on Qubole Data Platform

Shivaji Dutta

Smart Data Conference: DL4J and DataVec

Josh Patterson

Spark MLlib and Viral Tweets

Asim Jalis

eScience Cluster Arch. OverviewFrancesco Bongiovanni

BigDL webinar - Deep Learning Library for Spark

DESMOND YUEN

Social Networks Analysis

Joud Khattab

Anomaly Detection at Scale

Jeff Henrikson

Matrix Factorizations at Scale: a Comparison of Scientific Data Analytics on ...

Databricks

Explore the trade-offs of performing linear algebra for data analysis and machine learning using Apache Spark, compared to traditional C and MPI implementations on HPC platforms. Apache Spark is designed for data analytics on cluster computing platforms with access to local disks and is optimized for data-parallel tasks. This session will examine three widely-used and important matrix factorizations: NMF (for physical plausibility), PCA (for its ubiquity) and CX (for data interpretability). Learn how these methods are applied to terabyte-sized problems in particle physics, climate modeling and bioimaging, as use cases where interpretable analytics is of interest. The data matrices are tall-and-skinny, which enable the algorithms to map conveniently into Spark’s data-parallel model. We perform scaling experiments on up to 1600 Cray XC40 nodes, describe the sources of slowdowns and provide tuning guidance to obtain high performance. Based on joint work with Alex Gittens and many others.

Tom Peters, Software Engineer, Ufora at MLconf ATL 2016

MLconf

Say What You Mean: Scaling Machine Learning Algorithms Directly from Source Code: Scaling machine learning applications is hard. Even with powerful systems like Spark, Tensor Flow, and Theano, the code you write has more to do with getting these systems to work at all than it does with your algorithm itself. But it doesn’t have to be this way! In this talk, I’ll discuss an alternate approach we’ve taken with Pyfora, an open-source platform for scalable machine learning and data science in Python. I’ll show how it produces efficient, large scale machine learning implementations directly from the source code of single-threaded Python programs. Instead of programming to a complex API, you can simply say what you mean and move on. I’ll show some classes of problem where this approach truly shines, discuss some practical realities of developing the system, and I’ll talk about some future directions for the project.

Scale up and Scale Out Anaconda and PyData

Travis Oliphant

Deeplearning on Hadoop @OSCON 2014

Adam Gibson

Distributed Deep Learning on Hadoop Deep-learning is useful in detecting anomalies like fraud, spam and money laundering; identifying similarities to augment search and text analytics; predicting customer lifetime value and churn; recognizing faces and voices. Deeplearning4j is an infinitely scalable deep-learning architecture suitable for Hadoop and other big-data structures. It includes a distributed deep-learning framework and a normal deep-learning framework; i.e. it runs on a single thread as well. Training takes place in the cluster, which means it can process massive amounts of data. Nets are trained in parallel via iterative reduce, and they are equally compatible with Java, Scala and Clojure. The distributed deep-learning framework is made for data input and neural net training at scale, and its output should be highly accurate predictive models. The framework's neural nets include restricted Boltzmann machines, deep-belief networks, deep autoencoders, convolutional nets and recursive neural tensor networks.

Introduction To TensorFlow

Spotle.ai

Develop a fundamental overview of Google TensorFlow, one of the most widely adopted technologies for advanced deep learning and neural network applications. Understand the core concepts of artificial intelligence, deep learning and machine learning and the applications of TensorFlow in these areas. The deck also introduces the Spotle.ai masterclass in Advanced Deep Learning With Tensorflow and Keras.

Snorkel: Dark Data and Machine Learning with Christopher Ré

Jen Aman

Building applications that can read and analyze a wide variety of data may change the way we do science and make business decisions. However, building such applications is challenging: real world data is expressed in natural language, images, or other “dark” data formats which are fraught with imprecision and ambiguity and so are difficult for machines to understand. This talk will describe Snorkel, whose goal is to make routine Dark Data and other prediction tasks dramatically easier. At its core, Snorkel focuses on a key bottleneck in the development of machine learning systems: the lack of large training datasets. In Snorkel, a user implicitly creates large training sets by writing simple programs that label data, instead of performing manual feature engineering or tedious hand-labeling of individual data items. We’ll provide a set of tutorials that will allow folks to write Snorkel applications that use Spark. Snorkel is open source on github and available from Snorkel.Stanford.edu.

Data Programming: Creating Large Datasets, Quickly -- Presented at JPL MLRG

Thamme Gowda

Mentoring Session with Innovesia: Advance Robotics

Dony Riyanto

Image Recognition on AWS with Apache Spark and BigDL

Amazon Web Services

by Dave Nielsen, Technical Program Manager, Big Data Technologies, STO, Intel In this workshop we explain how to use Deep Learning to recognize objects in photos using the BigDL framework for Apache Spark. The workshop starts by explaining how Convolution Neural Network models like SSD, VGG and Inception identify edges and then higher level characteristics in multiple layers in order to identify the object. Then we will describe how to build an end-to-end pipeline for image classification including data prep, model building, training, scoring. The workshop includes a Zeppelin Notebook with python code you can modify to train the model and test for accuracy. The code provided in this workshop can be used to run examples on your laptop, on-premises or on Amazon Web Services managed by Qubole. Level 100

Guglielmo iozzia - Google I/O extended dublin 2018

Guglielmo Iozzia

Suneel Marthi - Deep Learning with Apache Flink and DL4J

Flink Forward

http://flink-forward.org/kb_sessions/deep-learning-with-apache-flink-and-dl4j/ Deep Learning has become very popular over the last few years in areas such as Image Recognition, Fraud Detection, Machine Translation etc. Deep Learning has proved to be very useful in handling unstructured data and extracting value from them. A big challenge with having to build deep learning models was the high cost of training them. With the recent advent of distributed frameworks like Apache Flink, Apache Spark etc.. it’s faster to train Deep Learning models in parallel on modern platform architecture. In this talk, we’ll be showing how to use Apache Flink Streaming with the open source Deep Learning framework, DeepLearning4j to perform large scale deep learning model training. We will show a demo of a Recurrent Neural Net that is trained for language modeling and have it generate text.

What's hot

Apache Toree

Asim Jalis

Amazon Deep Learning

Amanda Mackay (she/her)

Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017

StampedeCon

KERAS Python Tutorial

MahmutKAMALAK

Deep Learning on Qubole Data Platform

Shivaji Dutta

Smart Data Conference: DL4J and DataVec

Josh Patterson

Spark MLlib and Viral Tweets

Asim Jalis

eScience Cluster Arch. OverviewFrancesco Bongiovanni

BigDL webinar - Deep Learning Library for Spark

DESMOND YUEN

Social Networks Analysis

Joud Khattab

Anomaly Detection at Scale

Jeff Henrikson

Matrix Factorizations at Scale: a Comparison of Scientific Data Analytics on ...

Databricks

Tom Peters, Software Engineer, Ufora at MLconf ATL 2016

MLconf

Scale up and Scale Out Anaconda and PyData

Travis Oliphant

Deeplearning on Hadoop @OSCON 2014

Adam Gibson

Introduction To TensorFlow

Spotle.ai

Snorkel: Dark Data and Machine Learning with Christopher Ré

Jen Aman

Data Programming: Creating Large Datasets, Quickly -- Presented at JPL MLRG

Thamme Gowda

Mentoring Session with Innovesia: Advance Robotics

Dony Riyanto

What's hot (19)

Apache Toree

Amazon Deep Learning

Getting Started with Keras and TensorFlow - StampedeCon AI Summit 2017

KERAS Python Tutorial

Deep Learning on Qubole Data Platform

Smart Data Conference: DL4J and DataVec

Spark MLlib and Viral Tweets

eScience Cluster Arch. Overview

BigDL webinar - Deep Learning Library for Spark

Social Networks Analysis

Anomaly Detection at Scale

Matrix Factorizations at Scale: a Comparison of Scientific Data Analytics on ...

Tom Peters, Software Engineer, Ufora at MLconf ATL 2016

Scale up and Scale Out Anaconda and PyData

Deeplearning on Hadoop @OSCON 2014

Introduction To TensorFlow

Snorkel: Dark Data and Machine Learning with Christopher Ré

Data Programming: Creating Large Datasets, Quickly -- Presented at JPL MLRG

Mentoring Session with Innovesia: Advance Robotics

Similar to Hands on image recognition with scala spark and deep learning4j

Image Recognition on AWS with Apache Spark and BigDL

Amazon Web Services

Guglielmo iozzia - Google I/O extended dublin 2018

Guglielmo Iozzia

Suneel Marthi - Deep Learning with Apache Flink and DL4J

Flink Forward

Jeremy Nixon, Machine Learning Engineer, Spark Technology Center at MLconf AT...

MLconf

Convolutional Neural Networks at scale in Spark MLlib: Jeremy Nixon will focus on the engineering and applications of a new algorithm built on top of MLlib. The presentation will focus on the methods the algorithm uses to automatically generate features to capture nonlinear structure in data, as well as the process by which it’s trained. Major aspects of that include compositional transformations over the data, convolution, and distributed backpropagation via SGD with adaptive gradients and an adaptive learning rate. Applications will look into how to use convolutional neural networks to model data in computer vision, natural language and signal processing. Details around optimal preprocessing, the type of structure that can be learned, and managing its ability to generalize will inform developers looking to apply nonlinear modeling tools to problems that they face.

Why scala for data science

Guglielmo Iozzia

Skymind Company Profile

Shu Wei Goh

Read: issuu.com/shuweigoh/docs/skymind At Skymind, we’re tackling some of the most advanced problems in data analysis and machine intelligence. We offer state-of-the-art, flexible, scalable deep learning for industry. Deep learning is becoming an important tool set for natural-language processing (NLP), computer vision, database predictions, pattern recognition, image/video processing and fraud detection.

Sjug #26 ml is in java but is dl too - ver1.04 - tomasz sikora 2018-03-23

Tomasz Sikora

Vint Cerf, recognised as one of "the fathers of the Internet" as co-inventor of TCP/IP, said "And programming computers was so fascinating. You create your own little universe, and then it does what you tell it to do.". Now when computers are learning these words give different ground to debate on developer context here. So what can we do from old good Software Craftsmanship perspective? I, as a developer who still believes in Java, can I use my beloved platform? How can I tackle problems requiring deeper models? On this session Tomasz tried to answer those, opening 100s more questions ;). He executed and configured a few models trying to explore the field from as practical perspective as possible. To get more visit https://twitter.com/tomaszsikora and http://silesia.jug.pl/

Deep Learning with DL4J on Apache Spark: Yeah it's Cool, but are You Doing it...

DataWorks Summit

DeepLearning4J (DL4J) is a powerful Open Source distributed framework that brings Deep Learning to the JVM (it can serve as a DIY tool for Java, Scala, Clojure and Kotlin programmers). It can be used on distributed GPUs and CPUs. It is integrated with Hadoop and Apache Spark. ND4J is a Open Source, distributed and GPU-enabled library that brings the intuitive scientific computing tools of the Python community to the JVM. Training neural network models using DL4J, ND4J and Spark is a powerful combination, but the overall cluster configuration can present some unespected issues that can compromise performances and nullify the benefits of well written code and good model design. In this talk I will walk through some of those problems and will present some best practices to prevent them. The presented use cases will refer to DL4J and ND4J on different Spark deployment modes (standalone, YARN, Kubernetes). The reference programming language for any code example would be Scala, but no preliminary Scala knowledge is mandatory in order to better understanding the presented topics.

PPT5: Neuron Introduction

akira-ai

Machine Learning with Scala

Susan Eraly

Build a deep learning pipeline on apache spark for ads optimization

Craig Chao

Bringing Deep Learning into production

Paolo Platter

Big Data Processing with Spark and Scala

Edureka!

Running Emerging AI Applications on Big Data Platforms with Ray On Apache Spark

Databricks

Top Deep Learning Frameworks.pdf

Appdeveloper10

Integrating Deep Learning Libraries with Apache Spark

Databricks

The combination of deep learning with Apache Spark has the potential to make a huge impact. Joseph Bradley and Xiangrui Meng share best practices for integrating popular deep learning libraries with Apache Spark. Rather than comparing deep learning systems or specific optimizations, Joseph and Xiangrui focus on issues that are common to many deep learning frameworks when running on a Spark cluster, such as optimizing cluster setup and data ingest (clusters can be configured to avoid task conflicts on GPUs and to allow using multiple GPUs per worker), configuring the cluster (setting up pipelines for efficient data ingest improves job throughput), and monitoring long-running jobs (interactive monitoring facilitates both the work of configuration and checking the stability of deep learning jobs). Joseph and Xiangrui then demonstrate the techniques using Google’s popular TensorFlow library.

Started with-apache-spark

Happiest Minds Technologies

Emiliano Martinez | Deep learning in Spark Slides | Codemotion Madrid 2018

Codemotion

Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk

Saurabh Saxena

Deep Learning libraries and ﬁrst experiments with Theano

Vincenzo Lomonaco

In recent years, neural networks and deep learning techniques have shown to perform well on many problems in image recognition, speech recognition, natural language processing and many other tasks. As a result, a large number of libraries, toolkits and frameworks came out in different languages and with different purposes. In this report, ﬁrstly we take a look at these projects and secondly we choose the framework that best suits our needs: Theano. Eventually, we implement a simple convolutional neural net using this framework to test both its ease-of-use and efﬁciency.

Similar to Hands on image recognition with scala spark and deep learning4j (20)

Image Recognition on AWS with Apache Spark and BigDL

Guglielmo iozzia - Google I/O extended dublin 2018

Suneel Marthi - Deep Learning with Apache Flink and DL4J

Jeremy Nixon, Machine Learning Engineer, Spark Technology Center at MLconf AT...

Why scala for data science

Skymind Company Profile

Sjug #26 ml is in java but is dl too - ver1.04 - tomasz sikora 2018-03-23

Deep Learning with DL4J on Apache Spark: Yeah it's Cool, but are You Doing it...

PPT5: Neuron Introduction

Machine Learning with Scala

Build a deep learning pipeline on apache spark for ads optimization

Bringing Deep Learning into production

Big Data Processing with Spark and Scala

Running Emerging AI Applications on Big Data Platforms with Ray On Apache Spark

Top Deep Learning Frameworks.pdf

Integrating Deep Learning Libraries with Apache Spark

Started with-apache-spark

Emiliano Martinez | Deep learning in Spark Slides | Codemotion Madrid 2018

Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk

Deep Learning libraries and ﬁrst experiments with Theano

Recently uploaded

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

Leading Change strategies and insights for effective change management pdf 1.pdf

OnBoard

Essentials of Automations: Optimizing FME Workflows with Parameters

Safe Software

Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place. Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects. Here’s what you’ll gain: - Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows. - Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy. - Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency. - Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity. We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic. Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Elevating Tactical DDD Patterns Through Object Calisthenics

Dorra BARTAGUIZ

After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Tobias Schneck

As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other? Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.

DevOps and Testing slides at DASA Connect

Kari Kakkonen

Knowledge engineering: from people to machines and back

Elena Simperl

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

Connector Corner: Automate dynamic content and events by pushing a button

DianaGray10

Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to: Create a campaign using Mailchimp with merge tags/fields Send an interactive Slack channel message (using buttons) Have the message received by managers and peers along with a test email for review But there’s more: In a second workflow supporting the same use case, you’ll see: Your campaign sent to target colleagues for approval If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team But—if the “Reject” button is pushed, colleagues will be alerted via Slack message Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors. And... Speakers: Akshay Agnihotri, Product Manager Charlie Greenberg, Host

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Product School

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

Bits & Pixels using AI for Good.........

Alison B. Lowndes

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Recently uploaded (20)

When stars align: studies in data quality, knowledge graphs, and machine lear...

FIDO Alliance Osaka Seminar: Overview.pdf

Leading Change strategies and insights for effective change management pdf 1.pdf

Essentials of Automations: Optimizing FME Workflows with Parameters

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

Monitoring Java Application Security with JDK Tools and JFR Events

GraphRAG is All You need? LLM & Knowledge Graph

Elevating Tactical DDD Patterns Through Object Calisthenics

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

DevOps and Testing slides at DASA Connect

Knowledge engineering: from people to machines and back

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

Connector Corner: Automate dynamic content and events by pushing a button

Securing your Kubernetes cluster_ a step-by-step guide to success !

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Neuro-symbolic is not enough, we need neuro-*semantic*

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Bits & Pixels using AI for Good.........

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

Hands on image recognition with scala spark and deep learning4j

1. Hands-on Image Recognition with Scala, Spark and DeepLearning4j Presented by Guglielmo Iozzia Kiev, April 20th 2018

2. Something About Me  Big Data Delivery Lead at Optum (UHG)  Previously at IBM and FAO of the UN  Current fields of expertise are Big Data, ML/DL and DevOps  Past experience in JVM languages (Java, Groovy, Scala) development, test automation, CI/CD

3. Something About Me  Author of the upcoming book “Hands-on Deep Learning with Apache Spark”  I love preparing home-made pizza

4. The Dublin Tech Hub

5. Agenda  Scala for Data Science  Deep Learning  Definition  Multilayer Neural Networks  Apache Spark  Overview  DeepLearning4j  ETL  CNN  Image Recognition Example

6. Scala for Data Science  Most part of the systems/tools in the Big Data/ML space run on the JVM.  Robustness and performance when it comes to production system and large datasets.  Plenty of frameworks available:

7. Deep Learning  It is a subset of machine learning that can solve particularly hard and large-scale problems in areas such as natural language processing or image classification.  It is based on Multilayered Neural Networks.

8. Deep Learning

9. Artificial Neural Networks

10. Multilayer Neural Networks  The first layer, called the input layer, is where features are input.  The last layer is called the output layer.  Any layer that is not an input or output layer is a hidden layer.

11. Multilayer Neural Networks  Different variations.  CNN: used in image recognition.  RNN: used in NLP.

12. Apache Spark  It is an Open Source fast cluster-computing platform.  Data are loaded in distributed memory (RAM) over a cluster of machines.  Compared to Hadoop MapReduce, it runs programs up to 100x faster when the data fits in memory, or 10x faster on disk.  It provides support for Java, Scala, Python and R.

13. Apache Spark: job execution

14. Apache Spark: components

15. DeepLearning4j: Overview  It is an Open Source distributed Deep Learning library for the JVM.  It is integrated with Hadoop and Spark.  It provides support for both CPUs and GPUs.  It allows the import of neural net models from the most major frameworks via Keras.

16. DeepLearning4j: libraries  Deeplearning4J: Neural Net Platform  ND4J: Numpy for the JVM  DataVec: Tool for Machine Learning ETL Operations  JavaCPP: The Bridge Between Java and Native C++  Arbiter: Evaluation Tool for Machine Learning Algorithms  RL4J: Deep Reinforcement Learning for the JVM.

17. ETL with DL4j Live Example

18. DL4j CNN Live Example

19. Image Recognition Live Example

20. Useful Links Apache Spark: http://spark.apache.org/ DeepLearning4j: https://deeplearning4j.org/ Keras: https://keras.io/

21. Q & A

22. Wrap Up Linkedin: https://ie.linkedin.com/in/giozzia Twitter: @GuglielmoIozzia Blog: googlielmo.blogspot.com DZone: https://dzone.com/users/2532948/virtualramblas.html Hands-On Deep Learning with Apache Spark: https://www.packtpub.com/big-data-and-business-intelligence/hands- deep-learning-apache-spark

Hands on image recognition with scala spark and deep learning4j

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Hands on image recognition with scala spark and deep learning4j

Similar to Hands on image recognition with scala spark and deep learning4j (20)

Recently uploaded

Recently uploaded (20)

Hands on image recognition with scala spark and deep learning4j