1.
AI in Manufacturing
neXt LIVE with Dr. Amit Sheth

2.
I
II
III
IV
V
VI
VII
AMIT SHETH, PhD
• Founding director of the
university-wide Aritificial
Intelligence Institute at UofSC
(AIISC)
• Core research on AI topics such as
knowledge infused learning and
neuro-symbolic computing,
• AIISC has translational research
with nearly al of the colleges at
UofSC
• Fellow of IEEE, AAAI and AAAS
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing2

3.
I
II
III
IV
V
VI
VII
OUTLINE
I. AIISC Introduction
II. AI in Manufacturing
III. Knowledge Graph/Ontology
IV. Computer Vision in Manufacturing
V. Predictive Maintenance
VI. NLP and Conversational AI
VII. Applications of AI in Manufacturing
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing3

4.
AIISC Introduction
Section I
4

5.
I
II
III
IV
V
VI
VII
I | AIISC DRIVERS AND DISTINCTIONS
• To be recognized as the top
institution in interdisciplinary AI,
AI applications and impact in
Southeast US, and among the top
in its chosen of selected AI
subareas
● Exceptional Student Outcomes
○ Education: 20+ courses in AI
● High impact from translational
research
● Apply AI and realize impact
across the university and state
● High engagement with
communities and industry
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing5
I

6.
I
II
III
IV
V
VI
VII
I | UNIVERSITY-WIDE MANDATE
● College of Medicine (5)
● College of Nursing (2)
● College of Arts & Science (2)
○ Hazard & Vulnerability Res Inst
○ Institute of Mind and Brain
● College of Pharmacy
○ Colorectal Cancer
○ Digestive Inflammation Index
● College of Information & Communication
● College of Engineering & Computing
○ Civil and Environmental
○ Mechanical & Aerospace
○ Computer Sc & Engg
● College Education
○ ALL4SC
● College of Public Health
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing6
Practically all our work involves real world challenges, real-world data, interdisciplinary
collaborators, path-breaking research and innovations, real-world deployments, real
world use, and measurable real world impact.
I

7.
AI in Manufacturing
Section II
7

8.
I
II
III
IV
V
VI
VII
II | BIG CHANGES IN MANUFACTURING NEED AI
● Automation supported by myriad of technologies including Robots; IoT,
Digital Twins
● Strategic changes in Supply Chain - massive disruptions, hiccups in
globalization
● Sustainability - traceability and accountability
Result: Data Tsunami -> Analytics [CAGR of 30.9% over the forecast period, 2020-
2025: ResearchAndMarkets.com]
Possible Solution: AI can help
(Recommendation/Planning/Decision Making)
[AI in manufacturing is expected to grow at a CAGR of 57.2% during 2020 and 2027:
MarketsAndMarkets.com]
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing8
II

9.
I
II
III
IV
V
VI
VII
II | BIG CHANGES IN MANUFACTURING NEED AI
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing9
“Industry 4.0 is the information-intensive transformation of manufacturing (and related industries) in a
connected environment of big data, people, processes, services, systems and IoT-enabled industrial assets
with the generation, leverage and utilization of actionable data and information as a way and means to
realize smart industry and ecosystems of industrial innovation and collaboration.”
From: https://www.i-scoop.eu/industry-4-0/
II

10.
I
II
III
IV
V
VI
VII
II | BIG CHANGES IN MANUFACTURING NEED AI
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing10
Information is cheap.
Understanding is expensive.
Karl Fast,
Professor of UX Design,
Kent State University
AI is about converting data into
knowledge, insights and actions.
II

11.
I
II
III
IV
V
VI
VII
II | WHAT IS EXPECTED FOR FACTORY OF FUTURE (FOF)
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing11
1. Detect defects throughout the production process.
2. Deploy predictive maintenance to reduce downtime.
3. Respond to real-time changes in demand across the supply chain.
4. Validate whether intricate goods like microchips have been perfectly produced.
5. Reduce costs of small-batch or single-run goods, enabling greater customization.
6. Improve employee satisfaction by shifting mundane tasks to machines.
II
From: Luke A. Renner, How Can Artificial Intelligence Be Applied in Manufacturing?

12.
I
II
III
IV
V
VI
VII
II | AI IN MANUFACTURING (WHY? -> BENEFITS)
• Direct Automation
• 24/7 production
• Safety
• Low operational cost
• Greater efficiency
• Quality control
• Quick decision making
https://www.rowse.co.uk/blog/post/7-manufacturing-ai-benefits
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing12
https://www.industryweek.com/technology-and-iiot/article/22027119/benefits-of-
ai-on-manufacturing-a-visual-guide
II

13.
I
II
III
IV
V
VI
VII
II | MANUFACTURING FAILURES: SEVERITY OF THE PROBLEM
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing13
II

14.
I
II
III
IV
V
VI
VII
II | AI IN MANUFACTURING
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing14
“EVEN THOUGH AI HAS BECOME ONE OF THE HOTTEST TOPICS IN
MANUFACTURING TODAY, MOST MANUFACTURERS ARE AT THE
START OF THE ADOPTION CURVE. “
THOMAS LEESON
“BY THE TIME A LATE ADOPTER HAS DONE ALL THE NECESSARY
PREPARATION, EARLIER ADOPTERS WILL HAVE TAKEN
CONSIDERABLE MARKET SHARE; THEY’LL BE ABLE TO OPERATE AT
SUBSTANTIALLY LOWER COSTS WITH BETTER PERFORMANCE. IN
SHORT, THE WINNERS MAY TAKE ALL AND LATE ADOPTERS MAY
NEVER CATCH UP.”
VIKRAM MAHIDHAR & THOMAS H. DAVENPORT, HBR, DEC 2019
II

15.
I
II
III
IV
V
VI
VII
II | AI IN MANUFACTURING
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing15
KEY AI
SUBAREAS
Conversational
AI
Machine & Deep
Learning
Natural
Language
Processing (NLP)
Computer
Vision
Robotics
Knowledge Graph
(Ontology)
II

16.
Knowledge Graph/Ontology
Section III
16

17.
I
II
III
IV
V
VI
VII
III | TYPICAL NW ARCHITECTURE FOR FOF: EDGE, FOG, CLOUD
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing17
Figure: Li et al, Online Workload Allocation via Fog-Fog-Cloud Cooperation to Reduce IoT Task Service Delay, 2019
III

18.
I
II
III
IV
V
VI
VII
II | INDUSTRY NEXT MANUFACTURING @ MCNAIR, UOFSC
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing18
Digital Cell
Actual Cell
II

19.
I
II
III
IV
V
VI
VII
III | CONNECTED MANUFACTURING: SMART IOT AS SOLUTION
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing19
III
http://wiki.aiisc.ai/index.php/Smart_Data

20.
I
II
III
IV
V
VI
VII
III | SEMANTICS AT DEVICE AND FACTORY FLOOR NW PROTOCOL LEVELS
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing20
Reference: Gyrard, Amelie, Pankesh Patel, Amit P. Sheth, and Martin Serrano. "Building the web of knowledge with smart iot applications." IEEE Intelligent Systems 31 (5), 2016)
P. Desai, A. Sheth, P. Anantharam: Semantic Gateway as a Service Architecture for IoT Interoperability, 2015.
● Moving computation and intelligence closer to data generation.
● Semantic Gateway as a service for interoperability between devices
that are using different protocols.
III

21.
I
II
III
IV
V
VI
VII
III | DATA INTEROPERABILITY
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing21
III

22.
I
II
III
IV
V
VI
VII
III | ANALOGY WITH HUMAN HEALTH
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing22
III

23.
I
II
III
IV
V
VI
VII
III | TYPES OF INTEROPERABILITY
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing23
Interoperability of
● NWs & protocols
● Data
Data interop:
● Domain
independent
● Domain specific
SenML, SSN
Semantic annotation
Liu et al, Device-Oriented Automatic Semantic Annotation in IoT, 2017
III

24.
I
II
III
IV
V
VI
VII
III | ROLE OF ONTOLOGY/KG FOR INTEROPERABILITY: SSN
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing24
Semantic annotation/
labeling help with shared
meaning/uniform
interpretation
of data
SSN ontology
provides framework for
semantic annotation of
sensor/device data;
Similarly application/
domain specific
ontology/knowledge graph
can support semantic
annotation wrt to the
application/domain/task
III

25.
I
II
III
IV
V
VI
VII
III | DIKW: DATA, ANNOTATION, ABSTRACTION, ACTION
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing25
Adapted from: Gyrard, et al,
Building the Web of Knowledge with Smart IoT Applications (Extended Version), 2016
ISA 95
model
III

26.
I
II
III
IV
V
VI
VII
III | FACTORY OF FUTURE (FOF) NETWORKING
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing26
Intizar Ali, Pankesh Patel, John Breslin, “Middleware for Real-Time Event Detection and Predictive Analytics in Smart Manufacturing”, 15th International Conference on
Distributed Computing in Sensor Systems (DCOSS), 2019.
III

27.
I
II
III
IV
V
VI
VII
III | USE OF KNOWLEDGE GRAPHS IN SMART MANUFACTURING
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing27
VISUAL SENSORS
Security Cameras, Drones,
Inspection Cameras
PHYSICAL SENSORS
INDUSTRIAL SENSORS
Load cell, Accelerometer,
Optical sensor, Potentiometer,
RTD temperature sensor
HEAT MAP SENSOR
Infra-Red heat map sensor
DIGITAL TWIN
Factory Configuration
Process simulators
Loops
Manufacturing Process
Manufacturing Knowledge
Representation
MANUFACTURING ONTOLOGY
MANUFACTURING KG
Downstream Tasks
ADAPTIVE KG UPDATE MODULE
Calculating
Ont. + KG
update
MANUFACTURING SCENE/ EVENT
UNDERSTANDING
FAULT DETECTION
Events
Features of Interests
Computer Vision + Signal
Processing Module
KG facts extraction
and infusion
Enhanced Fault
Detection
Feedback
III

28.
I
II
III
IV
V
VI
VII
III | REVISITING ARCHITECTURE: DATA TO ABSTRACTION
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing28
III

29.
Computer Vision in Manufacturing
Section IV
29

30.
I
II
III
IV
V
VI
VII
IV | COMPUTER VISION EXAMPLES IN FOF
• Predictive maintenance of machinery: Using IoT sensors to monitor the
production line in real-time to reduce unscheduled downtime and increase
productivity
• Inspection of defectives: Monitor the assembly lines and identify the
defective components
• Accurate assembly of components: Alert system for misassembly or mid-
operation failure.
• Quality control for products: Eg: Acquire Automation implements machine
vision that permits manufacturers to inspect bottles in a complete 360-
degree view to verify that products are placed in the correct packaging
• Health and safety: Deep learning-based AI to track the movement of people
and predict where the machines are going to be to avoid dangerous
interactions
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing30
IV

31.
I
II
III
IV
V
VI
VII
IV | INSPECTION SYSTEM
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing31
• Visual sensors(IoT) are deployed to monitor defects
• They generate a lot of data and it is difficult to manage large volumes
of data
• A system to handle and make sense out of such large volumes of data
is necessary
• Convolutional Neural Networks can be used for defect classification
• Besides defect type, the degree of defect can also be quantified
IV

32.
I
II
III
IV
V
VI
VII
IV | DEFECT DETECTION
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing32
L Li et al, Deep Learning for Smart Industry: Efficient Manufacture Inspection System With Fog Computing, IEEE Transactions on Industrial Informatics, 14 (10), October 2018
IV

33.
I
II
III
IV
V
VI
VII
IV | VIDEO ANALYTICS FOR INDUSTRY 4.0 : DRONE AND SAFETY
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing33
● Health, Safety and Environment (HSE) inspection
● "Bird eye" view
● Camera to capture images, evidences
Drone@ Construction site
● Grid inspection
● Camera to capture any potential issues
● Remote inspection for worker safety
Drone@ grid inspection
Image source: https://bit.ly/2RgKxeL https://bit.ly/2zKXN4N
IV

34.
Predictive Maintenance
Section V
34

35.
I
II
III
IV
V
VI
VII
V | INTELLIGENT PREDICTIVE MAINTENANCE FOR FAULT DIAGNOSIS
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing35
ML/DL
Algorithms
used for Fault
Diagnosis
How AI Affects the Future Predictive Maintenance: A Primer of Deep Learning
Li et al, ML algorithms used: SVM, Decision Trees, ANNs, Self-Organizing-Maps and other Statistical Machine Learning techniques
V

36.
I
II
III
IV
V
VI
VII
V | PREDICTIVE MAINTENANCE BASED ON DEEP LEARNING
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing36
Wang and Wang, How AI Affects the Future Predictive Maintenance: A Primer of Deep Learning, 2018
V
Prognostics: probabilities
that the system can fail in
different time horizon/
Maintenance decision.

37.
NLP and Conversational AI
Section VI
37

38.
I
II
III
IV
V
VI
VII
VI | CHATBOT AND SMART MANUFACTURING
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing38
Image source: : https://bit.ly/2zLZ9Mo
Chatbot features
● Easy to use
● Real-time interactions with devices
● Questions- answer structure
● Natural communication
● Continuous improvement over time
● Personalized relation with engineers (context,
history)
● Helping maintenance crews to verify
factory's condition
○ Field operation – "What is the
temperature reading of a motor #1
of floor #3?"
● Feedback from users on trial runs
○ Improved customer-manufacturer
relationship
● Scalable
VI

39.
I
II
III
IV
V
VI
VII
VI | TAKEAWAY
• Manufacturing is a data rich environment. More automation and new
manufacturing add to the growth of data
• Different area of AI provide ability to improve decision making from
different types of data, and for different applications
• AI is at the center of the future differentiation and progress in
manufacturing
Dr Harik's neXt LIVE with Dr. Amit Sheth on AI in Manufacturing39
VI

40.
THANK YOU!
neXt LIVE with Dr. Ramy Harik
For more information email harik@cec.sc.edu
Slide layout by Alex Brasington