759

1. Application of Parallel Computing
in Artificial Intelligence
Reporter：

2. 1
2
3
4
Detailed Description of AI
Parallel Computing in AI
Conclusion
C O N T E N T
Application and influence of AI

3. Introduction to Artificial Intelligence
Part One
Definition and development history of artificial intelligence

4. What is AI?
Artificial intelligence, also known as intelligent machinery and machine intelligence, refers to the
intelligence shown by the machines made by people. Generally, artificial intelligence refers to the
technology that represents human intelligence through ordinary computer programs. Figuratively speaking,
artificial intelligence is a technology that enables machines to see, hear, think, speak, and move like
humans

5. History of Artificial Intelligence
1. Birth of artificial intelligence
In 1950, the computer scientist Alan Turing published a landmark paper predicting the possibility
of creating a machine with true intelligence. Given that "intelligence" is not easy to measure, he produced
the famous Turing Test, which measures a machine's intelligence by its ability to disguise a human
conversation.
In 1956, the Dartmouth Conference brought together leading scientists such as Marvin Minsky,
John McCarthy, Claude Shannon, Nathan Rochester, Allen Newell and Herbert Simon, Jointly determined
the name and task of artificial intelligence, marking the formal birth of the subject of artificial intelligence.

6. History of Artificial Intelligence
2.Two lows and three highs of artificial intelligence
• The First High Tide (1956 - 1974)
 The decade after 1956 - golden age of artificial intelligence.
 Computers were used to prove mathematical theorems, solve algebraic word problems and other fields.
• The First Low (1974-1980)
 By the early 1970s, people gradually found that only the ability of logical reasoning was not enough to achieve AI,
and many problems were not solved over time.
 The previous optimism made people expect too much and lack effective progress.
 Many institutions gradually stopped funding AI research.

7. History of Artificial Intelligence
2.Two lows and three highs of artificial intelligence
• The Second High Tide (1980-1987)
 1980s - many companies began to develop and apply expert systems.
 Therefore, knowledge engineering, which expert systems rely on, has become the focus of AI research.
 John Hopfield invented the Hopfield network and solved the famous traveling salesman (TSP) problem.
 David Rumelhart proposed Back Propagation, BP algorithm, which solved the learning problem of multi-layer
neural network.
• Second trough (1987 - 1993)
 From the late 1980s to the early 1990s, the problems of expert systems, such as narrow application fields, difficult
knowledge acquisition, and high maintenance costs, gradually emerged.
 AI has encountered a series of financial problems and entered the second low.

8. History of Artificial Intelligence
2.Two lows and three highs of artificial intelligence
• The third climax (since 1993)
 Since the mid-1990s, with the rapid development of computer performance, the accumulation of massive data and
the unremitting efforts of AI researchers, AI has continuously made breakthrough achievements in many fields,
setting off a new round of climax.

9. Parallel Computing in AI
Part Two

10. Parallel Computing
Training：Data Parallelism
Distribute datasets
to multiple GPUs
Forward training,
Compute gradient in
every single GPU
Compute average
gradient of all
Update params
with average
gradient

11. Parallel Computing & Training & Inference
Training：Pipeline Parallelism
 Divide the model into different parts to run on different
GPUs in order, which can save a lot of GPU memories.
 Model layers are computing in order, so some GPU will
wait for others’s result

12. Parallel Computing & Training & Inference
Training：Pipeline Parallelism
Optimization：Pipeline Parallelism + Data Parallelism

13. Application and influence of AI
Part Three

14. CV
Auto
pilot
NLP
Audio
Computer Vision
• Obeject Detection
• Object Classification
• Face Recognition
• ......
Autopilot
Natural Language Processing
Audio
The Application of AI
• Man-Machine Interaction
• Translating
• Emotional Analysis
• ......
• Perceptual Algorithm
• Decision-Making
• ......
• Speech Recognition
• Text Reading
• Audio Audit
• ......
......

15. Impact of Artificial Intelligence
1. As the core force of a new round of scientific and technological revolution and industrial transformation, AI
promotes the overall leap of social productivity, promotes the upgrading of traditional industries, drives the rapid
development of "unmanned economy", and has a positive impact on the development of people's livelihood in the
fields of intelligent transportation, smart home, smart medical care, etc.
2. The application of artificial intelligence not only affects the human way of thinking and traditional concepts, but
also changes people's way of thinking and concepts. For example, traditional knowledge in the past was usually
published in magazines or books and newspapers, so the content of traditional knowledge cannot be changed.
3. The application of artificial intelligence will make the contradiction in labor employment more prominent. As AI
can take the place of human beings to do all kinds of mental work, the labor efficiency of the whole society will be
greatly improved, but at the same time, some people will have to change their types of work, or even cause their
unemployment.
The use of artificial intelligence will change the employment mode of the whole mental labor. These mental workers
will not be reused in the labor market. Once this situation occurs, it will be a big disaster for the society.

16. Conclusion
Part Four

17. Conclusion
GPU high performance parallel computing drives the AI industry to start. The proposal of deep
learning has brought about a fundamental breakthrough in AI technology, which has greatly
improved the accuracy of complex classification tasks. However, deep learning requires complex
calculations on a large amount of data, so the significant improvement of GPU parallel computing
performance and big data have created conditions for the popularization of deep learning and the
launch of the AI industry.
Computing power is the basis of AI. The breakthrough of large-scale parallel computing
technology has brought about an upward turning point in the development of AI. Parallel
computing is also called parallel computing (compared with serial computing). Parallel
computing is an algorithm that can execute multiple instructions at the same time. The purpose of
parallel computing is to expand the scale of problem solving, improve the computing speed, and
solve large and complex computing problems. The so-called parallel computing can be divided
into time parallel and space parallel. Temporal parallelism refers to pipeline technology, while
spatial parallelism refers to parallel computing performed by multiple processors. Therefore, the
realization of large-scale parallel computing capability has made AI take a big step forward

18. THANK YOU
Reporter：