Slides from the introductory lecture I gave for students at Camp IT 2019. I tried to cover artificial inteligence, machine learning, most popular algorithms and their applications to business as broadly as possible - for in-depth materials on the given topics, see links and references in the presentation.
Handwritten Digit Recognition and performance of various modelsation[autosaved]SubhradeepMaji
This presentation is all about handwritten digit recognition of different people using Convolution Neural Network and compare the performance of different models based on different sequence of layers.
Using Deep Learning to Find Similar DressesHJ van Veen
Report by Luís Mey ( https://www.linkedin.com/in/lu%C3%ADs-gustavo-bernardo-mey-97b38927/ ) on Udacity Machine Learning Course - Final Project: Use Deep Learning to Find Similar Dresses.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Image classification is perhaps the most important part of digital image analysis. In this paper, we compare the most widely used model CNN Convolutional Neural Network , and MLP Multilayer Perceptron . We aim to show how both models differ and how both models approach towards the final goal, which is image classification. Souvik Banerjee | Dr. A Rengarajan "Hand-Written Digit Classification" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd42444.pdf Paper URL: https://www.ijtsrd.comcomputer-science/artificial-intelligence/42444/handwritten-digit-classification/souvik-banerjee
Handwritten Digit Recognition and performance of various modelsation[autosaved]SubhradeepMaji
This presentation is all about handwritten digit recognition of different people using Convolution Neural Network and compare the performance of different models based on different sequence of layers.
Using Deep Learning to Find Similar DressesHJ van Veen
Report by Luís Mey ( https://www.linkedin.com/in/lu%C3%ADs-gustavo-bernardo-mey-97b38927/ ) on Udacity Machine Learning Course - Final Project: Use Deep Learning to Find Similar Dresses.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Image classification is perhaps the most important part of digital image analysis. In this paper, we compare the most widely used model CNN Convolutional Neural Network , and MLP Multilayer Perceptron . We aim to show how both models differ and how both models approach towards the final goal, which is image classification. Souvik Banerjee | Dr. A Rengarajan "Hand-Written Digit Classification" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd42444.pdf Paper URL: https://www.ijtsrd.comcomputer-science/artificial-intelligence/42444/handwritten-digit-classification/souvik-banerjee
https://telecombcn-dl.github.io/2017-dlsl/
Winter School on Deep Learning for Speech and Language. UPC BarcelonaTech ETSETB TelecomBCN.
The aim of this course is to train students in methods of deep learning for speech and language. Recurrent Neural Networks (RNN) will be presented and analyzed in detail to understand the potential of these state of the art tools for time series processing. Engineering tips and scalability issues will be addressed to solve tasks such as machine translation, speech recognition, speech synthesis or question answering. Hands-on sessions will provide development skills so that attendees can become competent in contemporary data anlytics tools.
In this article you will learn hot to use tensorflow Softmax Classifier estimator to classify MNIST dataset in one script.
This paper introduces also the basic idea of a artificial neural network.
https://github.com/telecombcn-dl/dlmm-2017-dcu
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
https://telecombcn-dl.github.io/2017-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
https://telecombcn-dl.github.io/2017-dlsl/
Winter School on Deep Learning for Speech and Language. UPC BarcelonaTech ETSETB TelecomBCN.
The aim of this course is to train students in methods of deep learning for speech and language. Recurrent Neural Networks (RNN) will be presented and analyzed in detail to understand the potential of these state of the art tools for time series processing. Engineering tips and scalability issues will be addressed to solve tasks such as machine translation, speech recognition, speech synthesis or question answering. Hands-on sessions will provide development skills so that attendees can become competent in contemporary data anlytics tools.
In this article you will learn hot to use tensorflow Softmax Classifier estimator to classify MNIST dataset in one script.
This paper introduces also the basic idea of a artificial neural network.
https://github.com/telecombcn-dl/dlmm-2017-dcu
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
https://telecombcn-dl.github.io/2017-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
The field of Artificial Intelligence (AI) has been revitalized in this decade, primarily due to the large-scale application of Deep Learning (DL) and other Machine Learning (ML) algorithms. This has been most evident in applications like computer vision, natural language processing, and game bots. However, extraordinary successes within a short period of time have also had the unintended consequence of causing a sharp difference of opinion in research and industrial communities regarding the capabilities and limitations of deep learning. A few questions you might have heard being asked (or asked yourself) include:
a. We don’t know how Deep Neural Networks make decisions, so can we trust them?
b. Can Deep Learning deal with highly non-linear continuous systems with millions of variables?
c. Can Deep Learning solve the Artificial General Intelligence problem?
The goal of this seminar is to provide a 1000-feet view of Deep Learning and hopefully answer the questions above. The seminar will touch upon the evolution, current state of the art, and peculiarities of Deep Learning, and share thoughts on using Deep Learning as a tool for developing power system solutions.
This presentation focuses on Deep Learning (DL) concepts, such as neural networks, backprop, activation functions, and Convolutional Neural Networks. You'll also learn how to incorporate Deep Learning in Android applications. Basic knowledge of matrices is helpful for this session, which is targeted primarily to beginners.
A fast-paced introduction to Deep Learning concepts, such as activation functions, cost functions, backpropagation, and then a quick dive into CNNs. Basic knowledge of vectors, matrices, and elementary calculus (derivatives), are helpful in order to derive the maximum benefit from this session.
Next we'll see a simple neural network using Keras, followed by an introduction to TensorFlow and TensorBoard. (Bonus points if you know Zorn's Lemma, the Well-Ordering Theorem, and the Axiom of Choice.)
Provides a brief overview of what machine learning is, how it works (theory), how to prepare data for a machine learning problem, an example case study, and additional resources.
M7 - Neural Networks in machine learning.pdfArushiKansal3
Neural networks are a set of algorithms, modeled loosely after the human brain, designed to recognize patterns. They interpret sensory data through a kind of machine perception, labeling, or clustering of raw input. The patterns they recognize are numerical, contained in vectors, into which all real-world data (images, sound, text, etc.) must be translated.
The presentation gives a brief introduction to artificial neural networks, their types, their characteristics and properties, their advantages and disadvantages, their architectures, different activation functions, popular learning approaches, the gradient descent algorithm, the back propagation algorithm and the application domains.
For the full video of this presentation, please visit:
https://www.edge-ai-vision.com/2021/02/introducing-machine-learning-and-how-to-teach-machines-to-see-a-presentation-from-tryolabs/
Facundo Parodi, Research and Machine Learning Engineer at Tryolabs, presents the “Introduction to Machine Learning and How to Teach Machines to See” tutorial at the September 2020 Embedded Vision Summit.
What is machine learning? How can machines distinguish a cat from a dog in an image? What’s the magic behind convolutional neural networks? These are some of the questions Parodi answers in this introductory talk on machine learning in computer vision.
Parodi introduces machine learning and explores the different types of problems it can solve. He explains the main components of practical machine learning, from data gathering and training to deployment. He then focuses on deep learning as an important machine learning technique and provides an introduction to convolutional neural networks and how they can be used to solve image classification problems. Parodi will also touches on recent advancements in deep learning and how they have revolutionized the entire field of computer vision.
This slide was used by Mr.Viju Chacko at FAYA:80 that gave a basic introduction to Ai. It act as an introduction to different terminologies related to AI that could enable its audience to understand the technology better.
A fast-paced introduction to Deep Learning (DL) concepts, such as neural networks, back propagation, activation functions, CNNs, RNNs (if time permits), and the CLT/AUT/fixed-point theorems, along with a basic code sample in TensorFlow.
During this session you will learn how to manually create a basic neural network that acts as a classifier, and also the segue from linear regression to a neural network.
You'll also learn about GANs (Generative Adversarial Networks) for static images as well as voice, and the former case, their potential impact on self-driving cars.
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptxEduSkills OECD
Andreas Schleicher presents at the OECD webinar ‘Digital devices in schools: detrimental distraction or secret to success?’ on 27 May 2024. The presentation was based on findings from PISA 2022 results and the webinar helped launch the PISA in Focus ‘Managing screen time: How to protect and equip students against distraction’ https://www.oecd-ilibrary.org/education/managing-screen-time_7c225af4-en and the OECD Education Policy Perspective ‘Students, digital devices and success’ can be found here - https://oe.cd/il/5yV
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
Ethnobotany and Ethnopharmacology:
Ethnobotany in herbal drug evaluation,
Impact of Ethnobotany in traditional medicine,
New development in herbals,
Bio-prospecting tools for drug discovery,
Role of Ethnopharmacology in drug evaluation,
Reverse Pharmacology.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Camp IT: Making the World More Efficient Using AI & Machine Learning
1. CAMP IT, MILÓWKA, 7/09/2019
Krzysztof Kowalczyk
Visiting Data Scientist at BCG GAMMA
President of Machine Learning Society at MIM UW
Making the World
More Efficient Using
AI & Machine Learning
2. 1
Mat Velloso (@matvelloso on Twitter, 17:25 - 22 Nov 2018)
Difference between machine learning and AI:
If it is written in Python, it's probably
machine learning
If it is written in PowerPoint, it's probably AI
3. 2
It is still important to know
some of the popular definitions,
especially when it comes to an
often misused term such as "AI".
Weak (narrow) AI
Can apply intelligence
to one specific task,
solving it at least as
good as a human being
It's already used
everywhere
Strong (general) AI
Has a capacity to
understand or learn any
intellectual task that a
human being can
It's debatable whether
it can exist
There is no clear
definition of what
AI means
5. 4
Example: house price
can be predicted using
weights tuned to match
historical data
house price = w1 ∙ area + w2 ∙ district + w3 ∙ price per m2 in city + …
Data: area, district, price per m2 in city, …
Model: w1, w2, w3, …
Prediction: house price
6. 5
What are the historical values
of area, district, average price
associated with house prices?
What model do we choose?
How to come up with an
accurate model?
How to measure model
quality?
What should values of w1,
w2, w3, … be equal to?
What price should we expect
for a house with a given area,
district, average city price?
Data Model Parameters Prediction
Human work is still necessary to make machine learning models work
house price = w1 ∙ area + w2 ∙ district + w3 ∙ price per m2 in city + …
Comes from historical
observations, usually needs to
be cleaned by a human
Typically it’s a human job to
design or select an
appropriate model and
quality metrics
This part is usually done by
computers, based on
historical data and specified
model
Correct model and
parameters enable automatic
prediction of price for new
houses
7. 6
Data science is a human process of analyzing and
cleaning historical data, in order to design
a model that can be trained to predict future values
of some variable based on new data
Machine learning is an automated process of finding
optimal model parameters (training), based on
historical data and an evaluation method specified
by a human Data Scientist
Data science vs machine learning
9. 8
Predicting value of a continuous
variable (house price, air
temperature, airplane delay)
Predicting value of a categorical
variable (plant species, illness,
handwritten characters)
Separating data into groups (types of
customers, similar parts of image,
spam e-mails)
Regression Classification Clustering
Machine learning tasks are easy to understand
10. 9
Linear regression – the simplest machine learning model
Assume target variable is a weighted mean of features:
y = w1 ∙ x1 + w2 ∙ x2 + … + wn ∙ xn
Given historical values of xs and ys,
find weights (ws) that minimize mean squared error:
MSE = meanover all historical samples([ytrue– ypredicted]2)
where for each historical sample:
ypredicted = w1 ∙ x1 + w2 ∙ x2 + … + wn ∙ xn
11. 10
Idea
• change the weights in direction that will decrease the error
• repeat until local minimum is reached
How to know whether to increase or decrease a weight?
Calculate the derivative of function:
J(w) = MSE(prediction(w), y)
where:
prediction(w) = w1 ∙ x1 + w2 ∙ x2 + … + wn ∙ xn
note that X=(x1, x2, …, xn) and y are just constants from historical data:
• matrix consisting of vectors of features' values
• vector of values for target variables
Learning algorithm 1: gradient descent
13. 12
Good features are more important that the sophisticated model – example:
By using radial basis functions (left picture) we can easily model seasonality (right picture).
Some other use cases for regression (including the linear regression):
life expectancy analysis, stock market prediction, airplane delay forecasts
An excellent talk on feature engineering for simple models: https://www.youtube.com/watch?v=68ABAU_V8qI
Visualizations: https://facebook.github.io/prophet/docs/quick_start.html
Regression:
Predicting demand for
store items
14. 13
Decision tree – a simple classification model
Build a binary tree where each node makes a
binary decision and leaves contain classes.
Algorithm:
• Select the best feature and threshold value
to make a decision
• Repeat recursively until all values matching
previous decisions have same class
How to select best feature for a tree node?
• Gini impurity
• Information gain
15. 14
Text
Gradient boosting is a model ensembling algorithm – it creates a model by combining other
models. It's often applied to trees because of their simplicity and relatively fast generation.
The algorithm:
• Fit a weak learner (any simple model – usually a decision tree) on a random small subset of
historical data and add it to a pool of already trained weak learners
• Increase weights of other samples that were wrongly predicted by the newly created tree
• Create a new tree and training it on the subset (sampled with weights, that reflect which
samples are poorly predicted by the trees we already have)
Gradient boosted trees are in practice the most often used classifier. They are much cheaper than
even the most sophisticated neural networks to train and maintain, and keep the explainability
properties of ordinary decision trees (you can always see their gain distributions)
More about gradient boosting:
https://medium.com/mlreview/gradient-boosting-from-scratch-1e317ae4587d
More about ensembling models in general: https://mlwave.com/kaggle-ensembling-guide/
By applying gradient boosting we can make weak models very powerful
16. 15
Clustering: an unsupervised machine learning task
Unsupervised machine learning is a family of
algorithms that work without labeled data.
They are often used for grouping similar data
points, to find similar ones or to identify
anomalies.
Breakthroughs in unsupervised learning
may be necessary for general-purpose AI.
Clustering is the process of separating data
points into groups of similar ones.
Note the difference: on the image above,
the algorithm identified similar pixels – not
classified them as human, car or sign!
Popular algorithms: K-Means, Agglomerative
Hierarchical Clustering, DBSCAN
Read more about clustering algorithms: https://scikit-learn.org/stable/modules/clustering.html
18. 17
Brain is a network of cells called neurons,
which can receive, process and send
signals to other cells
Similarly, artificial neural network is made
of layers of nodes, each node transforms
data and passes it on to the next layer
Computer science can
be inspired by biology
19. 18
Neuron – the building block of artificial neural networks
vk
x1
x2
xm
wk,1
wk,2
wk,m
yk
Φ(.)
xi – inputs
wk,i – neuron weights
vk – weighted inputs
Φ – transfer function
yk – output of the neuron
yk = Φ(vk) = Φ(wk,1x1 + wk,2x2+ … + wk,mxm)
20. 19
Neural networks are solving a simple optimization problem
By using gradient descent & backpropagation we can converge to a local minimum of any differentiable function
DATA
INPUT
LAYER
HIDDEN
LAYER
OUTPUT
LAYER
92%
HOT DOG
8%
PIZZA 0.08
TRUE VALUE
HOT DOG
PREDICTION LOSS FUNCTION VALUE
This can be any differentiable
function that we wish to minimize
UPDATE WEIGHTS (BACKPROPAGATION):
1. Calculate the gradient (derivative
of loss function wrt weights of the
network) for the output layer
2. Update weights of the output layer in
correct directions
3. Recursively calculate gradient &
update all other layers (always using
outputs from the following layer to
calculate gradient)
REPEAT UNTIL LOSS FUNCTION MINIMUM IS REACHED
FEED-FORWARD NEURAL NETWORK (PERCEPTRON)
This is a simplified example – in real life we always use deep neural networks, which have
much more hidden layers. There are also other types of layers than feed-forward ones.
21. 20
Architecture:
Output one value for every variable you want to predict
Architecture:
Output one value for every possible class
Regression Classification
Neural network: architecture + loss function = selection of machine learning task
Price of Google stock
Price of Apple stock
Probability of input being letter "A"
Probability of input being letter "B"
Probability of input being letter "C"
Loss function:
Mean square error (measure of distance between predicted and actual values)
MSE = mean([ytrue– ypred]2)
Loss function:
Cross-entropy (measure of error between probability distributions)
LogLoss = – sum(ytrue ∙ log(ypred))
How to select an appropriate loss function for your problem? Use a maximum likelihood estimator1
How can we make sure our output probabilities are in [0,1]? Use a sigmoid function2
1. https://machinelearningmastery.com/loss-and-loss-functions-for-training-deep-learning-neural-networks/
2. https://en.wikipedia.org/wiki/Sigmoid_function
22. 21
• 1943: First mathematical model of neurons, implemented with electrical
circuits (Warren McCulloch, Walter Pitts)
• 1957: Definition & implementation of perceptron: a single-layer neural
network, exactly same as we use today (Frank Rosenblatt)
• 1969: Proof that perceptron is limited: cannot solve the XOR problem
(Marvin Minsky, Seymour Papert)
• 1986: Neural networks with multiple layers can in theory learn non-linear
functions using backpropagation (Geoffrey E. Hinton). Despite rising
interest in neural networks and discovery of new layers (convolutional,
recurrent) they are simply too inefficient to be applied to any real problem
• 2012: Creation of AlexNet, an improved convolutional neural network,
trained using GPUs, that won a ILSVRC-2012 ImageNet classification
competition, beating classical machine learning methods (Alex Krizhevsky,
Ilya Sutskever Geoffrey E. Hinton)
• 2016: AlphaGo neural network model beats Lee Sedol in Go – a game
which has about 10^170 possible move combinations (comparing to about
10^50 for chess) and cannot be solved by even the most sophisticated
search algorithm (Google DeepMind)
A brief history of artificial neural networks
An old theory becomes possible thanks to Graphics Processing Units (GPUs)
Source: https://blogs.nvidia.com/blog/2016/01/12/accelerating-ai-artificial-intelligence-gpus/
Neural networks proved
suitable for solving real
problems only recently.
This change is mainly a
result of improvement in
GPU efficiency, as both
forward and backward
propagation in a network
are just a simple vector
and matrix operations.
23. 22
Deep learning eliminates the need for feature engineering
Humans did this part manually
a few years ago (2011)
Hot dog
Pizza
Number of round edges node
Yellow to green color proportion node
Pepperoni node
Sausage node
24. 23
Deep learning:
Image classification
Three popular neural network architectures used for image classification: ResNet with Skip (top), plain
convolutional network (middle), VGG-19 (bottom). Each box denotes a network layer,
"NxM conv K" denotes a convolutional layer with kernel dimensions (N,M) and K input variables.
Convolutional layers are extremely powerful for extracting information out of images, to read more
about them visit: https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-
networks-the-eli5-way-3bd2b1164a53
Image source: https://towardsdatascience.com/review-resnet-winner-of-ilsvrc-2015-image-
classification-localization-detection-e39402bfa5d8
25. 24
Deep learning: object detection & image segmentation
To create models that are actually useful in the real world, we need even more sophisticated network architecture
Example: to perform image segmentation (left picture), the network needs multiple
outputs (right picture) and a more complex loss function taking them all into account.
To read more on image segmentation see: https://arxiv.org/pdf/1703.06870.pdf
and the implementation: https://github.com/matterport/Mask_RCNN
26. 25
By representing data
as vectors, we can
apply deep learning
to any problem
Images were a natural first choice for deep learning applications: the image is essentially a matrix of
color values
Other types of data needs to be embedded into a fixed-size vectors in order to train neural networks
on it. For example, natural language processing (document classification, named entity detection)
usually requires mapping words into a vector space in a way that preserves semantic relationships
(as in the picture above).
For this purpose, unsupervised models such as word2vec and GloVe were invented.
king – man + woman = queen
27. 26
Some models allow to create
new data from the same
distribution as historical data.
Neural networks can have this
property too, giving computers
ability to generate images,
films, music or stories.
Instead of training models on historical data,
we can have an environment which reacts to
network outputs. This approach is perfect for
train game bots, or even real robots!
Generative models Reinforcement learning
Neural networks expand the scope of machine learning
Image sources: https://prisma-ai.com/ https://openai.com/blog/how-to-train-your-openai-five/
28. 27
Generative adversarial networks – a simple trick that makes models generative
RANDOM
NOISE
REAL DATA
GENERATOR NETWORK
Tries to fool discriminator to classify
generated data as real
PREDICTION: IS THE DATA REAL?
DISCRIMINATOR NETWORK
Predicts the probability of
the data being a real
FAKE DATA
LOSS FUNCTION VALUE
Is also propagated to the generator network,
to improve its generation ability
30. 29
Reinforcement learning: a new task in the realm of AI
The one thing game bots, industrial robots and self-driving cars have in common
INTERPRETER
ENVIRONMENT
REWARD
AGENT
STATE REPRESENTATION
STATE ACTION
Neural network:
input=state,
result=action
Backpropagation:
calculates loss based on
rewards received from the
environment
Agents often need to combine multiple
machine learning tasks:
- Classification to select an action:
move, shoot, etc.
- Regression to select action values:
how much power to use,
which direction to shoot
31. 30
Reinforcement learning: can we make AI curious?
Right now, even the most sophisticated models can look stupid in some situations
By adding a penalty to the loss function when the agent can accurately predict what's
going to happen in the environment, we can trick it to explore more, showing behavior
similar to human curiosity. This allowed OpenAI to create a state-of-the-art model for
Montezuma's Revenge – a game that requires a lot of exploration to earn points.
However, such creative agents are still very stupid – placing such AI in a Quake
environment with a TV on a wall made AI stare at it forever (left animation).
While TV is off the agent continues to explore the world (right animation).
https://openai.com/blog/reinforcement-learning-with-prediction-based-rewards/
33. 32
Common foundation for doing machine learning
KNOWLEDGE TOOLS SKILLS
Probability theory
Statistics
Machine learning models
Evaluation metrics
Python
SQL
Numpy & Pandas
Scikit-Learn
Data exploration
Model selection
Model evaluation
Data visualization
34. 33
Using machine learning is simple thanks to many good libraries and extremely helpful community
You can easily achieve good results without understanding what you're doing in detail
Solutions to real-world problems often need to be theoretically sound and explainable
Theoretical background
- is it really necessary?
35. 34
Universities: Top AI & Machine Learning research hubs worldwide
• Carnegie Mellon University
• Cornell University
• Tsinghua University
• Stanford University
• Technion
• Peking University
• Georgia Institute of Technology
• Massachuesetts Institute of Technology
• National University of Singapore
• HKUST
• Chinese Academy of Sciences
• Columbia University
• University of California – Berkley
• University of Toronto
• University of Illinoi at Urbana-Champaign
• University of Maryland – College Park
Machine learning & artificial intelligence are extremely international subjects
– most researchers travel around the world to work or present their results.
It's not that important to be in a top university worldwide – what matters,
is that you have access to international conferences & local researchers
that are up-to-date with latest findings in machine learning.
In practice, being in the top university in your country is enough,
unless you don't want to get into research (but I strongly encourage to try)
Most university rankings are created based on the number of publications in
certain domains – this isn’t always the best metric. Always try to find what
people are giving lectures and what kind of research are they doing exactly.
Top AI & ML Universities Worldwide1 Practical tips on choosing where to apply
1. Ranked by number of publications in AI, ComputerVision, ML & Data Mining, Natural Language Processing, The Web & Information Retrieval from 1990 to 2019. Source: http://csrankings.org
36. 35
Learn the tools by using
them in your projects
Find an interesting problem
Design your solution
Implement it, learn
new tools if necessary
Measure & evaluate results
Find alternative
approach, repeat
37. 36
• Competitions – solve problems submitted by real
companies, win prizes (or at least get experience)
• Datasets & notebooks
• Explore solutions of other people shared as
public notebooks
• Use hosted Jupyter Notebook with easy access
to datasets & a free GPU
• Edit other people's notebooks to improve
or play with their solutions
• Discussion – data science community is very helpful,
even during competitions people often share tips
• Courses – new feature of Kaggle, for a start
this is probably where you want to go
Text Text
Popular source of interesting projects #1
Taking part in data science competitions
is often the best way to learn & test skills
Kaggle – the cornerstone of
every data science career
Website: https://www.kaggle.com/
38. 37
• Assumes that you already know how to program
(and know basics of Python as well)
• Focused on practical aspects of machine learning
• Most courses focus on deep learning (neural networks)
• You won't learn much theory there
• Plenty of practical tips:
• Where to get free (or cheapest) computing
power – especially GPUs
• Best practices when working with machine
learning models & deploying them
• Links to tutorials & other relevant content
• Community is more goal-oriented (similar to coding
bootcamp) rather than process-oriented
(similar to university)
Popular source of interesting projects #2
If you are more of a developer than a
mathematician, this courses are for you
Fast.ai – hack your way
into machine learning
Website: https://www.fast.ai/
39. 38
• Machine learning subjects
• Quality depends heavily on your university
• Can be a good way of getting into ML & AI
• Students societies
• Usually ran by people interested in machine
learning and willing to share their skills
• Good places to look for competition teammates,
project ideas and new knowledge sources
• Machine learning & AI research groups
• Researchers often could use free help, and will
teach you necessary skills to make it possible
• This is excellent way of getting an interesting
topic for your BSc, MSc or PhD
• If you're in one of the top universities worldwide,
ML & AI subjects should be the best way to learn
• Otherwise, being in the top university in any country
should still allow you to meet skilled researchers
Popular source of interesting projects #3
Enroll in a machine learning subject or reach
out to fellow students & researchers
University: student societies
& research groups
Machine Learning Society at University of Warsaw: http://knum.mimuw.edu.pl
40. 39
Share your work with others
Take time to clean and publish your work – make it easy for other people to discover and use
GitHub
• A perfect place to
store & showcase
your code
• You will use git to
manage your source
code version anyway
• Nice place to
discover new
software projects
Kaggle Notebooks
• If you're working on
Kaggle, publishing a
notebook is easy
• Very useful during
competitions, to
suggest ideas or
present your results
• Attracts less people
but more discussion
than GitHub project
Medium blogs
• If you like to write,
consider making a
small blog
• Platforms like
Medium & LinkedIn
make it easy to
publish new articles
• Depending on your
taste, you may
LinkedIn Slideshare
• When you make a
presentation, post it
to Slideshare in
order to reach a
wider audience
• Using LinkedIn will
also make your work
easy to discover
41. 40
Video tutorials
• Almost always too simplified
• Only introduce confusion when you really
need to understand what you're doing
Entry-level courses
• Aimed at people without programming or
math skills
• Not approved by any scientific institution
Online communities
• Social media groups
Blog posts
• Published by top researchers & professionals
Kaggle notebooks
• Less popular ones are almost never correct
• Sometimes, notebook is not very popular,
but is approved by many experienced users
(by commenting / upvoting)
• Ones that are both popular and approved by
many experienced users can be quite
reliable – also, seeing references to related
research papers is a good sign
Presentations
• From local meetups (student club meetings,
smaller conferences, etc.)
Research papers
• Popular (referenced often)
• Reviewed & publish in a journal
Library documentation
• Or alternatively, GitHub readme and issues
Professional forums
• Stackoverflow, stackexchange
• Only answers that are both valid & upvoted
University lectures
• Can be outdated, but always correct
• Top universities (MIT, Stanford) also publish
their lectures & related materials online
Presentations
• From top scientific & technical conferences
(PyCon, PyData, NIPS, ICML, etc.)
Likely unreliable Simplified, but still useful Mostly correct
Always use the most reliable source of knowledge
43. 42
AI Researcher ML Developer Data Scientist Data Engineer
Do what you enjoy the most: 4 typical career paths
Optimizes models that are
already proven to solve a
specific problem well
Necessary only in projects
where model inference
cost is very high
Works with algorithms &
code, rather than data
Chooses the right tools
and methods to solve a
specific, real-life problem
In smaller companies also
performs development &
data engineering work
Uses existing libraries &
tools, spends more time
interacting with people
Manages the flow of data
from its sources to final
model predictions
Crucial in projects with
high data throughput or
real-time predictions
Uses database & pipeline
management tools
Designs new algorithms or
applies existing ones to
new domains
Typically works at a
university or research
team in large company
Implements things from
scratch, uses well-known
data to compare results
44. 43
AI researcher – exploring the unknown
Designs new algorithms or applies existing ones to new domains
Skills
Understand optimization algorithms in detail
Follow scientific process of hypotheses testing
Create deep learning models from scratch
Tools
High-performance programming language
Experiment & data management tools
Low-level machine learning libraries
Responsibilities
Stay up to date with current research
Teach & share knowledge with others
Adapt fresh research to new domains
Products
Research papers & other publications
Tools for developers & data scientists
Contributions to open-source projects
45. 44
ML developer – working close to the metal
Optimizes models that are already proven to solve a specific problem well
Skills
Understand optimization algorithms in detail
Trade off accuracy for speed or memory
Develop efficient & correct software
Tools
High-performance programming language
High- and low-level ML libraries
Testing, build & deployment tools
Responsibilities
Stay up to date with current research
Improve efficiency of existing software
Help others improve code efficiency
Products
Optimized machine learning models
Models ported to embedded devices
Workshops & talks on code efficiency
46. 45
Data scientist – the sexiest job of the XXI century?
Chooses the right tools and methods to solve a specific, real-life problem
Skills
Analyze data & create visualizations
Finding the right model for a given task
Measure & compare model effectiveness
Tools
High-level machine learning libraries
Data manipulation & visualization tools
Optimization & simulation tools
Responsibilities
Solve real-world business problems
Create models & prove their effectiveness
Gather insights from large pools of data
Products
Presentation of business problem solution
Model optimally solving a given problem
Dashboards clearly visualizing data
47. 46
Data engineer – making the most of "big data"
Manages the flow of data from its sources to final model predictions
Skills
Design software architecture & data model
Choose right databases & compute platform
Integrate & manage many data sources
Tools
High-level machine learning libraries
Databases & pipeline management tools
Cloud infrastructure monitoring tools
Responsibilities
Optimize data processing for a given model
Improve speed of model inference
Stay up to date with software & infrastructure
Products
Efficient pipeline for data processing
Models scaled to handle "big data"
Dashboards clearly visualizing data
48. 47
PhD in computer science,
mathematics or related field
Algorithms & data structures
Advanced machine learning
Numerical methods
Low-level ML library1
High-performance language2
BSc in computer science, related
industry experience or projects
Algorithms & data structures
Parallel programming
Software & systems architecture
Low-level ML library1
High-performance language2
MSc in mathematics, statistics,
computer science or related field
Advanced probability & statistics
Advanced machine learning
Probabilistic programming
Data visualization libraries3
Optimization algorithms & tools
BSc in computer science or
experience with large datasets
Software systems architecture
Parallel programming
Distributed systems
Databases (SQL and No-SQL)
Pipeline management tools4
AI Researcher ML Developer Data Scientist Data Engineer
Specializations have flexible requirements and a common foundation
EXPERIENCE
TO GET
KNOWLEDGE
TO LEARN
TOOLS
TO MASTER
1 Low-level ML libraries: Tensorflow, Pytorch, Theano
2 High-performance languages: C++, Scala, Go, Rust
3 Data visualization libraries: Matplotlib, Seaborn, Plotly, Bokeh
4 Pipeline management tools: Spark, Airflow, Dagster, DVC
SKILLSKNOWLEDGE TOOLS
49. 48
Learn the basics Try different specializations Dig deeper when it's interesting
No matter what you decide to do in your life, knowing
the basics of machine learning won't hurt
Start by taking university classes or online courses
(or both, one may not be enough to start quickly)
Take part in machine learning competitions,
don’t hesitate to share your successes
(or takeaways from failures)
Write a personal project or publish
code you already have
Do everything to get your first internship or job,
after that it gets much easier to try new things
Learn enough to get an internship in one of the 4
roles
When working, learn as much new tools as possible,
you will likely continue to use at least some of them
The same is true for people: get to know as many as
you can – some of them may help you in the future,
others may need help from you
Internships are a perfect way of exploring different
career paths – you don't need to get the first one right
Note down your experiences and take them into
account when applying for the next job - until you
find one you know you can do longer than 3 months
The more interesting the problem you are facing,
the more time you should take to explore it
If people want you to do other things,
learn to politely say no – more often than not
it will be beneficial for everyone
Value your time – if you get this far, it's likely you will
find most problems interesting
Find mentors – people a little more experience than
you, don't fear reaching out to them
How to choose the right development path?
50. 49
Final thought: AI is not limited to machine learning
CURRENT MACHINE LEARNING SYSTEMS
CANNOT SOLVE ALL OF OUR PROBLEMS
KNOWLEDGE REPRESENTATION
PLANNING & SCHEDULING
RESOURCE ALLOCATION
EXPERT SYSTEMS
CONSTRAINT SATISFACTION PROBLEM
51. 50
Solving the unsolvable: NP-complete problems in artificial intelligence
How quantum computers and unsupervised learning can bring us closer to general AI
PLANNING & SCHEDULING
RESOURCE ALLOCATION
CONSTRAINT SATISFACTION PROBLEM
KNOWLEDGE REPRESENTATION
EXPERT SYSTEMS
Currently solved using approximation algorithms,
quantum computers will make optimal solutions achievable
Currently built manually using well-established algorithms,
breakthroughs in unsupervised learning have potential
to make this process much more efficient