Artificial Intelligence has had a transformative impact on the modern economy. Technologies like machine learning, deep learning, and neural networks are being combined with cloud computing, open source software tools, and big data to power unprecedented breakthroughs in AI and revolutionize Finance. From compliance to traders, AI is being used across the industry to automate repetitive tasks, improve efficiency, and augment human decision making. Download full version: https://streamdata.io/resource-library/

1. White Paper
Artificial Intelligence in Finance
http://streamdata.io
Prepared by:
Theodore Marrinan
& Kin Lane
tmarrinan@gmail.com
kinlane@gmail.com

2. Computer scientist John McCarthy was the first to
coin the phrase artificial intelligence (AI) in 1956
at Dartmouth College while organizing a conference
to examine how machines could be made to
simulate aspects of human intelligence. Since that
foundational conference, AI has developed into an
integral part of daily modern life and transformed
our economy. From internet search engines to
autonomous vehicles, AI has rapidly evolved
into a critical modern technology with extensive
applications in numerous fields. According to the
Harvard Business Review, AI has become “the
most important general-purpose technology of our
era”. AI leverages computers’ intrinsic advantages
over humans like speed, accuracy, and memory
storage. Accordingly, implementing AI can make
many processes more efficient and effective for both
businesses and researchers alike.
Forms of Artificial Intelligence (AI)
• Artificial Intelligence is a diverse field that aims
to enable computers to mimic aspects of human
intelligence.
• Machine Learning is a form of AI that teaches
itself and learns from provided data sets.
• Deep Learning is a form of ML that uses multiple
levels of abstraction to form a hierarchy of
concepts.
• Supervised Learning is a form of ML that uses
labeled training data where respective inputs and
outputs are paired together.
• Unsupervised Learning is a form of ML that uses
unlabeled training data where no categorization
of the data is provided.
• Semi-supervised Learning is a hybrid form of ML
that uses a combination of labeled and unlabeled
training data.
Within the field ofAI, the majority of recent research
and development efforts have primarily focused on
Artificial Intelligence has had a transformative impact on the modern economy. Technologies like machine learning,
deep learning, and neural networks are being combined with cloud computing, open source software tools, and big
data to power unprecedented breakthroughs in AI and revolutionize Finance. From compliance to traders, AI is being
used across the industry to automate repetitive tasks, improve efficiency, and augment human decision making.
artificially intelligent programs that can learn and
teach themselves to adapt to data called Machine
Learning (ML). ML uses a variety of algorithms to
parse data, learn from it, and determine or predict
some characteristic or value. Thanks to advances in
computing and the Big Data movement, powerful
ML algorithms can now parse vast new data sets and
generate unprecedented insights for businesses and
researchers. ML techniques have powered some of
the greatest breakthroughs in the field ofAI, and their
broad applicability has led to widespread adoption.
ML saves both time and money, as previously mind-
numbing tasks can now increasingly be automated,
freeing up human talent for higher level work with
greater added value. ML techniques are generally
most effective when there are large historical data
sets available for training the algorithms.
Training is a term used to describe the initial
optimization of general ML techniques for specific
applications. Algorithms are “trained” as they
analyze and learn from a provided training data
set. Once they are trained, algorithms can use what
they learned to analyze new, previously unforeseen
data. Training is a critical process for effective
ML, as it forms the basis for how the algorithms
actually function. Training can entail supervised,
semi-supervised, or unsupervised learning methods
depending on the type of training data provided and
the ML algorithm used.
There are also several factors to consider when
choosing between supervised and unsupervised ML
methods. Oftentimes, the structure and quantity
of available training data will dictate which kind
of algorithm is selected. In general, supervised
learning methods will normally require more effort
to set up than unsupervised methods, as all of the
training data must first be accurately labeled before
any training can start. Labeling training data can
be time-consuming, particularly if the data set is
large or complex, but it can provide some important
benefits. Namely, labeling specific input and
output pairs can create a structure for the data that
algorithms can then learn. Knowing the structure of
data can greatly improve the accuracy of algorithms’
Artificial Intelligence in Finance

3. predictions or classifications, so long as the labeled
structure is actually an accurate representation of the
real structure. Conversely, if there is no previously
knowndatastructure,unsupervisedlearningmayhelp
generate new insights into the data. Unsupervised
learning methods can therefore often be particularly
helpful when one is analyzing massive data sets that
are too large to be labeled. Some ML algorithms,
like the Naïve Bayes Classifier, are particularly
useful in applications where there is little training
data available and a fast approximation is required.
Other ML algorithms, like Neural Networks, excel
in applications where there are vast amounts of data
that can be used for training.
Currently, most practical ML applications utilize
supervised learning methods. Supervised learning is
popular because labeling can be a way to represent
data’s underlying structure in a form algorithms can
easily understand. This structural information can
help improve ML algorithms accuracy and greatly
reduce the amount of time needed for training. As
the amount of data produced by modern society
has continued to grow at an exponential pace and
unlabeled data is generally more abundant than
labeled data, semi-supervised and unsupervised
learning methods are increasingly thought to be the
future of ML.
When training ML algorithms, there are a number
of important elements one must consider. Firstly, it
is important to have some understanding of the level
of complexity possessed by the underlying function
that the ML algorithm is attempting to learn. If the
underlying function is complex, it will be necessary
to use very large training data sets and a more
flexible learning algorithm like a neural network.
In general, the more training data that is available
the better. Another important consideration is the
available training data’s quality. If the data is very
noisy or contains a lot of redundant information,
ML algorithms will generally perform poorly. After
training, it is also very important to test the accuracy
of algorithms with new data points that were not
used for training to help detect overfitting. When an
algorithm’sperformanceismaximizedforsomesetof
training data, it can often follow the specific training
data too closely instead of actually approximating
the underlying model structure. This is a problem
known as overfitting, and it causes algorithms to
have higher error rates when presented with new
data points. Testing algorithms with new data can
also provide important information regarding the
accuracy of their classifications or predictions.
Often as time marches on and new data points are
created, the underlying model that a ML algorithm
is trying to approximate can shift and change.
For example, if you trained a ML algorithm using
data from a bullish period in the stock market, the
resulting algorithm would likely perform well when
there is a bull market. However, if the economy were
to go into a recession and the stock market shifted to
a bearish period, the algorithm trained on a bullish
period would likely not perform as well, since the
underlying model has changed from a bull market
to a bear market. This concept drift occurs when
the statistical properties of a target variable change
over time, and it is an unavoidable part of modelling
complex phenomenon.
There are two main methods for adjusting for
inevitableconceptdrifts.Thefirstisanactivesolution
that relies on a triggering mechanism like a change-
detection test. This method is a form of statistical
analysisthatseekstoexplicitlyidentifywhenconcept
driftoccurssothatanew,moreaccuratemodelmaybe
implemented. The second method is a more passive
solution that focuses on retraining models using the
most recently received data, thereby continuously
updating it. Econometrically speaking, concept drift
is often the result of unobserved contextual data,
like seasonality or the business cycle. By adding in
useful additional data, it may be possible to greatly
reduce the deterioration in model performance
caused by concept drift. For example, if a model
used macroeconomic data to predict consumer sales
and its performance declines due to concept drift, the
model’s performance could possibly be bolstered by
adding seasonal data into the model. By helping to
contextualize trends like holiday shopping, concept
drift’s negative effects could potentially be reduced,
or even eliminated. While contextual data may
help to reduce some instances of concept drift, not
all trends can necessarily be easily contextualized
through additional data, and new factors may arise
or known factors may change at any time. Despite
these limitations, adding new contextual data to
models can still help to minimize concept drift’s
harmful effects on performance, particularly if a
second adjustment method like continuous retraining
is also used.
ML has developed a number of highly capable and
powerful techniques over the years that are used for a
widearrayofdifferentapplications.Whilestilllargely
basedonprobabilitytheoryandstatisticaltechniques,
ML has also come to incorporate concepts from a