This document discusses autonomous learning for autonomous systems. It begins with an outline and context, noting the huge volumes of streaming data from sources like the internet of things. True intelligence is described as evolving intelligence that can adapt to non-stationary data streams.
Case studies on applications are presented, including fast, transparent deep fuzzy rule-based learning for image classification. Other applications discussed include real-time video processing to detect moving targets, self-calibrating sensors in oil refining, and high frequency trading.
Technical challenges of data streams are described, such as the inability to analyze data offline and the need for online, real-time models. Historical approaches to evolving intelligent systems are also summarized.
18 common techniques used across various disciplines of computer science, which are: caching, blocking vs. non-blocking, profiling, pipelining, speculation (prediction), relaxation, eager vs. lazy, filtering, reuse, parallelization, coupling vs. decoupling, multilevel / clustering, lookahead / prefetching, dynamic (adaptive), replication, virtualization, checkpoints / snapshots, and transactions.
Streaming Random Forest Learning in Spark and StreamDM with Heitor Murilogome...Databricks
We present how to build random forest models from streaming data. This is achieved by training, predicting and adapting the model in real-time with evolving data streams. The implementation is on the open source library StreamDM, built on top of Apache Spark.
Deep Learning: Chapter 11 Practical MethodologyJason Tsai
Lecture for Deep Learning 101 study group to be held on June 9th, 2017.
Reference book: https://www.deeplearningbook.org/
Past video archives: https://goo.gl/hxermB
Initiated by Taiwan AI Group (https://www.facebook.com/groups/Taiwan.AI.Group/)
In planet-scale deployments, the Operation and Maintenance (O&M) of cloud platforms cannot be done any longer manually or simply with off-the-shelf solutions. It requires self-developed automated systems, ideally exploiting the use of AI to provide tools for autonomous cloud operations. This talk will explain how deep learning, distributed traces, and time-series analysis (sequence analysis) can be used to effectively detect anomalous cloud infrastructure behaviors during operations to reduce the workload of human operators. The iForesight system is being used to evaluate this new O&M approach. iForesight 2.0 is the result of 2 years of research with the goal to provide an intelligent new tool aimed at SRE cloud maintenance teams. It enables them to quickly detect and predict anomalies thanks to the use of artificial intelligence when cloud services are slow or unresponsive.
18 common techniques used across various disciplines of computer science, which are: caching, blocking vs. non-blocking, profiling, pipelining, speculation (prediction), relaxation, eager vs. lazy, filtering, reuse, parallelization, coupling vs. decoupling, multilevel / clustering, lookahead / prefetching, dynamic (adaptive), replication, virtualization, checkpoints / snapshots, and transactions.
Streaming Random Forest Learning in Spark and StreamDM with Heitor Murilogome...Databricks
We present how to build random forest models from streaming data. This is achieved by training, predicting and adapting the model in real-time with evolving data streams. The implementation is on the open source library StreamDM, built on top of Apache Spark.
Deep Learning: Chapter 11 Practical MethodologyJason Tsai
Lecture for Deep Learning 101 study group to be held on June 9th, 2017.
Reference book: https://www.deeplearningbook.org/
Past video archives: https://goo.gl/hxermB
Initiated by Taiwan AI Group (https://www.facebook.com/groups/Taiwan.AI.Group/)
In planet-scale deployments, the Operation and Maintenance (O&M) of cloud platforms cannot be done any longer manually or simply with off-the-shelf solutions. It requires self-developed automated systems, ideally exploiting the use of AI to provide tools for autonomous cloud operations. This talk will explain how deep learning, distributed traces, and time-series analysis (sequence analysis) can be used to effectively detect anomalous cloud infrastructure behaviors during operations to reduce the workload of human operators. The iForesight system is being used to evaluate this new O&M approach. iForesight 2.0 is the result of 2 years of research with the goal to provide an intelligent new tool aimed at SRE cloud maintenance teams. It enables them to quickly detect and predict anomalies thanks to the use of artificial intelligence when cloud services are slow or unresponsive.
This Tutorial will discuss and demonstrate how to implement different realtime streaming analytics patterns. We will start with counting usecases and progress into complex patterns like time windows, tracking objects, and detecting trends. We will start with Apache Storm and progress into Complex Event Processing based technologies.
With the rise of IoT and the increasing complexity of applications, clouds, networks and infrastructure, the battle to keep your data and your infrastructure safe from attackers is getting harder. As groups of bad actors collaborate, sharing information and offering illegal access, and botnets as a service, terabits of attack can be launched cheaply. Meanwhile, it’s hard to find enough security analysts to catch and prevent these attacks.
This is where community collaboration and open source efforts like Apache Metron come in. Metron presents a comprehensive framework for application and network, security built on Apache Hadoop and open source Streaming Analytics(ie Apache Nifi, Apache Kafka) tool’s highly scalable data management and processing stacks. Advanced features like profiling, machine learning, and visualization work with real-time streaming detection to make your SOC analysts more efficient, while the intrinsic extensibility of open source helps your data scientists get security insights out of the lab and into production fast.
We will discuss and demonstrate how some real-world businesses and managed service providers are using Apache Metron to identify and solve security threats at scale, and some approaches and ideas for how the platform can fit into your security architecture.
Speaker: Laurence Da Luz, Senior Solutions Architect, Hortonworks
A Science Information Committee (SIC) presentation authored by Smith, V.S., Blagoderov, V., Kitching, I. and Simonsen T., given at the Natural History Museum, London, UK. May 14th, 2010.
Workshop nwav 47 - LVS - Tool for Quantitative Data AnalysisOlga Scrivner
In the format of hands-on session, this workshop will introduce participants to the Language Variation Suite (LVS), a user-friendly interactive web application built in R. LVS provides access to advanced statistical methods and visualization techniques, such as mixed-effects modeling, conditional and random tree analyses, cluster analysis. These advanced methods enable researchers to handle imbalanced data, measure individual and group variation, estimate significance, and rank variables according to their significance.
Automated machine learning lectures given at the Advanced Course on Data Science & Machine Learning. AutoML, hyperparameter optimization, Bayesian optimization, Neural Architecture Search, Meta-learning, MAML
This talk was presented in Startup Master Class 2017 - http://aaiitkblr.org/smc/ 2017 @ Christ College Bangalore. Hosted by IIT Kanpur Alumni Association and co-presented by IIT KGP Alumni Association, IITACB, PanIIT, IIMA and IIMB alumni.
My co-presenter was Biswa Gourav Singh. And contributor was Navin Manaswi.
http://dataconomy.com/2017/04/history-neural-networks/ - timeline for neural networks
Provenance for Data Munging EnvironmentsPaul Groth
Data munging is a crucial task across domains ranging from drug discovery and policy studies to data science. Indeed, it has been reported that data munging accounts for 60% of the time spent in data analysis. Because data munging involves a wide variety of tasks using data from multiple sources, it often becomes difficult to understand how a cleaned dataset was actually produced (i.e. its provenance). In this talk, I discuss our recent work on tracking data provenance within desktop systems, which addresses problems of efficient and fine grained capture. I also describe our work on scalable provence tracking within a triple store/graph database that supports messy web data. Finally, I briefly touch on whether we will move from adhoc data munging approaches to more declarative knowledge representation languages such as Probabilistic Soft Logic.
Presented at Information Sciences Institute - August 13, 2015
A presentation for the Ottawa AI/ML Meetup Group, Sept 9, 2019.
A person’s Facebook newsfeed from their vacation is to the real-world as Kaggle competitions are to industrial ML/AI projects. This presentation shares with the audience some common challenges in ML/AI that real-world practitioners face while doing their craft, along with suggestions on how to address them. This is a non-technical presentation but does assume some knowledge of data science process and ML/AI.
Machine learning applications in clinical brain computer interfacingJenny Midwinter
Brain computer interfaces (BCIs) provide alternative communication channels from the brain to external devices for severely disabled individuals and can be used to induce and guide adaptive plasticity for recovery after central nervous system trauma. Clinical BCI effectiveness depends on robust and accurate modeling of the relationship between brain signals and behaviour. Dr. Boulay will give a brief survey of BCI technologies and discuss common BCI paradigms and implementations, with an emphasis on clinical BCI brain signals and machine-learning applications.
* From the Ottawa AI/ML Meetup June 2018.
More Related Content
Similar to Autonomous Learning for Autonomous Systems, by Prof. Plamen Angelov
This Tutorial will discuss and demonstrate how to implement different realtime streaming analytics patterns. We will start with counting usecases and progress into complex patterns like time windows, tracking objects, and detecting trends. We will start with Apache Storm and progress into Complex Event Processing based technologies.
With the rise of IoT and the increasing complexity of applications, clouds, networks and infrastructure, the battle to keep your data and your infrastructure safe from attackers is getting harder. As groups of bad actors collaborate, sharing information and offering illegal access, and botnets as a service, terabits of attack can be launched cheaply. Meanwhile, it’s hard to find enough security analysts to catch and prevent these attacks.
This is where community collaboration and open source efforts like Apache Metron come in. Metron presents a comprehensive framework for application and network, security built on Apache Hadoop and open source Streaming Analytics(ie Apache Nifi, Apache Kafka) tool’s highly scalable data management and processing stacks. Advanced features like profiling, machine learning, and visualization work with real-time streaming detection to make your SOC analysts more efficient, while the intrinsic extensibility of open source helps your data scientists get security insights out of the lab and into production fast.
We will discuss and demonstrate how some real-world businesses and managed service providers are using Apache Metron to identify and solve security threats at scale, and some approaches and ideas for how the platform can fit into your security architecture.
Speaker: Laurence Da Luz, Senior Solutions Architect, Hortonworks
A Science Information Committee (SIC) presentation authored by Smith, V.S., Blagoderov, V., Kitching, I. and Simonsen T., given at the Natural History Museum, London, UK. May 14th, 2010.
Workshop nwav 47 - LVS - Tool for Quantitative Data AnalysisOlga Scrivner
In the format of hands-on session, this workshop will introduce participants to the Language Variation Suite (LVS), a user-friendly interactive web application built in R. LVS provides access to advanced statistical methods and visualization techniques, such as mixed-effects modeling, conditional and random tree analyses, cluster analysis. These advanced methods enable researchers to handle imbalanced data, measure individual and group variation, estimate significance, and rank variables according to their significance.
Automated machine learning lectures given at the Advanced Course on Data Science & Machine Learning. AutoML, hyperparameter optimization, Bayesian optimization, Neural Architecture Search, Meta-learning, MAML
This talk was presented in Startup Master Class 2017 - http://aaiitkblr.org/smc/ 2017 @ Christ College Bangalore. Hosted by IIT Kanpur Alumni Association and co-presented by IIT KGP Alumni Association, IITACB, PanIIT, IIMA and IIMB alumni.
My co-presenter was Biswa Gourav Singh. And contributor was Navin Manaswi.
http://dataconomy.com/2017/04/history-neural-networks/ - timeline for neural networks
Provenance for Data Munging EnvironmentsPaul Groth
Data munging is a crucial task across domains ranging from drug discovery and policy studies to data science. Indeed, it has been reported that data munging accounts for 60% of the time spent in data analysis. Because data munging involves a wide variety of tasks using data from multiple sources, it often becomes difficult to understand how a cleaned dataset was actually produced (i.e. its provenance). In this talk, I discuss our recent work on tracking data provenance within desktop systems, which addresses problems of efficient and fine grained capture. I also describe our work on scalable provence tracking within a triple store/graph database that supports messy web data. Finally, I briefly touch on whether we will move from adhoc data munging approaches to more declarative knowledge representation languages such as Probabilistic Soft Logic.
Presented at Information Sciences Institute - August 13, 2015
A presentation for the Ottawa AI/ML Meetup Group, Sept 9, 2019.
A person’s Facebook newsfeed from their vacation is to the real-world as Kaggle competitions are to industrial ML/AI projects. This presentation shares with the audience some common challenges in ML/AI that real-world practitioners face while doing their craft, along with suggestions on how to address them. This is a non-technical presentation but does assume some knowledge of data science process and ML/AI.
Machine learning applications in clinical brain computer interfacingJenny Midwinter
Brain computer interfaces (BCIs) provide alternative communication channels from the brain to external devices for severely disabled individuals and can be used to induce and guide adaptive plasticity for recovery after central nervous system trauma. Clinical BCI effectiveness depends on robust and accurate modeling of the relationship between brain signals and behaviour. Dr. Boulay will give a brief survey of BCI technologies and discuss common BCI paradigms and implementations, with an emphasis on clinical BCI brain signals and machine-learning applications.
* From the Ottawa AI/ML Meetup June 2018.
Augmented Intelligence Bridging the Gap Between BI and AIJenny Midwinter
In this session Matt will give an overview of the Qlik Cognitive Engine and Qlik’s vision for Augmented Intelligence. Using Artificial Intelligence and Machine Learning the Qlik Cognitive Engine help analytics users find insight in their data faster in a self-service framework. Matt will give highlights of the Augmented Intelligence approach and implementation and share the technical journey from research to delivery of the functionality.
** From the Ottawa AI/ML Meetup November 26, 2018.
Introduction to Natural Language Processing. Review of techniques , both classic (legacy) and recent deep learning. Presentation from the Ottawa Machine Learning and Artificial Intelligence Meetup, April 2018.
Instructions on how to build a Natural Language Processing Deep Neural Network in 5 minutes. Presentation from the Ottawa Machine Learning and Artificial Intelligence Meetup, April 2018.
Machine Learning meets Granular Computing: the emergence of granular models in the Big Data era
** Presentation Slides from Dr Rafael Falcon, from Larus Technologies, for the February 2018 Ottawa Machine Learning & Artificial Intelligence Meetup
Abstract
Traditional Machine Learning (ML) models are unable to effectively cope with the challenges posed by the many V’s (volume, velocity, variety, etc.) characterizing the Big Data phenomenon. This has triggered the need to revisit the underlying principles and assumptions ML stands upon. Dimensionality reduction, feature/instance selection, increased computational power and parallel/distributed algorithm implementations are well-known approaches to deal with these large volumes of data.
In this talk we will introduce Granular Computing (GrC), a vibrant research discipline devoted to the design of high-level information granules and their inference frameworks. By adopting more symbolic constructs such as sets, intervals or similarity classes to describe numerical data, GrC has paved the way for a more human-centric manner of interacting with and reasoning about the real world. We will go over several granular models that address common ML tasks such as classification/clustering and will outline a methodology to appropriately design information granules for the problem at hand. Though not a mainstream concept yet, GrC is a promising direction for ML systems to harness Big Data.
Presentation slides from the Sept, 2016 Ottawa Machine Learning Meetup "Catching Bad Guys with Math: Real World Data Science Use Cases for Cyberattack Detection and Prevention", by Stephan Jou.
Ottawa Machine Learning Meetup April 24, 2017 Slides.
Abstract:
Amazon is one of the world's fastest growing companies and is showing no signs of slowing down. This staggering growth often presents serious challenges to the company's ability to scale. In the past, this has tested (and occasionally crushed) conventional software systems, but now this is testing the human decision makers in areas from Product Recommendation to Produce Inspection, from Text Classification to Voice Assistants and more. Matthew Spencer, a Machine Learning Engineer at Amazon will discuss these challenges and how they are being met by a wide variety of Machine Learning applications.
AI and Machine Learning: The many different approachesJenny Midwinter
Ottawa Machine Learning Meetup slides for event March 20, 2017, A whirlwind tour of AI and Machine Learning: When to use which technique, by Robin Grosset, Chief Technology Officer, MindBridge.Ai.
Applying Deep Learning Vision Technology to low-cost/power Embedded SystemsJenny Midwinter
Slides from Ottawa Machine Learning Meetup from January 16, 2016.
Pierre Paulin, Director of R&D at Synopsys (Embedded Vision Subsystems) , will be will be making a presentation on:
“Applying Deep Learning Vision Technology to Low-Cost, Low-Power Embedded Systems: An Industrial Perspective”
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
Autonomous Learning for Autonomous Systems, by Prof. Plamen Angelov
1. Autonomous Learning
for Autonomous Systems
Prof Plamen Angelov, PhD, DSc, FIEEE, FIET
Vice President International Neural Network Society
School of Computing and Communications
Lancaster University, UK; e-mail: p.angelov@lancaster.ac.uk
3. Ottawa, 24/09/18
Outline
Part I
1. The new Data-rich Reality, Motivation,
Challenges, Historical remarks
2. Case studies/applications
Part II
3. Under the bonnet: how we can achieve
this
4. Ottawa, 24/09/18 4
Context
In this talk the rationale and basics of EIS will be
introduced while drawing close links with well
established methodologies and algorithms, e.g.:
✓Machine learning
✓Clustering
✓Classification
✓Adaptive systems
✓Mining data streams
✓Feature selection,
✓etc.
5. Ottawa, 24/09/18
Context
➢ Huge volumes of streaming data streams
➢True Intelligence is the Evolving Intelligence
➢Social networks, Internet, IoT, banking
➢Advanced industrial processes, transport
➢Intelligence, surveillance, defence
6. Ottawa, 24/09/18
Big Data (streams) – the new reality
• Although somewhat debatable as a term –
• we live in a new reality:
➢ GB of data in our pockets
➢ TB per day
➢ zettabytes (1021) of data and growing, exo-…
➢ streaming, heterogeneous, irregular sampling,…
➢The Moor’s law seems now to be more applicable to
the digital data than to hardware
7. Ottawa, 24/09/18
Offline is not an option
➢ Data streams cannot be analyzed in a batch mode
(storing & manipulating complete data is impossible)
➢ Instead, systems have to be developed that extract
knowledge from the data streams in real-time, on-line,
‘on the fly’
➢ For non-stationary data it is logical to assume a
dynamic/evolving structure
➢ Evolving – gradually developing ► a higher level of
adaptation
8. ▪ Non-stationary data streams - characterised
by shift and drift phenomena
▪ We would like to access our data and
information anywhere anytime
Ottawa, 24/09/18
Specifics of Data Streams
9. Ottawa, 24/09/18 9
✓ Extracting model structure – layered
structure, deep learning
✓ Streaming data – recursive, non-iterative ,
parallelizable, collaborative algorithms,
(low memory and computational) costs
✓ On-line down-selection of best inputs
✓ Even routine pre-processing (normalisation
and standardisation) when applied to data
streaming on-line are not trivial
Technical Challenges
10. Ottawa, 24/09/18 10
✓ Evolving vs Evolutionary
✓ First Evolving Fuzzy/Connectionist/Self-
organising NN/NF Systems (1998-2001; P.
Angelov, N. Kasabov, CTLin)
✓ First really dynamically evolving (2001-2004):
✓ eR (2001), eTS (2002), DENFIS (2002),
simpl_eTS (2005),…
✓ EIS – first time Angelov & Kasabov (2005, 2006)
✓ EIS (2010, Springer), ALS (2012)
Historical Remarks
11. Ottawa, 24/09/18
Applications
1) Fast, transparent Deep Fuzzy Rule-based Learning for
Image Classification
2) On-board Real-time Video Processing (detecting moving
targets on the ground – AURORA project)
3) Self-calibrating sensors in oil refining (chemical and petro-
chemical industry)
4) High Frequency Trading (HFT)
13. Ottawa, 24/09/18
Fast Interpretable Deep Learning
➢After the training process, each ALMMo system will generate 10
(1 per class/digit) AnYa type fuzzy rules in the following form:
14. Ottawa, 24/09/18
Fast Interpretable Deep Learning
➢Each ALMMo has 10 fuzzy rules, and each rule will give its output as a score of
confidence based on the “winner takes all” principle.
16. Ottawa, 24/09/18
Fast Interpretable Deep LearningApproaches
Accuracy
TrainingTime
PCParameters
GPUUsed
ElasticDistortion
Reproducibility
Parallelization
The Proposed
Approach
99.55%
Less than 2 minute
for each part of the
network
Core i7-4790
(3.60GHz),
16 GB DDR3
None NO YES YES
Large Convolutional
Neural Networks [6]
99.47% NO YES NO
Committee of 35
Convolutional Neural
Networks [4]
99.77%
Almost 14
hours for each one
of the 35 DNNs.
Core i7-920
(2.66GHz),
12 GB DDR3
2 GTX 480 &
2 GTX 580
YES NO NO
17. Fast Interpretable Deep
Learning (Classification Stage)
➢ Every ALMMo system passes its scores of confidence
corresponding to the 10 digits to the decision-making
committee and the committee integrates the outputs of all (154
per digit) into 10 overall scores of confidence:
➢ In general cases, the overall decision is made based on the
“winner takes all” principle:
( ) ( ) ( )( )
( ) ( )( )
, ,
1,2,...,1
, ,
1,2,...,1
1 1
max
2
1 1
max
2
SR
M
j i j i j
i SRi
SR
i j i j
i SRi
C Image
SR
SR
==
==
= +
+ +
g g
h h
( )( )0,1,...,9
arg max M
j
j
Label C Image
=
=
20. A Massively Parallel Deep Rule-Based Ensemble
Classifier for Remote Sensing Scenes
21. A Massively Parallel Deep Rule-Based Ensemble
Classifier for Remote Sensing Scenes
22. Semi-supervised Deep Rule-based Classifier using
Deep Representation
• In this work, the pre-
trained vgg-verydeep-
16 convolutional
neural network is
used as the feature
descriptor, and the
1x4096 dimensional
activations from the
first fully connected
layer is used as the
feature vector of the
image.
24. Ottawa, 24/09/18 24
DFRBL summary
• Fast deep learning;
• Rule-base (as oppose to NN black box) –
therefore, transparent;
• 0-order prototypes based – therefore,
interpretable;
• Fully parallelisable
• Non-iterative
• Non-parametric
• Evolving
• Human level precision
• Fully repeatable
25. Ottawa, 24/09/18
HFT problem description
• Due to fast changes on the global markets, individual agents
have to deal with a huge amount of HF financial data to:
– determine buy and sell recommendations;
– submit trades to exchanges for execution;
– manage trades after execution;
– analyse market data.
• Many HFT algorithms are based on technical analysis that
uses historical prices and indicators to identify zones of
supply/demand where sellers/buyers are likely to change
the price of the product.
26. Ottawa, 24/09/18
HFT experiment
• Data stream: QuantQuote Second Resolution Market dB
• Attributes: 1) Time; 2) Open price; 3) High price; 4) Low price; 5) Close
price; predicting the future value of 3).
• Five steps ahead prediction of the High price.
• Left plot: prediction; right plot: evolution of the model structure
29. Results
• The time interval of
between 2 data ticks
ranges from as little as a
second to few minutes,
however the proposed
method only needs less
than 0.001 second in
average to process each
tick
33. eSensor – self-callibrating
Final Rule base in the Abel inflammability test
R1: IF (P is 5.4%) AND (Tco is 323.3 oC) AND …AND (Tne is 126.8 oC)
THEN (A1=20.2 + 92.7P + … + 0.12 Tne)
R 2: IF (P is 11.7%) AND (Tco is 365.0 oC)AND …AND (Tne is 147.6 oC)
THEN (A2=42.1 + 63.4P + … + 0.10 Tne)
R 3: IF (P is 5.4%) AND (Tco is 335.14 oC) AND…AND (Tne is 136.1 oC)
THEN (A3=25.2 + 71.9P + … + 0.19 Tne)
Low end
point of
Naphtha
High end point
of Naphtha
34. Ottawa, 24/09/18 34
✓ Because traditional offline approaches,
using supervised train vs test vs validate
data split is not applicable
✓ Unsupervised concept learning form data
distributions and realisations
✓ Data → Concepts→ Relations and/or Rules
✓ Data Clouds (AnYa,2010) – suitable for
Concept representation
Concepts Leanring
36. Ottawa, 24/09/18
➢ One can use various mathematical
frameworks to model big data, e.g. (but
not limited to!):
✓ Probabilistic (HMM, Bayesian, PF, MC,…)
✓ Computational Intelligence (ANN, FRB,…)
✓ Decision Trees (DT)
✓ Grammar (Natural language, NL)
✓ EDA (Empirical data analytics, Angelov, 2014)
Model Frameworks
37. Ottawa, 24/09/18
➢ Traditionally, system identification– mostly
parameter identification (system structure is
selected, pre-defined!)
a) Divide et impera
b) Adapt/Evolve or die
c) Layered structure
Principles
38. Divide et Impera (Lat.)
Behind all framework types is the old Roman
principle ‘Divide et impera’ - The data space
is decomposed into Granules (states, pdf,
neurons, rules, words, etc.).
0
0.5
1
0.5
0 -0.5
-1 -1
-0.5 0 0.5 1
Ottawa, 24/09/18
39. Ottawa, 24/09/18
Divide et impera
)x(fy =Decompose complex processes
into multiple simple models, x
which Adapt & Evolve
n
Rx
m
Ry T
y =
TT
e
NT
e xx ],...,[ 1
=
y
x1* x2*
y1
y2
x y
…
yR
Local model 1
Local model 2
Local model R
40. Ottawa, 24/09/18
Local sub-Models
This leads to the concept of local sub-
models/sub-systems, e.g.:
✓States (HMM):
✓Activation functions
✓pdf
✓Rules
✓m, etc.
S1 Sn
Snew
p11 p1n pnn
43. Data Clouds vs Clusters
Having data clouds (areas for local models) we can now define the ALS
as a set of rules of the following form:
Ottawa, 24/09/18
Table A Comparison between Cluster and Clouds
( ) ( )iii
LMTHENCxIFRule ~:
Features Cluster Data Clouds
Boundaries Defined as hyper-ellipsoids Voroni tessellation
Centre/Prototype Defined Extracted post factum
Distance between a data point and Centre/Mean Focal point
Membership function
Parameterised Non-parametric
Approximation of an ideal
distribution, assumed a priori
Reflect the real data
distribution
45. Empirical data Analytics
• Probability theory, statistics– number of restrictive
assumptions which usually do not hold in reality
➢ pre-defined smooth, “convenient to use” types of
distribution;
➢ infinite amount of observations/data points;
➢ independence between data points (so called iid –
independent and identically distributed data)
➢ pdf has a number of paradoxes
Ottawa, 24/09/18
paradoxes
46. Empirical data Analytics
➢ EDA-Entirely based on the empirical
observations of discrete data points and their
mutual position forming a unique pattern in the
data space
➢ An effective combination of the frequency and
the space distance.
Ottawa, 24/09/18
47. EDA: Cumulative Proximity
Cumulative proximity is a measure indicating the degree of
closeness/similarity of a particular data point to all other existing
data points:
Ottawa, 24/09/18
( ) ( ) ( ) 01,
1
2
= =
ik
k
j
jiik xkxxdx
48. EDA – basic measures, ε
2. Standardized Eccentricity - represents the association
of the data point with the tail of the distribution and
the property of being an outlier/ anomaly:
ε is very convenient to represent well known Chebyshev
inequality. It turns into a simple check if >10 or not
for n=3 because:
Ottawa, 24/09/18
( ) ( ) ( )
( )
( ) 01
1
2
1
==
=
ikk
j
jk
ik
ikik xk
x
k
x
xkx
( )N x
( )( )2
2
1
1 1NP n
n
+ −x
49. EDA – basic measures, D
3. Data density is inversely proportional to the
standardized eccentricity.
It can be proven that for Euclidean and Mahalnobis
distances D gets the form of Cauchy function:
Ottawa, 24/09/18
( ) 2
T
1
1
N i
i N
N N N
D
X
=
−
+
−
x
x
1 1 1
1 1
;N N N
N
N N
−
−
= + =μ μ x μ x
T T
1 1 1 1
1 1
;N N N N
N
X X X
N N
−
−
= + =x x x x
50. Ottawa, 24/09/18
KDE and RDE
• The density in the data space is a key charac-
teristic of anomalies and model structure (focal
points for local sub-models).
• Traditionally–KDE, Parzen (‘62):
• RDE–Cauchy type, Angelov(‘02)
( )=
−=
k
i
ikk xxK
k
xD
1
1
)(
...
1
1
11
)(
1
21
1
1
2
1
2
+ −+
=
=
=
−
−−
=
=
k
i
ki
xx
k
xx
k
k
xx
ke
exD k
i
ki
k
i
ki
51. Ottawa, 24/09/18
RDE (patented)
• It can be recursively updated (Angelov, 2008):
• One can use RDE for:
a) Model structure update (Angelov, 2002);
b) Anomaly/outliers detection in RT (Angelov, 2007)
c) Typicality & Eccentricity (Angelov, 2014)
d) RTSDE (Angelov, 2014)
22
1
1
)(
kkkk
k
Xx
xD
−+−+
=
111
11
xx
kk
k
kkk =+
−
= − 2
1
2
1
1
11
xXx
k
X
k
k
X kkk =
−
+
−
= −
54. EDA – basic measures, τ
4. Multi-modal (discrete global) typicality:
Ottawa, 24/09/18
( )
( )
( )
( )
( )
1
1
1 1
N N
j N ii N iD
N i L L
j N j j N j
j j
f qf D
f D f q
−
−
= =
= =
uu
u
u u
55. EDA – basic measures, τ
• Continuous global density and typicality:
• Integrates to 1, can be
used as pdf, but is derived
entirely from data with no
prior assumptions
Ottawa, 24/09/18
( )
( )
( )
( )
( )
, ,
1
, ,
1
N
N
C
L
G N i N i
NG i
N C
G L
N N i N i
i
S D
D
D d S D d
=
=− −
= =
x
x
x
x x x x
( ) ,
1 1
1 22 2
,
, 2
,
1
2
1
NC
N iG
N d d
i
N id
N i
N i
d
S
N
+ +
=
+
=
−
+
x
x
59. 59
The EIS approach
Predict/
classify/
control
Life-cycle
EIS
Data acquisition
pre- process
hard sensors
EIS (evolve
or die)
RDE,
detect
outliers
Model structure
(data clouds)
Learn
parameters
Learning from experience (extract knowledge from
data streams) throughout the whole life-cycle:
Reflects human ability to
acquire, summarize, and
manage knowledge by
learning “on the fly”
Ottawa, 24/09/18
60. Ottawa, 24/09/18
Input … Output(s)
Data
Model structure update
based on R(TS)DE or m
Model evolution
The approach can be summarised as:
✓Decomposition of the Data into Clouds (2010)
✓Joint RT identification of local sub-models
✓Overall output – a soft blend of local outputs
61. Ottawa, 24/09/18
Evolving based on Density
▪ The core of ALS - recursively estimate data
density from data streams and to react on its
variations by modifying the model structure:
– A1) a data sample with high density is eligible to be a
focal point of a Cloud/local sub-model
– A2) a data sample that lies in an area of data space
not covered by other local sub-models is also eligible
to form a new local sub-model
– B3) avoid overlap and information redundancy in
forming new local sub-models
– C4) remove old clouds and low support & utility ones
– D5) select on-line the input variables that contribute
most to the output
62. Local models weighting
based on relative density
x
y 1
newD
G2
G1
Ottawa, 24/09/18
2
newD
2
newz
1
newz
The (fuzzy) weight, λi to a cloud is described by
the relative local density:
=
= N
j
j
k
i
ki
k
D
D
1
1
1
==
N
i
i
10 i
63. Learning local model parameters
with proven convergence/stability
➢ The error ϵk caused by approximating f(x) is
bounded with the constant ε given by
➢ fwRLS – Globally optimal stable/converging solution:
➢ 𝛼 𝑘 is the time-varying (but bounded) learning rate
given as:
Ottawa, 24/09/18
𝑦 𝑘=𝐻 𝑘
𝑇
𝑄∗ + ϵk
|𝜖 𝑘|≤ 𝜀
𝑄 𝑘+1=𝑄 𝑘 + 𝛼 𝑘Σ 𝑘 𝐻 𝑘 𝑒 𝑘
Σ 𝑘+1 = Σ 𝑘 − 𝛼 𝑘Σ 𝑘 𝐻 𝑘 𝐻 𝑘
𝑇
Σ 𝑘
𝛼 𝑘 =
1
1 + 𝐻 𝑘
𝑇
Σ 𝑘 𝐻 𝑘
65. EIS as FRB systems
..........................................................................................
( )
+=
+= ==
n
j
j
m
jR
m
R
m
R
n
j
jjRRR
RR
xaayANDANDxaayTHEN
GranuleisxIFRule
1
0
1
11
0
1
...
:
On-line input variables selection
remove
add
rulesrules
( )
+=
+= ==
n
j
j
m
j
mm
n
j
jj xaayANDANDxaayTHEN
GranuleisxIFRule
1
1011
1
1
1
1
01
1
1
11
...
:
Ottawa, 24/09/18
66. Prediction,
Classification or Control
Anomaly/outliers detection (low D)
Clustering (focal points with high D)
Classification:
Prediction:
Control
Ottawa, 24/09/18
The problem formulation is generic. It applies to:
( ) ( )iii
LabelxTHENCxIFRule →~: ( )j
N
j
Label
1
maxarg
=
=
=
=+
N
i
ii
LMky
1
)1(
67. Ottawa, 24/09/18
Analyze rule quality
– Support – No of samples
– Age:
*
1
minarg1 i
k
N
i
ll
zzlforSS −=+
=
;
*
1
minarg i
k
N
i
l
k
l
k zzlforkII −=+
=
],1[;
)(
)(
)(
1
Ri
kN
I
kkAge
i
kN
l
l
i
l
=−=
=
68. Ottawa, 24/09/18
Detecting shift by rule age
▪ Shift in the data stream can be detected by the
age of the rule which corresponds to the
inflexed point of the Age curve (when the
derivative of Age changes its sign:
( )
dk
Aged
69. Ottawa, 24/09/18
Rule utility
▪ The utility of a fuzzy rule represents the
accumulated strength of a fuzzy rule
▪ It is defined as the accumulated firing level of
the respective fuzzy rule for the span of its life
],1[;)( 1
Ri
tk
k
i
k
l
l
i =
−
=
=
70. Ottawa, 24/09/18
On-line inputs selection
▪ Because sub-models are locally linear, the
sensitivity analysis reduces to analysis of the
consequent parameters:
▪ The importance of each input (feature) can be
evaluated by the ratio of the accumulated sum
of the consequent parameters for the specific
jth input (feature) in respect to all n inputs:
],1[];,1[;
)(
)(
)(
1
njRi
kT
kT
k n
r
ir
ij
ij ===
=
=
=
k
l
ijij lk
1
)()(
],1[];,1[;)()(
1
1*
*
njRikTkj
n
r
irij == =
72. Ottawa, 24/09/18
Researchers
• Gruffydd Morris (2005-now), finishing PhD
• Dr. Xiaowei Zhou (2005-2008), now CEO, China
• Dr. Ramin Ramezani (2007-9), now UCLA
• Dr. Pouria Sadeghi-Tehran (2009-2015), now RI
• Dr. Javier Andreu (2008-2012), now Imperial College
• Tu Vong (2011-2012), now ARM, Cambridge
• Chris Clarke (2013-2014), now doing PhD here
• Antoniou Antreas (2014-2015), now Amazon
• Many other MSc students
73. Ottawa, 24/09/18 73
Publications
• P. Angelov, Autonomous Learning Systems: From Data to Knowledge in Real
Time, Wiley, 2012
• P. Angelov, X. Gu, J. Principe, A generalized methodology for data analysis,
IEEE Transactions on Cybernetics, DOI: 10.1109/TCYB.2017.2753880, 2017.
• P Angelov et al,Empirical data analytics,Int.J.Intel.Syst,2017,DOI:10.1002/int.21899
• P. Angelov, X. Gu, J. Principe, Autonomous learning multi-model systems from
data streams, IEEE Trans. on Fuzzy Systems, DOI:10.1109/TFUZZ.2017.2769039, 2017.
• P. Angelov, X. Gu, Empirical Fuzzy Sets, Int. Journal of Intel. Syst., DOI:
10.1002/int.21935, 2017.
• P. Angelov, X. Gu, MICE: Multi-layer multi-model images classifier ensemble,
IEEE Intern. Conference on Cybernetics (CYBCONF), Exeter, UK, 2017, pp. 1-8.
• P. Angelov, X. Gu, A Cascade of Deep Learning Fuzzy Rule-based Image
Classifier and SVM, IEEE Intern. Conf. on SMC (SMC2017), Banff, Canada, 2017
• P. Angelov, et al., Fast feedforward non-parametric deep learning network
with automatic feature extraction, IJCNN-2017, Anchorage, USA, pp. 534-541.
74. Ottawa, 24/09/18
Highlights
– True Intelligence is Dynamically Evolving
– Evolving Model Structure
– HFT and Evolving Fast Transparent Deep
Learning classifier applications of EIS