These slides use concepts from my (Jeff Funk) course entitled analyzing hi-tech opportunities to show how AHaH (anti-Hebbian and Hebbian) computing is becoming economically feasible. Traditional computing with the von Neumann architecture requires constant interactions between the processor and the memory (usually DRAM) and improvements in memory access time are occurring at a much slower rate than that of microprocessor speeds. This performance gap is becoming a bottleneck for von Neumann based computers. AHaH computing (and Synaptic computing http://www.slideshare.net/Funk98/neurosynaptic-chips) address this bottleneck in that they use a different architecture that mimics the processing of the brain. AHaH computing has additional differences from von Neumann (and synaptic) architectures in that it reduces the number of interactions between the memory and the processor by combining some aspects of memory on the processing chip. This is done with so-called memristors, which are naturally adaptive systems, and which are experiencing rapid improvements in cost, storage density, and storage capacity. With memristors, widely used pathways become stronger and less widely used pathways become weaker thus facilitating machine learning. Although machine learning can also be done with software, memristors and AHaH computing enables machine learning at the hardware level. The optimism for AHaH computing partly comes from the rapid improvements in memristors, which are rapidly improving the economics of AHaH computing.

1. Anti-Hebbian and Hebbian
(AHaH) Computing
MT5009
Analyzing Hi-Technology
Opportunities
1. Chow Ka Yau Daniel A0145207M
2. Muhammad Dzahir Bin Mohamed Zain Affandi A0129428Y
3. Gregory Chee Ken Khyun A0132405W
4. Jayapathma Herath Madhushanka Meranjan A0132398Y
5. Lim Yee Hao Marcus A0132390N
http://www.riken.jp/en/research/rikenresearch/highlights/7918/

2. • Current worldwide buzz - Big Data analytics
• Future: Not just analytics but also
• Solutions: Predictive and Prescriptive analytics
• We need
- Machine learning
- Intelligent computing
- Large scale simulations
Problem: Current Von-Neumann computing bottleneck
We believe, AHaH Computing can break the Von-Neumann
bottleneck and open up a new era of big data analytics
Problem Statement

3. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

4. http://hartenstein.de/cited/Damian_Millers-Award.pdf
Von-Neumann (VN) Architecture
and Limitations http://sybaseblog.com/2013/05/0
6/need-of-in-memory-technology-
sap-hana/
Memory access speed
Processor speed

5. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

6. http://www.gridgain.com/wp-content/uploads/2014/09/insideBIGDATA-Guide-to-In-
Memory-Computing.pdf
- Keeping data in a server’s RAM instead
of hard disk or flash devices
- Massive parallelization for faster
processing speeds
- Inexpensive way to speed up enterprise
software applications, including but not
limited to analytics
Challenges:
- Requires lots of RAM!
- Unsustainable brute force method
since data volumes continue to
explode in a big data world
- High power consumption
- Processor and memory are still
separated
http://www.toddmace.io/
1. Algorithmic Approach: In-Memory Computing
Solutions to break VN bottleneck

7. Increase
bandwidth
Cache
Pre
fetching
Multi
threading
Parallel
processing
Pipelining
In-
Memory-
Computing
(IMC)
1. Algorithmic Approach: Software
http://tonycosentino.ventanar
esearch.com/
Solutions to break VN bottleneck

8. 2. Neuromorphic Approach
http://www.slideshare.net/Funk98/neurosynaptic-chips
IBM’s True North
supercomputer incorporates
the largest neuromorphic
chip in the world, but the
chip is not capable of
learning on its own
Solutions to break VN bottleneck
Limitations
 No active machine
learning on chip
 No unsupervised
learning
 Required
supercomputer
http://www.slideshare.net/Funk98/neurosynaptic-chips?qid=e85f972a-2571-49d0-ac6c-4b4395525901&v=default&b=&from_search=1

9. Aim: to achieve the level
whereby it is able to do
what brain does –
processing and memory
are performed by the
same component
https://upload.wikimedia.org/wikipedia/commons/d/df/PPTExponentialGrowthof_Computin
g.jpg
The Future of Computing
http://www.slideshare.net/Funk98/neurosynaptic-chips?qid=e85f972a-2571-
49d0-ac6c-4b4395525901&v=default&b=&from_search=1

10. 3. Self - Organizational Approach
AHaH computing is more than just the integration of
memory and processing!
Anti-Hebbian and Hebbian
(AHaH) Computing
Solutions to break VN bottleneck

11. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

12. • Creation of conductive path through a common
medium
• It just happens naturally (self organization)
• No need for external control to produce path of
conduction
• Occurs in rivers, air, blood system etc.
Bifurcation video
AHaH Phenomenon
Examples of Natural Adaptive System
http://knowm.org/blog/
It is this
energy
dissipating
pathways
competing and
conducting for
resources
It is a Learning Process

13. AHaH Phenomenon
Biology – Brain Synapse
- Strong Spikes Strengthen the connection
- Open up new pathways
- Hebbian Learning
- Weak Spikes (misfiring) weaken the
connection
- Delete up pathways
- Anti - Hebbian Learning
Model
• Firing strengthen pathways by increasing
synaptic weight change. This Adaptation
is called Hebbian learning
• Misfiring weaken the pathway connection
by decreasing synaptic weight change.
This Adaptation is called Anti - Hebbian
learning
Which component can help to mimic this synaptic adaptation ?
?
?
http://www.smashinglists.com/top-10-amazing-facts-
about-the-human-brain/
http://www.intechopen.com/books/reinforcement_learning/interaction_between_the
_spatio-temporal_learning_rule__non_hebbian__and_hebbian_in_single_cells__a_c
MEMRISTORS

14. AHaH Phenomenon
Memristors - Mimic Synaptic Adaptation
Analogous to the adaptive water pipeline
1) High pressure difference -> more water flow -> diameter gets bigger
2) Cut water supply -> diameter stays same – Remembers how much water has flown
3) Less pressure difference -> less water flow -> diameter gets smaller
4) Water flows in the opposite direction if diameter gets two small
– Erasing the path and flow is bi-directional
Replace the water flow with Current flow
Replace the pressure with Resistance
Synaptic Adaptation
Memristors adapts its resistance as current flows through the memristor
Non - Volatile memory
Remembers how much current flowed through the memristor
Inputs
Output
Memristors
http://cacm.acm.org/news/33675-memristor-minds-the-future-of-artificial-intelligence/fulltext

15. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

16. http://www.hpcwire.com/2015/09/09/knowm-snaps-in-final-piece-of-memristor-puzzle/
AHaH Computing Architecture
Overall Architecture
Connected
Memristors
Connected KT-
Synapse in Parallel
– AHaH node
Map Many KT-Cores
into RAM
architecture
KT-Core
• Connected AHaH Nodes in
Parallel
• Column decoder and Row
decoder to select AHaH Nodes
• Controller to control Instruction
flow into AHaH Nodes
Basic RAM ArchitectureAHaH-NodeINPUTS
Output
http://i.cmpnet.com/pldesignline/20
05/07/zeidmanfigure1.gif

17. AHaH Unites Memory and processing
Example- A . B = C
CPU
MEMORY
Row
Decorder
Column Decoder
Input
Output
B
A C
Row
Select
Column
SelectRead Write
CPU
1) Inputs are
connected with
multiple cores
KT-RAM
Activation of
KT-Cores
2)Activation
Instruction
Act of accessing the
memory Becomes the act
of configuring the
Memory
- Weight Change
occurs inside memristors
KT-Core
During
Activation,
AHaH
Controller
connects with
Multiple AHaH
nodes
InputA
AHaH
Von-Neumann
3) Adaptation
Instruction
- AND Logic
4) Data Out
Output
C• Creates logics within the memory using sequential Instructions flow
• Adaption happens for free, because memristors adopt as we use them
• No back and forth data transfer between memory and CPU
ALU
C
B
C

18. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

19. Comparison of AHaH, Von-Neumann and
Neuromorphic Computing
Architecture:
Conventional
Computing
Neuromorphic
Computing
AHaH
Computing
Architecture Von Neumann Neural Network AHaH Architecture
Computing Unit CPU Synaptic Chip Synaptic Chip
Storing Unit Memory Synaptic Chip Synaptic Chip
Storing Element DRAM DRAM/SRAM Memristors
Suitability Logical and Analytical Machine Learning
(pattern recognition)
Logical & analytical and
Machine learning
(pattern recognition)
Processing Serial Processing
(multi cores)
Parallel Processing Parallel Processing
Backward
Compatibility
Only in Von-Neumann
Architecture
Unable to use in von-
Neumann architecture
directly
(Require Supercomputer)
Able to use in both Von-
Neumann & AHaH
Architecture
Power Consumption High Low Ultra Low
Speed Slow Fast Fast

20. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

21. • Barrier potential inherently exists between
CPU and RAM especially when the it is
physically separated.
• Sufficient energy must be applied to
overcome this barrier potential
• Power between the CPU and RAM heavily
depends on the distance between the CPU
and RAM
• Each read operation lowers the switch
barriers. Thus, the act of accessing the
memory becomes the act of configuring
the memory over time
In brains, d = 0 which
would mean high
amounts of power
saving
AHaH Advantages:
Lower power
https://www.youtube.com/watch?v=CFSrC7kjbJo

22. Less processing
requests and reply
between CPU and
memory due to a
internal machine
learning
AHaH Advantages:
Power and Time Reduction
https://www.youtube.com/watch?v=CFSrC7kjbJo
More back-and-forth operations! Single package reply!

23. http://www.colocationamerica.com/blog/energy-wasting-data-centers
http://perspectives.mvdirona.com/2009/05/the-datacenter-
as-a-computer/
AHaH Advantages:
Data Center Power Consumption Servers and Storage
• Increasing energy consumption
per year due to increasing data
storage demands
• Higher energy consumption
equates to higher electricity cost
expenditure
http://www.itbusinessedge.com/info/PP-BuildDataCenter-pg9.aspx

24. AHaH Advantages:
Machine Learning (Example)
http://image.slidesharecdn.com/bmvass2014breckonml-140709055858-phpapp01/95/machine-learning-fro-computer-vision-a-whirlwind-of-key-concepts-for-the-uninitiated-7-638.jpg?cb=1405575596
http://images.slideplayer.com/11/3288366/slides/slide_2.jpg

25. AHaH Advantages:
Machine Learning (Example)
Example:
Rich person has a high confidence level of having
high education
Rich person has a high confidence level of being old
Sequence 3: Unsupervised learning
Sequence 1: Supervised Learning
Sequence 2: Assign Label for KT- Cores (rich and poor)
Model
Input
Spikes
https://www.youtube.com/watch?v=CFSrC7kjbJo
https://www.youtube.com/watch?v=CFSrC7kjbJo
http://knowm.org/thermodynamic-ram-
technology-stack-published/

26. AHaH Advantages:
Flexibility of AHaH
• Integration AHaH circuits with existing
integrated circuits technology by
apply it at the end of the process line
at the very end
• Front-end-of-line refers to the current
integrated circuits.
• Back-end-of-line refers to AHaH
circuit would be fabricated on top of
it.
• Able to take advantage of an
existing process that is working very
well and bringing it to the next level
of performance
• Create new computers that adapts as
it is used
https://www.youtube.com/watch?v=w7q07eKPM9U
AHaH
circuits
Current
integrated
circuits

27. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

28. AHaH Enabler - Memristor
Memristor as
building
block
ReRAM AHaH NVRAM
Symbol
Unit: 

29. “Most Memristors that I
have seen do not
behave like fast,
binary, non-volatile,
deterministic switches.
This is a problem
because this is how HP
wants them to behave”
• Alex Nuegent – Lead
Inventor and CEO of Knowm
HP’s Memristor Problem
The incumbents are limiting the applications of memristors
within the existing memory technology framework
 Limit the use of memristor
http://knowm.org/the-problem-is-not-memristors-its-how-hp-is-trying-to-use-them/

30. https://indico.cern.ch/event/345619/session/1/contribution/10/attachments/681170/935777
/HW_trends_market_costs_BPS_Apr2015_v14.pdf
Memristors – Memory Trends:
Cost
NVRAM become cheaper
NVRAM vs SRAM
• Closing gap
• IBM’s synapse chips uses
SRAM
• Potential opportunity to
expand AHaH computing
market (memristor –
NVRAM)

31. Memristors - Memory Trends:
Storage Capacity
ReRAM
ReRAM vs other
emerging NVRAM
• Highest capacity
over others
http://www.maltiel-consulting.com/ISSCC-2013-Memory-trends-FLash-NAND-DRAM.html

32. http://www.tomsitpro.com/articles/flash-data-center-advantages,2-744-3.html
Memristors - Memory Trends:
Manufacturability
ReRAM vs other
emerging
NVRAM
• The only memory
compatible for mass
production

33. http://www.storagenewsletter.com/rubriques/market-reportsresearch/non-volatile-memories-yole/
NVRAM Trend Analysis
Current NVRAM Suppliers
ReRAM
Potential
Market
Non-Memristor based Memristor based
BIG DATA
Analytics

34. 2015 onwards, prediction of the mass manufacturing of Memristor to be available
This will bring forth the further improvement of AHaH computing architecture where
AHaH synaptic chip can be produced based on memristors
http://www.reram-forum.com/2013/03/21/predicting-the-reram-
roadmap/
Future opportunity for Memristor production

35. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

36. http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data
Definition of Big Data
Big Data 3V Model

37. Definition of Big Data
http://www.csc.com/insights/flxwd/78931-
big_data_universe_beginning_to_explode
The global production of data
is expanding and reach
~40ZB by 2020
https://www.capgemini.com/blog/capping-it-
off/2014/07/are-you-effectively-using-big-data
> 85% of an
organization’s data
is unstructured
Time and energy
consuming to
process unstructured
data
https://web-assets.domo.com/blog/wp-
content/uploads/2014/04/DataNeverSleeps_2.0_v2.jpg
Data velocity is
measured against
time
Enable real time
streaming
processing
Volume, Variety & Velocity

38. http://wikibon.com/wp-content/uploads/kalins-pdf/singles/big-data-vendor-revenue-and-
market-forecast-2011-2026.pdf
Wikibon forecasted Big Data
market to have 17% Compound
annual growth rate over 15 years
(2011-2026)
McKinsey Global Institute, Game changers: Five
opportunities for US growth and renewal, July 2013
Big Data identified as one of the
game changer that can boost US
annual GDP by 2020
Big Market for Big data

39. Business Intelligence
comprises of tools and
methodology for data
analyzing
Data / Big Data Analytics
can be grouped under
Business Intelligence
Past in Nature:
Descriptive and Diagnostic
Analytics
Future in Nature:
Predictive and
Prescriptive Analytics
http://www.fyisolutions.com/blog/advanced-analytics-seminar/
Big Data Analytics

40. The Bottleneck is in technology
 Not only need new algorithms and techniques but
breakthrough computing architecture
The Big Hurdle
http://image.slidesharecdn.com/finalpresentation-150305004602-conversion-gate01/95/presentation-on-big-data-analytics-15-638.jpg?cb=1425516438

41. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

42. http://ayata.com/stage/wp-content/uploads/ayata-infographic-2012-09-04.jpg
Big Data Analytics
Applications
Healthcare Industry
Given agility to government
to combat flu epidemic
dealing with vaccine
production/delivery rate vs
outbreak numbers in various
states
Autonomous Vehicle
Google AV to recognize &
anticipate what might be
coming in real time at a
junction
Oil & Gas Industry
Chevron need to analyze 50
terabytes of seismic data
Drilling miss cost USD$100M
Retail Industry
Starbucks marketing strategy
aligned to real time data
and responses

43. CONTENTS
1. The Von-Neumann Architecture and Limitations
2. Solutions to break Von-Neumann bottleneck
3. AHaH Phenomenon
4. AHaH Computing Architecture
5. Comparison of AHaH, VN and Neuromorphic Computing
6. AHaH Advantages
7. Memristor – Memory Trends
8. Big Data bottleneck, Model, Market and Challenges
9. Big Data Analytics Applications
10. Conclusion

44. CONCLUSION
• Big market and growth of Big Data applications
• Von-Neumann architecture bottleneck is hitting the limits
• Cutting edge of AHaH computing architecture
• Real time processing (Integrated memory & processing)
• Ultra less power consumption and less heat generate
• Self-Organized approach
http://ekvv.uni-bielefeld.de/bilddb/bild?id=87240

45. ANY QUESTIONS?
For further readings on AHaH computing, please
visit www.knowm.org
(Startup for AHaH – started July 2015)