3. What is a database ?
.ball
•An organised collection of information
•Allows reading and writing .
•Provides authorisation and authentication.
•Provides some level of data safety.
4. Traditional RDBMS
.ball
•Developed by E F Codd in early 1970s
•This model is based on tables rows and columns
and the manipulation of data stored within.
•A Relational DB is the collection of all these table
•Example: Oracle, mysql & microsoft access
5. What is a database ?
.ball
•An organised collection of information
•Allows reading and writing .
•Provides authorisation and authentication.
•Provides some level of data safety.
6. Data store for typical RDBMS
.ball
•Data resides on disk.
•Data maybe cached into memory for access.
7. PROBLEM
.ball
• Existing disk-based systems can no longer offer
timely response due to the high access latency to
hard disks
•The unacceptable performance an obstacle for a
meaningful real-time service.
•Eg :Real-time bidding, advertising, social gaming,
Stock market .
8. “Memory is the new disk, disk is the new tape”
Jim Gray
Data scientist
Creator IBM system R
.ball
11. IN-MEMORY DATABASE SYSTEMS
.ball
•For in-memory DB ,Data resides permanently on main memory.
•Source data is loaded into system memory in a compressed,
non-relational format
•Only backup copy on disk.
•Memory optimised data structures are used
12. Disk VS Memory
.ball
•Order of magnitude of access time is less for main memory.
•Main memory is normally volatile while disk storage is not.
•The layout of disk is much more critical than layout of main
memory
15. .ball
•SAP HANA is the market leader in IMDB systems. It is also a platform
for big data processing analysis and prediction.
•SAP HANA can help business for building real-time applications and
analytics for accelerating the process
22. .ball
•In-Memory Big Data Management and Processing:
By Hao Zhang, Gang Chen, Member, IEEE, Beng Chin Ooi, Fellow, IEEE,
Kian-Lee Tan, Member, IEEE, and Meihui Zhang, Member, IEEE
•SAP HANA Distributed In-Memory Database
System: Transaction, Session, and Metadata Management
Juchang Lee#1, Yong Sik Kwon#2, Franz Färber*3, Michael Muehle*4, Chulwon
SAP Labs, Korea
•In-memory database
www.wikipedia.org
REFERENCES
From one core to multi-core, to multiple processors per servers, to multi-threaded cores, where we now have servers with up to 8 CPUs (with 24Mb caches each) and 160 threads!
Relentless technology progress by Intel, AMD, ARM and others, will lead to even bigger caches and cores. The name of the game is data-locality and parallelization. Just released “Sandy Bridge” generation for servers.
By accessing data in column-store order, you benefit immensely from simplified table-scan and data pre-caching. This can make all the difference in performance.
Big building 1910
Basketball hoop – 10 feet
Ratio of 106M to 4.9k
Memory access is 1M – 10M times faster than disk. In the past memory was so expensive that database vendors optimized for disk. However, with memory costs dropping so dramatically over last 20 years, it’s not possible to harness the power of in-memory computing.