MELJUN CORTES research memory_database_position_paper_research_

1. Memory Database, it’s too Expensive
RESEARCH
POSITION PAPERS
MELJUN CORTES

2. In today’s world, organizations like Google, Yahoo, Amazon, Facebook etc. are facing drastic increase in data. This
leads to the problem of capturing, storing, managing and analyzing terabytes or petabytes of data, stored in multiple
formats, from different internal and external sources. Moreover, new applications scenarios like weather forecasting,
trading, artificial intelligence etc. need huge data processing in real time. These requirements exceed the processing
capacity of traditional on-disk database management systems to manage this data and to give speedy real time results.
Therefore, data management needs new solutions for coping with the challenges of data volumes and processing data
in real-time. An in-memory database system (IMDS) is a latest breed of database management system which is
becoming answer to above challenges with other supporting technologies. IMDS is capable to process massive data
distinctly faster.
With the increasing demand of real time data processing, traditional (on-disk) database management systems are in
tremendous pressure to improve the performance. With the increasing amount of data, which is expected to touch
40ZB (1ZB = 1 billion terabytes) by 2020, means 5247 GB of data per person, and with traditional DBMS architecture, it
is becoming more and more challenging to process the data and to produce analytical results in almost real time. For
on-disk databases, disk I/O operations are the main bottleneck, which are very slow operations and can’t be optimized
beyond a limit being mechanical in nature.
Here comes the in-memory database system concept, which actually changed the whole architecture paradigm for the
database management system. An in-memory database system or main-memory database system is a breed of
database management system that stores data entirely in main memory instead of keeping it on disk. With decreasing
cost of main memory, and advance technological innovations, it becomes quite feasible to store large amount of data in
main memory. Once data is stored in main memory, speed of reading and writing the data will be improved drastically
as it eliminates disk I/O operations.
Introduction

3. In Memory Database, it’s too Expensive.
Maximum Practical Size for an IMDS is Measured in Gigabytes,
While On-Disk Databases Can Grow to Terabytes
Volatility is IMDSs’ Achilles Heel – When the System Goes
Down, Data is Lost.
Counter Arguments

4. Nikita Ivanov Founder & CTO, GridGain Systems
The interesting trend is that price of RAM is dropping 30% every 12 months
or so and is solidly on the same trajectory as price of HDD which is for all
practical reasons is almost zero (enterprises care more today about heat,
energy, space than a raw price of the device).
The price of 1TB RAM cluster today is anywhere between $20K and $40K -
and that includes all the CPUs, over petabyte of disk based storage,
networking, etc. Cisco UCS, for example, offers very competitive white-label
blades in $30K range for 1TB RAM setup.
Claims

5. IMDS technology scales well beyond the terabyte size range. McObject’s
benchmark report, In-Memory Database Systems (IMDSs) Beyond the
Terabyte Size Boundary (to download, see
http://www.mcobject.com/130/EmbeddedDatabaseWhitePapers.htm)
detailed this scalability with a 64-bit in-memory database system deployed
on a 160-core SGI Altix 4700 server running SUSE Linux Enterprise Server
version 9 from Novell. The database grew to 1.17 terabytes and 15.54 billion
rows, with no apparent limits on it.
Claims

6. C. Diaconu et al. Hekaton: SQL Server’s Memory-Optimized OLTP Engine. In SIGMOD,
pages 1243–1254, 2013.
Data needn’t be lost in the event of system failure. Most in-memory database systems offer
features for adding persistence. One important tool is transaction logging, in which periodic
snapshots of the in-memory database (called “savepoints”) are written to non-volatile media.
Non-volatile RAM or NVRAM provides another means of in-memory database persistence.
One type of NVRAM, called battery-RAM, is backed up by a battery so that even if a device
is turned off or loses its power source, the memory content—including the database—
remains. Newer types of NVRAM, including ferroelectric RAM (FeRAM), magnetoresistive
RAM (MRAM) and phase change RAM (PRAM) are designed to maintain information when
power is turned off, and offer similar persistence options.
Claims

7. CONCLUSION
The dominance of database management on disk system is on the verge to
disappear. In the coming years IMDB’s effectiveness and the affordability as well as
abundance of the processors and memory will compel the database community to
make this shift over. IMDB is not only faster than disk-based database system but
also eliminates the need for the use of tier 1 storage in database deployment
significantly reducing operational costs associated with the database storage.
While making this shift over, the users should consider the data they intend to
move to IMDB as well as the frequent operations performed on the database. It is
definitely not hyperbolic to conclude that main memory will be seen as the prime
storage of a database and disks will serve only as recovery system.