SlideShare a Scribd company logo
1 of 9
Download to read offline
Big Data : A Rescue Plan
Big Data: A Rescue Plan
Ritu Vashistha1
Vikram Sharma2
Abstract
As the time has grown the mankind has also grown in terms of technology,
living standard, the way of keeping records. In the initial years the
requirement of remembering information and keeping information intact
for future use was very less. But as the human developed himself, the
requirement of keeping information in an effective manner and in terms of
capacity becomes a crucial issue to be resolved. To solve this problem
several techniques came up in the meanwhile. And now when data capacity
has no limit and humans doesn’t want the pattern of storage and
processing to be a hurdle; BIG DATA came up as a rescue plan.The motive
of writing this paper is to understand the storage pattern of data
developed over the years.
Keywords: Data, Technology, Information, Storage, Communication.
Introduction
he data are the raw facts and figures which individually do not
have any meaning, but if it is processed properly then it
becomes information, which is useful for mankind to
understand things. People use to store data from ancient times, but the
way or the pattern of storing data is different in every period.
1
Assistant Professor, Amity Business School, Amity University
2
Team Leader, A3logics(IND) Pvt. Ltd.
T
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 68
In the starting phase, the human race use to write on walls, rocks or on
animal skins and papyrus, but it didn’t last for long. After that the
invention of paper gives the human race a new way to store data but it
also has its own problems, the paper gets deteriorated quickly and when
the data grows the problem of managing data and retrieving it efficiently
becomes the next issue. Then comes the digital era where computer were
used to store the data, the storage capacity of computer was very high in
comparison to the older forms, it can store the data until it was not
deleted intentionally or accidentally, but it was still unstructured. Then
with the help of database management system the humans got a way to
store the data in more structured form and from where the efficient
retrieval is also not an issue.
As data is growing day by day, the industries were demanding new
Database technologies and infrastructures.
Technology Enhancements during the Years
The data are the raw facts and figures which don’t have meaning and the
information is the processed data which is meaningful. As D. Kimbrough
Oller and Ulrike Griebel has mentioned in their book ‘Evolution of
Communication Systems’ that when the mankind came into existence
they don’t have any communication medium which is verbal they have
gradually developed it. First it was symbolic they came few alphabets,
words and then sentences. But it took several years to become a formal
form of communication.
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 69
Then came the next issue and that was how to remember everything,
people forget things, people die and the medium they have at that time
was also not of permanent in nature and it can’t be carried easily if
desired. Then paper was invented around 100 BC in china. In 105 AD,
under the Han Dynasty emperor Ho-Ti, a government official in China
named Ts'ai Lun was the first to start a paper-making industry. It was
the revolution in the industry, people learnt how to write, how to keep
data in different forms. But this technology also had its disadvantage that
it is not permanent in nature, paper gets deteriorated by the time. So
when the computers came into existence in 19th century, then also after
several years database management system came into existence. A
database is a storage where the data can be stored in an organized way,
so it can be processed easily and converted into useful information. Now
the databases are able to store terabytes of information into it. DBMS is a
structured approach, in a single database only similar kind of
information is able to get stored. Now with the growing requirements, it
was felt that sometimes different kind of information which comes from
different sources are required to store at one place, and it discovers a
new concept that is Big Data.
Big Data
In 2001 Doug Laney specified challenge of data management using three
V’s. Volume, Variety and Velocity. A phase of new concept was started to
begun to overcome with the old challenges. Big Data is a new revolution
in data warehousing and data analytics. This concept is introduced to
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 70
store and handle large number of data without define data type with
high speed of data processing. This concept can handle structured and
unstructured data with large volume, which was very difficult to handle
with the older technology.
Few Important Technologies for Handling Large Data Infrastructure
(1) RDBMS: Relational Database Management system designed with
the architecture in that data is organized in a defined structured
pattern. This type of databases limits themselves with the size of
database and with the defined set of data type values stored.
These databases need a predefined pattern of values to be stored.
(2) No SQL Database Systems: This type of database system contain
horizontally scalable; as data continues to explode, high
performance, agile processing of information at massive scale
properties. In other words, a true smell of Big Data.
Therefore, to understand the present requirement of storage pattern and
capacity researchers have talked to few people who are working in
software companies. We have chosen this industry because these people
deal with the basic problems of storage and they solve them for the user
industries of their software’s.
After several years of development and analysis of products, suddenly
they found a new kind of problem. Their company had developed five
products in last ten years and providing services and support for all
these five products. All five products have large number of data attached
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 71
with them in different format. All five products developed on different
technologies with different type of data base structure.
Gradually company started realising that there should be some method
or technique through which they can connect all five product databases
with each other. To accomplish this task they need a new system which
will gather data from all the five products and can generate its query
results. With the help of that system; different type of data flow or search
or requirement of client(s) can be easily analysed.
With the current structure it will be a difficult task. As they need to take
data from all the five systems database and then they have to write in a
system that process that data and provide them a logical report.
The biggest challenge was the collection of information from different
structured database. After a long thought process and problem solving
exercises they thought of two approaches
(1) Create an independent tool that contains all the information
about the 5 database structures and from there it will take data in
raw format, process that data and provide the results. In this
process a small change in any of the 5 application may create a big
change in new tool. Another one is if developers want to add 6th
application data then they have to update the tool to
accommodate the new changes, which is not really very feasible.
(2) And the other approach to solve this problem was; Big Data.
Big-Data is non-structured database that can contain any type of value,
with large number of volume, with faster processing speed and have
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 72
capability to grow up as much required. And so they have gone for the
second option which is more practical, feasible and suitable according to
the current scenario. Big Data simply can be defined by three
dimensions:
Volume: Big data contains a large amount of data. It can also call the
volume as larger sets of historical data.
Variety: Data variety in big data is a feature that allows storing assorted
data without pre-specified of their type. It can store images, videos, text,
audio or any type of data that comes randomly into the system.
Velocity: Velocity is to consider as a high speed on which data processing
will be done through Big data.
Table 1: Difference between Big data and Traditional Analytics
Basis Big Data Traditional analytics
Type of data Unstructured formats Formatted in rows and
columns
Volume of data 100 terabytes to peta-
bytes
Tens of terabytes or
less
Flow of data Constant flow of data Static pool of data
Analysis methods Machine learning Hypothesis-based
Primary purpose Data-based products Internal decision
support and services
Source: Big Data Big Analytics by Wiley
Above specified table 1 describe basic differences in between
“Traditional analytical” format and Big Data. As per the table “Big Data”
supports unstructured formats of data with high volume storage while
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 73
traditional analytical formats required predefined structure to store the
data will less storage handling. Big data also provides constant flow of
data on high speed while traditional analytical format handled data
transactions through static pools. Big Data is a good option to develop
Data-base products for analytical purpose while traditional analytical
format is good for internal decision support systems.Combining multiple
application data in a single database structure to generate a Data-
Informational system is not an easy task to complete along with
traditional approaches. So, Big Data can consider resolving the problem.
In this multiple application data as input, without bother about their data
structure/value type and can store incoming data using ‘Big Data’ format.
Incoming data can also outflow towards the requested application
database individually to allow each data processed by individual
applications. Stored data in “Big Data” format can be used to
create/generate a “Data-Informational” application to provide processed
result. To understand the above specified problem and its solution the
diagram is shown below:
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 74
Figure1: Big Data Representation
(Source: Primary Data)
Conclusion
Above specified problem of software engineers can be treated as one
example to understand the utility and importance of BIG DATA.
Current Implementation
The industries that are in real need of Big Data are very larger in size
then we can think of. Few of those can be Health-Care organizations, B2B
firms, Industrial products like, Media and entertainment, Retail and
Electric utilities.
Tools
There are tools / software’s available in the market which can help in
implementing the concept of “BIG-DATA”, and HADOOP is one of them.
Hadoop is open-source software for storing and processing of Big data.
Professional Panorama: An International Journal of Management & Technology
Big Data : A Rescue Plan 75
Like its name BIG-DATA is a big work, big analytic, big science, Big
transform of information.
Bibliography
1. Davenport, T. H. (2014),Big Data @ Work, Harvard Business School
Publishing, Boston.
2. Michael Minelli, M. C. (2014),Big Data Big Analytics, Willey, New
Delhi.
3. Viktor Mayer-Schonberger, K. C. (2013),Big Data,John Murray,
london.
4. https://datajobs.com/what-is-hadoop-and-nosql
5. http://www.sas.com/en_us/insights/big-data/hadoop.html"
6. http://quickbase.intuit.com/articles/timeline-of-database-history"
7. http://math.hws.edu/vaughn/cpsc/343/2003/history.html"
8. http://ecomputernotes.com

More Related Content

What's hot

Data Mining in the World of BIG Data-A Survey
Data Mining in the World of BIG Data-A SurveyData Mining in the World of BIG Data-A Survey
Data Mining in the World of BIG Data-A SurveyEditor IJCATR
 
Addressing Big Data Challenges - The Hadoop Way
Addressing Big Data Challenges - The Hadoop WayAddressing Big Data Challenges - The Hadoop Way
Addressing Big Data Challenges - The Hadoop WayXoriant Corporation
 
Big data issues and challenges
Big data issues and challengesBig data issues and challenges
Big data issues and challengesDilpreet kaur Virk
 
Data mining & big data presentation 01
Data mining & big data presentation 01Data mining & big data presentation 01
Data mining & big data presentation 01Aseem Chakrabarthy
 
Implementation of application for huge data file transfer
Implementation of application for huge data file transferImplementation of application for huge data file transfer
Implementation of application for huge data file transferijwmn
 
Big data unit 2
Big data unit 2Big data unit 2
Big data unit 2RojaT4
 
Introduction to Big Data
Introduction to Big DataIntroduction to Big Data
Introduction to Big DataAkshata Humbe
 
An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...
An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...
An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...IJERDJOURNAL
 
Semantic Web Investigation within Big Data Context
Semantic Web Investigation within Big Data ContextSemantic Web Investigation within Big Data Context
Semantic Web Investigation within Big Data ContextMurad Daryousse
 
Big Data in Distributed Analytics,Cybersecurity And Digital Forensics
Big Data in Distributed Analytics,Cybersecurity And Digital ForensicsBig Data in Distributed Analytics,Cybersecurity And Digital Forensics
Big Data in Distributed Analytics,Cybersecurity And Digital ForensicsSherinMariamReji05
 
DSSG Speaker Series: Paco Nathan
DSSG Speaker Series: Paco NathanDSSG Speaker Series: Paco Nathan
DSSG Speaker Series: Paco NathanPaco Nathan
 

What's hot (18)

Big data
Big dataBig data
Big data
 
Data Mining in the World of BIG Data-A Survey
Data Mining in the World of BIG Data-A SurveyData Mining in the World of BIG Data-A Survey
Data Mining in the World of BIG Data-A Survey
 
Addressing Big Data Challenges - The Hadoop Way
Addressing Big Data Challenges - The Hadoop WayAddressing Big Data Challenges - The Hadoop Way
Addressing Big Data Challenges - The Hadoop Way
 
Data mining with big data
Data mining with big dataData mining with big data
Data mining with big data
 
Big data issues and challenges
Big data issues and challengesBig data issues and challenges
Big data issues and challenges
 
Big data Paper
Big data PaperBig data Paper
Big data Paper
 
Data mining & big data presentation 01
Data mining & big data presentation 01Data mining & big data presentation 01
Data mining & big data presentation 01
 
Big data
Big dataBig data
Big data
 
Implementation of application for huge data file transfer
Implementation of application for huge data file transferImplementation of application for huge data file transfer
Implementation of application for huge data file transfer
 
Big data survey
Big data surveyBig data survey
Big data survey
 
Big Data analytics
Big Data analyticsBig Data analytics
Big Data analytics
 
Big Data: Issues and Challenges
Big Data: Issues and ChallengesBig Data: Issues and Challenges
Big Data: Issues and Challenges
 
Big data unit 2
Big data unit 2Big data unit 2
Big data unit 2
 
Introduction to Big Data
Introduction to Big DataIntroduction to Big Data
Introduction to Big Data
 
An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...
An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...
An Investigation on Scalable and Efficient Privacy Preserving Challenges for ...
 
Semantic Web Investigation within Big Data Context
Semantic Web Investigation within Big Data ContextSemantic Web Investigation within Big Data Context
Semantic Web Investigation within Big Data Context
 
Big Data in Distributed Analytics,Cybersecurity And Digital Forensics
Big Data in Distributed Analytics,Cybersecurity And Digital ForensicsBig Data in Distributed Analytics,Cybersecurity And Digital Forensics
Big Data in Distributed Analytics,Cybersecurity And Digital Forensics
 
DSSG Speaker Series: Paco Nathan
DSSG Speaker Series: Paco NathanDSSG Speaker Series: Paco Nathan
DSSG Speaker Series: Paco Nathan
 

Viewers also liked

Impact of octapace model on banking employees a comparative study of private...
Impact of octapace model on banking employees  a comparative study of private...Impact of octapace model on banking employees  a comparative study of private...
Impact of octapace model on banking employees a comparative study of private...Tapasya123
 
Achieving organisation excellence through diversity management
Achieving organisation excellence through diversity managementAchieving organisation excellence through diversity management
Achieving organisation excellence through diversity managementTapasya123
 
Library automation in india and co operation
Library automation in india and co operationLibrary automation in india and co operation
Library automation in india and co operationTapasya123
 
Gold, as an investment avenue
Gold, as an investment avenueGold, as an investment avenue
Gold, as an investment avenueTapasya123
 
Use of six sigma a practical approach
Use of six sigma a practical approachUse of six sigma a practical approach
Use of six sigma a practical approachTapasya123
 
Scenario of commodity market
Scenario of commodity marketScenario of commodity market
Scenario of commodity marketTapasya123
 
Training and development through e learning a case study
Training and development through e learning a case studyTraining and development through e learning a case study
Training and development through e learning a case studyTapasya123
 
Cost analysis for management decisions
Cost analysis for management decisionsCost analysis for management decisions
Cost analysis for management decisionsTapasya123
 
Corporate social responsibility status in india
Corporate social responsibility status in indiaCorporate social responsibility status in india
Corporate social responsibility status in indiaTapasya123
 
Changing dimensions of corporate social responsibility in india
Changing dimensions of corporate social responsibility in indiaChanging dimensions of corporate social responsibility in india
Changing dimensions of corporate social responsibility in indiaTapasya123
 
An evaluation study of mid day meal programme
An evaluation study of mid day meal programmeAn evaluation study of mid day meal programme
An evaluation study of mid day meal programmeTapasya123
 
Seed spices production in rajasthan– an overview
Seed spices production in rajasthan– an overviewSeed spices production in rajasthan– an overview
Seed spices production in rajasthan– an overviewTapasya123
 
Financial inclusion for sustainable development
Financial inclusion for sustainable developmentFinancial inclusion for sustainable development
Financial inclusion for sustainable developmentTapasya123
 
Working capital analysis in rajasthan financial corporation
Working capital analysis in rajasthan financial corporationWorking capital analysis in rajasthan financial corporation
Working capital analysis in rajasthan financial corporationTapasya123
 
Consumer buying behaviour towards the gold jewellery speciall in jaipur city
Consumer buying behaviour towards the gold jewellery speciall in jaipur cityConsumer buying behaviour towards the gold jewellery speciall in jaipur city
Consumer buying behaviour towards the gold jewellery speciall in jaipur cityTapasya123
 
A survey on intelligent textile export business in rajasthan
A survey on intelligent textile export business in rajasthanA survey on intelligent textile export business in rajasthan
A survey on intelligent textile export business in rajasthanTapasya123
 
A study of consumer protection act, 1986 in Banking Sector
A study of consumer protection act, 1986 in Banking SectorA study of consumer protection act, 1986 in Banking Sector
A study of consumer protection act, 1986 in Banking SectorTapasya123
 
Management thinker series
Management thinker seriesManagement thinker series
Management thinker seriesTapasya123
 
Telecommunication sector a prime driver in accelerating india's economic growth
Telecommunication sector a prime driver in accelerating india's economic growthTelecommunication sector a prime driver in accelerating india's economic growth
Telecommunication sector a prime driver in accelerating india's economic growthTapasya123
 
A study on patient satisfaction with special reference to government hospital...
A study on patient satisfaction with special reference to government hospital...A study on patient satisfaction with special reference to government hospital...
A study on patient satisfaction with special reference to government hospital...Tapasya123
 

Viewers also liked (20)

Impact of octapace model on banking employees a comparative study of private...
Impact of octapace model on banking employees  a comparative study of private...Impact of octapace model on banking employees  a comparative study of private...
Impact of octapace model on banking employees a comparative study of private...
 
Achieving organisation excellence through diversity management
Achieving organisation excellence through diversity managementAchieving organisation excellence through diversity management
Achieving organisation excellence through diversity management
 
Library automation in india and co operation
Library automation in india and co operationLibrary automation in india and co operation
Library automation in india and co operation
 
Gold, as an investment avenue
Gold, as an investment avenueGold, as an investment avenue
Gold, as an investment avenue
 
Use of six sigma a practical approach
Use of six sigma a practical approachUse of six sigma a practical approach
Use of six sigma a practical approach
 
Scenario of commodity market
Scenario of commodity marketScenario of commodity market
Scenario of commodity market
 
Training and development through e learning a case study
Training and development through e learning a case studyTraining and development through e learning a case study
Training and development through e learning a case study
 
Cost analysis for management decisions
Cost analysis for management decisionsCost analysis for management decisions
Cost analysis for management decisions
 
Corporate social responsibility status in india
Corporate social responsibility status in indiaCorporate social responsibility status in india
Corporate social responsibility status in india
 
Changing dimensions of corporate social responsibility in india
Changing dimensions of corporate social responsibility in indiaChanging dimensions of corporate social responsibility in india
Changing dimensions of corporate social responsibility in india
 
An evaluation study of mid day meal programme
An evaluation study of mid day meal programmeAn evaluation study of mid day meal programme
An evaluation study of mid day meal programme
 
Seed spices production in rajasthan– an overview
Seed spices production in rajasthan– an overviewSeed spices production in rajasthan– an overview
Seed spices production in rajasthan– an overview
 
Financial inclusion for sustainable development
Financial inclusion for sustainable developmentFinancial inclusion for sustainable development
Financial inclusion for sustainable development
 
Working capital analysis in rajasthan financial corporation
Working capital analysis in rajasthan financial corporationWorking capital analysis in rajasthan financial corporation
Working capital analysis in rajasthan financial corporation
 
Consumer buying behaviour towards the gold jewellery speciall in jaipur city
Consumer buying behaviour towards the gold jewellery speciall in jaipur cityConsumer buying behaviour towards the gold jewellery speciall in jaipur city
Consumer buying behaviour towards the gold jewellery speciall in jaipur city
 
A survey on intelligent textile export business in rajasthan
A survey on intelligent textile export business in rajasthanA survey on intelligent textile export business in rajasthan
A survey on intelligent textile export business in rajasthan
 
A study of consumer protection act, 1986 in Banking Sector
A study of consumer protection act, 1986 in Banking SectorA study of consumer protection act, 1986 in Banking Sector
A study of consumer protection act, 1986 in Banking Sector
 
Management thinker series
Management thinker seriesManagement thinker series
Management thinker series
 
Telecommunication sector a prime driver in accelerating india's economic growth
Telecommunication sector a prime driver in accelerating india's economic growthTelecommunication sector a prime driver in accelerating india's economic growth
Telecommunication sector a prime driver in accelerating india's economic growth
 
A study on patient satisfaction with special reference to government hospital...
A study on patient satisfaction with special reference to government hospital...A study on patient satisfaction with special reference to government hospital...
A study on patient satisfaction with special reference to government hospital...
 

Similar to Big data a rescue plan

Unit 1 Introduction to Data Analytics .pptx
Unit 1 Introduction to Data Analytics .pptxUnit 1 Introduction to Data Analytics .pptx
Unit 1 Introduction to Data Analytics .pptxvipulkondekar
 
RESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWRESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWieijjournal
 
RESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWRESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWieijjournal
 
RESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWRESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWieijjournal1
 
Research in Big Data - An Overview
Research in Big Data - An OverviewResearch in Big Data - An Overview
Research in Big Data - An Overviewieijjournal
 
A Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE TheoremA Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE TheoremAnthonyOtuonye
 
Database Management System ( Dbms )
Database Management System ( Dbms )Database Management System ( Dbms )
Database Management System ( Dbms )Kimberly Brooks
 
In memory big data management and processing
In memory big data management and processingIn memory big data management and processing
In memory big data management and processingPranav Gontalwar
 
Seminar presentation
Seminar presentationSeminar presentation
Seminar presentationKlawal13
 
DEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIDEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIIJCSEA Journal
 
DEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIDEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIIJCSEA Journal
 
DEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIDEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIIJCSEA Journal
 
Big data lecture notes
Big data lecture notesBig data lecture notes
Big data lecture notesMohit Saini
 
Whitepaper: Know Your Big Data – in 10 Minutes! - Happiest Minds
Whitepaper: Know Your Big Data – in 10 Minutes! - Happiest MindsWhitepaper: Know Your Big Data – in 10 Minutes! - Happiest Minds
Whitepaper: Know Your Big Data – in 10 Minutes! - Happiest MindsHappiest Minds Technologies
 
SWOT of Bigdata Security Using Machine Learning Techniques
SWOT of Bigdata Security Using Machine Learning TechniquesSWOT of Bigdata Security Using Machine Learning Techniques
SWOT of Bigdata Security Using Machine Learning Techniquesijistjournal
 

Similar to Big data a rescue plan (20)

Unit 1 Introduction to Data Analytics .pptx
Unit 1 Introduction to Data Analytics .pptxUnit 1 Introduction to Data Analytics .pptx
Unit 1 Introduction to Data Analytics .pptx
 
GADLJRIET850691
GADLJRIET850691GADLJRIET850691
GADLJRIET850691
 
RESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWRESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEW
 
RESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWRESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEW
 
RESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEWRESEARCH IN BIG DATA – AN OVERVIEW
RESEARCH IN BIG DATA – AN OVERVIEW
 
Research in Big Data - An Overview
Research in Big Data - An OverviewResearch in Big Data - An Overview
Research in Big Data - An Overview
 
LITERATURE SURVEY ON BIG DATA AND PRESERVING PRIVACY FOR THE BIG DATA IN CLOUD
LITERATURE SURVEY ON BIG DATA AND PRESERVING PRIVACY FOR THE BIG DATA IN CLOUDLITERATURE SURVEY ON BIG DATA AND PRESERVING PRIVACY FOR THE BIG DATA IN CLOUD
LITERATURE SURVEY ON BIG DATA AND PRESERVING PRIVACY FOR THE BIG DATA IN CLOUD
 
Data mining with big data
Data mining with big dataData mining with big data
Data mining with big data
 
A Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE TheoremA Model Design of Big Data Processing using HACE Theorem
A Model Design of Big Data Processing using HACE Theorem
 
Database Management System ( Dbms )
Database Management System ( Dbms )Database Management System ( Dbms )
Database Management System ( Dbms )
 
In memory big data management and processing
In memory big data management and processingIn memory big data management and processing
In memory big data management and processing
 
Seminar presentation
Seminar presentationSeminar presentation
Seminar presentation
 
DEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIDEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AI
 
DEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIDEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AI
 
DEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AIDEALING CRISIS MANAGEMENT USING AI
DEALING CRISIS MANAGEMENT USING AI
 
Big data lecture notes
Big data lecture notesBig data lecture notes
Big data lecture notes
 
AIS 3 - EDITED.pdf
AIS 3 - EDITED.pdfAIS 3 - EDITED.pdf
AIS 3 - EDITED.pdf
 
Big Data Analysis
Big Data AnalysisBig Data Analysis
Big Data Analysis
 
Whitepaper: Know Your Big Data – in 10 Minutes! - Happiest Minds
Whitepaper: Know Your Big Data – in 10 Minutes! - Happiest MindsWhitepaper: Know Your Big Data – in 10 Minutes! - Happiest Minds
Whitepaper: Know Your Big Data – in 10 Minutes! - Happiest Minds
 
SWOT of Bigdata Security Using Machine Learning Techniques
SWOT of Bigdata Security Using Machine Learning TechniquesSWOT of Bigdata Security Using Machine Learning Techniques
SWOT of Bigdata Security Using Machine Learning Techniques
 

Recently uploaded

A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 

Recently uploaded (20)

A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 

Big data a rescue plan

  • 1. Big Data : A Rescue Plan Big Data: A Rescue Plan Ritu Vashistha1 Vikram Sharma2 Abstract As the time has grown the mankind has also grown in terms of technology, living standard, the way of keeping records. In the initial years the requirement of remembering information and keeping information intact for future use was very less. But as the human developed himself, the requirement of keeping information in an effective manner and in terms of capacity becomes a crucial issue to be resolved. To solve this problem several techniques came up in the meanwhile. And now when data capacity has no limit and humans doesn’t want the pattern of storage and processing to be a hurdle; BIG DATA came up as a rescue plan.The motive of writing this paper is to understand the storage pattern of data developed over the years. Keywords: Data, Technology, Information, Storage, Communication. Introduction he data are the raw facts and figures which individually do not have any meaning, but if it is processed properly then it becomes information, which is useful for mankind to understand things. People use to store data from ancient times, but the way or the pattern of storing data is different in every period. 1 Assistant Professor, Amity Business School, Amity University 2 Team Leader, A3logics(IND) Pvt. Ltd. T
  • 2. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 68 In the starting phase, the human race use to write on walls, rocks or on animal skins and papyrus, but it didn’t last for long. After that the invention of paper gives the human race a new way to store data but it also has its own problems, the paper gets deteriorated quickly and when the data grows the problem of managing data and retrieving it efficiently becomes the next issue. Then comes the digital era where computer were used to store the data, the storage capacity of computer was very high in comparison to the older forms, it can store the data until it was not deleted intentionally or accidentally, but it was still unstructured. Then with the help of database management system the humans got a way to store the data in more structured form and from where the efficient retrieval is also not an issue. As data is growing day by day, the industries were demanding new Database technologies and infrastructures. Technology Enhancements during the Years The data are the raw facts and figures which don’t have meaning and the information is the processed data which is meaningful. As D. Kimbrough Oller and Ulrike Griebel has mentioned in their book ‘Evolution of Communication Systems’ that when the mankind came into existence they don’t have any communication medium which is verbal they have gradually developed it. First it was symbolic they came few alphabets, words and then sentences. But it took several years to become a formal form of communication.
  • 3. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 69 Then came the next issue and that was how to remember everything, people forget things, people die and the medium they have at that time was also not of permanent in nature and it can’t be carried easily if desired. Then paper was invented around 100 BC in china. In 105 AD, under the Han Dynasty emperor Ho-Ti, a government official in China named Ts'ai Lun was the first to start a paper-making industry. It was the revolution in the industry, people learnt how to write, how to keep data in different forms. But this technology also had its disadvantage that it is not permanent in nature, paper gets deteriorated by the time. So when the computers came into existence in 19th century, then also after several years database management system came into existence. A database is a storage where the data can be stored in an organized way, so it can be processed easily and converted into useful information. Now the databases are able to store terabytes of information into it. DBMS is a structured approach, in a single database only similar kind of information is able to get stored. Now with the growing requirements, it was felt that sometimes different kind of information which comes from different sources are required to store at one place, and it discovers a new concept that is Big Data. Big Data In 2001 Doug Laney specified challenge of data management using three V’s. Volume, Variety and Velocity. A phase of new concept was started to begun to overcome with the old challenges. Big Data is a new revolution in data warehousing and data analytics. This concept is introduced to
  • 4. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 70 store and handle large number of data without define data type with high speed of data processing. This concept can handle structured and unstructured data with large volume, which was very difficult to handle with the older technology. Few Important Technologies for Handling Large Data Infrastructure (1) RDBMS: Relational Database Management system designed with the architecture in that data is organized in a defined structured pattern. This type of databases limits themselves with the size of database and with the defined set of data type values stored. These databases need a predefined pattern of values to be stored. (2) No SQL Database Systems: This type of database system contain horizontally scalable; as data continues to explode, high performance, agile processing of information at massive scale properties. In other words, a true smell of Big Data. Therefore, to understand the present requirement of storage pattern and capacity researchers have talked to few people who are working in software companies. We have chosen this industry because these people deal with the basic problems of storage and they solve them for the user industries of their software’s. After several years of development and analysis of products, suddenly they found a new kind of problem. Their company had developed five products in last ten years and providing services and support for all these five products. All five products have large number of data attached
  • 5. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 71 with them in different format. All five products developed on different technologies with different type of data base structure. Gradually company started realising that there should be some method or technique through which they can connect all five product databases with each other. To accomplish this task they need a new system which will gather data from all the five products and can generate its query results. With the help of that system; different type of data flow or search or requirement of client(s) can be easily analysed. With the current structure it will be a difficult task. As they need to take data from all the five systems database and then they have to write in a system that process that data and provide them a logical report. The biggest challenge was the collection of information from different structured database. After a long thought process and problem solving exercises they thought of two approaches (1) Create an independent tool that contains all the information about the 5 database structures and from there it will take data in raw format, process that data and provide the results. In this process a small change in any of the 5 application may create a big change in new tool. Another one is if developers want to add 6th application data then they have to update the tool to accommodate the new changes, which is not really very feasible. (2) And the other approach to solve this problem was; Big Data. Big-Data is non-structured database that can contain any type of value, with large number of volume, with faster processing speed and have
  • 6. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 72 capability to grow up as much required. And so they have gone for the second option which is more practical, feasible and suitable according to the current scenario. Big Data simply can be defined by three dimensions: Volume: Big data contains a large amount of data. It can also call the volume as larger sets of historical data. Variety: Data variety in big data is a feature that allows storing assorted data without pre-specified of their type. It can store images, videos, text, audio or any type of data that comes randomly into the system. Velocity: Velocity is to consider as a high speed on which data processing will be done through Big data. Table 1: Difference between Big data and Traditional Analytics Basis Big Data Traditional analytics Type of data Unstructured formats Formatted in rows and columns Volume of data 100 terabytes to peta- bytes Tens of terabytes or less Flow of data Constant flow of data Static pool of data Analysis methods Machine learning Hypothesis-based Primary purpose Data-based products Internal decision support and services Source: Big Data Big Analytics by Wiley Above specified table 1 describe basic differences in between “Traditional analytical” format and Big Data. As per the table “Big Data” supports unstructured formats of data with high volume storage while
  • 7. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 73 traditional analytical formats required predefined structure to store the data will less storage handling. Big data also provides constant flow of data on high speed while traditional analytical format handled data transactions through static pools. Big Data is a good option to develop Data-base products for analytical purpose while traditional analytical format is good for internal decision support systems.Combining multiple application data in a single database structure to generate a Data- Informational system is not an easy task to complete along with traditional approaches. So, Big Data can consider resolving the problem. In this multiple application data as input, without bother about their data structure/value type and can store incoming data using ‘Big Data’ format. Incoming data can also outflow towards the requested application database individually to allow each data processed by individual applications. Stored data in “Big Data” format can be used to create/generate a “Data-Informational” application to provide processed result. To understand the above specified problem and its solution the diagram is shown below:
  • 8. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 74 Figure1: Big Data Representation (Source: Primary Data) Conclusion Above specified problem of software engineers can be treated as one example to understand the utility and importance of BIG DATA. Current Implementation The industries that are in real need of Big Data are very larger in size then we can think of. Few of those can be Health-Care organizations, B2B firms, Industrial products like, Media and entertainment, Retail and Electric utilities. Tools There are tools / software’s available in the market which can help in implementing the concept of “BIG-DATA”, and HADOOP is one of them. Hadoop is open-source software for storing and processing of Big data.
  • 9. Professional Panorama: An International Journal of Management & Technology Big Data : A Rescue Plan 75 Like its name BIG-DATA is a big work, big analytic, big science, Big transform of information. Bibliography 1. Davenport, T. H. (2014),Big Data @ Work, Harvard Business School Publishing, Boston. 2. Michael Minelli, M. C. (2014),Big Data Big Analytics, Willey, New Delhi. 3. Viktor Mayer-Schonberger, K. C. (2013),Big Data,John Murray, london. 4. https://datajobs.com/what-is-hadoop-and-nosql 5. http://www.sas.com/en_us/insights/big-data/hadoop.html" 6. http://quickbase.intuit.com/articles/timeline-of-database-history" 7. http://math.hws.edu/vaughn/cpsc/343/2003/history.html" 8. http://ecomputernotes.com