This document discusses inconsistencies in big data. It begins by introducing the concept of big data and defining its key characteristics: volume, velocity, variety, variability, veracity, and complexity. It then examines different types of inconsistencies that can occur in big data, including temporal inconsistencies dealing with conflicting time-related information, spatial inconsistencies involving geometric or location-based data, and text inconsistencies found in unstructured language. The document also looks at inconsistencies relating to functional dependencies where certain attribute values should determine other attributes. It concludes by discussing how identifying and resolving inconsistencies can help improve big data analysis through techniques like inconsistency induced learning.

1. INCONSISTENCIES IN BIG DATA
1
Prepared by,
Minu Joseph
Guided by,
Mr. Thomas Varghese

2. Contents
• Introduction.
• Problem Statement.
• 3V’s
• Big data.
• Defining Big data.
• Dimensions of big data.
• Sources, applications of big data.
• Inconsistencies in big data.
• Inconsistency induced learning.
• Conclusion.
• References.
2

3. Introduction
• A torrent of data is generated and captured in
digital form due to advancement in science
and technology.
• Everything we do is increasingly leaving a
digital trace.
• Large data sets which are so large and
complex that traditional data processing
applications are inadequate.
3

4. Problem Statement
• Big Data-The next big thing in IT industry.
• Classification of big data inconsistencies.
• Big Data and Big Data analysis in terms of
issues and challenges.
• Inconsistency Induced Learning- A tool to turn
big data inconsistencies into helpful formulas
for better analysis of results.
4

5. 5

6. Big Data
• Big data can be described by:
Volume
Velocity
Variety
Variability
Veracity
Complexity
6

7. What is BIG DATA?
7

8. 8

9. Dimensions In Big Data
9

10. 10

11. 11

12. Levels of Knowledge
12

13. INCONSITENCIES IN BIG DATA
• Temporal
• Spatial
• Text
• Functional Dependency
13

14. Temporal Inconsistencies
• Conflicting information.
• Data items with conflicting circumstances may
coincide or overlap in time.
• SRS often contain inconsistent information.
• Inconsistent information affects the
correctness and performance of the system.
• Due to concurrent programming errors
Therac-25(1985-1987) lead to 6 accidents.
14

15. List of temporal inconsistencies
15

16. Spatial Inconsistencies
• Happens in datasets which include geometric
or spatial dimensions.
• Traditional DB systems are enhanced to
include spatially referenced data.
• Spatial inconsistencies can arise from
 Geometric representation of objects
 Spatial relationship between objects
 Aggregation of composite objects.
16

17. Spatial Inconsistencies contd..
17

18. Text Inconsistencies
• Inconsistencies found in unstructured natural
language text.
• Data generated from social media, blogs,
emails etc.
• If two texts are referring to same event or
entity they are said to be of co-reference.
• Contradiction Detection detects text
inconsistencies and has many applications.
18

19. Text Inconsistencies contd..
19

20. Functional Dependency Inconsistency
• When certain attribute values are equal, then
other attribute values must also be equal.
• Many big databases are stored , aggregated
and cleaned through the help of RDBMS.
• Here Functional dependencies play an
important role in enforcing the integrity
constraints for the database.
20

21. Functional Dependency Inconsistency
contd…
21
• Variation of Functional Dependencies will
result in inconsistencies in data and
information.

22. Inconsistency Induced Learning
• Improves data quality
• Helps to enhance big data applications.
• Accommodates lifelong learning by allowing
successive learning episodes to be triggered
through inconsistencies an agent encounters
during its problem solving episodes.
• Basic idea is to identify the cause of
inconsistency and then apply cause specific
heuristics to resolve inconsistencies.
22

23. Conclusion
• Multidimensional issues and challenges in big
data and big data analysis.
• Types of inconsistencies.
• How to improve quality of big data analysis.
23

24. References
• www.slideshare.com
• dl.acm.org
• www.ieeexplore.ieee.org
• D. Zhang, On Temporal Properties of Knowledge Base
Inconsistency. Springer Transactions on Computational
Science.
• M. Schroeck, R. Shockley, J. Smart, D. Romero-Morales, and P.
Tufano, Analytics: the real-world use of big data: how
innovative enterprises extract value from uncertain data,
Executive Report, IBM Institute for Business Value and Said
Business School at the University of Oxford.
• Nasrin Irshad Hussain ,Big Data,www.slideshare.com
24

25. QUESTIONS?
25

26. 26