This document discusses big data concepts and patterns. It covers topics like data mining, data visualization, data analytics, open data, data science, cloud computing, mobile technologies, and the internet of things as they relate to big data. It also discusses data issues around volume, velocity, and variety of data as well as infrastructure needs. Additionally, it covers big data principles, scalability, fault tolerance, availability, flexibility, and research domains in big data including optimization, data science, design, security, relationships to other trends, and applications in various business fields.

1. Thinking BIG
Big Data Concepts and
Patterns
.p
Dr. Lilia Sfaxi
April 2017

2. DATA
2

3. DATA
Data Mining
DataViz Data Analytics
Open Data
Data Science
Cloud Mobile IoT BI
Big Data
3

4. Collection
Processing
Storage
Visualization
Security - Monitoring
Business
Data Issues
4

5. Data Issues
Available Data Value Extraction On-Time
5

6. Data Issues
Infrastructure

7. Data Issues
Infrastructure
Scalable
Available
Flexible
Volume
Velocity
Variety7

8. Data Issues
3V 8

9. Store THEN Process
Big Data Principles
Data-driven Decisions (NOT Decision-Driven Data)
Redunduncy is GOOD
There is NO NEEDLESS Data
GO Polyglot!
9

10. Scalability
Scale UP vs Scale OUT
10
Classical : Scale UP

11. Scalability
Scale UP vs Scale OUT
11
Big Data : Scale OUT

12. Ring Master - Slave
12
Scalability
Architectures

13. Classical Architectures Big Data Architecture
Database Application Server
13
Scalability
Co-Localization of Processing & Storage

14. 14
Scalability
FaultTolerance
Data Replication

15. 15
Scalability
FaultTolerance
Data Replication
Cluster Replication

16. 16
Scalability
FaultTolerance
Data Replication
Cluster Replication
Rack Awareness

17. Availability
CAPTheorem
17
Consistency
Availability PartitionTolerance
Pick
Any
Two

18. Availability
CAPTheorem
18
Consistency
Atomicity
Isolation
Durability
Basically Available
Soft-State
Eventual Consistency
ACID BASE

19. Availability
Time
19
Stream Processing Support
Dynamic and Interactive Charts and Reports
In-Memory Processing
In-Memory Storage

20. Flexibility
ONE application can support…
20
Diverse Data Sources
Schema-less Data
Multiple Processing Paradigms
Multiple Storage Systems

21. Research Domains
21
In the Big Data Domain
EVERYTHING
IsYet to Be Done

22. Research Domains
Optimization
22
Processing Time Optimization
Storage Size and Compression
Data Access Optimization
Tradeoff bw Consistency and Availability

23. Research Domains
Data Science
23
Distributed Algorithms for Machine Learning
Semantic & Sentiment Analysis
Visualization Algorithms
Data Mining, Data Prediction, Data Analytics

24. Research Domains
Big Data Design
24
Design Methodologies for Big Data Systems
Standardization of Big Data Architectures
Design and Architectural Patterns
Modeling Language(s) for Schema-less Data

25. Research Domains
Big Data Security
25
Non-Relational Databases Security
Logs Gathering and Analysis
Source Data Validation and Filtering
Access Control and Cryptography

26. Research Domains
Big Data & OtherTrends
26
Big Data & Business Intelligence
Big Data & Cloud Computing
Big Data & Internet of Things
Big Data & Mobile

27. Research Domains
Big Data & Business Fields
27
Big Data in Education
Big Data in Health
Big Data in Art
Big Data in Finance

28. In Other Words…
28
Big Data is Here to STAY!

29. Thinking BIG
Big Data Concepts and
Patterns
.p
Dr. Lilia Sfaxi
April 2017