This Presentation provides the basics for understanding data

1. Understanding
Data
Shahd Salama
Software Engineer

2. Outline (Part 1)
• What are Structured, unstructured, semi-structured data?
• DBMS (Database Management System)
• ACID consistency
• Distributed DB
• Replicas and Sharding
• What is No-SQL?
• When to use No-SQL and when not to use No-SQL?

3. Outline (Part 2)
• GCP options for database
• What is managed cloud services
• Cloud storage
• Cloud SQL
• Cloud datastore (no sql)
• Cloud big table (no sql)

4. Structured data
• Can be stored in DB in tables (rows and columns)
• Tables have relational keys
• High degree of organization
• Database contains a schema
• Schema defines tables, tables fields and relationships

5. Click to add text
This erd represents one
to many relation

6. Unstructured data
• Data has no structure
• Data has no model
• Cannot fit in relational databases
• Can be represented as a sequence of bytes
• Examples: Images, Videos, files and emails

7. NLP, text analysis and data mining provide
methods to find patterns of info in unstructured
data
Unstructured data

8. Semi-structured data
• It is a type of structured data, but it lacks the model
• Tags are used to identify certain elements
• Data does not have rigid structure
• A group of data that belongs to a certain class can
have different attributes
• Data cannot fit into tables
• Examples : JSON, XML

9. Semi-structured data

10. DBMS
• A DBMS makes it possible for end users to create,
read, update and delete data in a database
• The DBMS essentially serves as an interface between
the database and end users or application programs

11. DBMS

12. Transactions in DBMS
Transactions is a group of tasks
Example:
A bank Employee wants to transfer 500 $ from account
X to account Y

13. Transactions in DBMS
open_account(A)
old_balance = A.balance
new_balance = old_balance – 500
A.balance = new_balance
close_account(A)
open_account(B)
old_balance = B.balance
new_balance = old_balance + 500
B.balance = new_balance
close_account(B)
Example

14. A Transaction in DB must maintain
ACID consistency

15. ACID
• Atomic
• Consistency
• Isolation
• Durability

16. Atomic
• A transaction must be treated as atomic unit
• All operations must be executed or none
• As in the bank transfer example we cannot
execute
The transaction cannot be partly excuted for example
subtract the transferred money from the first account and
not send it to the other account

17. Consistency
• A transaction either creates a new and valid state of
data, or, if any failure occurs, returns all data to its
state before the transaction was started
• If database was consistent before transaction then DB
must be consistent after transaction

18. Isolation
• If more than one transaction is executed at the same
time both should be in total isolation of each other

19. Isolation
• Example:
One day you go to a restaurant and the restaurant had
no other customers and you make an order then you
will get your order
The next day you go to the same restaurant and it was
full and you make an order then you should get the
same order as the day before

20. Isolation
In other words you should get the
same order whether another
orders are taken place in
the restaurant or not

21. Isolation
So If you execute T1 alone you will get
The same result when you execute T1 while T2 is
executing

22. Durability
• Guarantees that transaction that have committed will
survive permanently
• Whatever changes are made to DB those must have
affect irrespective to hardware or software failure
• Durability can be achieved by flushing the
transaction's log records to non-volatile
storage before acknowledging commitment.

23. Distributed database
•It is a database in which storage devices are not all attached to a
common processor.
• It may be stored in multiple computers, located in the same
physical location; or may be dispersed over a network of
interconnected computers

24. Distributes Database Management System
•DDBMS sync all data periodically and insures any
changes (updates, deletes and additions ) are
performed on data in one place will be automatically
reflected on data stored elsewhere
•User will always see data consistent with data seen by
another user

25. Replicas
• It is frequent copying data in db from one computer or
server to another
• So that all users share the same level of information
• The result : is distributed DB which portions of DB are
stored in multi physical locations and processing is
distributed among different nodes

26. Sharding
• A database sharding is partitioning of data in DB
• Each individual portion is referred to as shard
• Breaking BD into much smaller DBs

27. Eventually consistent data
• Eventual consistency is a consistency model used in
distributed computing to achieve high availability that
informally guarantees that, if no new updates are made to a
given data item, eventually all accesses to that item will
return the last updated value(consistency is maintained
later)
• Eventually-consistent services are often classified as
providing BASE (Basically Available, Soft state, Eventual
consistency) semantics, in contrast to traditional ACID
(Atomicity, Consistency, Isolation, Durability) guarantees.

28. Cap for distributed DB
• System made up of multi nodes (scaled out)
communicating with each other over a network

29. CAP Theorem
• Consistency : if you write to one node and read from
another node then you get what you wrote
 Data is consistent across all nodes
 All nodes must get the same most recent writes
• Availability: when you talk to a node it will respond
without a guarantee that respond is the most recent
• Partition tolerance: when the network is partitioned it
a node fails the system continues to work

30. CAP Theorem says
we can only have two of these properties
at most in distributed system

32. No-SQL
A NoSQL (originally referring to "non SQL" or "non-
relational") database provides a mechanism for storage
and retrieval of data that is modeled in means other
than the tabular relations used in relational databases.

33. No-SQL
A better was to describe no-sql is is Not Only SQL
because you can actually use SQL in No-SQL

34. Types and examples of NoSQL databases

36. No-SQL cannot support
• Joins
• No constraints support (example: null constraints)
• No support for complex transactions
• Data integrity
Example :
 Insert 3 records
Update 2 records
Check something if true rollback

37. No-SQL cannot support data integrity

38. No-SQL vs SQL

39. SQL vs NO-SQL
SQL NO-SQL (non-relational )
Structured data only Unstructured, semi-
structured, structured data
Fixed schema Flexible schema
Non-scalable scalable
Non distributed Distributed
transactional Non transactional

40. When to use NO-SQL
1. Storing and retrieving big quantities of data (Big Data)
2. Relationships between elements is not important
3. Dealing with growing lists of elements (social media
posts)
4. Unstructured data or the structure of data changes
rapidly
5. Constraints and validations of data can be performed
in application layer and no need to implement
constraints in DB

41. When not to use NO-SQL
1. Complex transactions (bank transfer from one
account to another example)
2. Joins must be handled by DB
3. Validations and constraints must be andeled by DB

42. GCP options for database
We need persisted durable
storage
Persistent storage means
data doesn’t go away after
device is turned off unlike
cache and ram

43. Cloud storage for unstructured data
Buckets : where you store your data
Objects : things you are storing

44. Creating buckets in cloud storage
You specify
• Name
• Class
• Location

45. Bucket class in cloud storage
1. Standard :
Best latency , highest availability 99.9%
2. Reduced availability :
Availability 99% , less expensive
3. Nearline :
Higher latency (few more seconds for the first byte)
way less expensive
if you will access object less than once a month
Best for archival scenarios
99 % available

46. Bucket location in data storage
1. Multi regional (boarder geographic area)
2. Regional : corresponding to region supported in
other cloud platform service like compute engine

47. Uploading objects to cloud storage
1. From developer console in gcp website
2. From command line using gutil in GC SDK:
Can access files in file system and objects stored in amazon S3

48. Cloud SQL for structured data
Cloud sql is hosted sql
All that you can do with sql can be done
with cloud sq
Google takes care of keeping the os up-
to-date, preforming backups...

49. Cloud SQL for structured data
Cloud SQL is a fully-managed
database service that makes it easy to set up,
maintain, manage, and administer your
relational PostgreSQL and MySQL databases
in the cloud.

50. Cloud SQL Supports
• Rich query language
• Primary and secondary indexes
• Acid transactions
• Relational integrity
• Stored procedure

51. Cloud SQL Supports

52. Cloud datastore for semi-structured
data
• Cloud data store is from the no-sql Family
• Scales gracefully from very small to very large

54. Cloud datastore for semi-structured
data

55. Cloud big table for semi-structured
and structured data
• Cloud big table is from the no-sql Family
• Stores over a terabyte of structured data
• Low lever DB
• High scalability
• Low latency (uses single zone)
• Does not scale down to small size

56. Thanks