Dynamodb ppt

REALTIME
RECORDIN
G OF DATA
IN AWS
DYNAMO
DB
BY: Shelly & Sagar

overview
■ BRIEF HISTORY OF DATA PROCESSING
■ RELATIONAL (SQL)VS. NONRELATIONAL
(NOSQL)
■ Why noSQL?
■ ACIDVS CAP
■ DynamoDB- what is it?
■ DynamoDB ARCHITECTURE
■ Conditional Writes
■ Provisioned throughput
■ QUERYVS SCAN
■ Operations
■ Benefits
■ Limitations
■ DEMO

TIMELINE OF
DATABASE
TECHNOLOG
Y

RELATIONAL (SQL)VS. NONRELATIONAL
(NOSQL)
Optimised for storage Optimised for compute
Normalized/relational Denormalized/hierarchical
Ad hoc queries Instantiated views
Scale vertically Scale horizontally
Good for OLAP Built for OLTP at scale
SQL NOSQL

Why noSQL?
■ DataVelocity – Huge data getting generated
in less time
■ DataVariety – Structured, semi-structures
and unstructured
■ DataVolume –Terabytes and petabytes of
data
■ Dynamic Schema – Flexible data model
■ Auto-sharding – Horizontal scalability
■ Continuous Availability
■ Integrated Caching
■ Replication – Automatic replication to support
high availability
■ Dynamic provisioned throughput

Acid vs CAP
■ ACID properties relates to relational databases
• Atomicity
– Requires each transaction to be all or nothing
• Consistency
– Brings databases from one valid state to another
• Isolation
–Takes care of concurrent execution and avoid overwriting
• Durability
– Committed transactions will remain so in the event of power loss,
crashes etc.
■ CAP theorem relates to noSQL database
• Consistency
– All nodes see the same data at the same time
• Availability
– A guarantee that every request receives a response about whether it
was successful or failed
• Partition tolerance
–The system continues to operate despite arbitrary message loss or
failure of part of the system)

DynamoDB-
what is it?
■ Fully managed nosql database service on
AWS
■ Data model in the form of tables
■ Data stored in the form of items (name –
value attributes)
■ Automatic scaling
■ – Provisioned throughput – Storage scaling
– Distributed architecture
■ Easy Administration
■ Monitoring of tables using CloudWatch
■ Integration with EMR (Elastic MapReduce)
■ – Analyze data and store in S3

BENEFITS OF DYNAMODB
 CONSISTERNT,PREDICTABLE PERFORAMNCE
• single milli second latency
• < 5ms reads OR <10ms writes
• backed by SSD drives
 FLEXIBLE DATA MODEL
• key/attribute pairs
• no schema required
• easy to create and adjust
 SEAMLESS SCALABILITY
• no table size limits
• unlimited storage
• no downtime
 DURABLE
• consistent disk writes only
• replication across data canters and availability zones

DynamoDB ARCHITECTURE
PRIMARY
KEY
(MANDATO
RY FOR
EVERY
TABLE) –
HASH OR
HASH +
RANGE
DATA
MODEL IN
THE FORM
OF TABLES
DATA
STORED IN
THE FORM
OF ITEMS
(NAME –
VALUE
ATTRIBUTE
S)
SECONDAR
Y INDEXES
FOR
IMPROVED
PERFORMA
NCE
–
LOCALSEC
ONDARYIN
DEX
–
GLOBALSE
CONDARYI
NDEX
• SCALAR
DATA TYPE
(NUMBER,
STRING
ETC) OR

QUERYVS
SCAN
• Query
– Search based on primary key and examines only matching data
– Maximum 1MB result
– Query result sorted by range key
– Query result can be opted to give strong consistency
– Query can be done based on secondary index
– Requires less provisioned throughput
– Query performance depends on amount of data retrieved
• Scan
– Examines every item and then applies filter
– Maximum 1MB result
– Scan result is always eventually consistent
– Required high provisioned throughput
– Scan performance depends on table size
– Secondary index does not have any impact on scan performance
– Parallel scan can be performed usingTotalSegments

Components of the System
Nodemcu
DHT11 temperature and
moisture sensor
OLED Display

Code –To
connect to
AWS IOT
server
■ #include "FS.h"
#include <ESP8266WiFi.h>
#include <PubSubClient.h
#include <NTPClient.h>
#include <WiFiUdp.h>
const char* AWS_endpoint = "a3hu9vk57nb9k8-ats.iot.us-east-
1.amazonaws.com"; //MQTT broker ipvoid callback(char* topic, byte*
payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
}
WiFiClientSecure espClient;
PubSubClient client(AWS_endpoint, 8883, callback, espClient)

Library for
showing data on
OLED Display
■ #include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>
#define SCREEN_WIDTH 128 // OLED
display width, in pixels
#define SCREEN_HEIGHT 64 // OLED
display height, in pixels
// Declaration for an SSD1306 display
connected to I2C (SDA, SCL pins)
Adafruit_SSD1306
display(SCREEN_WIDTH,
SCREEN_HEIGHT, &Wire, -1);

Library for
reading from
DHT sensor
■ #include <DHT.h>
void loop()
{
float h = dht.readHumidity(); // Reading
Temperature form DHT sensor
float t = dht.readTemperature(); //
Reading Humidity form DHT sensor
float tF=(t*1.8)+32;
if (isnan(h) || isnan(t))
{
Serial.println("Failed to read from DHT
sensor!");
return;

CREATINGTHE URLTO CONNECTTO
AWS IOT CORE

STRATINGTHE MQTT PROTOCOLTO
RECEIVE DATA

DATA RECEIVEDTHROIGHT MQTT
PROTOCOL

DATA SEEN FROMTHE OUTPUT
SCREEN OF ARDUINO IDE

LIMITATION
S of
DynamoDB
64KB limit on item size (row size)
1 MB limit on fetching data
Pay more if you want strongly consistent data
Size is multiple of 4KB (provisioning throughput wastage)
Cannot join tables
Indexes should be created during table creation only
No triggers or server side scripts
Limited comparison capability (no not_null, contains etc)

References ■ https://docs.aws.amazon.com/amazondyn
amodb/
■ http://en.wikipedia.org/wiki/Consistent_ha
shing
■ https://youtu.be/vCRXqVld8_c
■ https://youtu.be/28FS2qix2u4
■ https://electronicsinnovation.com/storing-
esp8266-data-into-amazon-dynamodb-
using-aws-iot-coremqtt-arduino/
■ https://www.slideshare.net/AmazonWebSe
rvices/introduction-to-amazon-dynamodb-
73191648

Dynamodb ppt

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Dynamodb ppt

Similar to Dynamodb ppt (20)

Recently uploaded

Recently uploaded (20)

Dynamodb ppt