Using Amazon SimpleDB
with Rails
http://webonrails.com

Amazon SimpleDB
● Database web service advertised as:
● Simple, Flexible, Scalable, Fast, Reliable, Inexpensive
● No RDBMS: no SQL, no joins, no schema,
no referential integrity, no transactions
● Ability to store, process and query data
sets
http://webonrails.com

Amazon SimpleDB
● HTTP-Interface
● Pricing: Similar to other AWS
http://webonrails.com

Before we move
Further
http://webonrails.com

Let, have a look
at
Amazon SimpleDB
Common Terms
http://webonrails.com

Amazon SimpleDB Common
Terms
● Domain:
storage container ~ table
● Item:
~ table rows accessed by ID ~ primary key
● Attribute:
~ table columns; every item may have a different
set of upto 256 attributes
http://webonrails.com

Amazon SimpleDB Common
Terms
● Value:
each Attribute may havemultiple Values, always
varchar(1024)[]
http://webonrails.com

SimpleDB
Domain A
Attributes
Items
Values (multiple)
http://webonrails.com

PUT (item, 123),
(description, sweater),
(color, blue),
(color, red)
PUT (item, 456),
(description, dress shirt),
(color, white),
(color, blue)
http://webonrails.com

Amazon SimpleDB API
● Domain level:
● CREATE, LIST, DELETE
● Item level:
● GET, PUT, DELETE attributes with values
● QUERY for unordered item IDs by
attribute values within one domain
● Beware: Eventual Consistency Approach!
http://webonrails.com

Benefits over traditional databases
● No need to pre-define data types
● 'Size' can be 9 or XL or 1.5
● Add attributes anytime
● Attributes specifically for some items
● Index all data
http://webonrails.com

API Summary
● CreateDomain — Create a domain(100).
● DeleteDomain — Delete a domain.
● ListDomains — List all domains.
● DomainMetadata — Retrieve information
about the domain.
http://webonrails.com

API Summary
● PutAttributes — Add or update an item and
its attributes, or add attribute-value pairs to
items that exist already.
● GetAttributes — Retrieve an item and all or
a subset of its attributes and values.
● DeleteAttributes — Delete an item, an
attribute, or an attribute value.
http://webonrails.com

API Summary
● Query — Query the dataset using a query
expression which specifies value tests on
one or more attributes.
● Supported value tests are:
=, !=, <, > <=, >=, starts-with.
[“price” < “12.00”] INTERSECTION [“color” = “green”].
● Order results using the SORT operator
http://webonrails.com

API Summary
● QueryWithAttributes — Enables developers
to retrieve all or a subset of the information
associated with items returned as a
response to a particular query.
http://webonrails.com

SimpleDB vs S3
S3 SimpleDB
● Stores raw data ● Stores indexed data
● Uses higher dense ● Uses less dense
storage drives storage drives
http://webonrails.com

SimpleDB Limits (Cont...)
● Domain size 10 GB per domain
● Domain size 250,000,000 attribute name-
value pairs
● Domain name 3-255 characters
● Domains per account 100
● Attribute name-value pairs/item 256
● Attribute name length 1024 bytes
● Attribute value length 1024 bytes
● Item name length 1024 bytes
● All UTF-8 characters that are valid in XML documents are
allowed in name/value.
http://webonrails.com

SimpleDB Limits
● Attributes per PutAttributes operation 100
● Maximum items in query response 250
● Maximum query execution time 5 seconds
● Maximum comparisons per predicate 10
● Maximum predicates per query 10
expression
● Maximum response size for 1MB
QueryWithAttributes
http://webonrails.com

Working with Numerical Data
● SimpleDB is a schema-less data store
● Everything is stored as a UTF-8 string value
http://webonrails.com

Working with Numerical Data
● Ensure that every number is positive
http://webonrails.com

Working with Numerical Data
● Ensure that every number is positive
What about Negative
Numbers?
http://webonrails.com

Negative Numbers Offsets
● Choose an offset
● That should be arger than the smallest
expected negative number in your data set
● Ex:
● if the smallest expected number in your data set is -12,
choosing offset = 100 might be safe
http://webonrails.com

Negative Numbers Offsets
The following is a sample original data set.
14.58, -12.7, 20, 65, -23
If you apply an offset of 100 the following is the resulting
data set:
114.58, 87.3, 120, 165, 77
http://webonrails.com

Zero Padding
● Since simpleDB uses lexicographical
comparisons “10” comes before “2”
● If we zero pad the numbers to five digits,
\"00002\" comes before \"00010\"
http://webonrails.com

Zero Padding
● Since simpleDB uses lexicographical
comparisons “10” comes before “2”
● If we zero pad the numbers to five digits,
\"00002\" comes before \"00010\"
http://webonrails.com

Dates
● ISO 8601 format
http://webonrails.com

Query dataset
['attribute1' = 'value1']
intersection
not ['attribute2' = 'value2']
union
['attribute3' = 'value3']
http://webonrails.com

SimpleDB with Rails
AWS SDB
Proxy Plugin
http://webonrails.com

http://webonrails.com

SimpleDB with Rails
● Install aws-sdb gem
● gem install aws-sdb
● Install aws_sdb_proxy plugin:
● script/plugin install
git://github.com/bansalakhil/aws_sdb_proxy.git
*Plugin originaly written by martin.rehfeld@glnetworks.de
http://webonrails.com

AWS SDB Proxy Plugin
● Configure config/aws_sdb_proxy.yml
● development:
aws_access_key_id: <aws key>
aws_secret_access_key: <aws secret key>
port: 8888
● Create new domain on Amazon SimpleDB
● rake aws_sdb:create_domain DOMAIN=MyDataStore
http://webonrails.com

AWS SDB Proxy Plugin
● Start AWS SDB proxy
● rake aws_sdb:start_proxy_in_foreground
http://webonrails.com

Mapping RESTful URLs
http://webonrails.com

Using AWS SDB Proxy Plugin
● Create demo ActiveResource model
class Post < ActiveResource::Base
self.site = \"http://localhost:8888\"
self.prefix = \"/MyDataStore/\"
end
http://webonrails.com

Lets try at script/console
>> Article.create(:title => \"This is my First article\")
=> #<Article:0xb7005d04 @attributes={\"updated_at\"=>Sat Dec 13
15:53:30 UTC 2008, \"title\"=>\"This is my First article\",
\"id\"=>601834...98, \"created_at\"=>Sat Dec 13 15:53:30 UTC 2008},
@prefix_options={}>
>> a = Article.create(:title => \"This is my Second article\")
=> #<Article:0xb6fe1f08 @attributes={\"updated_at\"=>Sat Dec 13
15:53:52 UTC 2008, \"title\"=>\"This is my Second article\",
\"id\"=>112...9, \"created_at\"=>Sat Dec 13 15:53:52 UTC 2008},
@prefix_options={}>
>> a.body = \"Nothing\"
>> a.save
=> true
http://webonrails.com

Lets try at script/console
>> Article.find(:all).size
=> 2
>> Article.find(:all, :from => :query, :params => \"['title' starts-with 'This
is' ]\")
=> [#<Article:0xb6f73080 @attributes={\"updated_at\"=>Sat Dec 13
15:53:30 UTC 2008, \"title\"=>\"This is my First article\", \"id\"=>60...98,
\"created_at\"=>Sat Dec 13 15:53:30 UTC 2008},
@prefix_options={}>, #<Article:0xb6f7306c
@attributes={\"updated_at\"=>Sat Dec 13 15:54:14 UTC 2008,
\"body\"=>\"Nothing\", \"title\"=>\"This is my Second article\",
\"id\"=>11...79, \"created_at\"=>Sat Dec 13 15:53:52 UTC 2008},
@prefix_options={}>]
http://webonrails.com

References
● http://aws.amazon.com/simpledb/
● http://inside.glnetworks.de
http://webonrails.com

Thanks
Akhil Bansal
http://webonrails.com
http://webonrails.com