Exploring Modeling - Best Practices with Aerospike Data Types

Exploring Data Modeling
Best Practices with Aerospike Data Types

2 Proprietary & Confidential | All rights reserved. © 2018 Aerospike Inc.
 Hi! I’m Ronen Botzer, and I’m a solutions architect at Aerospike
 I’ve worked at Aerospike since June 2014, first on the Python and PHP clients
 I’ve talked to a lot of enterprise and community users about modeling and getting things
done in Aerospike
 I’m active on StackOverflow and the Aerospike community forum
 I’m excited to be here!
Welcome to Aerospike User Group – Tel Aviv!

 I have some questions for you
 Aerospike Data Types
 The Map API
 Modeling with ordered Maps
 Maps as containers for capturing and searching events, using timestamp keys
 Maps as containers for capturing and searching messages, using timestamp values
 Return types for Map operations
Agenda

Aerospike is a Primary Key Database
Objects stored in Aerospike are called records
A bin holds the value of a supported data type: integer, double, string, bytes, list, map,
geospatial
Every record is uniquely identified by the 3-tuple (namespace, set, primary-key)
A record contains one or more bins
(namespace, set, primary-key)
EXP – Expiration Timestamp
LUT – Last Update Time
GEN – Generation
RECORD
EXP LUT GEN BIN1 BIN2

 Aerospike is a row-oriented distributed database
 Rows (records) contain one or more columns (bins)
 Similar to an RDBMS with primary-key table lookups
 Single record transactions
 Namespaces can be configured for strong consistency
Seems Familiar?
Aerospike RDBMS
Namespace Tablespace or Database
Set Table
Record Row
Bin Column
Bin type
Integer
Double
String
Bytes
List (Unordered, Ordered)
Map (Unordered,
K-Ordered, KV-Ordered)
GeoJSON

Single-bin with integer value; single-bin with bytes value; multi-bin with string, integer, list of maps
and bytes values.
Example Records
('test', 'users’, 1234) {
'name': 'Apolonius Kensington'
'age': 31
'cards': [
{ 'type': 'visa', 'last4': 5996,
'expires': '2019-09'},
{
'type': 'mastercard', 'last4': 4177,
'expires': '2018-11'}
]
'pass': p??Ȇ??"R*?vw96Q
}
('users', 'eu', 'froedrick@example.com’) ?C"ӽN'?z?h?B?‫ژ‬
('counters', 'frequency-cap', 'ad-1234:user-987') 3

 The Aerospike data types have powerful APIs. Get to know them.
 Native operations perform and scale better than UDFs.
 Atomic operations simplify application logic and reduce network load.
 Complex Data Types (List, Map) can be nested to many levels.
 Operations can execute only at the top level.
 Query operations have interesting behaviors.
Data Modeling in Aerospike

Map operations supported by the server. Method names in the clients might be different.
set_type() (unordered, k-ordered or kv-ordered)
add(), add_items(), increment(), decrement()
clear()
remove_by_key(), remove_by_index(), remove_by_rank()
remove_by_key_interval(), remove_by_index_range()
remove_by_value_interval(), remove_by_rank_range(), remove_all_by_value()
remove_all_by_key_list(), remove_all_by_value_list()
size()
get_by_key(), get_by_index(), get_by_rank()
get_by_key_interval(), get_by_index_range()
get_by_value_interval(), get_by_rank_range(), get_all_by_value()
get_all_by_key_list(), get_all_by_value_list()
Map Operations

{
'name': 'Apolonius Kensington'
'age': 31
'cards': [
{ 'type': 'visa', 'last4': 5996,
'expires': '2019-09'},
{
'type': 'mastercard', 'last4': 4177,
'expires': '2018-11'}
]
'pass': p??Ȇ??"R*?vw96Q
}
 increment() is atomic, can execute on top level values (‘age’)
 You cannot increment the integer value at ‘last4’ because it’s nested
 Use atomic operations wherever possible
Incrementing a Map Value

{'a': 1, 'b': 2, 'c': 3, 'yy': 0, 'z': 26}
 The map key is the ID part of the element. For example 'a’
 The map value is the value part of the element. For example 1 for key 'a’
 The index is the order of the key. The index of 'z' is 4 or -1
 The rank is the order of the value. The rank of 'yy' is 0 or -5
Map Terms

If events come in with unique identifiers (UUID, millisecond timestamp, etc) they can be
collected in a map.
A timeseries of events can be modeled as a map with unique timestamps as its map keys
{
t1: ['event-type', {'event-attr1':10, 'event-attr2':20} ],
t2: ['event-type', {'event-attr1':11, 'event-attr2':22} ],
:
}
The events can now be queried on time (map key) and event type (map value) dimensions.
Maps as Containers for Events – Timestamp Keys

Starting Aerospike Database 4.3.1, lists are compared as follows
1. [1, 1] < [1, 3] – Compare the entries in order.
2. [1, 2] < [1, 2, 0, 1] – Length is the secondary criteria.
So [1, 1] < [1, 2] < [1, 2, 0, 1] < [1, 3]
This can be used by these Map API methods
• get_by_value_interval( [v1, nil], [v2, nil] )
• get_all_by_value( [v1, *])
• get_all_by_value_list( [v1, *], [v2, *])
* In the C client * is AS_CMP_WILDCARD, in the Python client it’s Aerospike.CDTWildcard(), etc
How List Values are Compared

{
1523474230000: ['fav', {'sku':1, 'b':2}],
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474236006: ['viewed', {'foo':'bar', 'sku':3, 'zz':'top'}],
1523474235005: ['comment', {'sku':1, 'c':1234}],
1523474233003: ['viewed', {'sku':3, 'z':26}],
1523474234004: ['viewed', {'sku':1, 'ff':'hhhl'}]
}
get_all_by_value(['comment', *])
{
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474235005: ['comment', {'sku':1, 'c':1234}]
}

{
1523474230000: ['fav', {'sku':1, 'b':2}],
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474235005: ['comment', {'sku':1, 'c':1234}],
1523474233003: ['viewed', {'sku':3, 'z':26}],
}
get_all_by_value_list([['comment', *], ['fav', *]])
{
1523474230000: ['fav', {'sku':1, 'b':2}],
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474235005: ['comment', {'sku':1, 'c':1234}]
}

{
1523474230000: ['fav', {'sku':1, 'b':2}],
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474235005: ['comment', {'sku':1, 'c':1234}],
1523474233003: ['viewed', {'sku':3, 'z':26}],
1523474234004: ['viewed', {'sku':1, 'ff':'hhhl'}],
}
get_all_by_key_interval(1523474233000, 1523474234999)
{
1523474233003: ['viewed', {'sku':3, 'z':26}],
}

If the events have unique identifiers (UUID), but time still needs to be a dimension for
querying, a timeseries can be modeled with UUID map keys and timestamp map values.
{
uuid1: [t1, {'event-attr1':10, 'event-attr2':20} ],
uuid2: [t2, {'event-attr1':11, 'event-attr2':22} ],
:
}
Maps as Containers for Events – Timestamp Values

{
'0edf5b73-535c-4be7-b653-c0513dc79fb4': [1523474230, "Billie Jean is not my lover", "MJ"],
'29342a0b-e20f-4676-9ecf-dfdf02ef6683': [1523474241, "She's just a girl who", "MJ"],
'31a8ba1b-8415-aab7-0ecc-56ee659f0a83': [1523474245, "claims that I am the one", "MJ"],
'9f54b4f8-992e-427f-9fb3-e63348cd6ac9': [1523474249, "...", "Tito"],
'1ae56b18-7a3c-4f64-adb7-2e845eb5094e': [1523474257, "But the kid is not my son", "MJ"],
'08785e96-eb1b-4a74-a767-7b56e8f13ea9': [1523474306, "ok...", "Tito"],
'319fa1a6-0640-4354-a426-10c4d3459f0a': [1523474316, "Hee-hee!", "MJ"]
}
get_all_by_value_interval([1523474240, nil], [1523474246, nil])
{
'31a8ba1b-8415-aab7-0ecc-56ee659f0a83': [1523474245, "claims that I am the one", "MJ"]
}

{
}
get_by_rank_range(-2, 2)
{
}

Return Types for Map Operations
Result Type Description
None No results
Index Key order: 0 = smallest key, -1 = largest key
RevIndex Reverse key order: 0 = largest key
Rank Value order: 0 = smallest value, -1 = largest value
RevRank Reverse value order: 0 = largest value
Count Return number of items matching criteria
Key Key for single item operations and list of keys for multi-ops
Value Value for single item operations, list of values for multi-ops
KeyValue Key Value pairs. The exact format is client dependent

{
1523474230000: ['fav', {'sku':1, 'b':2}],
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474235005: ['comment', {'sku':1, 'c':1234}],
1523474233003: ['viewed', {'sku':3, 'z':26}],
}
get_all_by_value(['viewed', *], resultType=count)
3
get_all_by_value(['comment', *], resultType=count)
2
Counting Events

{
}
remove_by_rank_range(-1000, 1000, INVERTED, resultType=none)
Trimming the Map – Timestamp Values

{
1523474230000: ['fav', {'sku':1, 'b':2}],
1523474231001: ['comment', {'sku':2, 'b':22}],
1523474235005: ['comment', {'sku':1, 'c':1234}],
1523474233003: ['viewed', {'sku':3, 'z':26}],
1523474234004: ['viewed', {'sku':1, 'ff':'hhhl'}],
}
remove_by_index_range(-1000, 1000, INVERTED, resultType=none)
Trimming the Map – Timestamp Keys

 We talked about Aerospike’s data types, and how they can be used for modeling
 We went in more detail into the flexible and well used Map API
 We modeled two types of collections (user events, messages) using ordered maps
 We saw how a list as the map-value can allow for different kinds of queries
What next?
 Go to GitHub; clone the code samples repo; run it; read the code
 Read the Aerospike blog. Get familiar with all the database features
 Participate in the community forum (https://discuss.aerospike.com)
 Sign up for our developer newsletter. You’ll find blog posts, code samples and more in it
Summary

List & Map API
 https://www.aerospike.com/docs/guide/cdt-map.html
 https://www.aerospike.com/docs/guide/cdt-list.html
 https://www.aerospike.com/apidocs/python/client.html#map-operations
 https://www.aerospike.com/apidocs/java/com/aerospike/client/cdt/MapOperati
on.html
Code Samples
 https://github.com/rbotzer/aerospike-cdt-examples
Aerospike Training
 https://www.aerospike.com/training/
More material you can explore:

Thank You!
Any questions?
ronen@aerospike.com

Exploring Modeling - Best Practices with Aerospike Data Types

More Related Content

What's hot

Similar to Exploring Modeling - Best Practices with Aerospike Data Types

Recently uploaded

Exploring Modeling - Best Practices with Aerospike Data Types

Editor's Notes