Cubes 1.0 Overview
light data warehouse and conceptual modelling
Štefan Urbánek, @Stiivi
stefan.urbanek@gmail.com November 2014

understanding
through metadata

model data
reporting apps / modules
metadata

❄
logical
physical

Categorical Data
Σ =

OLAP
(online analytical processing)
lightweight framework for
conceptual modelling and
analytics

Original Cubes
before 1.0

Workspace 1 × 1 ×
store
process or server
|
model

We needed more!

Models
Stores
file
Σ
database API
Postgres Mongo API
multiple model parts,
different sources
multiple data sources,
heterogenous

Cubes 1.0

Python ≥ 3.4
works with ≥ 2.7 too for the “two” series

■ analytical workspace
■ model providers
■ new and improved backends
■ better extensibility
■ authorisation

Analytical
Workspace

Model Providers
Cubes
Stores
Static Model
Provider
API Model
Provider
sales churn activations events
BI Data
(Postgres)
BI Data 2
(Mongo)
Events
(API)

Workspace
Model Providers
Cubes
Stores
Static Model
Provider
sales churn activations events
BI Data
(Postgres)
BI Data 2
(Mongo)
crm sales events
[workspace]
models_path: /var/lib/cubes/models
[models]
crm: crm.cubesmodel
sales: sales.cubesmodel
events: events.cubesmodel
[store crm]
type: sql
url: postgresql://localhost/crm
[store events]
type: mongo
host: localhost
collection: events

BYOB
bring your own backend
Slicer

Backend

|
Browser
"
Store
#
Provider

Logical Physical
create model connect
physical data store
(database or API)
|
Browser
"
Store
#
Provider
Σ
aggregate
model
cubes
dimensions
model
backend objects

Model Provider
model
cubes
dimensions

Model Provider
■ metadata on-the-fly
■ local or external source
■ might be linked to a store
model
cubes
dimensions

Model
required
required
automatic
automatic
automatic
Dimensions
column (table)
key/attribute
property
dimension
Cubes / Facts
table
collection
event
metric
Backend
SQL
MongoDB
Mixpanel
Google Analytics
Slicer cube dimension

Model Improvements

Model
■ measures → aggregates
■ more front-end metadata
cube categories, dimension role and cardinality
■ customised dimension linking

"measures": [
{
"name": "amount",
"label": "Sales Amount"
},
{
"name": "vat",
"label": "VAT"
}
]
"aggregates": [
{
"name": “total_sales",
"label": "Total Sales Amount",
"measure": "amount",
"function": "sum"
},
{
"name": “total_vat",
"label": "Total VAT",
"measure": "vat",
"function": "sum"
},
{
"name": "item_count",
"label": "Item Count",
"function": "count"
}
]

Aggregates
■ custom name
■ can refer to other aggregates
post-aggregation calculations
■ functions are backend-specific
SQL aggregations: sum, count, count_nonempty, count_distinct, min,
max, avg, stddev, variance, …

Contextual Dimensions
{
"measures": [ … ],
"dimensions": [
{"name": "date", "hierarchies": ["ym", "yqm"]},
{"name": "date", "alias": "contract_date"}
],
…
}
customisable linking properties:
alias, hierarchies, exclude_hierarchies,
default_hierarchy_name, cardinality,
nonadditive

Dimension Roles
dimension.role
time
level.role
year, month, day, …
hint for reporting applications or backends

Cardinality
overload precautions
dimension.cardinality
level.cardinality
tiny < low < medium < high
< <

Browser
Σ

Browser
■ uses logical model
■ implements aggregation
■ builds queries
■ retrieves data
Logical Physical
physical data store
(database or API)
|
Browser
"
Store
Σ
aggregate
model

Browser Methods
■ features()
■ aggregate(cell, drilldown,…)
■ members(cell, dimension, …)
■ facts(cell, …)
■ fact(id)
■ cell_details(cell, drilldown, …)

Split Cell
False True
aggregate(split=cell)
__within_split__
generated dimension

Post-aggregation
“statutils”
■ computed on aggregation result
in Python
■ moving averages, deviation, variance
wma, sma, sms, smstd, smsrd, smsvar
■ aggregate property: window_size

Store

Store
■ provides database or API connection
■ might provide a model
■ slicer tool actions (future)
validation, schema, optimization, ... Logical Physical
physical data store
(database or API)
|
Browser
connect
"
Store

SQL Backend
also known as ROLAP
or SQL query generator

SQL Overview
■ new query builder
■ join optimisation
■ support for outer-joins
■ support for “split” dimension
■ new aggregate functions

❄
fact table
join optimisation

facts match date
master detail
facts detail date
master detail
facts master date
master detail
"joins" = [
{
"master": "fact_contracts.contract_date_id",
"detail": "dim_date.id",
"method": "detail"
}
]

Authentication and
Authorisation

{
“lidia”: {
“allowed_cubes”: [“sales”],
“cube_restrictions”: {
“sales”: [“store:3”]
}
},
“martin”: {
“allowed_cubes”: [“sales”],
“cube_restrictions”: {
“sales”: [“store:5”]
}
}
}

[workspace]
authorization: simple
[authorization]
rights_file: access_rights.json
!
Authorizer

Slicer
server
✂

Model Queries
■ GET /cubes
overview of cubes from all providers
■ GET /cube/sales/model
detailed cube model with described dimensions

Browser Queries
■ GET /cube/name/aggregate
■ GET /cube/name/members/dim
■ GET /cube/name/facts
■ GET /cube/name/fact
■ GET /cube/name/cell

Aggregate
GET /cube/sales/aggregate? cut=date:2010
& drilldown=date|region & split=status:1
& page=10 & page_size=100

{
"cell": [],
"total_cell_count": 2,
"drilldown": [
{
"record_count": 31,
"amount_sum": 550840,
“date.year": 2009
},
{
"record_count": 31,
"amount_sum": 566020,
“date.year": 2010
}
],
"summary": {
"record_count": 62,
"amount_sum": 1116860
}
}

Special Characters
“category:10-24”
→ “10-24”
“city:Nové Mesto nad Váhom”
→ “Nové Mesto nad Váhom"

Relative Time
uses dimension roles and Calendar
date:yesterday
date:90daysago-today
expiration_date:lastmonth-next2months

Output Format
format=csv
format=json
format=json_lines
*
*for facts and members

Deployment
reporting for your app or stand-alone

Public
HTML & JS
Application
Slicer server
store
HTTP request
JSON reply
model
Public
HTML & JS
Application
WSGI
store
HTTP request
JSON reply
Slicer Flask App
model
Public
HTML
Django, Flask, …
store
JSON reply
Cubes
Python API
model
Public
Public
store
Flask
HTML
HTML
Web Application
PHP, RoR, Django
Slicer server
Slicer Blueprint
model
Internal
store
HTTP request
JSON reply
model

Front-ends
generic ad-hoc reporting

✂
Slicer

Cubes Viewer
Jose Juan Montes, ! jjmontesl/cubesviewer

checkgermany.de
*
Front-end by: Felix Ebert (@femeb)
Data by: Friedrich Lindenberg (@pudo) Cubes 0.10.2

Summary & Future

Summary
■ heterogenous pluggable environment
■ easier to extend
■ better SQL query generator

Not Mentioned
■ localisation
■ namespaces
■ calendar
■ query logging

Incubated
■ non-additive properties
■ periods-to-date
■ modeler app
■ cubes.js

Future
■ arithmetic expressions
■ SQL improvements
■ improved API for custom browsers
■ cubes.js

Nutrition Facts
Serving Size 1 cube
Amount Per Serving
Total Fat 0g
Saturated Fat 0g
Trans Fat 0g
% Daily Value
Total Carbohydrate 0g
Dietary Fiber 0g
Sugars 0g
0%
0%

Want to contribute?
#TODO, #FIXME, Issue #
https://github.com/DataBrewery/cubes/issues

Credits

Thanks for 1.0
Robin Thomas
Ryan Berlew
Jose Juan Montes
Squarespace
and all contributors on Github

Thank You
"Stiivi

github.com/DataBrewery/cubes
cubes.databrewery.org