Download as PDF, PPTX
![one file or one directory bundle
✂
model
browser
formatters
http
backends
one per serving:
server
workspace
[workspace]
backend=sql
url=postgresql://localhost/database
modules](https://image.slidesharecdn.com/cubes-pluggablemodel-140224105228-phpapp02/85/Cubes-pluggable-model-explained-4-320.jpg)
![Workspace
[workspace]
models_path: /var/lib/cubes/models
Model Providers
crm
sales
!
Static Model
Provider
events
[models]
crm: crm.cubesmodel
sales: sales.cubesmodel
events: events.cubesmodel
Cubes
!
sales
churn
activations
events
[datastore_bidata]
type: sql
url: postgresql://localhost/crm
!
Stores
BI Data
(Postgres)
BI Data 2
(Mongo)
[datastore_bidata2]
type: mongo
host: localhost
collection: events](https://image.slidesharecdn.com/cubes-pluggablemodel-140224105228-phpapp02/85/Cubes-pluggable-model-explained-32-320.jpg)
Uploaded byStefan Urbanek
Cubes – pluggable model explained
This document proposes a redesign of the Cubes analytical workspace to make it more pluggable and flexible. Key points of the redesign include: 1. Splitting backends into separate objects for browsers, stores, and model providers for more modular composition. 2. Allowing browsers and stores to work with different data sources and schemas within a single workspace. 3. Using an external workspace object to provide the appropriate browser and manage configuration, replacing the previous single backend concept.
More Related Content
Similar to Cubes – pluggable model explained
Cubes – pluggable model explained
- 1. data brewery Pluggable Model CubesAnalytical Workspace Redesign Stefan Urbanek – @Stiivi February 2014
- 2.
- 3. Model ■ single JSONor a model bundle ■ contains all model objects ■ full description required
- 4. one file orone directory bundle ✂ model browser formatters http backends one per serving: server workspace [workspace] backend=sql url=postgresql://localhost/database modules
- 5. Browser Aggregation Browser SQL Snowflake Browser SQL Denormalized Browser MongoDBBrowser Some HTTP Data Service Browser ? multiple backends available
- 6. Backend ■ implemented aspython module with an entry point create_workspace() ■ provides Workspace and Browser workspace represents data storage ■ only one Workspace per serving only one kind of storage per serving
- 7.
- 8. Model ■ composed ofmultiple parts ■ external model definition provided from external source, such as analytical service ■ shared dimension descriptions only one dimension description is necessary per composed model
- 9. Backend ■ heterogenous storage multipledata stores, different types of data stores ■ different schemas in same store ■ multiple environments dev, test, production, ...
- 10.
- 11. Backend ■ “backend” aremultiple objects: ! | Provider Browser ■ better plug-in system instead of Python module ■ more flexible composition Store
- 12. Backend Objects ■ Browser– performs aggregated browsing ■ Store – maintains database connection ■ Model Provider – provides model Note: not every kind has to be implemented
- 13. Logical Physical ∑ create model connect aggregate | Provider Browser Store model cubes model physical datastore (database or API) dimensions backend objects
- 14.
- 15. Browser ■ depends onthe logical model ■ implements aggregation aggregate(), values(), … ■ gets data from associated store
- 16.
- 17.
- 18.
- 19. Store * ■ provides databaseor API connection ■ might provide a model ■ slicer tool actions physical mapping validation, model from schema generation, schema from model generation, schema conversions and optimization, ... *former backend’s “Workspace” object
- 20.
- 21. Store Methods Store isnot required to implement any methods at this time. Future: ■ validate(cube) – does logical map to physical? ■ create(object) – create physical structure
- 22.
- 23. Model Provider ■ createsmodel from external source ■ might suggest store to be used
- 24.
- 25. Provider Methods ■ dimension_metadata(name,temps,locale) ■cube_metadata(name,locale) or ■ dimension(name,temps,locale) ■ cube(name,locale)
- 26. SQL Backend Mongo Backend | | SnowflakeBrowser Mongo Browser SQL Store Google Analytics Backend | Mongo Store GA Model Provider example backends GA Browser GA Store
- 27. from cubes import| AggregateBrowser, Store ! class SQLStore( Store): | default_browser_name = “sql_snowflake” ! def __init__(self, **options): # initialize the store here ! def validate_cube(self, cube): return True # if valid ! ! class | SQLSnowflakeBrowser(| Browser): def __init__(self, model, locale): # initialize the browser ! def features(self): # return list of browser features def aggregate(self, cell, ...): # return aggregation of the cell from slicer.ini
- 28. New Workspace * ■ globalobject at library level ■ provides appropriate browser ■ contains run-time configuration ■ might have state persistence *former backend Workspace is now Store
- 29. Future Workspace ■ caching ■cube composition ■ …?
- 30.
- 31. Workspace Model Providers API Model Provider StaticModel Provider Cubes sales churn activations events BI Data 2 (Mongo) Events (API) Stores BI Data (Postgres) heterogenous environment
- 32. Workspace [workspace] models_path: /var/lib/cubes/models Model Providers crm sales ! StaticModel Provider events [models] crm: crm.cubesmodel sales: sales.cubesmodel events: events.cubesmodel Cubes ! sales churn activations events [datastore_bidata] type: sql url: postgresql://localhost/crm ! Stores BI Data (Postgres) BI Data 2 (Mongo) [datastore_bidata2] type: mongo host: localhost collection: events
- 33.
- 34. Conclusion ■ heterogenous pluggableenvironment ■ externally provided models ■ easier backend implementation
- 35.