MicroStrategy
for
Data Engineers
Francesco Mucio
London, 10 June 2016
What is MicroStrategy
- Born in 1989
- BI Reporting Tool
- The last of the old one not acquired and not acquiring
- Etc...
MicroStrategy Architecture
dwh MicroStrategy
iServer
web server
with MSTR webapp
user
MicroStrategy
metadata
What MicroStrategy do wants!
- A Dimensional model (snowflake flavour)
- Dummy Records for Null values
- Good Data Quality
Star vs Snowflake schema
fact
dimension
2
dimension
4
dimension
5
dimension
3
dimension
1
fact
dimension
2 - lvl1
dimension
4
dimension
5 - lvl1
dimension
3
dimension
1 - lvl2
dimension
1 - lvl1
dimension
5 - lvlb
dimension
5 - lvl2a
dimension
2 - lvl2
dimension
2 - lvl3
Star vs Snowflake schema
Star Schema Dimension Snowflake Schema Dimension
date_id day_of_week month_id month_name quarter_id
20160101 Friday 201601 January 20161
20160102 Saturday 201601 January 20161
20160103 Sunday 201601 January 20161
date_id day_of_week month_id
20160101 Friday 201601
20160102 Saturday 201601
20160103 Sunday 201601
month_id month_name quarter_id
201601 January 20161
201601 January 20161
201601 January 20161
quarter_id quarter_name
20161 2016-Q1
20161 2016-Q1
20161 2016-Q1
DIM_CALENDAR
DIM_DATE
DIM_MONTH
DIM_QUARTER
MicroStrategy Objects
➔ Schema Objects
◆ mapping of physical objects: Tables, Columns, etc…
➔ Public Objects
◆ application objects: Metrics, Reports, Dashboards, etc…
MicroStrategy Schema - Tables
A MicroStrategy Tables maps a
database table or view.
Multiple physical tables can be
mapped as partition of a
MicroStrategy table.
MicroStrategy Schema - Logical Tables
A Logical Table allows to use a
query as a table.
In the definition you must:
- Set the database instance to use
- Map all the columns returned by
you query
MicroStrategy Schema - Attributes
- A level of a Dimension.
- Needs a lookup table.
- Can have parents and children.
- Has multiple forms.
- Each form can have multiple
expressions.
MicroStrategy Schema - Facts
A numeric* value that needs to be
measured by the business.
Can have multiple expressions.
Doesn’t know how it will be used.
Can be extended, if required.
* usually
MicroStrategy Schema - Facts
A numeric* value that needs to be
measured by the business.
Can have multiple expressions.
Doesn’t know how it will be used.
Can be extended, if required.
* usually
MicroStrategy Schema - Advanced topics
Expressions can contain intra-row operations.
Expressions can contain 1-1 functions.
Attributes determine the Logical Weight of a table.
Transformations defines ways to transform an attribute
element to one or more elements of the same attribute:
i.e. Previous Year or Year-to-date.
MicroStrategy Metrics
A Fact defines which column of
which table we want to use.
A Metric defines what, how,
where, when to do with that
fact/column.
MicroStrategy Metrics
How to aggregate?
The Formula defines the kind
of aggregation we want to use.
Metrics can be compound of
other metrics:
Profit = Revenue - Cost
MicroStrategy Metrics
Where to aggregate?
The Level allows to fine tune
the aggregation, depending on
attributes/dimension.
Filtering: ignore or not a
where condition?
Grouping: aggregate using the
standard grouping or not.
MicroStrategy Metrics
When to aggregate?
Condition: a where condition.
This can become a CASE statement or a
separated sub-query.
Transformation: a rule to
transform an attribute value
MicroStrategy Reports
Reports or Datasets are glorified
SQL queries.
Components:
- Report Objects
- Filter
- Template
MicroStrategy Reports
OLAP capabilities:
- Moving objects in template
- Moving objects between Report
object and template
- Modify the View Filters
Non-OLAP:
- Add new objects
- Change the Report Filter
MicroStrategy Reports: SQL
...
Pass0 - Query Pass Start Time: 09/06/2016 23:22:50
Query Pass End Time: 09/06/2016 23:22:52
Query Execution: 0:00:01.78
Data Fetching and Processing: 0:00:00.00
Data Transfer from Datasource(s): 0:00:00.00
Other Processing: 0:00:00.02
Rows selected: 131
with gopa1 as
(select /*Administrator - job 1049 - New Report - 20160609:232250*/ a13.PROFILE_GENDER_ID
PROFILE_GENDER_ID,
a12.ACQUISITION_BRAND ACQUISITION_BRAND,
a11.PROFILE_REGISTERED_DT Reporting_Date_DT,
count(distinct a11.PROFILE_USER_ID) n_users
from MICRO.MS_F_PROFILE_REGISTRATION a11
...
For the full SQL code click here.
MicroStrategy Intelligent Cubes
In memory copy of a query result.
Can be also partially re-published.
A report based on an Intelligent Cubes:
- has access only to that cube objects
- generates an MDX query
Dynamic Sourcing allows also not cube-based reports to use
available cubes.
MicroStrategy Reports - Free Form SQL
A FFSQL is a report created
starting from an existing query.
Very good for prototyping
Very bad for maintenance
You can also have FFSQL cubes.
MicroStrategy Reports - Free Form SQL
Things to define:
- Database Instance
- Query
- Free form objects
MicroStrategy Filters
Filters are like where
conditions.
Multiple filters can be
combined, using AND,
OR, and NOT, to make
complex ones.
MicroStrategy Filters: Advanced
Reports can be used as
filters.
Set Qualification are
used to filter
attributes based on
elements not always
present in the final
report.
MicroStrategy Filters: Advanced
gopa1 as
(select /*Administrator - job 2158 - All TYPE REG Active with Revenue - 20160610:132345*/ a11.PROFILE_USER_ID PROFILE_USER_ID
from MICRO.MS_F_PROFILE_REGISTRATION a11
group by a11.PROFILE_USER_ID
having sum(a11.PROFILE_PHOTOS_TOTAL_COUNT) >= 3.0
),
gopa9 as
(select /*Administrator - job 2158 - All TYPE REG Active with Revenue - 20160610:132345*/ a15.REGISTRATION_METHOD REGISTRATION_METHOD,
a13.PROFILE_REGISTERED_DT first_click_date,
a14.ACQUISITION_BRAND ACQUISITION_BRAND,
a14.ACQUISITION_TRANSLATION ACQUISITION_TRANSLATION,
a13.PROFILE_GENDER_ID PROFILE_GENDER_ID,
count(distinct a11.PROFILE_USER_ID) WJXBFS1
from INGRES.MS_DIM_PROFILE a11
join gopa1 pa12
on (a11.PROFILE_USER_ID = pa12.PROFILE_USER_ID)
join INGRES.MS_DIM_PROFILE a13
on (a11.PROFILE_USER_ID = a13.PROFILE_USER_ID)
join ingres.DIM_ACQUISITION_PLATFORM a14
on (a13.PROFILE_ACQUISITION_PLATFORM = a14.ACQUISITION_PLATFORM)
join ingres.MS_DIM_PROFILE_EXTRA a15
on (a11.PROFILE_USER_ID = a15.PROFILE_USER_ID)
where (a13.PROFILE_REGISTERED_DT >= DATE '2016-01-01'
and a13.PROFILE_IS_TEST_USER_ID = 0
and a14.ACQUISITION_TRANSLATION in ('Android', 'iOS', 'Webapp', 'Windows')
and a14.ACQUISITION_BRAND in ('Badoo'))
group by a15.REGISTRATION_METHOD,
a13.PROFILE_REGISTERED_DT,
a14.ACQUISITION_BRAND,
a14.ACQUISITION_TRANSLATION,
a13.PROFILE_GENDER_ID
)
MicroStrategy Dashboards
A more modern tool for data exploration.
- Simpler to develop and use
- Slick design
- Just nicer
MicroStrategy Dashboards
MicroStrategy Dashboards
A dashboard has at least one tab.
A tab has one or more panels.
A panel contains visualizations.
Each visualization is based on one
or more datasets.
MicroStrategy Dashboards
A dashboard has at least one tab.
A tab has one or more panels.
A panel contains visualizations.
Each visualization is based on one
or more datasets.
MicroStrategy Debugging
A simple report sql .
[originally here there was a link to a SQL report were I was discussing the
different sections, results and how to interpreter the SQL, with common
table expressions, generated by MicroStrategy]

Microstrategy for Data Engineers

  • 1.
  • 2.
    What is MicroStrategy -Born in 1989 - BI Reporting Tool - The last of the old one not acquired and not acquiring - Etc...
  • 3.
    MicroStrategy Architecture dwh MicroStrategy iServer webserver with MSTR webapp user MicroStrategy metadata
  • 4.
    What MicroStrategy dowants! - A Dimensional model (snowflake flavour) - Dummy Records for Null values - Good Data Quality
  • 5.
    Star vs Snowflakeschema fact dimension 2 dimension 4 dimension 5 dimension 3 dimension 1 fact dimension 2 - lvl1 dimension 4 dimension 5 - lvl1 dimension 3 dimension 1 - lvl2 dimension 1 - lvl1 dimension 5 - lvlb dimension 5 - lvl2a dimension 2 - lvl2 dimension 2 - lvl3
  • 6.
    Star vs Snowflakeschema Star Schema Dimension Snowflake Schema Dimension date_id day_of_week month_id month_name quarter_id 20160101 Friday 201601 January 20161 20160102 Saturday 201601 January 20161 20160103 Sunday 201601 January 20161 date_id day_of_week month_id 20160101 Friday 201601 20160102 Saturday 201601 20160103 Sunday 201601 month_id month_name quarter_id 201601 January 20161 201601 January 20161 201601 January 20161 quarter_id quarter_name 20161 2016-Q1 20161 2016-Q1 20161 2016-Q1 DIM_CALENDAR DIM_DATE DIM_MONTH DIM_QUARTER
  • 7.
    MicroStrategy Objects ➔ SchemaObjects ◆ mapping of physical objects: Tables, Columns, etc… ➔ Public Objects ◆ application objects: Metrics, Reports, Dashboards, etc…
  • 8.
    MicroStrategy Schema -Tables A MicroStrategy Tables maps a database table or view. Multiple physical tables can be mapped as partition of a MicroStrategy table.
  • 9.
    MicroStrategy Schema -Logical Tables A Logical Table allows to use a query as a table. In the definition you must: - Set the database instance to use - Map all the columns returned by you query
  • 10.
    MicroStrategy Schema -Attributes - A level of a Dimension. - Needs a lookup table. - Can have parents and children. - Has multiple forms. - Each form can have multiple expressions.
  • 11.
    MicroStrategy Schema -Facts A numeric* value that needs to be measured by the business. Can have multiple expressions. Doesn’t know how it will be used. Can be extended, if required. * usually
  • 12.
    MicroStrategy Schema -Facts A numeric* value that needs to be measured by the business. Can have multiple expressions. Doesn’t know how it will be used. Can be extended, if required. * usually
  • 13.
    MicroStrategy Schema -Advanced topics Expressions can contain intra-row operations. Expressions can contain 1-1 functions. Attributes determine the Logical Weight of a table. Transformations defines ways to transform an attribute element to one or more elements of the same attribute: i.e. Previous Year or Year-to-date.
  • 14.
    MicroStrategy Metrics A Factdefines which column of which table we want to use. A Metric defines what, how, where, when to do with that fact/column.
  • 15.
    MicroStrategy Metrics How toaggregate? The Formula defines the kind of aggregation we want to use. Metrics can be compound of other metrics: Profit = Revenue - Cost
  • 16.
    MicroStrategy Metrics Where toaggregate? The Level allows to fine tune the aggregation, depending on attributes/dimension. Filtering: ignore or not a where condition? Grouping: aggregate using the standard grouping or not.
  • 17.
    MicroStrategy Metrics When toaggregate? Condition: a where condition. This can become a CASE statement or a separated sub-query. Transformation: a rule to transform an attribute value
  • 18.
    MicroStrategy Reports Reports orDatasets are glorified SQL queries. Components: - Report Objects - Filter - Template
  • 19.
    MicroStrategy Reports OLAP capabilities: -Moving objects in template - Moving objects between Report object and template - Modify the View Filters Non-OLAP: - Add new objects - Change the Report Filter
  • 20.
    MicroStrategy Reports: SQL ... Pass0- Query Pass Start Time: 09/06/2016 23:22:50 Query Pass End Time: 09/06/2016 23:22:52 Query Execution: 0:00:01.78 Data Fetching and Processing: 0:00:00.00 Data Transfer from Datasource(s): 0:00:00.00 Other Processing: 0:00:00.02 Rows selected: 131 with gopa1 as (select /*Administrator - job 1049 - New Report - 20160609:232250*/ a13.PROFILE_GENDER_ID PROFILE_GENDER_ID, a12.ACQUISITION_BRAND ACQUISITION_BRAND, a11.PROFILE_REGISTERED_DT Reporting_Date_DT, count(distinct a11.PROFILE_USER_ID) n_users from MICRO.MS_F_PROFILE_REGISTRATION a11 ... For the full SQL code click here.
  • 21.
    MicroStrategy Intelligent Cubes Inmemory copy of a query result. Can be also partially re-published. A report based on an Intelligent Cubes: - has access only to that cube objects - generates an MDX query Dynamic Sourcing allows also not cube-based reports to use available cubes.
  • 22.
    MicroStrategy Reports -Free Form SQL A FFSQL is a report created starting from an existing query. Very good for prototyping Very bad for maintenance You can also have FFSQL cubes.
  • 23.
    MicroStrategy Reports -Free Form SQL Things to define: - Database Instance - Query - Free form objects
  • 24.
    MicroStrategy Filters Filters arelike where conditions. Multiple filters can be combined, using AND, OR, and NOT, to make complex ones.
  • 25.
    MicroStrategy Filters: Advanced Reportscan be used as filters. Set Qualification are used to filter attributes based on elements not always present in the final report.
  • 26.
    MicroStrategy Filters: Advanced gopa1as (select /*Administrator - job 2158 - All TYPE REG Active with Revenue - 20160610:132345*/ a11.PROFILE_USER_ID PROFILE_USER_ID from MICRO.MS_F_PROFILE_REGISTRATION a11 group by a11.PROFILE_USER_ID having sum(a11.PROFILE_PHOTOS_TOTAL_COUNT) >= 3.0 ), gopa9 as (select /*Administrator - job 2158 - All TYPE REG Active with Revenue - 20160610:132345*/ a15.REGISTRATION_METHOD REGISTRATION_METHOD, a13.PROFILE_REGISTERED_DT first_click_date, a14.ACQUISITION_BRAND ACQUISITION_BRAND, a14.ACQUISITION_TRANSLATION ACQUISITION_TRANSLATION, a13.PROFILE_GENDER_ID PROFILE_GENDER_ID, count(distinct a11.PROFILE_USER_ID) WJXBFS1 from INGRES.MS_DIM_PROFILE a11 join gopa1 pa12 on (a11.PROFILE_USER_ID = pa12.PROFILE_USER_ID) join INGRES.MS_DIM_PROFILE a13 on (a11.PROFILE_USER_ID = a13.PROFILE_USER_ID) join ingres.DIM_ACQUISITION_PLATFORM a14 on (a13.PROFILE_ACQUISITION_PLATFORM = a14.ACQUISITION_PLATFORM) join ingres.MS_DIM_PROFILE_EXTRA a15 on (a11.PROFILE_USER_ID = a15.PROFILE_USER_ID) where (a13.PROFILE_REGISTERED_DT >= DATE '2016-01-01' and a13.PROFILE_IS_TEST_USER_ID = 0 and a14.ACQUISITION_TRANSLATION in ('Android', 'iOS', 'Webapp', 'Windows') and a14.ACQUISITION_BRAND in ('Badoo')) group by a15.REGISTRATION_METHOD, a13.PROFILE_REGISTERED_DT, a14.ACQUISITION_BRAND, a14.ACQUISITION_TRANSLATION, a13.PROFILE_GENDER_ID )
  • 27.
    MicroStrategy Dashboards A moremodern tool for data exploration. - Simpler to develop and use - Slick design - Just nicer
  • 28.
  • 29.
    MicroStrategy Dashboards A dashboardhas at least one tab. A tab has one or more panels. A panel contains visualizations. Each visualization is based on one or more datasets.
  • 30.
    MicroStrategy Dashboards A dashboardhas at least one tab. A tab has one or more panels. A panel contains visualizations. Each visualization is based on one or more datasets.
  • 31.
    MicroStrategy Debugging A simplereport sql . [originally here there was a link to a SQL report were I was discussing the different sections, results and how to interpreter the SQL, with common table expressions, generated by MicroStrategy]