Case Study : MVNO MVNE Data Warehouse Transatel (2008 version)

Case study : MVNO / MVNE Data Warehouse Seminar for EFREI engineering school Nov 2008 Mathieu DESPRIEE – BI Manager [email_address] www.transatel.com

Agenda
Part 1 – MVNO / MVNE / Transatel
Part 2 – Data-warehouse & Business Intelligence :
Emergence of the need
Part 3 – Business Intelligence :
Tools & concepts
Part 4 – Project organization

Part 1 MVNO / MVNE / Transatel

Introduction / Definitions
MVNO : Mobile Virtual Network Operator
An MVNO is a Service Provider offering a mobile service under its own brand without having a mobile radio network
GSM operators are referred to as MNO : Mobile Network Operator
An MNO has a license on radio frequency ranges (GSM, UMTS…), and all the radio infrastructure
MVNE : MVNO-Enabler
An MVNE is a service company delivering tools & services to companies wishing to market their services over a mobile network, and thus becoming MVNOs

MVNO and MVNE responsibilities Radio Network Core Network OSS BSS Customer Service Life-cycle Management Marketing Distribution
GSM
GPRS
UMTS…
HLR
SMS-C
MSC
GGSN
Interco …
Service activation
CDR collect
CRM
Billing
Processes
Operations
Call-centre
Support…
Brand
Offer definition
Communicat°
Marketing mix
Sales
Distribution network animation
MVNO responsibility MNO responsibility MVNE capabilities

Transatel Company introduction

Transatel
Transatel is an European MVNE (Mobile Virtual Network Enabler)
Founded in August 2000
Backed with institutional funds such as Natexis and Credit Agricole
80+ employees
HQ in France with subsidiaries in UK, Belgium, Netherlands and Luxembourg
Serving MVNOs & GSM operators
Established brands and businesses: turn key solution to become MVNO
GSM operators: infrastructure & services to easily integrate new MVNO and/or build specific co-branded offer
A innovative approach to mobile service differentiation and USP
Team & Infrastructure
Dedicated team with know-how (business, regulatory, technical dev & op.)
IT & Telecom platforms
Several MVNO launches in several countries
Over 200k managed subscribers

Transatel skills & operational capabilities
Telecom Business Plan set-up: Consulting
Mobile offer definition & positioning, business plan
Interco model / cost structure / pricing
Regulatory issues
Relationship with GSM operators (MNO):
Airtime procurement in selected European countries
Support in negotiation
Mobile Service management: Development and Operations
Order entry / Activation / Back office / Customer Management / Call centre
CDR mediation / Real-time IN control / Billing (post-paid & prepaid)
Fraud detection / Money collection / Bad debt Management
Value added services: Development & Operations (incl. « live » prototyping)
SIM cards management, personalisation and procurement
Voicemail, Unified Messaging, IVR, Follow-me
WISP over GPRS
Fixed / mobile convergence – Voice / data convergence – GSM / Wifi (VoIP convergence)
M2M secured connectivity
Pan-European footprint with centralised & mutualised platform
Turn key solution for MVNOs in Europe from Marketing & Regulatory support to Technical Implementation & Operations

Service portfolio & Operations capabilities Customer Acquisition & Order Entry Provisioning (activations, portability) Billing Customer Care & CRM Value added services SIM cards Order-entry tool web enabled for end-users, distributors Prov. & workflow MNO, service platforms, billing system CRM system web enabled for call-centre & self care Prepaid reload management Prepaid engine IN based Post-perso process for small batches SIM profile definition for large batches SIM mngt & OTA server SIM applets Customisation & branding, call mngt, security Switch & Service node UMS, follow-me, call termination, SMS-GW WISP APN for user authentication & mobile portal mngt + WAP gtway + MMS + remote handset configuration HLR / GMSC (roadmap) Marketing / Business / Regulatory support (offer definition, economic modelling, business processes, regulatory requirements) Logistics (3 rd party) Call centre (3 rd party) Post-paid, invoicing, balance management Payment Collection Revenue Assurance

Transatel interconnection with GSM operators

Transatel MVNE operating models MVNO TRANSATEL MNO End-User MNO End-User MVNO MVNE - ASP E X E M P L E Aggregator TRANSATEL

Vision of a possible efficient market structure Mass market MVNOs Segmented / niche MVNOs Ethnic Sports SME Switchless resellers MVNA ( Aggregator ) M2M Media Distributor Fixed telco MVNE MVNE MVNE MVNO mass market MNO MVNO mass market MVNO niche MVNO niche MVNO niche MVNO niche MVNO niche MVNO niche MVNO niche MVNO niche MVNO niche MVNO niche

EasyBorder : Transatel offer for frequent travellers & cross- border population (FR, BE, NL, LU)
Dedicated offer for high roamers
30% to 100% price reduction on roaming
Better continuity & dedicated services
Commercialised in FR , BE , NL , LUX
Niche marketing
Direct marketing
Flyers & billboards in int’l train stations
Advertising in targeted newspapers & websites
Active referral program
Partnership with frequent traveller programs
Sales: telesales & web
www.transatel.com

Eurokeitai (post-paid) – France www.eurokeitai.com

Transatel international reach Roam Triple Wireless

IT systems to support CRM & billing Billing Customer Management Provisioning CRM (sales & mkt) Order-entry / Customer service Real-time rating IN call-control CDR mediation Prepaid top-up mngmt Post-paid mngmt & invoicing WWW IVR SMS PDF email WWW print-shop Bank interface Reporting & data w/h Customers Distributors MVNO back-office IT interface INAP / CAMEL CDR files from 3 rd parties GSM network Other P/F & systems MNO interface IN P/F Service P/F 3 rd parties, etc. Web Front End (O/E & C/S) API Account Status & Network features Trouble ticketing Subscribers & Catalogue Resources (SIM & MSISDN)  Accounting reports

Transatel Service infrastructure

Part 2 Data-warehouse & Business Intelligence : Emergence of the need

Reporting needs
Information is one of the biggest asset of the enterprise. Business users NEED to analyze that data.
Finances :
Analysis of revenues, costs, profitability, customer/supplier accounting, human resources, financial audits…
Revenue assurance
Analysis of costs and revenues, detection of revenue leaks
Fraud detection
Sales & marketing :
Sales analysis, orders evolution, customer bills, CRM reports, marketing channels efficiency, offers appropriateness, calls analysis (destination, duration, distribution along time)
CRM & Customer support :
Statistics on incidents, delay of incident resolution, hotline statistics…
Production
SLA control. Ex : 99% of billing data processed in 24h
CEO, Board
Overall performance dashboards, KPI = Key Performance Indicators…

Reporting needs are …
Repetitive
Finances and marketing will share lots of requirements, but expressed differently
Subtly different
Finances would like a report starting on Jan, 1 st .
Marketing would like the same report, starting on Dec, 10 th (Christmas offer…)
Complex
Marketing would like to know the revenues generated by an offer (billing data), but with differentiation of the different acquisition channels (CRM data)
Expressed by business users
Business vocabulary : they won’t use the name of table and columns !
They don’t care about the naming conventions in data-models
They don’t care about technical constraints

First approach
Custom SQL queries
Queries against the different databases of the different applications
Use Excel sheets (with embedded queries) to federate data, and produce charts
Use existing applications to deduce stats (from displayed grids in a search screen for instance…)

Enterprise IT « in real life »
Many systems
Many data sources
Many different data models
Different references
Duplicates
 Problems in data consolidation

Complexity
Queries are complex
Samples :
mvno_activation_monitoring.sql
mvno_dump_accounts.sql
mvno_financial_report.sql
process_monitoring.sql
This kind of queries :
Use many tables, sometimes from several databases, sometimes from several systems
Needs many indexes
Produce large result sets
Are long to execute

Volumes
1 MVNO = 10,000 to 100,000 subscribers
Around 200 CDRs (call data records) per subscriber per month
(Up to 800 CDRs per month for some specific offers with a lot of data traffic)
100,000 subs * 200 CDRs * 12 months = 240 M CDRs / year
800 bytes of storage per CDR (including indexes) =~ 190 GB of storage per MVNO per year
 doing a SELECT query over the whole data of the year (for instance, sum of all traffic) would require a lot of time of execution, and a lot of resources (disk, CPU) on production server
1 MVNO = 5 tariff-plans = 10,000 rates implemented in billing
Call destination analysis tables (numbering plans) = 600,000 destinations in the world

No reactivity to change
One report = one query  catastrophic !!
Each set of reports evolution needs a project organization :
specification by business,
technical specification (complexity to interpret business requirements)
testing,
move to production
Even for minor changes !!

OLTP models are not adapted
OLTP is the common modeling for business applications
transaction-oriented
data is highly normalized (cf. 3 normal forms)
OLTP models are not designed for large queries
Performance impact, database locking
OLTP models are not designed to maintain history
Often, OLTP systems store a snapshot of reality. For instance, they store the current status of some entity, but not all the history of statuses for that entity and the dates of each change.
Keeping all activity for a long time in OLTP systems would have an important impact :
Difficult to administrate indexes
Performance impact
Time to backup & restore database (because systems fail one day or another)
OLTP models are not adapted to reporting
Data normalization decompose data into many tables and joins, resulting in complex queries

The result :
90% of time spent in developing and producing the reports, 10% spent to analyze them
Use of user-interfaces not designed for reporting
Too few people know how to produce the key reports
Performance impact on production systems
Reports are incomplete, no sufficient history
“ I have data everywhere, and sometimes I forgot some data-source in my reports”
“ I get data from many different systems to fill-in my Excel sheet”
“ I’m not totally confident in my reports”
“ I make a lot of hypotheses in my simulation, but I cannot verify them”

All these problems = the field of BI
Difficulties in data acquisition
Need dedicated tools to collect data from different systems, with cleaning and consolidation capabilities
Difficulties in data storage with OLTP on production systems
Need different data-models to store history
Need different databases to store history
Need different engine to do the analysis and calculations
Difficulties in data restitution with SQL
Need tools usable by business users
BI = Business Intelligence = Field of technologies that facilitate acquisition, analysis and restitution of data to help in taking decisions

Part 3 Business Intelligence : Tools & concepts

Business Intelligence tools & concepts Production systems Other sources CRM Billing Activation systems Other DBs… Files ETL DataWarehouse OLAP cubes Business Users Querying & Reporting tools Acquisition Storage / Archiving Restitution Datamarts

Business Intelligence tools & concepts (2)
ETL
Extract : from various production data stores
Transform : transform data to match target data-model, merge the different sources
Load : in a target database (the data-warehouse)
Data Warehouse
Repository of the whole enterprise’s historical data
Subject oriented : business data
Time variant : every change is dated
Non-volatile : read-only, never deleted nor changed once committed
Integrated : related data from different production systems are gathered
Datamart
A small repository dedicated to a business subject.
Example : one datamart dedicated to cost &revenues analysis, another dedicated to traffic analysis.
In the case of Transatel, our data-warehouse is simply constituted of a set of datamarts.
OLAP cubes : explained next slide…
Reporting tools
Querying tools that can be used by business users, non computer specialists (such as excel !)
Reports, dashboards

OLAP & Datamarts

Datamart creation : the OLAP approach
Reorganize data in an hypercube
N-dimensional cubes
Facts : the data to be aggregated
Data quantifying the business activity (facts are also called “measures” in microsoft tools)
Price, revenue, cost, duration of calls, size of data packets, currencies exchange rates, number of visits on web-site, number of incidents…
Aggregation operations = sum, average, min, max…
Dimensions : the information structuring the facts
Time, time, time !
Customers, offers, currencies, countries, type of charge, type of call (SMS, voice, data), destinations of calls, marketing channels, …
Hierarchies :
Time : Year > Quarter > Month > Day
Geography : Zone > Country > Region > City
Organization : Business Unit > Line of business > Market
Time Geography Offers Cells contains facts data

OLAP : On-Line Analytical Processing
Cubes are the way OLAP technology organizes summary data into multidimensional structures.
Dimensions and their hierarchical levels reflect the queries that can be asked to the cube.
Aggregations are precalculated and stored in the multidimensional structure in cells at coordinates specified by the dimensions.
Sophisticated algorithms in modern OLAP engines minimize storage
Calculating all possible aggregations would require substantial processing time and storage, which is unnecessary. As a result, there is a tradeoff between storage requirements and the percentage of possible aggregations that are precalculated.
Different OLAP storage modes for different needs :
MOLAP : multidimensional OLAP : fastest calculation, larger storage, not updated real-time
ROLAP : relational OLAP : smaller storage overhead, slowest calculation, real-time
HOLAP : hybrid mode
OLAP engines use specific querying language (MDX)
OLAP client tools generate MDX queries
MDX can also be written by developers for advanced querying

OK, I need a cube, where do I start ?
The star data-model (or snowflake)
Facts Dimension Dimension Dimension Dimension

Dimensions hierarchies
Example of hierarchy for a time dimension
The cube is configured so that the OLAP engine knows the hierarchy structure, and can precalculate aggregates at different dimension levels

Real world example 1
Cube dedicated to revenue analysis from post-paid billing
Advanced point :
Transatel billing produces bills for customers in several countries, and in several currencies.
Currency is a special point of interest, because we can’t sum prices in different currencies.
Currency conversion rate variations along time are stored in a dedicated fact table
A “reporting currency” dimension has been created
The business user chooses in which currency he wants the aggregates
Cube has been configured with advanced calculation rules to help in correct processing

Real world example 2 : cube for billed charges
The datamodel (snowflake)

The cube structure

The cube structure Main Facts (charges) Measures : charge, sum minutes, sum SMS, sum MBytes …

The cube structure Measure : Subscriber count

The cube structure Measure : {subscriber x month} counting an artificial measure to ease some computations conceptually it’s a crossjoin, in this case, it’s better to create it from the datamodel, with a dedicated SQL query

The cube structure Measure : the currency exchange rate .. to display the data in €, even if the bill is in another currency (conversion is done by an advanced MDX script)

The cube structure Dimensions the Date dimension is used several times (role-playing dimension)

The cube structure Relationships a « regular » relationship ie : a BilledCharge has a DateKey for the BillingDate dimension

The cube structure Relationships a « fact » relationship ie : the same table (subscriber) is used as a dimension (organizing attributes) and as a fact (for counting)

The cube structure Relationships a « referenced » relationship ie : the Country dimension is not directly related to facts. We have to go through another dimension (customer) to get it

The cube structure Relationships a « many-to-many » relationship ie : subscriber measures (count) are not related to products. We have to go through the billed charges to know that. This is a many-to-many join : Dim Product  Fact Charges Fact Charges  Dim Subscriber Dim Subscriber  Fact Subscriber

ETL
Extract
Multi-source
Batch oriented : usually ETL batches are ran when load is low on production systems (by night)
Connectivity to different technologies, to various DBRMS implementations
Transform
Manipulate complex data
Merge from different sources
Implement business rules.
Cleanup, sorting, deduplication…
Optimized for performance and large volumes
Load
Load data in fact tables and in dimension tables
Produce logs to files or database for traceability
Others
Controls execution flows : manage dependencies between tasks, control execution, manage errors, logs

Examples of transformation available in SSIS
Fuzzy Lookup Transformation
Look up values in a reference table using a fuzzy match
Example : the field “how did you heard about Transatel” is a free-text field on the website. The data contains a lot of variants for same words (“friend”, “a friend”, “freind”, etc). The fuzzy logic can help in merging all the variants
Slowly Changing Dimension Transformation
Help in managing a the update of a slowly changing dimension
Example : Information about the customers change continuously, but we want to keep some history.
A change in customer address  we don’t keep the old value (override)
A change in company VAT Number  we ‘duplicate’ the customer entry, to keep old financial data aligned with the old VAT Number (for annual reports)
This is called a surrogate key

ETL : control the execution !
Data is very sensitive, quality of reports rely on the quality of data
Loading a datawarehouse can imply very complicated rules on extraction, transfer and transformations
Business users won’t trust a datawarehouse, if they’re not confident on how it has been loaded.
So, one of the key aspects in ETL process implementation is the control of how runs the data transfer and transformation
ETL solutions come with a lot of tools to control process execution, and with alerting systems (eg: mail sent to the DBA) to raise the error.
Microsoft SSIS organize ETL packages in control-flow and data-flows.
Cf next slide

ETL package example in SSIS
Control flow and data flow

Restitution & Reporting

Reports topology Dashboards Adhoc reports Mass reporting executives operational teams
KPIs, trends
5% of users
data and trends analysis
need of interactivity
need of drill-down, drill-through
20% of users
static reports designed for specific purposes
75% of users

Reporting : OLAP cube browsing
Users can browse data through pivot charts
Colums and rows are taken amongst available dimensions
Content of pivot-chart is measures (attributes of facts)
Excel is a good low-end tool for cube browsing
Demo

Reporting : Control access to data
Security issues :
Security of access and segregation of data is important : anybody may not access to any data.
Especially in the case where reports are produced for external partners / customers : they can connect to a secured web-site and download reports they’re interested in.
Some business data may also be confidential, and destined to top-executives only.
Reporting tools and OLAP engines provide the ability to define roles and permission on data, at every level (dimension, attribute of dimension, cell, etc.)
Simplification for business users :
Cubes may contains a lot of dimensions, dimension attributes, and facts. In some tools, it’s possible to define a subset of a cube, and restrain some users to access only at this part of data (“perspectives” in microsoft tools)

Part 4 Project organization

Collect the business needs
The best way is to start from existing reports, and existing excel sheets frequently used by managers & executives
Organize reporting needs by functional domain, and if possible by data-source
Define at a macro level the model you need
Determine dimensions
Determine facts
Look at data access permissions

What users want

What they specify
“ Put all information you have, we will select things afterwards”

What you’ll manage to do

What they really need ( thanks to Yves Cointrelle for the joke idea)

Select requirements and organize phases
Talk with business users, again, again and again !
Organize requirements in a matrix of choice :
Eliminate complex and non-urgent needs, and organize project phases
complexity business priority P1 P2 P3

Define granularity of data
A radar diagram can help you to :
Identify the needed dimensions and their hierarchies
Decide the level of granularity you need to keep in your datamarts

Learnings from experience…
Dimension design : beware of cardinality !
Example : Time dimension
First design : one table for all granularities, from year to second
365*24*3600 = up to 31’536’000 rows by year !!!
This design was catastrophic on performance of everything : loading, pre-processing, querying, custom calculations (advanced MDX)
In fact, bad analysis of business need
Day granularity was enough for most of time dimension usage (date of bill-cycle, date of call, date of subscriber creation, etc.) = 365 rows per year
Second granularity was useful in a unique case : time of call placement, only 86400 rows and rarely used.
2 different time dimensions : a “Date” dimension (used a lot), and a “Time” dimension (used once in the model)
Result : Pre-processing time improved by a factor of 20…

Learnings from experience…
ETL flows design
6 first months of loading of the datawarehouse after each bill-cycle : the ETL flows NEVER finished successfully !
There are always little problems or unexpected inconsistencies in data
Restarting a whole ETL process is long and costly for hardware resources
Don’t hesitate to separate the 3 parts of E-T-L
Design your flows so you can restart a phase without restarting everything, using intermediary storage if necessary

Datamart development methodology
Be AGILE !
Estimate the quality of data
Estimate constraints on data (ETL rules)
Design core part of the data model
Show to the users what is possible
Help users to define what they need
Do a prototype if necessary (with manually loaded data)
Implement ETL flows, rules, and data clean-up
Implement the data model and the cubes
Produce the first reports
Control the results with pre-existing reports
Discuss the first results and problems with users
Adapt and improve ETL rules and flows
Improve cube model to adapt to users comments
Add the advanced computed indicators (MDX scripts)
Refine the aggregation designs to optimize cube performance from real usage statistics

Project methodology
Start small !!
Build progressively, use iterative approach with frequent delivery to production
Benefits : give a quick feedback to business users
Validate the technical architecture
Organize a weekly project committee
Involve final users
Get trained
Developers should be trained specifically on the chosen BI tools
Business users have to be trained to use the new tools (and avoid a reject)
Buy consulting from specialists of necessary

Issues not to under-estimate
It can take some time to stabilize the flows and the rules in the ETL
With some applications, data is never totally clean.
You may need additional rules or even some manual processing from time to time
You may spend some time in operations with the first versions of your cubes
To adapt the aggregation configuration from the real usage statistics
To adapt the indicators you compute to what the users want
You need to spend time in the modeling phase
With a bad model, cube may be technically impossible to optimize. You can even crash you server with a bad query
With a bad model, you can obtain false calculations
MDX is a very difficult language
Business users will become greedy of your reports !
One day behind schedule to produce a sensitive report, and all the management will be on your back !!
One false statistic about the business, and everybody will hate you … Do a lot of testing to stay confident !

Some vendors of BI software…
ETL
Informatica
IBM
SAS
Cognos
DataMirror
Microsoft SSIS
Sunopsis
Business Objects
Oracle
Open Source : Talend, Enhydra
Storage
Generalists : Oracle, Microsoft SQL Server
Specialized :
Red brick
Teradata
Sybase IQ
SAS
Netazza
Datallegro
Open Source : MySQL
OLAP
Microsoft Analysis Services
Hyperion
Oracle
SAP BW (dedicated to SAP solutions)
SAS
Reporting
Business Object / Crystal Report
Cognos
Hummingbird
Hyperion Solutions
Microsoft reporting services
Information Builders
Oracle
SAS
Informatica
MicroStrategy
Actuate
Open Source : JasperReports, Pentaho

Bibliography
Yves Cointrelle
BI training session – Homsys Group
Miscrosoft MSDN for Analysis Services
http://msdn2.microsoft.com/en-us/sql/Aa336310.aspx
Wikipedia

Thank you ! electronic version www.slideshare.net/mathieu.despriee feel free to contact me for any question [email_address]

Jobs and Internships at Transatel

Case Study : MVNO MVNE Data Warehouse Transatel (2008 version)

by mathieu.despriee on Nov 03, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 48

Statistics

Case Study : MVNO MVNE Data Warehouse Transatel (2008 version) — Presentation Transcript

http://www.slideshare.net	47
http://www.fachak.com	1