This document discusses IBM's development of a first-of-a-kind big data solution for telecommunications companies. It describes IBM Research activities related to advanced analytics using telco data. The solution includes an advanced analytics platform that derives insights from telco data through predictive analytics, behavioral analysis, and other techniques. It then discusses two use cases: using aggregated anonymous location data from telcos for city-scale transit optimization, and developing enriched consumer profiles through individual-level mobility analytics.

1. © 2013 IBM Corporation
A Deep Dive into “First of A Kind” Big Data Telco
Solution
Session #: ISA-3638
Sambit Sahu, Arvind Sathi, Tommy Eunice, Mathews Thomas
Ken Kralick
IBM

2. Please note
IBM’s statements regarding its plans, directions, and intent are subject to
change or withdrawal without notice at IBM’s sole discretion.
Information regarding potential future products is intended to outline our general
product direction and it should not be relied on in making a purchasing decision.
The information mentioned regarding potential future products is not a
commitment, promise, or legal obligation to deliver any material, code or
functionality. Information about potential future products may not be
incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
Performance is based on measurements and projections using standard IBM
benchmarks in a controlled environment. The actual throughput or performance
that any user will experience will vary depending upon many factors, including
considerations such as the amount of multiprogramming in the user’s job
stream, the I/O configuration, the storage configuration, and the workload
processed. Therefore, no assurance can be given that an individual user will
achieve results similar to those stated here.
2

3. Acknowledgements and Disclaimers
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in
which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for
informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant.
While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without
warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this
presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or
representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use
of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have
achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended
to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other
results.
© Copyright IBM Corporation 2013. All rights reserved.
• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with
IBM Corp.
• Please update paragraph below for the particular product or family brand trademarks you mention such as WebSphere, DB2,
Maximo, Clearcase, Lotus, etc
IBM, the IBM logo, ibm.com, [IBM Brand, if trademarked], and [IBM Product, if trademarked] are trademarks or registered
trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM
trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols
indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks
may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml
If you have mentioned trademarks that are not from IBM, please update and add the following lines:
[Insert any special 3rd party trademark names/attributions here]
Other company, product, or service names may be trademarks or service marks of others.
3

4. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
4

5. Telco and cross industry data to create a unified view of the
customer. Mobility information is becoming increasingly
valuable…..
Structured
Repeatable
Linear
Monthly sales reports
Profitability analysis
Customer surveys
Other
Industries
Other
Data
Industry
Reports
Retail
Social
Media
Data
Customer
• Segment
• Social
Network
• Demographics
• Sex,
Age
Group,
etc
• Tenure
• Rate
plan
• Credit
RaBng,
ARPU
Group
Device
• Class
• Manufacturer
• Model
• OS
• Media
Capability
• Keyboard
Type
TransacBons
• Voice,
SMS,
MMS
• Data
&
Web
Sessions
• Click
Streams
• Purchases
• Downloads
• Signaling,
AuthenBcaBon
• Probe/DPI
Network
• Availability
• Throughput/Speed
• Latency
• LocaBon
• FaciliBes
Interface
• Discovery
• NavigaBon
• RecommendaBons
Product/Service
• SubscripBons
• Rate
Plans
• Media
Type
• Category/ClassificaBon
• Price
Starts,
Stops
Success
Rates
Errors
Throughput
Setup
Time
ConnecBon
Time
Usage
Recency
Frequency
Monetary
Latency
Telco Data Cross Industry Data
5

6. Building Context and Intent from Location data
• Deriving location: location information may be derived using multi-
modal information
– CDR data, tower data, device data, Wi-fi etc.
– Accuracy of location information depends on data fidelity etc.
• Building context: making sense of the location information
– Correlate location information with business data
– Various other correlation rules may be used to build a rich context
• Inferring intent: infer consumer level intents by leveraging location
and mobility patterns
Deriving Location Inferring IntentBuilding Context
6

7. 7
Data
Cell tower locations
Wi-fi locations
Device locations
Device usage data – apps, web
sites
Customer data – demographics
Refined locations
Mobility Patterns
Hang outs
Hang outs correlated with
business locations
Mode of transportation
Traveling buddies
Analytics
7

8. 8
Type of Location Analytics…..
Habitual Journey
Patterns
Demographic
customer profiles
Common origins
and destinations
Direction of
travel
Level of Mobility +
Segmentation
Aggregated
Mode of transport
Average journey
times
Travel pattern
anomalies
Accurate
Location
Congestion
Real time traffic
incident flags
Optimal route
planning
Foot traffic Customer wait times
Individual
mode of transport
Possible with event data More detailed data required
VCC Board Morph Update, June 2011

9. First-of-a-Kind Program
§ Experimental technology-based
solutions engagements
§ Testing tomorrow’s innovations on
today’s business problems
§ Yielding prototype solutions across a
range of industries
§ Creating valuable intellectual capital
for IBM’s portfolio
§ Value to IBM Clients
– Early market advantage
– Access to world class researchers
9

10. FOAK Deliverables
• Early thought leadership and experiences
with new technologies
• Working prototype of an innovative
solution not yet available in the
marketplace
• The know-how to improve a business
process or solve a problem
• Software components, methodologies
and tools
• Press & media coverage
10

11. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
11

12. Two Scenarios: Aggregate and Individual
• Aggregate Anonymized Analytics: Sensing City-scale
people movement from Telco data and leveraging for Transit
Optimization
• Enriched Consumer Profile: Customer Analytics with
Mobility Profiles from Telco Data
12

13. Enriched Consumer Profiles for Enabling Telco
Data Monetization
• We develop enriched consumer profiles by deriving insights
about consumer preferences, life style, and intent from location,
mobility and call data joined with use case appropriate data
sources.
• Enriched consumer profiles are utilized to enable new services
and effective campaign through targeted segmentation.
13

14. Two Scenarios: Aggregate and Individual
• Aggregate Anonymized Analytics: Sensing City-scale
people movement from Telco data and leveraging for Transit
Optimization
• Enriched Consumer Profile: Customer Analytics with
Mobility Profiles from Telco Data
14

15. Sensing City Scale People Movement from Telco Data
Cities Demonstrated: Istanbul (Turkey), Dubuque (USA) for Transit
Optimization and a series of subsequent client pipeline
Challenge Cities have very little real understanding of where citizens, goods and
transportation move during the day. Without this information it is
difficult to accurately plan and manage the usage of roads and
infrastructure.
Solution Using a variety of real time data from “smart phones”, GPS devices,
terminals, traffic cameras, public transportation schedules and transit
data, develop models of zonal density, flow of goods and origin /
destination pairs. From these models, drive processes to manage this
flow against a specific objective.
Benefits Evaluates the efficacy of existing transit system and transportation
infrastructure; provides the structure for design incentive strategies to
win new riders – information, incentives, services; optimize fleet
operations in situations where demand outpaces supply; manage
revenue through better zoning and permits. comprehensive solution
that will address the management of congestion, fleet management,
people attending events, and multimodal transit
1515

16. Sensing People Movement from Telco Data
16

17. Identifying Meaningful Locations
Where People Live Where People Work
Istanbul Movement Analysis
- 4.7 million phones w. 3B+ events/week
- Accurate detection of home, work & meaningful locations
17

18. Traffic Monitoring
Uses basic analytics building blocks already seen to display time based
traffic flow levels mapped to city road system. A snapshot at 8:30am:
18

19. Commuter Pain Index
19

20. Feeder Bus Route Optimization for M4 Metro
Line on Anatolian side of Istanbul
Feeder bus routes based on demand to 4 metro stations on Kadikoy-Kartal metro line
20

21. Optimal Bus Stop Location Design
• Stops are added by
considering the greatest
potential demand for transit
and accessibility at origin and
destination
• Some stops are added to far
places in which demand to the
area already served by
existing stops is potentially
large
21

22. Two Scenarios: Aggregate and Individual
• Aggregate Anonymized Analytics: Sensing City-scale
people movement from Telco data and leveraging for Transit
Optimization
• Enriched Consumer Profile: Customer Analytics with
Mobility Profiles from Telco Data
22

23. Consumer Analytics with Enhanced Consumer
Profiles
• Derive advanced location/mobility attributes and patterns from Telco data to
enrich consumer profiles with mobility context
• Derive predictive model about consumers location and mobility patterns
• Leverage enriched consumer profiles for data monetization opportunities by
correlating and joining other data sources
• Build an operational asset on IBM Big Data platform to enable Telco to
extract mobility attributes and patterns efficiently
23

24. Set of example mobility attributes
• Base set of example mobility attributes
– Home and work location
– Weekday top locations
– Weekend top locations
– Meaningful location detection
– Classification of where and when time spent
– Detecting tourism pattern
– Detecting specified habits related to mobility
– Trip purpose
– Anomaly in mobility from baseline patterns
– Detecting who’s who in the household based on mobility pattern
• Advanced predictive models (Next Best Location)
– Likely place a person would be at a future time
– Likelihood of a person going to a Mall during this weekend
– When this person is likely to be a tourist
24

25. 25
Enhanced Micro-segmentation with Mobility Model
Mobility
Patterns
Buying
Patterns
Social
Patterns
Demographics
• Gender
• Age group
• Address
• Income
Historical buying patterns
Social network
influencers
Mobility Model
• Location and movement pattern (space,
time)
• Meaningful location detection
• Meaningful location classification
• Trip purpose
• Estimated Duration of stay
• Estimated Duration of travel
• Mode of travel
• Calling patterns
• Detecting tourist patterns
• Detecting student patterns
• Estimated demographic profile of user of
phone
• Anomalies in regular patterns
Enhanced Attributes for Customer Segmentation

26. Retailer Customer Profile
Real Time Targeted Advertisement for IPTV
AAP
(Advanced
Analytics
Platform)
3 - AAP catches the
new football interest
flag, his frequent
sports shopping, and
in realtime matches
Tom’s profile with an
offer for 20% off
coupon to an Nike
store.
4 - Tom is also an
existing SMS Opt-
In mobile cust.
5 – Tom receives
targeted IPTV
advertisements based
on his IPTV, mobility
and social profiles
2 - Tom is channel surfing,
mostly sports channels,
primarily football games where
Nike advertises a lot (AAP enhances
his customer profile, after 10 football
games viewed in 1st month,
with an interest flag as a “football fan”)
Enhanced Cust. Profile
Interest / Mobile # / Email
1- Tom activates IPTV service
with the America 50 package and
adds the ESPN sports ala carte
option (we have an initial
customer profile with his fixed #
and a mobile#)
A la carte option
Sports Packages
tom@gmail.com
212-­‐201-­‐1234
Language
Package
26

27. Location Based Real Time Offering on Mobile Phone
Lisa
4 - AAP catches that
Lisa is entering a mall,
and matches her
“Fashion” interest flag
and “Perfume”
preference, sends in
realtime an offer for
20% off coupon for
Byonce fragrance at
Sephora in that mall.
5 - Lisa receives
an SMS/email/App
notification that
her mobile app
account contains a
new offer for
Beyonce perfume.
Beyonce Fan Page
2 - She follows a
friend’s post on FB and
clicks the Like button on
the Beyonce Fan Page.
3 - Lisa’s IPTV viewing
& mobile clickstream
behaviors set her Interest
flag to “Fashion” and one
preference to “Perfume”.
6 - Lisa uses
the mWallet
app on her
smartphone to
purchase some
perfume at POS
via NFC.
1- Lisa is a mobile subscriber
with Telco and downloads the
mobile app and agrees to receive
offers related to her interests.
AAP
(Advanced
Analytics
Platform)
Retailer Customer Profile
Enhanced Cust. Profile
Interest & Preference
IPTV a la carte option &
Mobile Features/Apps
IPTV Lang
Pkg &
Mobile Pkg
27

28. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
28

29. 1
2
3
Advanced Analytics Platform
End-use
Applications
Analytics
Visualization
Big Data Analytics
Warehouse
Predictive Analytics
Sens
e
Analyze Act
Search / Explore
KPIs
Dashboards
Drill-Downs
Reports
Marketing
Campaigns
Rules Engine
Behavioral
Analysis
Outcome
Optimization
Propensity
Scoring
Model
Creation
Structured /
Unstructured
Data
Data Governance
Data Integration
ETL/ELT
ChangeCapture
DataQuality/Validity/Security-Privacy
Format/UnitConversion
Consolidation/De-duplication
DataRepositories
Network
Data
Customer
Behavior Data
Customer
Data
ProductDataNetworkTopology
Data
ContinuousFeed
Sources
Usage Data
Reference
Data
Historical
Analysis Data
Demographics
Segmentation
Location
Past Actions
Propensity
Scores
Behaviors
Predictive Model
Deployment
Actionable
Insight
Stream Processing
Streaming Data
Operational
Systems
4
5
AAP Capabilities
High Performance Historical analysis (Big Data Platform)
Model Based Analytics - behavioral scoring, micro segmentation,
correlation detection analysis
Real-time scoring, classification, detection and action
Visualize, explore, investigate, search and report
Take action on analytics
IBM’s Advanced Analytics
Platform (AAP) Supports Use
Cases across the business
with New Era Capabilities
Create new Services
and Business Models Transform Operations
Build Smarter
Networks
Personalize Customer
Engagements
1
1
2
3
4
5
5
29

30.
Social
Informa-on
Loca-on
Informa-on
Customer
database
informa-on
InfoSphere Streams
Low Latency Analytics for streaming data
InfoSphere BigInsights
Hadoop-based low latency analytics for
variety and volume
IBM Netezza
BI and Ad Hoc Analytics
Structured Data
Customer database
Coremetrics
Low Latency Analytics for streaming data
Data sources…..
30

31. The carmel
frappuccino
in starbucks is
just heavenly.
IBM
BigInsights
Text
Analy-cs
Accelerators
BigInsights
Custom
analyBcs
First
Name:
Joe
Last
name:
Smith
Address:
1234
Anyroad
….
[
X
]
Likes
coffee
[
X
]
Likes
frappuccino
[
]
Likes
cappuccino
….
[
posiBve]
SenBment
coffee
Social Media Profile Creation
31

32. URL Analysis- Extract Implicit User Profile
analysis"
URL Analysis: for each user,
report the most meaningful
interests to describe her profile.
Large scale analysis
Update users
profiles"
Consume"
Adaptive user
segmentations: create
new users segmentation by
clustering similar interests
Data Cleansing
32

33. 3
3
CDR Relationship Analytics
33

34. Collecting and analyzing in real-time millions of events from multiple sources to detect the
right time to respond to the event
CDRs
Billing
CRM
Location
Account Mgt
Internet
Network
Millions of events
per second
Microsecond Latency
Dropped Calls
Outgoing International Calls
Call Duration
Extra Call
Contract Expiration
Entered new cell
New Top-Up
5 minutes left on pre-paid
EDW
Invoice Issued
Predictive Models
3 dropped calls in 10 minutes
Customer is close to a store
Customer entered a shopping area
Invoice paid + called competitor
Smart phone browsing pattern
Customer is watching a video
Congested Cells
Invoice Paid
Acquired new products
Change contracts
Brand Reputation
Customer Sentiment from Social network
Customer is roaming
Customer is at home
Campaign
ManagementInvoke appropriate campaign
Score
Real-time Stream Analytics
34

35. 35
Campaign Management

36. Campaign Response
36

37. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
37

38. Determining Buddies, Hangouts, Life Style
Example Lifestyle Attributes for marketing demonstration
§ Subscriber Lifestyles
§ Popular Locations
§ Subscriber Pairings
Who Are You?
Homebody
Daily Grinder
Delivering the Goods
Globetrotter
Nomad
10 Top Hangouts
Best Buddies
Next Steps
• Given the lifestyles, popular locations, and best buddy data => predict where
individuals or groups of similar individuals will be and when.
• Use time series modeling and clustering we can create time/location based marketing
campaigns targeted at homogenous groups in specific locales.
38

39. © 2012 IBM Corporation
Buddies, Hangouts, Globtrotters
Areas of mobility analytics
n Individual Lifestyle and Usage profiles
n Popular Locations with specific profiles
n Who are the Buddies
n Predicting where people go
Who Are You?
Homebody
Daily Grinder
Delivering the Goods
Globetrotter
Sofa Surfer
10 Top Hangouts
Mobile ID Buddy Rank
2702 1
1256 2
8786 3
4792 4
8950 5
39

40. What are Profiles
• Lifestyle Profiles are defined by marketing analysts for specific
use cases or marketing programs
• Usage Profiles are created using data mining algorithms and
define how a person uses services during the day
• Location Affinity is created with algorithms and determines
preferred locations for individuals throughout the day and week
• Together these uniquely define a person with relation to how
the retailer or marketer might want to market to them
40

41. Creating Groups of Mobility Profiles Enables
Better Prediction for Certain Groups
l profiles breakdown like this
l Homebody, doesn't visit too many unique locations
l Daily Grinder, back and forth to work, quiet weekends, makes
stops along the way
l Norm Peterson, inside the lines, no deviations
l Delivering the goods, no predictable patterns, many different
locales during the day
l Globe Trotter, either not in town, or keeps their phone turned off
l Rover Wanderer, spends evenings at various location (sofa
surfers www.couchsurfing.org)
l “Other”, is a group hard to categorize
41

42. By Profile, when is it easy or difficult to predict
where they will be?
Profile Day Time Predictability
Daily Grinder Thursday Dinner Highest
Daily Grinder Friday Afternoon Lowest
Homebody Saturday Night Highest
Homebody Wednesday Morning Lowest
These are the 2 most predictable profiles, yet there is diversity in their predictability.
To best communicate with Daily Grinders, contact them on Thursday Afternoons just before dinner
42

43. Preferred Locations of by profile type at
Lunchtime Weekdays (Central Stockholm)
Delivering
the Goods
Night
Shifters
Daily
Grinders
43

44. What analysis is available (Anonymous
Data)
From the mobility profiles, summarized, anonymous analysis is
available
l Summarized to ensure anonymity, analysis of popular locations
by time of day and profile of subscribers is possible
l For retailers this information can help understand what types
of people are nearby at lunch time
l What types of people prefer which areas. Some obvious
results are Globe Trotters go to airports, Daily Grinders go to
office buildings. Other non-obvious results show up also.
l Are there predictable patterns that we can use to target
certain groups in the future?
44

45. What Makes this Possible?
l Using the power of Netezza and modeling capabilities of SPSS we
can literally throw all the data at data mining algorithms and create
discrete clusters of subscribers by activity, mobility
l Apply the data mining outputs to the entire subscriber base by
creating detailed specific analyses for each subscriber refined by the
mobility profiles
45

46. Enriched Consumer Profile Hub
Customer
Profile
Hub
IPTV
-­‐
SubscripBon
Billing
-­‐ VOD
Billing
&
viewed
-­‐
channel
viewing
history
-­‐ -­‐
contents
purchased
-­‐ Logs
&
Tuning
Events
-­‐
package
subscripBon
Mobile
-­‐
LocaBon
-­‐
URL+App
Transac-ons
-­‐
xDRs
and
inb.
roaming
-­‐
RAN
(incl.
HLR/VLR)
-­‐
Top
Up
-­‐
Pkgs
-­‐
Billing
-­‐
SMS,
browing
URLs
Other:
-­‐
Devices
-­‐
Dealer
Network
-­‐
Contact
Center
-­‐
Call
Recordings
-­‐
Trouble
Tickeing
-­‐
Campaign
Results
(Imagine)
-­‐
Loyalty
-­‐
CompeBBon
Website
-­‐
Retail
Transac-ons
Fixed
-­‐
CDR
-­‐
URL
(IP)
-­‐ Radius
(IP-­‐Cust)
-­‐
Pkgs
-­‐
Billing
Historical
TransacBons/
Events
Partners/Retailers
AdverBsers
Other/Internal
GIS
-­‐
Business
map
and
numbers
-­‐
Point
of
Interest
maps
Consumers
of
new
Insights
Feedback
Social
Media
Data
46

47. Agenda
• Motivation and Background
• IBM Research activities
• Advanced Analytics Platform
• Life Style Analytics
47

48. Thank You
Your feedback is important!
• Access the Conference Agenda Builder to
complete your session surveys
o Any web or mobile browser at
http://iod13surveys.com/surveys.html
o Any Agenda Builder kiosk onsite
48