Monitizing Big Data at Telecom Service Providers

  • 751 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
751
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
95
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • GeolocationSensorMachine GeneratedSocial NetworksWeb lgos, Click streamsDocuments, EmailsOLTP, ERP, CRMNot shown: videos, pictures
  • Network infrastructureService and securitySales and marketingNew and adjacent business
  • 1) Proactive customer care and fault resolution:3 call attempts to same number within several secondsTop 5% customers with highest call drop ratio; targeted according to error type
  • CDR analytics & archiving for compliance, billing & congestion monitoringContact center log analyticsFraud reduction for pre-paid mobile servicesSecurity analytics
  • Cross-channel 360 degree view of the customerPersonalized marketing campaigns, notably for upselling & cross-sellingNext best offer predictive recommendations at the point of saleSocial network and deep packet inspection analysisWeb site optimizationAdvanced pricing, with segmentation based on Price seekers, “In danger”, “Up-sell”Deep packet analytics for churn prevention and counter-offersSales completion actions:Empty basket re-targetingCheck-out completionSearch completion (iPhone 4s)2) Web page personalization:Offerings based on previous exp.Offerings based on affinity3) Path-to-transaction optimization:1) Churn prevention:Tracking competitors web-sites visitedRe-calculating propensities to churn2) Counter offerings on products searched via Google:USB-modemsBranded handsetsDSL connectionsFlat voice & data tariff

Transcript

  • 1. Monetizing Big Data at Telecom Service Providers Juergen Urbanski Tech Alpha
  • 2. Hadoop Makes Shareholders Happy The World's Largest Telcos are Driving Business Performance with Hadoop at the Center of an Enterprise-Wide Modern Data Architecture Juergen Urbanski CEO, Tech Alpha Board Member Big Data & Analytics, BITKOM (German IT Industry Association)
  • 3. Agenda • Telco Data Management Challenges • Hadoop Business Value • Data Lake Business Value • Data Lake Reference Architecture • 21 Telco Use Cases for Hadoop – Network Infrastructure – Service and Security – Sales and Marketing – New and Adjacent Business 3
  • 4. Enterprise Data Management Challenges Limited Insight: • Schema On Write • Data In Silos Limited Scale: • Not Designed to Scale • Not Affordable at ScalePhysical Infrastructure Presentation & Application Data Access Data Management Engineered Systems Shared Storage Systems OLTP OLAPTraditional Analytics = = – 4 –
  • 5. Business Value of Hadoop Data Access Layer Data Management Layer Hadoop Core Capabilities: Broader Insights: • Allows simultaneous access by and timely insights for all your users across all your data • Irrespective of the processing engine, analytical application or presentation • Enabled by schema on read and enterprise-wide pool of data Unlimited Scale: • Allows to acquire all data in its original format and store it in one place, cost effectively and for an unlimited time • Affordable and performing well into the 100+ petabyte scale = = – 5 –
  • 6. A New Approach for Broader Insights HADOOP Iterate over structure Transform and analyze Hadoop Approach • Apply schema on read • Support range of access patterns to data stored in HDFS: polymorphic access Batch Interactive Real-time Right Engine, Right Job In-memory Traditional Approach • Apply schema on write • Heavily dependent on IT Determine list of questions Design solution Collect structured data Ask questions from list Detect additional questions Single Query Engine SQL – 6 –
  • 7. Compelling Economics Allow Scale 0 5 10 15 20 25 30 35 40 SAN EDW / MPP Engineered System* NAS HADOOP Cloud Storage Min Max Fully Loaded Cost per Raw TB Deployed US$ ‘000s Hadoop Provides Highly Scalable Data Storage at 5% of the Cost of Alternatives 36 to 180 20 to 80 12 to 18 10 to 20 0.250 to 1 0.1 to 0.3 * E.g., Oracle Exadata – 7 –
  • 8. 5 Capabilities of Hadoop 2.x Enable the Data Lake – 8 – Data Integration & Governance Integrate with existing systems. Move data into, within and out of the environment Security Provide layered approach to security Operations Deploy and manage a multi-tenant, environment easily, using existing tools where possible Environment and Deployment Model Run anywhere Data Lake Functional Requirements 1 32 4 Data Access = Insight …ask questions later (or in the moment) Data Management = Scale Store first… Presentation & Application Enable existing and new applications 5
  • 9. Data Lake Reference Architecture – 9 – Deployment Model Environment Data Integration & Governance Data Access Security Operations Data Management Storage: HDFS (Hadoop Distributed File System) Multitenant Processing: YARN (Hadoop Operating System) Online HBase Accumulo Real- Time Storm Others Commodity HW Linux Windows Appliance On Premise Virtualize Cloud/Hosted Authentication Authorization Accountability Data Protection across Storage: HDFS Resources: YARN Access: Hive, … Pipeline: Falcon Cluster: Knox Provision, Manage & Monitor Ambari Scheduling Oozie Data Workflow Data Lifecycle Falcon Real-time and Batch Ingest Flume Sqoop WebHDFS NFS Batch Map Reduce Script Pig SQL Hive In- memory Spark Metadata Management HCatalog Presentation & Application
  • 10. Multiple Use Cases and Tools Run on Hadoop as a Shared Service – 10 – Hadoop 2.x: Shared Service = Data Lake Hadoop 1.x: Dedicated Project Silos = Data Ponds BU2 BU3BU1 Customer Intimacy Hbase Opera- tional Excellence Lucene New Business Storm Risk Manage- ment Map- Reduce BU4 Customer Intimacy Hbase Opera- tional Excellence Lucene New Business Storm Risk Manage- ment Map- Reduce Enterprise-wide • Poor resource management • Limited governance • Batch processing, no streams
  • 11.  Shared service operational benefits similar to infrastructure cloud  Speed of provisioning and de-provisioning for capacity and users  Fast learning curve and reduced operational complexity  Consistent enforcement of data security, privacy and governance  Optimal capital efficiency driven by scale and load balancing  Value grows exponentially as data from more applications lands in one Hadoop 2.x data lake  Marginal cost of retaining data is less than marginal value  Able to run a broader range of analyses  More data in one place usually leads to better answers  Results is order-of-magnitude better insights Data Lake Business Rationale – 11 –
  • 12. Technical and Business Drivers – 12 – Foundation for a modern data architecture New data types Sensors Machine Generated Geolocation Documents, Em ail, Voice to Text Social Networks Web Logs, Click Streams Operational excellence E.g., Network Maintenance Compliance & Risk Mgt. E.g., Fraud Reduction Customer Intimacy E.g., 360 o View of Customer New Business E.g., Data as a Product Business drivers
  • 13.  Network capacity planning  Network upgrades  Network maintenance  Network performance management  Network traffic shaping 21 Telco Use Cases for Hadoop – 13 – Use Case Network Infrastructure Function  Customer experience analytics  Contact center productivity  Field service productivity  Data protection and compliance  End-user device security Service and Security  360-degree view of customer value  Personalized marketing campaigns  Upselling and cross-selling  Next-product-to-buy (NPTB)  Churn reduction Sales and Marketing  New product development  Actionable intelligence serving:  Advertisers  Merchants/retailers  Payment processors  Federal governments  Local governments New and Adjacent Business Network Care Sales New Biz
  • 14. Hadoop in Network Infrastructure – 14 – Business Problem  Network capacity planning  Network upgrades  Network maintenance  Network performance management  Network traffic shaping  Hadoop is used to optimize the rollout of 4G coverage in time and space to match the likely pick-up in service revenue, allowing an operator to defer more than 10% of capex for the same resulting revenue.  Hadoop helped detect that only a small number of congested cable network nodes were responsible for the majority of churn, and could thus be prioritized for maintenance and upgrades.  Network function virtualization, software defined networking and unified all IP networks vastly increase the amount of machine and log data relevant for trouble shooting. Hadoop helps with root cause analysis and may even be used to reason on the data in real-time. Value Realized Network Care Sales New Biz
  • 15. Network Infrastructure – Network Capacity Planning – 15 – Business Problem  The consumption of services and resulting bandwidth in a particular neighborhood may be out of sync with a telco’s plans to build new towers or transmission lines in that same neighborhood.  This leads to a mismatch between expensive infrastructure investments and the actual revenue from those investments.  Examples:  4G (LTE)  FTTC (fiber to the curb)  FTTH (fiber to the home)  One European carrier used Hadoop to optimize the rollout of 4G coverage in time and space to match the likely pick- up in service revenue, based on detailed cell tower traffic data of the last few years.  With their prior, less informed approach, they would have had to spend 10% more capex for the same outcome. Value Realized Network Care Sales New Biz
  • 16. Network Infrastructure – Network Upgrades – 16 – Business Problem  Hadoop is used for targeted network maintenance and upgrades by cable companies.  One large US cable MSO was unsure how cable network congestion affects churn, and where exactly network upgrades produce the most incremental revenue.  The result was that only a small number of nodes were responsible for the majority of the negative customer experience, and could therefore be prioritized for upgrades. Value Realized Network Care Sales New Biz
  • 17. Hadoop in Network Infrastructure – Network Upgrades Improve the Customer Experience – 17 – • Correlate network congestion and customer experience • 11 different data sources • 4m subscriber records, 12m work orders, 9m calls, 42m IPDRs, 20m Tivoli NPMs • Finding: Only a few nodes responsible for most of the negative customer experience Network Node TNMP CMTS Performance Network Sensors IPDR Cable Modem Usage Competitive Spend Data HouseholdHousehold Master Subscriber Record Marketing Demo- graphics Caller Experience Work Orders Mobile Devices Customer Premise Equipment Online Transactions Social Media Interactions SOURCE DATA Network Care Sales New Biz
  • 18. Network Infrastructure – Network Maintenance – 18 – Business Problem  Radio access networks provide the air interface between a mobile provider and the end user mobile devices.  Maintenance and repair of radio access networks poses substantial logistical challenges. In most countries, mobile networks cover more than 95% of a country’s surface area.  Many transmission towers are in remote and difficult to access locations.  In high-density areas, pico- and femto- cells optimize local coverage, but in turn require coordination with the building owner for maintenance.  Hadoop improves a provider’s ability to service equipment proactively, which is always cheaper and less disruptive than the replacement of equipment that has already failed. Value Realized Network Care Sales New Biz
  • 19. Network Infrastructure – Network Performance Management – 19 – Business Problem  Existing network management platform meant to diagnose poor cellular service such as dropped calls or poor audio quality.  Overwhelmed by data volume, ingesting 10 million messages per second  Each analysis was limited to a 24-hour time window and only one-fiftieth the surface area of the United States.  Same customer issue may generate multiple support calls, but the operator’s team cannot see relationships between multiple variables across time.  Is the problem with the customer’s device? Is it their neighborhood or proximity to a tower? Is it because of how they use their phone?  With more history, they are able to explore root causes that they have never been able to identify by reviewing just one day’s data, allowing them to to improve cell phone service. Value Realized Network Care Sales New Biz
  • 20. Hadoop in Service and Security – 21 – Business Problem  Customer experience analytics based on call detail records (CDRs)  Contact center productivity  Field service productivity  Data protection and compliance  End-user device security  With Hadoop, one operator detected that 25% of callers were contacting the call center merely to have their late fees on the monthly bill waived. Clearly a case for call deflection to interactive voice recognition and online self- service.  Contact center agents had insufficient ways of diagnosing what was wrong with customers, leading to many unnecessary truck rolls. Hadoop helped avoid these.  3% of smartphones account for 10-15% of traffic because of malware (notably on Android phones) and some fair use violations. Hadoop helps detect that so operators can take remedial action. Value Realized Network Care Sales New Biz
  • 21. Service and Security – Customer Experience Analytics Based on Call Detail Records (CDRs) – 22 – Business Problem  A typical mobile service provider generates >1 billion CDRs per day, ingesting millions of CDRs per second.  System holds >100 billion records, half a petabyte added every month!  Due to the cost of existing solutions, the data expires after 60 days  CDRs need to be analyzed and archived for compliance, billing and congestion monitoring.  Example: forensics on dropped calls and poor sound quality.  High volume makes pattern recognition and root cause analysis difficult.  Often those need to happen in real- time, with a customer waiting for answers.  With Hadoop the carrier can to retain some data for up to three years  Hadoop provides both a cost advantage – Hadoop provides storage 20x cheaper than enterprise-grade storage – and better insights.  Better analysis to continuously improve call quality, customer satisfaction and servicing margins. Value Realized Network Care Sales New Biz
  • 22. Service and Security – Contact Center Productivity – 23 – Business Problem  A US-based mobile provider struggled with a combination of high costs but low customer satisfaction related to customer care.  An increasing share of support cases are related to mobile data usage and associated charges.  Traditionally, contact center agents did not have granular insights into a particular customer’s data usage, hence were unable to provide effective call resolution.  With Hadoop, one operator detected that 25% of callers were contacting the call center merely to have their late fees on the monthly bill waived.  The provider was able to off-load these cases to online self-service and interactive voice recognition.  Frees up the agents to focus on more valuable customer interactions.  The provider is now extending this solution to focus on issue resolution. Value Realized Network Care Sales New Biz
  • 23. Service and Security – Field Service Productivity – 24 – Business Problem  A provider’s contact center agents had insufficient ways of diagnosing what was wrong with customers, leading to many unnecessary truck rolls.  In particular, the agents were not able to triage network vs. home-based problems accurately enough.  Therefore, technicians were dispatched to the customer premises for problems that reside within the network.  The provider was able to avoid a large number of “false positive” truck rolls.  With each truck roll costing about $150 fully loaded, the provider was able to save several million dollars already in the first year. Value Realized Network Care Sales New Biz
  • 24. Service and Security – End User Device Security – 26 – Business Problem  A mobile operator needed to identify real-time malware threats from non- trusted application stores and contain their impact on customers.  3% of smartphones account for 10-15% of traffic because of malware (notably on Android phones) and some fair use violations.  Hadoop helps detect that so operators can take remedial action, thus eliminating a disproportionate share of network tonnage.  Options ranged from notifying an affected customer all the way to blocking certain URLs for the whole network. Value Realized Network Care Sales New Biz
  • 25. Hadoop in Sales and Marketing – 27 – Business Problem  360-degree view of customer value  Personalized marketing campaigns  Upselling and cross-selling  Next-product-to-buy (NPTB)  Churn reduction  Telesales revenue increase by 50% by tracking competitors web-sites visited and counter offers to products searched  +20% conversion rate increase by optimizing and personalizing the path- to-transaction  $1.65 ARPU increase for 1 million customers boosts topline by $20 million per year.  Reducing cable subscriber churn (“cord cutting”). Every 100,000 subscribers equates to customer lifetime value of $1 billion  Churn model quality increase  Price related churn down by 40% Value Realized Network Care Sales New Biz
  • 26. Sales and Marketing – 360 Degree View of Customer Value – 28 – Business Problem  Telcos and cable companies interact with customers across many channels and points in time.  Data about those interactions is stored in silos.  Difficult to correlate data about customer purchases, marketing campaign results, and online browsing behavior.  Problem is exacerbated by recent acquisitions and a proliferation in the volume and type of customer data.  Merging that data in a relational database structure is slow, expensive and technically difficult.  Enterprise-wide data lake of several petabytes  360-degree unified view of the customer (or household) life time value based on usages patterns across time, products and channels. Value Realized Network Care Sales New Biz
  • 27. Sales and Marketing – Personalized Marketing Campaigns – 29 – Business Problem  Marketers have long sought ways to tailor their marketing campaigns to the needs of each individual customer.  Telcos are uniquely positioned to deliver on that goal because mobile phones not only follow their owners everywhere, but also reveal a lot about their owners’ interests through browsing behavior and the applications present on the phone.  Telcos are looking for ways to mine that information.  Provider risked losing substantial revenue as prepaid customers were starting to switch to a competitor as a result of a particularly effective marketing campaign.  The provider used Hadoop to pinpoint those individual customers most at risk of churning, and then built a highly targeted campaign to retain the remaining customers in that segment.  A churn alarm system was established and revenue leakage was minimized.  Telesales revenue increase by 50% by tracking competitors web-sites visited and counter offers to products searched  +20% conversion rate increase by optimizing and personalizing the path- to-transaction  $1.65 ARPU increase for 1 million customers boosts topline by $20 million per year. Value Realized Network Care Sales New Biz
  • 28. Sales and Marketing – Up-selling and Cross-selling – 30 – Business Problem  The provider needed to find an approach to upsell smart phones into a user base that was still largely on legacy feature phones.  The operator converted many hundred thousand feature phone users to smart phones with associated data plans. Value Realized Network Care Sales New Biz
  • 29. Sales and Marketing – Next Product to Buy (NPTB) – 31 – Business Problem  As telco product portfolios grow more complex, there are ever more opportunities to sell additional services to the same customer base.  Many sales reps however are overwhelmed with that complexity and struggle to translate the breadth of the product portfolio into incremental sales.  Confident NPTB recommendations, based on data from all its customers, empower sales associates and improve their interactions with customers pre- transaction. Value Realized Network Care Sales New Biz
  • 30. Sales and Marketing – Churn Reduction – 32 – Business Problem  A North American provider faced the following challenge: 50% of new customers churned off within 6 months of acquisition.  The average customer life time in this segment was 13 months, well short of the 18 months needed to break even.  The provider increased the “right” customer acquisitions by 27% and decreased subsequent churn in this segment by 50%.  Price related churn down by 40%  Reducing cable subscriber churn (“cord cutting”). Every 100,000 subscribers equates to customer lifetime value of $1 billion Value Realized Network Care Sales New Biz
  • 31. Hadoop in New and Over-the-Top / Adjacent Businesses – 33 – Business Problem  New product development  Actionable intelligence serving:  Advertisers  Merchants/retailers  Payment processors  Federal governments  Local governments  Hadoop-as-a-Service  Telcos are well positioned to provide big data as a service to retail, hospitality and logistics customers. This can generate $50-100m in annual revenue for each medium-sized country. Value Realized Network Care Sales New Biz
  • 32. New and Adjacent Businesses – New Product Development – 34 – Business Problem  Mobile devices produce large amounts of data about where, when, how and why they are used.  This data is extremely valuable for product managers, yet much of it is out of reach. Either it is never captured or never converted into business insight. Its volume and variety make it difficult to ingest, store and analyze at scale.  One provider who logged 27m devices with more than 1bn events per month has developed more than 20 projects and pilots within 18 months after launch, leading to increased revenue and profitability. Value Realized Network Care Sales New Biz
  • 33. New and Adjacent Businesses – Actionable Intelligence Serving Advertisers – 35 – Business Problem  Europe’s leading real estate marketplace Scout24 – a subsidiary of Deutsche Telekom – features more than one million properties for rent or sale at any given time, and has facilitated more than 20 million property transactions over the last few years.  The company wanted to drive more market share to Scout24 by offering advertisers – typically real estate agents and brokers – an even better service.  A small team consisting of a product manager, a data scientist and a few developers was able to make a meaningful contribution to revenue growth. Value Realized Network Care Sales New Biz
  • 34. Big Data as a Product: ImmobilienScout (Deutsche Telekom) – 36 – Network Care Sales New Biz
  • 35. New and Adjacent Businesses – Actionable Intelligence Serving Merchants – 37 – Business Problem  A French mobile service provider is a great example for how location information per customer segments can be used to optimize promotions and point-of-sale locations of bricks-and- mortar retailers.  The retailers were able to increase their reported same-store-sales through better campaign management and in- store optimizations. They also gained valuable insights to optimize their store network. Value Realized Network Care Sales New Biz
  • 36. New and Adjacent Businesses – Actionable Intelligence Serving Payment Processors – 38 – Business Problem  Credit card issuers experience increasing fraud when their card members are travelling abroad.  95% of travelers opted into the SMS alerting service, resulting in a substantial decrease in fraud related to card use in foreign countries. Value Realized Network Care Sales New Biz
  • 37. New and Adjacent Businesses – Actionable Intelligence Serving Federal Governments – 39 – Business Problem  The Eastward expansion of the European Union has resulted in a longer and more porous border to non-EU member states.  This has made it more difficult to protect the EU against a stream of illegal goods and refugees, which often travel over land from the EU’s Eastern and South- Eastern neighbors.  Law enforcement agencies are able to target their scarce resources much more effectively, for instance choosing to intercept suspicious cars traveling in certain directions at speeds above 130km/h.  This radically increases their hit rate per mission. Value Realized Network Care Sales New Biz
  • 38. New and Adjacent Businesses – Actionable Intelligence Serving Local Governments – 40 – Business Problem  In a large French city, traffic to large events regularly caused massive congestion on the city’s streets and highways.  The city identified and implemented dozens of specific traffic management measures, relieving congestion around major events.  They are also exploring how to use these insights for environmental impact studies, city planning and disaster management. Value Realized Network Care Sales New Biz
  • 39. • Makes capital investments more efficient • Leads to a better customer experience • Lowers churn • Increases conversions • Strengthens security • Opens up new markets Hadoop Drives Business Outcomes for the World’s Telcos and Cable Companies! – 41 –
  • 40. Questions? Email juergen@techalpha.com for a copy of the presentation. LinkedIn: juergenurbanski Download 200-page BITKOM / Forrester Guide to Big Data Technologies (in German): http://www.bitkom.org/files/documents/BITKOM_Leitfaden_Big-Data- Technologien-Wissen_fuer_Entscheider_Febr_2014.pdf