Hadoop Boosts Profits in Media and Telecom Industry


Published on

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Operational value = departmental value
    Enterprise data lake = enterprise value
  • Moving from stage 2 to stage 3 is qualitatively different.
  • 2 maturity stages – early stage model shown here might have 5 people, easy to fund and staff
  • 2 maturity stages – mature stage model shown here might have 15 people, and more of the roles start to diversify.
  • Result of stage 2: Retain all data in its original format cost-effectively. Insights for a few, based on data silos

    Result of stage 3 : Multiple lenses on the same data and multiple tenants enabled by YARN

    A note on publication and consumption models:
    If data from my department is shared across the enterprise, the consuming departments have an expectation of availability, stability (e.g. of formats, similar to APIs), quality (similar to "referential integrity” in the MDM / RDBMS world), no surprises particularly if the insights are used for front-line decision making.
    HDP offers write vs. read privileges / protection today, but will not prevent an authorized writer from changing data that others rely on.

    As a shared service, Hadoop can no longer be a Tier 5 – completely unmanaged, best efforts, no SLA service.

    As Hadoop moves up the service tiers, expectations include defined SLAs, DR, defined RPO and RTO, follow the sun support / service desk, security, run books and other procedures expected of other high-tier IT services
  • Network infrastructure
    Service and security
    Sales and marketing
    New and adjacent business

  • 1) Proactive customer care and fault resolution:
    3 call attempts to same number within several seconds
    Top 5% customers with highest call drop ratio; targeted according to error type
  • Hadoop Boosts Profits in Media and Telecom Industry

    1. 1. Driving Business Performance in Telco with Hadoop 21 Use Cases with Lessons Learned at the Intersection of People, Processes and Technology Juergen Urbanski Former Enterprise CTO, Deutsche Telekom Board Member for Big Data & Analytics, BITKOM
    2. 2. Agenda • The Journey to a Data Driven Organzation • 21 Telco Use Cases for Hadoop • Overall Lessons Learned – 2 –
    3. 3. Business Value from Hadoop Flight Plan for a Journey in Four Phases * Timeline varies by company size. Often smaller or focused online businesses achieve milestones at the shorter end of the range. 1 2Evaluation – Business Value Awareness & Interest Evaluation – Technical Enterprise Deployment Enterprise Production Industry Leadership Point Deployment Point Production 3 4 Operational Value Strategic Value Data-Driven Organization Flight plan – typical elapsed time* from start of phase 1 in months: 2-6 9-15 18-36 Potential Value – 42 –
    4. 4. 1 2 3 4 What Would You Like to Accomplish? Levels of Success with Hadoop Potential Value Operational Value Strategic Value Data-Driven Organization CXO • Recognition of potential • Mandate to explore • Recognition of value realized • Sponsorship to expand use • Recognition of material value realized • Sponsorship to transform organization • Competitive advantage • CDO part of Exec Team Line of Business • Basic understanding of the value of Hadoop to the business • Value realized in 1 area ‒ Customer intimacy ‒ Operational excellence ‒ Risk, security, compliance ‒ New business • Value realized and tracked in many areas ‒ Customer intimacy ‒ Operational excellence ‒ Risk, security, compliance ‒ New business • Data managed like capital • Intelligence at the front line • JIT decision making • Widespread value creation Analytics & Applications • Basic understanding how Hadoop fits into existing landscape • BI and EDW access to Hadoop • Some new analytic apps, often batch • Few use cases and processing engines • Many sources and time periods • Mostly departmental silos • 10-50 enterprise users • Hadoop consumable by any department, both technically and process-wise • New apps natively on Hadoop, often transactional or real-time • Many use cases and processing engines • Multiple lenses into common data pool • Emerging data science team • 50-500 enterprise users • Data-driven culture • High-performing data science team • Use cases build on each other • 500-5000 enterprise users Data Mgt. & Security • Basic understanding how Hadoop fits • Benefitting from schema on read • Professionalizing data definitions and models • Collaboration and granular security controls governing use of shared data • Incentives and process to encourage consumption of shared data Infra- structure • Basic fluency with core technical concepts of Hadoop • 1 or more production environments • Multi-tenant shared service worldwide • Data Lake • Service Desk / CoE • Hadoop community participation and contribution – 43 –
    5. 5. Early Stage Center of Excellence Mastering the Hadoop Journey 2 – 49– Roles Analytic Application Development (LOB) Infrastructure Operations Change & Program Management ArchitectureHead of Big Data Business Analysts (usually in LOB)
    6. 6. Roles Later Stage Center of Excellence Mastering the Hadoop Journey Legend Red = New at this stage – 50 – 3 Analytic Application Development (LOB) Infrastructure Administration End User Service Desk Change & Program Management Advocacy & Demand Management Data Science & Machine Learning OperationsBusiness Team Architecture & Service Portfolio Management SVP Big Data / Chief Data Officer Group CIO or EVP Business Analysts (usually in LOB)
    7. 7. A Data Lake Establishes Hadoop as a Shared Service Mastering the Hadoop Journey Data Lake Characteristics • Timely insights for all authorized users / tenants • Many use cases, often building on each other • The right processing engine for the right job • All data, across all time periods • Multiple lenses on the same data 3 – 45 – User Use case Processing engine Data
    8. 8. Stakeholder Expectations of a Shared Service Mastering the Hadoop Journey • Multi-tenancy, workload fencing, resource isolation • Data security, governance and workflows • Data privacy, policy and regulatory compliance • Consumption models (e.g., self-service, charge-back, on-boarding) • Data publication guidelines (availability, stability, quality) • Operational processes, standards, service tiers with SLAs Stage 3 – Enterprise Data LakeDepartmental Project Silos • Security largely via restricting physical access to a few friendly users • Best efforts service for batch use cases • Data largely owned by each department 3 – 46 – User Use case Processing engine Data
    9. 9. Hadoop Can Create Competitive Advantage Mastering the Hadoop Journey 2-3 Data Science & Machine Learning* Wave C Wave B Wave A Use case New Front-line Applications, often Real-Time Integration with Front-line Applications, often Real Time Data Mining – New Analytic Applications Data Refinery – Integration with Analytic Applications Active Archive and Data Offload Creates Operational Value Drives Competitive Advantage * A practice area that is relevant to many Hadoop workloads. – 44 –
    10. 10. Agenda • The Journey to a Data Driven Organzation • 21 Telco Use Cases for Hadoop • Overall Lessons Learned – 2 –
    11. 11.  Network capacity planning  Network upgrades  Network maintenance  Network performance management  Network traffic shaping 21 Telco Use Cases for Hadoop – 11 – Use Case Network Infrastructure Function  Customer experience analytics  Contact center productivity  Field service productivity  Data protection and compliance  End-user device security Service and Security  360-degree view of customer value  Personalized marketing campaigns  Upselling and cross-selling  Next-product-to-buy (NPTB)  Churn reduction Sales and Marketing  New product development  Actionable intelligence serving:  Advertisers  Merchants/retailers  Payment processors  Federal governments  Local governments New and Adjacent Business Network Care Sales New Biz
    12. 12. Network Infrastructure – Network Capacity Planning – 12 – Business Problem  The consumption of services and resulting bandwidth in a particular neighborhood may be out of sync with a telco’s plans to build new towers or transmission lines in that same neighborhood.  This leads to a mismatch between expensive infrastructure investments and the actual revenue from those investments.  Examples:  4G (LTE)  FTTC (fiber to the curb)  FTTH (fiber to the home)  One European carrier used Hadoop to optimize the rollout of 4G coverage in time and space to match the likely pick- up in service revenue, based on detailed cell tower traffic data of the last few years.  With their prior, less informed approach, they would have had to spend 10% more capex for the same outcome. Value Realized Network Care Sales New Biz
    13. 13. Hadoop in Network Infrastructure – Network Upgrades Improve the Customer Experience – 13 – • Correlate network congestion and customer experience • 11 different data sources • Millions of subscriber records, work orders, calls, IPDRs, Tivoli NPMs • Finding: Only a few nodes responsible for most of the negative customer experience Network Node TNMP CMTS Performance Network Sensors IPDR Cable Modem Usage Competitive Spend Data HouseholdHousehold Master Subscriber Record Marketing Demo- graphics Caller Experience Work Orders Mobile Devices Customer Premise Equipment Online Transactions Social Media Interactions SOURCE DATA Network Care Sales New Biz
    14. 14. Service and Security – Customer Experience Analytics Based on Call Detail Records (CDRs) – 14 – Business Problem  A typical mobile service provider generates >1 billion CDRs per day, ingesting millions of CDRs per second.  System holds >100 billion records, half a petabyte added every month!  Due to the cost of existing solutions, the data expires after 60 days  CDRs need to be analyzed and archived for compliance, billing and congestion monitoring.  Example: forensics on dropped calls and poor sound quality.  High volume makes pattern recognition and root cause analysis difficult.  Often those need to happen in real-time, with a customer waiting for answers.  With Hadoop the carrier can to retain some data for up to three years  Hadoop provides both a cost advantage – Hadoop provides storage 20x cheaper than enterprise-grade storage – and better insights.  Better analysis to continuously improve call quality, customer satisfaction and servicing margins. Value Realized Network Care Sales New Biz
    15. 15. Service and Security – Contact Center Productivity – 15 – Business Problem  A US-based mobile provider struggled with a combination of high costs but low customer satisfaction related to customer care.  An increasing share of support cases are related to mobile data usage and associated charges.  Traditionally, contact center agents did not have granular insights into a particular customer’s data usage, hence were unable to provide effective call resolution.  With Hadoop, one operator detected that 25% of callers were contacting the call center merely to have their late fees on the monthly bill waived.  The provider was able to off-load these cases to online self-service and interactive voice recognition.  Frees up the agents to focus on more valuable customer interactions.  The provider is now extending this solution to focus on issue resolution. Value Realized Network Care Sales New Biz
    16. 16. Service and Security – Field Service Productivity – 16 – Business Problem  A provider’s contact center agents had insufficient ways of diagnosing what was wrong with customers, leading to many unnecessary truck rolls.  In particular, the agents were not able to triage network vs. home-based problems accurately enough.  Therefore, technicians were dispatched to the customer premises for problems that reside within the network.  The provider was able to avoid a large number of “false positive” truck rolls.  With each truck roll costing about $150 fully loaded, the provider was able to save several million dollars already in the first year. Value Realized Network Care Sales New Biz
    17. 17. Sales and Marketing – 360 Degree View of Customer Value – 17 – Business Problem  Telcos and cable companies interact with customers across many channels and points in time.  Data about those interactions is stored in silos.  Difficult to correlate data about customer purchases, marketing campaign results, and online browsing behavior.  Problem is exacerbated by recent acquisitions and a proliferation in the volume and type of customer data.  Merging that data in a relational database structure is slow, expensive and technically difficult.  Enterprise-wide data lake of several petabytes  360-degree unified view of the customer (or household) life time value based on usages patterns across time, products and channels. Value Realized Network Care Sales New Biz
    18. 18. Sales and Marketing – Personalized Marketing Campaigns – 18 – Business Problem  Mobile phones not only follow their owners everywhere, but also reveal a lot about their owners’ interests through browsing behavior and the applications present on the phone.  Telcos are looking for ways to mine that information.  Provider risked losing substantial revenue as prepaid customers were starting to switch to a competitor as a result of a particularly effective marketing campaign.  Pinpoint those individual customers most at risk of churning, and then built a highly targeted campaign to retain the remaining customers in that segment.  A churn alarm system was established and revenue leakage was minimized.  Telesales revenue increase by 50% by tracking competitors web-sites visited and counter offers to products searched  +20% conversion rate increase by optimizing and personalizing the path- to-transaction  $1.65 ARPU increase for 1 million customers boosts topline by $20 million per year. Value Realized Network Care Sales New Biz
    19. 19. Sales and Marketing – Up-selling and Cross-selling – 19 – Business Problem  The provider needed to find an approach to upsell smart phones into a user base that was still largely on legacy feature phones.  The operator converted many hundred thousand feature phone users to smart phones with associated data plans. Value Realized Network Care Sales New Biz
    20. 20. Sales and Marketing – Next Product to Buy (NPTB) – 20 – Business Problem  As telco product portfolios grow more complex, there are ever more opportunities to sell additional services to the same customer base.  Many sales reps however are overwhelmed with that complexity and struggle to translate the breadth of the product portfolio into incremental sales.  Confident NPTB recommendations, based on data from all its customers, empower sales associates and improve their interactions with customers pre- transaction. Value Realized Network Care Sales New Biz
    21. 21. Sales and Marketing – Churn Reduction – 21 – Business Problem  A North American provider faced the following challenge: 50% of new customers churned off within 6 months of acquisition.  The average customer life time in this segment was 13 months, well short of the 18 months needed to break even.  The provider increased the “right” customer acquisitions by 27% and decreased subsequent churn in this segment by 50%.  Price related churn down by 40%  Reducing cable subscriber churn (“cord cutting”). Every 100,000 subscribers equates to customer lifetime value of $1 billion Value Realized Network Care Sales New Biz
    22. 22. New and Adjacent Businesses – Actionable Intelligence Serving Advertisers – 22 – Business Problem  Europe’s leading real estate marketplace Scout24 – a subsidiary of Deutsche Telekom – features more than one million properties for rent or sale at any given time, and has facilitated more than 20 million property transactions over the last few years.  The company wanted to drive more market share to Scout24 by offering advertisers – typically real estate agents and brokers – an even better service.  A small team consisting of a product manager, a data scientist and a few developers was able to make a meaningful contribution to revenue growth. Value Realized Network Care Sales New Biz
    23. 23. Big Data as a Product: ImmobilienScout (Deutsche Telekom) – 23 – Network Care Sales New Biz
    24. 24. New and Adjacent Businesses – Actionable Intelligence Serving Payment Processors – 24 – Business Problem  Credit card issuers experience increasing fraud when their card members are travelling abroad.  95% of travelers opted into the SMS alerting service, resulting in a substantial decrease in fraud related to card use in foreign countries. Value Realized Network Care Sales New Biz
    25. 25. New and Adjacent Businesses – Actionable Intelligence Serving Federal Governments – 25 – Business Problem  The Eastward expansion of the European Union has resulted in a longer and more porous border to non-EU member states.  This has made it more difficult to protect the EU against a stream of illegal goods and refugees, which often travel over land from the EU’s Eastern and South- Eastern neighbors.  Law enforcement agencies are able to target their scarce resources much more effectively, for instance choosing to intercept suspicious cars traveling in certain directions at speeds above 130km/h.  This radically increases their hit rate per mission. Value Realized Network Care Sales New Biz
    26. 26. Agenda • The Journey to a Data Driven Organzation • 21 Telco Use Cases for Hadoop • Overall Lessons Learned – 2 –
    27. 27. Lessons Learned – Technology Disciplines – 48 – Security. XA Secure is a big step forward. But internal security sign-offs are complicated. New possibilities opened up by the data lake imply a steep learning curve. Data quality. Early use case was volume of structured data. Gets harder now with unstructured data use cases. Metadata management needs to be unified across BI and Hadoop. Coordination between data publishers and consumers on our internal social network breaks down barriers.
    28. 28. Lessons Learned – Business Disciplines Sponsorship. Even our CEO is aware that Hadoop has a role to play in the transformation of our business. Ownership of projects always sits with business people. Success is measured. On-boarding 10 new use cases per year. Governance is difficult in a federated organization. Hence the CEO pushes for success in a few areas, and the rest of the organization can opt in. Funding and incentives. We pay the internal data producers, there is no free data. If you consume our data, you have to share all of your data, no cherry picking. Application development not that different. Important to apply usual coding best practices here as well. Cloud is intriguing for rapid prototyping, side-stepping procurement or where sources are in the cloud. However, different clouds are not fully interoperable, resulting in some lock-in. Power plays. Data has power. A data lake brings lots of power. Beware of internal politics over who should own Hadoop. – 47 –
    29. 29. Driving Business Performance in Telco with Hadoop 21 Use Cases with Lessons Learned at the Intersection of People, Processes and Technology Questions? juergen@techalpha.com @juergenurbanski LinkedIn: juergenurbanski