• Save
Big Data Journeys: Review of roadmaps taken by early adopters to achieve their big data goals
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Big Data Journeys: Review of roadmaps taken by early adopters to achieve their big data goals

on

  • 2,224 views

Implementing a Big Data program can be a long and arduous journey. Each organization has its own unique business drivers and technical considerations that drive their big data adoption roadmaps. ...

Implementing a Big Data program can be a long and arduous journey. Each organization has its own unique business drivers and technical considerations that drive their big data adoption roadmaps. Whatever be your organization's specific big data driver - be it managing a rapid surge of data, implementing a new set of analytic capabilities, incorporating unstructured data as part of your enterprise data platform or accessing real time information for actionable intelligence - the approach and roadmap that you put in place to reach that end goal becomes all the more critical in a space where early success stories are relatively rare, skill sets are hard to find and technologies are still evolving.
In this session we will chronicle the journeys of four different organizations that were early adopters of big data. Each of them charted a different path to achieve their big data goals. We will look at what were the key drivers behind their respective approaches, what worked and what did not work for them.

Statistics

Views

Total Views
2,224
Views on SlideShare
2,087
Embed Views
137

Actions

Likes
1
Downloads
0
Comments
1

1 Embed 137

http://analyzingmedia.com 137

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Big Data Journeys: Review of roadmaps taken by early adopters to achieve their big data goals Presentation Transcript

  • 1. Big Data Journeys A review of roadmaps taken by earlyadopters to achieve their big data goals TDWI Big Data Solution Summit San Deigo, CA // June 4-6, 2012 Krishnan Parasuraman CTO, Digital Media Netezza& Big Data Solutions
  • 2. Big Data Journeys A review of roadmaps taken by earlyadopters to achieve their big data goals Talking Points • Journeys of 4 organizations • Different Drivers and Considerations • Different paths to big data realization • Key learning
  • 3. Big Data Considerations1 Leading Financial Services Solution Provider Volume2 Large Online Content Publisher Value3 Global Telecommunications Major Velocity4 Emerging Digital Media Marketer Variety
  • 4. 1 Leading Financial Services solution provider Provider of Financial Services, Products and Services to both businesses and consumers • Provide personalized customer experience online • Anticipate user behavior andStrategic shift to deliver goods and guide them to specific services via Digital Channels functionality • Maintain consistent experience across online, mobile and social channels
  • 5. 1 Leading Financial Services solution provider Provider of Financial Services, Products and Services to both businesses and consumers Volume • Large volumes of Data • Data integrationBig Data Solution Considerations • Deep Analytics • Large number of attributes
  • 6. 1 Leading Financial Services solution provider 2007 : Before the Digital Shift EDW Analytics 4 3 Biz. UsersInternal Data Sources 2 Data Analysts 1 • Top 10 display advertiser in the US Digital Data • 25Billion Impressions per quarter (Clickstream) • 1Billion clicks per day during peak usage • Regression analysis for conversion tracking
  • 7. 1 Leading Financial Services solution provider 2008 : Roadmap 1.0 EDW Analytics Biz. UsersInternal Data Sources Data Analysts Digital Data Step 1: Move to Massively Parallel Data Warehousing Appliance – (Clickstream) Address volume, scale and performance considerations
  • 8. 1 Leading Financial Services solution provider 2010 : Roadmap 2.0 EDW + Analytics Biz. UsersInternal Data Sources In DB Anal- ytics Data Analysts Digital Data Step 2: Leverage In Database Analytics – (Clickstream) Run analytics at scale, closer to data
  • 9. 1 Leading Financial Services solution provider 2010 : Roadmap 3.0 EDW + Analytics Biz. UsersInternal Data Sources In DB Anal- ytics Data Analysts Digital Data(Clickstream) Step 3: Offload data pre-processing, cleansing and normalization to Hadoop – Elastic scalability + Analytics sandbox
  • 10. 2 Large Online Content Publisher One of the internet’s top destinations for specialized content Value • Provide regular data feeds – no Support business partners and performance SLAs affiliate marketer’s data needs • Manage cost of infrastructure
  • 11. 2 Large Online Content Publisher 2010 : Roadmap 1.0 EDW + Analytics 1 Biz. Users Data Sources ETL• 15 Million unique visitors Data• 210 million page views 2 Analysts• 2TB of new data per day• 1 million+ new content items per day Partners & Affiliates
  • 12. 2 Large Online Content Publisher 2011 : Roadmap 2.0 EDW + Analytics 1 Biz. UsersData Sources ELT Data Analysts 2 3 4 Partners & Affiliates
  • 13. 3 Global Telecommunications Major Leading cell phone carrier networks in the world Velocity • Predict outages and congestions React to network disruptions before they appear immediately • Address disruptions in Real Time
  • 14. 3 Global Telecommunications Major Till 2007: World of Voice and limited Data EDW Analytics Call Centers Data Sources Network engineers• Call Detail Records• Network Transmissions logs• Thousands of events per second 1 Response Latency = Hours or Days
  • 15. 3 Global Telecommunications Major 2008 – 2011 – Voice, Data, Smartphones and 3G EDW + Analytics Call Centers Data Sources In-database modeling and scoring Network• Call Detail Records engineers• Network Transmissions logs• Millions of events per second Response Latency = Minutes Step 1: Adoption of Massively Parallel Data Warehousing Appliance reduced overall latency from hours to minutes
  • 16. 3 Global Telecommunications Major 2011+Video Services, 4G LTE Stream processing EDW + Analytics Call Centers Data Sources In-database modeling and scoring Network• Call Detail Records engineers• Network Transmissions logs• Hundreds of Millions of events per second Response Latency = Seconds Step 2: Stream processing provided Real Time analytics capability, took processing workload off DW and was designed to scale
  • 17. 4 Digital Media Marketer Specializes in multi-channel marketing across online, offline, mobile and social channels Variety • Manage large volumes of unstructured dataMonitor social media channels and engage with customers • Correlate structured and unstructured data to increase targeting and relevance
  • 18. • 100TB+ data under management4 Digital Media Marketer • “Listen” to 100Million+ Tweets per day • Manage large volumes of unstructured data 2010: Roadmap 1.0 EDW + Analytics 3 1 Biz. UsersData Sources 2 Data Analysts • 100 Node Hadoop Cluster • Manage Structured and Unstructured Data • Components included Hive, HBase, Mahout
  • 19. • 100TB+ data under management4 Digital Media Marketer • “Listen” to 100Million+ Tweets per day • Manage large volumes of unstructured data 2012: Roadmap 2.0 EDW + Analytics Biz. UsersData Sources Data Analysts Hadoop and Massively Parallel Data Warehouse co-existence EDW – Manage unstructured and structured data at scale
  • 20. Key Takeaways1 Adopt your roadmap based on the big data consideration Massively Parallel Data-warehouse Appliances and Hadoop2 are complementary technologies3 Consider Evolutionary approach over a Big Bang approach4 Your EDW will be non-monolithic – understand intra-product integration implications
  • 21. Big Data Journeys A review of roadmaps taken by earlyadopters to achieve their big data goals Bon Voyage! Krishnan Parasuraman @kparasuraman