Big data Introduction by Mohan
Upcoming SlideShare
Loading in...5

Big data Introduction by Mohan



Big data Introduction by Mohan

Big data Introduction by Mohan



Total Views
Views on SlideShare
Embed Views



1 Embed 2 2


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.


12 of 2

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Excellent presentation
    Are you sure you want to
    Your message goes here
  • I like your Big Data presentation.
    I would like to share with you document about application of Big Data and Data Science in retail banking.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Big data Introduction by Mohan Big data Introduction by Mohan Presentation Transcript

    • Trendwise AnalyticsCopyright Trendwise AnalyticsBig Data/HadoopIntroduction
    • Trendwise AnalyticsAgenda1.Introduction to Big Data and Hadoop2.Big Data Business cases3.Technology4.Q&A - Wind up
    • Trendwise AnalyticsWhat is Big DataThree Vs• Volume• Variety• Velocity
    • Trendwise AnalyticsCopyright Trendwise AnalyticsVolume of Data•A commercial aircraftgenerates 3GB offlight sensor data in 1hourAn ERP system for an midsize company grows by1-2TB annuallyA Video SuveillanceCamera generates 1-3TB data in 3 monthsAirtel or Vodafonegenerates 3TB ofCall DetailsRecords (CDR)every dayEvery day 2.5 quintillion(2.5×10^18) bytes of datais createdi.e., 2,500,000TB
    • Trendwise AnalyticsCopyright Trendwise AnalyticsInternet Minute ....
    • Trendwise AnalyticsMarket opportunityIDC, a research firm, predicts thatthe market for Big Data technology and services will reach $16.9 billion by 2015,up from $3.2 billion in 2010. That is a 40 percent-a-year growth rate —about seven times the estimated growth rate for the overall informationtechnology and communications business, according to IDC.Billions and billions: big databecomes a big deal :Deloitte predicts that in 2012, “bigdata” will likely experienceaccelerating growth and marketpenetration.
    • Trendwise AnalyticsHow Companies areusing Big Data?
    • Trendwise AnalyticsCommon Big Data Customer Scenariosin your industryWeb appoptimizationSmart metermonitoringEquipmentmonitoringAdvertisinganalysisLife sciencesresearchFrauddetectionHealthcareoutcomesWeatherforecastingNaturalresourceexplorationSocial networkanalysisChurnanalysisTraffic flowoptimizationITinfrastructureoptimizationLegaldiscovery
    • Trendwise AnalyticsCopyright Trendwise Analytics916 Apr 2013Watson wins Jeopardy!Feb 14th2011 – Watson wins Jeopardy!beating its human opponents.Watson is IBM’s super computer builtusing Big Data Technology.
    • Trendwise AnalyticsBig data Applications Social media analytics ‒ ”People You May Know” atLinkedIn Voice analytics ‒ Call center Text analytics ‒ Voice of customer,sentiment analysis, warranty analysis Video analytics ‒ Intelligence, policing, retailapplications Telecom – customer churn
    • Trendwise AnalyticsBig Data at GE11 New $1B corporate center for software and analytics Hiring 400 data scientists Includes financial and marketing applications,but with special focus on industrial uses of big data When will this gas turbine need maintenance? How can we optimize the performance of a locomotive? What is the best way to make decisions about energy finance?
    • Trendwise AnalyticsCopyright Trendwise AnalyticsFord Gets Smarter About Marketing andDesign• Ford collects and aggregates data from the 4 million vehicles that usein-car sensing and remote app management software• The data allows to glean information on a range of issues, from howdrivers are using their vehicles, to the driving environment that couldhelp them improve the quality of the vehicle• Partnered with Microsoft to develop SYNC
    • Trendwise AnalyticsCopyright Trendwise AnalyticsHow Amazon Uses Big Data To Make YouLove Them• Amazon has been collecting customer information for years--not justaddresses and payment information but the identity of everything that acustomer had ever bought or even looked at.• They’re using that data to build customer relationship
    • Trendwise AnalyticsCopyright Trendwise AnalyticsHow LinkedIn is Riding a Wave of Big DataAll the Way to the Bank• LinkedIn is a trove of data not just about people, but how people aremaking their money and what industries they are working in and howthey connect to each other.
    • Trendwise AnalyticsCopyright Trendwise AnalyticsHow AT&T is using cell phone to watchuser movements?• AT&T has 300 million customers• A team of researchers is working to turn data collected through thecompany’s cellular network into a trove of information forpolicymakers, urban planners and traffic engineers.• The researchers want to see how the city changes hourly by looking atcalls and text messages relayed through cell towers around the region,noting that certain towers see more activity at different times
    • Trendwise AnalyticsCopyright Trendwise AnalyticsGovt of India• Aadhar project by Govt. of India uses Hadoop
    • Trendwise AnalyticsCopyright Trendwise AnalyticsTECHNOLOGYHadoop Non Hadoop
    • Trendwise AnalyticsCopyright Trendwise AnalyticsHadoop Components
    • Trendwise AnalyticsCopyright Trendwise AnalyticsHadoop HDFS and MapReduceHadoop runs on HDFS,Hadoop Distributed FilesystemAny data stored is converted to blocksand distributed across the cluster nodes
    • Trendwise AnalyticsCopyright Trendwise AnalyticsOther componentsHive• Data Warehouse infrastructurethat provides datasummarization and ad hocquerying on top of HadoopPIG• A high-level data-flow language andexecution framework for parallelcomputationSqoop• Sqoop is a tool designed tohelp users of large dataimport existing relationaldatabases into their HadoopclustersZookeeper• Zookeeper is a centralizedservice for maintainingconfiguration information,naming, providingdistributed synchronization,and providing group services
    • Trendwise AnalyticsBenefits of Hadoop• Hadoop is designed to run on cheap commodityhardware• It automatically handles data replication and nodefailure• Handles large volumes of unstructured data easily• Last but not least – its free! ( Open source)
    • Trendwise AnalyticsCopyright Trendwise AnalyticsCommercial Hadoop Distributions• Cloudera• Hortonworks• Greenplum, A Division of EMC• IBM InfoSphere BigInsights
    • Trendwise AnalyticsTechnology – Non Hadoop•HPCC - HPCC Systems from LexisNexis Risk Solutions offers a proven,• open-source, data-intensive supercomputing platform• designed for the enterprise to solve big data problems.• SAP HANA is SAP AG’s implementation of in-memory database technology.•NoSQL DatabasesKey-Values Stores – Redis, RiakColumn Family Stores – Cassandra, HBaseDocument Databases – CouchDB, MongoDBGraph Database – InfoGrid, Infinite Graph
    • Trendwise AnalyticsSAP HANA In-memory Database System Hana is an in-memory database systemdeveloped by SAP AG. It takes the advantage of – low-cost of main memory Fast data access of solid state drives. Data processing abilitiesof multi-core processors. It supports both row-orientedand column-oriented data storage. It incorporates powerful graph andtext processing capabilities to workwith semi and full unstructured data. SAP has positioned HANA as its solutionto big data challenges at the low end of thisscale.
    • Trendwise AnalyticsCopyright Trendwise AnalyticsThank You!
    • Trendwise AnalyticsAdditional resourcesHadoop: