Your SlideShare is downloading. ×

Impala Unlocks Interactive BI on Hadoop

4,897

Published on

The Cloudera Impala project is pioneering the next generation of Hadoop capabilities: the convergence of interactive SQL queries with the capacity, scalability, and flexibility of a Hadoop cluster. In …

The Cloudera Impala project is pioneering the next generation of Hadoop capabilities: the convergence of interactive SQL queries with the capacity, scalability, and flexibility of a Hadoop cluster. In this webinar, join Cloudera and MicroStrategy to learn how Impala works, how it is uniquely architected to provide an interactive SQL experience native to Hadoop, and how you can leverage the power of MicroStrategy 9.3.1 to easily tap into more data and make new discoveries.

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,897
On Slideshare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
132
Comments
0
Likes
10
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • More & Faster Value from Big DataProvides an interactive BI/Analytics experience on HadoopPreviously BI/Analytics was impractical due to the batch orientation of MapReduceEnables more users to gain value from organizational data assets (SQL/BI users)Makes more data available for analysis (raw data, multi-structured data, historical data)Removes delays from data migrationInto specialized analytical DBMSsInto proprietary file formats that happen to be stored in HDFSInto transient in-memory storesFlexibilityQuery across existing data in HadoopHDFS and HBaseAccess data immediately and directly in its native formatSelect best-fit file formatsUse raw data formats when unsure of access patterns (text files, RCFiles, LZO)Increase performance with optimized file formats when access patterns are known (Parquet, Avro)Run multiple frameworks on the same data at the same timeAll file formats are compatible across the entire Hadoop ecosystem – i.e. MapReduce, Pig, Hive, Impala, etc. on the same data at the same timeRun multiple frameworks on the same data at the same timeAll file formats are compatible across the entire Hadoop ecosystem – i.e. MapReduce, Pig, Hive, Impala, etc.Cost EfficiencyReduce movement, duplicate storage & computeData movement: no time or resource penalty for migrating data into specialized systems or formatsDuplicate storage: no need to duplicate data across systems or within the same system in different file formatsCompute: use the same compute resources as the rest of the Hadoop system – You don’t need a separate set of nodes to run interactive query vs. batch processing (MapReduce)You don’t need to overprovision your hardware to enable memory-intensive, on-the-fly format conversions10% to 1% the cost of analytic DMBSLess than $1,000/TBFull Fidelity AnalysisNo loss of fidelity from aggregations or conforming to fixed schemasIf the attribute exists in the raw data, you can query against it
  • Our design strategy is to tightly integrate and couple Impala within the Hadoop system. Impala (and interactive SQL) is just another application that you bring to your data. It’s integrated with Hadoop’s existing security and resource management frameworks and is completely interoperable with existing data formats and processing engines.One pool of dataStorage platforms (HDFS & HBase)Open data formats (files & records)Shared across multiple processing frameworksOne metadata modelNo synchronization of metadata between 2 different systems (analytical DBMS and Hadoop)Same metadata used by other components within Hadoop itself (Hive, Pig, Impala, etc.)One security frameworkSingle model for all of HadoopDoesn’t require “turning off” any portion of native Hadoop securityOne set of system resourcesOne set of nodes – storage, CPU, memoryOne management consoleIntegrated resource managementScale linearly as capacity or performance needs grow
  • Interactive BI/Analytics on more dataRaw, full fidelity data – nothing lost through aggregation or ETL/LTNew sources & types – structured/unstructuredHistorical dataAsking new questionsExploration and data discovery for analytics and machine learning – need to find a data set for a model, which requires lots of simple queries to summarize, count, and validate.Hypothesis testing – avoid having to subset and fit the data to a warehouse just to ask a single questionData processing with tight SLAsCost-effective platformMinimize data movementReduce strain on data warehouseQuery-able storageReplace production data warehouse for DR/active archiveStore decades of data cost effectively (for better modeling or data retention mandates) without sacrificing the capability to analyze
  • Query-able archive w/full fidelityReplace production data warehouse for DR/active archiveStore decades of data cost effectively (for better modeling or data retention mandates) without sacrificing the capability to analyzeExample: Global financial services companyOffloaded DB25x performance improvement over HiveSaved 90% on incremental EDW spend
  • Agile analytics is the hottest area in BI right now and the reason for that is that fundamentally, it is a revolutionary, new approach that enables anybody to very quickly find new insight, share that insight with everybody, and do it without the involvement of IT. That’s really a game changer considering that business intelligence has traditionally has required high levels of IT involvement in order to create dashboards, create reports, and then distribute those results out to large numbers of people. MicroStrategy Visual Insight is a pioneering new technology in agile analytics. It combines stunning visualizations with on-screen filtering and at the back end, there’s a very high speed in-memory database that powers everything for speed of thought interactivity between user and data. There are really two common use cases for this technology. The first is just pure visual data discovery, the idea of using visualizations rapidly shifting between different types of interactive visualizations to find information that you’re looking for, to isolate that information, to find trends, to look for root causes, to find the nuggets of insight into your data. The second major use case is dashboard creation. That’s the idea of bringing together multiple visualizations in an interactive dashboard that can then be published and shared with a variety of other people.
  • Transcript

    • 1. Impala unlocks Interactive BI onHadoop with MicroStrategyJustin Erickson | Cloudera | Senior Product ManagerJochen Demuth | MicroStrategy | Director, Partner EngineeringMay 2013
    • 2. Agenda• Why Impala?• Architectural Overview• Real-World Use Cases• Interactive Analytics with MicroStrategy• Taking Big Data Out of Isolation©2013 Cloudera, Inc. All RightsReserved.2
    • 3. Why Hadoop?• Scalability• Simply scales just by adding nodes• Local processing to avoid network bottlenecks• Flexibility• All kinds of data (blobs, documents, records, etc)• In all forms (structured, semi-structured, unstructured)• Store anything then later analyze what you need• Efficiency• Cost efficiency (<$1k/TB) on commodity hardware• Unified storage, metadata, security (no duplication orsynchronization)©2013 Cloudera, Inc. All RightsReserved.3
    • 4. What’s Impala?• Interactive SQL• Typically 5-65x faster than Hive (observed up to 100x faster)• Responses in seconds instead of minutes (sometimes sub-second)• Nearly ANSI-92 standard SQL queries with Hive SQL• Compatible SQL interface for existing Hadoop/CDH applications• Based on industry standard SQL• Natively on Hadoop/HBase storage and metadata• Flexibility, scale, and cost advantages of Hadoop• No duplication/synchronization of data and metadata• Local processing to avoid network bottlenecks• Separate runtime from MapReduce• MapReduce is designed and great for batch• Impala is purpose-built for low-latency SQL queries on Hadoop©2013 Cloudera, Inc. All RightsReserved.4
    • 5. Benefits of Impala5More & Faster Value from “Big Data” BI tools impractical on Hadoop before Impala Move from 10s of Hadoop users per cluster to 100s of SQL users No delays from data migrationFlexibility Query across existing data Select best-fit file formats (Parquet, Avro, etc.) Run multiple frameworks on the same data at the same timeCost Efficiency Reduce movement, duplicate storage & compute 10% to 1% the cost of analytic DBMSFull Fidelity Analysis No loss from aggregations or fixed schemas©2013 Cloudera, Inc. All RightsReserved.
    • 6. Impala and Hive6Shares Everything Client-Facing Metadata (table definitions) ODBC/JDBC drivers SQL syntax (Hive SQL) Flexible file formats Machine pool Hue GUIBut Built for Different Purposes Hive: runs on MapReduce and ideal for batchprocessing Impala: native MPP query engine ideal forinteractive SQLStorageIntegrationResource ManagementMetadataHDFS HBaseTEXT, RCFILE, PARQUET, AVRO, ETC. RECORDSHiveSQL Syntax ImpalaSQL Syntax +Compute FrameworkMapReduceCompute FrameworkBatchProcessingInteractiveSQL©2013 Cloudera, Inc. All RightsReserved.
    • 7. Not All SQL on Hadoop is Created Equal7Batch MapReduceMake MapReduce fasterSlow, still batchRemote QueryPull data from HDFS overthe network to the DWcompute layerSlow, expensiveSiloed DBMSLoad data into aproprietary database fileRigid, siloed data,slow ETLImpalaNative MPP query enginethat’s integrated intoHadoopFast, flexible,cost-effective$©2013 Cloudera, Inc. All RightsReserved.
    • 8. Our Design Strategy8StorageIntegrationResource ManagementMetadataBatchProcessingMAPREDUCE,HIVE & PIG…InteractiveSQLIMPALAMachineLearningMAHOUT, DATAFUHDFS HBaseTEXT, RCFILE, PARQUET, AVRO, ETC. RECORDSEnginesOne pool of dataOne metadata modelOne security frameworkOne set of system resourcesAn Integrated Part ofthe Hadoop System©2013 Cloudera, Inc. All RightsReserved.
    • 9. Impala Use Cases9Interactive BI/analytics on more dataAsking new questionsQuery-able archive w/ full fidelityData processing with tight SLAsCost-effective, ad hoc query environment thatoffloads the data warehouse for:©2013 Cloudera, Inc. All RightsReserved.
    • 10. Global Financial Services Company10Saving 90% on incremental EDW spend &improving performance by 5xOffload data warehouse for query-able archiveStore decades of data cost-effectivelyProcess & analyze on the same systemImprove capabilities through interactive queryon more data©2013 Cloudera, Inc. All RightsReserved.
    • 11. Six3 Systems11Boosting performance by 20X for mission-critical, real-time cyber securityAnalyze unstructured data with flexibility &real-time responseIntegrate with existing desktop & BI toolsDeploy in minutes with Cloudera Manager©2013 Cloudera, Inc. All RightsReserved.
    • 12. Expedia12Implementing self-service BI on big data,reducing data latency by 50%Offload data warehouse for archiving, ETL &analyticsUnify IT environmentContinuously ingest & analyze at scaleDrive greater usability & adoption of big datastack©2013 Cloudera, Inc. All RightsReserved.
    • 13. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.14About MicroStrategyInnovator and Leader In Interactive BICompany• Top independent analyticssoftware platform vendor• 20+ years old, publicly traded• Approximately $600M revenue in 2012.No debt, $200M+ cash in the bank• Global presence with operations in 23countriesTechnology• Long-time market leader andinnovator in analytics• Unique unitary architecture,known for high performance and scalability• Revolutionary Cloud-based analytics services• Innovations in mobile commerce and identityAnalysts• Leader for six consecutiveyears in Gartner’s BIMagic Quadrant• Leader in Forrester BI Self Service Wave• #1 Ranked Mobile BI Vendor by Gartner &Dresner Advisory• Top ranking BI vendor in the BI ScorecardCustomers• Millions of business users• Thousands of mission-criticalapplications• Nearly 4,000 customer institutions globallyacross all industries and government• Customers range from Global 500 giants likeChevron and Carrefour to cutting edgetechnology innovators like eBay and LinkedIn
    • 14. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.15RetailFinancial ServicesCommunicationsOther Major CompaniesInnovators and Leaders Worldwide4 of the Top 5 Global RetailersManufacturing5 of the Top 10 AutomotiveCompanies8 of the Top 10 Communications& Media CompaniesPharmaceuticals7 of the Top 10 Healthcare & LifeSciences Companies6 of the Top 10 Financial ServiceCompaniesGovernmentFederal, State, and LocalGovernment InstitutionsConsumer Packaged GoodsLeading Consumer PackagedGoods CompaniesOur Customers Are Leaders In All IndustriesSupporting the Most Demanding, Mission Critical BI Applications
    • 15. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.16Intuitive Interface• Interactive dashboards• Visual Analytics• Build once, deployanywhereData Federation• Virtual model spanningmultiple data sourcesHigh Performance• Push-down analytics• In-memory cubeaccelerationFlexible, Reliable, andEasy to Manage• High-efficiency object reuse• Powerful SDK• Comprehensive admin toolsMicroStrategy Analytics PlatformComprehensive Analytics Suite for Big DataWeb | Mobile | Portals | Office™Data from Across the EnterpriseDashboards Statements Visual DiscoveryMicroStrategy Analytics PlatformReportsData MartsRelationalDatabasesCloudera Impalafor HadoopMultiDimensionalSources
    • 16. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.17MicroStrategyVisual Insight• Stunningvisualizations• On-screen filtering• Speed-of-thought inmemory databaseCommon UseCases• Interactive dataexploration and rootcause analysis• Dashboard creation• Self-service BIMicroStrategy Visual InsightInteractive Analysis, Drag-and-Drop to Build Intuitive Dashboards in MinutesData MartsRelationalDatabasesMultiDimensionalSourcesData from Across the EnterpriseCloudera Impalafor Hadoop
    • 17. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.18Combine Data from Multiple Federated SourcesTake Big Data Out of IsolationPut Big Data analysis in context with information from federateddata sources into one single dashboardUser / DepartmentalDataData WarehouseAppliancesHadoopDatabasesRelational DatabasesMultidimensionalDatabasesColumnarDatabases112232 & 3Bring AllRelevant Datato DecisionMakers,No MatterWhere ItResides
    • 18. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.19Browsers Portals EnterpriseApplicationsWebEmailEmailPDF OfficeDocumentsMobileAndroidiOSBlackBerryBuild Once, Deploy AnywhereMakes Big Data Accessible to a Wider Business AudienceBuildonceDeploy via any media12
    • 19. CONFIDENTIALThe Information Contained In This Presentation Is Confidential And Proprietary To MicroStrategy. The Recipient Of This Document Agrees That They Will Not Disclose Its Contents ToAny Third Party Or Otherwise Use This Presentation For Any Purpose Other Than An Evaluation Of MicroStrategys Business Or Its Offerings. Reproduction or Distribution Is Prohibited.20From Big Data to Business ValueMicroStrategy Delivers Insights on Big Data FasterAny and AllDataWorld’s MostIntuitive InterfaceBenchmarkingProjectionsTrend AnalysisData SummarizationRelationshipAnalysisRelationalMultidimensionalHadoop-basedStructuredSemi-StructuredUnstructuredComprehensiveAnalyticsCloudera Impalafor HadoopShortened time-to-value for data scientistEnables self-service for the business user
    • 20. • Submit questions in the Q&A panel• Watch this webinar on-demand athttp://cloudera.com• Follow Cloudera at @Cloudera• Follow MicroStrategy at @microstrategy• Thank you for attending!Learn more about the ClouderaMicroStrategy partnershiphttp://cloudera.com/Microstrategy.htmlDownload Impalahttp://cloudera.com/downloadsLearn more about Impala athttp://cloudera.com/impala

    ×