The SAP Startup Focus Program – Tackling Big Data With the Power of Small by Soenke Moosmann


Published on

Geared exclusively towards helping startups master big data to the benefit of their users, the SAP Startup Focus Program has truly gone global since its initiation in March 2012. The in-memory database platform SAP HANA forms the basis of this initiative.

Marcus and Sönke from the SAP Innovation Center will introduce the program and provide technical insights into the unique capabilities of SAP HANA in a hands-on manner.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Data ComplexityBig dataStructured and unstructuredNoiseHidden patternsAlgorithmic ComplexityMore complex information needs: Predictive, simulationsComplex workflows: provisioning data, cleaning, denoising, pattern miningPresentation ComplexityHow to present complex results in an intellible way
  • Critical slide!!!Developing a database to solve these two critical challenges requires a careful design and development from the ground up of every aspect of the database. Relabeling an existing DB “in-memory” doesn’t do it. Carful optimizing for optimal cache utilization and for hundreds of parallel threads is what makes the difference, and allows HNA to reach the speeds I just discussed. I can’t over-emphasize hwo important solving these two challenges is to the performance of SAP HANA.
  • By accessing data in column-store order, you benefit immensely from simplified table-scan and data pre-caching. This can make all the difference in performance.
  • Intuitively design complex predictive modelsRead and write from data stored in SAP HANA, Universes, IQ, and other sourcesDrag-and-drop visual interface for data selection, preparation, and processing
  • Visualize, discover, and share hidden insightsAdvanced visualization designed where you’d expect it – natively from within the modelling toolShare insights via PMML and with other BI client tools
  • Predictive Analytics is a Goldmine for Startups (Forbes, March 2013)
  • Predictive Analytics is a Goldmine for Startups (Forbes, March 2013)
  • The SAP Startup Focus Program – Tackling Big Data With the Power of Small by Soenke Moosmann

    1. 1. The SAP Startup Focus Program –Tackling Big Data With the Power of SmallMarcus Krug and Sönke Moosmann, SAP Innovation Center
    2. 2. © 2013 SAP AG. All rights reserved. 2PublicAgenda SAP Innovation Center Why Startups Matter To Us? SAP Startups Focus Program (SFP)
    3. 3. SAP Innovation Center
    4. 4. © 2013 SAP AG. All rights reserved. 4PublicWho We Are Established in February 2011 First of its kind for SAP AG Now: 35 FTEs Soon: 100 FTEs plus 200students
    5. 5. © 2013 SAP AG. All rights reserved. 5PublicLooking Beyond ERPPersonalizedMedicineOnlineGamingFilmProductionSmartEnergySupply-ChainInnovationTechnologies In-Memory-Technology Cloud Mobility
    6. 6. SAP and StartupsHow Does That Fit Together?
    7. 7. © 2013 SAP AG. All rights reserved. 7PublicSome Might See Us as… Grow old SlowBUT TURTLES AREALSO… Quite „fundamental“,according to ancientAsian mythology
    8. 8. © 2013 SAP AG. All rights reserved. 8PublicCommonalities - Innovation….Business(viable)Technology(feasible)InnovationHuman Values(desirable,usable)
    9. 9. © 2013 SAP AG. All rights reserved. 9PublicBIG DataVideoAudioDemandContentGPSCustomer DataServiceCallsEmailsVirtual GoodsSocial MediaMobileInstantMessagesSAP HANA
    10. 10. © 2013 SAP AG. All rights reserved. 10PublicBIG AmbitionsSAP Strategic Goals 2015 20 Bn in Revenue 35% Margin 1,000,000,000 Users→ Dramatically extend SAP‘s ecosystemAnd if we join forces…
    11. 11. © 2013 SAP AG. All rights reserved. 11PublicThis is What Will Happen…
    12. 12. SAP Startup Focus ProgramWhat Startups Can Expect From Us
    13. 13. © 2013 SAP AG. All rights reserved. 13PublicAccess…1. Technology Many startups face „big data“ and „real-time“ challenges → SAPHANA2. Customers SAP Install Base of close to 200,000 customers across 25 industries3. Financing SAP Ventures
    14. 14. © 2013 SAP AG. All rights reserved. 14PublicSFP – All About AccessTechnology SAP HANA One Developer Edition (free) → 1 year per default (flexible) Physical and virtual HANA bootcamps, HANA virtual learning platform technical advisorJoint Go-to-Market Appearances at SAP and non-SAP events Solution showcases on HANA marketplace Dedicated GTM advisor → GTM plan Pipeline creation to drive SFP startups‘ revenueAccess to SAP Ventures and other VCs SAP Ventures and other VCs (155 Mio $ HANA Real-time Fund)
    15. 15. © 2013 SAP AG. All rights reserved. 15PublicSFP – From Idea to (Market) ImpactAttend StartupForumDevelopmentAccelerator GTMBoot CampSFP ≈ 1 year ≥1 year: commercial stateMarket-readysolutionSFP selectionPitch at Forum
    16. 16. © 2013 SAP AG. All rights reserved. 16PublicSFP – Then, Now and BeyondMarch - May 2012- The first 10startups- Recruited fromfriends + family- High touchMarch – Sept 2012- From 10 -100- startup forumsheld globally- HANA boot camps- HANA developeredition on AWSSapphire EMEA2012- ≈150 startups- 50 with Proof-of-Concept2013 – Transitionto Scale!As of now:- 200+ startups inSFP- 60+ productivesolutionsGoals 2013- 1000+ startups inSFP- 200+ productivesolutions
    17. 17. © 2013 SAP AG. All rights reserved. 17PublicSome References(Israel)(US) (US)(France)(UK)(UK)(Israel)(US)(Canada)(US) (US) (Germany)etc….
    18. 18. SAP HANACrunching Big Data Made Easy
    19. 19. © 2013 SAP AG. All rights reserved. 19PublicThe Microscope :: A Tool for Biological ExplorationBefore the invention of the microscope Difficult to study tiny structures Only models, hypotheses about Cells Micro-organisms Difficult to verify / falsify hypothesesAfter the invention of the microscope Tiny structures are plain to see Can be studied in real time
    20. 20. Challenges... and how HANA helps tackle these
    21. 21. © 2013 SAP AG. All rights reserved. 21PublicData ChallengeCRM* dataGPSDemandSpeedVelocityTransactionsOpportunitiesServicecallsCustomerSales ordersInventoryE-mailsTweetsPlanningThingsMobileInstantmessagesVELOCITYVOLUME VARIETY
    22. 22. © 2013 SAP AG. All rights reserved. 22PublicAlgorithmic ChallengeChallengesForecastingKeyInfluencersTrendsAnomaliesRelationships
    23. 23. © 2013 SAP AG. All rights reserved. 23PublicPresentation Challenge
    24. 24. © 2013 SAP AG. All rights reserved. 24PublicApplicationServerSAP HANA OverviewPredictiveanalyticsScriptingDataModelingR IntegrationMathLibrariesColumn androw store+Multi-core/parallelizationIn-memoryCompressionSQL interface oncolumns & rowsSQLTTextEngine
    25. 25. Data ChallengesHow do you crunch big data in real time?
    26. 26. © 2013 SAP AG. All rights reserved. 27PublicOptimizing Data Access PatternsChallenge: Data locality! Yes, DRAM is 100,000 times faster than disk… But DRAM access is still 4-60 times slower than on-chip caches
    27. 27. © 2013 SAP AG. All rights reserved. 28Public SAP HANA supports rows, but is optimized for column-order data organizationOrder Country Product Sales456 France corn 1000457 Italy wheat 900458 Italy corn 600459 Spain rice 800Column and Row Store456 France corn 1000457 Italy wheat 900458 Italy corn 600459 Spain rice 800456457458459FranceItalyItalySpaincornwheatcornrice1000900600800Row order organizationColumn order organizationSingle-record access:SELECT * FROM SalesOrdersWHERE Order = ‘457’SQLSingle-scan aggregation:SELECT Country, SUM(sales) FROMSalesOrders WHERE Product=‘corn’ GROUP BYCountry
    28. 28. © 2013 SAP AG. All rights reserved. 29PublicCombining OLTP and OLAP Write operations are accumulatedin a dedicated data structure (deltastore) Write operations are insert-only! Integration of differential data in async.merge process. MVCC enables processing ofOLTP workloads Insert only approach favors implementation ofMVCCMain Memoryat Blade iLogSnapshotsPassive Data (History)Non-VolatileMemoryRecoveryLoggingTimetravelDataagingQuery Execution Metadata TA ManagerInterface Services and Session ManagementDistribution Layerat Blade iMain Store DifferentialStoreActive DataMergeColumnColumnCombinedColumnColumnColumnCombinedColumnIndexesInvertedObjectData Guide
    29. 29. © 2013 SAP AG. All rights reserved. 30PublicOrder Country Product Sales456 France corn 1000457 Italy wheat 900458 Spain rice 600459 Italy rice 800460 Denmark corn 500461 Denmark rice 600462 Belgium rice 600463 Italy rice 1100… … … …Columnar Dictionary Compression Dictionary per column Uses data-driven fixed-length bit encodings Operations directly on compressed data, using integers More in cache, less main memory access1 Belgium2 Denmark3 France4 Italy5 Spain1 32 43 54 45 26 27 18 4… …1 72 5,63 14 2,4,85 3Logical TableDictionary5 entries, soneed 3 bits toencode!Compressedcolumn(bit fields)InvertedindexDictionaryWhere wasorder 460?Which ordersin Italy?
    30. 30. © 2013 SAP AG. All rights reserved. 31PublicMore Columnar Compression Techniques
    31. 31. © 2013 SAP AG. All rights reserved. 32Public Concurrent users Concurrent operations within a query Data partitioning, on one host 
ordistributed to multiple hosts Horizontal and vertical 
parallelization of asingle query
operation, usingmultiple
cores / threadsTransparent to developerParallelizationInter Transaction Intra TransactionInter Query Intra QueryInter Operation Intra OperationPipelineParallelismData ParallelismPipelineParallelismData ParallelismParallelism
    32. 32. Algorithmic ChallengesGoing Beyond Descriptive
    33. 33. © 2013 SAP AG. All rights reserved. 34PublicExtending your analytics capabilitiesANALYTICS MATURITYLEVELOFINSIGHTSense & Respond Predict & ActRawDataCleanedDataStandardReportsAd HocReports &OLAPGenericPredictiveAnalyticsPredictiveModelingOptimizationWhat happened?Why did it happen?What will happen?What is the bestthat couldhappen?
    34. 34. © 2013 SAP AG. All rights reserved. 35PublicSAP HANA ModelingAnalytical View Attribute View Column TableCalculation View
    35. 35. © 2013 SAP AG. All rights reserved. 36PublicImplementing Predictive AnalyticsSQLScript set of SQL extensions to push data-intensive logic into the database and leverage parallelexecution strategies of the databasePAL (Predictive Analysis Library) Built-in C++ statistical and data mining algorithms K-means, k-nearest neighbor, decision trees, multiple linear regression, classification, and manymoreR integration Leverage R’s 3000+ external packages to perform wide-range data mining and statisticalanalysis.
    36. 36. © 2013 SAP AG. All rights reserved. 37PublicIntuitively Design Predictive Models using Predictive Analysis
    37. 37. © 2013 SAP AG. All rights reserved. 38PublicSAP Predictive Analysis
    38. 38. Presentation ChallengeDesign HANA apps easy and fast
    39. 39. © 2013 SAP AG. All rights reserved. 40PublicSAP HANA XS EngineRationale: Enable application developmentand deployment – minimize layers HTTP-based UI (browser, mobile apps) Runs directly on HANA, minimizes TCO Leverages built-in strengths of SAP HANAfor the best possible performanceScope From lightweight environment for small web-based applications To robust environment for complex high-speed business applicationsControl flow logicCalculation logicDataClientsPresentation logicHANAXS
    40. 40. © 2013 SAP AG. All rights reserved. 41PublicImplement UIs with SAPUI5SAPUI5 is an extensible JavaScript-based HTML5browser rendering library for BusinessApplications. Uses the jQuery library as a foundation Open AJAX compliant and can be used togetherwith/uses other standard JS libs Supports RIA like client-side features based onJavaScript Supports an extensibility concept regardingcustom controls Allows usage of own JavaScript and HTMLInternetExplorerVersion 9Version 8ChromeLatestversionFirefoxVersion 3.6and latestversionSafariLatestversion
    41. 41. © 2013 SAP AG. All rights reserved. 42PublicSAPUI5 Templates
    42. 42. © 2013 SAP AG. All rights reserved. 43PublicDemo :: In-Game Promotion ManagementBigpoint Europe‘s largest browser game provider with 300 Miousers Revenue through selling „virtual goods“ 1-3% of users are buying virtual offers.Goals Perform real-time massive amount of event streamanalytics Filter and monitor online players Enabling in-game promotion offers and track results inreal-time
    43. 43. © 2013 SAP AG. All rights reserved. 44PublicIn SummaryIn-memory data management Column-oriented data layout Compression Parallelization Optimized for big data Transparent to developerHANA Applications XS Engine Application Server Control logic SAPUI5 Reduced TCOPredictive Analytics SQLScript R integration Predictive AnalysisLibrary Unlock new insights
    44. 44. © 2013 SAP AG. All rights reserved. 45PublicTake Your Chance!The SAP Startup Forum is coming toBerlin again! When: June 19th Where: SAP Office Berlin(Rosenthaler Str. 30) For more info, visit our event website: Or turn to us directly
    45. 45. Thank you!Sönke MoosmannSAP Innovation Centersoenke.moosmann@sap.comMarcus KrugSAP Innovation