What is Big Data for Turkey and Sybase IQ?


Published on

This is a presentation that I have presented during IDC Big Data Event on Dec 2012 on behalf of Sybase Turkey

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Ortaya çıkan ilk gerçek ; 83% ile büyük veriyi direk hacimsel büyüklük ile eşleştirdiğimizdir. Katılımcıların sadece 11% unstructured yani yapısal olmayan verileri büyük veri kapsamında değerlendiriyor . 6% lık kesime büyük veri verinin depolanmasını anımsatıyor. Ekranda gördüğümüz kotasyonlar ise araştırma sırasında karşılaşılanlardan sadece bazıları. Nitekim 20TB hacimden büyük hacimdeki veriye büyük veri diyenler var. Yada yapısal olmayan yönetimi zor veriyi büyük veri olarak adlandıranlar var. 83% lük kesimde yer alanların 36% sı ise zaten büyük veriye cevabını çok spesifik rakamlarla TB olarak vermiş..
  • V
  • What is Big Data for Turkey and Sybase IQ?

    1. 1. What is Big Data?Muzaffer YöntemSybase Turkey
    2. 2. Research FindingsIDC – August 2012, Turkey
    3. 3. Data VolumeQ: How many Tbytes of data does your company index per day on average? 2% 12% 20 + Tbyte per day 11 - 20 Tbyte per day 6 - 10 Tbyte per day 1 - 5 Tbyte per day < 1 Tbyte per day 86%© 2012 SAP AG. All rights reserved. 5
    4. 4. Data Variety: priorityQ: How important is structured/unstructured data analysis? 100% 6% 14% 90% 80% 22% 70% 60% 60% Not important 50% Somewhat important 40% 71% Very important 30% 20% 10% 26% 0% Structured data Unstructured data© 2012 SAP AG. All rights reserved. 6
    5. 5. Data Variety: data sourcesQ: What type of data do you analyze today? Data that comes from relational databases 74% Production data (analysis of e.g. quality data) 62% E-mail 58% Internal documents 32% Customer and market surveys 26% Social media (blogs etc.) 22% Content on internal web sites 18% Web logs 18% Pictures 16% News articles 16% On-line forum 14% Other (please specify): 6% 0% 10% 20% 30% 40% 50% 60% 70% 80%© 2012 SAP AG. All rights reserved. 7
    6. 6. Data SolutionsQ: What types of data solutions are you using for Big Data analysis? Traditional relational databases 64% BI solution/front-end tools (e.g. Business Objects) 30% Appliances 14% In-memory databases 8% Column databases 6% Other (please specify): 0% 0% 10% 20% 30% 40% 50% 60% 70%© 2012 SAP AG. All rights reserved. 8
    7. 7. Data PlansQ: Is your organization planning any Q: If yes, when do you envisionBig Data Analysis projects? implementation? Don’t know 10% Yes Planning but no targeted 24% 17% implementation date yet 18 to 24 months 0% 12 to Less than 18 Months 8% 6 to Less than 12 Months 42% No Within 6 months 33% plans 66% 0% 10% 20% 30% 40% 50%© 2012 SAP AG. All rights reserved. 9
    8. 8. © 2012 SAP AG. All rights reserved. 10
    9. 9. Big Data inReal-Time© 2011 SAP AG. All rights reserved. 11
    10. 10. SAP Real-Time Data Platform Architectural Components SAP SAP SAP Big Business SAP SAP Custom Business Data Warehouse Analytics Mobile Apps 3rd Party Suite Applications BI Clients SAP NetWeaver (On Premise / Cloud) Present SAP Real-Time Data Platform Open Developer API’s and Protocols Process Sybase PowerDesigner SAP Sybase Common Landscape Common Modeling Scale-Out HADOOP SQLA NoSQL Management SAP Sybase MPP SAP Sybase ASE SAP HANA Platform IQ SAP Sybase Store ESP SAP Sybase SAP Data SAP MDG, MDM, DQ Replication Server Services Enterprise Information Management Ingest© 2012 SAP AG. All rights reserved. 12
    11. 11. SAP Sybase IQ : A complete “Near-Time” platform for data analytics use cases Sybase PowerDesigner, SAS, SPSS, K Bradmark, Sy Sybase Replication XEN, Fuzzy mantec, White BMMSoft, Server, Logix, Zementi SAP BusinessObjects sands, Quest, SOLIX, PBS s, Visual ISYS, Panopticon ZEND Numerics Optimized Dev and admin Predictive Packaged ILM BI,EIM, Model, Re EcoSystem tools Analytics apps plicate App Services Hadoop, DBMS R Built-in Full InDB Analytics w/ Comprehensive Web 2.0 Big Data Text MapReduce + ANSI SQL w/OLAP APIs OpnSrc APIs Search simulator Most mature High Structured + Comprehensive MPP queries + Virtual column Speed Unstructured lifecycle tiering Marts + User scaling store loads Store© 2012 SAP AG. All rights reserved. 13
    12. 12. SAP Sybase IQ : Architecture SAP Sybase IQ Engine Data is Stored Vertically • Each column is stored separately  Bit-Mapped Index Web Enabled Analytics  Index on every column  FAST ACCESS and LOAD Communications & Security Information Lifecycle Management Optimized Storage Multiplex Grid Management Administration Framework • Input data is compressed Loading Query Engine Engine  Usually = 40%-70% • Database smaller than input data Text Search  Even with all the indexes In-Database Analytics Query Engine Retrieves Only Columns Used in the Query Column Indexing • Reduces system I/O dramatically Sub-system  Average 90% less than competition • Permits better data manipulation Column Storage Processor  Easy to alter and manage Storage Area Network Schema Design Not Restricted • Design based on application use Flat, Star, Relational, Snowflake Any Schema© 2012 SAP AG. All rights reserved. 14
    13. 13. Would you like to take an action even before data is being stored ?© 2012 SAP AG. All rights reserved. 15
    14. 14. Our Customer Examples
    15. 15. comScore Digital Business Analytics 40% compression Business Challenges  Providing customers quick results to help market more effectively and generate business  Supporting intense data reporting  Technical Challenges 147 TB data warehouse providing  Scalability, performance and data compression industry’s most preferred service  Handling 150GB of data daily Benefit 150 GB of data  Economically scales to large amounts of data and supports data- loaded daily intense reporting while controlling costs  Largest data warehouse running on an NT platform  Industry’s most preferred audience measurement service – beating the nearest competitor by a margin of 25%“ ”We have tremendous amounts of data – more data than anyone will ever see. The only database that can handle this volume of data withease is SAP Sybase IQVP comScore © 2012 SAP AG. All rights reserved. 17
    16. 16. SunGard Leading software and IT services company Business Challenges  Enabling the building of newer and larger systems – allowing expansion into new markets and business areas. 1 Trillion rows data stored  Technical Challenges  Handle very large and continuously growing volumes of data without performance degradation.  Existing system began to experience performance deterioration that was unacceptable to end-users 80 TB of compressed data Benefit  Slashes query response time regardless of data volumes  Enables analytics and reporting against virtually unlimited data“ ” SAP Sybase IQ is simple to manage and operate and it’s enabling us to easily build really big systems in a way that is cost- effective, manageable and sustainable…It doesn’t matter what we throw at it, it seems to take it in stride and give us a great response…We feel like it’s a solution that will carry us forward into uncharted territory. We see no limit to how far we can go with it. Product Architect, SunGard © 2012 SAP AG. All rights reserved. 18
    17. 17. A leading Turkish customerBusiness Analytics 60+% compression Business Challenges  Providing users fast results to sustain market leading position and generate profitable business  Supporting intense ad hoc reporting  Technical Challenges 65+ TB compressed  Scalability, performance and data compression data warehouse  Handling 1.5 – 2.5 TB of data daily (ELT)  50,000+ queries daily 1.5+ TB of data Benefit transformed and loaded daily  Economically scales to large amounts of data and supports data- (25billion+ intense reporting while controlling costs transactions)  Reduction of HW+Storage+IT Staff to be utilized in other projects“ ”© 2012 SAP AG. All rights reserved. 19
    18. 18. BNP Paribas Corporate and Investment Banking Financing, advisory and capital markets services Business Challenges  Provide senior management with a real-time view of liquidities and the bank’s cash position in order to implement appropriate hedging strategies 600 x increase in  Fully optimize operational and financial risk management processes reporting speeds to ensure highest levels of performance  Technical Challenges  Providing access for more than 200 users daily to make various types of requests including immediate, pre-defined, complex, and 75% reduction in ad hoc reports. business intelligence storage requirements  Benefit  Ensures required reliability and security for risk management assessments  Guarantees improved analytical decision-making capacity capabilities for senior management“ ” The performance of SAP Sybase IQ, combined with SAP Business Objects, is crucial for our banking activities. The two technologies allow us to obtain a centralized view of liquidity risk that we can now manage in real-time Applications Manager, BNP Paribas Corporate and Investment Banking © 2012 SAP AG. All rights reserved. 20
    19. 19. SAP SYBASE IQ CUSTOMERS© 2012 SAP AG. All rights reserved. 21
    20. 20. © 2012 SAP AG. All rights reserved. 22
    21. 21. THANK YOU© 2012 SAP AG. All rights reserved. 23