Downloaded 34 times
Uploaded bySnapLogic
The Impact of SMACT on the Data Management Stack
The document discusses the changing landscape of data management and integration challenges driven by technologies like big data, cloud computing, and the Internet of Things. It outlines the limitations of legacy integration approaches and emphasizes the need for modern solutions that can accommodate the rapid growth and complexity of data. SnapLogic is presented as a flexible integration platform that addresses these challenges by streamlining data and application integration through innovative design and user-friendly self-service capabilities.
More Related Content
![[Infographic] Cloud Integration Drivers and Requirements in 2015](https://cdn.slidesharecdn.com/ss_thumbnails/cloudintegrationdriversandrequirementsin2015-141217184007-conversion-gate02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to The Impact of SMACT on the Data Management Stack
The Impact of SMACT on the Data Management Stack
- 1. The Impact ofSMACT on the Data Management Stack Darren Cunningham, SnapLogic
- 2. About Me •Responsible for Marketing @ SnapLogic o Co-founded by Gaurav Dhillon, co-founder and former CEO of Informatica o www.SnapLogic.com • Prior to SnapLogic: o 4 years at Informatica (Cloud Team) o 2 years at LucidEra (Cloud BI) o 1 year at Salesforce (Analytics / AppExchange) o 7 years at Business Objects
- 3. The New EnterpriseIntegration Challenges Big Data Access and Analytics Disconnected SaaS Silos API Proliferation The Internet of Things
- 4. We Recently PostedThis Infographic…
- 5. of line ofbusiness employees admit to using non-approved SaaS applications in their jobs The global Hadoop Market is estimated to reach $50.2 billion by 2020. It was valued at $1.5 billion in 2012 and is expected to grow at a CAGR of 58.2% between 2013 to 2020
- 6. The Integrator’s Dilemma Old Approaches Not Built for the New Data Challenges Legacy EAI • Not built for the web • On-prem ESB (xml, soap) • Code-intensive Legacy ETL • Built for rows and columns • Batch-oriented • Struggles with real-time
- 7. The Integrator’s Dilemmais faced when legacy integration solutions are no longer effective in the new world of Social, Mobile, Analytics (Big Data), Cloud and the Internet of Things 2003 2014
- 8. So What’s Changed? 1 Speed
- 9.
- 10. So What’s Changed? 2 User (and Buyer) Expectations Citizen integrators are proliferating, in most cases outside of the control or visibility of whoever in the company is commissioned with fulfilling integration.
- 11. So What’s Changed? T3he Data (Volume, Velocity, Variety, right?) Top 5 Use Cases: 1) Customer Analytics 2) Operational Analytics: Understanding Machines, Devices and Human Interactions 3) Fraud and Compliance 4) Data-Driven Products & Services 5) EDW Optimization Source: Datameer
- 12. So What’s Changed? C4loudification and “Data Gravity” (iPaaS, elastic scale, opex) CRM HRMS ITSM SCM Analytics ERP?
- 13. The Cloud AnalyticsWave Analytics
- 14. So What’s Changed? 5Standards, Protocols and Architectural Styles
- 15. Whitepaper on CloudIntegration • From XML to JSON • From SOAP to REST • From ESB to iPaaS Working on ETL Paper…stay tuned! Buses Don’t Fly in the Cloud Why the ESB Is the Wrong Approach For Cloud Integration
- 16. What’s the Same?It’s All About the Pipes!
- 17. Your Integration Strategy: Innovation On Ramp or Roadblock?
- 18.
- 19. Option • Determineif you’re suffering from the “Integrator’s Dilemma” o Multiple teams and tools for EAI, SOA, ESB / ETL, ELT; old tech used to solve new challenges • Review your ICC / COE practices o Command and control won’t fly in the age of the “Citizen Integrator” and need for speed • Do an audit of your cloud applications currently in use o Find out how/if they’re integrated and how it’s going (typically many SaaS silos) • Dig into Hadoop o The economics and scale will make it a key component of your data infrastructure • Investigate AWS Redshift other cloud DW / BI options
- 20. SnapLogic Elastic Integration Single Platform Connecting Data, Apps, APIs
- 21. SnapLogic: Connecting Data,Apps and APIs • Experienced Team: Leadership from Informatica, Salesforce, Microsoft • Investors: Andreessen Horowitz & Ignition • Advisory Board: AstraZeneca, HP, Symantec, Yahoo • Headquarters: San Mateo, CA • Customers: Adobe, Acxiom, Blackberry, Bloomin’ Brands, CapitalOne, Cisco, GE, iRobot, Netflix…
- 22. Why SnapLogic ElasticIntegration? Fast Multi-Point Modern Easily Design Monitor, Manage Deploy in Days not Months • EAI, ETL, APIs, Hybrid Deployment • Modern Standards: REST, JSON • Scale Out Architecture REST SOAP WEB APIs
- 23. Hybrid Architecture •Streams: No data is stored/cached • Secure: 100% standards-based • Elastic: Scales out & handles data and app integration use cases Metadata Data
- 24. Software Defined Integration Integration Cloud Designer, Manager, Dashboards (Multi-tenant cloud service) Elastic Execution Snaplex Snaps (Buy or Build) “Cloudplex” “Hadooplex” “Groundplex”
- 25. SnapReduce and theHadooplex Acquire, Prepare, Deliver Big Data YARN YARN MapReduce MapReduce MapReduce MapReduce MapReduce MapReduce MapReduce MapReduce Snaplex YARN Application MapReduce Generation Snaplogic iPaaS + Hadoop = Snaplex Container
- 26. Common SnapLogic UseCases Cloud App Integration • Workday: HR On- Boarding • Salesforce: CRM Back Office • Eliminate SaaS Silos Digital Marketing • AWS Redshift • Tableau, Social, CRM • Cloud Analytics Big Data Analytics Enterprise Platform • Data Access • Data Preparation • Data Delivery • Self Service • Data, Apps, APIs • Integrator’s Solution
- 27. We looked atSnapLogic as an opportunity to think differently about integration. With a document-centric processing approach and schema-less platform, we’ve been able to eliminate some of the rigidity and time-consuming tasks related to traditional integration patterns. – Jim Teal, Information Architect, iRobot $555M revenue, 534 employees
- 28. Designs and buildsrobots that make a difference in people’s lives Challenges Faced: • Geographically distributed manufacturing • Wanted to improve data quality and eliminate FTP and VPN channels • Growing cloud app adoption Why SnapLogic? • Flexibility: Control plane in Boston, data planes in China, schema-less approach • Connectivity: Standards connectivity, eliminated VPNs, etc. • Productivity: 68 to 6 pipelines
- 29. Largest apparel brandever built on the web in the United States. • 1 month vs. 6m-1yr • Reduction in headcount to administer and maintain • Lowers the barrier to entry for advanced analytics Speed Flexibility Scale Amazon Redshift Reporting Platform Analytics Platform Data Integration Platform Real-Time Dashboards
- 30. 50+ stores servingneeds of Hispanic community in Southern CA and AZ Challenges Faced: How to integrate all of the company's applications and keep its 24/7 operations running smoothly. Why SnapLogic? Speed: Northgate was able to achieve a significant improvement in sales order throughput and accuracy. Productivity: In a matter of months, the company went from 30 pipelines to 64, all while adding 14 additional stores Connectivity: Modernized and connected the ERP and WMS systems in only 12 weeks
- 31. The Impact ofSMACT on the Data Management Stack? • Presents an opportunity to re-think your approach to data and application integration • Means you need to accept / embrace self-service data access (citizen integrators) • Data, App, API integration converging quickly • Innovation is back in the DI market (so don’t settle for same old, same old – SO SO Integration)
- 32. Discussion @SnapLogic Facebook.com/SnapLogic Plus.google.com/+SnapLog ic
Editor's Notes
- #2 This session will introduce the concept of the "Integrator's Dilemma" and review some of the challenges faced by traditional data and application integration technologies when it comes to keeping up with the new enterprise data, application and API connectivity and management requirements. We'll review the landscape and share examples of the steps more and more IT organizations are taking to improve business alignment through faster access to trusted data.
- #7 This need for both cloud integration power and the need for speed and simplicity has lead to an “Integrator’s Dilemma” for many of the IT organizations we talk to today. The good news is there’s now an awareness that without the right approach to integration, the promise of SaaS and cloud computing will not be met. The bad news is that their existing legacy middleware technologies were conceptualized and built before the SMAC stack became an enterprise reality – Social, Mobile, Analytics/Big Data and Cloud Computing. When it comes to Legacy Enterprise App Integration tools, “buses don’t fly.” On-premises enterprise service bus (ESB) are known to be brittle and code-intensive. They were designed to speak XML and SOAP, not the more modern web protocols like JSON and REST. They weren’t designed to run at cloud speed. When it comes to Legacy Extract, Transformation and Loading tools, they’re great for large batches of structured data – rows and columns. This works well for initial migrations and periodic data loading requirements, but customers inevitably want both real-time and batch application and data integration capabilities. According to a recent Gartner report, (read the quote on the slide) - “Organizations are increasingly turning to iPaaS offerings because of their close affinity with SaaS and the anticipated greater ease of use, lower costs and faster time-to-integration than traditional integration platforms”. http://www.snaplogic.com/dilemma .
- #12 http://datascience.berkeley.edu/what-is-big-data/ Use Case #1: Customer Analytics To improve customer conversion rates, personalize campaigns, reduce customer churn, and more, marketers need to analyze data from lots of customer interaction points like mobile, social media, stores, and e-commerce sites. With big data analytics from Datameer, you can aggregate and analyze all of this data at once, yielding insights you never had before – for example, who are your high-value customers, what motivates them to buy, how they behave, and how to best reach them. Use Case #2: Operational Analytics: Understanding Machines, Devices and Human Interactions Manufacturing, operations, service and product executives face intense pressure to optimize asset utilization, budgets, performance and service quality. IT executives can help by using big data analytics to unlock insights buried in log, sensor and machine data and structured CRM, ERP, other data. Datameer customers are detecting outliers; running time series and root cause analyses; and parsing, transforming and visualizing data for insights that improve asset management decisions and ROI. Use Case #3: Fraud and Compliance Data-driven insights can help you uncover what’s hidden and suspicious – and in time to mitigate risks. With big data analytics, you can combine, integrate and analyze all of your risk-related data at once to generate the insights and metrics needed to detect fraud and compliance issues. For example, you can perform time series analysis, data profiling and accuracy calculations, data standardization, root cause analysis, breach detection, and fraud scoring, identity verifications, risk profiles, and data visualizations. Use Case #4: Data-Driven Products and Services Savvy companies are leveraging big data analytics to create new, data-driven products and services that differentiate their business and drive revenue. For example, a media company is using Datameer to provide brands and advertisers with reports about how customers behave using mobile apps so they can optimize ads and boost responses. And a leading provider of enterprise cloud applications uses Datameer to provide customers with reports on how end users are actually using their software. Use Case #5: EDW Optimization EDWs are critical business and IT resources today – but as the size and complexity of the data to be analyzed increases, you’ll eventually hit the limits of traditional data warehouses. By offloading the most challenging data management and analytics activities to big data analytics solutions like Datameer that runs on Hadoop, you can cost effectively scale to any volume of data and store and analyze any and all data types together – both structured and unstructured. http://www.datanami.com/2014/10/06/top-five-big-data-analytics-use-case-yield-high-returns/
- #27 Fix into colums
- #29 “We’ve been able to eliminate some of the rigidity and time-consuming tasks related to traditional integration patterns.” – Jim Teal, Information Architect, iRobot
- #30 Architecture diagram - SnapLogic pulls data from all of the services, CSVs, APIs, databases and pushes it into all of the BI tools Good Data is the primary BI tool Easy to use web interface (minimal training) easy to maintain Tableau for deeper business analysts – internal case studies Real-time reporting – Gecko Board (API calls for quick dashboards) Everything goes into Redshift Good Data is subset, more highly aggregated Use Python to do the data science on Redshift Predictive, product recommendation “I would rather work with the business than with the hardware!” – David Glueck, Sr. Director of Data Science and Engineering, Bonobos
- #31 . "Future development is going to be faster and cheaper because we can reuse the Snaps," Lewis said. The IT staff is small -- only one person is fully focused on integration -- and that works because the SnapLogic support staff fills in the gaps, “Working with SnapLogic has enabled us to make better use of both our on-premise and cloud applications and drive greater benefits from our overall investments.” – Harrison Lewis, CIO, Northgate Markets