Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Secure Big Data Analytics - Hadoop & Intel

2,167 views

Published on

Our Keynote presentation at Gartner Catalyst

Published in: Technology
  • Be the first to comment

Secure Big Data Analytics - Hadoop & Intel

  1. 1. Secure Big Data Analytics Combining APIs, Security and Big Data + Data Center Software Division1
  2. 2. Two Red Hot Trends - How do they Intersect? API Management Big Data Analytics • Enterprise extending reach • Increased Volume, Variety, ? through APIs Velocity of unstructured data • API traffic overtaking web traffic • Drivers: mobile, cloud, social • Defacto communication for • Tremendous ROI mobile to server How does this effect application architecture to support growth?2
  3. 3. Big Data Fundamentals Traditional Data Analysis Big Data Analysis Unstructured Cluster Relational Data Analyze Database Warehouse Organize AnalyzeTransaction Batch Streaming Devices (MapReduce) • Structured data • Unstructured, variety of data: “mashup” • Data ~ GBs to TBs • Data ~ TBs to PBs • Centralized: Data moves to analytics • Distributed: Analytics move to the data • Batch analytics • Streaming analytics Focus ventures in one of two “Only business model tech has left” areas: monetization of data or infrastructure to enable monetization of data March 12, 20123
  4. 4. Today’s Big Data Tools & Hurdles New BI Tools “Big Data” includes tools like Hadoop, NOSQL technologies, massive parallel processing, and in-memory databases Existing Hurdles with Hadoop • Job Control - Enable clients to run jobs with security controls • Data On-ramping - Get data into Hadoop for processing, from internal sources, cloud services or network-connected devices • Data Off-ramping - Data availability to clients via APIs, suitable for mobile applications • Security and Compliance - Big Data processing provides PII protection, data security and PCI compliance4
  5. 5. Connecting Data Movement: Back End to Device to ALL Departments 1 Problem: Today’s platforms are 2 Problem: Data and Potential value fragmented and not securely locked in fragmented solutions inhibit connected, limiting scale E2E analytics Dept Dept Dept Dept Dept Dept Dept Dept A B A B A B A B Retail platform Home Energy Platform Telco Service Provider Smart City 10k devices, 1M customers 300K home pilot in Germany Real-time CDR: 12TB/ day 3000+ cameras, 1PB/3mo API Control Point Analytics Edge Devices NB/ULT Phone Cameras Kiosk PoS DS API Control Point5
  6. 6. API/Service Gateway Fundamentals Service API Data Mediation Management Transformation • Consistent policy enforcement for API CENTRALIZED across Service Gateway Central Proxy departments Enterprise • Use Models: CSB, ESB-light, Edge Security, API Gateway Monetization/Charge Back App Service Gov & Integration Security, Access, Compliance Developer Community • Meter usage • API management • Edge threat protection • Configuration not code • Throttle per SLAs • Policy creation & exe • Data Loss Protection • Discovery of aggregated • API Analytics • Legacy & SOA integration • Federated ID Brokering services from IT • Orchestrate & transform • PCI PII Data Tokenization • Meta data • Protocol translation Move from Line of Business to “Enterprise” Wide6 API Mgt & Utilization of Analytics
  7. 7. Last Mile Device Mobile Middleware • High Performance • Version Management • Content Optimization • Quality of Service • Ubiquitous Compatibility • External Cloud Service Support7
  8. 8. Information Greed • Greedy Users: Instant response from touch-screens, context aware smart phones, etc • Greedy Business: Expect real time intelligence on the consumer derived from social, data warehouses, and data mining Addressing this greed requires new thinking for how to build Composite Applications8
  9. 9. Composite Distributed Application Apps • Hybridized – New functionality with legacy code and data • Location Independent- 1-n clouds (private and public) and datacenters simultaneously • Knowledge Complete - Access to disparate “Big Data” warehouses owned by the business • Contextual – Produces just-in-time results based on client context, e.g. identity and location • Accessible & Performs – Produces data compatible with any client on any operating system, and does it instantaneously • Secure and Compliant - Meets compliance and security requirements for data in transit and data at rest Realizing composite apps can be done with a service gateway, which secures, brokers and mediates data for API access, and a Hadoop Cluster which provides data analysis and processing9
  10. 10. “APIfication” of real-time Hadoop datasets PaaS Services Internal Client (Storage, RDMS) Users HTTP/REST Smartphone interactions with Network- & JSON Results Connected Tablet Clients Devices Partner Web Services Data On-ramping from the cloud with Types of Clients selective protection (FPE/Tokenization) Service Gateway Gateway Control Point DMZ Hadoop API Job Scheduler Legacy Apps and RDBMS IDM Web Services Metadata Server Existing Apps, Data and Infrastructure Node1 Node2 Node3 HDFS10
  11. 11. Pulling it all Together: Ref Arch for Composite Apps11
  12. 12. Field Case Study Secure ‘Big Data’ Storage and REST API • Authenticate IP cameras based on IP address, 2-way SSL or message security • Codeless insertion and retrieval to and from HBase. Drag and drop with no Java coding • Expose ‘Big Data’ using a REST facade, ideal for native mobile applications and partner services • Provide a secure REST API with authentication and authorization based on OAuth and internal identity stores such as LDAP12
  13. 13. Suggested Roadmap to Composite Apps & Big Data13
  14. 14. More: www.cloudsecurity.intel.com Gartner Cloud Service Broker API Patterns Secure Big Data Hype Cycle White Paper Solution Brief14

×