Your SlideShare is downloading. ×
Accelerate Digital London Technical Masterclass
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Accelerate Digital London Technical Masterclass

521

Published on

Architecture for Tomorrow …

Architecture for Tomorrow
APIs are Interaction, not just Integration
Big Data
Big Data for Predictive APIs

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
521
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Technical Master Class 17 July 2014 Accelerate Digital London
  • 2. 17.07.2014 Architecture for Tomorrow bernhard@apigee.com
  • 3. • The Challenge • Shortcomings of the current Enterprise IT • Introduction of the "Digital Business Platform" • Location of the Digital Business Platform in the Enterprise Architecture • Digital Business Platform and Orchestration
  • 4. © 2014 Apigee Confidential – All Rights Reserved Partner Integration Web site Wearable Electronics Gaming Consoles Smart Appliances Smart Sensors Smart phones Connected Cars Tablets KiosksAPI API Your Digital Assets API API 4 Your Digital Assets The changing demands for Enterprise IT
  • 5. • The Challenge • Shortcomings of the current Enterprise IT • Introduction of the "Digital Business Platform" • Location of the Digital Business Platform in the Enterprise Architecture • Digital Business Platform and Orchestration
  • 6. © 2014 Apigee Confidential – All Rights Reserved SOA implementation projects • choose tools with care • built well designed services • designed contracts through solid engineering principles Overall system-to-system communication across the company got • decoupled to ease maintenance • properly secured • made reliable One of the current standards of Enterprise IT: SOA SoR Data base Back end SOA Middleware SOA Service SOA Service The created infrastructure is great for complex business logic and process integration e.g. with long running transactions and human interaction
  • 7. © 2014 Apigee Confidential – All Rights Reserved Is your architecture fit for these challenges? 7 Partner Integration Web site Wearable Electronics Gaming Consoles Smart Appliances Smart Sensors Smart phones Connected Cars Tablets KiosksAPI API Your Digital Assets API API SoR Data base Back end SOA Service SOA Service ? ? SOA Middleware
  • 8. © 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved Different Focus 8 DeveloperUser API API Team BackendApp Inside-Out approach Integration Transactional Predictable usage Outside-In approach Consumption Interactive Unpredictable usage External Internal
  • 9. © 2014 Apigee Confidential – All Rights Reserved Aspect SOA / ESB Apps and APIs Some Technical Aspects Enable developers, either external or internal, to build nifty, compelling apps, and allow users to run them High-latency, untrusted (Mobile wireless network) Web technologies, REST, JSON, Javascript Internet standards including oAuth and OpenID Of primary importance Enable Internal developers and systems to connect, while complying with IT department standards Low-latency, trusted SOAP, XML, proprietary Internal mechanisms, WS- Security, LDAP Limited use, secondary importance. Core goal Network Technologies Authentication and Authorization Usage Analytics 9
  • 10. • The Challenge • Shortcomings of the current Enterprise IT • Introduction of the "Digital Business Platform" • Location of the Digital Business Platform in the Enterprise Architecture • Digital Business Platform and Orchestration
  • 11. © 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved Backend Speed up while preserving the existing invests Digital Business Platform layer SOA Middleware layer System of Record YearsMonths IT Applications Partner Apps Mobile Clients "Internet of Things" Days Database
  • 12. © 2014 Apigee Confidential – All Rights Reserved • Fast, light and stateless • Quickly configurable, but at the same time extensible • Improve response time (caching, paging, …) • Self service onboarding for developers • Comprehensive and holistic view on API usage and performance • Support both vertical and horizontal scaling • Deployable across regions, e.g. disaster recovery, geo load balancing • … Selected features of a Digital Business Platform 12
  • 13. © 2014 Apigee Confidential – All Rights Reserved A Digital Business Platform – Logical View Mobile clients / Apps Provide data or need access to digital assets ESB/MQ System of Record System of Record API Router Calls / data needs to be routed to the ideal target API Proxy Some calls / data will be routed to an SOR App Services Custom data for the apps App metrics Apps require metrics API metrics APIs require metrics Data mining Machine Learning Predictive Analytics Data Lake Insights APIs driven by data Developer Portal Devs need access
  • 14. © 2014 Apigee Confidential – All Rights Reserved • To achieve fault tolerance, horizontal scalability and high performance (high throughput, low latency) a Digital Business Platform needs to be thoroughly designed Design of a Digital Business Platform API Router API Proxy Data Platform API Router API Proxy API Router API Proxy runtime db runtime db … … load balancer Metrics Insights Stateless components to easily scale horizontally Replicating NoSQL to be fault tolerant and to scale horizontally (even across regions) Geo location aware, round robin, monitoring API Router/Proxy availability Scalable, Real time access, central data platform for modeling Dev Mgmt
  • 15. • The Challenge • Shortcomings of the current Enterprise IT • Introduction of the "Digital Business Platform" • Location of the Digital Business Platform in the Enterprise Architecture • Digital Business Platform and Orchestration
  • 16. © 2014 Apigee Confidential – All Rights Reserved A Generic Digital Business Platform – Summary API Router API Proxy App Services App metrics API metrics Developer Portal Insights Developer Management API Services API Intelligence
  • 17. © 2014 Apigee Confidential – All Rights Reserved The simplified new Integration Architecture SOA Services External Clients Partner Clients SOA Middle-ware (ESB) Back end API Services Back end Back end Adapter Developer Management API Intelligence SOA Manage- ment Digital Business Platform Integration / Soa Architecture API API Internal Clients Data Plat- form
  • 18. © 2014 Apigee Confidential – All Rights Reserved OUTSIDE DMZ It can can be deployed on premise … External Clients Partner Clients Internal Clients Developer Management API Intelligence API Services SOA Services SOA Middle-ware (ESB) Back end Back end Back end Adapter SOA Manage- ment API API Data Plat- form CORE
  • 19. © 2014 Apigee Confidential – All Rights Reserved DMZ … but for agility reasons cloud is best SOA Services SOA Middle-ware (ESB) Back end Back end Back end Adapter Developer Management API Intelligence SOA Manage- ment API API External Clients Internal Clients API Services Partner Clients OUTSIDE CORE Data Plat- form
  • 20. • The Challenge • Shortcomings of the current Enterprise IT • Introduction of the "Digital Business Platform" • Location of the Digital Business Platform in the Enterprise Architecture • Digital Business Platform and Orchestration
  • 21. © 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved A composite service combines different services to deliver a value-added new service and can be categorized into different types: 1. User Experience centric Mashup • Enhance UI – by adding public cloud data (e.g. maps, stock info, weather info, ...) – by combination of content from different sources (e.g. RSS, ATOM,…) 2. Intermediation / Customization • Enhance services – with public cloud data (e.g. maps, stock info, weather, ...) or partner data – by payload/security mediation to homogenize the service use 3. Arbitration/Aggregation • Unify services by combination in a read-only way or with data changes that are completely independent • Route services to provide one access point onto several different services without knowing implementation details 4. Process Integration/Package • Create new services by tight integration into business workflows • Resource changes will most probably be transactional and depend on each other Definitions 21 Unsafe with depen-dencies Indepen- dently consistent Safe Safe
  • 22. © 2013 Apigee. Confidential – All Rights Reserved© 2013 Apigee. Confidential – All Rights Reserved Placement in the Integration Architecture 22 Type Exposure Layer / API Gateway Implementation Layer / Integration Architecture 1. User Experience centric Mashup Yes No 2. Intermediation/Cust omization Yes Not advised 3. Aggregation/Arbitrat ion Yes Not advised 4. Process Integration No Yes SOA Services SOA Middle- ware (ESB) Back end Back end API API Services Back end Back end Adapter Developer Management API Intelligence SOA Manage- ment API API Back end API DIGITAL BUSINESS PLATFORM
  • 23. © 2014 Apigee Confidential – All Rights Reserved • The need for higher speed and agility demands for a new layer / system in the Enterprise IT • The Digital Business Platform optimizes access for a large amount of different devices and an increasing number of business partners from the outside of the Enterprise • It has to be deployed in a fault tolerant, high available, horizontally scalable manner to be prepared for the load of the near future • It complements existing capabilities in the architecture: – Services exposed by Middleware / SOA – (Big) Data platform for data analysis / data mining / machine learning Summary 23
  • 24. 17 July 2014 APIs are Interaction, not just Integration S
  • 25. © 2013 Apigee Confidential – All Rights Reserved Agenda 25 Interaction Explosion of devices - User centric Developers Empowered developer - Usability is key Integration Ubiquitous - Cheap and easy - Integration Explosion
  • 26. Interaction
  • 27. © 2013 Apigee Confidential – All Rights Reserved Partner Integration Web site Wearable Electronics Gaming Consoles Smart Appliances Smart Sensors Smart phones Connected Cars Tablets KiosksAPI API Your Digital Assets API API 27 Your Digital Assets App and Partner Interaction take Centre Stage
  • 28. © 2013 Apigee Confidential – All Rights Reserved Interaction Patterns Govern APIs Backend Services App Server Web Partner ESB Mobile API A API B API C API Tier
  • 29. © 2013 Apigee Confidential – All Rights Reserved Interaction Patterns Govern APIs Backend Services App Server Web Partner ESB Mobile API A API B API C API Tier Chattiness Mash Ups and Aggregation Data & Format Feedback Loop (email, push notification) Security
  • 30. Developers
  • 31. © 2013 Apigee Confidential – All Rights Reserved Developers are the Key Actor 31 DeveloperUser API API Team BackendApp
  • 32. © 2013 Apigee Confidential – All Rights Reserved This applies … 32 Jakob's Law of the Internet User Experience: Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know. Developers spend most of their time developing against other APIs. This means that developers prefer your API to work the same way as all the other APIs they already know.
  • 33. 33
  • 34. 34 Application developers are raison d'être for APIs.
  • 35. 35 Be pragmatic. For the benefit of application developers.
  • 36. Design – intuitive and simple Consumption – ease of use and relevance Packaging – developer and use case centric Security – simple and consistent
  • 37. © 2013 Apigee Confidential – All Rights Reserved API Design & Pragmatic REST 37 • https://groups.google.com/forum/#!forum/api-craft • http://info.apigee.com/Portals/62317/docs/web%20api.pdf • http://www.youtube.com/user/apigee
  • 38. APIs are Integration Technology
  • 39. © 2013 Apigee Confidential – All Rights Reserved APIs – Easy Integration 39 Complex and Visible CORBA, RMI, RPC Simple and Abstract EAI, EJB, ESB Simple and Visible REST/HTTP Integration is hard however System Impedance and Remote Nature is obvious
  • 40. © 2013 Apigee Confidential – All Rights Reserved APIs – Easy Integration 40 Complex and Visible CORBA, RMI, RPC Simple and Abstract EAI, EJB, ESB Simple and Visible REST/HTTP Integration is easier System Impedance and Remote Nature is hidden
  • 41. © 2013 Apigee Confidential – All Rights Reserved APIs – Easy Integration 41 Complex and Visible CORBA, RMI, RPC Simple and Abstract EAI, EJB, ESB Simple and Visible REST/HTTP Integration is easier System Impedance and Remote Nature is hidden Abstractions break!
  • 42. © 2013 Apigee Confidential – All Rights Reserved APIs – Easy Integration 42 Complex and Visible CORBA, RMI, RPC Simple and Abstract EAI, EJB, ESB Simple and Visible REST/HTTP Easy integration based on simple, standardised principlesUsed everywhere!
  • 43. © 2013 Apigee Confidential – All Rights Reserved App Backend APIs are Everywhere API
  • 44. © 2013 Apigee Confidential – All Rights Reserved App Front end Backend APIs are Everywhere App Backend API API
  • 45. © 2013 Apigee Confidential – All Rights Reserved App Front end Backend AppApp APIs are Everywhere API API APIAPI
  • 46. © 2013 Apigee Confidential – All Rights Reserved Front end Backend AppApp Service Service Service Service Service Service Service Service Service Micro service architecture APIs APIs APIAPI API API APIs are Everywhere
  • 47. APIs not an Integration Platform
  • 48. © 2013 Apigee Confidential – All Rights Reserved • Every Developer is an Integration Developer – Bye bye centralized integration platform – Hello point to point integration • Integrations are use case centric – Bye bye canonical data model – Bye bye one size fits all interaction model What’s the consequence of ubiquitous APIs?
  • 49. © 2013 Apigee Confidential – All Rights Reserved Front end Backend AppApp Service Service Service Service Service Service Service Service Service Micro service architecture APIs APIs APIAPI API API The API Landscape will shift! Requires • Making APIs consumable and accessible • Adding governance
  • 50. © 2013 Apigee Confidential – All Rights Reserved Add Visibility, Governance and Decoupling Backend Services App Server Web apps Social apps ESB Mobile apps Persistence Security Orchestration Analytics All Apps API Tier
  • 51. © 2013 Apigee Confidential – All Rights Reserved Wrap Up 51 Interaction Explosion of devices - User centric API Design Empowered developer - Usability is key Integration Ubiquitous - Cheap and easy - Integration Explosion
  • 52. Your Questions?
  • 53. Thank you 17 July 2014
  • 54. Data: Powering the Digital Economy Pankaj Sodhi, Accenture UK/I Big Data Lead @AccentureDigiUK
  • 55. Every business is a Digital Business
  • 56. Data is the lifeblood of the organisation Most business leaders – CEO, CMO and CFOs now view data as among their most valuable assets
  • 57. Copyright © 2014 Accenture All rights reserved. 57 Data Supply Chain Data should flow easily and usefully through the entire organisation 80% of high performing companies are investing in analytics, yet only 40% day they are getting the value that they want
  • 58. Application Service Interface Layer In-Memory IMDB, CEP, Cache IMDB, CEP Distributed Cache In-Memory DB Streaming CEP Graph Stream 1 Pass Stream Big Data Core Query Engine In-Memory Analytics InteractiveBulk Transfer Sources Streaming Organisations can choose from different data technology components to build the architecture needed to support data acceleration. These include big data platforms, complex event processing, ingestion, in-memory databases, cache clusters, and appliances.
  • 59. Decision Learning Discovery Memory Perception Optimization Prediction Causality Tagging Filtering Raw Data Actionable Insights Data Ingestion Capabilities Data Discovery & Visual Analytics Capabilities Advanced Data Analytics Capabilities Known Business Questions Business Hypothesis Data Raising Interesting Questions Insight Generation Patterns in the Data Operational Analytics Opportunities Accenture Data Discovery
  • 60. Accenture’s Data Discovery Platform
  • 61. Advanced Visualizations Get deeper insights from analytics by using interactive best practice advanced visualizations that enable drill-to-detail. For maximum effectiveness use these visualizations in combination with blended data and advanced analytic models. Text & Unstructured Segmentation/Clustering Predictions & Simulations Behavior Networks and Relationships Geospatial Treemaps and scatter plots visualize aggregate and individuals size, proximity and concentration of clusters Parallel coordinates visualize patterns in multivariate discrete and continuous datasets with parameterized colors Visualize location and geographic network data on multi-layered maps give insights into event concentrations. Parameterize time-series predictions for ‘What-if’ analysis. Confidence bands visualize the models strength Find relationship clusters and patterns with network diagrams are used chord charts for hierarchical entity relationships Text clouds combined with sentiment scores are powerful for generating insights from social and unstructured data.
  • 62. Thames Water Copyright © 2013 Accenture All rights reserved.
  • 63. @AccentureDigiUK Join the Conversation
  • 64. July 25, 2014 Apigee Insights
  • 65. Who we are Helps business quickly achieve digital acceleration, with scale, insights and agility More API deployments run on Apigee than any other platform Customers include 20% of the Fortune 100, 50% of the top global brands, 50% of the top retailers Customer-driven product leadership with APIs and Big Data Over 100k developers in the fast growing Apigee developer community Products Services Knowledge
  • 66. Apigee customers span multiple industries 66Updated 5/31/14
  • 67. Agenda Predictive Analytics on Big Data, enabling a new generation of businesses Customer Success Apigee Insights
  • 68. Enterprises need to operate differently to stay relevant 68 Omni-channel AgilityIndividualization Individualized interactions Relevant experiences Accelerate decisions Proactive response Adaptive processes Continual learning Be where customers are Holistic view Contextual journey Proliferation of Mobile and Digital Accelerating Pace of Change Evolving Customer Expectations
  • 69. Predictive Analytics on Big Data key to operating differently 69 DeveloperUser API API Team Backend Predictive Insight Hadoop Data Warehouse AppAdaptive App Data Scientist Adaptive API
  • 70. Existing Big Data investments lack key capabilities 70 Data Lake / Data Warehouse • Big Data Infrastructure • Lack specialized data structures to analyze fine-grain event data • Lower precision resulting from predictive analytics on summarized, tabular data • Difficult to incorporate signals from unstructured data in analysis • Difficult to access insights in real-time • Difficult to integrate predictive insights into apps and APIs Business Intelligence • BI Visualization & Reporting Data Science • Statistical Analysis • Scientific Visualization ChallengesCurrent Big Data Investments
  • 71. 71 In addition, many challenges to address Best Practices Global Deployment Unstructured Data Processors Real Time Processing Continual Improvement APIs & Automation Sophisticated Machine Learning Model Building Monetization Security System Management Big Data Structure Adaptive Apps & APIs
  • 72. Agenda Predictive Analytics on Big Data, enabling a new generation of businesses Customer Success Apigee Insights
  • 73. Business & Technical User Rapidly realize value API Team Apps Developer Develop more precise insights Discover complex hidden patterns Data Scientist Apigee Insights: Enable adaptive apps and APIs Hadoop Data Warehouse Other Developer & Business User Tools Predictive & Descriptive Analytics Graph & Sequence Processing (GRASP)
  • 74. Business User Tools Developer Tools RESTful APIs Graph Machine Learning Modeling Workbench Real Time Scoring Data Loaders Unstructured Data Processor Temporal Graph Database Apigee Insights: Enable adaptive apps and APIs Business & Technical User API Team Apps Developer Data Scientist Hadoop Data Warehouse Other Predictive & Descriptive Analytics Developer & Business User Tools Graph & Sequence Processing (GRASP)
  • 75. How do you detect hidden patterns and relationships? 75 Data Data • How do you discover hidden patterns in interactions across touch points & channels? • How do you understand the customer journey across siloed channels? • How do you deliver a contextual journey vs. channel optimized interactions?
  • 76. • Unique big data structure (GRASP) purpose-built to analyze fine-grain time-aware interaction data – at scale • Identify complex hidden patterns and relationships in customer behavior • Dramatically improve ability to predict future actions Use specialized big data structure to analyze customer journey 76 PROFILE ConsumerID: U56 Gender: M Geo: San Francisco Interests: Bikes PROFILE ConsumerID: U60 Gender: F Interests: News Age: 35-40 2 4 1 3 40 0 3 View product A on web Buy product A on mobile Call customer service Return Product A in store 3 2 4 1 0 Time GRASP (Graph and Sequence Processing) time-sequenced graph database on Hadoop Fine-grain event and entity data
  • 77. How do you improve prediction accuracy? 77 • How do you avoid losing precision when working with sampled or summarized data? • How do you detect signals hidden in unstructured data such as text? • How do you prevent prediction accuracy from degrading over time?
  • 78. Apply powerful machine learning on fine-grained event data • Significantly improve prediction accuracy by applying powerful machine learning algorithms directly on fine-grained event data in GRASP • Always deliver fresh, relevant interactions with a continual learning feedback loop 78 Rules Based Approach Traditional Statistical Analytics Apigee Insights Precision Traditional Analytics Apigee Insights (Machine Learning on Big Data) Static Model Dynamic Model Focuses only on top attributes Leverages both strong and weak signals Batch only Batch and real time Adapts to changing conditions Easier to put into action Significantly better results
  • 79. Discover hidden signals in unstructured data 79 MBRS payment was rcvd the same day as dunning took effect. Dunning was not posted until after payment was processed. Final notice sent out after payment was sent out as well. Payment was sent out and postmarked before due date. The member was termed correctly. 88% 45% With Unstructured Data Without Unstructured Data Prediction Accuracy (Likelihood for member to complain) Healthcare Payer Example 2x • Unstructured data is difficult to analyze but holds valuable signals • Improve prediction accuracy by extracting signals hidden in text
  • 80. How do you realize value from data? 80 • How do you enable developers to build new adaptive apps and APIs? • How do you enable business users and analysts to interact with predictive insights? • How do you integrate predictive insights into existing apps and systems? Predictive Insights Business Value
  • 81. Access insights in real-time 81 • In-memory real-time processor to enable fast access to insights • Regularly updated to always provide fresh, relevant scores Real-time access to insights using NoSQL Propensity Free Shipping 10% Off Churn User 1 0.72 0.68 0.33 User 2 0.56 0.23 0.55 User 3 0.32 0.45 0.67 User 4 0.20 0.32 0.18 User 5 0.44 0.69 0.22
  • 82. Easily consume insights via APIs and visualization 82 Predictive insights Descriptive insights • Use RESTful APIs to build new adaptive apps and APIs or extend existing ones • Enable developers and business users to act on predictive insights Build new adaptive apps and APIs, extend existing ones, or use or extend visualization tools provided
  • 83. Deliver individualized interactions across multiple channels 83 Direct Mail Email Web Mobile Outreach Fine grained event data Individualized interactions across multiple channels Machine Learning on GRASP Predictive Models Targeting Recommendations Churn Other Advanced 3 2 4 1 0 Propensity Free Shipping 10% Off Churn User 1 0.72 0.68 0.33 User 2 0.56 0.23 0.55 User 3 0.32 0.45 0.67 User 4 0.20 0.32 0.18 User 5 0.44 0.69 0.22 0 View product page on web 1 Receive mobile offer 2 Call customer service 3 Agent note: dissatisfied 4 Return product in- store 5 Purchase Online Customer Profile
  • 84. Agenda Predictive Analytics on Big Data, enabling a new generation of businesses Customer Success Apigee Insights
  • 85. 85 IBC: Proactive Customer Service with Predictive Analytics “Apigee has helped us move from simply answering customers’ calls to proactively reaching out to our members before they have issues.” Somesh Nigam, SVP and Chief Informatics Officer, Independence Blue Cross • $1.2B Medicare Advantage business • 50+% Reduction in complaints to Medicare • Transformed from reactive to proactive customer service • Identified root causes for dissatisfaction
  • 86. Typical Use Cases Proactive Churn Reduction Reduce churn by predicting customers likely to be at-risk and proactively responding to them Individualized Targeting & Recommendations Improve customer retention, cross-sell, up-sell, loyalty, and acquisition 20% increase in profit Individualized daily deals emails Leading Online Services Company 100% increase in revenue & profit Individualized website recommendations Leading Shopping Site 164% lift in conversions 60% more clicks, 55% fewer emails Recommended products - game launch Leading Gaming Company 50% fewer complaints 6x more precise predictions 1 month in advance Leading Healthcare Insurer
  • 87. Apigee Insights 87 Deep Predictive Analytics Expertise • Team led Big Data @ Yahoo!, IBM, and others • Thought leaders in machine learning and big data • Extensive Vertical expertise from client engagements Accelerate Business Growth • Increase revenue, profitability and customer satisfaction • Individualized targeting and recommendations • Proactive Churn Reduction Develop Adaptive Apps & APIs • GRASP on Hadoop • Machine Learning on GRASP • Unstructured & API data • Real Time • APIs and Automation • Cloud or on-premise Apigee Insights
  • 88. Thank you

×