Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

California Breakfast Seminar

587 views

Published on

The applications being built today have drastically different requirements than those built five, 10 or 20 years ago.

In this seminar, you heard from Chief Architect Steve Emmerich about how this company that processes $13 trillion daily is reconsidering database deployment architecture and working toward an always-on, high-performance, elastically scalable database.

You also heard NuoDB CTO Seth Proctor discuss key database points to consider when building or migrating applications for the cloud and modern data center, including:
- Spanning multiple geographies / data centers
- Delivering elasticity
- Providing high performance even with heavy application usage
- Offering around-the-clock resiliency and uptime

Published in: Technology
  • Be the first to comment

California Breakfast Seminar

  1. 1. June 2015! Steve Emmerich! Chief Architect / Technical Fellow! ACI Worldwide Inc.! Data Management for Mission-Critical Real-Time Financial Systems!
  2. 2. Who am I, Steve Emmerich? ●  Involved in data analysis/management software since 1970 ●  High School, University, Grad School studies in Computer Science ●  Consistent computer-related interests ●  Overcoming I/O challenges of Super- & Parallel-processing ●  Persistence technologies that support OLTP, analytics ●  I/O architecture matching other hardware trends (e.g. Cloud) ●  Design (anti-) patterns for non-functional requirements ●  Continuous learning, teaching, mentoring, influencing ●  Professional experience ●  Engineering (leadership) in UNIX / microprocessor / parallel processing startups ●  Founded and ran data warehousing/business intelligence firm ●  General manager of national systems integration practice ●  Chief Architect, VP Engineering for quite a few data-centric firms ●  Currently Chief Architect at ACI Worldwide Inc. ●  Helped establish strong foundational NFR focus ●  Leading many aspects of data management strategy ●  Influencing software architecture across 20+ strategic areas 2
  3. 3. Who (in the World) is ACI Worldwide Inc.? $1B Payment Software Provider (Maybe the World’s Largest) Global customer base in 80+ countries on 6 continents 40 years of payments expertise 24x7x365 global support organization 4,600+ customers use hosted solutions ~$1 Billion Revenue 18% on R&D annually 21 of the world’s top 25 banks rely on ACI software; many payment processors as well Prevent fraud for 360+ payment organizations worldwide Serve 300+ retailers globally Handle bill pay for 3,600+ organizations
  4. 4. Four Market Segments of ACI’s Business … ACI delivers solutions that serve four broad but distinct market segments PAYMENT RISK MANAGEMENT Consumer Bank ** Transaction Bank † Retailers Billers FRAMEWORK, MOBILITY & TOOLS HOSTED & ON PREMISE SUPPORT MODELS ** Includes Consumer-facing payment ecosystem players, such as Card payment acquirers and issuers, payment processors, associations, etc. † Includes Business-facing payment ecosystem players, such as Treasury departments of corporations, network and institutional intermediaries supporting “wire” payments, ACH payments, etc.
  5. 5. Channels and Interfaces ACI’s Legacy: 30-Year Old Real-Time Architecture BASE24 ATM Auth DB Bank’s System Other Systems HOST HSM ATM POS Networks On Tandem Non-Stop Architecture … ACI’s original claim to fame in the Banking and Merchant Retail worlds, NSK (TAL) only BASE24 ATM Auth DB Bank’s System Other Systems HOST HSM Asynchronous Replication
  6. 6. router Integrated Server Integrated Server SAN Websphere MQ Websphere MQ OS Clustering DB Server DB Server ICE-XS ICE-XS High-Availability Cluster Multi-Processing ACI’s Next-Generation Switch: “Base24-eps” … C++, database-neutral, runs on modern HW architectures, more or less infinitely scalable router Integrated Server Integrated Server SAN Websphere MQ Websphere MQ OS Clustering DB Server DB Server ICE-XS ICE-XS High-Availability Cluster Multi-Processing Asynchronous Replication
  7. 7. BASE24-eps Performance/Scalability ●  Deeply performance-architected with no known performance inhibitors ●  Generalized resource partitioning (no bottlenecks) ●  Symmetric, no process- or memory-affinities that create asymmetries ●  Potential database bottleneck mitigated by extensive partitioning of application journals ●  Throughput ●  >= 2000+ sustained business transactions-per-second / “node” ●  Linear resource consumption per transaction ●  No resource constraints at peak (other than cpu) ●  Response-time (latency) ●  Extremely low, stable response times ●  50-100ms at peak load ●  The lower the latency, the more room in the transaction path for additional value-added services (e.g. Fraud detection) … Requirements include completely predictable performance and scalability, with no spikes Fast, scalable authorization & switching of payment transactions
  8. 8. ●  Continuous availability ●  No single points of failure ●  Real-time logic and schema changes ●  Strong geo-distribution with active/active support ●  Elimination of both planned and unplanned outages ●  Removes the need for fault-tolerant hardware ●  System behavior under stress predictable and manageable ●  Queuing to handle volume spikes ●  Application is first line of defense ●  No single point of failure in transaction path ●  Context/state-free processing ●  Designed from ground-up for High Availability at all levels ●  Network Overload Management (NOM) at endpoints BASE24(-eps) Availability … Requirements include withholding permission for downtime (planned or unplanned) No-fail authorization & switching of payment transactions with deployment-time adaptability
  9. 9. ●  Deployment-time/On-line logic updates ●  Rolling process changes ●  Instantaneous introduction of new authorization scripts ●  Dynamic addition and removal of business services ●  Deployment-time/On-line application and schema changes ●  Application-level database versioning ●  Real-time table conversions ●  Rolling software upgrades ●  Deployment-time/On-line configuration changes ●  Command-level or bulk BASE24(-eps) Availability Requirement No-fail authorization & switching of payment transactions with deployment-time adaptability … Requirements include withholding permission for downtime (planned or unplanned)
  10. 10. Next-Gen Architecture Requirements for ACI ●  Experience since Base24-eps indicate that real unmet business needs exist ●  Away from discrete systems for siloed LOBs and payment types and towards combined functionalities, to avoid redundancy ●  Towards real-time (“Immediate”, “Faster”) payments – not just payment initiation, but also clearing, settlement, reconciliation ●  Towards any-to-any payments (e.g. persons and/or businesses) ●  Towards truly global (notwithstanding cross-border challenges) ●  Existing payment systems will remain and be very important. But the traditional LOB boundaries will diminish ●  Example: solutions for Merchant Retail POS and BillPay support will converge (since the only essential difference as far as the consumer is concerned is when payment is made) ●  Example: the need for and trend towards real-time payments will apply to any type of payment (high or low value, high or low volume). These types of payments have historically been handled by different systems.
  11. 11. Implications of Change for Next-Gen Architecture ●  Imperative: create an architecture that enables satisfying both continuing needs of existing payments ecosystems and new payments ecosystems, that overcomes the following impediments to business and technical change ●  Payment service consumers: some don’t want change ●  Payment service suppliers: existing LOBs threatened by change ●  Regulatory: government-led change varies greatly by region ●  Integration: legacy systems are hard to replace ●  Nevertheless, our customers are continually demanding much more comprehensive functionalities that span our traditional segments (Consumer, Transaction, Biller, Merchant Retail) ●  Deployment architectures desired by customers are trended towards outsourced, hosted, “Cloud-based” elastic deployments that shield them from security threats
  12. 12. Implications of Change for Next-Gen Architecture ●  Conclusions: ●  Move towards integration of discrete functionalities (“products”) into configurable “solutions” that – at deployment-time, based on specific customer needs – blends multiple functionalities. ●  Support elastic, hosted deployment architecture that enable customers to outsource security threats and in which elastic system resources can be applied transparently
  13. 13. CHANNELS BATCH MOBILE ONLINE BRANCH POS ATM Bill Pay P2P FI Core Systems Batch Network Debit / Credit Remittances SOA Business Services SOA Business Services SOA Business Services SOA Business Services SOA Business Services SOA Business Services SOA Business Services SOA Business Services SOA Business Services Payment Information Model Monitoring Management Templates Session Orchestration Reporting ACI’s Next-Gen Architecture: SOA + Framework … Meets the need to continue to support incumbent requirements and support unmet needs
  14. 14. ConfidentialMEETS THE CHALLENGE OF CHANGE Component Architecture Channel Interfaces Network Interfaces ATM POS MobileOnlineBranch Debit / Credit Bill Pay P2PFI Core BatchRemittances Payment Services and Frameworks Service Enabled Solutions Consumer Payments Transaction Services Transaction Security Customer Account Journal Services Instrument Verify Limits and Velocities Transaction Banking Template Management Liquidity Management ACH Services Wire Services Sanctions Filtering Exception Management Retailers Transaction Services Loyalty Services Value Card Management Refund Services Wallet Management Cheque Processing Billers Bill Payment Bill Presentment VCA Services Account Validation Liquidity Management Scheduled Payments Payments Risk RT Fraud Analysis NRT Fraud Analysis Entity Block Services Account Activity Fraud Scoring Demographic Profile Omni-Channel Teller Services Platform Services Balances Transfers Cheque Services Online Banking Services Consolidated Payment Data Management Core Infrastructure and Common Services But what are the requirements for this piece? … Meets the need for SOLUTIONS that support incumbent requirements and unmet needs
  15. 15. ConfidentialMEETS THE CHALLENGE OF CHANGE Consolidated Payment Data Management Shared Service Data Shared Solution Data Data Management Requirements Users Fraud Tokenization Entitlements Notification Accounts Customers Transactions Config Data Availability Security ManageabilityPerformance ScalabilityMulti Tenancy Service Data Supportability Contributing Apps UOB CG MCM Bill Pay Private Data Private Data Private Data Private Data … Supports the data needs for SOLUTIONS that support incumbent and unmet needs Distributed Deployment with SQL and Transactional Access Required
  16. 16. Performance/Scalability Needs at the Data Tier ●  Deeply performance-architected with no known performance inhibitors ●  Generalized resource partitioning (no bottlenecks) ●  Symmetric, no process- or memory-affinities that create asymmetries ●  Potential database bottleneck mitigated by extensive partitioning of database journals ●  Throughput ●  >= 20000+ (10X) sustained business transactions-per-second ●  Linear resource consumption per transaction ●  No adverse impact of one component’s workload on another’s throughput ●  Response-time (latency) ●  Extremely low, stable response times ●  No adverse impact of one component’s workload on another’s latency … Requirements include completely predictable performance and scalability, with no spikes Fast, scalable payment transactions (anticipating future volumes across all solution components, not just one discrete product)
  17. 17. ●  Continuous availability ●  No single points of failure ●  Real-time logic and schema changes ●  Strong geo-distribution with active/active support ●  Elimination of both planned and unplanned outages ●  Removes the need for fault-tolerant hardware ●  System behavior under stress predictable and manageable ●  Queuing to handle volume spikes ●  Database becomes first line of defense ●  No single point of failure in transaction path ●  Context/state-free processing ●  Designed from ground-up for High Availability at all levels ●  Network Overload Management (NOM) at endpoints … Requirements include withholding permission for downtime (planned or unplanned) No-fail operation at the data-tier with deployment-time adaptability Availability Needs at the Data Tier
  18. 18. ●  Deployment-time/On-line logic updates ●  Rolling process changes ●  Instantaneous introduction of new authorization scripts ●  Dynamic addition and removal of business services ●  Deployment-time/On-line application and schema changes ●  Application-level database versioning ●  Real-time table conversions ●  Rolling software upgrades ●  Deployment-time/On-line configuration changes ●  Command-level or bulk BASE24(-eps) Availability Requirement … Requirements include withholding permission for downtime (planned or unplanned) No-fail operation at the data-tier
  19. 19. Elasticity Needs at the Data Tier ●  Enable transparent run-time elasticity of transaction processing bandwidth ●  Enable transparent, run-time elasticity of I/O bandwidth ●  Enable transparent, run-time elasticity of availability … Requirements include completely predictable performance and scalability, with no spikes Fast, scalable payment transactions (with elasticity of both capacity and availability)
  20. 20. Q & A
  21. 21. Architecting for the Cloud Seth Proctor, CTO @technicallyseth
  22. 22. What’s unique about “cloud”?
  23. 23. Cloud architecture   On-demand   Scale-out for capacity & availability   Public infrastructure; dynamic provisioning   Flexible   Commodity   Hybrid (public & private)   Simple   Monitoring & management   Platform APIs and automation   Resilient
  24. 24. Goals   All the reasons Steve cited…   Greater capacity   Cost-effectiveness   Higher availability and better failure- handling   Lower latencies for global deployment   Online upgrade & evolution   Hybrid workloads
  25. 25. Challenges   Distribution brings challenges   Lots of failures happen with frequency   More difficult to get a global view   Security & data lifecycle is harder   Everything else about “distributed computing”   Still, we can scale most layers   Load-balancers & name services at the top   Horizontally-scaled app servers   Caches & CDNs for content   Redundant disks and object stores
  26. 26. Scaling the database is the real challenge
  27. 27. Traditional database design   RDBMS architectures start at the disk   Vertical scale follows   Caching helps, but often breaks consistency   HA systems become very expensive   Schema & operation is hard to evolve   Hard to harness commodity infrastructure   Not designed to scale-out
  28. 28. Common options   Replication   Active-passive or (gulp) multi-master   Replicated data but visible delays & conflict Sharding   Split one database into many sub-sets   More capacity but hard to evolve and relate   Abandon consistency   Push correctness & conflict to the application   Simpler core architecture but painful for applications and hard to reconcile failures
  29. 29. Side-effects   Applications are tied to deployment   Driver for dev-ops   Complex for on-demand changes, failures   More, independent pieces   Harder to interpret failures   Complexity
  30. 30. Global deployment   Many motivations   Disaster Recovery   Lower-latency for distributed users   Data access & storage residency rules   Trade-offs between latencies and safety or consistency   Storage should be separate from service
  31. 31. Approach Shared Disk Shared-Nothing/ Sharded Durable Distributed Cache Key Idea Sharing a file system. Independent databases for disjoint subsets of data. Replicating data in memory on- demand. Topology Example Oracle RAC DB2 Pure Scale MySQL Cluster and most NoSQL/NewSQL solutions Distributed Database Designs *Note: Most major web properties include custom-sharded MySQL or sharded PostgreSQL, including Facebook, GOOGLE, Wikipedia, Amazon, Flickr, Box.net, and Heroku.   11
  32. 32. Peer to Peer Architecture P P P S3Disk , ... P P NuoDB Database Peer Process Provisioned, Manageable Resources Peer to Peer Communications SQL Client Management Client SQL Front-End SQL Optimizer Transaction Handling Object Caching Object Coordination Durability P
  33. 33. Magic Quadrant 2013 About NuoDB Magic Quadrant 2013 & 2014 NuoDB delivers a distributed SQL database management system specifically designed for the cloud and the modern datacenter. Magic Quadrant 2013
  34. 34. Summary   When architecting for the cloud..   Look for distributed architectures with on- demand capabilities   Layer & abstract to support evolution and react gracefully to failures   Assume your needs will evolve; plan with scale in mind   Please try out NuoDB!   http://dev.nuodb.com
  35. 35. Thank you!

×