In this session, we discuss our learnings from migrating the financial ledger and accounting system that Amazon uses from Oracle to AWS. We share the performance and cost benefits to enterprises who migrate critical systems from Oracle to AWS, the decision frameworks used to pick the appropriate AWS service for appropriate application, and best practices in project management.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrating Financial and Accounting Systems
from Oracle to Amazon DynamoDB
Phani Bhamidipati
Sr. Software Dev Manager
Amazon.com
D A T 3 4 1
Chris Brack
Software Dev Manager
Amazon.com

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
About Us
DynamoDB – Distributed Uniqueness & Order
DynamoDB, Amazon S3, & SNS – Streaming Aggregation
Pipeline
Amazon S3 & Amazon Athena – Subledger Convergence &
Archival
Data Migration Service (DMS) & RDS – Lift & Shift
Q&A

4. 185 instances
93 databases
120 Terabytes
20+ years of financial data
Nine teams
Largest single-instance offered

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon FLASH team
Financial Ledger & Accounting Systems Hub
Secure, complete, timely and accurate financial statements for Amazon
Historical data for audit
SOX Compliance (Corporate and Auditing Accountability, Responsibility, and
Transparency Act)
Sub-ledger technology to substantiate the general ledger
Systems to support sub-ledger onboarding and accounting needs

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
FLASH customers
Amazon worldwide accounting & finance
Non-technical analyst teams
Technical ops teams
Amazon lines of business
Teams launching new businesses or products
General ledger / Corporate reporting
Securities & Exchange Commission (SEC)
Investor Relations

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data flow
Financial statements~13 MM entries / hour ~200 batches / hour
General
ledger
Transactional
business systems
Retail Digital
ShippingPrime
InventoryMarketplace

8. 13MM entries / hour
3600 TPS
1.15 billion lines / day
50 – 180% YoY growth

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Oracle DBs—Monolithic, expensive
FLASH eventually used largest-available Oracle-certified single instance
hardware
High latency despite optimizations, worsened with YoY traffic growth
Instead of costs reducing over time, saw 10% annual increase, on top of
increasing operational/administrative costs
Software naturally monolithic due to relational DB architecture, prone to
coupling due to logic embedded in queries
Oracle seems simple & cheap until you need to scale it out

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Path to #DBFreedom
Amazon DynamoDB
Fully managed, virtually infinite storage, virtually unlimited throughput
Consistent single-digit responsiveness
Serverless, auto scaling RCU/WCU, point-in-time recovery
PCI, HIPAA, FIPS 140-2 eligible
Encryption at rest and transit
Amazon DynamoDB Streams
Each stream record appears exactly once in the stream
For each item that is modified in a DynamoDB table, the stream records
appear in the same sequence as the actual modifications to the item
Writes stream records in near real time

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Path to #DBFreedom
Amazon S3 & Athena
Serverless, pay-per-query of archive data
Cheap, durable storage in Amazon S3 standard-infrequent storage tier
AWS Database Migration Service (AWS DMS)
Easy lift-and-shift from Oracle to Amazon RDS
SOX-compliant source-to-target controls
Amazon Relational Database Service (Amazon RDS) for
PostgreSQL
Easy setup, easy deployment, frictionless managed database service
Pay only for what you need – no archaic licensing

12. 13 million entries / hour
3600 TPS
200ms to 20ms
Distributed uniqueness & order

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Constraints
• Bounded wall clock time
assigned synchronously for
records upon insert
• Exactly-once persistence &
record versioning
• 100% completeness of
inserted records
• Scalability for YoY growth
up to 2x on peak days
without latency degradation
• Exactly-once, ordered delivery in an
eventually consistent world
• Avoiding long-running stream
processing tasks
• Eventual consistent reads vs.
strongly consistent reads w/
historical record support
• Time skew across Amazon Elastic
Compute Cloud (Amazon EC2)
hosts
Challenges

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Distributed uniqueness & order
GSI
Item
Table Stream
Poller Interval
tracking
Retail Digital
ShippingPrime
And moreMarketplace
Business systems Booking flow Interval processing
Booking host
Durable
backup
KCL workers
Tumbling
windows
Indexer API

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How we leveraged Amazon DynamoDB
• Exactly-once, ordered delivery—DynamoDB Streams
• 100% completeness—two-phase persist with ‘unindexed’ Global
Secondary Index
• Record versioning without strongly consistent read—optimistic locking
• Data migration—simple backfill script
• Historical records & partition size constraints—hot & cold tables
• Time skew—booking hosts w/NTP and consensus algorithm

16. Zero joins
Zero downstream inserts
Streaming architecture
Streaming aggregation pipeline

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming aggregation pipeline
Interval processing General ledger
Business intelligence
Batching pipeline
Event log
Stream
processor
Stream
processor
Interval
processing
Audit log Batch status
Stream Queue
Queue
QueueBatch map
Batch status
Batches Listener
Storage
Storage
Job runner
Job queue
table
Interval
tracking
Interval
artifacts
Batches Batch map

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming w/AWS—Highlights
• Single source of truth—one table, many updates, streamed changes
• Disparate tables consolidated to one table. No more joins
• No more JDBC for downstream systems. Good fences, good neighbors
• Durable audit log with Amazon S3 stream archive without extra tables
• Centric non-determinism. On failures, restart from the original event

19. 74,876,091,283 rows
$300 migration bill
~99% monthly cost savings
Subledger convergence & archival

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Subledger convergence & archival
Migration
fleet
Daily slices
Database
Migration
operator
Dev team
Manifest
PartitionChunk
artifact
Encrypted
chunk
KMS Key
Auditor
Database Database
Migration
tool
Query
Query
Comparison
Oracle to Amazon S3 Validate & Query as needed
Manifests Manifest
Database Database Database

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 & Amazon Athena—Highlights
• Cost-effective, pay-as-you-go. No maintenance
• Query data similar to before. Easy migration verification. JDBC support if
you really want it. Plug & play with others (for example: Amazon
QuickSight)
• Defined common model, persisted as compressed, encrypted, columnar
artifact
• Simple migration tool with manifest (control file) for easy comparison
• Scaling migration is easy—add Amazon EC2 hosts (if Oracle can handle it)

22. Six billion rows
Five days run time
95% cost savings
Lift & shift with AWS DMS & Amazon RDS

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lift & shift w/
AWS DMS &
Amazon RDS
Migration
workflow
PostgreSQL
instance
Stack
Template
Database
Database
Database
DMS Schema
Conversion
Tool
Stack
Stack
Migration
workflow
Migration
workflow
PostgreSQL
instance
PostgreSQL
instance
Operator
Inflight audit
Source to
Target
reports
Easy replicationOne time configuration

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Database Migration Service (AWS DMS) & Amazon
Relational Database Service (Amazon RDS)—Highlights
• Hands-off migration, easy conversion to non-Oracle DB engines
• SOX-compliant source-to-target validation
• One-click setup for manageable partition comparison/audit
• Inflight insights during migration, granular detail for triaging issues
• Pay only for what you use, scale up & down with a few clicks
• AWS CloudFormation integration for effortless multi-region migrations

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon DynamoDB related sessions
DAT201 – A Deep Dive into What’s New for Amazon DynamoDB
DAT303 – Protecting Your Greatest Asset (Your Data): Security Best Practices on DynamoDB
DAT311– Building Serverless Applications with Amazon DynamoDB & AWS Lambda – Workshop
DAT314 – Migrating Your NoSQL Database to Amazon DynamoDB
DAT320 – Becoming a Nimble Giant: How Amazon DynamoDB Serves Nike at Scale
DAT321 – Amazon DynamoDB Under the Hood: How We Built the Hyper-Scale Database of the Future
DAT325 – How Oath (a Verizon Company) Built a Multi-Region GDPR Application with Amazon DynamoDB
DAT332 – Why GE Aviation Migrated from Cassandra to Amazon DynamoDB
DAT333 – Real-World Use Cases for Amazon DynamoDB
DAT341 – Migrating Financial and Accounting Systems from Oracle to Amazon DynamoDB
DAT342 – Implementing Microservice Architectures with Amazon DynamoDB and AWS Lambda
DAT345 – How GumGum Migrated from Cassandra to Amazon DynamoDB
DAT347 – How Amazon Migrated Items and Offers for Retail, Marketplace, and Digital to DynamoDB
DAT349 – Deep Dive on Amazon DynamoDB Global Tables
DAT352 – Migrate Your Nonrelational Database to AWS
DAT357 – Build Internet-Scale Apps with Amazon DynamoDB
DAT365 – Separating Fact from Fiction: Amazon DynamoDB Scaling and Adaptive Capacity
DAT401 – Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB
DAT404 – Advanced Design Patterns for Amazon DynamoDB – Workshop

27. Follow @dynamodb to get the latest

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

29. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phani Bhamidipati
Chris Brack
SDMs, FLASH

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.