The document discusses different database models and their common use cases. It describes relational, key-value, document, in-memory, graph, and time-series database models. For each model it provides examples of common data structures, features, and example use cases such as ERP/CRM systems, shopping carts, content management, fraud detection, and IoT applications. The goal is to help readers understand which database model to select based on their specific data and requirements.
자 그럼 지금부터 본격적인 세션을 진행할텐데요.
오늘 세션 순서에 대해 먼저 설명을 드리도록 하겠습니다.
Amazon.com 회장이신 Andy Jessy는 2018년 11월 9일 Amazon Consumer Business (Retail, Ecommerce, Seller) 가 Oracle Datawarehouse를 Redshift 로 전환하였고, 2018년 말까지 88% 오라클 DB를 Aurora 와 DyanmoDB로 전환한다고 발표하였습니다.
아마존 닷컴은 몸소 database 다변화 전략을 보여주었습니다.
보시는 주요 서비스들에서 탈 상용 Database를 실천하였고
수익을 발생시키는 Tier1 서비스인 경우에는 NoSQL인 DynamoDB를
수익에 간접적으로 기여하는 서비스인 경우에는 Open Source PostgreSQL 을 채택하였습니다.
Wallet은
Prime Video 는
Advertising 은
Buyer Fraud는
Items and Offer는
FLASH는
https://aws.amazon.com/solutions/case-studies/AmazonWalletService/
https://aws.amazon.com/solutions/case-studies/prime_video_dynamoDB/
https://aws.amazon.com/solutions/case-studies/Amazonadvertising/
https://aws.amazon.com/solutions/case-studies/AmazonBuyerFraud/
https://aws.amazon.com/solutions/case-studies/itemsandoffers/
https://aws.amazon.com/solutions/case-studies/AmazonFinancialSystems/
https://aws.amazon.com/solutions/case-studies/AmazonFinancialSystems/
The Financial Ledger and Accounting Systems Hub (FLASH) is a suite of microservices that ingests these financial transactions, performs complex and business-critical functions to substantiate the general ledger, and generates financial statements such as balance sheet, cash flow, and income.
Over the years, FLASH grew to require more than 90 Oracle databases, totaling more than 120 terabytes of data and growing. “Simply maintaining and administering the database technology took hundreds of hours per month,” says Marisamy Krishnan, senior technical program manager at Amazon.
Migrated more than 120 TB of financial data
Reduced latency by 40%
Cut admin overhead by 70%
Cut costs by 70%
Customers are migrating their workloads to AWS in record numbers
Verizon is migrating over 1,000 business-critical applications and database backend systems to AWS, several of which also include the migration of production databases to Amazon Aurora—AWS’s relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases.
Wappa, a market leader in taxi expense management, available in 22 countries, migrated from their Oracle database to Amazon Aurora and improved their reporting time per user by 75 percent.
Trimble, a global leader in telematics solutions, had a significant investment in on-premises hardware running Oracle databases. Rather than refresh the hardware and renew their licenses, they opted to migrate the databases to Amazon RDS and project they will pay about 1/4th of what we were paying when managing their private infrastructure.
Thomas Publishing is using AWS to save hundreds of thousands of dollars in data center costs, quickly launch new websites by spinning up resources in one day instead of several weeks, and rapidly migrate critical Oracle applications to the cloud. The company links industrial buyers and suppliers by offering an array of information and services online and in print. Thomas runs its primary website on AWS, and worked with AWS Partner Apps Associates to migrate key content management applications to Amazon Aurora on RDS. Thomas Publishing was one of the first customers to use DMS to migrate from Oracle to Aurora.
Amazon.com have been moving off Oracle as fast as we can. Today 92% of Amazon’s Fulfillment Centers worldwide have migrated off Oracle to AWS Database services, mostly Aurora PostgreSQL (338 out of 368 FC’s). (Note: By the end of December 2018 Amazon.com will have migrated 170 (56%) of the Critical service databases to DynamoDB and 4316 (58%) of the non-Critical service databases to Aurora and RDS PostgreSQL in a single year).
Amazon’s DW: Amazon builds and operates thousands of micro-services to serve millions of customers. These include catalog browsing, order placement, transaction processing, delivery scheduling, video services, and Prime registration. Each service publishes datasets to Amazon’s massive analytics infrastructure, including over 50 petabytes of data and 75,000 tables, processing 600,000 user analytics jobs each day. They migrated this Data Warehouse from Oracle to AWS (S3, EMR, and Redshift). The new analytics infrastructure has one Data Lake with over 100 petabytes of data – almost twice the size of the previous Oracle Data Warehouse. Teams across Amazon are now using over 2700 Amazon Redshift or Amazon EMR clusters to process data from the Data Lake.
Samsung - Samsung Electronics migrated their Cassandra cluster to DynamoDB for their Samsung Cloud workload, saving 70%.
Intuit - Intuit is a business and financial software company that develops and sells financial, accounting, and tax-preparation software and services for small businesses, accountants, and individuals. Intuit migrated from Microsoft SQL Server to Amazon Redshift to reduce data-processing timelines and get insights to decision makers faster and more frequently. "With Amazon Redshift, our small team has handled a tenfold increase in data volumes while reducing the amount of time spent on system administration—freeing up time for value-add development." -Jason Rhoades, Systems Architect, Intuit
Equinox - Equinox Fitness Clubs transitioned from on-premises Data Warehouses and data marts in Teradata to a cloud-based, integrated data platform, built on AWS and Amazon Redshift. They went from static reports, redundant data, and inefficient data integration to a modern and flexible Data Lake and Data Warehouse architecture that delivers dynamic reports based on trusted data.
Eventbrite – Eventbrite’s mission is to bring the world together through live experiences. To achieve this goal, Eventbrite relies on data-driven decisions at every level. Previously they had Cloudera on-premises. With Amazon EMR, they are able to intelligent resize clusters, and cut costs considerably. They only pay for what they use and reduce costs further by purchasing Reserved instances and bidding on Spot instances (saving > 80%)
Smoothing - 특정 Outlier 제거
Interpolation – Missing Value 유추
Explanding – Attribute 추가나
DynamoDB – Oath is new brand after merging AOL and Yahoo
become GDPR compliant within 6 weeks by requiring 1B users to be notified of Privacy Policy changes. Deployed Amazon DynamoDB (global tables). Was able to deploy privacy controls for all of their web assets to meet GDPR.
They were able to test, snapshot, and deploy in 2 months.