Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Amazon DynamoDB
[AWS Meister – re:Generate] 웨비나 시리즈 발표자료
작성: 今井雄太 솔루션스 아키텍트
번역: 정윤진 솔루션스 아키텍트
이 자료의 내용은 웨비나가 진행된 2013년 10월...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
Upcoming SlideShare
Loading in …5
×

20131002 AWS Meister re:Generate - DynamoDB (Korean)

1,769 views

Published on

2013년 10월 02일 일본에서 진행된 AWS 今井雄太 솔루션스 아키텍트의 기술 웨비나 발표 자료를 한국의 정윤진 솔루션스 아키텍트가 한글로 번역한 자료입니다. 웨비나 당시와 현재의 내용이 상이한 부분이 있을 수 있으니 자료 열람에 이 점 참고하시기 바라며, 혹 내용에 대한 문의사항이 있으신 경우 info-kr@amazon.com으로 연락 부탁드리겠습니다.

20131002 AWS Meister re:Generate - DynamoDB (Korean)

  1. 1. Amazon DynamoDB [AWS Meister – re:Generate] 웨비나 시리즈 발표자료 작성: 今井雄太 솔루션스 아키텍트 번역: 정윤진 솔루션스 아키텍트 이 자료의 내용은 웨비나가 진행된 2013년 10월 02일에 맞춰져 있어 현재의 내용과는 다를 수 있습니다. 내용에 대한 문의사항이 있으신 경우 info-kr@amazon.com으로 연락 주시기 바랍니다.
  2. 2. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Introduce myself 이름 • 今井雄太( @imai_factory ) 업무 • 솔루션 아키텍트 • 리스팅 광고의 개발, 광고 서버의 운영 및 Hadoop의 로그 분석 좋아하는 AWS 서비스 Amazon S3 DynamoDB
  3. 3. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Agenda NoSQL과 RDB DynamoDB 에 대하여 DynamoDB 사용 해 보기 도구 및 Eco 시스템 When to use 정리
  4. 4. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. NoSQL과 RDB
  5. 5. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 세상은 지금 NoSQL이 유행중? Neo4j CounchDB DynamoDB http://http://en.wikipedia.org/wiki/NoSQL MongoDB Riak HBase Cassandra 그 외에도 많은 NoSQL
  6. 6. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. NoSQL vs RDB RDB • ACID특성 • Atomicity • Consistency • Isolation • Durability • 트랜젝션 기반 • Rich 쿼리 • 조인 NoSQL • BASE 특성 • 대용량 데이터 • 확장 가능 • 빠른 쓰기 속도 • 유연한 스키마 설정
  7. 7. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. BASE?
  8. 8. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Basically Available, Soft state, Eventually consistent RDB 에서 제공하는 ACID 특징과 비교하기 위해 자주 사 용되는 정확한 일관성과 데이터의 즉시 반영을 포기하는 대신 확 장성을 특징으로 한다 모든 NoSQL이 완전히 BASE 의 특성을 가지지는 않지만 일관성 및 트랜젝션 등 기존 ACID의 RDB가 가진 특성을 포기해야 하는 것이 요구사항 이었다.
  9. 9. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. NoSQL • 일관성을 유지하는 것은 골칫거리 • 트랜젝션을 사용하지 않아도 되는 • 쿼리가 단순 • 편리한 운용
  10. 10. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. NoSQL이 쓰기가 빠르다고는 하지만… MySQL 에서도 보통 1000 QPS 정도는 나옵니다. NoSQL의 진가는 데이터의 양 이 방대해져도 성능이 떨어지지 않는것
  11. 11. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. ”RDB에서 사용하던 방법을 그대로 사용할 수 없다”는 점에 유의 하여 NoSQL 을 사용합시 다!!
  12. 12. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB
  13. 13. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB ・NoSQL as a Service ・초고속 / 예측 가능한 일관된 성능 ・원활한 확장성, 저렴한 비용 운영 및 관리 필요 없음 낮은 지연시간, SSD 처리량을 프로비저닝 무한에 가까운 스토리지
  14. 14. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB 만의 특징 관리가 필요 없으며 높은 신뢰 성 처리량을 프로비저닝 저장 용량에 제한이 없음
  15. 15. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 특징1:관리가 필요없는 높은 신뢰성 SPOF가 존재하지 않는 구성 데이터는 3개소의 AZ에 분산 저장되어 높은 신뢰성 스토리지는 필요에 따라 자동으로 분산 처리 Client
  16. 16. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 특징2:처리량을 프로비저닝 가능 Read 및 Write, 각각 필요한 만큼의 처리 용량을 할당 예를 들어 일반적인 Read Heavy DB 라면 • Read : 1,000 • Write : 100 약간 Heavy 한 DB의 경우 • Read : 500 • Write : 500 이 설정값은 DB 운영중에 온라인으로 변경 가능 • Scale-down에 관해서는 하루 4회만 가능
  17. 17. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 특징3:저장 용량에 제한이 없는 사용한 만큼 지불하는 종량제 스토리지 데이터 용량 증가에 따른 노드 추가와 같은 작업이 불필 요
  18. 18. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB의 요금체계 프로비저닝한 처리량에 따른 요 금 • Read/Write 각각 프로비저닝 된 처리량에 따라 시간당 요금이 결정됨 • 대규모로 사용하는 경우 예비 순위에 의한 할인도 가능 스토리지 사용량 • 저장한 데이터의 양에 따른 사용 요금 • GB 당 가격이 적용 실제 사용 비용에 대한 내용은 아래를 참조 http://aws.amazon.com/ko/dynamodb/pricing
  19. 19. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB를 시작하려면 1. 테이블의 Key 와 Index 를 결정 2. Read/Write 처리량을 결정 That’s it, write your code!
  20. 20. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB 사용해 보기
  21. 21. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB의 구성 요소 API Servers HTTP API SDK 오퍼레이션은 HTTP API 로 제공 Distributed Storage ClientSideServiceSide
  22. 22. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 제공 되는 API PutItem UpdateItem GetItem DeleteItem Query Scan BatchWriteItem BatchGetItem CreateTable DescribeTable UpdateTable DeleteTable
  23. 23. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. PutItem UpdateItem GetItem DeleteItem Query Scan BatchWriteItem BatchGetItem CreateTable DescribeTable UpdateTable DeleteTable SDK를 이용하는 것이 편리합니다 . 제공 되는 API
  24. 24. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 주로 사용되는 기능 • Get/Put/Update/Delete/BatchGet • Scan • 전체 테이블을 싹쓸이로 긁어옴 • Query • Hash + 범위 키만
  25. 25. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. AWS SDKs 각종 언어를 위한 공식 SDK 를 사용 Java Python PHP .NET Ruby nodeJS iOS Android
  26. 26. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 공식 SDK 이외에도 Perl • Net::Amazon::DynamoDB Erlang • wagerlabs/ddb • https://github.com/wagerlabs/ddb Go • go-dynamodb • https://github.com/fabiokung/go-dynamodb
  27. 27. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 디자인을 위한 요소(1) Table • Primary Key 로 ”Hash key” 혹은 ”Hash key & Range key”를 선택 기본 키:Hash key • Hash key 단체로 데이터를 고유하게 식별할 수 있는 경우 사용 기본 키:Hash key & Range key • Hash key에 해당하는 여러 데이터에서 Range key 로 검색 가능 Local Secondary Indexes • Range key 이외에 필터 검색을 위한 키를 가질 수 있음
  28. 28. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 디자인을 위한 요소(2) Attributes • 데이터의 내용. Hash key 에 해당하는 Attributes 이외에는 미리 정의 할 필요는 없다. 또한 레코드에서 Attributes 가 불규칙하더라도 문제 없 다. Attributes 형식 • String • Number • Binary • Array of String • Array of Number • Array of Binary
  29. 29. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB의 데이터 모델 테이블 아이템 기본 키 속성 Hash key 또는 Hash key & Range key
  30. 30. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB의 기본 키 • Hash key • 간단한 키 값 • Hash 이므로 정렬이 필요 없음 • Hash key + Range key • 복합 기본 키 • Range key 는 sort 가 있음
  31. 31. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 샘플(1) 상품 카탈로그 스키마 • 테이블명 Products • 상품 ID(ProductId) 를 테이블의 Hash key 로 사용 ProductId (Hash key) ProductName Price ・・・ ・・・ 1 Pen 100 ・・・ ・・・ 2 Pencil 50 ・・・ ・・・ 3 Eraser 100 ・・・ ・・・
  32. 32. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB 데이터 모델 Hash key 만 사용하는 경우 아이템 기본키 속성 (Attribute) A1-1 Hash key
  33. 33. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 만들기 • 상품 ID 를 Hash key로 가지는 테이블 • Primary Key에 Hash 를 선택하고 이름과 형태를 설정 샘플(1) 제품 카탈로그
  34. 34. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. node.js SDK 사용 예 • 데이터를 DynamoDB 로 저장 샘플(1) 제품 카탈로그 var aws = require(‘aws-sdk’); aws.config.loadFromPath(‘PATH/TO/CREDENTIAL’); var dynamoDb = new aws.DynamoDB.Client(); dynamoDb.putItem( { TableName:"Products", Item: { ProductId : {N:"1"}, ProductName: {S:"Pen"}, Price: {N:"100"} } }, function(err,data){ if(err){ console.log(err); }else{ console.log(data); } } );
  35. 35. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. node.js SDK 사용 예 • 테이블에서 ProductID 가 1인 데이터를 가져오기 샘플(1) 제품 카탈로그 dynamoDb.getItem( { TableName:"Products", Key: { ProductId : {N:"1"}, }, function(err,data){ if(err){ console.log(err); }else{ console.log(data); } } );
  36. 36. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 샘플(2) 로그 테이블 스키마의 이미지 • 테이블 이름은 Audience • 사용자 ID(AudienceId)를 Hash key로, Timestamp 를 Range key 로 설정 AudienceId (Hash key) Timestamp (Range key) Action Url ・・・ 1 2013-10-01 00:01:01 Login ・・・ ・・・ 2 2013-10-01 00:02:02 Login ・・・ ・・・ 1 2013-10-01 00:21:00 Login ・・・ ・・・ 1 2013-10-01 00:42:00 ViewHoge 1 2013-10-01 00:56:22 PostHoge
  37. 37. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB 데이터 모델 Hash key + Range key 의 경우 아이템 기본키 속성 (Attribute) A1-1 Hash key A Range key-1 Range key-2 Range key-3 속성 (Attribute) A2-1 속성 (Attribute) A3-1
  38. 38. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 만들기 샘플(2) 로그 테이블
  39. 39. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. node.js SDK 사용 예 • 데이터를 테이블에 입력 샘플(2) 로그 테이블 var data = new Array( { AudienceId: { N:"1" }, Timestamp: { S:"2013-10-01 00:01:01" }, Action: { S:"Login" } }, { AudienceId: { N:"2" }, Timestamp: { S:"2013-10-01 00:02:02" }, Action: { S:"Login" } }, { AudienceId: { N:"1" }, Timestamp: { S:"2013-10-01 00:21:00" }, Action: { S:"Login" } }, { AudienceId: { N:"1" }, Timestamp: { S:"2013-10-01 00:42:00" }, Action: { S:"ViewHoge" } }, { AudienceId: { N:"1" }, Timestamp: { S:"2013-10-01 00:56:22" }, Action: { S:"PostHoge" } } ); for (var i =0; i < data.length; i++){ dynamoDb.putItem( { TableName:"Audience", Item: data[i] }, function(err,data){ if(err){ console.log(err); }else{ console.log(data); } } ); }
  40. 40. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. node.js SDK 사용 예 • 사용자의 10월 1일 0시 10분 이후의 로그를 가져오기 샘플(2) 로그 테이블 dynamoDb.query( { TableName:"Audience", KeyConditions: { "AudienceId": { ComparisonOperator:"EQ", AttributeValueList:[ {N:"1"} ] }, "Timestamp": { ComparisonOperator:"GT", AttributeValueList:[ {S:"2013-10-01 00:10:00"} ] } } }, function(err,data){ if(err){ console.log(err); }else{ console.log(data); } } );
  41. 41. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 샘플(3) LSI를 적용한 로그 테이블 사용자 로그에 Local Secondary Indexes(LSI)를 보면 AudienceId (Hash key) Timestamp (Range key) Action (Action-Index) Url ・・・ 1 2013-10-01 00:01:01 Login ・・・ ・・・ 2 2013-10-01 00:02:02 Login ・・・ ・・・ 1 2013-10-01 00:21:00 Login ・・・ ・・・ 1 2013-10-01 00:42:00 ViewHoge 1 2013-10-01 00:56:22 PostHoge
  42. 42. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Local Secondary Indexes(LSI) • Get/Put/Update/Delete/BatchGet • Scan • 테이블의 전체 내용을 가져옴 • Query • Hash + Range 키만 가져옴 Local Secondary Index 를 사용하여 이 제한을 없앨 수 있다!
  43. 43. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. ‹#› Local Secondary Indexes (LSI) 테이블에서 어떠한 Index 를 Query 로 사용할지에 대한 예 • 5개의 LSI 지정이 가능 LSI없는 테이블에서는 Range Key를 사용한 Query 만 가능 LastPostIndex RepliesIndex LSI를 정의하면 Key가 아닌 속성에 대해서도 Query 가능 예: - Replies 가 10개 이상 - 마지막 게시물이 XX 이 후 LSI Forum 게시물을 보관하는 테이블의 구조
  44. 44. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. ‹#› 어째서 ”Local” Secondary Indexes 인가? Hash key 가 일치하는 범위 내에서의 Secondary Index 이므로 Hash Key 가 다른 항목에 대해 Query 결과로 가져올 수 없다 RepliesIndex 를 사용하여1개의 Query로 가져올 수 있는 데이터는 ? • ForumName 이 S3 이며, Replies 가 9개 이상 • 모든 ForumName 에서 Replies가 9 이상  3개의 Hash key가 있으므 로 3 회의 Query 필요 RepliesIndex
  45. 45. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. ‹#› LSI 를 설정한 경우의 실제 동작 Thread 테이블의 LastPostDateTime에 LSI 를 설정하면, LastPostIndex 별도의 Index 테이블이 뒤에서 생성, 관리 됨 • 지정된 속성이 Range-key • 원래 테이블의 Range-key 가 속성으로
  46. 46. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. ‹#› 속성의 프로젝션 Index 용 테이블에 지정된 속성을 복제 저장하는 기능 • 예:Thread 테이블에 Replies를 LastPostIndex에 프로젝션 LastPostIndex Query 수행시 Replies를 취득하는 경우 Load 비용을 절감  ReadThroughput 을 절약 가능
  47. 47. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 샘플(3) LSI를 적용한 로그 테이블 로그 테이블에 Local Secondary Indexes(LSI)를 만들어 보 면 • 이미 생성된 테이블의 LSI 는 변경이 불가능하기 때문에 필요하다면 Audience2 라는 신규 테이블을 생성 해야 함 • 이번에는 Action-Index라는 LSI를 생성, ”Action”이라는 속성으로 설정
  48. 48. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 샘플(3) LSI를 적용한 로그 테이블 Action-Index(LSI)를 사용하여 데이터를 검색 dynamoDb.query( { TableName:"Audience2", IndexName: “Action-Index”, KeyConditions: { "AudienceId": { ComparisonOperator:"EQ", AttributeValueList:[ {N:"1"} ] }, “Action": { ComparisonOperator:“EQ", AttributeValueList:[ {S:“Login"} ] } } }, function(err,data){ if(err){ console.log(err); }else{ console.log(data); } } );
  49. 49. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 디자인: 기본키의 사용 Hash key와 Range key의 개념 A,D B,E C,F 123456789 Parition1 Partition2 Parition3 Range key Partition내에서 데이터의 순서를 보장 Hash key Partition간의 데 이터 분산에 이용 되는 key DynamoDB의 처리량은 각 Partition에 부하가 적절히 분 산되었을때를 기준으로 설계 되 어 있다. 하나의 Partition 으 로 프로비저닝된 처리량이 나오 는 것이 아님에 주의
  50. 50. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 디자인: 기본키의 사용 Hot Key문제 :특정 Hash key에 데이터가 집중 되면 성능이 저하(예상대로의 성능이 아님)됨 에 주의 위의 Audience 테이블에서 AudienceId처럼 데 이터가 특정 키에 집중하기 어려운것을 선택한다 예를 들어,「년월」과 같이 데이터의 집중이 발 생하기 쉬운 데이터를 Hash key로 선택하는 것 은 안티 패턴
  51. 51. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 테이블 디자인: LSI Local Secondary Indexes는 저장소와 IO 비용 증가를 발생 시킬 수 있음 • LSI는 소위 “Projection”에서 구현된다 • 따라서 Projection 되는 속성이 많을수록 스토리지 비용이 증가 • Write시 쓰기 비용 역시 증가 반드시 필요한 조건에만 LSI 를 사용 Projection이 필요한 Attributes만 선택하도록 한다
  52. 52. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 도구 및 Eco 시스템
  53. 53. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 데이터의 조작은 관리 콘솔에서 가능 관리 콘솔에서는 테이블에 대해 SCAN, Query, PutItem 등의 동작을 수행 할 수 있다.
  54. 54. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB Local 개발 및 테스트용 도구 • 이전까지는 개발 및 테스트를 위해서는 반드시 DynamoDB테이 블을 생성해야 했다. 이는 ”비용 발생의 우려”、”내부 테스트 환 경구성 불가”、”오프라인으로 개발 불가능”과 같은 문제가 있었 음 • 이 도구를 사용하여 개발 및 테스트를 보다 편리하게 수행 할 수 있음 JAR파일로 제공되어 로컬에 설치 및 실행 가능 (Java7) 단순히 API의 기능 재현을 위한 테스트용 도구이기 때문 에 서비스 환경에는 사용이 불가능 프로비전된 처리량은 반영되지 않으므로 주의 자세한 내용은 AWS document 를 참조
  55. 55. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Transaction Library for DynamoDB AWS SDK for Java 에 추가된 기능 • 이 SDK를 사용하여 DynamoDB에서 Transaction 사용이 가능 • 라이브러리에서 Transaction 관리 테이블을 생성하여 Transaction을 구 현 사용 방법은 다음 페이지의 샘플 코드를 참조 http://bit.ly/16KbppP
  56. 56. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Transaction Library for DynamoDB
  57. 57. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Transaction Library for DynamoDB
  58. 58. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Amazon EMR의 Hive 에서 사용 CREATE EXTERNAL TABLE Audience ( AudienceId Int, ActionTimestamp string, Action string ) STORED BY 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler' TBLPROPERTIES ( "dynamodb.table.name" = ”Audience2", "dynamodb.column.mapping" = ”AudienceId:AudienceId, ActionTimestamp:Timestamp, Aciton:Action“ ); hive의 External Table로 이용 가능 • DynamoDB의 데이터 집계등이 필요한 경우등에 이용 • 자세한 내용은 http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/EMR_ Interactive_Hive.html
  59. 59. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. hive를 사용하여 S3에 데이터 백업 EMR에서 hive는 DynamoDB뿐만 아니라 S3 역시 External Table 사용 할 수 있는 장점을 살려 DynamoDB 를 External Table 로 Select한 데이 터를 다시 S3 의 External Table로 Insert 가능하 다! INSERT OVERWRITE TABLE s3_as_external_table SELECT * FROM dynamodb_as_external_table;
  60. 60. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Redshift 의 COPY 를 사용해 데이터를 로드 • 자세한 내용은 http://docs.aws.amazon.com/amazondynamodb/latest/devel operguide/RedshiftforDynamoDB.html Amazon Redshift 에 데이터 로딩 COPY audience FROM ‘dynamodb://Audience2’ CREDENTIALS 'aws_access_key_id=<Your-Access-Key- ID>;aws_secret_access_key=<Your-Secret-Access-Key>' READRATIO 50;
  61. 61. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. When to use
  62. 62. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 언제 DynamoDB 를 사용할 것인가 RDB를 대체할 목적으로는 쉽지 않다 시스템 전체를 DynamoDB로 구성할 필요는 없 음 RDB와 사용방법에 차이를 두어야 함 이를테면 샤딩과 같은 방법이 아니면 저장이 불 가능한 양의 데이터를 DynamoDB에 저장
  63. 63. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. RDB에서 성능을 유지하면서 대량의 데이터를 저장하려면
  64. 64. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. MySQL에서 샤딩을 구성하여 데이터를 운용하는 방법의 예 (1) ViewData index Shard 구성된 실제 data tables Shard Index S S S S S S S S
  65. 65. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. MySQL에서 샤딩을 구성하여 데이터를 운용하는 방법의 예 (2) ViewData index Shard 구성된 실제 data tables Shard Index S S S S S S S S 검색할 데이터의 목록을 가지고 있음
  66. 66. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. MySQL에서 샤딩을 구성하여 데이터를 운용하는 방법의 예 (3) ViewData index Shard 구성된 실제 data tables Shard Index S S S S S S S S 원하는 데이터가 있는 Shard node 의 정보를 가져옴
  67. 67. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. MySQL에서 샤딩을 구성하여 데이터를 운용하는 방법의 예 (4) ViewData index Shard 구성된 실제 data tables Shard Index S S S S S S S S 실제 데이터를 조회
  68. 68. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. MySQL에서 샤딩을 구성하여 데이터를 운용하는 방법의 예 (4) View cacheData index Shard 구성된 실제 data tables Shard Index S S S S S S S S Cache 된 View 를 조회
  69. 69. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 확장성(성능 x 데이터량)을 RDB에서 확보하는것은 매우 쉽지 않음
  70. 70. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 이러한 데이터에 DynamoDB가 제격!
  71. 71. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Shard node 와 별도의 Index 가 불필요하므로 운용이 편리! View cacheData index DynamoDB S S
  72. 72. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB가 적합한 케이스는 사용자의 로그를 저장 • Hash key에 사용자의 ID、Range key에 Timestamp • 광고 이벤트에서 사용자의 Subscription 수집 • 게임에서 사용자의 액션을 기록하는 로그 스토리지에 대한 Index • Hash key에 Object의 이름 • S3에 저장된 데이터의 Index 로 사용 투표를 시스템 또는 디지털 마케팅 • 투표 및 응모 대상의 ID 를 Hash key 로 지정 • 단기간에 스파이크 트래픽을 처리하기 위한 용도
  73. 73. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Use case
  74. 74. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 잘못된 Use case 로는? 집계 • 전체 데이터를 SCAN 해야하는 경우 부적합 함 • 이러한 경우엔 Hive 및 Redshift 와 함께 사용 Transaction • 라이브러리를 통해 사용은 가능하지만, NoSQL 의 특성과 맞지 않으므로 좋은 사용성이 아님
  75. 75. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 정리
  76. 76. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. DynamoDB는 NoSQL as a Service 운영이 필요 없다 NoSQL과 RDB는 서로 용도가 다르다 전체 시스템을 DynamoDB로 구축할 필요 는 없다. RDB 및 Cache 와 함께 적절히 사 용하는것이 좋다 정리
  77. 77. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 참조 Amazon DynamoDB 의 모범사례, 사용 방법 및 도구 등 의 정보를 다음의 링크에서 참조 가능 • http://aws.amazon.com/dynamodb/developer-resources/
  78. 78. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. 감사합니다!

×