Your SlideShare is downloading. ×
0
Amazon CloudSearch Meetup                                     San Francisco                                               ...
Welcome!   Housekeeping!   Slides will be posted!   Upcoming Events        •  April 3, 2013. Webinar 10:00am – 11:00am. "B...
Agenda6:30-7:30                               Networking (over Pizza/Beer/Drinks)7:00-7:30                               U...
Using CloudSearch with DynamoDB                      Jon Handler, Amazon CloudSearch Solution Architect© 2012 Amazon.com, ...
Do You Want Search With That?!   Amazon Dynamo DB       •  High throughput, provisioned latency       •  NoSQL model!   Am...
Amazon CloudSearch Overview                    DNS / Load Balancing                                                       ...
Automatic Scaling                                                           DATA         Document Quantity and Size       ...
The Big Picture© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed...
Working With DynamoDB and CloudSearch!        Create an Amazon CloudSearch domain!        Upload data from DynamoDB to Clo...
Create An Amazon CloudSearch Domain© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, mod...
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in...
Text fields for                                                                                        matching user terms...
Literal fields for                                                                      Faceting                          ...
Integer fields for                                                            ranking, narrowing© 2012 Amazon.com, Inc. an...
Configure the Domain© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distri...
Amazon Dynamo DB DataPrimary Key                                Attributesid = "tt0076759"                           title...
CloudSearch SDF[{"type":"add", !  "id": "tt0076759",!  "version": 1,!  "lang": "en", !  "fields": {!      "title":"Star Wa...
Uploading!   CloudSearchs batch update strategy!   Table scan one row is one document!   Batch multiple rows together and ...
Sending to CloudSearch: Main Loopdef _dynamo_to_cloudsearch(table, domain):!    doc_service = domain.get_document_service(...
Sending To CloudSearch: Fieldsdef _get_fields(table_item):!    Return a dict with key:value for all!       of the fields i...
Limits!   ID sizing       •  Dynamo: primary key 2K, range key 1K       •  CloudSearch: document id 128 bytes!   Items    ...
Keeping CloudSearch Up-To-Date© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified...
Conclusion!   Combine Amazon CloudSearch and Amazon DynamoDB       •  DynamoDB – throughput, durability       •  CloudSear...
Resources!   Amazon CloudSearch Overview Page       http://aws.amazon.com/cloudsearch/       •  FAQs       •  Community Fo...
Q&A© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or...
Were Hiring!© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in...
Thank You                                      Jon Handler / handler@amazon.com© 2012 Amazon.com, Inc. and its affiliates....
Upcoming SlideShare
Loading in...5
×

Meetup - Using CloudSearch with DynamoDB

3,283

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,283
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
15
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Meetup - Using CloudSearch with DynamoDB"

  1. 1. Amazon CloudSearch Meetup San Francisco March 28, 2013© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Welcome!   Housekeeping!   Slides will be posted!   Upcoming Events •  April 3, 2013. Webinar 10:00am – 11:00am. "Building Location-Based Search" https://www2.gotomeeting.com/register/264455602 •  April 30, 2013. AWS Summit, San Francisco. http://aws.amazon.com/aws-summit-2013/san-francisco/ © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  3. 3. Agenda6:30-7:30 Networking (over Pizza/Beer/Drinks)7:00-7:30 Using CloudSearch with DynamoDB – Jon Handler, Amazon CloudSearch Solution Architect7:30-8:00 EDU 2.0 User Talk - Graham Glass, Founder8:00-8:30 Getting More from Your Search Queries: How to Tune Search Requests – Tom Hill, Amazon CloudSearch Solution Architect8:30-8:45 SnapGuide User Talk - Sam Kimbrel, Senior Software Engineer © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  4. 4. Using CloudSearch with DynamoDB Jon Handler, Amazon CloudSearch Solution Architect© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  5. 5. Do You Want Search With That?!   Amazon Dynamo DB •  High throughput, provisioned latency •  NoSQL model!   Amazon CloudSearch •  Rich query API – free text, fielded, faceting and more •  User-customizable ranking© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  6. 6. Amazon CloudSearch Overview DNS / Load Balancing AWS QuerySearch Domain Search API Console Doc Command Console Config Command Console Svc API Line Tools API Line Tools SEARCH SERVICE DOCUMENT SERVICE CONFIG SERVICE© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  7. 7. Automatic Scaling DATA Document Quantity and Size SEARCH INSTANCE SEARCH INSTANCE SEARCH INSTANCE Index Partition 1 Index Partition 2 Index Partition n Copy 1 Copy 1 Copy 1TRAFFICSearchRequestVolume and SEARCH INSTANCE SEARCH INSTANCE SEARCH INSTANCEComplexity Index Partition 1 Index Partition 2 Index Partition n Copy 2 Copy 2 Copy 2 SEARCH INSTANCE SEARCH INSTANCE SEARCH INSTANCE Index Partition 1 Index Partition 2 Index Partition n Copy n Copy n Copy n © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  8. 8. The Big Picture© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  9. 9. Working With DynamoDB and CloudSearch!   Create an Amazon CloudSearch domain!   Upload data from DynamoDB to CloudSearch!   Configure: item attributes map to index fields!   Keep CloudSearch and DynamoDB in sync © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  10. 10. Create An Amazon CloudSearch Domain© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  11. 11. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  12. 12. Text fields for matching user terms Result enabled to retrieve source data© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  13. 13. Literal fields for Faceting Facet enabled to retrieve facet counts Search enabled for narrowing© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  14. 14. Integer fields for ranking, narrowing© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  15. 15. Configure the Domain© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  16. 16. Amazon Dynamo DB DataPrimary Key Attributesid = "tt0076759" title = "Star Wars", director = "Lucas, George", genre = {"Action","Adventure","Fantasy","Sci-Fi"}, actor = {"Ford, Harrison","Fisher, Carrie","Hamill, Mark”, "Jones, James Earl","Guinness, Alec","Johnston, Joe", "Mayhew, Peter","Cushing, Peter","Prowse, David","Daniels, Anthony”} year=1977id = "tt1411664" title = "Born to Be a Star", director = "Brady, Tom", genre = {"Comedy"}, actor = {"Ricci, Christina","Swardson, Nick","Dorff, Stephen”, "Johnson, Don","Bain, Robin","Herrmann, Edward","Goodman, Dana”, "Giangrande, Meredith","Dawn, Nadia","Locke, Tembi","Herschman, Adam"} year=2011© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  17. 17. CloudSearch SDF[{"type":"add", ! "id": "tt0076759",! "version": 1,! "lang": "en", ! "fields": {! "title":"Star Wars",! "director":"Lucas, George",! "genre": ["Action","Adventure","Fantasy","Sci-Fi"], ! "actor": ["Ford, Harrison","Fisher, Carrie",! "Hamill, Mark","Jones, James Earl",! "Guinness, Alec”, ...],! "year": 1977!} }, ...]! © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  18. 18. Uploading!   CloudSearchs batch update strategy!   Table scan one row is one document!   Batch multiple rows together and send!   Alternate strategy using Hive/S3/CLTs© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  19. 19. Sending to CloudSearch: Main Loopdef _dynamo_to_cloudsearch(table, domain):! doc_service = domain.get_document_service()! for table_item in table.scan():! docid = table_item[docid]! version = _cur_timestamp()! doc_service.add(docid, version,! _get_fields(table_item))! if len(doc_service.get_sdf()) >= FOUR_MEGABYTES: ! resp = doc_service.commit()!© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  20. 20. Sending To CloudSearch: Fieldsdef _get_fields(table_item):! Return a dict with key:value for all! of the fields in the table! attrs = {} ! for key,value in table_item.iteritems(): ! attrs[key] = ! [item for item in value]! return attrs !!© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  21. 21. Limits!   ID sizing •  Dynamo: primary key 2K, range key 1K •  CloudSearch: document id 128 bytes!   Items •  Dynamo: unlimited attributes, up to 64K of data •  CloudSearch: unlimited data, up to 200 fields and 100 field values •  Attribute/field naming conventions! Numerics© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  22. 22. Keeping CloudSearch Up-To-Date© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  23. 23. Conclusion!   Combine Amazon CloudSearch and Amazon DynamoDB •  DynamoDB – throughput, durability •  CloudSearch – Rich, powerful search!   Just a few lines of code© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  24. 24. Resources!   Amazon CloudSearch Overview Page http://aws.amazon.com/cloudsearch/ •  FAQs •  Community Forum •  Documentation & Getting Started Tutorial (IMDb)!   Demos and Tutorials •  What Is Amazon CloudSearch •  Introducing Amazon CloudSearch (Features) •  Building a Search Application Using Amazon CloudSearch© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  25. 25. Q&A© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  26. 26. Were Hiring!© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  27. 27. Thank You Jon Handler / handler@amazon.com© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×