AWS Summit 2013 | Singapore - Delivering Search for Today's Local, Social, and Mobile Applications

905 views

Published on

As more organizations seek to leverage the power and benefits of the cloud, they also need to combine new systems with exiting on-premises systems. Services such as Virtual Private Cloud, VPN and DirectConnect enable AWS customers to combine on-premises and cloud-based resources easily and effectively. This session will walk customers through the 4 main patterns of connectivity and will include a ""real time"" demonstration of how easy it is to setup your own VPC and start working in your own private section of the AWS Cloud.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
905
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

AWS Summit 2013 | Singapore - Delivering Search for Today's Local, Social, and Mobile Applications

  1. 1. Delivering Search for Today's Local, Social, and Mobile Applications Brian Pinkerton VP and GM Amazon A9 and CloudSearch July 18, 2013
  2. 2. Agenda • Local, social, and mobile search • Building with CloudSearch • CloudSearch in action • ManageFlitter on CloudSearch, James Peter, CTO & Co-Founder, 89n
  3. 3. The Rise of Mobile Search • 45% of users 18-29 use mobile search daily (Icebreaker Consulting) • Mobile searches (85.9B) are projected to exceed desktop searches (84B) for in 2015 (eMarketer) • 17% of users make a purchase after a mobile search (Juniper Research)
  4. 4. Local, Social, Mobile Search Needs • Location-based • Rapidly scale in response to social trends • Server-side importance • Social relevance based on friends • Optimized for the small screen – Better autocorrect, spelling suggestions/corrections
  5. 5. Search Challenges • Complex, expertise required • Costly, often with upfront expenditure • Long time to market, slow innovation & experimentation • Operational overhead is undifferentiated work
  6. 6. Amazon CloudSearch • Pay for infrastructure you need when you need it • Low cost • No need to guess capacity • Experiment fast with low risk • We do the undifferentiated heavy lifting • Go global in minutes
  7. 7. CloudSearch in Context
  8. 8. Amazon CloudSearch Overview DNS / Load Balancing AWS Query Search API Console Config API Command Line Tools ConsoleDoc Svc API Command Line Tools Console SEARCH SERVICE Search Documents DOCUMENT SERVICE Add Documents Update Documents Delete Documents Create Domains Configure Domains Delete Domains CONFIG SERVICE Search Domain
  9. 9. Automatic Scaling SEARCH INSTANCE Index Partition n Copy 1 SEARCH INSTANCE Index Partition 2 Copy 2 SEARCH INSTANCE Index Partition n Copy 2 SEARCH INSTANCE Index Partition 2 Copy n SEARCH INSTANCE DATA Document Quantity and Size TRAFFIC Search Request Volume and Complexity Index Partition n Copy n SEARCH INSTANCE Index Partition 1 Copy 1 SEARCH INSTANCE Index Partition 2 Copy 1 SEARCH INSTANCE Index Partition 1 Copy 2 SEARCH INSTANCE Index Partition 1 Copy n
  10. 10. Automatic Scaling SEARCH INSTANCE Index Partition n Copy 1 SEARCH INSTANCE Index Partition 2 Copy 2 SEARCH INSTANCE Index Partition n Copy 2 SEARCH INSTANCE Index Partition 2 Copy n SEARCH INSTANCE Index Partition n Copy n SEARCH INSTANCE Index Partition 1 Copy 1 SEARCH INSTANCE Index Partition 2 Copy 1 SEARCH INSTANCE Index Partition 1 Copy 2 SEARCH INSTANCE Index Partition 1 Copy n Compute Storage Load Balancing Security DATA Document Quantity and Size Search Request Volume and Complexity
  11. 11. Search App Server CloudSearch Query (URL) Results (JSON/XML)
  12. 12. Text Search
  13. 13. Highly Relevant Results
  14. 14. Faceted Drilldown
  15. 15. Integer Range Searching
  16. 16. Complex Queries
  17. 17. Text fields for matching user terms Result enabled to retrieve source data
  18. 18. Literal fields for Faceting Facet enabled to retrieve facets Search enabled for narrowing
  19. 19. Integer fields for ranking, narrowing
  20. 20. BUILDING WITH AMAZON CLOUDSEARCH
  21. 21. Create An Amazon CloudSearch Domain
  22. 22. Configure the Domain
  23. 23. Data Preparation and Upload SDF Batch Amazon CloudSearch Search Documents
  24. 24. CloudSearch SDF [{"type":"add", "id": "b007oznzg0", "version": 1, "lang": "en", "fields": { "title":"Kindle Paperwhite", "description":"World's most advanced e-reader", "category": ["Electronics","eBook Readers"], "price":11900 } }, ...]
  25. 25. Document Service API http(s)://< document service endpoint >/2011-02-01/documents/batch Accept: application/json Content-Length: 1176 Content-Type: application/json Host: doc.imdb-movies-rr2f34ofg56xneuemujamut52i.us-east- 1.cloudsearch.amazonaws.com [{ ,"id":"b007oznzg0","version": 1,"lang": "en","fields": {"title":"Kindle Paperwhite","description":"World's most advanced e- reader","category":["Electronics","eBook Readers"],"price":11900} }, { , "id": "tt0434409", "version": 1337648735 } ]
  26. 26. Search Service API http(s)://< search service endpoint>/2011-02-01/search? • Simple searches – q= text • Boolean combination of fields – bq= (or field:'value1' (and field:'value2' field:'value3')) • Faceting – facet= comma separated list of facet fields • Pagination – start=, size= • Customized ranking – rank= sort results based on the rank expression provided e.g. -text_relevance * log10(unit_sales)
  27. 27. Search Results {"rank": "-text_relevance", "match-expr": "(label 'kindle paperwhite')", "hits": { "found": 204, "start": 0, "hit": [ { "id": "sontsst12cf5f88b42" }, { "id": "sopvopr12ab017f082" }, { "id": "sorzrpw12ac468a13b" }, ] }, ... }
  28. 28. CLOUDSEARCH IN ACTION
  29. 29. The Week – Mobile App • Mobile news app with commentary and analysis of breaking news • CloudSearch enabled keyword search, results ranking, social sharing • Impact – Increased usage, user engagement – More page views – Higher user growth
  30. 30. Viddy • Capture, create, and share 30- second social videos with friends • Migrated entirely from Lucene to CloudSearch in 3 days • Operation costs reduced from $5- 6K/mo. to $1600/mo. • 5x increase in search usage • Freed up development resources for innovation
  31. 31. James Peter, 89n ManageFlitter on CloudSearch
  32. 32. ManageFlitter Background • ManageFlitter helps businesses and personal brands manage their Twitter accounts • Work faster and smarter with Twitter – Unfollowing & Following – Analytics – Engagement – Search • 1.7 million users over 3 years • Customers from over 100 countries
  33. 33. ManageFlitter Background • Billions of social connections processed daily • ManageFlitter makes the ‘Big Data’ analysis associated with social accounts accessible to everyone • We require tools that can keep up with our fast growth and high throughput
  34. 34. ManageFlitter Background Every 24 hours • Read 732,916,544 social graph connections • Write 7,517,513 social graph updates (940,622 follows - 6,576,891 unfollows)
  35. 35. ManageFlitter Background Cached data • 90 million unique Twitter accounts ? Search service
  36. 36. Search Infrastructure Requirements • Low operational overhead (no 3am wakeup calls) • Can really automatically scale • Handle large amounts of frequently updated data • Easy to prototype and low cost to try
  37. 37. ManageFlitter CloudSearch Deploy Timeline Prototype (1 day) Test (1 week) Launch NowMay 29, 2012
  38. 38. Search Architecture CloudSearch Data Provider Memcache MySQL Web Servers Visitors Social Data
  39. 39. Search Architecture Data Provider Web Servers Visitors Social Data 1000 CloudSearch Memcache MySQL Per second 950 records added/updated 400 records deleted
  40. 40. Search Architecture Data Provider Visitors 1000 CloudSearch Memcache MySQL Web Servers Social Data
  41. 41. Search Architecture CloudSearch Data Provider Memcache MySQL Web Servers Visitors Social Data
  42. 42. http://manageflitter.com/search
  43. 43. http://manageflitter.com/search
  44. 44. Key Benefits • Operational overhead has been incredibly low – Saving $100,000+ per year on headcount by not needing dedicated engineers – Saving $131,856 per year on hardware by running in the cloud • CloudSearch team has been very responsive • From prototype to production with very little work • HTTP API has been easy to work with & platform independent • Can handle 90,000,000+ frequently updated documents
  45. 45. Future Plans • Take advantage of transparent scaling & drive more people to our search product • Bring location based searches to the fore-front
  46. 46. Thank You James Peter CTO & Co-Founder 89n  Web http://www.manageflitter.com  Twitter @zemaj  Email james@89n.com
  47. 47. Summary • Powerful search is a critical component of today's local, social and mobile applications • Amazon CloudSearch makes adding search easy • Just create a domain, POST documents, GET search results
  48. 48. Resources and Q&A Get started for free: 30 days Amazon CloudSearch Overview Page http://aws.amazon.com/cloudsearch/ • FAQs • Community Forum • Documentation & Getting Started Tutorial (IMDb)
  49. 49. Technical Track

×