Download to read offline
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Is this a search engine?
where [field] like “%codecamp%”](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-6-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
How searching works
Id Title UserId ViewCount Tags
1 Controller Action ambiguity
even with [HttpPost]
decoration? (ASP.NET MVC4)
5 352 asp.net asp.net-mvc
asp.net-mvc-4 f#
2 Why can't I use a scrollwheel
on a webpage?
6 109 c# javascript asp.net
asp.net-mvc-4 twitter-
bootstrap-3
3 Access session variable of one
site in another"
7 78 asp.net .net
4 Check if SIM card exists 5 209 c# windows-phone-8](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-9-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Inverted index
How searching works
Title
Access session variable of one site in another" 3
Check if SIM card exists 4
Controller Action ambiguity even with [HttpPost] decoration? (ASP.NET
MVC4)
1
Why can't I use a scrollwheel on a webpage? 2
UserID
5 1, 4
6 2
7 3
ViewCount
78 3
109 2
209 4
352 1](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-10-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Inverted index
How searching works
Title
Access session variable of one site in another" 3
Check if SIM card exists 4
Controller Action ambiguity even with [HttpPost] decoration? (ASP.NET
MVC4)
1
Why can't I use a scrollwheel on a webpage? 2
UserID
5 1, 4
6 2
7 3
ViewCount
78 3
109 2
209 4
352 1
Query: UserID = 5](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-11-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Index creation
POST https://codecamp-en.search.windows.net/indexes
"name": "stackoverflow-posts",
"fields": [ {
"name": "name_of_field",
"type": “data_type",
"searchable": true (default where applicable) | false ,
"filterable": true (default) | false,
"sortable": true (default where applicable) | false
"facetable": true (default where applicable) | false ,
"key": true | false (default),
"retrievable": true (default) | false } ] …
Azure Search](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-23-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Searching through data
GET https://codecamp-en.search.windows.net/indexes/stackoverflow/docs?
search=[string]
+ (AND operator “code" and “camp")
| (OR operator “code" or “camp" or both)
- (NOT operator. “code–camp" “code" term and/or do not have “camp" )
* (Suffix operator. “cod*" - starts with “cod", ignoring case)
" (Phrase search operator)
( ) (Precedence operator - code+(camp|workshop)
searchMode=any|all
searchFields=[string]
Azure Search](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-26-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Filtering results
$filter=[string] - Odata syntax
$skip=#
$top=#
$count=true|false
$orderby=[string]
$select=[string]
Azure Search](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-27-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Emphasizing results
facet=[string] (field names)
count
sort
values
interval
highlight=[string] (field names)
highlightPreTag=[string] (default is em)
highlightPostTag=[string]
Azure Search](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-28-2048.jpg)
![April 25, 2015Cloud powered search
© EXPERT NETWORK
Suggestions
GET https://codecamp-en.search.windows.net/indexes/stackoverflow/docs/suggest
search=[string]
suggesterName=[string]
fuzzy=[boolean]
searchFields=[string]
Azure Search](https://image.slidesharecdn.com/cloudpoweredsearch-150513140926-lva1-app6891/75/Cloud-powered-search-29-2048.jpg)
Uploaded byCodecamp Romania
Cloud powered search
This document discusses cloud powered search using Azure Search. It begins with an introduction to search engines and how they work using indexing and relevance scoring. Lucene is presented as an example open source search library. The document then covers how Azure Search works as a managed search service, including indexing data from sources like SQL databases, performing CRUD operations, querying, filtering, faceting and highlighting results. Scaling and storage capabilities of Azure Search are also discussed. Examples of using Azure Search for online retail/ecommerce and user generated content are provided.
More Related Content
Similar to Cloud powered search
![Lets build a_search-based_application_in_share_point_2013_-_spsdc[2]](https://cdn.slidesharecdn.com/ss_thumbnails/letsbuildasearch-basedapplicationinsharepoint2013-spsdc2-130610081031-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Cloud powered search
- 2.
- 3. April 25, 2015Cloudpowered search © EXPERT NETWORK CODECAMP Challenges in distributed applications SQL Azure Federation HDInsight DocumentDB Previous subjects
- 4. April 25, 2015Cloudpowered search © EXPERT NETWORK Azure Search The need for search Search explained Development Case Scenarios Agenda
- 5. April 25, 2015Cloudpowered search © EXPERT NETWORK The need for search Why do we search for data? How do we store it to search efficiently? What’s important?
- 6. April 25, 2015Cloudpowered search © EXPERT NETWORK Is this a search engine? where [field] like “%codecamp%”
- 7. April 25, 2015Cloudpowered search © EXPERT NETWORK WHAT IS A SEARCH ENGINE? Efficient indexing of data On all fields / combination of fields Analyzing data Text Search Tokenizing Stemming Filtering Understanding locations Relevance scoring
- 8. April 25, 2015Cloudpowered search © EXPERT NETWORK Lucene Document: collection of fields Field: string based key-value pair Collection: set of documents Inverted index: a term can list the number of documents it contains Score: relevancy for each document matching the query
- 9. April 25, 2015Cloudpowered search © EXPERT NETWORK How searching works Id Title UserId ViewCount Tags 1 Controller Action ambiguity even with [HttpPost] decoration? (ASP.NET MVC4) 5 352 asp.net asp.net-mvc asp.net-mvc-4 f# 2 Why can't I use a scrollwheel on a webpage? 6 109 c# javascript asp.net asp.net-mvc-4 twitter- bootstrap-3 3 Access session variable of one site in another" 7 78 asp.net .net 4 Check if SIM card exists 5 209 c# windows-phone-8
- 10. April 25, 2015Cloudpowered search © EXPERT NETWORK Inverted index How searching works Title Access session variable of one site in another" 3 Check if SIM card exists 4 Controller Action ambiguity even with [HttpPost] decoration? (ASP.NET MVC4) 1 Why can't I use a scrollwheel on a webpage? 2 UserID 5 1, 4 6 2 7 3 ViewCount 78 3 109 2 209 4 352 1
- 11. April 25, 2015Cloudpowered search © EXPERT NETWORK Inverted index How searching works Title Access session variable of one site in another" 3 Check if SIM card exists 4 Controller Action ambiguity even with [HttpPost] decoration? (ASP.NET MVC4) 1 Why can't I use a scrollwheel on a webpage? 2 UserID 5 1, 4 6 2 7 3 ViewCount 78 3 109 2 209 4 352 1 Query: UserID = 5
- 12. April 25, 2015Cloudpowered search © EXPERT NETWORK Full text search Id Tags 1 asp.net asp.net-mvc asp.net-mvc- 4 f# 2 c# javascript asp.net asp.net-mvc- 4 twitter-bootstrap-3 3 asp.net .net 4 c# windows-phone-8 How searching works Term Doc .net 3 asp.net 1, 2, 3 asp.net-mvc-4 1, 2 c# 2, 4 f# 1 javascript 2 mvc 1 twitter-bootstrap-3 2 windows-phone-8 4
- 13. April 25, 2015Cloudpowered search © EXPERT NETWORK Full text search Id Tags 1 asp.net asp.net-mvc asp.net-mvc- 4 f# 2 c# javascript asp.net asp.net-mvc- 4 twitter-bootstrap-3 3 asp.net .net 4 c# windows-phone-8 How searching works Term Doc .net 3 asp.net 1, 2, 3 asp.net-mvc-4 1, 2 c# 2, 4 f# 1 javascript 2 mvc 1 twitter-bootstrap-3 2 windows-phone-8 4 Query: “javascript” in Tags
- 14. April 25, 2015Cloudpowered search © EXPERT NETWORK Full text search Id Tags 1 asp.net asp.net-mvc asp.net-mvc- 4 f# 2 c# javascript asp.net asp.net-mvc- 4 twitter-bootstrap-3 3 asp.net .net 4 c# windows-phone-8 How searching works Term Doc .net 3 asp.net 1, 2, 3 asp.net-mvc-4 1, 2 c# 2, 4 f# 1 javascript 2 mvc 1 twitter-bootstrap-3 2 windows-phone-8 4 Query: “asp.net” in Tags
- 15. April 25, 2015Cloudpowered search © EXPERT NETWORK Auto-completion Uses
- 16. April 25, 2015Cloudpowered search © EXPERT NETWORK Auto-correction Phrasing Iframe security – Security in an Iframe Word-level distance grey/gray color/colour Uses
- 17. April 25, 2015Cloudpowered search © EXPERT NETWORK Elasticsearch Distributed: aggregated results of search performed on multiple shards/indices Schema Less: is document oriented. Supports JSON format RESTful: supports REST interface Faceted Search: support for navigational search functionality Replication: supports index replication Fail over: replication and distributed nature provides inbuilt fail over. Near Real time: supports near real time updates
- 18. April 25, 2015Cloudpowered search © EXPERT NETWORK Distributed & highly available • Multiple servers (nodes) running in a cluster • Acting as single service • Nodes in cluster that store data or nodes that just help in speeding up search queries. • Sharding • Indeces are sharded (# shards is configurable) • Each shard can have zero or more replicas • Replicas on different servers (server pools) for failover • One in the cluster goes down? No problem. Elasticsearch
- 19. April 25, 2015Cloudpowered search © EXPERT NETWORK Azure search Elasticsearch as a managed service Platform as a service (PaaS) Admin by Rest API Data exchange with JSON
- 20. April 25, 2015Cloudpowered search © EXPERT NETWORK Where are we at Service Ease of use Scalability Easy Administration Manual search (SQL) No No Partial Elasticsearch Yes Yes No AzureSearch Yes Yes Yes
- 21. April 25, 2015Cloudpowered search © EXPERT NETWORK Resource model Service Index (schema type 1) Index (schema type 2) Document Document Field1 Field2 Field3 Field4 Indexers Azure Search
- 22. April 25, 2015Cloudpowered search © EXPERT NETWORK Management Portal Demo
- 23. April 25, 2015Cloudpowered search © EXPERT NETWORK Index creation POST https://codecamp-en.search.windows.net/indexes "name": "stackoverflow-posts", "fields": [ { "name": "name_of_field", "type": “data_type", "searchable": true (default where applicable) | false , "filterable": true (default) | false, "sortable": true (default where applicable) | false "facetable": true (default where applicable) | false , "key": true | false (default), "retrievable": true (default) | false } ] … Azure Search
- 24. April 25, 2015Cloudpowered search © EXPERT NETWORK Index documents Indexers Data sources: Azure SQL Database, DocumentDB Connects data sources with target search indexes An indexer can be used in the following ways: one-time copy of the data to populate an index sync an index with changes from the data source on a schedule invoke on-demand to update an index as needed Azure Search
- 25. April 25, 2015Cloudpowered search © EXPERT NETWORK CRUD Operations Add, Update, Delete POST https://codecamp-en.search.windows.net/indexes/stackoverflow/docs/index { "@search.action": "upload (default) | merge | mergeOrUpload | delete", "key_field_name": "unique_key_of_document", (key/value pair for key field from index schema) "field_name": field_value (key/value pairs matching index schema) } Azure Search
- 26. April 25, 2015Cloudpowered search © EXPERT NETWORK Searching through data GET https://codecamp-en.search.windows.net/indexes/stackoverflow/docs? search=[string] + (AND operator “code" and “camp") | (OR operator “code" or “camp" or both) - (NOT operator. “code–camp" “code" term and/or do not have “camp" ) * (Suffix operator. “cod*" - starts with “cod", ignoring case) " (Phrase search operator) ( ) (Precedence operator - code+(camp|workshop) searchMode=any|all searchFields=[string] Azure Search
- 27. April 25, 2015Cloudpowered search © EXPERT NETWORK Filtering results $filter=[string] - Odata syntax $skip=# $top=# $count=true|false $orderby=[string] $select=[string] Azure Search
- 28. April 25, 2015Cloudpowered search © EXPERT NETWORK Emphasizing results facet=[string] (field names) count sort values interval highlight=[string] (field names) highlightPreTag=[string] (default is em) highlightPostTag=[string] Azure Search
- 29. April 25, 2015Cloudpowered search © EXPERT NETWORK Suggestions GET https://codecamp-en.search.windows.net/indexes/stackoverflow/docs/suggest search=[string] suggesterName=[string] fuzzy=[boolean] searchFields=[string] Azure Search
- 30. April 25, 2015Cloudpowered search © EXPERT NETWORK Stackoverflow Posts 5.215.584 records 212 MB in Title column 118 MB in Tags column 10,5 GB in Body column Sample Data Column Name Data Type Id int CreationDate datetime Score float ViewCount int Body nvarchar OwnerUserId int Title nvarchar Tags nvarchar
- 31. April 25, 2015Cloudpowered search © EXPERT NETWORK Search API DEMO
- 32. April 25, 2015Cloudpowered search © EXPERT NETWORK Scaling Capacity measured in Search Units 1 Search Unit 1 Partition 1 Replica Horizontal scaling by increasing the number of partitions and/or replicas Cloud powered search
- 33. April 25, 2015Cloudpowered search © EXPERT NETWORK Storage Partition limitations: 15 million documents 25 GB data Every Index is split by default in 12 shards Each partition can store 1,2,3,4,6,12 shards Cloud powered search
- 34. April 25, 2015Cloudpowered search © EXPERT NETWORK SCENARIOS Online retail/ecommerce User generated/social content Not just for the web Hybrid Applications USE CASE
- 35. April 25, 2015Cloudpowered search © EXPERT NETWORK Conclusions The need for search Search explained Development Case Scenarios
- 36. April 25, 2015Cloudpowered search © EXPERT NETWORK Questions ?
- 37. April 25, 2015Cloudpowered search © EXPERT NETWORK THANK YOU