Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting started with Elasticsearch and .NET

34,665 views

Published on

Slides for my presentation about how to get started with Elasticsearch and .NET using Nest.

Published in: Technology
  • DOWNLOAD FULL. BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • hi, Very nice article. It is really helpful for all developer and other. I would like add more details regarding how to used ElasticSearch with your favourite programming language. Check out - http://www.multidots.com/what-is-elasticsearch How to integrate with PHP? You can find PHP client api on github: https://github.com/elastic/elasticsearch-php Thank you
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @roser137, thank you! Since things are always changing there might be some parts that are outdated in the presentation, hopefully I'll get time to update it soon to match the 1.0 release of Elasticsearch and Nest (when it is also updated).
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • One of best resources for a quick review of elasticsearch and .net . Thanks for sharing !
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Getting started with Elasticsearch and .NET

  1. 1. GETTING STARTED WITH ELASTICSEARCH ON WINDOWS AND .NET WITH NEST A short introduction Oslo/NNUG Meetup Tomas Jansson 29/01/2014
  2. 2. THIS IS ME Tomas Jansson Manager & Group Lead .NET BEKK Oslo @TomasJansson tomas.jansson@bekk.no github.com/mastoj blog.tomasjansson.com
  3. 3. TL;DR; https://github.com/mastoj/NestDemo
  4. 4. AUDIENCE N00b Expert N00b Expert
  5. 5. BACKGROUND This is the data and we need this new application
  6. 6. THE MASTERPLAN
  7. 7. WHAT I WANT TO SHOW YOU IS... Elasticsearch is awesome Indexing using NEST Querying using NEST ... not about advanced elasticsearch hosting
  8. 8. INSTALLATION Great news, install as a service added in 0.90.5 Powershell to the rescue
  9. 9. NEST Abstraction over Elasticsearch There is an low level abstraction as well called RawElasticClient
  10. 10. NEST Abstraction Fluent & Strongly over Elasticsearch typed
  11. 11. Functional C#
  12. 12. FUNC DEMO C:Devgit> scriptcs scriptcs (ctrl-c or blank to exit) > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 Func  executable
  13. 13. SIMPLE EXPRESSION DEMO > using System.Linq.Expressions; > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) Expression  ”function description” (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var binExpr = addExpr.Body as BinaryExpression; > Console.WriteLine(binExpr); (x + y) > var add2 = addExpr.Compile(); > add2(3, 1); 4
  14. 14. MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString
  15. 15. MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; Enables us to translate from one domain to another in an ”easy” manner > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString
  16. 16. Show me the code!
  17. 17. ELASTICSEARCH CONNECTION public class ElasticClientWrapper : ElasticClient { private static string _connectionString = Settings.ElasticSearchServer; private static ConnectionSettings _settings = new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200 .SetDefaultIndex(Settings.Alias) //"customer_product_mapping" .UsePrettyResponses(); public ElasticClientWrapper() : base(_settings) { } } //usage var client = new ElasticClientWrapper();
  18. 18. MAPPING public class Product { public double UnitPrice { get; set; } public int TotalQuantity { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string ProductName { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string CategoryName { get; set; } } public class Customer { public string CustomerID { get; set; } public string CompanyName { get; set; } public string Address { get; set; } public string City { get; set; } public string Country { get; set; } [ElasticProperty(Type = FieldType.nested)] public Product[] Products { get; set; } }
  19. 19. MAPPING & INDEXING _client = new ElasticClientWrapper(); Mapping created from attributes _client.CreateIndex("indexName", s => s.AddMapping<Customer>(m => m.MapFromAttributes())); var customers = _customerRepo.GetCustomers(); _client.IndexMany(customers, "indexName"); Indexing will use the mapping for the specified index There is async versions of the methods
  20. 20. ALIAS _client = new ElasticClientWrapper(); _client.Alias("indexName", "aliasName"); Alias Index_01
  21. 21. SWAPPING _client = new ElasticClientWrapper(); _client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } ); Alias 1. Index_02 Swap 3. Index_01 Create new index 2. Alias Delete old index
  22. 22. MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES) public class SearchModel { private int? _numberToTake; public string Query { get; set; } public Dictionary<string, IEnumerable<string>> Filter { get; set; } public int? NumberToTake { get { return _numberToTake.HasValue ? _numberToTake.Value : 25; } set { _numberToTake = value; } } }
  23. 23. QUERYING Elasticsearch NEST { _client.Search<Customer>(sd => sd .QueryString(Input.Query)); "query": { "query_string": { "query": "tomas" } } }
  24. 24. FUZZY Elasticsearch NEST { _client.Search<Customer>(sd => sd .Query(q => q .Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query)))); "query": { "fuzzy": { "_all": { "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } } } Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»
  25. 25. FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH { "query": { "bool": { "should": [{ "match": { "_all": { "query": "tomas" } } }, { "fuzzy": { "_all": { "boost": 0.1, "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } }] } } }
  26. 26. FUZZY IMPROVED (USING BOOL QUERY) - NEST _client.Search<Customer>(sd => sd .Query(q => q .Bool(b => b .Should(new Func<QueryDescriptor<Customer>, BaseQuery>[] { _ => _.Match(m => m .OnField("_all") .QueryString(Input.Query)), _ => _.Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query) .Boost(0.1)) }))));
  27. 27. HIGHLIGHT RESULT - ELASTICSEARCH { "query": { // see previous example }, "highlight": { "pre_tags": [ "<span class='highlight'>" ], "post_tags": [ "</span>" ], "fields": { "companyName": { "fragment_size": 100, "number_of_fragments": 1 } } } }
  28. 28. HIGHLIGHT RESULT - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight(h => h .PreTags("<span class='highlight'>") .PostTags("</span>") .OnFields(new Action<HighlightFieldDescriptor<Customer>>[] { _ => _ .OnField(c => c.CompanyName) .NumberOfFragments(1) .FragmentSize(100) })));
  29. 29. FACETS - ELASTICSEARCH { "query": { /* See previous example */ }, "highlight": { /* See previous example */ }, "facets": { "products.productName": { "nested": "products", "terms": { "field": "products.productName", "size": 1000 } }, "products.categoryName": { "nested": "products", "terms": { "field": "products.categoryName", "size": 1000 } }, "country": { "terms": { "field": "country", "size": 1000 } } } }
  30. 30. FACETS - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight( /* See previous example */ ) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].ProductName) .Size(1000)) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].CategoryName) .Size(1000)) .FacetTerm(f => f .OnField(c => c.Country) .Size(1000)));
  31. 31. http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html
  32. 32. FILTERS - ELASTICSEARCH { "query": { "filtered": { "query": { /* See previous example */ }, "filter": { "bool": { "must": [ { "terms": { "country": ["usa"] } }, { "nested": { "query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } }, "path": "products" } }, { "nested": { "query": { "terms": { "products.productName": ["Chai"] } }, "path": "products" } } ] } } } }, "facets": { /* See previous example */}, "highlight": { /* See previous example */ } }
  33. 33. FILTERS – NEST – PART 1, THE CUSTOMERS FILTER private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); }
  34. 34. FILTERS – NEST – PART 1, THE PRODUCTS FILTER private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); }
  35. 35. FILTERS – NEST – PART 1, THE MAGIC DICTIONARY public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };
  36. 36. FILTERS – NEST – PART 1, ALL THE HELPERS private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); } private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); } public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };
  37. 37. FILTERS – NEST – PART 2, THE QUERY _client.Search<Customer>(sd => sd .Query(q => q .Filtered(fq => { fq.Query(qs => { if (!string.IsNullOrEmpty(Input.Query)) { qs.Bool( /* See previous example */ )); } else { qs.MatchAll(); } return qs; }); if (Input.Filter.Count > 0) { var filters = Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray(); fq.Filter(fs => fs.Bool(bf => bf.Must(filters))); } })) .Highlight( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ );
  38. 38. SUMMARY Elasticsearch NEST Easy installation Strongly typed client Awesome search engine Fluent Abstraction over Elasticsearch
  39. 39. RESOURCES Demo code: https://github.com/mastoj/NestDemo Nest documentation: http://nest.azurewebsites.net/ Nest source code: https://github.com/Mpdreamz/NEST Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense
  40. 40. Questions?
  41. 41. Thank you! @TomasJansson

×