Getting started with Elasticsearch and .NET

32,865 views

Published on

Slides for my presentation about how to get started with Elasticsearch and .NET using Nest.

Published in: Technology
3 Comments
20 Likes
Statistics
Notes
  • hi, Very nice article. It is really helpful for all developer and other. I would like add more details regarding how to used ElasticSearch with your favourite programming language. Check out - http://www.multidots.com/what-is-elasticsearch How to integrate with PHP? You can find PHP client api on github: https://github.com/elastic/elasticsearch-php Thank you
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • @roser137, thank you! Since things are always changing there might be some parts that are outdated in the presentation, hopefully I'll get time to update it soon to match the 1.0 release of Elasticsearch and Nest (when it is also updated).
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • One of best resources for a quick review of elasticsearch and .net . Thanks for sharing !
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
32,865
On SlideShare
0
From Embeds
0
Number of Embeds
5,139
Actions
Shares
0
Downloads
238
Comments
3
Likes
20
Embeds 0
No embeds

No notes for slide
  • Present thetopic Deployment
  • Trenger en ny profilbilde
  • One of the 108 projects that Karel mentioned
  • C:\Dev\git> scriptcsscriptcs (ctrl-c or blank to exit)> usingSystem.Linq.Expressions;> Func<int, int, int> add = (x, y) => x + y;> add(1, 3)4> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;> addExpr(1, 3)(1,1): error CS1955: Non-invocablemember 'addExpr' cannot be used like a method.> var body = addExpr.Body as BinaryExpression;> Console.WriteLine(body);(x + y)
  • C:\Dev\git> scriptcsscriptcs (ctrl-c or blank to exit)> usingSystem.Linq.Expressions;> Func<int, int, int> add = (x, y) => x + y;> add(1, 3)4> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;> addExpr(1, 3)(1,1): error CS1955: Non-invocablemember 'addExpr' cannot be used like a method.> var body = addExpr.Body as BinaryExpression;> Console.WriteLine(body);(x + y)
  • C:\Dev\git> scriptcsscriptcs (ctrl-c or blank to exit)> usingSystem.Linq.Expressions;> Func<int, int, int> add = (x, y) => x + y;> add(1, 3)4> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;> addExpr(1, 3)(1,1): error CS1955: Non-invocablemember 'addExpr' cannot be used like a method.> var body = addExpr.Body as BinaryExpression;> Console.WriteLine(body);(x + y)
  • C:\Dev\git> scriptcsscriptcs (ctrl-c or blank to exit)> usingSystem.Linq.Expressions;> Func<int, int, int> add = (x, y) => x + y;> add(1, 3)4> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;> addExpr(1, 3)(1,1): error CS1955: Non-invocablemember 'addExpr' cannot be used like a method.> var body = addExpr.Body as BinaryExpression;> Console.WriteLine(body);(x + y)
  • C:\Dev\git> scriptcsscriptcs (ctrl-c or blank to exit)> usingSystem.Linq.Expressions;> Func<int, int, int> add = (x, y) => x + y;> add(1, 3)4> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;> addExpr(1, 3)(1,1): error CS1955: Non-invocablemember 'addExpr' cannot be used like a method.> var body = addExpr.Body as BinaryExpression;> Console.WriteLine(body);(x + y)
  • Getting started with Elasticsearch and .NET

    1. 1. GETTING STARTED WITH ELASTICSEARCH ON WINDOWS AND .NET WITH NEST A short introduction Oslo/NNUG Meetup Tomas Jansson 29/01/2014
    2. 2. THIS IS ME Tomas Jansson Manager & Group Lead .NET BEKK Oslo @TomasJansson tomas.jansson@bekk.no github.com/mastoj blog.tomasjansson.com
    3. 3. TL;DR; https://github.com/mastoj/NestDemo
    4. 4. AUDIENCE N00b Expert N00b Expert
    5. 5. BACKGROUND This is the data and we need this new application
    6. 6. THE MASTERPLAN
    7. 7. WHAT I WANT TO SHOW YOU IS... Elasticsearch is awesome Indexing using NEST Querying using NEST ... not about advanced elasticsearch hosting
    8. 8. INSTALLATION Great news, install as a service added in 0.90.5 Powershell to the rescue
    9. 9. NEST Abstraction over Elasticsearch There is an low level abstraction as well called RawElasticClient
    10. 10. NEST Abstraction Fluent & Strongly over Elasticsearch typed
    11. 11. Functional C#
    12. 12. FUNC DEMO C:Devgit> scriptcs scriptcs (ctrl-c or blank to exit) > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 Func  executable
    13. 13. SIMPLE EXPRESSION DEMO > using System.Linq.Expressions; > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) Expression  ”function description” (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var binExpr = addExpr.Body as BinaryExpression; > Console.WriteLine(binExpr); (x + y) > var add2 = addExpr.Compile(); > add2(3, 1); 4
    14. 14. MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString
    15. 15. MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; Enables us to translate from one domain to another in an ”easy” manner > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString
    16. 16. Show me the code!
    17. 17. ELASTICSEARCH CONNECTION public class ElasticClientWrapper : ElasticClient { private static string _connectionString = Settings.ElasticSearchServer; private static ConnectionSettings _settings = new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200 .SetDefaultIndex(Settings.Alias) //"customer_product_mapping" .UsePrettyResponses(); public ElasticClientWrapper() : base(_settings) { } } //usage var client = new ElasticClientWrapper();
    18. 18. MAPPING public class Product { public double UnitPrice { get; set; } public int TotalQuantity { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string ProductName { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string CategoryName { get; set; } } public class Customer { public string CustomerID { get; set; } public string CompanyName { get; set; } public string Address { get; set; } public string City { get; set; } public string Country { get; set; } [ElasticProperty(Type = FieldType.nested)] public Product[] Products { get; set; } }
    19. 19. MAPPING & INDEXING _client = new ElasticClientWrapper(); Mapping created from attributes _client.CreateIndex("indexName", s => s.AddMapping<Customer>(m => m.MapFromAttributes())); var customers = _customerRepo.GetCustomers(); _client.IndexMany(customers, "indexName"); Indexing will use the mapping for the specified index There is async versions of the methods
    20. 20. ALIAS _client = new ElasticClientWrapper(); _client.Alias("indexName", "aliasName"); Alias Index_01
    21. 21. SWAPPING _client = new ElasticClientWrapper(); _client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } ); Alias 1. Index_02 Swap 3. Index_01 Create new index 2. Alias Delete old index
    22. 22. MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES) public class SearchModel { private int? _numberToTake; public string Query { get; set; } public Dictionary<string, IEnumerable<string>> Filter { get; set; } public int? NumberToTake { get { return _numberToTake.HasValue ? _numberToTake.Value : 25; } set { _numberToTake = value; } } }
    23. 23. QUERYING Elasticsearch NEST { _client.Search<Customer>(sd => sd .QueryString(Input.Query)); "query": { "query_string": { "query": "tomas" } } }
    24. 24. FUZZY Elasticsearch NEST { _client.Search<Customer>(sd => sd .Query(q => q .Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query)))); "query": { "fuzzy": { "_all": { "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } } } Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»
    25. 25. FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH { "query": { "bool": { "should": [{ "match": { "_all": { "query": "tomas" } } }, { "fuzzy": { "_all": { "boost": 0.1, "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } }] } } }
    26. 26. FUZZY IMPROVED (USING BOOL QUERY) - NEST _client.Search<Customer>(sd => sd .Query(q => q .Bool(b => b .Should(new Func<QueryDescriptor<Customer>, BaseQuery>[] { _ => _.Match(m => m .OnField("_all") .QueryString(Input.Query)), _ => _.Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query) .Boost(0.1)) }))));
    27. 27. HIGHLIGHT RESULT - ELASTICSEARCH { "query": { // see previous example }, "highlight": { "pre_tags": [ "<span class='highlight'>" ], "post_tags": [ "</span>" ], "fields": { "companyName": { "fragment_size": 100, "number_of_fragments": 1 } } } }
    28. 28. HIGHLIGHT RESULT - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight(h => h .PreTags("<span class='highlight'>") .PostTags("</span>") .OnFields(new Action<HighlightFieldDescriptor<Customer>>[] { _ => _ .OnField(c => c.CompanyName) .NumberOfFragments(1) .FragmentSize(100) })));
    29. 29. FACETS - ELASTICSEARCH { "query": { /* See previous example */ }, "highlight": { /* See previous example */ }, "facets": { "products.productName": { "nested": "products", "terms": { "field": "products.productName", "size": 1000 } }, "products.categoryName": { "nested": "products", "terms": { "field": "products.categoryName", "size": 1000 } }, "country": { "terms": { "field": "country", "size": 1000 } } } }
    30. 30. FACETS - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight( /* See previous example */ ) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].ProductName) .Size(1000)) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].CategoryName) .Size(1000)) .FacetTerm(f => f .OnField(c => c.Country) .Size(1000)));
    31. 31. http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html
    32. 32. FILTERS - ELASTICSEARCH { "query": { "filtered": { "query": { /* See previous example */ }, "filter": { "bool": { "must": [ { "terms": { "country": ["usa"] } }, { "nested": { "query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } }, "path": "products" } }, { "nested": { "query": { "terms": { "products.productName": ["Chai"] } }, "path": "products" } } ] } } } }, "facets": { /* See previous example */}, "highlight": { /* See previous example */ } }
    33. 33. FILTERS – NEST – PART 1, THE CUSTOMERS FILTER private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); }
    34. 34. FILTERS – NEST – PART 1, THE PRODUCTS FILTER private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); }
    35. 35. FILTERS – NEST – PART 1, THE MAGIC DICTIONARY public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };
    36. 36. FILTERS – NEST – PART 1, ALL THE HELPERS private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); } private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); } public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };
    37. 37. FILTERS – NEST – PART 2, THE QUERY _client.Search<Customer>(sd => sd .Query(q => q .Filtered(fq => { fq.Query(qs => { if (!string.IsNullOrEmpty(Input.Query)) { qs.Bool( /* See previous example */ )); } else { qs.MatchAll(); } return qs; }); if (Input.Filter.Count > 0) { var filters = Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray(); fq.Filter(fs => fs.Bool(bf => bf.Must(filters))); } })) .Highlight( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ );
    38. 38. SUMMARY Elasticsearch NEST Easy installation Strongly typed client Awesome search engine Fluent Abstraction over Elasticsearch
    39. 39. RESOURCES Demo code: https://github.com/mastoj/NestDemo Nest documentation: http://nest.azurewebsites.net/ Nest source code: https://github.com/Mpdreamz/NEST Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense
    40. 40. Questions?
    41. 41. Thank you! @TomasJansson

    ×