Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Java clients for elasticsearch

2,036 views

Published on

Slides for a talk on Java clients for elasticsearch for Singapore JUG. Covers the TransportClient, RestClient, Jest and Spring Data Elasticsearch

Published in: Technology
  • Be the first to comment

Java clients for elasticsearch

  1. 1. Java Clients for Florian Hopf @fhopf
  2. 2. elasticsearch
  3. 3. Elasticsearch? „Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“
  4. 4. Elasticsearch? „Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“
  5. 5. Elasticsearch?
  6. 6. Installation # download archive wget https://artifacts.elastic.co/downloads/ elasticsearch/elasticsearch-5.0.0.zip unzip elasticsearch-5.0.0.zip # on windows: elasticsearch.bat elasticsearch-5.0.0/bin/elasticsearch
  7. 7. Accessing Elaticsearch curl -XGET "http://localhost:9200" { "name" : "LI8ZN-t", "cluster_name" : "elasticsearch", "cluster_uuid" : "UvbMAoJ8TieUqugCGw7Xrw", "version" : { "number" : "5.0.0", "build_hash" : "253032b", "build_date" : "2016-10-26T04:37:51.531Z", "build_snapshot" : false, "lucene_version" : "6.2.0" }, "tagline" : "You Know, for Search" }
  8. 8. Indexing curl -XPOST "http://localhost:9200/food/dish" -d' { "food": "Hainanese Chicken Rice", "tags": ["chicken", "rice"], "favorite": { "location": "Tian Tian", "price": 5.00 } }'
  9. 9. Indexing curl -XPOST "http://localhost:9200/food/dish" -d' { "food": "Ayam Penyet", "tags": ["chicken", "indonesian"], "spicy": true }'
  10. 10. Search curl -XGET "http://localhost:9200/food/dish/_search?q=chicken" ... {"total":2,"max_score":0.3666863,"hits": [{"_index":"food","_type":"dish","_id":"AVg9cMwARrBlrY9 tYBqX","_score":0.3666863,"_source": { "food": "Hainanese Chicken Rice", "tags": ["chicken", "rice"], "favorite": { "location": "Tian Tian", "price": 5.00 } }}, ...
  11. 11. Search using Query DSL curl -XPOST "http://localhost:9200/food/dish/_search" -d' { "query": { "bool": { "must": { "match": { "_all": "rice" } }, "filter": { "term": { "tags.keyword": "chicken" } } } } }'
  12. 12. Elasticsearch? „Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“
  13. 13. Distributed
  14. 14. Distributed
  15. 15. Recap ● Java based search server ● HTTP and JSON ● Search and Filtering Query-DSL ● Faceting, Highlighting, Suggestions, … ● Nodes can form a cluster
  16. 16. Transport-Client
  17. 17. Transport-Client dependencies { compile group: 'org.elasticsearch.client', name: 'transport', version: '5.0.0' }
  18. 18. Transport-Client TransportAddress address = new InetSocketTransportAddress( InetAddress.getByName("localhost"), 9300); Client client = new PreBuiltTransportClient(Settings.EMPTY) addTransportAddress(address);
  19. 19. Search using Query DSL curl -XPOST "http://localhost:9200/food/dish/_search" -d' { "query": { "bool": { "must": { "match": { "_all": "rice" } }, "filter": { "term": { "tags.keyword": "chicken" } } } } }'
  20. 20. Search using Search Builders SearchResponse searchResponse = client .prepareSearch("food") .setQuery( boolQuery(). must(matchQuery("_all", "rice")). filter( termQuery("tags.keyword", "chicken"))) .execute().actionGet(); assertEquals(1, searchResponse.getHits().getTotalHits()); SearchHit hit = searchResponse.getHits().getAt(0); String food = hit.getSource().get("food").toString();
  21. 21. Indexing XContentBuilder builder = jsonBuilder() .startObject() .field("food", "Roti Prata") .array("tags", new String [] {"curry"}) .startObject("favorite") .field("location", "Tiong Bahru") .field("price", 2.00) .endObject() .endObject(); IndexResponse resp = client.prepareIndex("food","dish") .setSource(builder) .execute() .actionGet();
  22. 22. Indexing
  23. 23. Transport-Client ● Connects to an existing cluster ● Uses the binary protocol also used for inter cluster communication
  24. 24. Transport-Client
  25. 25. Sniffing
  26. 26. Sniffing ● State of the cluster can be requested from elasticsearch ● Client side loadbalancing TransportAddress address = new InetSocketTransportAddress( InetAddress.getByName("localhost"), 9300); Settings settings = Settings.builder() .put("client.transport.sniff", true) .build(); Client client = new PreBuiltTransportClient(settings) addTransportAddress(address);
  27. 27. Transport-Client ● Full API-Support ● Efficient communication ● Client side loadbalancing
  28. 28. Transport-Client Gotchas ● Compatibility between elasticsearch versions ● Elasticsearch dependency
  29. 29. REST-Client
  30. 30. Elasticsearch 5 REST-Client
  31. 31. Elasticsearch 5 REST-Client dependencies { compile group: 'org.elasticsearch.client', name: 'rest', version: '5.0.0' }
  32. 32. Elasticsearch 5 REST-Client +--- org.apache.httpcomponents:httpclient:4.5.2 +--- org.apache.httpcomponents:httpcore:4.4.5 +--- org.apache.httpcomponents:httpasyncclient:4.1.2 +--- org.apache.httpcomponents:httpcore-nio:4.4.5 +--- commons-codec:commons-codec:1.10 --- commons-logging:commons-logging:1.1.3
  33. 33. Elasticsearch 5 REST-Client RestClient restClient = RestClient.builder( new HttpHost("localhost", 9200, "http"), new HttpHost("localhost", 9201, "http")).build();
  34. 34. Elasticsearch 5 REST-Client HttpEntity entity = new NStringEntity( "{ "query": { "match_all": {}}}", ContentType.APPLICATION_JSON); // alternative: performRequestAsync Response response = restClient.performRequest("POST", "/_search", emptyMap(), entity); String json = toString(response.getEntity()); // ...
  35. 35. Elasticsearch 5 REST-Client ● Less dependent on elasticsearch version ● Clean separation network application/cluster ● Minimal dependency ● Sniffing ● Error handling, timeout config, basic auth, headers, … ● No query support (for now)
  36. 36. Jest – Http Client
  37. 37. Jest dependencies { compile group: 'io.searchbox', name: 'jest', version: '2.0.0' }
  38. 38. Client JestClientFactory factory = new JestClientFactory(); factory.setHttpClientConfig(new HttpClientConfig .Builder("http://localhost:9200") .multiThreaded(true) .build()); JestClient client = factory.getObject();
  39. 39. Searching Jest String query = jsonStringThatMagicallyAppears; Search search = new Search.Builder(query) .addIndex("library") .build(); SearchResult result = client.execute(search); assertEquals(Integer.valueOf(1), result.getTotal());
  40. 40. Searching Jest JsonObject jsonObject = result.getJsonObject(); JsonObject hitsObj = jsonObject.getAsJsonObject("hits"); JsonArray hits = hitsObj.getAsJsonArray("hits"); JsonObject hit = hits.get(0).getAsJsonObject(); // ... more boring code
  41. 41. Searching Jest public class Dish { private String food; private List<String> tags; private Favorite favorite; @JestId private String id; // ... getters and setters }
  42. 42. Suche mit Jest Dish dish = result.getFirstHit(Dish.class).source; assertEquals("Roti Prata", dish.getFood());
  43. 43. Jest ● Alternative HTTP implementation ● Queries as Strings or via Elasticsearch-Builder ● Indexing and searching Java beans ● Node Discovery
  44. 44. Spring Data Elasticsearch
  45. 45. Spring Data ● Abstractions for different data stores ● Speciality of each store available ● Dynamic Repository implementations ● Popular modules ● Spring Data JPA ● Spring Data MongoDB
  46. 46. Dependency dependencies { compile group: 'org.springframework.data', name: 'spring-data-elasticsearch', version: '2.0.4.RELEASE' }
  47. 47. Entity @Document(indexName = "spring_dish") public class Dish { @Id private String id; private String food; private List<String> tags; private Favorite favorite; // more code }
  48. 48. Repository public interface DishRepository extends ElasticsearchCrudRepository<Dish, String> { }
  49. 49. Configuration <elasticsearch:transport-client id="client" /> <bean name="elasticsearchTemplate" class="o.s.d.elasticsearch.core.ElasticsearchTemplate"> <constructor-arg name="client" ref="client"/> </bean> <elasticsearch:repositories base-package="de.fhopf.elasticsearch.springdata" />
  50. 50. Indexing Dish mie = new Dish(); mie.setId("hokkien-prawn-mie"); mie.setFood("Hokkien Prawn Mie"); mie.setTags(Arrays.asList("noodles", "prawn")); repository.save(Arrays.asList(hokkienPrawnMie));
  51. 51. Searching Iterable<Dish> dishes = repository.findAll(); Dish dish = repository.findOne("hokkien-prawn-mie");
  52. 52. Repository public interface DishRepository extends ElasticsearchCrudRepository<Dish, String> { List<Dish> findByFood(String food); List<Dish> findByTagsAndFavoriteLocation(String tag, String location); List<Dish> findByFavoritePriceLessThan(Double price); @Query("{"query": {"match_all": {}}}") List<Dish> customFindAll(); }
  53. 53. Recap ● High level abstraction ● Entity beans ● dynamic repository implementation ● HTTP support in the making ● Slower feature development
  54. 54. Recap ● Transport-Client ● Full API support ● Elasticsearch dependency ● REST-Client ● Uses HTTP ● currently lacking search API
  55. 55. Recap ● Jest ● Http client ● Support for Java beans ● Spring-Data-Elasticsearch ● High level abstraction ● Dynamic repositories
  56. 56. Links ● http://elastic.co ● https://github.com/searchbox-io/Jest ● https://github.com/spring-projects/spring- data-elasticsearch ● https://github.com/fhopf/singajug-examples ● http://blog.florian-hopf.de

×