The document summarizes a presentation about using Elasticsearch to improve search performance for applications with large amounts of data. It describes how the presenter previously used Elasticsearch at a previous job to speed up searches of a growing product catalog. The presenter then demonstrates how to install and use Elasticsearch with .NET applications using the Nest client library. Issues that may arise with integrating Elasticsearch into existing applications are also discussed, such as differences from relational databases and potential rework of user interfaces.

1. Amazing Speed:
Elasticsearch for the .NET Developer
4:05 PM / 301-D

2. Adrian Carr
With special guest- Shaunak Kashyap,
Developer Advocate, Elastic.co

3. A little about me…
• Software Engineer & Team Lead
o The White Stone Group in Knoxville, TN.
• Previously
o Jewelry Television
o Alltel/Fidelity
o One terrible non-profit in Boone, NC that I don’t like to talk
about.
• Several small startups
Experience:
• Full stack developer, C#, Ruby/Rails, JavaScript, Java,
etc……. “Business Problem Solver”

4. A little background…
• We have some large clients, with lots of
patients.

5. Really slow sometimes…
• Some legacy database decisions that weren’t
so good back in the day.
• Worked just fine when volume was lower.
• Re-architecting would mean changing too
many apps.
• Maybe you just have too much data to search
efficiently, or it isn’t structured for search.

6. I’ve seen this before…
• Jewelry Television
• At the time, a lot of the database design
wasn’t very good, but it worked.
o ~$200 million when I started in 2005
o ~$550 million three years later

7. The JTV Solution:

8. The JTV Solution:
• It was awesome!

9. Business Problem Solver…

10. A little research…
I found Elasticsearch -built on top of Lucene.

11. I found:
http://blog.wikimedia.org/2014/01/06/wikimedia-moving-to-elasticsearch/
Elasticsearch is used here:

12. And here:
https://www.elastic.co/assets/bltdfb74654a34935fa/case-study-github.pdf

13. And here:
http://meta.stackexchange.com/questions/160100/a-new-search-engine-for-stack-exchange

14. What is Elasticsearch?
Elasticsearch is a distributed, open
source search and analytics engine,
designed for horizontal scalability,
reliability, and easy management

15. So, I did a proof of concept
Demo Here:
Results in SQL vs Results in Elastic

16. Terminology
• Node: DB Instance. (Java process running Elasticsearch)
• Cluster: Database Cluster- One or more nodes
with same cluster name
• Index: Database, logical grouping of tables
• Type: Database Table
• Document: Like a row. JSON, key-value pairs
• Fields: Columns
• Shard: Worker processes. (These mostly happen
automagically by Elastic)

17. Tools
• Elasticsearch (Obviously)
o Demo of how to install and run. (It’s easy.)
• Browser http://localhost:9200/, etc.
• Sense- a fantastic Chrome plugin
• C# with Nest
<Multiple Demos of Indexing and Searching>

18. How to Load & Synchronize Data?
• It depends…
This is what I did:
• SQL Server Triggers on data that matters
• Windows Service
o Nest, C#, Elastic’s Bulk API (Found 10,000 rows at
a time to be the sweet spot for insertion speed)

19. Gotchas
• Elasticsearch.net vs Nest
• Documentation on Elasticsearch is fantastic.
Documentation on Nest is very sparse.

20. More Gotchas
• It’s a different way of thinking.
o Not RDBMS
o Joins? Nope
• Case sensitivity
• Index size- disk space can grow a lot
• Integration with existing application. Are your
existing users going to have their cheese
moved?

21. There is much more to know…
• Analyzers- These can be very complex
• Scaling- Elastic has a lot built in, but it can be
tuned quite a bit.
• Field weighting- Relevance of blog post vs
body vs comments
• This is just a start.

22. Questions?
• Code samples will eventually be at http://adriancarr.com
• Contact:
o adriancarr@gmail.com
o shaunak@elastic.co
o https://discuss.elastic.co/
Thank You!