Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Run Solr on Docker and Why

11,682 views

Published on

Docker is all the rage these days. While one doesn't hear much about Solr on Docker, we're here to tell you not only that it can be done, but also share how it's done.

We'll quickly go over the basic Docker ideas - containers are lighter than VMs, they solve "but it worked on my laptop" issues - so we can dive into the specifics of running Solr on Docker.

We'll do a live demo showing you how to run Solr master - slave as well as SolrCloud using containers, how to manage CPU assignments, constraint memory and use Docker data volumes when running Solr in containers. We will also show you how to create your own containers with custom configurations.

Finally, we'll address one of the core Solr questions - which deployment type should I use? We will demonstrate performance differences between the following deployment types:

- Single Solr instance running on a bare metal machine
- Multiple Solr instances running on a single bare metal machine
- Solr running in containers
- Solr running on virtual machine
- Solr running on virtual machine using unikernel

For each deployment type we'll address how it impacts performance, operational flexibility and all other key pros and cons you ought to keep in mind.

Published in: Technology
  • Be the first to comment

How to Run Solr on Docker and Why

  1. 1. O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
  2. 2. How to run Solr on Docker. And why. Rafał Kuć, Radu Gheorghe Sematext Group, Inc.
  3. 3. 3 01 Next ~40 minutes
  4. 4. 4 02 Common issues we all know overprovisioned serversresources not utilized != != != differences in environments $$$money loss
  5. 5. 5 02 One of the solutions productionQAtestdevelopment
  6. 6. 6 03 Bare metal Hardware Operating System Libraries Application Application bare metal
  7. 7. 7 03 Traditional virtual machine Hardware Operating System Libraries bare metal Hardware Host Operating System Hypervisor Guest OS Guest OS Libraries Libraries Application Application Application Application traditional VM
  8. 8. 8 03 Docker container Hardware Operating System Libraries bare metal Hardware Host Operating System traditional VM Hypervisor Guest OS Guest OS Libraries Libraries Application Application Hardware Host Operating System Docker Engine Libraries Libraries Application Application Application Application container
  9. 9. 9 03 Unikernel architecture Hardware Operating System Libraries bare metal Hardware Host Operating System traditional VM Hypervisor Guest OS Guest OS Libraries Libraries Application Application Hardware Host Operating System container Docker Engine Libraries Libraries Application Application Hardware unikernel Kernel + App + Libs Application Application Host Operating System Hypervisor
  10. 10. 10 01 Solr + Docker how to +
  11. 11. 11 01 Docker + Solr – how to docker run solr:latest docker run -p 8983:8983 solr:latest docker run --name lr_single_solr -d -p 8983:8983 -t solr:6.2.1 docker run --name lr_alpine_solr -d -p 8983:8983 -t solr:6.2-alpine docker run -it --name lr_solr_mem -p 8983:8983 -d -m 1G solr:6.2.1 docker run -p 8983:8983 -d --memory-swappiness=0 solr:6.2.1 docker run -it --cpuset-cpus="0,1" -p 8983:8983 -d solr:6.2.1 docker run --name solr_one -d -p 8983:8983 –p 9983:9983 -t solr:6.2.1 -c docker run --name solr_two -d --link solr_one -p 6983:6983 -t solr:6.2.1 -z solr_one:9983
  12. 12. 12 01 Demo time! https://github.com/sematext/lucene-revolution-samples/tree/master/2016/solr_docker DEMO TIME
  13. 13. 13 01
  14. 14. 14 01 Very simple test infrastructure Ubuntu + Solr 6.2 Ubuntu + Docker 1.12.2 (native) VMs (Ubuntu) + Solr 6.2
  15. 15. 15 01 Solr configuration – collection
  16. 16. 16 01 Solr configuration 10 stored fields Hard commit – 15 sec 10 indexed fields Soft commit - 1 sec 7 doc values
  17. 17. 17 01 Other parts Completely out of the box experience!
  18. 18. 18 01 Ubuntu + Solr, 1 Solr instance Test 1 Indexing throughput - 3348 docs/sec Test 2 Indexing throughput - 2754 docs/sec Querying throughput - 209 q/sec Sprzedaż CPU Mem I/O
  19. 19. 19 01 Ubuntu + Solr, 4 Solr instances Test 1 Indexing throughput - 3618docs/sec Test 2 Indexing throughput - 3024 docs/sec Querying throughput - 267,7 q/sec Sprzedaż CPU Mem I/O
  20. 20. 20 01 Ubuntu + 1 VM, 1 Solr instance Test 1 Indexing throughput - 2052 docs/sec Test 2 Indexing throughput - 1944 docs/sec Querying throughput - 36,7 q/sec Sprzedaż CPU Mem I/O
  21. 21. 21 01 Ubuntu + Docker, 4 Solr instances Test 1 Indexing throughput - 2754 docs/sec Test 2 Indexing throughput - 2484 docs/sec Querying throughput - 108,6 q/sec Sprzedaż CPU Mem I/O
  22. 22. 22 01 Performance summary Test results on OSX != Test results on Linux Bare metal is the fastest from those tested Docker will be faster than traditional VM Bare metal > Docker > VMs
  23. 23. 23 01
  24. 24. 24 01 Bare metal based architecture One per machine vs Multiple per machine Multiple per machine may be a better choice Harder to scale -> machine provisioning needed Full control over resources Noisy neighbour problems can appear Best or almost best performance
  25. 25. 25 01 Virtual machine based architecture Multiple VMs per machine allowed Easy to scale, but requires preparation Clear boundaries between VM Guest operating system overhead Worst performance compared to the rest
  26. 26. 26 01 Docker based architecture Multiple per host easily possible Easy to scale until host resources allow Easily add constraints per container Low overhead compared to traditional VM Native on Linux == best possible performance
  27. 27. 27 01 http://cliparts.co/clipart/3733708
  28. 28. 28 01 Should I care about Solr? Tuning is still needed, no matter what ENV you use Keep in mind ENV constraints Watch out for: - merges – adjust or split data - memory issues - I/O subsystem - CPU utilization - garbage collection - proper Solr configuration
  29. 29. 29 01 Should I care about Docker? Docker reports events We can listen and react to events Example of events to react to - kill - start - stop - destroy - die - restart - pause - unpause - oom
  30. 30. 30 01 Should I care about Docker volumes? Docker volumes also report events We can listen and react to those as well We can automatically react to: - create - mount - unmount - destroy
  31. 31. 31 01 Solr + Docker, should I? Yes, when in need of dynamic scaling Yes, when in need of resources utilization Lots of nodes -> look at swarm, mesos and kubernetes
  32. 32. 32 01 Thank you! If you want to get in touch Rafał rafal.kuc@sematext.com @kucrafal Radu radu.gheorghe@sematext.com @radu0gheorghe http://sematext.com @sematext Join US! http://sematext.com/jobs/

×