Tsung Intro presentation 2013


Published on

Published in: Technology

Tsung Intro presentation 2013

  1. 1. Uncertainty is a recipe for failure! TSUNG LOAD TESTING Testing HTTP(S)/ Databases / LDAP/ XMPP.. <insert your plugin here> January 2013 Steffen Larsen (slarsen@braintrust) twitter: @zooldkSaturday, January 26, 13
  2. 2. TSUNG INTRO Introduction to Tsung Walk through the phases of a TSUNG load test Mainly talk about using TSUNG on the HTTP protocol Show simple use cases (my installation is fcked - sorry!) Drink a beer!Saturday, January 26, 13
  3. 3. TSUNG INTRO The purpose of Tsung is to simulate users in order to test the scalability and performance of IP based client/server applications. Open Source (GPLv2) Based on Erlang which implicitly defines it as: Highly concurrent and blazingly fast.. Small memory fingerprint (512 bytes for an erlang process vs. 512k default for a JVM stack for a thread) Distributed by the nature of Erlang and easily done (so we can do DDOS!) :-) Multi protocol: HTTP (REST, SOAP,WebDav etc), PostgreSQL, MySQL, LDAP, XMPP/Jabber. Support for SSL and Cookie handling Easy Setup! - no confusing GUI (vi and a DTD is enough). Xorg might not be available on the servers.Saturday, January 26, 13
  4. 4. TSUNG IN HTTP LAND Supports both old HTTP 1.0 and 1.1 (Web Sockets on a branch) GET, POST, PUT, DELETE etc. Methods support Cookies are automatically handled WWW basic authentication Basic support for SOAP using HTTP mode Proxy mode to record sessions using a web browser..Saturday, January 26, 13
  5. 5. TSUNG LIFE CYCLE 0.write your tsung.xml 1.run 2.tsung gen. data XML Tsung server 3.tsung measures reports 4.report generationSaturday, January 26, 13
  6. 6. TSUNG PHASES.. Define load clients (localhost or multiple for massive/distributed testing) Define servers you want to load test Define the load scenario (num. users, arrivals etc, timing) Define your monitors (servers you are load testing AND related systems) Record or define your user scenarios (proxy and record or manually edit config) Run! (tsung -f config.xml start) Grab a beer.. (cheers!) Generate Graphs (for the data reported by tsung demon)Saturday, January 26, 13
  7. 7. DEMO show: simple_example.xml config run: tsung -f simple_example.xml start see the log file (default $HOME/.tsung/log/<timestamp>Saturday, January 26, 13
  8. 8. RECORDING.. You can either define you data by hand in the configuration file or record it via. a proxy and a web browser. tsung-recorder -L 9000 start (recorder will listen on port 9000) Now start you browser, configure a new proxy and hit the page you want to record. Use the out put in your tsung xml file and voila!Saturday, January 26, 13
  9. 9. WHY TSUNG? Your own scripts with curl, wget.. - Stop that! AB Apache Bench (slow as he..) HTTPPerf (better but still not fast and not distributed) JMeter (Java based) - quite good, Nice GUI, distributed! but JVM mem and consumption ! Grinder (Java based)?? Gatling-Tools (JVM, scala, looks promising..using the akka actor principle for concurrency - instead of normal threads - similar to erlang)Saturday, January 26, 13
  10. 10. BURNING SERVERS 0/1? load testing or load generation! 0 VS 1Saturday, January 26, 13
  11. 11. MONITOR YOUR LOAD! Default SNMP OIDS for CPU usage, network traffic, load, memory etc. - No need to install a special agent on the tested server Define your own OIDs! (patch in 1.4.x done by my poor erlang skills) With this you can measure your JVM, disk, IO etc. Network always the problem... there should be open for SNMP and the ports your are testing!Saturday, January 26, 13
  12. 12. GRAPHS! Tsung have now generated data in the log directory (default $HOME/.tsung/log.. Can be set different with -d option) /usr/local/Cellar/tsung/1.4.2/lib/tsung/bin/tsung_stats.pl <log> Now HTML is ready and can be served and showed offSaturday, January 26, 13
  13. 13. DEMO tsung -f monitor_example.xml start see the log file (default $HOME/.tsung/log/<timestamp> Generate graphs from your monitors: /usr/local/Cellar/tsung/1.4.2/lib/tsung/bin/ tsung_stats.pl <log>Saturday, January 26, 13
  14. 14. TSUNG XML CONFIG more advanced stuff Many options: set browser agents, probability for hit Get content from files (CVS or.. define your pattern) Mix files or code into config with substitution Use xpath or jsonpath to MATCH on the response and make you action! Use regular expressions as well.. (GNU RegExp) Programmatically (use conditionals: for loops, if, repeats or variables etc. for testing)Saturday, January 26, 13
  15. 15. DISTRIBUTED TESTS Tsung distributed load testing is based on SSH shell and erlang distribution system. Distribution is easy! just add more clients! Remember to make network accessible for the controller and workers - ssh login with a key from controller to all the slave nodes (workers) should be available Same version of tsung/erlang on EVERY machine (controller/workers)Saturday, January 26, 13
  16. 16. DISTRIBUTED TESTS 0.write your tsung.xml 1.run Tsung generate/gather worker Tsung XML controller Tsung server worker Tsung worker reports multiple servers as workers 4.report generationSaturday, January 26, 13
  17. 17. MORE ADVANCED STUFF Overlay/merge your graphs, useful for comparing (tsplot) Besides SNMP we can monitor with Munin or Erlang agents on the remote servers.. Choose another backend: text, json, rrdtool etc. (if you want another graphing tool or what ever) Measure in realtime or generate afterwards.Saturday, January 26, 13
  18. 18. MORE ADVANCED STUFF The recording proxy also works for database protocols (query proxy) Mix your sessions types (mysq with HTTP or XMPP ...)Saturday, January 26, 13
  19. 19. TSUNG GOTCHA’S If you are new at Erlang, dumps of stack traces can be hard if anything is wrong! Remember to write the config in the correct order of the DTD (its picky!) The path to the DTD was wrong.. comes with funny exceptions! Turn off debugging when testing - it takes time to write. Ensure that your versions is the same and compiled (if not a package) in the same way if you are running distributed!Saturday, January 26, 13
  20. 20. TSUNG GOTCHA’S For massive testing set sysctl.conf for many client connections.. set the hard limit of “max files open” larger than 1024. Do that on every controller and node! (otherwise you limit yourself!) Do some tuning of the network stack! memory for TCP , read, write and backlog. Example: net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 # This gives the kernel more memory for TCP # which you need with many (100k+) open socket connections net.ipv4.tcp_mem = 50576 64768 98152 # Backlog net.core.netdev_max_backlog = 2048 net.core.somaxconn = 1024 net.ipv4.tcp_max_syn_backlog = 2048 net.ipv4.tcp_syncookies = 1Saturday, January 26, 13
  21. 21. TESTING PLAN GOTCHAS Understand what you are testing Prepare your self: write down your expectations, what went wrong and what to improve.. Use a wiki (JIRA) Store your test for later comparison (use tsplot etc.) Test on the correct network. Plan which one. Take into consideration that you have to reach the servers being tested (both test port and SNMP) Prepare your network a massive load!. Be prepared that a switch will blow up or so..Saturday, January 26, 13
  22. 22. LINKS ETC. http://tsung.erlang-projects.org http://dak1n1.com/blog/14-http-load-generate http://programmingobsession.blogspot.dk/2012/09/tsung- vs-jmeterj.html Installation: Mac (homebrew): brew install tsung Linux: apt-get install tsung Windows? Compile your own -- a lot of dependencies..Saturday, January 26, 13