Download to read offline
![PyConCZ’16 29
HOTFIX
def receive_session_stats(request):
[...]
session = SessionData()
session.player = stats
# 1 - 5
session.start = data.get("1",0)
session.end = data.get("2",0)
[...]
session.save()
return HttpResponse()
def receive_session_stats(request):
return HttpResponse()](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-29-320.jpg)
![PyConCZ’16 33
DJANGO ORM
class Author(models.Model):
first_name = models.CharField(max_length=64)
last_name = models.CharField(max_length=128)
class Book(models.Model):
title = models.CharField(max_length=128)
author = models.ForeignKey(Author)
pages = models.IntegerField()
def get_books_by_size(request, pages_min, pages_max):
books = Book.objects
.filter(pages__gte=pages_min, pages__lte=pages_max)
return JsonResponse({
"books": [{
"id": book.id,
"title": book.title,
"pages": book.pages,
"author": {
"id": book.author.pk,
"last_name": book.author.last_name,
}
} for book in books]
})](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-33-320.jpg)
![PyConCZ’16 34
DJANGO ORM
class Book(models.Model):
title = models.CharField(max_length=128)
author = models.ForeignKey(Author)
pages = models.IntegerField()
def get_books_by_size(request, pages_min, pages_max):
books = Book.objects
.filter(pages__gte=pages_min, pages__lte=pages_max)
return JsonResponse({
"books": [{
"id": book.id,
"title": book.title,
"pages": book.pages,
"author": {
"id": book.author.pk,
"last_name": book.author.last_name,
}
} for book in books]
})](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-34-320.jpg)
![PyConCZ’16 35
DJANGO ORM
class Book(models.Model):
title = models.CharField(max_length=128)
author = models.ForeignKey(Author)
pages = models.IntegerField()
def get_books_by_size(request, pages_min, pages_max):
books = Book.objects
.filter(pages__gte=pages_min, pages__lte=pages_max)
return JsonResponse({
"books": [{
"id": book.id,
"title": book.title,
"pages": book.pages,
"author": {
"id": book.author.pk,
"last_name": book.author.last_name,
}
} for book in books]
})](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-35-320.jpg)
![PyConCZ’16 36
DJANGO ORM
class Book(models.Model):
title = models.CharField(max_length=128)
author = models.ForeignKey(Author)
pages = models.IntegerField(db_index=True)
def get_books_by_size(request, pages_min, pages_max):
books = Book.objects
.filter(pages__gte=pages_min, pages__lte=pages_max)
return JsonResponse({
"books": [{
"id": book.id,
"title": book.title,
"pages": book.pages,
"author": {
"id": book.author.pk,
"last_name": book.author.last_name,
}
} for book in books]
})](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-36-320.jpg)
![PyConCZ’16 37
DJANGO ORM
class Book(models.Model):
title = models.CharField(max_length=128)
author = models.ForeignKey(Author)
pages = models.IntegerField(db_index=True)
def get_books_by_size(request, pages_min, pages_max):
books = Book.objects
.filter(pages__gte=pages_min, pages__lte=pages_max)
return JsonResponse({
"books": [{
"id": book.id,
"title": book.title,
"pages": book.pages,
"author": {
"id": book.author.pk,
"last_name": book.author.last_name,
}
} for book in books]
})](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-37-320.jpg)
![PyConCZ’16 38
DJANGO ORM
class Book(models.Model):
title = models.CharField(max_length=128)
author = models.ForeignKey(Author)
pages = models.IntegerField(db_index=True)
def get_books_by_size(request, pages_min, pages_max):
books = Book.objects
.select_related('author')
.filter(pages__gte=pages_min, pages__lte=pages_max)
return JsonResponse({
"books": [{
"id": book.id,
"title": book.title,
"pages": book.pages,
"author": {
"id": book.author.pk,
"last_name": book.author.last_name,
}
} for book in books]
})](https://image.slidesharecdn.com/pyconcz16-loadtesting-161028095606/85/PyConCZ-16-DoS-youtself-a-k-a-Load-Testing-38-320.jpg)
More Related Content
![[CB17] Trueseeing: Effective Dataflow Analysis over Dalvik Opcodes](https://cdn.slidesharecdn.com/ss_thumbnails/d2-t2-5entakahiroyoshimura-kenyayoshimura-171228184818-thumbnail.jpg?width=640&height=640&fit=bounds)
![[ENG] OHM2013 - The Quest for the Client-Side Elixir Against Zombie Browsers -](https://cdn.slidesharecdn.com/ss_thumbnails/ohmquest-130827053519-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to PyConCZ'16 - DoS youtself a.k.a. Load Testing
PyConCZ'16 - DoS youtself a.k.a. Load Testing
- 1. PyConCZ’16 DOS YOURSELF a.k.a. LoadTesting Dariusz Aniszewski ! DariuszAniszewski " @aniszewski_eu
- 2. PyConCZ’16 ➤ z polštiny ➤9 to 5 ➤ Senior Software Engineer @ Polidea ➤ Python user since 2010 ➤ Load testing aware since 2011/2012 ➤ After hours ➤ Home-brewer ➤ IoT enthusiast 2 $ WHOAMI
- 3.
- 4. PyConCZ’16 ➤ Introduction ➤ Exampleperformance problems ➤ Tools 4 AGENDA
- 5. PyConCZ’16 1. You mustnot perform load testing on server you are not authorised to test. 2. You should not perform load testing on live, production server, even yours. 5 DISCLAIMER
- 6.
- 7.
- 8.
- 9. PyConCZ’16 ➤ Load testingis the process of putting demand on a software system or computing device and measuring its response. 9 WIKI SAYS:
- 10. PyConCZ’16 ➤ Load testingis performed to determine a system's behavior under both normal and anticipated peak load conditions. 10 WIKI SAYS:
- 11. PyConCZ’16 ➤ Make yourusers happy ➤ Don’t loose money ➤ Ensure system mets non-functional requirements ➤ usable under expected load ➤ usable under N-times larger than expected load ➤ Ensure your hardware is working efficiently ➤ Ensure your architecture is working efficiently ➤ Determine how much traffic you can handle on single node 11 REAL REASONS
- 12.
- 13. PyConCZ’16 ➤ Unit Testsare Awesome ➤ Integration Tests are Awesome too! ➤ But still not enough… ➤ Unit tests are very isolated ➤ Both of them use minimal data 13 AREN’T UNIT TESTS JUST ENOUGH?
- 14.
- 15. “ PyConCZ’16 There is absolutelyno way that all of them will hit the same API exactly at the same time. - Dariusz Aniszewski, 2011 15
- 16. PyConCZ’16 ➤ Project forweekly magazine ➤ iPad app & on-premise backend ➤ ~4000 active iPads every week ➤ Everything was running smoothly 16 BACKGROUND
- 17.
- 18. PyConCZ’16 ➤ Virtual bookshelf ➤Introduced in iOS 5 ➤ Workflow: ➤ Silent push that there is new publication available ➤ Application downloads new publication in background ➤ Application displays nice badge on its icon ➤ User opens app and new publication is ready 18 NEWSSTAND
- 19. “ PyConCZ’16 There is absolutelyno way that all of them will hit the same API exactly at the same time. - Dariusz Aniszewski, 2011 19 !! WRONG !!
- 20. PyConCZ’16 ➤ Every deviceconnected to the Internet hit our API at once ➤ Hundreds of parallel downloads of ~90MB packages ➤ No crash ;-) ➤ Download speed was terrible, usability was poor ➤ On-premise network infrastructure was bottleneck ➤ Moved to S3 week later 20 NEWSSTAND
- 21.
- 22. PyConCZ’16 ➤ Gather lotsand lots of stats from mobile app ➤ Store them locally for short period of time ➤ Export them to BigTable ➤ Make it readable via Django 22 OBJECTIVES
- 23. PyConCZ’16 SESSION DATA 23 class SessionData(models.Model): player= models.ForeignKey(PlayerStats) # 1 - 5 start = models.IntegerField(blank=True,null=True) end = models.IntegerField(blank=True,null=True) length = models.IntegerField(blank=True,null=True) since_last = models.IntegerField(blank=True,null=True) referrer = models.CharField(max_length=2000,blank=True,null=True) # 6-7 NoAPS = models.TextField(blank=True,null=True) NoGCPAPS = models.IntegerField(blank=True,null=True)
- 24. PyConCZ’16 GETTING WORSE 24 # 8- 23 VoGCPAPS = models.IntegerField(blank=True,null=True) AVoGCPAPS = models.IntegerField(blank=True,null=True) NoDPAPS = models.IntegerField(blank=True,null=True) VoDPAPS = models.IntegerField(blank=True,null=True) AvoDPAPS = models.IntegerField(blank=True,null=True) USDSPS = models.FloatField(blank=True,null=True) VoDPPS_USD = models.FloatField(blank=True,null=True) VoGCPPS_USD = models.FloatField(blank=True,null=True) VoGCPPS_DINARS = models.FloatField(blank=True,null=True) EUPS = models.IntegerField(blank=True,null=True) EDPS = models.IntegerField(blank=True,null=True) EPPS = models.FloatField(blank=True,null=True) NoTAAPS = models.IntegerField(blank=True,null=True) VoTAAPS = models.FloatField(blank=True,null=True) DoTAAPS = models.TextField(blank=True,null=True) GLaSS = models.IntegerField(blank=True,null=True)
- 25. PyConCZ’16 AND WORSE 25 # 24-40 GLaSE= models.IntegerField(blank=True,null=True) PaSS = models.IntegerField(blank=True,null=True) PaSE = models.IntegerField(blank=True,null=True) SCBaSS = models.IntegerField(blank=True,null=True) SCBaSE = models.IntegerField(blank=True,null=True) GCBaSS = models.IntegerField(blank=True,null=True) GCBaSE = models.IntegerField(blank=True,null=True) GCUPS = models.IntegerField(blank=True,null=True) GCCPS = models.IntegerField(blank=True,null=True) GCBPS = models.TextField(blank=True,null=True) DBaSS = models.IntegerField(blank=True,null=True) DBaSE = models.IntegerField(blank=True,null=True) DUPS = models.IntegerField(blank=True,null=True) DCPS = models.IntegerField(blank=True,null=True) DBPS = models.TextField(blank=True,null=True) EBaSS = models.IntegerField(blank=True,null=True) EBaSE = models.IntegerField(blank=True,null=True) EUPS_all = models.IntegerField(blank=True,null=True)
- 26. PyConCZ’16 AND WORS… OHCOME ON... 26 # 41 - 56 OECOS = models.IntegerField(blank=True,null=True) PECPPS = models.FloatField(blank=True,null=True) EBPS = models.TextField(blank=True,null=True) APPS = models.TextField(blank=True,null=True) VoAAPPS = models.FloatField(blank=True,null=True) APwDPS = models.TextField(blank=True,null=True) VoAAPwDPS = models.FloatField(blank=True,null=True) APwCPS = models.TextField(blank=True,null=True) VoAAPwCPS = models.FloatField(blank=True,null=True) ExPPS = models.IntegerField(blank=True,null=True) VoAEPPS = models.FloatField(blank=True,null=True) EPwCPS = models.IntegerField(blank=True,null=True) VoEPwCPS = models.FloatField(blank=True,null=True) EPwDPS = models.IntegerField(blank=True,null=True) VoEPwDPS = models.FloatField(blank=True,null=True) poAMOBEP = models.TextField(blank=True,null=True)
- 27. PyConCZ’16 UH. END. 27 # 57- 63 APADtIF = models.TextField(blank=True,null=True) LABSE = models.CharField(max_length=200,blank=True,null=True) LPCaSS = models.IntegerField(blank=True,null=True) LPCaSE = models.IntegerField(blank=True,null=True) PADaPS = models.IntegerField(blank=True,null=True) LTaSP = models.IntegerField(blank=True,null=True) SPDO = models.IntegerField(blank=True,null=True)
- 28. PyConCZ’16 Overloaded database withbig inserts Drastic decrease of response time Service was unusable Drastic decrease of active users 28 PROBLEM
- 29. PyConCZ’16 29 HOTFIX def receive_session_stats(request): [...] session= SessionData() session.player = stats # 1 - 5 session.start = data.get("1",0) session.end = data.get("2",0) [...] session.save() return HttpResponse() def receive_session_stats(request): return HttpResponse()
- 30. PyConCZ’16 class NewSessionData(models.Model): player =models.ForeignKey(PlayerStats) data = models.TextField() ➤ Gather lots and lots of stats from mobile app ➤ Store them locally for short period of time ➤ Export them to BigTable ➤ Make it readable via Django 30 LONG TERM SOLUTION
- 31.
- 32. PyConCZ’16 ➤ Is awesome! ➤Is easy to use, powerful and generally great ➤ Is dangerous! ➤ It makes it very easy to forget about performance ➤ Is magical ➤ You don’t see queries that are generated ➤ Those queries might be not optimal 32 DJANGO ORM
- 33. PyConCZ’16 33 DJANGO ORM classAuthor(models.Model): first_name = models.CharField(max_length=64) last_name = models.CharField(max_length=128) class Book(models.Model): title = models.CharField(max_length=128) author = models.ForeignKey(Author) pages = models.IntegerField() def get_books_by_size(request, pages_min, pages_max): books = Book.objects .filter(pages__gte=pages_min, pages__lte=pages_max) return JsonResponse({ "books": [{ "id": book.id, "title": book.title, "pages": book.pages, "author": { "id": book.author.pk, "last_name": book.author.last_name, } } for book in books] })
- 34. PyConCZ’16 34 DJANGO ORM classBook(models.Model): title = models.CharField(max_length=128) author = models.ForeignKey(Author) pages = models.IntegerField() def get_books_by_size(request, pages_min, pages_max): books = Book.objects .filter(pages__gte=pages_min, pages__lte=pages_max) return JsonResponse({ "books": [{ "id": book.id, "title": book.title, "pages": book.pages, "author": { "id": book.author.pk, "last_name": book.author.last_name, } } for book in books] })
- 35. PyConCZ’16 35 DJANGO ORM classBook(models.Model): title = models.CharField(max_length=128) author = models.ForeignKey(Author) pages = models.IntegerField() def get_books_by_size(request, pages_min, pages_max): books = Book.objects .filter(pages__gte=pages_min, pages__lte=pages_max) return JsonResponse({ "books": [{ "id": book.id, "title": book.title, "pages": book.pages, "author": { "id": book.author.pk, "last_name": book.author.last_name, } } for book in books] })
- 36. PyConCZ’16 36 DJANGO ORM classBook(models.Model): title = models.CharField(max_length=128) author = models.ForeignKey(Author) pages = models.IntegerField(db_index=True) def get_books_by_size(request, pages_min, pages_max): books = Book.objects .filter(pages__gte=pages_min, pages__lte=pages_max) return JsonResponse({ "books": [{ "id": book.id, "title": book.title, "pages": book.pages, "author": { "id": book.author.pk, "last_name": book.author.last_name, } } for book in books] })
- 37. PyConCZ’16 37 DJANGO ORM classBook(models.Model): title = models.CharField(max_length=128) author = models.ForeignKey(Author) pages = models.IntegerField(db_index=True) def get_books_by_size(request, pages_min, pages_max): books = Book.objects .filter(pages__gte=pages_min, pages__lte=pages_max) return JsonResponse({ "books": [{ "id": book.id, "title": book.title, "pages": book.pages, "author": { "id": book.author.pk, "last_name": book.author.last_name, } } for book in books] })
- 38. PyConCZ’16 38 DJANGO ORM classBook(models.Model): title = models.CharField(max_length=128) author = models.ForeignKey(Author) pages = models.IntegerField(db_index=True) def get_books_by_size(request, pages_min, pages_max): books = Book.objects .select_related('author') .filter(pages__gte=pages_min, pages__lte=pages_max) return JsonResponse({ "books": [{ "id": book.id, "title": book.title, "pages": book.pages, "author": { "id": book.author.pk, "last_name": book.author.last_name, } } for book in books] })
- 39.
- 40.
- 41. PyConCZ’16 ➤ Inside job ➤You must ➤ Have a deep understanding of system ➤ Set up internal monitoring software on your system ➤ Without it, you just monitor average response time 41 BEFORE YOU BEGIN
- 42. PyConCZ’16 ➤ Application monitoring ➤Availability monitoring ➤ Error reporting ➤ SLA calculator ➤ Integrates with your app via agent ➤ Works very well with Python 42 NEW RELIC
- 43.
- 44.
- 45.
- 46. PyConCZ’16 ➤ Custom breakdownsegments available. 46 EVEN MORE DETAILED BREAKDOWN
- 47. PyConCZ’16 ➤ Good toknow ➤ Helps with maintenance planning 47 REQUESTS PER MINUTE
- 48. PyConCZ’16 48 SLOWEST SQLQUERIES
- 49.
- 50.
- 51.
- 52. PyConCZ’16 ➤ part ofApache Server ➤ ab -n 100 -c 10 http://hit--me.herokuapp.com/100ms ➤ -n <= total number of requests to be made ➤ -c <= maximum concurrency level 52 APACHE BENCH
- 53.
- 54.
- 55.
- 56. PyConCZ’16 JMETER 56 ➤ Pros: ➤ verypowerful load testing tool ➤ tests can be imported to some cloud services ➤ load tests almost everything ➤ Cons: ➤ Complicated ➤ Big entry threshold
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64. PyConCZ’16 ➤ Pros: ➤ Cantest exactly what you need ➤ Cons: ➤ Possible re-inventing a wheel ➤ Who said your testing script is optimal… :) 64 CUSTOM SCRIPT
- 65.
- 66. PyConCZ’16 ➤ locust.io ➤ Pythonbased, using gevent ➤ Well documented ➤ Load testing as Python code ➤ Distributed tests support ➤ Small problem: Python 2 only 66 LOCUST
- 67. PyConCZ’16 67 LOCUST from locustimport HttpLocust, TaskSet, task class HitMeTasks(TaskSet): @task(3) def index(self): self.client.get("/") @task(10) def test100ms(self): self.client.get("/100ms") class HitMeLocust(HttpLocust): host = "http://hit--me.herokuapp.com" task_set = HitMeTasks min_wait = 100 max_wait = 100
- 68.
- 69.
- 70.
- 71.
- 72. PyConCZ’16 ➤ Load Testingas a Service: ➤ loader.io ➤ blazemeter.io ➤ loadimpact.com ➤ and-so-on.com ➤ Server ownership verification ➤ Generally easy to use 72 CLOUD
- 73. PyConCZ’16 ➤ Distributed ➤ APIavailable ➤ Email results ➤ Free plan ➤ 1 host ➤ 10,000 users ➤ 2 endpoints ➤ 1 minute test 73 LOADER.IO ➤ Pro plan ➤ ∞ hosts ➤ 100,000 users ➤ 10 endpoints ➤ 10 minutes tests
- 74.
- 75.
- 76.
- 77.
- 78.
- 79. PyConCZ’16 ➤ Include LoadTesting into your workflow ➤ No magic formula, adopt solution to problem ➤ Be rational ➤ Don’t be afraid to break your system! ➤ Don’t Load Test your production! ➤ Don’t “Load Test” other servers!! 79 FINAL THOUGHTS
- 80. PyConCZ’16 THANK YOU Questions? ! DariuszAniszewski "@aniszewski_eu