FOSS4G Bonn 2016

1. Stress Testing and Analysing
MapServer Performance
Seth Girvin, Compass Informatics

2. • Used by 30 Local Authorities to manage local roads
• MapServer accessed through browser and desktop
• Built using MapServer, OpenLayers, GeoExt, ExtJS, .NET and SQL Server
Background: Ireland’s Pavement Management System

3. Why test performance?
It seems a little slower in the
new version?
What are your hardware
requirements for the new
hosting?
Would moving to Linux
speed things up?
One of my map layers isn’t
loading!
Why isn’t my A0
map printing?

4. Measure Usage: Import web server log files into a database
• Over 2 years of data
• Over 2 million requests
• Over 1 million MapServer requests

5. Review of logs:
• Avoid premature optimisation
• Most requests are for a few layers
• GetLegendGraphic called for each layer at each scale
Usage Review
• Over 700 distinct IP
addresses (users) in
2016
• Currently around 60
users a day

6. • Python code not a UI or XML - easy to customise and extend if you know
Python
• Each user is a lightweight process (not thread) so can simulate 100s of users
(using the gevent library)
• MIT license
“Locust is an easy-to-use, distributed, user load testing tool. Intended for
load testing web sites (or other systems) and figuring out how many
concurrent users a system can handle.”
http://locust.io/
Simulating Real Users

7. Modelling Users with Locust
from locust import HttpLocust, TaskSet, task
class OWSTaskSet(TaskSet):
def on_start(self):
self.layer = 'MyLayer'
self.app = "/cgi-bin/mapserv?"
def get_response(self, name, params):
with self.client.get(self.app, params=params, verify=False,
name=name, catch_response=True) as response:
if response.headers['content-type'] == 'application/vnd.ogc.se_xml':
# if MapServer returns an error then log this with Locust
response.failure(response.content)

8. @task(1)
def get_wfs_featurecount(self):
"""
Only supported by WFS 1.1.0 not 1.0.0
"""
params = {'SERVICE': 'WFS', 'VERSION': '1.1.0',
'REQUEST': 'GetFeature', 'resultType': 'hits'}
params["TYPENAME"] = self.layer
self.get_response("GetFeature_%s" % self.layer, params)
class OWSUser(HttpLocust):
task_set = OWSTaskSet
min_wait = 5000 # minimum time between user requests
max_wait = 15000 # maximum time
Then add task set to a user:
Add tasks and weightings:

9. TODO – ADD UI PICTURE
CHECK – MEASURED IN MS?

10. SELECT [date], DATEPART(HOUR, [time]), count(c_ip)
FROM pms
WHERE cs_uri_stem = '/mapserver/'
GROUP BY [date], DATEPART(HOUR, TIME), c_ip
ORDER BY count(c_ip) DESC
Getting the heaviest load on the server
Formula for concurrent users:
Hourly Sessions x Average Session Duration (in seconds) / 3,600
Monday 6th June 2016 at 2pm:
• 12,428 MapServer requests
• 19 concurrent users

11. When does MapServer fall down?
…cannot watch more than 1024 sockets

12. Time to experiment!

13. MapServer versus Lorem Ipsum

15. OGR versus Native Driver
http://mapserver.org/input/vector/mssql.htmlhttp://www.gdal.org/drv_mssqlspatial.html

17. • Processor: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
• Memory: 8 GB
Windows versus Ubuntu

19. • Monitor and review usage
• Optimise only what is necessary
• Don’t assume - measure
Key Points

20. sgirvin@compass.ie
Questions?
• http://mapserver.org/
• http://locust.io/
• http://bit.ly/2bxboRm (Gist)