Open Street Map Installation Tutorial (Ubuntu 12.04)

16,329 views
15,792 views

Published on

references:
switch2osm.org
wiki.openstreetmap.org
leafletjs.com

Published in: Technology
1 Comment
5 Likes
Statistics
Notes
  • Be carefull, I downloaded the pdf and copyed sample 'tile html' code from the end of the document to my server apache folder but it didn't run. The thing is that in pdf there is used these symbols ” ” but should be ' ' instead. It took me 2 days to figure it out.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
16,329
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
318
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide

Open Street Map Installation Tutorial (Ubuntu 12.04)

  1. 1. Open Street Map Installation Tutorial Ubuntu 12.04 LTS Marc Huang marchuang41@gmail.com
  2. 2. OSM Components http://wiki.openstreetmap.org/wiki/Component_overview
  3. 3. Serving Tiles
  4. 4. System requirements • Operating System • Ubuntu 12.04 or 13.04 • Processor • min: dual-core processor • max: quad-core processor • Storage • min:10-20GB • max: 300GB+ • Memory • min: 4GB • max:24GB
  5. 5. The toolchain • Apache • front end server, handle request from web browser and pass the request to mod_tile • mod_tile • check if the tile has already been created and is ready for use or whether it needs to be updated due to not being in the cache already • Mapnik • an open source toolkit for rendering maps, it is used to render the Slippy Map layers • PostgreSQL/postGIS • OpenStreetMap data is stored in this database • osm2pgsql • converts OpenStreetMap data to postGIS-enabled PostgreSQL databases
  6. 6. Software installation • The OSM tile server consists of 5 main components: mod_tile, renderd, mapnik, osm2pgsql and a postgresql/postgis database • In order to build these components, a variety of dependencies need to be installed first: $ sudo apt-get install subversion git-core tar unzip wget bzip2 build-essential autoconf libtool libxml2-dev libgeos-dev libpq- dev libbz2-dev proj munin-node munin libprotobuf-c0-dev protobuf-c-compiler libfreetype6-dev libpng12-dev libtiff4-dev libicu-dev libboost-all-dev libgdal-dev libcairo-dev libcairomm-1.0-dev apache2 apache2-dev libagg-dev liblua5.2-dev ttf-unifont
  7. 7. Installing postgresql/ postgis • On ubuntu there are pre-packaged versions of both postgis and postgresql $ sudo apt-get install postgresql-9.1-postgis postgresql-contrib postgresql-server-dev-9.1 • Create a postgis database • Substitute your username for <username> in the below (e.g., $ createuser marc) $ sudo -u postgres -i $ createuser <username> # answer yes for superuser (although this isn't strictly necessary) $ createdb -E UTF8 -O <username> gis $ exit
  8. 8. Set up PostGIS on the postgresql database • Give your newly-created user permission to access some of the PostGIS extensions’ data. Make sure you replace <username> with your user’s name: $ psql -f /usr/share/postgresql/9.1/contrib/postgis-1.5/ postgis.sql -d gis This should respond with many lines ending with: ... CREATE FUNCTION COMMIT ... DROP FUNCTION $ psql -d gis -c "ALTER TABLE geometry_columns OWNER TO <username>; ALTER TABLE spatial_ref_sys OWNER TO <username>;"
  9. 9. Installing osm2pgsql • Although there might be a osm2pgsql package in the repository, it is likely rather old and so we need to compile a newer one for source $ mkdir ~/src $ cd ~/src $ git clone git://github.com/openstreetmap/osm2pgsql.git $ cd osm2pgsql $ ./autogen.sh $ ./configure $ make $ sudo make install $ psql -f /usr/local/share/osm2pgsql/900913.sql -d gis
  10. 10. Install Mapnik library • Mapnik is used to render the OpenStreetMap data into the tiles used for an Openlayers web map $ cd ~/src $ git clone git://github.com/mapnik/mapnik $ cd mapnik $ git branch 2.0 origin/2.0.x $ git checkout 2.0 $ python scons/scons.py configure INPUT_PLUGINS=all OPTIMIZATION=3 SYSTEM_FONTS=/usr/share/fonts/truetype/ $ python scons/scons.py #takes time $ sudo python scons/scons.py install $ sudo ldconfig • Verify that Mapnik has been installed correctly $ python $ >>> import mapnik $ >>>
  11. 11. Install mod_tile and renderd • Compile the mod_tile source code $ cd ~/src $ git clone git://github.com/openstreetmap/mod_tile.git $ cd mod_tile $ ./autogen.sh $ ./configure $ make $ sudo make install $ sudo make install-mod_tile $ sudo ldconfig
  12. 12. Install Mapnik style-sheet • Next, we need to install the OpenStreetMap Mapnik tools, which include the default style file and tools to help Mapnik render OpenStreeMap data • Mapnik uses prepared files to generate coastlines and ocean areas for small scale maps since it is faster than reading the entire database for this information. • Downloading the coastline data requires about 400MB of download $ cd ~/src $ svn co http://svn.openstreetmap.org/applications/ rendering/mapnik mapnik-style $ cd ~/src/mapnik-style $ sudo ./get-coastlines.sh /usr/local/share
  13. 13. Software configuration • Now that all of the necessary software is installed, you will need to configure some of the software to work correctly
  14. 14. Configure mapnik style-sheet • In order for mapnik to find the correct postGIS database and the coastline data, you will need to configure the mapnik style-sheet to your local settings • In your style-sheet directory there should be a directory inc.There are a number of files you need to adapt in this directory $ cd ~/src/mapnik-style/inc $ cp fontset-settings.xml.inc.template fontset- settings.xml.inc $ cp datasource-settings.xml.inc.template datasource- settings.xml.inc $ cp settings.xml.inc.template settings.xml.inc • Now you need to modify each of these files • You can use any editor you like, such as vi, vim, and nano
  15. 15. settings.xml.inc <!ENTITY symbols "%(symbols)s"> <!ENTITY osm2pgsql_projection "&srs%(epsg)s;"> <!ENTITY dwithin_node_way "&dwithin_%(epsg)s;"> <!ENTITY world_boundaries "%(world_boundaries)s"> <!ENTITY prefix "%(prefix)s"> <!ENTITY symbols "symbols"> <!ENTITY osm2pgsql_projection "&srs900913;"> <!ENTITY dwithin_node_way "&dwithin_900913;"> <!ENTITY world_boundaries "/usr/local/share/world_boundaries"> <!ENTITY prefix "planet_osm"> replace following lines with
  16. 16. datasource-settings.xml.inc • In this file you will need to enter your database settings • You are running postgresql on the same machine as the rendering stack, so you can comment out the parameters “password”,“host” and “port” with an HTML-style comment • This will enable mapnik to use the “unix local user” as an authentication method <!-- Settings for your postgres setup. Note: feel free to leave password, host, port, or use blank --> <Parameter name="type">postgis</Parameter> <!-- <Parameter name="password">%(password)s</Parameter> --> <!-- <Parameter name="host">%(host)s</Parameter> --> <!-- <Parameter name="port">%(port)s</Parameter> --> <!-- <Parameter name="user">%(user)s</Parameter> -->
  17. 17. • Change the “dbname” from “%(dbname)s” to “gis”,“estimate_extent” to “false”, and “extent” to “-20037508,-19929239,20037508,19929239” <Parameter name="dbname">gis</Parameter> <!-- this should be 'false' if you are manually providing the 'extent' --> <Parameter name="estimate_extent">false</Parameter> <!-- manually provided extent in epsg 900913 for whole globe --> <!-- providing this speeds up Mapnik database queries --> <Parameter name="extent">-20037508,-19929239,20037508,19929239</ Parameter>
  18. 18. fontset-settings.xml.inc • This file contains font definitions, an information about how to change the default font • I strongly recommend you to set “unifont” as default, otherwise some languages (such as chinese, japanese) characters may not display correctly <FontSet name=”book-fonts”> <!-- <Font face-name=”DejaVu Sans Book” /> --> <Font face-name=”unifont Medium” /> </FontSet> <FontSet name=”bold-fonts”> <!-- <Font face-name=”DejaVu Sans Book” /> --> <Font face-name=”unifont Medium” /> </FontSet> <FontSet name=”oblique-fonts”> <!-- <Font face-name=”DejaVu Sans Book” /> --> <Font face-name=”unifont Medium” /> </FontSet>
  19. 19. Configure renderd • Change the renderd settings by editing the /usr/local/ etc/renderd.conf and change the following lines like so (remember to change <username> to your user’s name socketname=/var/run/renderd/renderd.sock plugins_dir=/usr/local/lib/mapnik/input font_dir=/usr/share/fonts/truetype/unifont XML=/home/<username>/src/mapnik-style/osm.xml HOST=localhost • Create the files required for the mod_tile system to run (remember to change <username> to your user’s name) sudo mkdir /var/run/renderd sudo chown <username> /var/run/renderd sudo mkdir /var/lib/mod_tile sudo chown <username> /var/lib/mod_tile
  20. 20. Configure mod_tile • Next, we need to tell the Apache web server about our new mod_tile installation by creating a file • And add one line into this file LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so $ sudo touch /etc/apache2/conf.d/mod_tile $ sudo vim /etc/apache2/conf.d/mod_tile #I use vim
  21. 21. Apache default configuration • Apache’s default website configuration file needs to be modified to include mod_tile settings • Include the following lines immediately after the admin e-mail address line • (looks like: ServerAdmin webmaster@localhost) LoadTileConfigFile /usr/local/etc/renderd.conf ModTileRenderdSocketName /var/run/renderd/renderd.sock # Timeout before giving up for a tile to be rendered ModTileRequestTimeout 0 # Timeout before giving up for a tile to be rendered that is otherwise missing ModTileMissingRequestTimeout 30 $ sudo vim /etc/apache2/sites-available/default
  22. 22. Tuning postgresql • Edit the file /etc/postgresql/9.1/main/ postgresql.conf and make the following changes shared_buffers = 128MB checkpoint_segments = 20 maintenance_work_mem = 256MB autovacuum = off • These changes require a kernel configuration change, which needs to be applied every time that the computer is rebooted • Edit /etc/sysctl.conf and add these lines at the end of this file, and reboot your computer # Increase kernel shared memory segments - needed for large databases kernel.shmmax=268435456
  23. 23. Loading data into your server • Get the latest OpenStreetMap data • Whole planet: planet.openstreetmap.org • Countries and provinces: download.geofabrik.de • City areas: metro.teczno.com $ mkdir ~/planet $ cd ~/planet $ wget http://planet.openstreetmap.org/pbf/planet- latest.osm.pbf
  24. 24. Importing data into the database • With the conversion tool compiled and the database prepared, the following command will insert the OpenStreetMap data you downloaded earlier into the database • This step is very disk I/O intensive and will take anywhere from 10 hours to several days depending on the speed of the computer performing the import • For smaller extracts the import time is much faster accordingly, and you may need to experiment with different -C values (mb) to fit within your machine’s available memory $ osm2pgsql --slim -d gis -C 16000 --number-processes 3 ~/ planet/planet-latest.osm.pbf
  25. 25. Starting your tileserver • Now that everything is installed, set-up and loaded, you can start up your tileserver and hopefully everything is working • We’ll run it interctively first, just to make sure that everything’s working properly $ sudo mkdir /var/run/renderd $ sudo chown username /var/run/renderd $ renderd -f -c /usr/local/etc/renderd.conf • and on a different session (open a new terminal window) $ sudo /etc/init.d/apache2 restart
  26. 26. • If any FATAL errors occur you’ll need to double-check any edits that you made earlier • If not, try and browse to http://yourserveraddress/osm_tiles/ 0/0/0.png to see if a small picture of the world appears • The actual map tiles are being created as “metatiles” beneath the folder /var/lib/mod_tile/default/ • If it does, you can stop the interactive (ctrl+c) renderd process and configure it to run automatically at machine startup as a daemon $ sudo cp  ~/src/mod_tile/debian/renderd.init /etc/init.d/ renderd $ sudo chmod u+x /etc/init.d/renderd • Edit the /etc/init.d/renderd file as root - you’ll need to make a couple of changes to the DAEMON and DAEMON_ARGS lines DAEMON=/usr/local/bin/$NAME DAEMON_ARGS="-c /usr/local/etc/renderd.conf" RUNASUSER=www-data #replace www-data to your user’s name
  27. 27. • You should now be able to start mapnik by doing the following • and stop it • Logging information is now written to /var/ log/syslog instead of to the terminal • Next, add a link to the interactive startup directory so that it starts automatically sudo /etc/init.d/renderd start sudo /etc/init.d/renderd stop sudo ln -s /etc/init.d/renderd /etc/rc2.d/S20renderd • and then restart your server, browse to http://yourserveraddress/osm_tiles/0/0/0.png and everything should be working!
  28. 28. Pre-render tiles • You can use render_list to pre-render tiles Usage: render_list [OPTION] ... -a, --all render all tiles in given zoom level range instead of reading from STDIN -f, --force render tiles even if they seem current -m, --map=MAP render tiles in this map (defaults to 'default') -l, --max-load=LOAD sleep if load is this high (defaults to 5) -s, --socket=SOCKET unix domain socket name for contacting renderd -n, --num-threads=N the number of parallel request threads (default 1) -t, --tile-dir tile cache directory (defaults to '/var/lib/mod_tile') -z, --min-zoom=ZOOM filter input to only render tiles greater or equal to this zoom level (default is 0)  -Z, --max-zoom=ZOOM filter input to only render tiles less than or equal to this zoom level (default is 18) • If you are using -a, you can restrict the tile range by adding these options • Check tile coordinate at http://tools.geofabrik.de/map/ -x, --min-x=X minimum X tile coordinate -X, --max-x=X maximum X tile coordinate -y, --min-y=Y minimum Y tile coordinate -Y, --max-y=Y maximum Y tile coordinate
  29. 29. Pre-render Examples • Suppose I would like to pre-render tiles form level 0 to 5 • Note that you have to set --socket=/var/run/renderd/ renderd.sock $ sudo render_list -a -f -z 0 -Z 5 --socket=/var/run/ renderd/renderd.sock • Or, I am going to re-render the map of level 7 of Taiwan $ sudo render_list -a -f -z 7 -Z 7 x 106 X 107 y 54 Y 56 --socket=/var/run/renderd/renderd.sock
  30. 30. Update OSM data and Tiles • First, download the latest osm pbf file from the aforementioned websites • Second, use osm2pgsql append mode to convert data into postgresql database • Lastly, remove expire tiles and restart renderd $ cd ~/planet $ wget http://planet.openstreetmap.org/pbf/planet- latest.osm.pbf #make sure you have deleted the old one $ osm2pgsql --slim -a -d gis -C 16000 --number- processes 3 ~/planet/planet-latest.osm.pbf $ sudo rm -R /var/lib/mod_tile/default/ $ sudo /etc/init.d/renderd restart
  31. 31. Personalize Map Style • You can make your map to personal style, for instance, displaying local features, changing road color or font color, by modify osm.xml • if you can not find osm.xml file, check the /usr/local/etc/ renderd.conf • In this example, I change the background-color from “#b5d0d0” to “#ff0000” (red) $ sudo vim ~/src/mapnik-style/osm.xml # in editor ... <Map background-color=”#ff0000” src... ... $ sudo /etc/init.d/renderd restart $ sudo render_list -a -z 0 -Z 0 --socket=/var/run/ renderd/renderd.sock • Now, browse http://yourserveraddress/osm_tiles/0/0/0.png you will see the red ocean
  32. 32. Using tile • An example using Leaflet • First, create a website folder (in my case is osm) under /var/www/ • Second, download Leaflet from http://leafletjs.com • Third, create and edit index.html (you can copy sample code from next slide) • Finally, browse http://yourserveraddress/osm/ $ sudo mkdir /var/www/osm $ cd /var/www/osm $ sudo wget http://leaflet-cdn.s3.amazonaws.com/build/ leaflet-0.6.4.zip $ sudo unzip leaflet-0.6.4.zip $ sudo touch index.html $ sudo vim index.html #see sample code in next slide
  33. 33. Using Tile:Sample code <html> <head> <title>My first osm</title> <link rel=”stylesheet” type=”text/css” href=”leaflet.css”/> <script type=”text/javascript” src=”leaflet.js”></script> <style> #map{width:100%;height:100%} </style> </head> <body> <div id=”map”></div> <script> var map = L.map(‘map’).setView([25.555,121.899],7); L.tileLayer(‘http://yourserveraddress/osm_tiles/{z}/{x}/{y}.png’, {maxZoom:18}).addTo(map); </script> </body> </html>
  34. 34. References • switch2osm.org • wiki.openstreetmap.org • leafletjs.com
  35. 35. Thank you.<Marc Huang> marchuang41@gmail.com

×