Vi:Mapnik

Mapnik
Giấy phép:	GNU Lesser General Public License
Nền tảng:	Windows, macOS, và Linux
Phiên bản:	2.2.0 (2013-06-03)
Ngôn ngữ:	Tiếng Anh
Trang chủ:	http://mapnik.org
Ngôn ngữ lập trình:	C++ và Python

Mapnik là phần mềm chúng tôi sử dụng để hiển thị lớp Slippy Map cho OSM.

Đây là một công cụ mã nguồn mở để hiển thị bản đồ. Phần mềm được viết bằng ngôn ngữ C++ và có hỗ trợ Python. Mapnik sử dụng thư viện AGG và hỗ trợ khử răng cưa với độ chính xác tới subpixel. Mapnik có thể đọc được dữ liệu ESRI Shapefile, PostGIS, TIFF rasters, .osm, và các định dạng mà GDAL hay OGR ho trợ. Mapnik có thể sử dụng được trên Linux, Mac OS X và Windows.

Trang chủ chính thức tại mapnik.org, người mới sử dụng Mapnik có thể tìm hiểu cách sử dụng tại The Mapnik Wiki và tham gia #mapnik channel on freenode

Báo lỗi và các đề xuất cho Mapnik

Nếu bạn gặp bất kì lỗi nào khi sử dụng Mapnik hoặc muốn yêu cầu các đối tượng chưa được hiển thị, hãy truy cập vào trac. Hãy kiểm tra xem yêu cầu của bạn có giống các yêu cầu gửi trước đó không tại trac tickets with component "mapnik" Phản hồi về các trục trặc, ý tưởng, các cải tiến về Mapnik nói chung (không phải OSM), xin truy cập http://trac.mapnik.org

Tổng quan

OSM sử dụng Mapnik để hiển thị các tiles có kích thước 256 × 256, được lưu trữ tại (tile.openstreetmap.org).

Trang Slippy Map sử dụng Javascript đọc đường dẫn tới tiles như sau : https://tile.openstreetmap.org/7/63/42.png

Xem thêm Slippy map tilenames

Dữ liệu

Dữ liệu OSM được chuyển đổi sang định dạng mà Mapnik có thể dọc được bằng chương trình osm2pgsql. osm2pgsql đọc dữ liệu tải về từ osm vào cơ sở dữ liệu PostGIS. Mapnik cũng có thể đọc được shapefiles.

Chuẩn bị

This is a work in progress.

This process will require quite a bit of diskspace if you plan on loading the entire world. Individual countries can be handled on any ordinary machine, but for the whole planet here are some indications of the requirements (as at 04 Apr 2009). This does not include the indexes:

Table	Size
roads	700 MB
line	6.25 GB
point	469 MB
polygon	700 MB
ways	4.87 GB
ways_nodes	16 GB

Which totals about 30 GB. If you are using slim-mode in osm2pgsql you will also need around 20 GB for the temporary tables. Please mention you would probably need indexes as well, and the whole database with indexes amounts to 61 GB!

Fresh news : As of 04/2010, europe.osm.bz2 in slim-mode will need around 90Go of disk space

If you have enough memory (>12 GB) then you can try running in normal mode. On a suitable machine the process should complete in 3-4 hours. If you don't have enough memory then you need to use the slim-mode which will work with any amount of memory you can give it. Configured for 3 GB RAM the process has been measured at around 24 hours, though it is strongly dependent on your disk subsystem.

Note: slim-mode is required if you want to apply the planet diff files (e.g. minutely diffs).

There is a graph explaining memory usage during the planet import, in a RAM mode:

Don't forget you count the 6.1 GB for the compressed planet dump itself!

DataSets

There are two large datasets needed to render tiles, so its probably a good idea to start downloading those first.

Planet.osm

Planet.osm is a snapshot of the OpenStreetMap database. See Planet.osm

$ wget https://planet.openstreetmap.org/planet-latest.osm.bz2

Note: If you want to render a small area as a quick test then you can import any .osm file, you do not need to render the whole planet. This is recommended if you want to quickly prove that the rendering is working before you spend several hours downloading 3+ GB and importing the whole planet. You can get a suitable .osm file by downloading and saving an area in JOSM or use one of the smaller planet extracts listed at Planet.osm#Extracts.

Ranh giới

Mapnik sử dụng dữ liệu đường bờ biển từ nguồn khác tại các mức zoom thấp. Xem tại Coastline#Main_Mapnik_Layer

$ wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz (50M)
$ wget http://tile.openstreetmap.org/processed_p.tar.bz2 (227M)
$ wget http://tile.openstreetmap.org/shoreline_300.tar.bz2 (46M)

Sau khi tải về và kiểm tra bằng các scripts của mapnik, (xem chie tiết bên dưới), tạo một thư mục mapnik/world_boundaries và giải nén vào đó

PostgreSQL / PostGIS

Để biết cách cài đặt và cấu hình PostgreSQL / PostGIS sử dụng với Mapnik, xem phần hướng dẫn tại Mapnik/PostGIS

osm2pgsql

osm2pgsql is a Openstreetmap data to PostgreSQL converter and loader.

For installation instructions, see osm2pgsql.

Before you can use osm2pgsql for the first time with the Spherical Mercator projection (see below), you need to initialize configuration data for this projection. Do this by running the .sql file included with osm2pgsql:

[Cú pháp trên Windows]
$ psql -d gis -f c:\osm2pgsql\900913.sql
[Syntax on Linux]
$ wget http://trac.openstreetmap.org/browser/applications/utils/export/osm2pgsql/900913.sql?format=raw 900913.sql
$ psql -d gis -f ./900913.sql

Enabled Intarray (if planning on using diffs)

psql gis < /usr/share/postgresql/8.3/contrib/_int.sql

Also you should make sure that the database schema used by osm2pgsql is in sync with the one Mapnik acts on. If they aren't, funny things will happen that aren't obvious right away.

The schema is described in the file default.style for osm2pgsql and osm.xml (resp. osm-template.xml) for Mapnik. default.style of your osm2pgsql package is very likely out of date when not building from svn source. So when fetching Mapnik OSM stuff from svn (see below) you should also update default.style e.g. via repository browser.

Cài đặt Mapnik

To install Mapnik, please follow the instructions here: Mapnik/Installation.

For more specific installation steps on Fedora 10, follow these steps: Mapnik/Fedora Installation.

Đọc dữ liệu

Simply call osm2pgsql (as the user who will be running mapnik):

$ osm2pgsql -m -d gis planet-latest.osm.bz2

This process requires significant amounts of free memory (by rough estimate, at least 512 MB). If it runs out of memory, it can fail in somewhat unpredictable ways (e.g. messages suggesting that errors exist in the .osm file). Make sure that you have sufficient free RAM and swap space available before running osm2pgsql.

The "-m" option enables the use of the Spherical Mercator projection which is recommended. This is the default since version 0.55, SVN r8320. Some older tools may still expect the data to be imported in a different projection such as "latlong" or the non-spherical Mercator. There are command line options available to select these and other projections.

The latest osm2pgsql code also supports a simple bounding box filter during the import which will make things a lot faster if you only want to render a small part of the planet. For example if you just wanted to import London from the UK planet extract:-

$ osm2pgsql --bbox -0.5,51.25,0.5,51.75 -m -d gis uk-080213.osm.bz2

Một số lỗi mà bạn có thể bắt gặp

Sử dụng sai phép chiếu

If all your map tiles appear to be off by 10 - 20 km in the North/South direction then you may have forgotten to add the "-m" option on the osm2pgsql import. Since osm2pgsql version 0.55 the spherical mercator (-m) is the default projection.

Không thể mở file dữ liệu

If you attempt to load a gzip'd planet file which is more than 7 GB you may see an error that it fails to open the file. This is due to a bug in some versions of zlib which fail to supply the O_LARGEFILE flag to the open(). The workaround is to use an external gzip process which does not suffer this bug. Specify the filename "-" will make osm2pgsql read the data from STDIN.

gzip -dc planet-080213.osm.gz | osm2pgsql -m -

Tràn bộ nhớ

If your load fails with an error similar to

Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Mid: Ram, scale=100

Reading in file: -
Entity: line 1: parser error : Extra content at the end of the document

^
- : failed to parse
Error occurred, cleaning up

You probably need to run osmpg2sql in slim mode, which stores intermediate objects on disk rather than in RAM. To do so run osm2pgsql with the '-s' switch:

osm2pgsql -m -s -d gis planet-080213.osm.gz

Phép chiếu không có giá trị trong pgSQL

You may receive this error right after starting:

Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
SELECT AddGeometryColumn('planet_osm_point', 'way', 900913, 'POINT', 2 );
  failed: ERROR:  AddGeometryColumns() - invalid SRID
CONTEXT:  SQL statement "SELECT AddGeometryColumn('','', $1 , $2 , $3 , 
$4 , $5
)"
PL/pgSQL function "addgeometrycolumn" line 4 at SQL statement

This means that the projection has not been intialized and you need to execute 900913.sql in the osm2pgsql folder. To do this :

wget http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/900913.sql && psql -f 900913.sql gis

to make it work after that, I had to restart the database daemon :

sudo /etc/init.d/postgresql restart

Lock

After several hours the import may quit with an error like:-

ANALYZE planet_osm_line;
 failed: ERROR:  deadlock detected
DETAIL:  Process 28511 waits for AccessExclusiveLock on relation 1064115 of database 18309; blocked by process 12776.
Process 12776 waits for ShareLock on transaction 572766655; blocked by process 28511.

Error occurred, cleaning up

This seems to be a fault in some versions of PostgreSQL and is caused when an auto-vacuum is attempted during the ANALYZE. The solution is to disable all auto-vacuums on the database. The data is not updated after the import so the vacuum process does nothing useful. In the postgresql.conf file set the option:

autovacuum = off

Then restart the database server

sudo /etc/init.d/postgresql-8.1 restart

Note: In Debian/Ubuntu you also need to update /etc/cron.d/postgresql-common to comment out the two pg_maintenance tasks (add a '#' in front of them) which would otherwise cause vacuums to occur at regular intervals:-

# Run VACUUM ANALYSE on all databases every 5 hours if pg_autovacuum is not
# running
# 2 0,5,10,15,20 * * 1-6 root if [ -x /usr/sbin/pg_maintenance ]; then /usr/sbin/pg_maintenance --analyze >/dev/null; fi

# On Sunday you may wish to run a VACUUM FULL ANALYSE as well
# If you do not run a 24/7 site, you may want to uncomment the next line 
# so as to do a regular VACUUM FULL.  If you need 24/7 connectivity, save
# VACUUM FULL for when you think you really need it.
# 10 3 * * Sun root if [ -x /usr/sbin/pg_maintenance ]; then /usr/sbin/pg_maintenance --full --analyze >/dev/null; fi

Don't forget to reenable them if you're doing anything else with postgres, or performance will slowly degrade.

Truy xuất bị từ chối

Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
SELECT AddGeometryColumn('planet_osm_point', 'way', 900913, 'POINT', 2 );
 failed: ERREUR:  droit refusé pour la relation geometry_columns
CONTEXT:  instruction SQL « DELETE FROM geometry_columns WHERE
		f_table_catalog = '' AND f_table_schema = 'public' AND f_table_name = 'planet_osm_point' AND f_geometry_column = 'way' »
PL/pgSQL function "addgeometrycolumn" line 94 at EXECUTE statement
instruction SQL « SELECT AddGeometryColumn('','', $1 , $2 , $3 , $4 , $5 ) »
PL/pgSQL function "addgeometrycolumn" line 4 at SQL statement

Error occurred, cleaning up

The user is not allowed to modify the gis database. You have to run osm2pgql as the user you created in the postgresql database setup.

Xác thực bị thất bại

If you can't connect to postgres with the psql command (this happens, for example, with Ubuntu 9.10 and postgres 8.3):

 $ psql -W gis username
 psql: FATAL:  Ident authentication failed for user "username"

Even after you've set the password for the user:

 $ sudo -u postgres -s -H
 postgres@aoeu$ psql 
 Welcome to psql 8.3.5, the PostgreSQL interactive terminal.
 ALTER ROLE username PASSWORD 'mypassword';

You may have to append the following lines to your pg_hba.conf (found e.g. in /etc/postgresql/8.3/main):

 local   all         all                               trust   
 host    all         all         127.0.0.1/32          trust
 host    all         all         ::1/128               trust

So that you can use password authentication from localhost.

Note that you may have to comment out any existing lines beginning with local all or host all. I had to comment out the following section on Ubuntu 8.10:

# "local" is for Unix domain socket connections only 
#local   all         all                               ident sameuser
# IPv4 local connections:
#host    all         all         127.0.0.1/32          md5
# IPv6 local connections:
#host    all         all         ::1/128               md5

Đối tượng không rõ ràng

Nếu nhận được lỗi sau:

Unknown Element name : changeset

bạn nên nâng cấp osm2pgsql lên phiên bản mới nhất.

Rendering với Mapnik

Unpack the lowzoom and coastline data you downloaded earlier:

cd ~/mapnik
tar xzf world_boundaries-spherical.tgz
tar xjf processed_p.tar.bz2 -C world_boundaries
tar xjf shoreline_300.tar.bz2 -C world_boundaries

The above should leave you with all the shapefiles in the world_boundaries directory.

If you haven't already done so, export the Mapnik stuff from the OSM subversion repository:

svn export http://svn.openstreetmap.org/applications/rendering/mapnik
cd mapnik

The osm.xml file is built up out of the core osm.xml file and XML include files in the inc/ directory. The inc/ directory holds 3 .template files, which are used as inputs for the generate_xml.py script:

./generate_xml.py -h

Check to see if the defaults that the script provides are ok for you. You'll have to provide the values where there are no defaults. For instance:

./generate_xml.py --host localhost --user `whoami` --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ --port 5432 --password 'something'

This command takes the .template files from inc/ and creates non-template versions of those files, with your settings. If you can not or do not want to run the script, you can do this by hand as well. Updating the stylesheet from SVN is now as easy as running an 'svn update'. SVN will never touch the generated settings files. If there is ever a new option added to the .template files, only then do you need to run the generate_xml.py script again.

After you have set up the stylesheet this way, you can point mapnik to the osm.xml file. The osm.xml file draws in the files in the inc/ directory, through the use of XML entities. You do not need to create another copy of the stylesheet.

You can decide which areas get rendered by setting bounding boxes and zoom levels in generate_(tiles|image).py

Optional: CJK font fallback support

Currently, mapnik includes a font to support CJK characters (Chinese, Japanese, Korean, et. al. or more generally East Asia). To add support you need to open the file inc/fontset-settings.xml.inc and uncomment the lines that have "unifont Medium".

Older mapnik versions did not have unifont included. There you need to:

1. Download the GNU Unifont Glyphs
2. Unpack and put the ttf file in your /usr/local/lib/mapnik/fonts (or appropriate path) directory with the other Mapnik fonts.

Rendering tiles

You can now call

MAPNIK_MAP_FILE="osm.xml" MAPNIK_TILE_DIR="tiles/" ./generate_tiles.py

generate_tiles.py generates tiles which can be served by a suitable server. (Look inside the file to find out what area of the planet it will render by default and at what zoom. Look for bbox and min,max zoom. If you want to select a different area go to openstreetmap.org and click the "export" tab; then "manually select a different area". Draw the area you want, and the four text boxes will show values you should provide in a sequence [left,bottom,right,top]). If you have multi-core machine, to speed-up generation of tiles, you can divide an area for e.g. four squares and run few instances in the same time. Further speed improvements are possible by reducing disk I/O bottlenecks (iowait time). These bottlenecks can for example be found with iofile.d; on Mac OS X this returns Spotlight, and Postgress statistics collector as parallel disk I/O consumers.

or

./generate_image.py

generate_image.py generates a single image for a given bounding box. That is, it essentially gets the tiles and stitches them into a single large image. This may be useful for people who want to just cover a limited area and thus want to save the hassle of handling tiles.

Troubleshooting

There are several possible problems you may encounter in your rendered tiles or while rendering tiles:

1. If the generated tiles or image show coastlines, but lack streets, street names, cities, and other features, please verify that the database user can indeed login. The scripts and Mapnik may quietly fail to read the data and thus render with coastlines, but no other data. Another cause is building Mapnik without PostGIS support.
2. The version of the last two scripts currently in svn is for use with Mapnik 0.5 and you will get some errors if you try to use them with an older version. (These scripts may try to invoke the 'convert' program. You may need to install ImageMagick or choose not to use 'convert'.)
3. If Asian place names appear as blocks instead of normal characters, then try another font face in your osm.xml file (see set-mapnik-env). AR PL ShangHeiSun Uni Regular seems to work well under Linux. It may be located at /usr/share/fonts/chinese/TrueType/uming.ttf. Copy that file to your mapnik fonts directory (probably /usr/local/lib/mapnik/fonts, but look in paths.py of the Mapnik Python extension).
4. If you are using the ext4 file system (the default in distributions such as openSUSE 11.2 and Ubuntu 9.10) and you are generating many tiles, it can occur that the script crashes because it can't create new tile files. This is a file system issue. Delete as many files as you can, force a file system check and/or switch to a more stable file system.
5. this error can happen is some unix distributions:

 untimeError: PSQL error:
 FATAL:  Ident authentication failed for user "username" (encountered during parsing of layer 'leisure')

On most Unix installs postgres is setup in the default configuration to work using the 'ident same user' authentication. To use this remove the following params from the config: user, port, host, password. Delete all the parameter lines for these, for example:

 ./generate_xml.py --accept-none --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/

Then mapnik will connect the same way as if you ran "psql gis" from the command line (without specifying a host, username or password).

Reducing disk space

If you are rendering the entire earth (that is, bbox = (-180.0, -90.0, 180.0, 90.0)) there are a few things you can do to dramatically reduce disk space usage.

1. Verify that you only have tiles you will use.
   1. For instance, the maximum x and y values for zoom 10 are 1023. So, the last tile should be TILEDIR/10/1023/1023.png.
   2. However, you may have tiles rendered with x and y values beyond that. (I had y values up around 1300)
   3. Delete those first for each zoom level.
2. At higher zoom levels (8 or so), many tiles are duplicated (e.g., empty oceans or lakes, and land areas with no features).
   1. On operating systems supporting it, you can create symbolic links to one set of blank water and land tiles.

Example savings (for me (athlon02)):

1. At zoom 9, the disk space used was 3.8 GB (if memory serves). It is now 1.4 GB.
2. At zoom 10, the disk space used was 12-13 GB to start. It is now 4.58 GB.
3. Your mileage may vary.

Customizing the rendering

Now you have the standard stylesheet rendering working you can start customizing your map.

Contours

It is possible to add contours to your map like the cyclemap has, the contours are based on the public domain SRTM (Shuttle Radar Topograhy Mission) from NASA. It is also possible to add hillshading and elevation colouring to your map based on the same data.

Stylesheet

The stylesheet is an XML document controlling most aspects of how features are rendered, such as color and widths of lines showing different types of roads. The format is not very compact or easy to understand, but it does give a lot of power/flexibility. The version we use on the live slippy map is probably the osm.xml file in the SVN head:applications/rendering/mapnik/osm.xml .

The stylesheet syntax is specified here: XMLConfigReference. MinScaleDenominator and MaxScaleDenominator specify the minimum and maximum scales at which each symbol/feature should be rendered.

You can play around with mapnik stylesheets online here: http://dev.openstreetmap.nl/~panman/styledit/

Look at Cascadenik and Spreadnik if you plan to develop complex stylesheets.

Regionalised rendering of names

To render name:* names instead of the standard name tags where to available you can either preprocess the data or modify the postgis database afterwards.

Customising default.style

If you're editing default.style, for instance to add cycle lanes or network data to the database, note that osm2pgsql does not use the default.style in the working directory, instead looking for /usr/share/osm2pgsql/default.style.

You will need to use the --style parameter to specify your own default.style file. Use .osm2pgsql -h to display more information.