For the activities of OSM data entry, morb_au's details are actually here: http://www.openstreetmap.org/user/morb_au

For the rest, read on.

Reference configuration procedures

From time to time I will need to rebuild my OSM-related servers. Hopefully this page will be my "memory" of how I got things to work the previous time. You might find them of use. but treat them as draft recipes, use at own risk etc etc.

WMS Server configuration

I thought I'd better lodge some details of how I got my WMS Server running, in the interests of a resilient server infrastructure.

I'm also assuming you know some basic PostgreSQL administration skills such as being able to remove SQL constraints (I use pgAdmin III).

I've chosen the Queensland Wetlands Data – Wetlands dataset as my example.

Example Technology Stack

I used:

Basic WMS

White Box Linux 3 virtual private server (GNU/Linux 2.6.16.38-xenU i686),
PostgreSQL 8.2
GEOS 1.0.0
PROJ 4.4.8
PostGIS 1.1.6
GDAL/OGR 1.2.1
UMN MapServer 4.10

WMS + TMS

White Box Linux 3 virtual private server (GNU/Linux 2.6.16.38-xenU i686),
PostgreSQL 8.2
GEOS 3.0.0RC3
PROJ 4.7.0
PostGIS 1.1.6
GDAL/OGR 1.6.2
MapServer 5.4.2

Fetch the Published Dataset

Already you hit a little snag because the data.australia.gov.au download URL has spaces in it, so you need to single-quote it, for example:

# wget -c 'http://www1.australia.gov.au/datasets/States%20&%20Territories/QLD/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.zip'

Then unzip it

# unzip IQATLAS.QLD_WETLAND_SYSTEM_100K_A.zip
Archive:  IQATLAS.QLD_WETLAND_SYSTEM_100K_A.zip
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.dbf
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.prj
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.sbn
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.sbx
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shp
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shx
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/QLD_WETLAND_SYSTEM_100K_A.xml
  inflating: IQATLAS.QLD_WETLAND_SYSTEM_100K_A/Queensland Wetland Data - Wetlands.html

Importing from Shapefile to PostGIS

Of course I have obfuscated some private details, just replace the parts in [square brackets] with your own server's details.

ogr2ogr -nln qld_wetlands -lco OVERWRITE=YES -a_srs EPSG:4283 -nlt POLYGON -f "PostgreSQL" PG:"host=localhost user=postgres dbname=[your-database-name] password=[your-postgres-password]" IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shp

(or, if you're running low on disk space and just want to set things up to trace from the geometry, then don't bother importing the attributes by adding the -select flag):

ogr2ogr -nln qld_wetlands -select '' -lco OVERWRITE=YES -a_srs EPSG:4283 -nlt POLYGON -f "PostgreSQL" PG:"host=localhost user=postgres dbname=[your-database-name] password=[your-postgres-password]" IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shp

This first run won't get very far before you get a dimension constraint from the database

ERROR 1: INSERT command for new feature failed.
ERROR:  new row for relation "qld_wetlands" violates check constraint "enforce_dims_wkb_geometry"

ERROR 1: Terminating translation prematurely after failed
translation of layer IQATLAS.QLD_WETLAND_SYSTEM_100K_A

For some reason ogr2ogr sets the constraint to effectively insist on 3D geometry, which is superfluous for our needs. You can see it in pgAdmin:

CREATE TABLE qld_wetlands
(
  ogc_fid serial NOT NULL,
  wkb_geometry geometry,
  wetland_id numeric(10),
  wetland_ar numeric(19,8),
  wetclass character(12),
  wetclassid numeric(1),
  hydromod character(12),
  salinmod character(12),
  wetre character(80),
  floodplain character(2),
  wetrepct character(40),
  wtrregime character(4),
  source character(8),
  remcover character(4),
  wetsub character(12),
  xre character(50),
  xre_percen character(14),
  xre_class character(32),
  drainagedi character(20),
  catchment character(24),
  habitat_id numeric(10),
  habitat_ar numeric(19,8),
  legend character(12),
  wetclass_l character(16),
  hydromod_l character(64),
  salinmod_l character(16),
  floodpla_1 character(40),
  wtrregime_ character(40),
  shape_area numeric(19,11),
  shape_len numeric(19,11),
  shape_ar_1 numeric(19,11),
  shape_le_1 numeric(19,11),
  shape_fid numeric(9),
  CONSTRAINT enforce_dims_wkb_geometry CHECK (ndims(wkb_geometry) = 3),
  CONSTRAINT enforce_geotype_wkb_geometry CHECK (geometrytype(wkb_geometry) = 'POLYGON'::text OR wkb_geometry IS NULL),
  CONSTRAINT enforce_srid_wkb_geometry CHECK (srid(wkb_geometry) = 4283)
) 
WITHOUT OIDS;
ALTER TABLE qld_wetlands OWNER TO postgres;

Use pgAdmin or your equivalent to remove the enforce_dims_wkb_geometry constraint:

ALTER TABLE qld_wetlands DROP CONSTRAINT enforce_dims_wkb_geometry;

Then try the following (the difference is the -append flag) (also try -skipfailures as perhaps some geometries are not well formed in the wetlands dataset):

ogr2ogr -nln qld_wetlands -append -lco OVERWRITE=YES -a_srs EPSG:4283 -nlt POLYGON -f "PostgreSQL" PG:"host=localhost user=postgres dbname=[your-database-name] password=[your-postgres-password]" IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shp

This one takes a few minutes. It doesn't wrap things inside one huge COMMIT so you can use pgAdmin count-rows feature to track its progress.

Finally, add a spatial index to fasten map drawing by a factor of 30:

CREATE INDEX index_qld_wetlands_wkb_geometry
  ON qld_wetlands
  USING gist
  (wkb_geometry);

OR ??? shp2pgsql from PostGIS package

shp2pgsql -s 4283 -d -I IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shp qld_wetlands | psql -d [your-database-name]

??? Row 4251 of QLD_WETLAND_SYSTEM_100K_A seems to be a problem (huge geometry?).

shp2pgsql -s 4283 -d -I IQATLAS.QLD_WETLAND_SYSTEM_100K_A/IQATLAS.QLD_WETLAND_SYSTEM_100K_A.shp qld_wetlands > pg.sql

2058616012 byte file results.

Note about coordinate reference systems

The DCDB metadata claims GDA94 coordinates but a projection code of EPSG:4938 [1]. However a search for GDA94 coordinates on the web shows the more popular EPSG:4283 [2]. I have gone with 4283 for now. I understand the difference between 4283 and 4326 (WGS84, the main OSM coordinate reference system) is only a matter of 1 metre therefore I am not getting precious about it.

Mapfile contents

Note that some of the config is just plain wrong (e.g. the EXTENT). However, it "works for JOSM, QGIS, and Merkaartor" so the following mapfile is presented as is. Of course I have obfuscated some private details, just replace the parts in [square brackets] with your own server's details.

#
# WMS OpenStreetMap Queensland CC-BY Map File definition for UMN MapServer 4.6
# Brendan Morley (morb_au), September 2009
#

# Map files begin with MAP keyword to signify the start of the map object.
# Well, the entire map file is THE map object.  Enclosed between MAP and END
# at the very bottom of this map file, are keyword/value pairs and other
# objects.
MAP
  NAME           "Queensland WMS for OSM"
  IMAGETYPE      PNG
  EXTENT         153.025 -27.475 153.05 -27.45
  SIZE           150 150
  IMAGECOLOR     255 255 255
  SHAPEPATH      "../mapdata"
  FONTSET        "../fonts/fonts.list"

OUTPUTFORMAT
  NAME           png
  DRIVER         "GD/PNG"
  MIMETYPE       "image/png"
  IMAGEMODE      PC256
  EXTENSION      "png"
  TRANSPARENT    on
END

  PROJECTION # OUTPUT
    "init=epsg:4326"  # WGS84 / Lat Lon
#    "init=epsg:28356"  # GDA94 / MGA Zone 56
  END

  # The web object is defined at the level below the map object.  All
  # web-related parameters (I interchange "parameters" and "keyword/value
  # pairs" quite frequently, sorry about that) are defined in this object.
  WEB
    TEMPLATE  'template-place.html'
    IMAGEPATH '[path-to-html-docs]/ms_tmp/'
    IMAGEURL  '/ms_tmp/'
    LOG       '[path-to-mapserv-logs]/mapserv.log'
    METADATA
      WMS_TITLE             'WMS Server for OSM-compatible CC-BY Queensland data.'
      WMS_ABSTRACT          'Maximum scale 1:10000.  Derived data must include attribution=State of Queensland (Department of Environment and Resource Management) 2009'
      WFS_TITLE             'WFS Server for OSM-compatible CC-BY Queensland data.'
      WFS_ABSTRACT          'Derived data must include attribution=State of Queensland (Department of Environment and Resource Management) 2009'
      OWS_ONLINERESOURCE    'http://206.123.75.6/cgi/ms/mapserv?map=../../mapserv/wms.osm.au.qld.map&'
      OWS_ATTRIBUTION_TITLE 'State of Queensland (Department of Environment and Resource Management) 2009'
    END
  END

  QUERYMAP
    COLOR 0 0 0
  END # QUERYMAP


  # Layer objects are defined beneath the map object.  You need at least one
  # layer defined in your map file before you can display a map...  You can
  # define as many layers as you'd like although a limit is typically hard-coded
  # in map.h in the MapServer source.  The default limit is set at 100.  You'd
  # have to have a very specialized application to need more than 100 layers in
  # your application.

  # Start of LAYER DEFINITIONS ---------------------------------------------
  LAYER
    NAME             dcdb_lite_geometry
    METADATA
      OWS_TITLE                      'Queensland Cadastral Boundaries (2009)'
      OWS_ABSTRACT                   'A copy of the geometry of the Property Boundaries Annual Extract Queensland (Lite DCDB) sourced from http://data.australia.gov.au/152 on 2009-10-01.'
      OWS_ATTRIBUTION_ONLINERESOURCE 'http://data.australia.gov.au/152'
    END
    TYPE             POLYGON
    CONNECTIONTYPE   POSTGIS
    CONNECTION       "host=localhost dbname=[your-database-name] user=postgres password=[your-postgres-password]"
    DATA             "wkb_geometry FROM qld_dcdb_lite_geometry USING UNIQUE ogc_fid"

    MAXSCALE         10000
    MAXFEATURES      5000

    DUMP             true

    PROJECTION # INPUT
#      "init=epsg:4326"  # WGS84 / Lat Lon
      "init=epsg:4283"  # GDA94 / Lat Lon
    END

    # The class object is defined within the layer object.  You can define as
    # many classes as you need (well, there are limits as with layers, but it's
    # senseless to define more than ten on a "normal" layer.  There are
    # situations, however, where you might have to do it.)
    CLASS
      NAME       "Cadastre"

      # There are styles in a class, just like there are classes in a layer,
      # just like there are layers in a map.  You can define multiple styles in
      # a class just as you can define multiple classes in a layer and multiple
      # layers in a map.
      STYLE
        OUTLINECOLOR  64  64  64
        COLOR        128 128   0
        WIDTH        1
#        ANTIALIAS    true
      END

    END # CLASS

  END # LAYER


  LAYER
    NAME "copyright"
    STATUS DEFAULT
    TYPE annotation
    TRANSFORM ll #set the image origin to be lower left
    FEATURE
      POINTS
        250 -10 #set the offset from lower left position in pixels
      END
      TEXT "CC-BY State of Queensland (Department of Environment and Resource Management) 2009" #this is your displaying text
    END
    CLASS
      LABEL #defines the font, colors etc. of the text
        FONT arial-narrow
        TYPE TRUETYPE
        SIZE 11
#        BUFFER 1
        COLOR 128 128 128
#        BACKGROUNDCOLOR 255 255 255
        FORCE TRUE
        ANTIALIAS FALSE
      END
    END
    UNITS          PIXELS  # sets the units for the feature object
    TRANSPARENCY   50      # doesn't seem to work for text?!
#    OPACITY        50      # doesn't seem to work for text?!
  END


  # End of LAYER DEFINITIONS -------------------------------

  SYMBOL
    NAME "circle"
    TYPE ellipse
    FILLED true
    POINTS
      1 1
    END
  END

END # MAP

TMS setup for Mobile GMaps

Mgmaps can allow you to enter a custom map URL. It acts as a map tile fetching client but it is possibly not a true TMS. This is because it seems to use a legacy Google Maps tile reference scheme. It adds the following template to your custom map URL's query string: x=123&y=456&zoom=2 (where 123, 456 and 2 are substituted with the real tile references, of course).

This template needs to be converted to the standard mapserver TMS template of mode=tile&tilemode=gmap&tile=123+456+16

A way to do it is to use an apache 2 rewrite module. You need to change 2 config files:

.htaccess in mapserv directory

This is not as easy as simply transposing the parameters from the MGMaps format to the mapserver format, since with MGMaps, the zoom parameter is "1" for the most zoomed-in level. This is equivalent to zoom level "17" for a modern TMS and/or the OSM slippy map layer.

Therefore there is one entry for each zoom level. On the bright side, since I've set up cadastre to only show up to 1:10000, you actually only need to set up OSM zoom levels 17 and 16. I've added more in case I want to add additional, and possibly less detailed, layers.

RewriteEngine On
RewriteBase /cgi/ms

# One for each zoom level, as zoom numbers are reversed
RewriteCond %{QUERY_STRING} ^x=(.*)&y=(.*)&zoom=0$ 
RewriteRule dcdb-lite.gmap mapserv?layers=dcdb_lite_geometry&format=image/png&map=../../mapserv/wms.osm.au.qld.map&mode=tile&tilemode=gmap&tile=%1+%2+17 [L]

RewriteCond %{QUERY_STRING} ^x=(.*)&y=(.*)&zoom=1$ 
RewriteRule dcdb-lite.gmap mapserv?layers=dcdb_lite_geometry&format=image/png&map=../../mapserv/wms.osm.au.qld.map&mode=tile&tilemode=gmap&tile=%1+%2+16 [L]

RewriteCond %{QUERY_STRING} ^x=(.*)&y=(.*)&zoom=2$ 
RewriteRule dcdb-lite.gmap mapserv?layers=dcdb_lite_geometry&format=image/png&map=../../mapserv/wms.osm.au.qld.map&mode=tile&tilemode=gmap&tile=%1+%2+15 [L]

RewriteCond %{QUERY_STRING} ^x=(.*)&y=(.*)&zoom=3$ 
RewriteRule dcdb-lite.gmap mapserv?layers=dcdb_lite_geometry&format=image/png&map=../../mapserv/wms.osm.au.qld.map&mode=tile&tilemode=gmap&tile=%1+%2+14 [L]

RewriteCond %{QUERY_STRING} ^x=(.*)&y=(.*)&zoom=4$ 
RewriteRule dcdb-lite.gmap mapserv?layers=dcdb_lite_geometry&format=image/png&map=../../mapserv/wms.osm.au.qld.map&mode=tile&tilemode=gmap&tile=%1+%2+13 [L]

# Ends

httpd.conf in /etc/httpd/conf directory

Make sure the following is active in the httpd.conf:

Turn rewrite on:

LoadModule rewrite_module modules/mod_rewrite.so

Setup the mapserver CGI directory to accept rewriting by paying particular attention to the AllowOverride and Options directives:

<Directory "path-to-your-mapserv-binary">
    AllowOverride All
    Options SymLinksIfOwnerMatch
    Order allow,deny
    Allow from all
</Directory>

Turn rewrite debugging on (if you like):

RewriteLog logs/rewrite_log
RewriteLogLevel 1

User:Morb au

Contents

Reference configuration procedures

WMS Server configuration

Example Technology Stack

Basic WMS

WMS + TMS

Fetch the Published Dataset

Importing from Shapefile to PostGIS

Note about coordinate reference systems

Mapfile contents

TMS setup for Mobile GMaps

.htaccess in mapserv directory

httpd.conf in /etc/httpd/conf directory

Navigation menu

User:Morb au

Reference configuration procedures

WMS Server configuration

Example Technology Stack

Basic WMS

WMS + TMS

Fetch the Published Dataset

Importing from Shapefile to PostGIS

Note about coordinate reference systems

Mapfile contents

TMS setup for Mobile GMaps

.htaccess in mapserv directory

httpd.conf in /etc/httpd/conf directory

Navigation menu

Search