Talk:LINZ

From OpenStreetMap Wiki
Jump to navigation Jump to search

Previous discussions and notes

? Is there any progress on this project? No edits for some months...

Yes. --Hamish 05:42, 16 September 2009 (UTC)

Also note recent OSM news about French and Canadian data being made available (I.e. if they can, why can't we?)

If there is going to be a big influx of LINZ data, is it still worth mapping roads etc --EliotB 02:51, 31 January 2009 (UTC) 

... the answers to all these questions - and more! - can be
    found on the NZOpenGIS mailing list (nzopengis•googlegroups•com)

Comparison maps

  • Reuben's NZTopoMaps site has a Google Maps mashup with a LINZ 1:50k paper topo maps overlay. These are essentially the same data as we are importing from LINZ.
http://www.nztopomaps.com
  • Gavin's similarly named NZ Topo Map is another mashup which includes both the LINZ 1:50k and the 1:250k topo map overlays.
http://www.topomap.co.nz
  • Joe Richard's old dev server had a nice comparorator, but it's currently only mostly functional.
https://joerichards.dev.openstreetmap.org/index-new.html

QA maps

Stats

Historial Method/Test Import

User:JoeRichards Pioneered the conversion of LINZ data to OSM. See the LINZ Trial Import wiki page. (current method used by JR is LINZ → .mp → .osm)


Key people

This list is in progress. Feel free to add yourself, e.g. for any particular area, even as an expert for a particular geographic zone (for checking merging)

  • User:Rcoup - Linz2osm tool, dev servers
  • User:Sdk - Linz2osm tool
  • User:JoeRichards - the script itself, dev server for map comparison (OSM vs import)
  • Graeme Williams - NZOGPS founder, knows a lot about what data is there and what "value-added" bits are present. Also MP (polish map format) and Garmin codes.
  • Sam Giffney - Zenbu founder & NZOGPS mapper, managed merge of new LINZ data to NZOGPS and added numbering, well practiced with MP data and written various scripts to manage it
  • User:Hamish - available as a general dogsbody to help with code & technical problems. Experienced with bulk editing massive files, mostly work in Bourne shell and C but passably multilingual with most UNIXy systems. On the software side pretty good with GDAL/OGR, PROJ.4, and GRASS GIS tools.
  • rcr - available where required. Helped out with coastline and border imports in other countries. Currently spend a lot of time manually fixing invalid data like unconnected or crossing ways in the existing data so don't mind long tedious JOSM sessions.
  • myfanwy - i'm keen to help wherever. i'm willing to help with merging/qa of data, particularly round the auckland area, where there is plenty of existing good quality data. imo a lot of the akl data needs to be kept, rather than replaced. i'm prepared to carry out manual joining work at the interface, and manually fill in small gaps, where the odd road is missing
  • Acrosscanadatrails - I'm happy to be of assistance where possible to share ideas from the Canada data import process, experiences & methods.

A Note on Specific Data Layers

NZOGPS road data

While LINZ has roads they have known quality issues. It is planned that road data will come from the NZOGPS project and not from LINZ directly. the NZOGPS project orginally sourced its data from LINZ but this has been cleaned up significantly.

  • NZ Open GPS homepage
  • Improved road data layer as of Sept 2010 (shapefiles, etc)
  • Prepared .osm files (may be a bit out of date now)
  • Contact: Graeme Williams
  • NZOGPS source data is in Polish format, intended for Garmins. We have been producing routable maps for Garmin equipment since 2007. The data was based on a LINZ import initially but has had substantial editing, manipulation, and other value added data.
  • LINZ data on it's own is raw - it's road center lines only. A good example are roundabouts (not included in LINZ) or dual carriageway roads (shown as a single line in LINZ).
  • The NZOGPS data has been very modified and edited since 2007. It includes full routing data such as road direction, turn restrictions, road center lines of both lanes on dual carriageway roads (eg Fenton Street in Rotorua), other information like LINZ SUFI, street numbering, suburbs / town association.
LINZ topo v16 data

These are 166 vector map layers covering everything from topographic contour lines to waterfall positions to power pylons to high resolution vector coastline. A full set of roads and railways for the country are included. Basically these layers are everything that they use create the 1:50,000 printed topo maps.

This data is the core of what the linz2osm tool will be used to import. It is envisaged that we will add other data to the tool as time goes on.

Other Data Layers

LINZ Address Data

See LINZ/Address Import

Cadastral boundary imports

Any people particularly interested and/or know about the existing datasets, please put your names here

  • User:JoeRichards - would like to know more :)

-- which datasets in particular were you thinking about? Property boundaries + house numbers? Regional council borders?

Statistics New Zealand

Some other public domain shapefile data is available from the statistics office:


"Copyright
Information obtained from Statistics New Zealand may be freely used, reproduced, or quoted unless otherwise specified. In all cases Statistics New Zealand must be acknowledged as the source."
Local Council Data

Many local councils are now releasing data under permissive licenses. Feel free to add any specific layers to the New_Zealand_data_import_projects page.

For example, council building outlines from Auckland Council ([1]) and Wellington City Council ([2]).

Other Useful Layers for Checking

  • The LINZ website has 1:50,000 GeoTiffs of the new-series topo maps (NZTopo50) for download as GeoTiffs, as well as some aerial 2.5m orthophotos. They can be added to a WMS server, or more cheaply into a TMS server (Merkaartor only?).
Old NZMG 2.5m res. orthophotos can be georegistered using their topo map bounds (see .csv file on website). Adjust bounds by 1/2 a cell to correct for grid vs. cell-centre convention.
Marco in Italy has all the NZ nautical charts registered in BSB format as part of the kapgen project. (GDAL can read BSB)


Geographic area volunteers

(north to south)

Primarily people who would like to be involved in merging of the data in any particular area

North Island

  • Auckland (CBD) -
  • Hamilton
    • rcr - not local but traced large parts of Hamilton from Yahoo, happy to help get rid of these unnamed roads and import the real thing.
  • Wellington
    • Olly Betts - I live in the Hutt Valley and have done a fair bit of mapping there and around the greater Wellington area.
    • Leigh - I live in Whitby, and may be able to help review/QA merged data in Porirua area.
    • Howdystranger - I live in Te Aro/CBD. I've been importing lots of building data from the WCC.

South Island

  • Nelson
  • Motueka, Golden Bay & Abel Tasman area
    • Joe Richards - did a bit of mapping here, willing to help, but happy to have others join me
  • Christchurch & North Canterbury
    • rcr - been doing mapping based on imagery and gps around Rangiora and in ChCh, happy to help with importing and merging.
    • EliotB - live in south Christchurch
    • gerkin - (West CHCH) - have been busy with CHCH and Canterbury road, walking track & back-country mapping etc., happy to help with data checking/general dogs-body
  • South Canterbury
  • Queenstown (CBD)
    • Joe Richards - just the central areas and the drive to Glenorchy
    • GrantH - any parts of Queenstown District
  • Dunedin
    • Joe Richards - used to live here so I don't mind helping with merging central/north Dunedin
    • JAYSONHOOGY - currently living in Dunedin and enjoying improving the map in this area.

Areas

("bbox" is short for "bounding box", and is given in x for degrees longitude and y for degrees latitude. Southern and Western hemispheres get a negative sign. Thus bbox=west,south,east,north bounds)
OSM will not export data across the antimeridian in one blob - these need to be extracted as two separate pieces, and joined together (or not, as needs) at a later point. The Chatham Islands are on the opposite side of this line to the rest of New Zealand.

bbox=157.5,-59.0,179.9,-25.5
bbox=171.90,-41.75,178.70,-33.95
bbox=166.3,-46.68,174.48,-40.31


bbox=-177,-44.5,-175.5,-43.5
bbox=-176.373,-44.450,-175.973,-44.115


bbox=167.25,-47.34,168.31,-46.66
bbox=175.2,-36.4,175.6,-36
bbox=174.98,-36.8525,175.206,-36.7186
The Snares

After the dance

  • Together with the contour data it would be great to make some TopOSM renderings
- TopOSM example
- TopOSM/Details
- TopOSM_howto
(but if you are just after pretty contour lines and hill shading, and don't really care about road names too much, you might as well just use gdal2tiles.py to convert the LINZ NZTopo50 300dpi GeoTiff raster scans of the topo sheets into a WMS/TMS server and save yourself a lot of trouble...)
  • What other nice OSM → hardcopy tools are out there?
OSM on Paper


To Done?

[dead url. try this one instead: http://joerichards.dev.openstreetmap.org/]
  • LINZ Trial Import needs to have all the issues in the current import fleshed out and addressed
  • agree on an approach for tagging
  • find out if the upcoming license change (to odbl) will affect the use of the data
  • ask linz if our attribution method is sufficient

Attribution methods

  • add a short line to the main map on the front page, something like "Crown Copyright", which displays whenever LINZ-sourced data is showing on screen, and hyperlinks to a page in the wiki with the full contents of the attribution text
include a line on the attributions page, stating that each element sourced from linz has it's attribution set to "linz", data which can be viewed by analysis of the planet file/through the api/with one of the editors. this data could be set to read-only, i.e. it will be persistent in the data set OSM use, unless the item is completely deleted
    • adding attributes to items which specify their source is approved
  • set the username to "New Zealand crown Copyright". at high zoom levels, the username of the last editor displays next to every object
  • Put the characters as a set of virtual roads on any empty area (preferably owned by linz ;-), and position map to this point and zoomfactor for very first access to data? It's a pity there's no redirect, as in HTML.
the idea of using cookies to determine if it's first visit or not, and re-directing to the attribution page/showing a translucent overlay (for a few seconds, before re-directing back to the map) if it is, is an idea that intrigues me
  • putting an 'attribution' tag on data, in a way that it can only be set (not removed/changed). Osmarener could detect these and mark the map appropriately, could even add the attribution as text on the other side of the way to the copyright holder's name when at maximum zoom. --Mungewell 23:29, 18 March 2008 (UTC)
permission to edit this tag could be given selectively - if another, overlapping data set were then imported, the importer could merge a new data set with the existing data, updating the 'attribution' tag to also represent the origin of the new data set

Summary of attribution methods - what OSM will and will not do

OSM will carry out

  • creating a wiki page with details of contributors, and information on how to view the contributor of a specific piece of data (planet.osm, osmxapi, potlatch, etc.)
  • adding a permanent link from the main map to the attributions page
  • allow to put a read-only changeset tag (attribution, source, ...) when uploading data

OSM will not carry out

  • adding attribution text to maps, beside each item requiring attribution
  • specific attribution labels which appear on-screen whenever an item requiring attribution is being shown (such as used by google maps)
  • re-directions to attribution text, when data requiring attribution is viewed for the first time
  • creating an 'attribution' tag, which can be edited by any member of the community. this tag will be applied only to items requiring attribution
  • creating an 'attribution' tag, which can only be edited by select members of the community. this tag will be applied only to items requiring attribution

Status @ June 2014

  • Stephen @ Koordinates has been working on a revamped and expanded version of the linz2osm web app (code avail on github), which includes workslice checkouts and better tracking than the personal text blurbs and wiki table we had been using on the past. Go Stephen!
The workslice check-out and check-off system is now working well, and the next major task is the tagging of the cleaned-up NZOpenGPS road data, which has now been loaded into the PostGIS database on the LINZ2OSM server.
We've come a long way re-approving the standardized object tagging, and are busy starting to upload the data.
  • We've made it ~ 40% of the way through uploading the mainland layers.
  • Permission has been granted by LINZ to continue uploading using the ODbL license
    • Some people are still worried/concerned about the new license so may not be motivated to continue. We're exploring methods to allow our pre-upload extracts to be fed into the FOSM and CommonMap forks with minimal duplication of effort.
  • Offshore island data is now available through linz2osm for the Antipodes, Auckland, Bounty, Campbell, Kermadec, and Snares archipelagos, as well as the Ross Dependency in Antarctica.
The Auckland, Bounty, Antipodes, Kermadec, and Snares Islands are now fully uploaded.
The JOSM Validator plugin is a good place to start.

Import Process @ June 2014

linz2osm Tool

You can view progress on the linz2osm site:

http://linz2osm.openstreetmap.org.nz/

Development has centred around the linz2osm tool which is currently under heavey development to allow for the remaining mainland layers to be easily uploaded and merged into OSM. The basic process is:

  • Raw LINZ shapefile data from the LINZ Data Service has been loaded into a PostGIS database
  • LINZ->OSM tagging and export is done via the online LINZ-2-OSM web app.
  • The user can 'check out' a slice of work (eg. 100 features from 1 layer).
  • The user gets a .osm file from the web app and saves it locally
  • The .osm file is imported into JOSM or Merkaartor.
  • The layer for import is checked with the JOSM Validator plugin
  • Data is merged, de-duped and cleaned as needed.
  • The layer is uploaded to OSM

Getting Access to linz2osm

We have various tasks to be done from coding to design to tag mapping and uploading.

For the source (GPL) and issue tracker, head to https://github.com/opennewzealand/linz2osm/.

Key Wiki Pages

Some pages that you really should read to understand the whole process. We are currently doing some wiki gardening to clean all of this up.


What should NZ OSM'ers *not* do?

The prospect of importing all this data is fantastic in many respects, but also raises the concern that in some places our work will be undone, and be replaced by bulk-uploaded material (as has happened on occasions elsewhere). Is there a list that offers any guidance about what we should or shouldn't work on in the meantime so that we don't waste time/effort?

Cheers - Gobeirne 02:52, 20 February 2011 (UTC)

place=island

Hi, please add place=island to the island import - this tag is missing on all islands. Greetings, -- Schusch 09:14, 21 February 2011 (UTC)

Done, for those with a name. --Hamish 13:02, 31 August 2012 (BST)

Topomap tiles as editor background

Hello,

Since the Linz topomap data is released under Creative Commons, is it possible to get the tiles as a background that can be traced from in the OSM map editor? Enzedrail 00:29, 3 January 2012 (UTC)

Yes, see the New Zealand/Imagery wiki page. BUT don't do that! Help us with the bulk upload instead! It's the same data and you'll only be making the merging process more difficult. --Hamish 12:51, 31 August 2012 (BST)

Issues

Incorrect waterway direction

Some ways tagged with waterway=stream were found to be flowing uphill. I fixed a few in a small region based on the original changeset. Would expect issue to be in all of "LINZ merge" changesets by mrsid_linz from 2016. Did not notice any discussion of this issue so may affect all waterway ways in import. Does the Linz data not contain directional information? If not, should be fairly easy for automated detection by starting where waterways join ocean, traverse upstream and flag any waterway ways which reverse direction. Likely existing documentation and/or scripts for addressing this issue since I'd imagine it's not uncommon. DFyson (talk) 00:25, 6 April 2024 (UTC)

Hi, you're correct that LINZ's waterways don't have directional information... I've only been involved since 2021, since then we've been manually correctly the directional of every waterway as they're imported (or mapped via the MapRoulette chanllenge). So rest assured that no new waterways will be mapped with the wrong direction. But I have no idea how bad the current situation is. Waterway Map suggests that there is a lot of clean-up work required... --Kylenz 08:35, 6 April 2024 (UTC)

wood=non-coniferous

this tag has been deprecated for a long time, it should be replaced by its equivalent leaf_type=broadleaved I'll take care of it. Any objections? Marc marc (talk) 11:54, 14 September 2024 (UTC)

Thanks, any help with tag clean-up would be great. Please add or update the status in LINZ/Invented Tags so we can keep track of the remaining work. Thanks :) --Kylenz 22:22, 14 September 2024 (UTC)

wrong/old fashion attribution to improve

THe modern attribution is :

changeset tag are read-only, so nobody 'll be able to change it. changeset tag also describe what's from this source, while avoiding giving the impression that any future improvements by osm contributors come from this source Marc marc (talk) 12:39, 14 September 2024 (UTC)

Hi, all these tags are deprecated and slowly being removed. Any imports after 2014 stopped using these tags on every feature. You can see a full list of all the 'discardable' tags here: LINZ/Invented Tags. The next step would be to add these tags to iD's 'discardable' list so that they're removed when anyone edits these features --Kylenz 22:22, 14 September 2024 (UTC)
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Talk:LINZ&oldid=2765396"