Fi:Maastotietokanta/Road Import Stage1 Workflow

Preparations

• Read this document fully through (surprisingly many still won't :-()
• If you have not mapped anything so far, you should get used to mapping by other means prior to participating in any imports such as this
• You should learn how to use JOSM validator, its use mandatory for this import to detect obvious geometry errors that are quite common in NLS.fi data
• You need to create a separate account for NLS.fi imports, see: http://wiki.openstreetmap.org/wiki/Import/Guidelines#Use_a_dedicated_user_account. Unfortunately due to technical limitations you cannot use the same email address as with your main account.
  • The recommended format for the import account is <normalusername>_nlsfi_import
  • You can make a copy of josm preferences folder and use java -Djosm.home=copiedfolder -jar josm.jar to start JOSM with the alternative preferences. This way you can switch between the import account and the normal account easily!
• NLS.fi orthophoto TMS can be setup using the link from the coordination tool, NLS.fi "peruskartta" TMS likewise. Both are also available as background layers for the coordination tool itself.
  • Unfortunately the kapsi provided NLS.fi orthoimagery are not complete, therefore we should focus first on the subnets that fall within the imagery coverage (either Bing or NLS.fi orthoimagery). We're trying to work a solution to this issue as soon as possible so the situation hopefully improves soon.
• As most of the classification for highway=road requires setting highway=track to the way, it is very useful to add a shortcut key for that in JOSM
• The coordination tool is able to directly feed the data into JOSM, however, you need to check from JOSM preferences that remote control is enabled (remote control is much easier to use than downloading an .osm file by hand which is also supported).

Terminology

• subnet is a collection of NLS.fi ways that form a network which should not overlap anything with OSM (overlap detection is automated, but later changes may appear and you need to check them).
• subnettool is used for distributing and coordinating the import among mappers. Subnettool is based on OSMTM but many extensions and improvements to it have been implemented.
• tail is a part of the subnet that needs to be extented in the post import process. It was cut in the middle because of existing nearby OSM data (< 40m).

Step-by-Step Process

Selecting Area and Confirming It Is Eligible

1. Select a subnet that is available from the subnettool: https://nlsfi.zeip.fi/ . The tool displays convex hull of the subnet. Please note that the largest networks require multiple hours of checking to complete and there are also some complications on uploading many objects (if you don't have enough experience on OSM to know how to deal with them properly, you SHOULD NOT pick networks larger than 100km, preferrably even smaller ones). 30km or smaller subnets usually take few minutes to go through fully and are quite safe to select until you acquire more experience.
2. Take the subnet in the subnettool.
3. If you at some point decide to abort the process for the subnet you have selected (and not continue it relatively soon), please unlock the subnet in subnettool. If you hold subnet for too long, the subnet is automatically unlocked (120 minutes currently).
4. Confirm that the canvas is indeed still empty around the subnet by loading slightly larger area into editor first. Subnettool provides a link that load the convex hull based boundary directly into JOSM using remotecontrol. The convex hull may shortcut on concave parts of the subnet such that there is a false-overlap where no actual subnet data exists. If that happens, you need to verify the overlap using the actual subnet data.
5. Check that there is Bing or NLS.fi orthoimagery coverage for the full subnet. NLS.fi orthoimagery coverage may hopefully get improved soon to nearly full coverage so please be patient if your favorite area currently lacks coverage.
6. Please keep an eye on existing, very badly positioned geometries in OSM. If there seems to be overlap to your subnet (e.g., the same road but very incorrect geometry outside the subnet area) abort the process and note the problem you found to the subnettool comment field and mark the subnet as done. Please try to improve the incorrect data in OSM manually. Once done that, go to first step and select another subnet. The subnet will be reprocessed later when additional analysis takes place and its parts can be then imported by following this same process.
7. Make sure you are not mixing existing OSM data with the import related ways on a same layer to avoid problems later on, the easiest way to ensure this is to delete the data layers before the next step.
8. Download the .osm file of the subnet, however, it is easier to use JOSM remotecontrol to load it directly into the editor.

Checking Through the Subnet

1. Please do not extend ways beyond how they appear in the subnet until post import process! Removal of incorrect data from the subnet itself is still perfectly fine and minor edits, e.g., to connect disjoint way segments within the subnet.
2. Classify all highway=road that appear in the subnet, use Bing or NLS.fi orthophotos to help you. The could be a small number ways for which you cannot determine highway=* from the imagery alone, in that case it's ok to leave them as highway=road until somebody surveys that particular way. If you notice some way does clearly does not seem to exist in neither imagery, at minimum add FIXME=* about your suspicions, although it is reasonably safe to delete such ways (be sure that the way is not just hiding fully under thick layer of trees). Note also Finnish road classification guideline.
3. Keep an eye on other errors (e.g., road misclassification near to your subnet, non-existing roads, grossly incorrect coastline, etc.). Typically fixing such errors requires touching geometries outside of the subnet, postpone that into post import process step. Take notes, if necessary, on what needs to be done. You MUST fix them yourself later on in post import process. Don't import anything unless you really intend to fix them (i.e., don't expect somebody else than you to fix them)!
4. Don't assume NLS.fi data to be perfect, you are quite likely to be correct when NLS.fi data disagrees what you can see in the aerial imagery!!! If available, checking against both Bing and NLS.fi orthophotos gives even higher certainty. Please remember than Bing can have reasonably large offsets around Finland whereas orthocorrection of the NLS.fi imagery is reasonably good.
5. Run JOSM validator. Check all errors and correct any geometry errors whenever incorrectly drawn in NLS.fi data. There could be other kinds of errors too which you need to also solve (you need some experience in mapping to handle such). Quite common error in NLS.fi data is not connected endpoints (i.e., two ways end very near each other but the nodes are not exactly the same, JOSM validator finds these for you but gives also some false positives when a way is short). Obviously such ways should be connected manual before uploading.
6. If you are unsure what to do in some specific situation (how to fix?, etc.), feel free to drop by and ask at irc://irc.oftc.net #osm-fi (also English is ok).

Uploading

1. Once everything is corrected and you're ready to upload...
2. It is recommended to save the subnet in its fixed state before trying to upload (to avoid redoing fixes if the uploading fails for some reason).
3. Please remember to use your NLS.fi IMPORT ACCOUNT (it is probably easiest to have different JOSM preferences altogether for this task)
4. Confirm that the upload dialog shows correct changeset tags. The subnet datafile tries to set correct changeset tags, however, due to bugs in JOSM handling them it is not always successful and they need to be added manually. The list of tags you need are shown below. It might help if you try to close and reopen the upload dialog multiple times (e.g., I got the correct tags on the second reopen) but please pay special attention on the subnet ID if you do this because JOSM might use the ID from the previous changeset. However, it probably varies based on JOSM version if this reopen trick works for you or not!
5. If you notice some problem during uploading, please abort the upload and revert the partial changeset (or full changeset if you realize it only after the upload completes). Note that the revert gets quite complicated if you have already upload any of the manual changes in the post import process (you might need to revert them first, unfortunately). If you need help in reverting something, please contact OSM user ij_ or ask help in IRC: irc://irc.oftc.net #osm-fi.

Key	Value
comment	Import NLS.fi road subnet ID
source	NLS.fi Topographic Database
source:date	2013-08-27
license	http://www.maanmittauslaitos.fi/en/NLS_open_data_licence_version1_20120501
import	yes

Post Import Process

1. Once the upload is complete, switch to use your NORMAL ACCOUNT. This is probably easiest to achieve if you have JOSM with two different preferences.
2. You should now connect the isolated imported network to the existing OSM highways. This step is necessary to avoid it appearing in some isolation detection tools that are periodically run. Please remember that some small subnets might not be connectable at all (e.g., connects only between landuse=farms), you don't need to invent bogus connectivity in such case.
3. Also fix all the errors you found earlier that requires adding more geometries or modifying surroundings of the subnet. It is expected that many problems are to be found in the existing OSM data near the subnets simply because we look now all around Finland. This is nothing to be afraid of.
4. Once all is done, mark your subnet into subnettool as done. If there was nothing noteworthy in the subnet itself, there is no need to fill anything to the comment box. The subnets are auto-validated. In case a subnet is fully connected it will turn validated (green) after few hours. In case there's some subnet which remains in done state (red) even after you think you've connected it fully, please check the auto-validator debug listing. The listing shows the tails which still require connecting, that is, if the connection/road really exists (there could be errors in NLS.fi dataset too). Please note that there could be other (short) connections to the existing OSM data which auto-validator tail analysis cannot detect, so don't trust the auto-validator tail analysis debug listing alone!
5. ...Then optionally, go to first step to select another area.