Canvec Import Guide

Imports and automated edits should only be carried out by those with experience and understanding of the way the OpenStreetMap community creates maps, and only with careful planning and consultation with the local community.
See Import/Guidelines and Automated Edits code of conduct for more information. Imports/automated edits which do not follow these guidelines might be reverted!

A few notes to read first:

• This is a beginners guide for importing data from the Canvec database in Canada. This guide is intended for beginners to importing Canvec, not beginners to OSM.
• Words like "I" and "Me" in this guide refer to the user "Jmarchon".
• You will need some experience with JOSM, .osm files, and OSM in general, along with the JOSM plugins "todo", "josm-batch-downloader". and "utilsplugin2".
• I will not be importing highways and railways from the Canvec dataset. These are too troublesome for me to have wanted to deal with.
• Please read through the guide carefully, as there is some important information.

Step 1 - Finding a tile to import:

The first step is to find a tile to import. Canvec data is divided up into tiles based on Canada's NTS grid. In short, the NTS grid divides Canada into a hierarchy of tiles, each with a unique code (for example "094L04.0.0" with "094", "L", "04", ".0", and the other ".0" being individual parts). The simplest way to find a tile to import is to use Canada's Toporama viewer. After you have opened the website, expand the "Map Layers and Legend(s)" section and check the mark next to "Grids (NTS, Lat/Long, UTM)". This should enable the NTS grid overlay.

Then, zoom to the area where you would like to import Canvec data. I recommend taking looks at the OSM main page to make sure you pick a tile that hasn't been imported yet. Zoom in all the way until squares with a 6-digit code in them show up. For this guide, I will use the tile "094L04" as an example.

Choosing a tile without large lakes, little existing data, and no coastlines will make conflation much easier.

Step 2 - Downloading the data:

Once you have the code for the tile which you would like to import, head over to the NRCAN ftp server for .osm files. That website is rather slow, so be patient and give it a good little while to load. For the example tile "094L04", you would then click on "094/", then "L/", and finally "094L04.zip". Save the file somewhere where you can easily access it, and then navigate to it in your file explorer of choice, and extract the files. You should end up with a folder containing a bunch of .osm files and one .txt file (the latter of which you won't be needing).

In JOSM, click "File" in the top left, then "Open..." and then navigate to the folder containing the .osm files. As a start, you should just open one of these files. I will describe a faster way to import these files later, but that process requires a bit more experience.

Step 3 - Data preprocessing:

With one of the files open in JOSM, right-click on it in the "layers" pane, and click on the "discourage upload" button to enable uploading.

Then, search for and delete any railways and highways present. I use the search query highway OR railway and then delete the objects selected by that query.

Next, you will have to fix the tagging for river areas. In the downloaded data, river areas are tagged as waterway=riverbank and area=yes, which has been deprecated. Use the search query riverbank to find any of these items. Then, remove the two deprecated tags from the objects, and add the replacement tags natural=water and water=river. Pipelines also require a small change. These are less common in the data, but still present. Change the deprecated key type= on any pipelines to the key substance=. Any missed tag changes for riverbanks and 'pipelines should be picked up by the JOSM validator later.

Finally, I would recommend deleted the source=NRCan-CanVec-10.0 tags from all objects. These are no longer necessary since the source is noted on each changeset.

Step 4 - Downloading existing OSM data:

The next step is to download any existing OSM data in and near the tile you want to import to another layer. There are multiple ways to do this (including simply using the built-in JOSM download dialogue), but my prefered method is as follows:

1. Draw an untagged way around the Canvec data with a little buffer around the edges
2. Select that way
3. Click on "Data", then "Batch Download", or use the keyboard shortcut Ctrl+Shift+D(on Windows & Linux). This uses the "josm-batch-downloader" plugin to semi-automatically download the existing OSM data.

You should now have 2 layers; one with the Canvec data, and another with the existing OSM data.

Step 5 - Conflation:

Now, check the existing OSM data for any objects that need to be conflated. This includes any items of the sort you are importing, such as forests, streams, and river areas, but does not include highways, railways, and any other objects of the sort that we are NOT importing. Select all the items that need to be conflated and click the "Add" button on the Todo list pane created by the "todo" plugin.

Then, right-click on the Canvec data layer in the "Layers" pane, and click "Merge". Select the layer with the existing OSM data as the target and click "Merge layer".

Once the merge has completed, you should have only one layer left. Go through the list of items to conflate (if any).

Step 6 - Validation:

The next step is to run the JOSM validator and fix any errors and warnings. I recommend selecting all objects in the data layer, and then running the validator with the keyboard shortcut Shift+V. I have found pressing "Upload" to run the validator to not always catch all the errors.

Once the validation has completed (which can take a while depending on how much data is present), work through the list of items to fix. Here are a few common things that may need extra attention:

• Since the Canvec data is cut into nice squares, duplicated nodes will exist all around the newly imported data, unless you are importing to an area where no adjacent tiles have been imported yet. These will show up in the JOSM validator pane as "... duplicated nodes". For example, the end nodes of 2 waterways will show up under the error category "Waterway duplicated nodes". These can be simply resolved by selecting the categories and then clicking the button labelled "Fix". Let JOSM think for a moment or 2, and then the issues should be resolved.
• Large river area ways are sometimes split into multiple unclosed ways, some with 2,000 nodes. Combine the ways, and then split the closed river area way using UtilsPlugin2's Split Object function.
• Duplicated ways are very common. There isn't really anything wrong with just leaving them as is, but I fix them anyway. Select each entry manually from the validator pane, and then use JOSM's "Join overlapping Areas" function (default keyboard shortcut is Shift+J) to combine the ways.

Ignore any errors in existing data, unless you feel like fixing those as well.

Next, I recommend going along the borders of each tile and checking for errors manually, as most appear at the borders. Use JOSM's "Join overlapping Areas" function again to merge adjoining lakes.

Step 7 - Upload:

Once all of the above steps are complete, you can continue to the upload dialogue. It may take quite a while to show up if there is a lot of data to upload.

Set the upload chunk size to 5,000 or 10,000 objects.

Use a changeset comment like "Imported Canvec tile ...".

Paste the following 2 tags to the changeset:

website=https://wiki.openstreetmap.org/wiki/CanVec

source=NRCan-CanVec-10.0

Then, click the upload button and wait. Afterwards, repeat the whole process with another tile!

Note that uploads are rate limited to 100,000 changes every hour on accounts without the importer role. The Import Guidelines page says to email the DWG to get that role.

Faster importing by uploading larger tiles:

Here is a tip for faster importing once you have gained a decent amount of experience with the method described above and there are no issues:

In the folders downloadable from the NRcan FTP server, there are many files. Open some or all of these at once. Then, select them in the JOSM layers pane, right-click, click "Merge", and then "Merge layer". This process can take upwards of 30 minutes, depending on your computer and the amount of tiles and data being merged.

When it has completed, you can continue the process as normal. Be sure to double check the borders of the tiles.

Easier tile picking:

For easier picking of which tiles to import, you can use QGIS or JOSM. Open the following WMS link either in JOSM or QGIS: https://maps.geogratis.gc.ca/wms/nts_grid_en?version=1.3.0&layers=nts_index_layers&legend_format=image/png&feature_info_type=text/html

I use the layers "NTS Labels - 1:50,000" and "NTS Data Sheets - 1:50,000".