JOSM/Plugins/Conflation

Conflation

for conflating (merging) of data

(Josh Doe)

This is a plugin for conflating (merging) objects in JOSM. For now it focuses on one-to-one matches between POIs, such as addresses and buildings, parks, schools, bus stops, etc. It leverages the Replace Geometry command from utilsplugin2 to replace or upgrade one OSM object to another.

Motivating use cases

GNIS updates

In 2009 feature data from GNIS was imported into OSM as nodes. Since that time some have been converted to areas, been moved, or even replaced. In this process the gnis:feature_id has sometimes been lost, and the conflation process could add back the correct ID. Also, GNIS has provided updates since 2009, which could be added to OSM with this tool. The conflation plugin could merge this data, with a strong weighting towards matching gnis_feature_id=* and name=*.

Replacing address nodes with building polygons

Consider replacing address nodes in OSM with high quality building data from a local authority. If the buildings have addresses associated with them, then matches would be heavily weighted towards matching addr:housenumber=* and addr:street=*. If there is no such addressing information, the data could still be conflated by setting a threshold on the distance of perhaps 20 meters.

Updating Building Footprints

Buildings have been imported or traced by hand. An external data source has been released, and you would like to update the building in OSM to use the best building footprint from either source.

Optimal parking space assignment

Assigning parking spaces to houses based on distance (duplicated houses to assign two spaces)

This is for fun rather than for conflating data with OSM. Imagine you have a neighborhood where you've mapped all the parking spaces and the houses. You can "conflate" the two only using distance in the cost calculation to determine the optimal assignment of houses to parking spaces. If each house gets two parking spaces, you could duplicate (copy and paste) the house nodes/polygons, and you'd get two spaces matched to each house.

Installation

Follow these instructions to install the conflation plugin automatically. If you plan on doing matches over 500 objects, you should use the 64 bit java VM, so you don't run out of memory.

Definitions

Reference: the dataset which is presumed to have the best spatial accuracy and attribute (tag) comprehensiveness, typically a third-party dataset such as from a government entity
Subject: the dataset which will have the geometry and/or attributes of the reference dataset merged into it, typically an up-to-date dataset downloaded from the OSM servers

Usage

Data Preparation

The plugin requires that the Reference dataset be in a format that can be loaded by JOSM (an OSM file). For simplicity, download the area that you would like edit first, then use the File | Open command to load in the external OSM file as a new layer. If you do open the external file first, then remember to select the "New layer" option before downloading the area to edit, otherwise JOSM will download the selected area into the existing layer, which will make selections very difficult.

Generating matches

Show the Conflation panel by clicking on the conflation icon in the Edit toolbar (on the left).
Click on the Configure button of the Conflation panel in the Dialogs panel (on the right).
Select the layer with your Reference data and make it active.
- Note: the Reference data will be merged into the Subject data.
Inside this layer, select your Reference data (You can select all data by pressing Ctrl+A).
On the dialog click on the first button Freeze to set your reference data.
Now you have to select your Subject data. You can select them from another layer or the same layer (If from another layer, you have to make it active then you can press Ctrl+A to select all data)
Click on the second button Freeze to set your subject data.
If needed set custom options, then click on Generate matches.

Selecting

Doing the selection will require using the Edit | Search window. If you are conflating ways, you need to only select ways and not points. Conversely, if you are conflating points, you don't want to have any ways selected. Currently the plugin does not support relations, so they should never be selected. For example, "building=* type:way", will correctly select buildings. The type:way is needed to that no relations or points are selected. It is not possible to just select an area with the mouse.

Conflating

In the conflation dialog, there are three tabs, Matches which lists matches found between the reference and subject layer, and Reference only and Subject only which lists unmatched objects from each dataset.

Double-click on a match or unmatched object to zoom and center on and to select the object(s).
Select one or more match and click Conflate to execute Replace Geometry. Select one or more objects from the Reference only list to copy them to the subject layer.

Note that you can also use the same selection for both target and source; the algorithm assigns a very high cost if both objects are the same, so a match should never occur with the same object as both target and source.

Implementation

Reuses existing search and selection functionality for creating reference and subject selections
Uses Java Conflation Suite (JCS) and Java Topology Suite (JTS) for calculating scores between reference and subject objects, and generating the matches
Use Replace Geometry from utilsplugin2 to conflate the objects

New features in v0.4 and v0.5

Allow for merging of just tags, and not geometry
Added costs for "distance" between tags, by using Levenshtein distance
Selecting elements in the editor will select the corresponding conflation matches in the list
In addition to Centroid distance:
- Standard Distance, which may be better suited when matching nodes to polygons
- Hausdorff distance which may be more appropriate when matching polygons to polygons
New «advanced configuration» section identical to the original JCS tool (taken from the OpenJump project), see explanations at the end of its User Manual, page 27 section 7.3

Future work

This is an incomplete and unordered list of tasks that I've thought of. If these or any others would be useful to you, please create a ticket for them.

Create ~~unique~~ better icon
Add thresholds and weights to individual costs (e.g. don't allow match if distance is greater than 100m)
Allow clicking on arrow in conflation layer to select candidate match
Allow declaring an invalid match, and using that information to regenerate the matches
Add keyboard shortcuts to quickly go through matches, with keys for next, previous, merge, "mismatch", etc.
Use color coding in arrows and table to indicate confidence of match
Allow for custom cost and assignment methods

Development

Initially created by User:Joshdoe, but other developers are welcome to fork it on GitHub. Please submit your bug reports and feature requests on the JOSM Trac using the "Plugin conflation" component.

External links