Import/Catalogue/Vicmap Address

From OpenStreetMap Wiki
Jump to navigation Jump to search

Vicmap Address is an import of Vicmap Address, address data covering Victoria, Australia. The bulk of the import was completed in December 2024. While there is still scope for further work, it is not currently being worked on.

Goals

The goal is to increase the coverage of address data in OSM from <10% currently to >95%, to allow for most addresses to be found by users searching with OSM. It's acknowledged that not all Vicmap addresses will be imported (due to low confidence on conflicts with existing OSM data).

It's is expected that Vicmap data once imported into OSM will be treated as native OSM data that can be further improved over time by mappers and not treated as "untouchable" or absolutely correct.

Schedule

Mon 23rd May 2022 - Import proposal sent to imports and cc talk-au mailing list.

Stage Description Status
Stage 0 Find and fix abbreviated addr:street=* values on existing addresses Completed
Stage 1 Add postal_code=* to existing Victorian admin_level=9 boundaries Completed
Stage 2 Set addr:unit=* as derived from existing addr:housenumber=* represented as X/Y Skipped
Stage 3 Add new addresses in blocks without any pre-existing addresses Completed
Stage 3 Repair corrupted data from Stage 3 Completed
Stage 4 Set addr:flats=* on existing addresses as derived from Vicmap To consider
Stage 5 Add new addresses in blocks with existing addresses Completed
Stage 6
Stage 7 MapRoulette challenge to manually review duplicate addresses from Vicmap far apart from each other In Progress
Stage 8 MapRoulette challenge to manually review imported addresses sharing the same location In Progress

Future work involves a process to import new addresses as they appear in future Vicmap releases, and deal with addresses which are modified or deleted from Vicmap.

Import Data

Background

Data source site: https://www.land.vic.gov.au/maps-and-spatial/spatial-data/vicmap-catalogue/vicmap-address
Data license: https://www.land.vic.gov.au/maps-and-spatial/spatial-data/vicmap-catalogue/vicmap-address
Type of license (if applicable): CC BY 4.0.
Link to permission (if required): https://wiki.openstreetmap.org/wiki/File:Vicmap_CCBYPermission_OSM_Final_Jan2018_Ltr.pdf
OSM attribution (if required): https://wiki.openstreetmap.org/wiki/Contributors#Victorian_Government_data
ODbL Compliance verified: yes

The import preparation code and further technical documentation is at https://gitlab.com/alantgeo/vicmap2osm/.

OSM Data Files

Generated at https://gitlab.com/alantgeo/vicmap2osm/-/pipelines/1583156942 with data files imported archived at https://f001.backblazeb2.com/file/vicmap2osm-imports/index.html

Import Type

One time automated import via upload.py and other tools per https://gitlab.com/alantgeo/vicmap2osm#import-process.

Data Preparation

Data Reduction & Simplification

The import repository readme documents how the source Vicmap data has been processed into upload ready OSM data, but to summarise:

  1. Convert into OSM schema
  2. Remove duplicates where all address attributes match at the same location or within a small proximity
  3. Reduce address points with the exact same coordinates but different address attributes

Tagging Plans

The import repository readme documents tags and the mapping, but to summarise it's roughly:

OSM Tag Vicmap Attrtibutes
addr:unit=* ${BUNIT_PRE1}${BUNIT_ID1}${BUNIT_SUF1}-${BUNIT_PRE2}${BUNIT_ID2}${BUNIT_SUF2}
addr:housenumber=* ${HSE_PRE1}${HSE_ID1}${HSE_SUF1}-${HSE_PRE2}${HSE_ID2}${HSE_SUF2}
addr:street=* ${ROAD_NAME} ${ROAD_TYPE} ${RD_SUF}

Community feedback mostly preferred to omit addr:suburb=*, addr:state=* and addr:postcode=* tags, so these have been omitted.

Changeset Tags

Per https://gitlab.com/alantgeo/vicmap2osm/#changeset-tags changeset tags for the import are:

Data Transformation

See the import repository readme.

Data Transformation Results

Post a link to your OSM XML files.

Data Merge Workflow

Team Approach

Currently the import has been developed and planned to be executed by System-users-3.svgaharvey (on osm, edits, contrib, heatmap, chngset com.), with the help of community feedback on talk-au. Anyone is free to volunteer to assist or be involved with this import.

References

List all factors that will be evaluated in the import.

Workflow

The import process is documented at https://gitlab.com/alantgeo/vicmap2osm#import-process.

The main part of the import will be done with a changeset per suburb. If for some reason a revert is needed, then likely this would be done with the JOSM reverter.

Conflation

This is described in full at the import repository readme, but to summarise:

Addresses were grouped by city block as it is rare that both OSM addresses and Vicmap addresses are incorrectly positioned in a different block.

  1. All the blocks without a single OSM address are considered to have no conflicts and therefore all Vicmap addresses in that block are imported as low risk for duplication.
  2. Where a candidate Vicmap address was found inside an existing address polygon from OSM, this is manually reviewed.

QA

Manual inspection performed of a subset of data prior to and post upload.

See also

The email to the Imports mailing list was sent on YYYY-MM-DD and can be found in the archives of the mailing list at [1].

The email to the local Australian community was sent in May 2021, archived at https://lists.openstreetmap.org/pipermail/talk-au/2021-May/014622.html