Helsinki region building import

From OpenStreetMap Wiki
Jump to navigation Jump to search

Goals

Add missing outlines and tags for buildings in the Helsinki region (Helsinki, Espoo, Vantaa).

Schedule

  • End of Nov 2014, write plan and receive feedback.
  • Before end of 2014, execute the first step of the import.

Import Data (Stage 1, Vantaa)

Background

As all the municipalities have different data formats, we are planning to separate the import in three stages based on the municipality. The source dataset is of very good quality because the same data is used for calculating the property tax (stakeholders have good reasons to have the data correctly).

Data source site: http://www.hri.fi/fi/dataset/vantaan-rakennukset
Data license: http://www.vantaa.fi/instancedata/prime_product_julkaisu/vantaa/embeds/vantaawwwstructure/101830_Vantaan_kaupungin_mittausosaston_avoimen_datan_lisenssi.pdf
ODbL Compliance verified: yes, explicit OK received from Sami Rapo on 24.9.2014 (attribution in OSM contributors list page only requirement explicitly discussed) 

From: Rapo Sami
Sent: 24 September 2014 15:56
Subject: VS: Avoimen rakennusdatan tekijänoikeusmerkintä ja käyttö Open Street Mapissa
 
Hei,
 
Vantaan avoimen datana julkaistun rakennusdatan lisenssimalli on  ”Vantaan kuntatekniikan keskuksen mittausosaston avoimen tietoaineiston lisenssi” eli ei siis HRI:n ”nimeä”.
http://www.vantaa.fi/instancedata/prime_product_julkaisu/vantaa/embeds/vantaawwwstructure/101830_Vantaan_kaupungin_mittausosaston_avoimen_datan_lisenssi.pdf
OSM:n Catalogue on meidän tapauksessa oikea paikka asiasta mainintaan.
Englanninkielinen versio on kielenkääntäjiltä aivan lähiaikoina tulossa eli viitataan siihen.
 
 
/Sami R.

Import Type

  • Import consists of three steps:
    • Step 1, Data that does not overlap with existing buildings or roads.
    • Step 2, Data that does overlap fully with existing buildings and where only tagging is added.
    • Step 3, Data that overlaps partially, and which has to be imported manually.

Data Preparation (Step 1, non-overlapping data)

Data Reduction & Simplification

Before the import can be started, the alignment of roads in OSM that overlap with buildings from the import must be fixed. The alignment using both Bing and NLS.fi orthophotos was completed in 2014-10 (mostly by user ij_).

The data is then reduced to only the buildings that do not overlap with already existing buildings as described in the workflow.

Tagging Plans

Vantaa source data fields, and their mapping
Field name Translation Mapping Note
kiinteistotunnus Plot ID None municipality-area-village-plot (might be replaced by the national permanent building id before the import)
rakennusnro Building number None Specifies the building on the plot (001-)
valmistumispvm Completion date start_date=* Year 1900 means no information and 01-01 and 12-31 year only
käyttötarkoitus_koodi Usage code Finland_Classification_of_Buildings_1994
käyttötarkoitus Usage None
rakennusmateriaali Building material None
julkisivumateriaali Facade material building:material=*
kerrostenlkm Number of floors building:levels=*
asuntoja Apartments building:flats=*
katuosoite_suomeksi Street address (in Finnish) addr:street=* and addr:housenumber=*
postitoimipaikka Post office addr:postcode=* Actually postal code
katuosoite_ruotsiksi Street address (in Swedish) None Information can be found from corresponding street
vir_rakennustunnus Official building ID building:fi:id=* plot ID + building number
pysyväraknro Permanent building id None (Only within Vantaa)
rakennusala Building area None

Changeset Tags

source=City of Vantaa source:date=2014-09-23

Workflow

  • Load building outlines to PostGIS database.
  • Calculate distance to nearest buildings:
CREATE TABLE vantaa_dist AS (SELECT DISTINCT ON (b.ogc_fid) b.ogc_fid,
b.vir_rakennustunnus, o.osm_id, o.building, ST_Distance(o.way,
b.wkb_geometry) AS distance FROM vantaa_rak AS b, planet_osm_polygon AS
o WHERE ST_DWithin(o.way, b.wkb_geometry, 1000) AND o.building != '' AND
b.vir_rakennustunnus != '' ORDER BY b.ogc_fid, ST_Distance(o.way,
b.wkb_geometry))
  • Select non-overlapping data and split it per postal code in order to keep the changesets reasonably sized:
POSTINUMEROT=(0 00420 00750 01200 01230 01260 01280 01300 01340 01350 01360 01370 01380 01390 01400 01420 01450 01480 01490 01510 01520 01530 01590 01600 01610 01620 01630 01640 01650 01660 01670 01680 01690 01700 01710 01720 01730 01740 01750 01760 01770 01800 02970 04260 04320)

for POSTI in ${POSTINUMEROT[*]}; do
    echo $POSTI
    ./ogr2osm.py -t vantaa_buildings -f -v --sql "SELECT DISTINCT ON (b.ogc_fid) b.*, d.distance FROM vantaa_rak as b, buildings_distance_vantaa AS d WHERE b.ogc_fid = d.ogc_fid AND d.distance > 2 AND b.postitoimipaikka = '${POSTI}'" -o ../vantaa/vantaa_${POSTI}.osm "PG:dbname=osm user=hannes host=localhost" 2> ${POSTI}.log
done
  • Validate generated .osm with JOSM
  • Upload data
  • Remove duplicate address and amenity nodes by finding them automatically and carefully editing manually.

OSM Data Files

http://koti.kapsi.fi/~hannes/vantaa/

Data Preparation (Step 2, fully overlapping data)

Only tags for geographies that are similar enough are updated.

Data Preparation (Step 3, partially overlapping data)

Data which is not consistant between both datasets has to be imported manually.

Import Data (Stage 2, Espoo)

Background

As all the municipalities have different data formats, we are planning to separate the import in three stages based on the municipality. The source dataset is of very good quality because the same data is used for calculating the property tax (stakeholders have good reasons to have the data correctly).

Data source site: http://www.hri.fi/fi/dataset/espoon-rakennukset
Data license: http://www.hri.fi/lisenssit/hri-nimea/
ODbL Compliance verified: yes, explicit OK received from Ari Purhonen on 10.10.2014 (attribution in OSM contributors list page only requirement explicitly discussed) 

From: Purhonen Ari
Sent: 10 October 2014 14:38
Subject: VS: Espoon rakennukset (HRI-aineisto) OSM-pilottiin
 
Moi
 
Espoon antaa luvan käyttää rakennusaineistoa Mirjan 24.9. viestin mukaisesti. Meille riittää myös, että tekijänoikeudet kirjataan pelkästään katalogiin.
 
Tästä on keskusteltu viimeksi tänään kaupungingeodeettimme Tuomas Frösen kanssa, joka myös hyväksyy käytön ja tekijänoikeuden kirjaamistavan.
 
yt. Ari
 
Ari Purhonen
ratkaisuasiantuntija/lösningsspecialist
paikkatietokoordinaattori/GIS koordinator
043 826 5242
ari.purhonen@espoo.fi
 
Teknisen ja ympäristötoimen esikunta/Teknik- och miljösektorns stab
PL/PB 49 (Virastopiha 2 C/Ämbetsplatsen 2 C)
02070 ESPOON KAUPUNKI/ESBO STAD

Import Type

  • Import consists of three steps:
    • Step 1, Data that does not overlap with existing buildings or roads.
    • Step 2, Data that does overlap fully with existing buildings and where only tagging is added.
    • Step 3, Data that overlaps partially, and which has to be imported manually.

Data Preparation (Step 1, non-overlapping data)

Data Reduction & Simplification

Before the import can be started, the alignment of roads in osm that overlap with buildings from the import must be fixed. The alignment is done using both Bing and NLS.fi orthophotos.

The data is then reduced to only the buildings that do not overlap with already existing buildings as described in the workflow.

Tagging Plans

Espoo source data fields, and their mapping
Field name Translation Mapping Note
id id None Used internally
laji Type None
lajin_selitys Type explanation None
z1 z1 None
z2 z2 None
teksti Official building ID building:fi:id=* Plot ID + building number
poimitaan_suunnitelmaan None
lupaperustelinkki None
rakennuksen rasterointi None
rakennuksen kaytto None
kiinteistotunnus Plot ID None municipality-area-village-plot (might be replaced by the national permanent building id before the import)
rakennusnumero Building number None Specifies the building on the plot (001-)
ekoordinaatti None
nkoordinaatti None
kaytossaolotilanne None
kayttotarkoitusnumeo Usage code Finland_Classification_of_Buildings_1994
kayttotarkoitus Usage None
muuta tilaa None
kerrosluku Number of floors building:levels=*
kokonaisala Number of floors None
valmistumispvm Completion date start_date=* Year 19000101 means no information and ????1231 year only
katunimi Street address (in Finnish) addr:street=*
osoitenumero House number addr:housenumber=*
osoitenumero2 House number2 addr:housenumber=* Format housenumber-housenumber2, eg. 10-12
osoitekirjain House letter addr:housenumber=* Format housenumberhouseletter, eg. 10A

Changeset Tags

source=City of Espoo source:date=2013-10-04

Workflow

  • Load building outlines to PostGIS database.
  • Calculate distance to nearest buildings:
CREATE TABLE espoo_dist AS (SELECT DISTINCT ON (b.ogc_fid) b.ogc_fid,
b.teksti, o.osm_id, o.building, ST_Distance(o.way,
b.wkb_geometry) AS distance FROM espoo_rak AS b, planet_osm_polygon AS
o WHERE ST_DWithin(o.way, b.wkb_geometry, 1000) AND o.building != '' AND
b.teksti != '' ORDER BY b.ogc_fid, ST_Distance(o.way,
b.wkb_geometry))
  • Select non-overlapping data and split it per postal code in order to keep the changesets reasonably sized:
ALUEET=(04901 04902 04903 04904 04905 04906 04907 04908 04909 04940 04941 04942 04943 04944 04945 04946 04987 04988 04989)

for ALUE in ${ALUEET[*]}; do
    echo $ALUE
    ./ogr2osm.py -t espoo_buildings -f -v --sql "SELECT DISTINCT ON (b.ogc_fid) b.*, d.distance FROM espoo_rak as b, buildings_distance_espoo AS d WHERE b.ogc_fid = d.ogc_fid AND d.distance > 2 AND substring(b.teksti for 5) = '${ALUE}'" -o ../espoo/espoo_${ALUE}.osm "PG:dbname=osm user=hannes host=localhost" 2> ${ALUE}.log
done
  • Validate generated osm with JOSM
  • Upload data
  • Remove duplicate address and amenity nodes by finding them automatically and carefully editing manually.

OSM Data Files

http://koti.kapsi.fi/~hannes/espoo/

Import Data (Stage 3, Helsinki)

TBD, based on experience from Vantaa and Espoo

Team Approach

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Helsinki_region_building_import&oldid=1917034"