Import/Catalogue/AssemblyPointsFVG

From OpenStreetMap Wiki
Jump to navigation Jump to search

About

This page is about importing 2k emergency earthquake assembly points of Friuli Venezia Giulia Region (RAFVG), Italy. Data is provided by scraping Protezione Civile (PCFVG) website. The import is being discussed on the OSM mailing list and the imports mailing list. The import will be the result of consensus there.

Goals

This import aims to have an updated and mantainable set of POIs for the RAFVG territory (OSM admin_level=4).

Schedule

Starting from Dec 2019, progress will be trackable in an audit map, for the first batch refer to AA_1230-1265. Next batches can be tracked in Progress paragraph. Due to today's low number of OSM assembly points in RAFVG, schedule should take 10-20 days in audit activities and import.

Import Data

Background

Dataset has been built starting from html's thru scraping and html2csv conversion process.

Metadata

Datasets feature the following:

  • source name: Piano regionale delle emergenze
  • scraping date: 2019-09-03
  • AOI: Friuli Venezia Giulia
  • operator: PCFVG

Legal

According to OSM ODbl compatibility table, licence is compatible for importing.

Record format and tagging plan

Data will be pruned and adapted to OSM tagging schema thru OpenRefine.

Find below a 2 rows exerpt of scraped and converted CSV: 

001 - Parco Borgo  S.Antonio,B.go S.Antonio - B.go S.Antonio,"13° 24' 07.34'' E, 45° 48' 05.90'' N"
002 - Piazzale dei Tigli,"S.Valentino - Via Gramsci, 8, 33059, Fiumicello Villa Vicentina","13° 24' 34.21'' E, 45° 47' 27.11'' N"

Import Type

Source data shall be checked by auditing.

Audit support map

The dataset will be imported on its regional base (OSM admin_level=4). OSM candidate nodes will be presented as pins on this general map. First batch in the following audit map. Next audit maps will be published in Progress paragraph.

Pins

  • Blue translucent: dataset position
  • Blue: OSM position (centroid if polygon)
  • Green: new POI, can be dragged in better position.

Fields

  • Yellow: proposed tag value substitution
  • Green: new tag

Goals

This audit aims to add missing source data POIs and to update OSM existing ones. Besides, you can take the chance to check:

  • name typos
  • position
  • duplicates

For any doubt, "skip" will postpone POI audit or a "fixme" will be inherited by OSM candidate object.

Team Approach

Import will be managed by OSM user Cascafico; audit will be open to any OSM user accessing audit map.

Workflow

Step by step operations:

  1. data scraping
  2. OpenRefine operations
  3. conflation
  4. community audit
  5. audit freeze
  6. osm data generation
  7. final check thru JOSM

In case of import problems, changeset involved will be reverted using proper reverter

Data Preparation

The data is presented in individual html files, one for each municipality. An iterative wget has been executed, based on ISTAT code uniquely defining municipalities.

Refining

Some normalizations require general refining operations, performed by OpenRefine:

  • name to title case
  • DMS to decimal coordinate conversion

Tagging

Refined data contains fields:

  • name (usually locality and/or address name)
  • ref (each municipality numbering)
  • lat
  • lon

which can be directly be mapped in OSM tags.

Conflation

Conflation parameters are set in profile file and are based on OSM tag emergency=assembly_point. Due to the present low density of POIs, conflation process should be rather simple.

Upload

Data shall be uploaded manually thru JOSM editor. Dedicated upload account shall be attilaimport.

Changeset Tags

Changesets should be tagged with:

Progress

tag audit audit status ready to upload changeset
assembly_point AA_1230-1265 0 % AA_1230-1265.osm TBD
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Import/Catalogue/AssemblyPointsFVG&oldid=2109946"