Orlando, Florida Microsoft Building Import

Imports and automated edits should only be carried out by those with experience and understanding of the way the OpenStreetMap community creates maps, and only with careful planning and consultation with the local community.
See Import/Guidelines and Automated Edits code of conduct for more information. Imports/automated edits which do not follow these guidelines might be reverted!

About

This page documents plans for an upcoming import of Microsoft Building Footprint Data supplied under ODbL by Microsoft in March 2017 tagged with available addresses. This effort began November 2017.

User:Valerietheblonde/User:Valerietheblonde_import initiated this import and plans to follow the import checklist.

Import Checklist

Step 1 - Prerequisites

1. Users working on this import are expected to be able to use JOSM, iD, and the HOT Tasking Manager
2. Participating users are expected to review what can go wrong with imports.
3. This import will be of microsoft building data tagged (in Orange County) with addresses.

Step 2 - Community Buy-in

1. It is recommended that before any actual work is performed on the import that contact is made with the community to see if there is interest in importing the data. Different geographic areas in OSM have different acceptance levels for imports. The exact same kind of data set might be welcomed in one area and be rejected in another.
2. Discuss your plan. Email the OSM community to notify them of your plans, including a link to your wiki page. You can do this with an email to (at a minimum) imports@openstreetmap.org, talk-(your country)@openstreetmap.org, and the OSM group specific to the area directly impacted by the import (note that you must join the lists at http://lists.openstreetmap.org/ before mailing to them). This will help gain the benefit of past experiences, which may include having already reviewed the data you're considering for import. Check for local user groups, local chapters, and country-specific mailing lists.
3. Be prepared to answer questions from the community. Discuss with the community the suitability of each layer for importing. Some data can be readily imported without much difficulty, while others are far more difficult (e.g. street centerlines). Also some are broadly accepted for import, while others haven't had much consensus (e.g. parcel boundaries).
4. More complex and large-scale imports should be reviewed with the assistance of more technically-oriented and experienced OSM volunteers.
5. You must not import the data without local buy-in.

Step 3 - License approval

1. You must obtain proper permissions and licenses to use the data in OSM from the data owner. If the license of the data is not compatible with the OpenStreetMap Open Database License, you can not use the data. Many localities already have progressive open data policies. Others have data policies that are almost open, but have conflicts with issues like prohibitions on commercial use or requirements for attribution. Sometimes, getting permission to use data, even if the existing license might seem prohibitive, is as simple as asking the appropriate authority if they are willing to comply with the terms of the OpenStreetMap Open Database License. See Import/GettingPermission for example emails that touch on important issues. See also ODbL Compatibility for a quick view of some compatible and incompatible licenses of data to import.

Step 4 - Documentation

1. You must register your permissions and project by adding a line to the table at Import/Catalogue.
2. You must write a plan for your import in the OSM wiki. Create a wiki page outlining the details of your plan. This plan must include information such as plans for how to convert the data to OSM XML, dividing up the work, how to handle conflation, how to map GIS attributes to OSM tags, how to potentially simplify any data, how you plan to divide up the work, revert plans, changeset size policies, and plans for quality assurance. An example for this can be found at Import/Plan Outline
3. If required by the data owners, you must add an acknowledgement of the list to Contributors.

Step 5 - Import Review

1. You must subscribe to the imports mailing list and post a review of your import to imports@openstreetmap.org. Don't upload any data until the project has been reviewed first. Note that the imports-us@openstreetmap.org list (or any other local imports list) is not considered authoritative, and is not a substitute for approval from imports@openstreetmap.org.
2. If possible, prepare the data and make it available for review.

Step 6 - Uploading

1. Follow your plan.
2. Track your progress.
3. Provide updates to the community on your efforts.
4. Let everyone know when you're done.
5. You must use a dedicated user account.

• Identify data to import
• Translate, tag, and otherwise tenderize the data
• Invite community members to help manually import the outlines
• Divvy up the work using the HOT Tasking Manager
• Do the work: Import/merge the data
• QA the data

Import Data

Background

The building outlines were made available by Microsoft. See Microsoft Building Footprint Data for details.

Addresses within Orange County are available on their FTP site in the public domain[1]. Osceola County releases no data to the public[2]. The City of Kissimmee and the City of Saint Cloud are similarly closed[3][4].

Import Type

This is a community building based effort.

There are currently no plans to script or automate this import.

Data Preparation

Tagging Plans

No source tags will be added.

Changeset Tag

Tag in import-related changesets: #msbuildingimport-orlando

Data Transformation

FROM TAMPA PAGE, TO BE UPDATED The source files will need to be converted to OSM XML. Shpfiles will be processed in Postgresql to expand street addresses. Two slighly different approaches are being proposed. A PostGIS approach github and a Python approach [5].

We have used Paul Norman's org2osm to convert from Postgresql to an OSM XML for this. The translation script is:

def filterTags(attrs):
    if not attrs: 
        return

    tags = {}
    tags['building'] = 'yes'
    if 'HEIGHT' in attrs and attrs['HEIGHT'] != '':
        tags['height'] = attrs['HEIGHT']    

    return tags

Data Merge Workflow

Team Approach

The work for this effort will be divided up into sections, with each section constituting a census block. Census Blocks will be used by the US Tasking Manager.

References

Using the JSOM, each volunteer will begin to work through the data for that district.

• Local knowledge
• Best available imagery
• Existing OSM data

Workflow

TO BE UPDATED

• Open JOSM and the OSM US Task Manager project 118
• Click on the task tab above and claim a task on the map to the right by clicking on an area you'd like to work in
• Click the "JOSM" button, this will open the area in JOSM and load up existing OSM data
• Click the ".osm" button, this will open new building outline data in a separate layer
• Select the new building layer and validate the layer, fix all issues
• Copy all geometry from building layer and paste it into the existing OSM data layer, Ctrl-c, Ctrl-Alt-v.
• Run validation on the existing OSM data layer that now contains new data, resolve all issues emanating from collisions between existing and new data (see merge rules).
• Do a sanity check on data:
• Merge with existing buildings.
  • To merge with a building outline, use the Replace Geometry (Ctrl+Sift+G) by selection the existing building outline and the new address node. Note: Only merge building outline with an address node if the building outline contains just one address node.
• Change visible storage tanks from building=yes to man_made=storage_tank
• Upload data to OSM
• Go back to the Tasking Manager and mark the task as done (but not as validated)
  

Dedicated Import Account

Users will be expected to obtain an unique import account, for example, user Foo might pick Foo_Import.

List of Participating Import Accounts

• User:Valerietheblonde_import

QA

Validation

• Use of validation tools in the Tasking Manager process
• Use Slack for questions and to help others during the import.