WikiProject Uganda/Microsoft building imports for UBOS pilot census preparation

Introduction

This activity is an import of building footprints covering the districts of Amuria, Kapelebyong and Butambala in western Uganda in support of the Uganda Bureau of Statistics’ (Uganda Bureau of Statistics ) technical preparations for a census pilot. The import is currently going through the Import Guidelines steps by which the local OSM community in Uganda and the global OSM import mailing list are engaged and consulted in the process.

Goals

For this building import activity, the goals are to provide a building footprint layer to Uganda Bureau of Statistics  to be able to guide the experimentation and improvement of data collection tools, methodologies and workflows deployed in previous censuses in a particular geographic area in Uganda that has been comprehensively mapped remotely. By knowing where and how building structures are located and/or distributed due a comprehensive building footprint map layer, Uganda Bureau of Statistics  will be able to experiment with different approaches for household surveying and ensure the final approach will enable the gathering, analysis and visualization of household data more efficiently and effectively. In more detail, the census pilot aims to achieve the following:

• Estimate the amount of resources (people and time) required to implement the national data collection activities based on the size/location of the area
• Determine whether new enumeration areas are needed to accelerate the field data collection process
• Enable the Uganda Bureau of Statistics  census team to Identify the most suitable methodology for data collection (sampling versus full range coverage) for implementing the national census.

Timeline

In early 2020, HOT received a request from the Manager of Geo-Information Services at Uganda Bureau of Statistics  for support in acquiring building footprints in four districts in Uganda; these datasets would provide a blueprint for parts of the country that have not been previously mapped and help guide the initial planning of the pilot census activity to take place in mid to late 2020.

The process of incorporating building footprints for Uganda (using RapID) began on 13 February 2020 and continued through 30 April 2020. When this was flagged by the DWG on 30 April 2020 as an import, all activities were halted. On 1 May 2020, a handful of quality checks were carried out to ensure data was completely validated. From 1 May onwards, our team has been working to develop and disseminate the import workflow according to the import guidelines.

Note: this data import is not a blind import - the process will be controlled and closely monitored, where users will be expected to check the validity of the data against aerial imagery and to correct the geometry of each single building structure according to the images.

Import data

Data description

Under Microsoft’s AI for Humanitarian Action program, Bing Maps is contributing to an initiative from Humanitarian OpenStreetMap Team that will bring Artificial Intelligence (AI) assistance to open building mapping. In 2019, Bing Maps released country-wide open building footprints datasets in Uganda (and Tanzania). The Uganda dataset contains 6,928,078‬ computer-generated building footprints derived using Bing Maps algorithms on satellite imagery. While this data has been permitted to be uploaded to OpenStreetMap, systematic and close inspection as well as validation will be needed beforehand. Mappers will be trained to avoid over-writing the hard work of other contributors or blindly importing data into OSM without first checking the local quality. While our metrics show that this data meets or exceeds the quality of hand-drawn building footprints, the data does vary in quality from place to place, between rural and urban, mountains and plains, and so on. For this reason, data quality will be inspected in detail, discussed with the local community to ensure there are no gaps in what data already exists in OSM and what data will be imported to OSM. The data is in geojson format and 1339MB in size.

Background

Data site: https://github.com/microsoft/Uganda-Tanzania-Building-Footprints

Data License: This data is licensed by Microsoft under the Open Data Commons Open Database License (ODbL).

ODbL Compliance verified: Yes

Data files

At the start of the import process, a link enabling the download of raw data in geojosn file format will be added here.

Import type

The plan is to import the building footprint data into OSM manually by a team of trained mappers working with HOT Uganda. Data will be split along village administrative boundaries that have been provided to our team from Uganda Bureau of Statistics  for this activity to ease the coordination of task allocation amongst the mapping team. Using the HOT tasking manager, remote mapping task projects will be set up with arbitrary task boundaries corresponding to the village administrative boundaries.  The benefits of this approach include the following:

• Trained community mappers will ensure proper data quality and can provide feedback quickly to the task management team
• Mapping progress of the import can be checked and validated at any time through HOT’s tasking manager and OSMcha platforms
• It is easy to set up and monitor over time

The list of remote mapping tasks created for this project are as follows:

Districts	Subcounties	Task
Amuria	Adepuru	7916
Amuria		8009
Amuria	Wilia	8141
Amuria	Arbriela	8155
Amuria		8220
Butambala	All villages	8180
Kapelebyong	All villages	8319
Total

Data Preparation

Data reduction and simplification

During the building importation process, buildings that will have been identified as no longer existing on the ground will not be uploaded to OpenStreetMap. No simplification or transformation of the data is expected, outside of correcting potential offset.

Tagging plans

ML Tags	Column Meaning	Action	OSM tags	Action
FID	Unique feature Identifier	Delete	building=”yes”	Add tag
			source= “Microsoft/BuildingFootprints”	Add tag
			mapwithai:source=“MapWithAI”	Add tag

Changeset tags

Changeset comment:#MapwithAI #Uganda #Uganda Bureau of Statistics census

Import=”yes”

Url:Microsoft buildings import for UBOS Pilot Census Preparation

Data Transformation

The original file (in geojson format) will first be converted into a shapefile format. This will allow the size of the file to be reduced significantly to enable reading in JOSM. The shapefile will then be loaded into JOSM using the open layers plugin where all irrelevant tags will be deleted and the proposed tags will be added. From this point, the resulting file will be split according to Uganda Bureau of Statistics ’ 2011 village boundaries and after saved in geojson format.

Data Merge Workflow

Team approach

This import (data integration) will be done through the HOT Tasking Manager, so the number of people importing the data is known and easily tracked. We expect to use 5 mappers during this import process with the following skills:

• Good experience with JOSM.
• Experience with previous imports through the Tasking Manager is not necessary, but a plus.
• S/he knows how to use JOSM filters.
• Skilled working with buildings. It will be crucial for mappers to know how to orthogonalize buildings (Q), round buildings (O), combine ways (C) and ungluing (G).
• S/he knows how to use the ToDo JOSM plugin.
• S/he knows how to create and deal with building relations.
• S/he knows how to deal with conflicts.

References

Following the imports guidelines, this import activity will be discussed in the local Uganda OSM mailing list, and later with the HOT mailing list to gather additional feedback/comments on the proposed process.After gathering feedback from the HOT and OSM Uganda mailing list the document will be shared with global OSM import mailing list for any final thoughts and feedback on the import activity.

Workflows

JOSM Workflow Instructions

These remote mapping tasks will be tailored for intermediate and/or advanced mappers only. With that said, strict supervision and monitoring protocols will be applied throughout the activity to ensure high quality mapping.

Once remote mapping tasks are set up, mappers are instructed to navigate the village task assigned to them and download it to his/her JOSM application. Once downloaded in JOSM, the mapper will add the (identified) village boundary to allow him/her to be sure of the mapping limits. Then, the mapper will add the Microsoft AI buildings for that specific village into JOSM as a new layer.

After loading Microsoft AI buildings into JOSM, the mapper must activate the Bing imagery layer. Since Microsoft AI buildings were largely generated using Bing imagery, it is important to use the same imagery to ensure building structures are properly aligned. From this point on, the mapper must assess each building feature individually using the todolist plugin in JOSM to ensure buildings are aligned with Bing imagery. Once this process is complete, the mapper will be required to activate the Maxar imagery service in JOSM. Maxar, in most areas in Uganda, is newer than Bing imagery (2015-2018 for Maxar versus 2011-2013 for Bing) which he/she will use to arbitrate what buildings to import or not. From here, the mapper will also manually trace the missing buildings that could be newly constructed buildings structures. The mapper will also  manually remove building structures that no longer exist based on the newest imagery  and then add the appropriate tags - building=yes, source=Microsoft/BuildingFootprints.

After completing this process, the mapper will apply a second assessment using the todolist plugin for all features added to the layer. The mapper will then run the validation process to identify overlapping, fix wrongly tagged or poorly traced building features. If any errors are identified, the mapper will work to resolve these issues before uploading them to OSM and moving to the next task.

Changeset Policy

Changesets will be quite small in size, so no issues are expected in this regard.

Revert Plan

In case of needing to retract any changes, the activity team will use the JOSM Revert Tool if/when needed as well as keep monitoring and tracking the mapping activity (Changesets made).This process will make it easier to identify errors that need to be reverted.

Conflation

If an existing building in OSM is no longer visible on the most recent imagery (as is the case in this activity because Maxar imagery is more up to date in the AOI), the building will be removed. The mappers will be required to apply proper validation checks of their tasks before uploading to OSM to ensure buildings and highways do not cross or overlap one another.

Quality Assurance

To ensure high data quality, validation of the import tasks will be carried out by a second user in the Tasking Manager. Two Senior Mappers will be dedicated to validating and cross-checking the imported data through OSMcha and the Tasking Manager to ensure data quality during the mapping process.