Canada/Building Canada 2020

From OpenStreetMap Wiki
Jump to navigation Jump to search
Logo of Building Canada 2020

NEWS, March 2, 2019

Statistics Canada released the Open Database of Buildings (ODB), version 2.0 to celebrate International Open Data Day.

Additionally, Microsoft ran a computer vision algorithm that outlines buildings (developed with deep neural networks and version 1.0 of the ODB) on satellite imagery of all of Canada and released the result with an open data license (ODbL). See Microsoft's GitHub Repository for details and the data. The results can also be viewed in Rapid.

NEWS, January 2019

OSM's Canada_Building_Import has been updated with specific steps to take with our Import process, in conjunction with OSM Canada's Tasking Manager. Please examine these data and steps now: Ontario, Québec, Alberta, British Columbia, New Brunswick and Nova Scotia are available! As these steps get fine-tuned (first draft was January 25) and consensus emerges that there are well-qualified OSM mappers to enter into OSM high-quality data, we will lift the "On Hold" or "Stopped" status on the Tasking Manager one province at a time. Ask around, post here by clicking on the Discussion tab (start a new section if you must, or indent with a: to add to a thread) and be a part of OSM's community of adding buildings in Canada!

Imports (this one included) are done by intermediate and advanced OSM volunteers; JOSM is the recommended editor and the steps can be technical, detailed and require a certain judgement of quality. If you are up for it, take a look at the documentation, data, ask questions, build community, roll up your sleeves and help us map from these federal data, vetted to be ODbL-compatible (OSM's data licensing agreement). Thank you!

NEWS, November 2018

November 1, 2018 - Statistics Canada released the Open Database of Buildings (ODB), a collection of open data on buildings, primarily building footprints, and is made available under the Open Government License - Canada which has been deemed compatible with OpenStreetMap by the Legal Working Group. The ODB brings together 61 datasets originating from various government sources of open data. The database aims to enhance access to a harmonized collection of building footprints across Canada. The current version of the database (version 1.0) contains approximately 4.3 million records and includes provinces and territories where open building footprints were found during the collection period from January to August 2018. The coverage of the ODB is expected to increase as more government sources of building information become available under an open data license and are integrated into the database.


How to assist.

If you are an experienced JOSM user please help us import the building outlines following the Canada_Building_Import using a dedicated import account.


If you are are not an experienced JOSM user or even if you are you can usefully contribute to the project by enriching the tags on the building outlines.


The tags of most interest are the type of building so building=detached, semi, commercial, terrace, hotel, school and others mentioned in Map Features

The house number if known is useful but only with the street as well. Postcode is useful. The city can be derived from the city boundaries and does not need to be added to each building.

The number of levels is of interest so a two storey home would be building:levels=2

Streetcomplete is a useful mobile app that can be used to fill in gaps. https://www.openstreetmap.org will take you to the iD editor if you click on edit.

Teachers may find the following link of use

http://teachosm.org/en/

The formal guidelines for organised edits are here:

https://wiki.osmfoundation.org/w/images/1/13/Organised_editing_guidelines_version_20180908.pdf


OSM from a municipal point of view:

WikiProject Canada/Building Canada 2020/municipalities

The Idea

Building Canada 2020 initiative (BC2020i) is an OpenStreetMap (OSM) WikiProject. It is a community-led initiative that has a vision to map in OSM all buildings in Canada by the year 2020. This vision emerged from a combination of factors: the collaborations and discussions triggered from a related (now defunct) crowdsourcing project by Statistics Canada, the growing need to improve existing georeferenced data on buildings across all communities in Canada, and a genuine opportunity to explore new forms in co-production of open data. Creating a freely accessible, non-proprietary source of nationwide information on buildings will contribute to the development of new data infrastructures and workflows of the future, upon which a multitude of public and private projects could thrive.

Simple information on buildings (e.g. WGS84-based geolocation, building footprint, full address including street # and street name, postal code, city, province and "Type Of Use") is of major societal value. This information, however, is often lacking in some areas or is not completely accessible on a single open data platform. Creating a nationwide, freely accessible and non-proprietary source of building information will contribute to development of the data infrastructures of the future, upon which a multitude of public and private projects could thrive.

This WikiProject came as an outgrowth from the pilot project done in Ottawa/Gatineau. There, the building outlines were imported from the City of Ottawa under their Open Data license. Then, the missing buildings were added and tags were added to the buildings in OSM. The value of combining Open Data to ensure completeness and accurately map the buildings with local knowledge yields data resources useful for many purposes. Although the project title says Canada the same techniques can be used anywhere in the world. In Africa, mapping buildings accurately has been used to estimate population: how many schools are needed, etc. Note the Ottawa pilot specifically made an effort to incorporate tags useful for disabled persons.

Why a community-led initiative?

Building Canada 2020 concept

Many communities might be mobilized to achieve this vision; the figure at right sketches six, more might be identified. By joining OSM, we share a space where these perspectives intersect and resources are available: dialogue can occur, activities can be coordinated and goals can be achieved with efficiency in a single, open database.

As a community-led initiative, Building Canada 2020 can foster collaboration between a multitude of stakeholders (civic groups, private sector, academia, public sector...) to achieve a specific common goal benefiting each‎ stakeholder, as well as society as a whole. This approach is inspired by principles of civic science, more specifically, civic data. A community-led approach enables stakeholders sharing a common vision ‎to share ownership and accountability for its realization as well as the creation of its output: open, accurate and complete data on all buildings in Canada, accessible through OSM.

Launch of the initiative

A workshop was held on September 15, 2017, at Statistics Canada head office in Ottawa. Participants were invited to provide comments and suggestions about how this vision could benefit their organization or activity, the challenges they foresee, additional stakeholders that might be engaged, possible tools that could be used, possible forms of governance that could be adopted, milestones to monitor progress, etc. Ideas and perspectives from many different communities were brought together to enrich the vision and help ensure its sustainability. The workshop was attended by 52 people from 25 organizations intending to:

  • Ascertain whether there was enough consensus among stakeholders to determine if it is possible and beneficial to move forward with this vision within the context of OSM.
  • Outline a "high-level roadmap" that might be used to turn the vision into a reality that will benefit all.

Note that many of participants were interested in what could the data be used for rather than how to get the information into OSM.

Known benefits

For the Canadian economy and society, buildings are a major capital asset, the space in which a large part of economic and social activities are concentrated, an essential element of human safety and security, physical infrastructures that shape our relationship with the natural environment and contribute to energy efficiency and consumption, and are landmarks of cultural and historical identity. For these reasons, there is a constellation of information domains that relate to buildings, at municipal, provincial and federal levels.

As of today, however, there is no comprehensive, open database on buildings in Canada. Such an open database could have a multitude of uses by a multitude of stakeholders.

Governance / Project Management

This project will be managed like all other OSM projects in Canada: by the local community using this wiki, the talk-ca mailing list and the OSM Canada Task Manager to coordinate various efforts. If there are lots of volunteers and/or funding becomes available to do additional things, more elaborate coordination (a project steering committee, ongoing project management...) can be set up.

This wiki (the document you are reading) helps guide BC2020's development, consensus and progress. We make progress by implementing various sub-projects and activities at a "more local level." Each of these "more local" sub-projects and activities has its own governance structure, depending on the organization or entity that implements it.

Communication and coordination

Building Canada 2020 is an open and inclusive initiative. Anybody sharing its vision is invited to be part of it and connect with individuals or organizations who contribute to this vision. The following communication and coordination tools are used:

  • OSM community at large: this OSM Wiki page
  • OSM community in Canada: done through Talk-Ca
  • Tactical how-to questions on Slack, see: https://www.osmcanada.ca/about.html (join the #buildings2020 channel)
  • Tasks in a specific community: OSM Canada Task Manager (ask on talk-ca or the slack channel if you would like a new task to be created)

What is the end goal?

At the launch of the initiative, no precise count of the total number of buildings in Canada was known. We could only estimate: using the benchmark ratio of 2 to 3 people per building, the total count of buildings in Canada might range between 10 and 15 million. As of September 2017, there were approximately 2 million ways in Canada tagged building=* in OpenStreetMap. As of February 2019, there were nearly 4 million buildings. With the new release of openly licensed databases (Statistics Canada and Microsoft, March 2019), we now know that there are approximately 13 million building footprints.

The data that could be mapped

Target data types (specific tags) for this initiative might include:

  • Building locations (OSM nodes) or footprints (OSM ways or relations), tagged building=* with their correct "Type Of Use" (see below),
  • Address and postcode of building, using the addr=* namespace,
  • Height of building in meters, tagged height=*,
  • Number of floors (called "levels" in OSM), using building:levels=*, max_level=*, min_level=* and non_existent_levels=*,
  • Name of building, if there is one, tagged name=*,
  • Year of construction, tagged start_date=*,
  • Entrance and/or exit location(s) of building, if they are known, as nodes tagged entrance=*,
  • Any amenities the building may provide, as nodes tagged amenity=*,
  • Other building attributes (e.g., web page links about well-known, landmark and famous buildings using the website=* tag, potentially more).

OSM key building=* currently has over 60 documented values ("Type Of Use"), including building=residential, building=commercial, technical infrastructure, whether private or public with building=industrial and/or man_made=*, educational buildings at schools, colleges and universities with a variety of values, civic and cultural buildings, etc. The man_made=* key now has over 50 documented values, some of which are not exactly buildings (adit, flagpole, snow_net, water_tap) and others which are (gasometer, observatory, silo, windmill). Amenity (Map Features#Amenity) is another useful tag. Please correctly and appropriately apply these key-value pairs as OSM documents them!

How to get buildings into OSM?

We use OSM as a data repository, so OSM tenets must be followed. If you are unfamiliar with OSM here is some starting documentation:

High level techniques

There are several methods to add buildings to OSM. And for any one area of country it is likely the various methods will need to be combined to generate a comprehensive set of building and building attributes. Some locations have existing data in other sources, some none. Some existing data have some subset of interesting attributes, others may only have the building outline. These methods require different data sources and sometimes newly-developing tools. Several approaches to entering data into OSM are discussed below.

Map it based on satellite images

A first and frequent method is by "manually" mapping building outlines. It is recommended you use JOSM and the building_Tool plugin for this. The more-beginner-friendly iD editor can be used, but it can be difficult for new mappers to accurately map buildings using iD. However, JOSM has a steeper learning curve than iD. If you are organising a group of new mappers, the OSM Canada Task Manager tiling system is recommended. It incorporates a facility for validation where someone else checks recently-entered work.

(see an example of how this was done in Nepal)

Import from Open Data sources

A second method is a formal Import of municipal and provincial data (often Open Data). As Imports are not the usual method by which data enter OSM, Imports require following OSM's Import procedures. Open Data imports need expert attention to ensure the data are high quality and meet OSM's standards for imported data. Data arising from automated image processing (e.g. Lidar data) to map buildings is currently frowned upon. This may change as potential imports move forward successfully following OSM's Import process (the steps of which should become incorporated as part of this wiki). Part of this is assuring that the licence for the data are aligned with OSM's licence (ODbL). Part is a technical alignment of the data about the buildings (which may include their height, number of floors, etc.) with OSM's data tags in a matching, harmonious, mathematical/logical mapping. (See "The data that could be mapped" above).

Local Knowledge

A third method is of greater importance and depends on local knowledge: what additional data can you add to a building which already exists in OSM? (See "The data that could be mapped" above, especially notes on the building=* and man_made=* tags). Certainly the address Map Features#Addresses, its use and other attributes. And Map Features contains other important attributes that might be added. For example, tags such as cafe and wlan are helpful to thirsty mappers wanting to check their email. There are many different (smartphone-based) apps and web-based approaches to enter into OSM these additional data, such as streetcomplete (an Android App) and Field Papers, but these involve some post-processing. Vespucci is an Android-based full editor and OSMand can be used to add POI (Point Of Interest) information. There are many other similar methods to add minor attribute data to buildings in OSM, please list your favorites here!

Details on importing

Note for properly licensed imports: A very important phase of any data importation in OSM is the logical mapping of existing data to be imported "onto" the correct OSM key-value pair tags. For example, a building dataset of an area with both a university and a city might distinguish between residential buildings which appear similar in the real world, but some are known to be dormitories (best tagged building=dormitory) while others are apartments (best tagged building=apartments). Careful consideration must preclude any effort or script to translate these data before they enter OSM. It is NOT satisfactory to simply import data in a quick, sloppy or unconsidered method hoping to then "improve them in place" with subsequent OSM edits: we must do our best to first get them right! Of course, if (building) data already exist in OSM in error, or updates are needed, such improvements are welcome.

Some building data to be imported into OSM will be sparse (perhaps only footprint and/or address data) and some will be rich (including all of the above-listed attributes and perhaps even more). The best practice is to logically map the source data onto OSM's key-value pair tags as carefully and accurately as possible. Specifying an exact list of this logical mapping from source data to target OSM tags is a critical component of any local Import Plan.

More target data types may be added to the list above. During this WikiProject's presently "open" phase, you are encouraged to add additional target data you believe might be useful. At some point, we will "close" our target data specification and no longer accept additional submissions of data types to consider. Note there is no restriction within OSM on keys or values.

Implementation tools and workflows

Import plans: examples, tools, best practices

Each data import needs to file an Import Plan with the Imports mail list within OSM.

Here are some examples, the first of which is local to this project:

Ottawa Building Import video tutorial
  • Ottawa Import Plan provides extensive documentation on how to implement a bulk import of building data from municipal open data source. This documentation includes a video tutorial, which explains the import process from A to Z (see link to video).
Getting organized: OSM Tasking Manager

WikiProject Building Canada 2020 is a massive undertaking of coordination. It benefits from organization and structure of the tasks being completed at any given time. The OSM Canada Task Manager (sometimes abbreviated TM or OSM-TM or OSM-CA-TM) is one organization tool that can be used to coordinate work.

Currently (2018-Q1), out of 42 tasks there, 33 have the word "building" in them, you can search for this to filter tasks.

Two OSM Tasking Manager projects are active around Ottawa: http://tasks.osmcanada.ca/project/100 and http://tasks.osmcanada.ca/project/114.

Inventory of Current Building Data Sets

We are creating an inventory of existing building related datasets from municipal and provincial open data sources as well as other sources. The tables and the link below document these datasets in Canada -- openly licensed, proprietary, ODbL compatible for inclusion in OSM, or not. As some of these data source's licenses remain in a state of flux and/or negotiation with OSM's Legal Working Group (status should be indicated in the table), please pay particular attention to this before entering these data into OSM.
In addition, Canada's "Open Government" Programs website has a table of municipal open data portals that can be reviewed for building data sets: http://open.canada.ca/en/maps/open-data-canada. 

* See tables of municipal and provincial open data for possible import into OSM.

Creating building polygons with machine learning techniques

This is a new and emerging area of work that has not yet been accepted by OSM due to data quality issues. It appears particularly promising with areas in Canada that are not covered by municipal open data, or to maintain OSM as "up to date." Here are some resources:

Community mapping projects and mapping parties: tools and best practices

This section provides information on how to implement community Mapping_projects and/or Mapping_parties, aimed at adding building information to OSM. Find links to existing toolkits, best practices and lessons learned. There are examples from Canada and examples and inspiration from around the world. Some of the tools in community mapping activities include:

CODAP - The Crowdsourced Open Data Acquisition Platform - beta version Access a beta version of CODAP here. CODAP is distributed under the MIT Licence. Code files of CODAP are available online here.
Tools and toolkits: examples and inspiration

Mapathons: tools and best practices

A Mapathon is a coordinated mapping event, often held indoors (sometimes known as "armchair mapping") but it can also be an outdoor or combined activity. Mapathons should OSM tutors who are happy to share their knowledge, sometimes have GPS units available to loan for the duration. If you want to run one note that it is common to offer snacks and refreshments. Using new mappers can offer challenges for data quality. If you are running one have some mice available to help the accuracy.

Building Canada 2020 can be a theme during many sorts of related events (see: Events).

Stakeholders of Building Canada 2020 can work towards the development of a set of "How to" toolkits, to help various organizations and educational institutions to plan and implement successful Mapathons across Canada.

Some general tools

Do-it-Yourself (DIY) Open Data Toolkit (Treasury Board of Canada Secretariat and OpenNorth).

An instructional manual that provides a step-by-step guide on how to develop your open data initiative. It brings together training materials, best practices, tools and resources to help you prepare for and implement an open data project.

How to Monitor progress?

This WikiProject will greatly benefit by better methods to display/demonstrate regular progress towards the goal of mapping all the buildings. It might be good to break down the status by CSD.

Active Monitoring tools

The following projects are actively monitoring the state of the buildings in Canada:

OSM Canada Task Manager is used to measure progress for many building projects in Canada: in fact, over 75% of the tasks there have "building" in their title. However, a more tightly-coupled method of the "% complete" reported there and some mechanism to "capture and report status" (on a province- or city-at-a-time basis), BC2020-wide, here or in another wiki seems a useful milestone to achieve.

[Keep adding to this list here...]

Background

If you are looking for statistics and the location of a municipality you can search Census Profiles 2016. Here are some project-specific tables:

The following tools can be used to generate insights on the mapping of buildings across Canada.

Major Obstacles to Solve

The following obstacles make achieving the goal difficult.

Getting local participation

Local buy-in is needed both for support for imports and to add tags to existing buildings. Ottawa demonstrated the importance of this.

Licence compatibility

For data imports, the compatibility of municipal/provincial open data licences with OSM's licence (ODbL, Contributor Terms) needs to be verified. As it appears for documentation listed in this page, Canadian municipalities have generally used a limited set of licences types (OGL 2.0; OGL 1.0; OGL-BC 2.0, etc). What is believed to be a fairly current licence status page is under construction.
Important note: data for a number of major municipalities have already been entered into OSM; of course, their licence was cleared and accepted as compatible with OSM's ODbL. See the documentation on OSM Wiki page "Contributors".

This Building Canada 2020 OSM WikiProject is expected to facilitate a discussion of OSM's ODbL license compatibility on two fronts. First, with municipalities so that they will adopt a standard licence (a lot of work has been done on this already). (HOW? Document that here, please!) Second, the OSMF Licensing Working Group will be engaged in this discussion. (HOW? WHEN?) A necessary component of this WikiProject is to streamline a process of verification of these licences, documenting the different open data licences used by Canadian municipalities and clarify issues of compatibility with ODbL for each type of licence.

Background documentation on previous cases

Ottawa import plan. The OSMF Licensing Working group (LWG sometimes known as the Legal Working Group) determined in their meeting on 2017-03-02 that data under the Ottawa Open Data, Licence Version 2.0 (Ottawa ODL 2.0) can be included in the OpenStreetMap dataset and distributed on ODbL 1.0 terms. Quoting from the draft minutes (draft link may not be generally readable):

The LWG has determined [(1)] that the attribution requirements of the Ottawa ODL 2.0 can be met by adding the required text to the wiki contributor page and corresponding changeset source attribute values, and that there is no downstream attribution requirement, [(2)] that we are not using "Personal Information" as defined in the licence and referenced legislation, and that so licensed material can be included in the OpenStreetMap dataset and distributed on ODbL 1.0 terms.

One of the difficulties to keep in mind is that OSM's LWG has limited resources to examine many licences: to approve "city after city after city..." is an unrealistic use of their time. A major goal of this project is to harmonize OD licenses in Canada so that as they might be used to enter building data into OSM, the LWG can be presented with a SINGLE licence (or very small number of licenses) to approve as harmonious with OSM's ODbL. This work is ongoing, progress should be documented here.

Postal Codes

Postal Code data are not open data in Canada.

Funding

At this time, nobody is dedicated to this project, and it will only go as fast as there are people to do work on it. Organizations who stand to benefit from realizing the vision and achieving the goals of this project are encouraged to provide funding to allow people to spend more time on this project than they otherwise would. Currently (2018-Q!) the project would benefit from a Project Manager who can coordinate the many moving parts using "methods familiar to OSM." These include writing/updating this wiki to inform new and existing users and provide ongoing status as to what is going on in the project at all times, becoming a "one stop shop" to learn about and contribute to the project.

Tag Standardization / Data quality assurance

OSM building data can be assessed across the following (and more) dimensions: completeness, positional, temporal, semantic, attribute and shape/size accuracy as well as logical OSM tagging consistency. For example, some building data might include height=* or start_date=* (date built) tags, some may not, but the general OSM tagging guidelines (building tags, the Buildings wiki and the man_made=* tag) are being followed as best as possible so that data are consistent across the country. However, given the diversity that makes up Canada, 100% consistency throughout the country is a challenge. Canada-specific usage can be documented here as and when these challenges are identified.

Academic institutions involved in this initiative are welcome to provide leadership on research and best practices related to QA for building data in Canada.

Figure 1. OSM tagging logical consistency: example of residential building tags used in the Ottawa-Gatineau area. October 2017. (source: ITO Map)

Existing OSM tools show some of the QA challenges related to building data in Canada. Figure 1 is an example of OSM tagging logical consistency: in this case how consistently residential building polygons are tagged with existing tags.

Data maintenance and accessibility

As with OSM in general, the ongoing maintenance to keep buildings up-to-date must be discussed. A specific plan to update data and how that will occur in the future should be outlined here.

Stories, Examples and Inspiration

In order to provide motivation for volunteers to work on this project, it is important to understand success stories of how building data (or OSM in general) is used in Canada. OSM is used in multitude of ways: on the ground by humanitarian operators, online with amazing apps and data visualization tools.... Please add more success stories here.

  • The Canadian Red Cross and the Gatineau floods 2017. "During spring 2017 when Ottawa and Gatineau experienced severe flooding, the Canadian Red Cross used building information within OpenStreetMap to validate the number of homes impacted by the high-water levels. They also used these data to produce base maps of the affected area for planning and delivering assistance." (Read more ...)
  • Data visualization tool for building property assessment values (Vancouver, Edmonton). "Open data facilitates innovation. I couldn’t have created this visualization without that kind of access to data." says Eugene Chen, of Darkhorse Analytics, (read the full blog post)

Other Examples and Inspiration from in Canada

  • Happy Valley-Goose Bay. Here is an example of how a small municipality makes a smart use of open source technology and data (QGIS2web and OSM) to create an interactive map.
  • Interactive maps created by Vancouver-based data analytics company MountainMath, show the power of combining OSM and municipal open data to visualize building and housing statistics.
  • City of Edmonton, Building age map. Map created by Yourtruhome.com. in partnership with BCYEG, which found a volunteer to redeploy the Chicago Building Age map in Edmonton.
  • CBC/Radio-Canada. Montréal a 375 ans, mais quel âge ont ses bâtiments? Building Age map of Montreal.
  • Simple 3D buildings is a OSM wiki page that describes tags for basic 3D attributes of buildings. There is a list of amazing Demos (for example, see these F4map versions of Edmonton, Toronto and Halifax)
  • Vancouver Land Prices Heat Map. Heat Map is showing price per sq.ft for Vancouver parcels of land. All of the data displayed on the map are from the Open data Vancouver Catalogue (2014 BC Assessment data from tax reports and current parcels polygons).
  • Housing Price Maps (Montreal and other international cities). These maps show overall city-wide trends in residential real estate prices. The source code is also provided.

Other Examples and Inspiration from Around the world

Events and Activities

Please add any relevant ongoing activities and upcoming events.

Ongoing Activities

  • Statistics Canada Crowdsourcing Project: This project focuses on collaborating with local OSM groups and meeting with major municipalities to help them make open their data on building footprints (where available). Contact for this project: statcan.crowdsourceapprocheparticipative.statcan@canada.ca. The following municipalities are involved:
    • City of Ottawa
    • Ville de Gatineau

Upcoming events

Past events

Contacts for Groups and Organizations

If your group or organization is or wants to be involved in this project, but not listed, please add it!

OpenStreetMap groups

Not-For-Profit

  • Open North
  • Canadian Red Cross

Academics

  • McGill University - Geographic Information Centre
  • Carleton University
  • York University - Lassonde School of Engineering - Geomatics
  • University of Calgary - Department of Geography

Private Sector

  • MapBox
  • OpenConcept

Municipal

  • ...

Federal

  • Statistics Canada - Data Exploration and Integration Lab (DEIL): jean.lemullec-at-canada.ca; alessandro.alasia-at-canada.ca
  • Natural Resources Canada - Centre of Mapping and Earth Observation
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Canada/Building_Canada_2020&oldid=2751418"