Tasking Manager/Validating data

Validation is an important part of ensuring that contributions to OpenStreetMap (OSM) through a Tasking Manager are high quality. Fundamentally it involves more experienced volunteers checking the contributions of other volunteers to ensure that the OSM data is complete, accurate and meets the OSM features and tagging requirements. This means that the creator of a Tasking Manager project can be confident that a fully validated project has all the data they require and mapped to a high standard.

Partially mapped and validated project on HOT's Tasking Manager

OSM has valuable resources including its data and the volunteers who spend time updating and improving it. The validation process attempts to ensure both these resources are maintained to a high standard: By (1) Spotting and directly fixing data issues (2) Giving feedback to volunteers to encourage them to improve their work and continue mapping.

A Tasking Manager splits areas that need to be mapped (called projects) into small tasks (see example on right). The tasks are first mapped by volunteers and marked complete (light blue). Next a more experienced mapper (validator) enters the same task to check the work that has been done, fix any issues and provide feedback. Once the validator is happy with the task, they can mark it validated (green) or they can send it back for more mapping (yellow). The project creator will be looking to see that their project is 100% mapped and 100% validated (completely green) before using the data from the project area.

Why, Who, When and What do we validate?

Why do we validate?

OSM is often the most complete spatial dataset, but its data quality can vary. It is crucial that map users have confidence in the quality of OSM so it is important to ensure that there is a system in place to check contributions from volunteers. As with any crowdsourcing project, new contributors are constantly getting involved. New mappers have to find their way and may make errors as they progress to being experienced mappers. These errors must be corrected and new mappers need guidance so that they do not continue to make the same errors and they are encouragement to keep mapping. 

Who can validate?

This depends on the Tasking Manager project. Validation restrictions are set by the project creator. The 'lowest bar' that a project creator is advised to set is that anyone with intermediate or advanced level can validate (which means they need at least 250 saves or changesets in OSM). However, ideally a project creator restricts validation permissions to only specified teams of very experienced contributors.

On HOT's Tasking Manager HOT manages two main validation teams at present. Validator Trainees for those starting out and HOT Global Validators for the most advanced validators. These teams have been given permission to validate a wide range of projects, both HOT projects and those from other organizations that are on HOT's Tasking Manager. Other organizations also manage their own teams of validators, for example Médecins Sans Frontières manage the MSF Validators team. The join methods for a team vary, but typically a user can visit the team page and request to join, which will then be considered by the team manager(s). The HOT Global Validators team has a join request form.

Irrespective of experience level, a contributor is blocked from validating the same task that they marked 'mapped'.

What types of people validate?

All kinds of people validate, however, for HOT Disaster Mapping we have noticed they typically fall into one of these categories:

Although originally written for the HOT Disaster 'Activators', it holds true across all types of mapping efforts.

When do we validate?

The aim is to have 'Stage 1' validating taking place at the same time as the initial mapping, so that any amendments can be made while the nature of the terrain, imagery, and features is still fresh in everyone's mind. Prompt validation is likely to lead to the overall task being completed quicker and any problems can be quickly dealt with. Feedback should be as soon as possible to help and be presented in an encouraging manner. The aim is to prevent the same problem occurring again. If the task has been mapped properly then Stage 1 validating of a task will take considerably less time than the initial mapping did.

Shouldn't we all be doing all of the things listed in Stage 1 validating when we do the initial mapping?

In an ideal mapping scenario the contributor follows the instructions and maps the the task perfectly. However this is not always the case. If you are validating and find some mistakes it is best to provide feedback to the mapper using the comment section in the Tasking Manager. If a mapper has made mistakes while tagging or squaring buildings they should be directed to the following material:

The following link provides more information on Tagging Buildings: https://wiki.openstreetmap.org/wiki/Key:building

The following link provides more information on Squaring Buildings (with JOSM): https://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data/Buildings_feedback or when mapping with iDeditor: https://learnosm.org/en/hot-tips/tracing-rectangular-buildings/

What do we validate?

Validators read project instructions very carefully to establish exactly what a project requires and often read any wiki pages with more information about the area and tags.

• Validators often add the JOSM Style 'HOT-OSM-Validation', which changes the colours of the tagged highways to indicate if they comply with Highway Tag Africa. Instructions on how to load it are at JOSM Style wiki . This makes validating much easier - if you need to see more details & less of the 'Data Layer' in for instance residential areas, remember to toggle backwards & forwards into wireframe mode with Ctrl+w.
• It is recommended to use JOSM for validation. One of the common problems in a task is highways that end prematurely or do not have a node where they cross over one another. This means that routing software, that is, "find me the way from A to B" cannot be implemented as it does not recognize that two highways are connected if there is a half inch gap between them. JOSM validation will detect them. It is recommended that crossing ways, duplicates, overlapping ways, unclosed ways, and untagged ways are checked with JOSM validation.

For detecting duplicate buildings, run the validation tool in JOSM and duplicated buildings, node, ways and other features will be listed as a folder and fixing them has been made easier by just clicking on fix button within the validation panel for them to be auto deleted.

OR

You could also choose to have a look up on all duplicated nodes by running the validation tool and then add the folder with duplicated features into the to-do list and fix each feature independently.

Selects duplicate, or near duplicate, area buildings in JOSM's active data layer. A "near duplicate" is a building whose bounding box overlaps another building's bounding box by more than 50% (I couldn't get a pure area compare to work).  Only the first building encountered of an overlapping pair is selected. The selected buildings are added to the current selection. Note clear the selection first then use the todo plugin to go through the duplicated buildings.

Stage 1. At neighborhood/TM task scale

This process is described for JOSM users, which includes a 'validator'. It is possible to validate using other applications and tools – if you are an expert in these please add details below.

• Using the Tasking Manager (TM), load a square into JOSM.

Initial Scan

Have an initial quick scan around to see if you will be continuing or will be invalidating the square. If everything is fine then click on Yes button and then Submit task. If it is obvious that major improvements need to be made, click on the No button to invalidate the task and +Comment to add a comment of what needs to be improved plus the Submit task. The most frequent problem is a simple one that anyone can make, and the square has somehow been marked as complete, when it is obvious that the whole or majority of the square is still needs to be mapped – this type of error does not really need a message to the mapper, and may even be a software glitch (That's what we all claim). Do not invalidate a square for something that is not a requirement of the task. You may still invalidate a task later if it is later found that the requirements are not met. With JOSM when scrolling through the tile <Crtl><Down arrow>  is very effective. A large high quality screen on a desktop shows much more detail than a fourteen inch laptop. Typically you will find that zooming to one level less than you would when mapping works. In urgent cases zoom in as normal using - and + buttons.

• If your initial inspection is okay, then Run the Validator and correct everything that can be corrected. These are often things like crossing buildings or ways, but there are also other more complex errors such as an entry (e.g., highway or waterway) that has been carried over from another tile. Note right clicking on the error in the validation tab allows you to zoom to the error.

Buildings

• Now check if buildings are missing. Not all buildings are rectangular - in this image we can see some round housing units or huts (tagged as 'building=hut'). If there are a large number of entities that should be mapped it can be invalidated. If there are only a few then it is simpler and faster for the validator to make these corrections while examining the tile. With the density of habitations in some areas it is quite common that buildings may be missed.
• Are the buildings correctly traced? Sometimes mappers do not trace buildings accurately and if this is significant then it needs to be corrected.
• The most frequent errors are buildings that have not been squared. In this case it is best to send a message explaining how buildings should be squared.
• Note: If you find one building that was not squared you should search for more. This can be done in JOSM by searching "building nodes:-7". Then select the mapper and press "q". You should leave a comment advising the mapper that their buildings were not squared.  Information on how to square a building for JOSM and iD is often included in the Tasking Manager where the project is listed or under the Instructions tab. The JOSM building_tool plugin is useful for adding missing buildings. You can also use this technique for moving buildings that are out of alignment. Select them and then move them all at once.

Paste the following into a message about Buildings for the mapper concerned: Help here https://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data/Buildings_feedback

• OSM_Tasking_Manager/Validating_data/Buildings_feedback

Highways

• Are highway tags correct? Some mappers tag every highway as a track because they are not paved. This is not a meaningful criteria because 99% of the roads are unpaved in developing countries. Check the wiki definitions and the Highway_Tag_Africa before tagging any highways.
• Should it be a path or an unclassified? Be careful about subjective interpretations. This depends on the type of vehicle (is it a four by four?) that will be used to access it as well as the expertise of the driver. Leave it as it is unless you can clearly identify two parallel tracks and that one is definitely wide enough for a four by four vehicle. Tracks are different therefore you must inspect closely and change the tag if necessary. The JOSM To Do list plugin is invaluable especially when you are faced with fifty tracks that need to be individually checked to see if they should be unclassified etc. You can use this plug-in to make notes to your self on the map as you review the area. Remember that the season can affect what is seen in the image. If its recently rained then the highways will be narrower.
• Should a highway be unclassified or unclassified and surface unpaved?  There is much debate about this issue. From a satellite image you cannot tell if it is paved with a layer of dust over the top or unpaved. This can only be accomplished in person by sweeping the highway with a brush and dustpan.  When I map I often will use surface=unpaved but when I am validating I will probably leave it as the previous mapper tagged it.
• Road geometry. Some mappers do not put enough details and others too much (for example, a node every 10 or 20 m even if the road is straight). The first case is quickly corrected with the (magical) Improve Way Accuracy mode in JOSM. The second case, when the extra nodes actually make a straight road weave requires deleting the extra nodes.
• Start/End of Roads. Some mappers produce long winding ways which meander through several junctions. Click on the 'way' and break it into more obvious parts, probably at junctions or changes in the tagging. Some other streets or roads are broken up without any reason (same tags for all the sections), and need to be combined.
• See Stage 2: Town or city scale validation regarding further problems with highways, some of which may be visible within the single square you are validating.Other highways will cross over more than one tile.

General issues

• Connections between objects. There are some entities that should be connected and those that should not. Frequent problems include residential boundaries being connected to highway or building nodes. This can be determined by using the JOSM validator plug-in.
• Want to know when the object was mapped and who mapped it?  Highlight the object in JOSM. Look under the view menu for advanced info or use <crtl>i under edit count or select the object , click on Windows and add the Authors panel on the right side.  At the top you will see the mapper’s name.  If they have thousands of edits to their credit and you have only a few you should double check your own interpretation.  The https://wiki.openstreetmap.org/wiki/Map_Features page provides useful feedback in this area.
• If you see a zebra crossing on a path in Africa just delete (zap) the tag. The iD editor seems to add these on occasion. It is a type of pedestrian crossing that is probably only found in the United Kingdom and is not related to actual zebras.
• If there are two landuse=areas that overlap select them both and use the <shift>j keys to join them.  This happens on a tile boundary where two mappers have drawn the line down the edge of the tile.
• Changeset comment: While you are reviewing the tile for validation you may notice that the mapper has not included the changeset comment, in which case send this link to them through the OSM messenger system https://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data/Changeset_comment_missing. It provides instructions on how and why to use the changeset comment feature.

• When you are satisfied your loaded square is completed, upload any changes and press the 'validate' button on the TM. Add a comment in the validation box that acknowledges the mappers' efforts, provides constructive feedback and if necessary links to information sources and any other relevant comments. For example, the following comment "All complete, thanks. Couple of minor errors with crossing buildings highlighted by the validator programme & I've fixed them" provides feedback and a thank you. You may also send a message directly to the mapper concerned but it may be better if you first review any other squares they have mapped. To send a message directly just type their username like this: @Gollum (or if username has spaces: @[Bilbo Baggins]) in the comment box in the Tasking Manager. You will find that many of the mappers are new to OSM especially during a humanitarian crisis. It is worth letting them know that their work was satisfactory and it would be appreciated if they did some more.

Stage 2: Town or city scale

• Load as many of the tiles into JOSM as you are able, covering the area of the tasking manager project. You may have to do a larger town or city in stages. If a project is not complete yet, you might do this just over areas which are fully validated and green.
• Run the JOSM Validator and make corrections where possible. If the Stage 1 validators have being making the correct entries this process should not take long.
• As you are now looking at a much larger area it should be apparent which sections of highway should be joined together and tagged as Primary, Secondary and Tertiary, for example. They are often larger and frame a larger area or can be a parallel way to main roads. It is also important to check where they start and when they stop, which is often not possible to do when you are looking at individual squares in the Tasking Manager as the highways run across more than one tile.

Missing parts of Highways

The following image looks unusual on Mapnik. If you check this the imagery confirms the two sides of the road separated by a drain are not finished.

Road continuity

In the image below we would expect the two unclassified roads to be connected and not joined by a simple path. The imagery confirms that the southern highway looks the same, and should be tagged the same. In this example it shows that in the farther south section the highway changes to tertiary. This may not be apparent in the imagery. It should be cut when it becomes a straight road.

Isolated sections of Highways

Highways that can be driven upon cannot be isolated and connected to the drivable road network by paths. Instead they must be connected to it by driveable highways.

Tagging coherence

In this example the primary and tertiary road are connected by an unclassified road. And the same thing is occurring between the two tertiary roads. After having checked the imagery this road would deserve to be tagged as tertiary. This requires that you check the streets width to identify the main ones that needs not to be tagged as residential but unclassified or even tertiary.

Useful Tools for Validating

Multiple-Task Validation

Selecting all tasks by a beginner mapper

• In Tasking Manager version 3 several tools were introduced to allow validators to select multiple tasks to validate at once
• Note: multi-task validation should be used very carefully and should not replace or be used as an excuse to verify every task
• In version 4 of the Tasking manager, the following multi-task validation functions are available:
  • Multiple selection by mouse: in the contribution tab, you can click multiple tasks and select them for validation
  • Selecting by user: in the contributions tab, select all the tasks mapped by a certain user, see image for example
  • Note that you can also filter by experience level or new users of the Tasking Manager
  • Visit LearnOSM Module: Tasking manager Mapper's Guide - Validation For more information

External Tools

Main list is at Quality_Assurance

• Overpass_turbo
• Osmose Web browser based. Whole world coverage. Multilingual
• Keep_right Web browser based. Whole world coverage. Multilingual
• That_Shouldnt_Be_Possible Web browser based. Requires a gps trace of the area / route you wish to check. At present only limited parts of the world are covered.
• Visualizing OSM data with cartodb to aid HOTOSM validation. Please see this blogpost

Messages to Mappers

Should you comment every time someone does something wrong?  One mapper quite often misses a settlement, however they’ve mapped over a thousand tiles.  Most of the time I just add the settlements and do not send comments to the mapper who missed this. 

If the entries are more than a month old you should just correct, validate and do not bother to comment.  Many mapathon mappers only map once and even if they are still mapping another validator may have commented and they may be mapping correctly now.  Multiple messages for something they did a few times months ago lessens the fun.

You don't have to send a message to the mapper involved, but if you are going to, this section may help. If your message concerns a problem with the mapping, see the links which are being developed for adding to your message - Problems with Buildings is the first.

Of course, you may send your own message to any mapper, Johnwhelan has been known to use:

For a mapper who has omitted fifty settlements, "added fifty settlements" and validate it, it gets the message to be more careful across.

For a mapper who has carefully filled the tile with every path and track, or has left settlements with no connections. "We're interested in where people live and how to get there so tracks leading to fields are of less interest." Link to African Highway wiki.

For mappers who leave ways or areas untagged, "tagged twenty buildings / highways." and validate it, it gets the message to be more careful across.

For a mapper who has tagged every highway as a road. "JOSM validation has identified twenty three highway=road as a temporary highway type. highway=road is historical from when the main source of input was a GPS track. These were converted to highways but could be either a motorway or a highway=footway with steps and needs closer inspection to add the correct tag. As a result many routing systems will ignore them. In this area highway=unclassified is the most common highway type."

For a mapper who tags highways between settlements as highway=pedestrian in rural areas. "The African Highway wiki recommends the use of highway=path. These are often used by small motorcycles."

"JOSM validation detected seventeen crossing highways and eight highways that almost met. This confuses the routing software and has been corrected."

Tallguy is much more tactful.

This is a template of a message being sent by Tallguy which may give you some ideas about what you could include.

"Hi & thank you for helping the Humanitarian OpenStreetMap Team (HOT).

I am an experienced mapper (still learning though!), and have looked through one of the squares you have completed for task from the OSM Tasking manager. You got it right, thanks!

If you are a regular contributor to HOT then keep up the good work.

If this is one of your first contributions, then thank you & please keep up the good work. There are many tasks outstanding in http://tasks.hotosm.org/ , all are for very good causes & it would be great if you could keep contributing.

You could also help HOT with more than just mapping. Here are a few ideas; Validating – there's a section on this in https://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager Perhaps if we each validated 3 squares and mapped one? We could learn a lot from others & help to raise the standard of mapping throughout. Sometimes you map a square & think “Why doesn't that highway come into my square from that square – when you've finished your square take a moment to have a look – perhaps it needs a lot more work & shouldn’t be left showing as complete? You could also get involved in Stage 2 validation?

Take a moment to read http://learnosm.org/en/ which contains sections on advanced mapping. After 5 years of mapping I read it & still learnt quite a lot,

Help with translating; http://learnosm.org/en/ is multilingual (English | Bahasa Indonesia | 日本語 | Português | Italiano | Français | Deutsch | Español ) but needs more translated especially on the advanced sections for JOSM, https://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager (either edit the original, or add to Talk section, or send to Tallguy for adding in – good screenshots are also a help),

Software writers. Could you help the team with https://github.com/hotosm/osm-tasking-manager

https://hot.openstreetmap.org/ and https://wiki.openstreetmap.org/wiki/Humanitarian_OSM_Team will tell you more about HOT. You can keep up to date about HOT, or particular areas by subscribing to the relevant lists at https://lists.openstreetmap.org/listinfo . There is also irc://irc.oftc.net #hot for a link to others mapping for HOT, and the page https://wiki.openstreetmap.org/wiki/Irc lists all of the IRC channels if you have an interest in a particular area.

Keep up the good work & I hope to see more squares completed by you in the future.

Regards

Tallguy"

Other Validation material

HOT validation Webinar Training - March 2019 - https://youtu.be/ee3XkQ2VwaY

Missing Maps validation page - https://www.missingmaps.org/validate/

HOT slide deck - Validation using JOSM

HOT TM validator checker