Quality assurance

From OpenStreetMap Wiki
(Redirected from Monitoring Tools)
Jump to navigation Jump to search
Quality assurance - Other languages
See also: Accuracy and Completeness

Quality Assurance tools help lead to better quality of OSM data. Often, the tools achieve this by providing a list of bugs in the data, which mappers can then go and fix using editing tools. The bugs are either automatically detected based on rules and data analysis, or the tools provide a means of manually reporting them, or some combination of the two. There are many different ideas for how to do this, and what kind of bugs to focus on, hence many different tools.

It is important to understand that warnings are just warnings, they should be examined and fixed where possible but never at the cost of introducing wrong data. Some validators warnings are generated for valid data. Often in such cases validator code should be fixed, not OSM data. Sometimes false positives are unavoidable.

Bug reporting tools

These are tools that try to highlight and point out parts of the data that are likely wrong. There are also reports from manual quality checks by humans. In both cases, the reported issues may contain false positives and, when editing, care should be taken to ensure their accuracy. However, they can provide a very valuable visualisation of which areas need attention and can help spot and correct errors.

  • Notes↙
    Notes is a feature of the OpenStreetMap website. Error reports can be added by using the 'Add a note' button in the lower right corner.

Error detection tools

Error Detecting Tools check the OSM data for potential data errors, inaccuracy or sparsely mapped places. Users should check if these structures are really wrong (false positives usually occur and there are not really mapping rules which are set in stone) and correct the data for a continuously rising data quality.

Comparison of some of the following tools↙
Tool Coverage Error types Display mode Fix suggestion Downloadable API Correction guide Development Quality
JOSM/Validator Local (scan data downloaded into editor) Many List + map yes yes N/A For some problems Active The best OSM validator, very few false positives, false positives are quickly fixed
Osmose World Many (200+) Marker map yes yes yes yes Active
frontend backend 		Highlights many things that may be issues that are probably not. Osmose's authors say not to use as the sole source when editing.
OSM Inspector World/Partial Many Rendered map no yes N/A no Active
MapRoulette World/Partial Many (10+) One feature at time yes yes yes no Active Reports are user provided, range from excellent to actively harmful
BRouter Suspects World Car Routing List or Osmoscope no no no no Active
Nominatim QA World Addressing, places Marker map yes
?
?
?
 Active Excellent, but limited to issues impacting Nominatim
Keep Right World Many (50+) Marker map no yes yes German only Inactive (last commit 2017)
OSM NightWatch World Many (6): admin polygons & coastline List no no no no Active Perfect, 0 false positives, minutely updated
Public Transport Stop Analysis World Public transport Rendered map
?
?
?
?
 Active
  • Keep Right↙
    Keep Right (keepright.at) shows a large number of automatically detected potential errors on a map or in list form. It has a system for reporting false positives and for labelling a bug as fixed. It has rules to automatically detect the following error types: non-closed areas, dead-ended one-ways, almost junctions, deprecated tags, missing tags, bridges/tunnels without layer (careful - not always an error), motorways without ref, places of worship without religion, POIs without name, ways without nodes, floating islands, un-tagged railway crossings, wrongly-used railway crossing tag, objects with fixme=* tags, and relations without type.
    The bottom-right corner of the interface has links to data outputs, including GeoJSON.
    There is a documentation on how to solve keep-right-errors, currently only in German language: DE:Keep Right Users Guide though here is an auto-translation into English.
    Possible downsides: Keep Right shows tens of thousands of errors, including many that have little impact for most data uses (e.g. a stream intersecting a highway: technically some feature is needed here (a bridge, a culvert or a ford). Keep Right uses flawed assumptions about layers of bridges and tunnels sometimes leading to false or misleading error messages (discussion).
  • Osmose↙
    Osmose (osmose.openstreetmap.fr) is a tool similar to Keep Right but offers even more error types. Obviously, this also includes smaller issues, like minor imprecisions or missing information that should be present in an ideal map.
  • JOSM Validator↙
    JOSM/Validator. A feature of JOSM that checks data loaded into the editor, highlights errors and warnings, and can (on request) perform some automatic fixes. By default, it checks all objects modified in that session (and also reports errors if the error wasn't introduced by the uploader, but merely touched), but the validator can also perform complete validations on the downloaded data.
  • The Gary68 tools↙
    Gary68 has created a whole bunch of quality assurance tools. The tools are currently not maintained. Most of the tools are written in Perl (scripts for your computer – no web tools) and the source code of all Gary68 tools is published online. See also Category:gary68.
    • WayCheck is a program which checks the OSM data (file) for open ends and crossings. The found positions must be evaluated by a user and he or she can correct the data in JOSM or Potlatch or post a mark in OpenStreetBugs. The WayCheck page lists reports generated by the program and any user is encouraged to run reports and correct errors listed on the page and its links. This is especially useful for the quality of routing related data.
    • MotorwayCheck lists problems related to highway=motorway. Page also shows reports.
    • osmdiff is a program which compares two OSM files and documents the changes in list and map form. Some reports can be seen here: osmdiff reports.
    • unmappedplaces finds a list of potentially unmapped places. These are reported on the wiki page. The query can be run for other countries as well.
    • Mapping Quality is the successor of unmappedplaces. See the wiki page for the results. You get automatic street directory, KPIs (numbers) per place, maps that show status and CSV files for further processing.
    • SomeChecks performs some checks: One-way check, Roundabout check, Double-node check (ways), Area Check.
    • Relation Diff - Differences in relations
    • Relation Check
  • OSM Inspector↙
    OSM Inspector (short: OSMI) has several views for specific mapping needs.
  • TIGER Edited Map↙
    TIGER Edited Map ([1]) - Map showing which areas of the U.S. have been edited since the TIGER import.
  • Coarse Highway↙
    Ragged highways with few points (blog entry): Coarse Highways.
  • Map of Turn Restrictions by Zartbitter↙
    ahorn.lima-city.de/tr shows turn restrictions and has separate overlays for errors and warnings showing potential problems with turn restrictions. See also Relation:restriction#Tools for more tools.
  • Restriction Analyzer by MorbZ↙
    OSM Restriction Validator shows unnecessary restrictions and restrictions that block access to streets.
  • US Interstate refs↙
    US interstate refs List of ref numbers and corresponding OSM interstates.
  • layers.openstreetmap.fr↙
    FR:Servers/layers.openstreetmap.fr provides a slippy map with image layers to help people find possible errors in the data with worldwide coverage. The service provides:>
    • layers with roads/street …
      • having no name
      • no ref
      • no one-way
      • drawing fixmes and notes texts.
    • This tool also generates half-transparent coloured layers showing administrative areas by admin_level, as well as other "political" areas (used in France for mapping electoral subdivisions, as well as intercommunal structures of cooperation that do not fit the hierarchy of admin levels, but used as well in other countries for similar non hierarchical administrative, judiciary and electoral boundaries). These coloured layers allow searching easily for missing/broken/incomplete/unclosed boundaries as well as overlapping/duplicate boundaries of the same type, or boundaries that are not tagged correctly to be found by querying the OSM database with simple filters (for example missing tag "boundary=administrative", missing or incorrect value in "admin_level=*").
    This tool shows polygons which present a problem for Nominatim. These ‘broken’ polygons could be where the interior of a multipolygon is disconnected or, where a polygon intersects itself.
  • MapRoulette↙
    MapRoulette (MapRoulette) is a customizable web application to crowd-source the fixing of specific mapping errors by randomly displaying an item of a specific type in potential need of fixing.
  • netzwolf's opening_hours & co. map↙
    [2] - Check Opening_hours, Service_times...
  • Routino error log↙
    Routino a router for OSM data can generate an error log file (when run with the --errorlog option) that lists problems found that could be data errors or Routino's limited understanding of the data and which may affect the ability to calculate valid routes. Since the purpose of the Routino error log is to find data that causes problems for Routino it does not mean that all reported items are data errors.
  • Bicycle tags map↙
    Bicycle tags map is a web map that shows bicycle related tags. The goal of this web map with bicycle tags is rendering bicycle related tags that are not (properly) rendered on OSM’s 'Mapnik' map or on the cyclemap version. It can be handy for detecting errors in OSM.
  • Max Speed Overpass↙
    Max Speed Overpass is a web map that shows maximum speeds. The goal of this web map is to show max speed and edit this from this map using your favorite OSM editor. The data shown is always up to date because it uses overpass query.
  • CheckAutopista↙
    CheckAutopista [3] is a web map for checking information on freeways. It allows to check if maxspeed, lanes and name are used on the ways and it checks if the exits have their ref and their exit_to (or destination). Also it shows the freeway's service and rest areas and it's tollbooths.
  • Relation Analyzer↙
    The Relation Analyzer (http://ra.osmsurround.org/) helps to view several quality-assurance-related aspects of relations: (amongst others) relation on a map with segments' start and end marked, warning if the relation has gaps accompanied by a list of the segments if there are gaps, a height profile, type of ways included, tags.
  • qa.poole.ch↙
    qa.poole.ch is a tool to show streets with no names (respects noname=*, noaddress=* and access=yes tags).
    The layers are updated very frequently, after a minute you should be able to see your edits reflected.
  • Improve OSM↙
    This set of tools from Telenav helps fix OSM based on the vast amounts of GPS data Telenav is gathering from their GPS Navigation & Maps - Scout App (blog post).
    This includes identification of potential missing roads, oneway streets and turn retrictions.
    The tools are available as a website and a JOSM plugin.
  • OSM-Sidewalker by Mapbox↙
    OSM-Sidewalker by Mapbox [ dead link ] shows an overlay of ways which appear to be a sidewalk but are missing the footway=sidewalk tag. The tool was introduced in a blog post [ dead link ].
  • Unmapped places↙
    Unmapped places is a bug tool done by neis-one.org which marks places without roads.
  • Errors in Alternativas libres↙
    In download section of Alternativas libres you can find a text file that marks the errors detected when they do their maps for Garmin. It is divided by countries and information is updated daily.
  • Is OSM up-to-date? ↙
    Is OSM up-to-date looks for old and potentially outdated nodes and ways.
  • BRouter Suspect Manager ↙
    BRouter Suspects are potential problems that affect the connectivity of the road network. The detection is based on BRouter's routing engine using a car profile, and mostly focuses on dead-ends or dead-starts on highway=tertiary an higher, plus some other detection patterns related to blocked roads touching the routable network. In addition to marking a suspect as fixed or false-positive, the suspect manager also allows hiding suspects for a given time, which makes it also applicable for the tracking of temporary construction edits or more complicated issues involving changeset discussions or OSM notes. Also check the Osmoscope view on the BRouter Suspects.
  • housenumbervalidators↙
  • Osmoscope
  • OSM POI Analyzer (OPA)↙
    OSM POI Analyzer (OPA) is an online platform to assess and measure the positional plausibility of POIs in OSM. Currently only 6 cities and 15 Tags in the world are covered by OSM POI Analyzer, but developers are open to suggestions to add more cities and tags to this platform. The platform is using a recommendation system which is based on association rule mining. Distance-based and topological rules are generated for each city and tag (object) and saved in the database. How it works? Users can click on map and try to add an objects. For example choose Paris city and ATM object from right side menu and then click in a corner of street in Paris. The analyzer needs 1 to 2 seconds to process and then brings up the recommendation score with some additional results. However this is experimental and the new POI will not be added to OSM database.This allows users freely navigate, click on places and measure the accuracy of recommendations in this system. For example you can try to click inside jungles and rivers and try to add a fuel station. This platform is developed at the end of Phd work by Mr. Alireza Kashian at University of Melbourne to improve concepts around association rule mining in OSM. The platform is currently hosted and funded by donations. Publications about methodology and how the whole system works is accessible in publication page. There are plans to cover almost 100 major cities of world by Mid 2020.
  • staty↙
    staty is a tool to assess the quality of public transport station tagging in OSM. It checks for suspicious station names and provides suggestions to fix names or group similar stations into public_transport=stop_area or public_transport=stop_area_group relations.
  • Subway Preprocessor↙
    Subway Preprocessor helps to maintain the quality of subways and other rapid transit networks. It reveals logical and topological errors like a stop position being not connected to a station via public_transport=stop_area relation, or incorrect order of stops in a route relation. It also allows to view a subway scheme on the map. Related links are:
https://cdn.organicmaps.app/subway/ — refreshed multiple times daily
https://github.com/alexey-zakharenkov/subways — active repository fork
https://maps.vk.com/osm/tools/subways/latest/ — refreshed nearly every hour
https://maps.vk.com/osm/tools/subways/ — history of recent validations
https://maps.vk.com/osm/tools/subways/weekly/ — weekly historical validation reports
  • Atlas Checks↙
    Atlas Checks are a suite of open-source data quality checking tools using the Atlas framework and Spark. There are a number of different checks looking at topology issues, tags, and more looking at all kinds of OSM data. The results can be used in MapRoulette or in JOSM or your favorite editor. Currently a number of different organizations use Atlas Checks as well.
  • OSM highway name interpolation (ohni)
    The OSM highway name interpolation (ohni) script finds unnamed highways in OSM data whose name can be derived from neighbouring ways and thus be armchair-mapped. (False-positives are possible, use common sense!)
  • OSM NightWatch↙
    The https://nightwatch.openstreetmap.si/ finds errors in coastlines and on various levels of admin polygons (countries, states, regions, subdivisions, municipalities, cities, towns...).
    Main idea that differentiates if from other validation/QA tools is that it updates minutely, with errors and fixes detected near real-time. That combined with clear, reachable goal of no problems makes fixing them a more satisfying process.
    First version was built as part of a week-long hackathon at Microsoft by Milica Jovanović and David Karlaš.

Visualization tools

  • Pedestrian overlay ↙
    [4] – A map of pedestrian/walking data; useful when checking coverage or connectivity of pedestrian mapping.
    It aims to show pedestrian mapping coverage (how many of the pedestrian-taggable elements are tagged with pedestrian data); this may be useful to see how well an area is mapped for pedestrians.
    It also aims to give an idea of connectivity (how well elements are connected, i.e. whether pedestrian elements have nodes connected to each other via nodes, for example, sometimes pedestrian elements are mapped without showing how they connect, for example a path which connects to a sidewalk may not yet have a node connecting one another); this information may be helpful for those working on routing.
  • OSRM debug option↙
    [5] - The debug option of the OSRM demo page provides a map overlay of the highways' speeds calculated by OSRM.
  • OSM Lane Visualizer↙
    [6] OSM Lane Visualizer is a tool to show several attributes of a highway, selected by a overpass query or by relation id.
  • Parking lanes viewer from OpenStreetMap↙
    [7] Parking lanes viewer from OpenStreetMap is a map to show street parking attributes of a highway. The source code is located under: zlant/parking-lanes GitHub.
Note: As of December 2022 the tool uses the deprecated parking:lane=* schema and is therefore not up to date.
  • Bus lanes viewer from OpenStreetMap↙
    [8] Bus lanes viewer from OpenStreetMap is a map to show attributes of a highway, such as lanes:bus=* or lanes:psv=*, the source code is located under: zlant/bus-lanes GitHub.
  • WaterwayMap↙
    [9] WaterwayMap is a map specialized in waterways, which shows basins in different colours (different colours means waterways not connected), endpoints and loops. the source code is located under: amandasaurus/waterwaymap.org GitHub.

Monitoring tools

In addition to the above error and bug reporting tools, there are a number of tools that allow you to spot erroneous changes and edits. For example if you are very familiar with an area and have thoroughly mapped it, you might want to follow all changes and verify that no unintended damage or vandalism has happened in the area. The Planet History includes all changes that have been made, but might not be as easy to use.

In case that a change is bad (by 'mistake' or vandalism) you can rollback the change.

The following and other tools also listed on the List of OSM based Services page. For approaches to automatically detect suspicious changesets see Detect Vandalism. For monitoring tools on notes, see Notes#Monitoring notes in a selected area.

  • osm.org↙
    • 'history' display showing recently opened changesets possibly affecting (bounding box overlap) the currently shown map area
    • 'User edits' display showing recently opened changesets by a particular user.
  • Real-time change following↙
    Currently nearly the same content: List of OSM based Services#Live/real-time edits to OSM data
    • LiveEditMapViewerJ is a Java program that allows you to follow changes as they happen on a world map.
    • OSMZmiany is based on LiveMapViewerJ but has some additional features.
  • History Browser↙
    virtuelle-loipe's History Browser with diff tool for single nodes/ways/relations (side by side comparison of two versions).
  • magOSM - tracking changes↙
    magOSM - tracking changes shows changes grouped by thematic and change type on a heat map and give you access to the change detail with several links to other quality assurance tools. Available only on metropolitan France.
  • Mapki's Deep Diff↙
    Mapki's Deep Diff is a diff tool for single nodes/ways/relations. Showing all versions in a table. A bookmarklet is available.
  • OSM Deep History↙
    https://osmlab.github.io/osm-deep-history/ - diff tool for single nodes/ways/relations (but without nodes of ways and without members of relations). Showing all versions in a table. A bookmarklet is available.
  • OSM Visual History↙
    https://aleung.github.io/osm-visual-history/ - visualized diff tool for single nodes/ways/relations. Showing all versions position on map and tags in a table.
  • OSM History Viewer (by PeWu)↙
    OSM History Viewer (by PeWu) (pewu.github.io/osm-history) - diff tool for single nodes/ways/relations. Showing only the colour-coded changes in a list. A bookmarklet is available on the page.
  • OSM History Viewer↙
    OSM History Viewer (OSMHV) shows modifications by single changesets in a visual way, highlighting object positions before and after a given changeset and tag changes. See also: the tool achavi has a similar feature.
  • OSM Aware↙
    OSM Aware is a program (only Python source code, no web service anymore) to create KML files of OSM change positions.
  • UserActivity↙
    UserActivity Statistics and maps of user activity in two OSM files. Possibly detect vandalism... Reports (for Germany) generated periodically.
  • WhoDidIt↙
    The WhoDidIt: OpenStreetMap Changeset Analyzer analyses what was changed in your area. You may need to select "month" or even "half a year" for the "age" parameter to see edits; display could take some seconds. Features an RSS feed to get informed about new changes.
    WTFPL licensed. At least two alternative versions exist. Longer description.
    Live implementations:
  • Latest OSM Edits per Tile↙
    https://resultmaps.neis-one.org/osm-change-tiles#13/50.0042/8.2912 - analyses latest OpenStreetMap edits per tile. Features an RSS feed to get informed about new changes.
  • RSS History Filter↙
    RSS History Filter filters out large changesets from RSS feeds generated from the OpenStreetMap history tab
  • OSM controltool↙
    https://wheregroup.com/produkte/osmct – commercial product by Wheregroup that allows to monitor changes in freely definable polygons. Only small demo version available without registration.
  • tyrasd's RSS feed link creator↙
    QA RSS feed links for several services can be generated for a self-defined boundingbox at tyrasd.github.io/osm-qa-feeds.
  • tyrasd's Latest Changes service↙
    at https://tyrasd.github.io/latest-changes shows all recent changesets (within a week, day or month) which touched the current map area. Changed objects are highlighted in the map. The object highlight is changing colour (to less intense) with the age of the relevant changeset. Deleted objects (and old versions of modified objects) are shown as faint outlines. OSM_Latest_Changes tyrasd/latest-changes GitHub
  • Explore latest changes on OpenStreetMap↙
    at https://rene78.github.io/latest-changes shows all recent changesets (within a week, day or month) which touched the current map area. Changed objects are highlighted in the map. The object highlight is changing colour (to less intense) with the age of the relevant changeset. Deleted objects (and old versions of modified objects) are shown as faint outlines. OSM_Latest_Changes rene78/latest-changes GitHub
  • OpenStreetMap Recent Change Monitoring System↙
    at https://altilunium.github.io/osm_pantau shows recent changesets by colored dots altilunium/osm_pantau GitHub
  • achavi↙
    achavi (Augmented Change Viewer) visualizes OpenStreetMap changes based on the Overpass API Augmented Delta (adiff) query. Supports continuous monitoring of an area of interest and visualizing single changesets (with some limitations).
  • Overpass attic queries↙
    With Overpass' attic queries you can query the OSM data based on time – example for objects which existed at a specific point in time or that were edited during a period of time.
  • OpenStreetMap Analytic Difference Engine↙
    The OpenStreetMap Analytic Difference Engine (also called "OSM Analytic Tracker" or OSMAT for short) continually analyses activity in a given region (depending on the level of activity, ranging up to small countries) and presents tag changes in fairly easy to digest lists, allowing experienced contributors to spot and react to mapping mistakes in their region within minutes. Also allows opening "diffmaps" (similar to achavi etc) for quick overviews of geometric changes. User Cascafico on his blog documents how to set up an instance on an OrangePI PC Singleboard computer.
  • OSMarelmon - The OSM Relation Monitor↙
    OSMarelmon monitors relations of OpenStreetMap. Monitored relations are periodically checked and the changes are distributed via RSS feeds.
  • IFTTT↙
    Helper tools like IFTTT might be useful to convert (possibly a subset of) e.g. QA RSS feed entries into a mail notification.
  • Where are the new OSM Contributors?↙
    Pascal's resultsmaps contain a "Where are the new OpenStreetMap Contributors?" feature. You can spot new contributors who have edited in a specific area (may be useful to check their edits for big errors, offer help, send welcome messages and spot vandalism). Create a RSS feed.
  • OSMCHA - OSM Changeset Analyzer↙
    OSMCha features an extensive changeset filtering, including detecting suspicious edits and export as an RSS feed. See here for instructions and details.
  • Osmlab Changeset-map - OSM Changeset Viewer↙
    Osmlab Changeset-map Example CS Visualise a changeset on a map. Github
  • osm-suspicious↙
    osm-suspicious by Pascal Neis for finding suspicious changesets.
  • OSM Hall Monitor↙
    OSM Hall Monitor With OSM Hall Monitor, you can track edits made by specified users, made to specified objects, or made with certain tags. You can also enable notifications to receive emails when any of the flagged people or items are edited. Basic functionality for suspicious changeset monitoring is also included.
  • OSM Edits Made With MAPS.ME ↙
    [10] With this tool you can see edits made with MAPS.ME. You can also see by country, by user, etc. open the objects in JOSM or Level0. etc. .

Assistant tools

  • Traffic Sign Tool↙
    Traffic Sign Tool / Verkehrszeichen_Tool (de) - Assistant to get the right tags for German road signs.
  • Road lists↙
    Users in Germany requested different official lists: Straßenverzeichnis.
  • CheckTheMonuments↙
    Check The Monuments! More Quality for the Key Monument.
  • Overpass turbo↙
    Overpass API and the interface Overpass turbo lets you easily create custom queries which can be used to detect potential errors. The results get shown on a map with object detail links in popups and can be easily opened in JOSM. The example page lists some quality assurance uses.
  • (Missing) Maxheight map↙
    The Maxheight Map (aka OSM Truck QA Map) (website) is a Overpass API-based browser tool which helps you check and improve truck relevant features in OSM. Besides identifying missing maximum height tags under (railway) bridges and tunnels, you can also validate existing tagging for maximum height, width, length and weight amongst others.
  • OSM Quality Assurance Editor (by osmsurround)↙
    The OSM Quality Assurance Editor (website) helps to highlight (via Overpass API) missing data in several categories (tracks without tracktype tag, tracks without surface tag, streets without any sidewalk tag, ways without incline tag, ways without surface tag, ways without smoothness tag, buildings without address, residential without name) and allows to directly edit the objects via a built-in online editor.
  • Public Transport Network Analysis (PTNA)↙
    PTNA finds and aggregate information about public transportation lines mapped in OSM. Additionally a quality assurance is integrated. GTFS data is visualized and tags for route relations recommended.
  • StreetComplete detects missing or incomplete data and allows to easily add it while surveying (Android application)

Tag statistics

To compare the use of a tag or to see typos, use these statistical tools. See also: Statistics.

  • Taginfo↙
    Taginfo is a system for finding and aggregating information about OSM tags and making it browsable and searchable.
  • OSM Tag History↙
    OSM Tag History displays the number of times a key or a key-value pair has been used as a function of time on a visual graph
  • ohsome↙
    The ohsome dashboard and API allow to perform in-depth analysis of the development of OSM data.

Other

External compares

See also: Research

See also

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Quality_assurance&oldid=2774047#Monitoring_Tools"