Osmose

From OpenStreetMap Wiki
Jump to navigation Jump to search
Osmose - Other languages
Osmose
Osmose1.png
License: GNU General Public License v3
Website: https://osmose.openstreetmap.fr/
Source code:
Programming language: Python


Osmose is one of many quality assurance tools available to detect issues in OpenStreetMap data. It detects a very wide range of issue types. It is also useful for integrating third-party data sets or Conflation.

Website: osmose.openstreetmap.fr

Warning 1: Do not map for Osmose, keepright, etc. Only fix real errors which you understand and know how to improve correctly!!!

Yes, Osmose has some false positives, like any other validator. Systematic issues should be reported

Warning 2: Avoid mechanical edits. If it was a case for automatic edit, osmose was able to do so!

Usage

The idea is to show anomalies identified in the OpenStreetMap data on a map.

Filter settings

In the upper left are several filters which can be used for only showing issues matching the filter settings. To see all filter options, click on "More filters".

  • Severity: Show only issues with marked with a specific severity levels: High, Normal and higher, All, Normal only, Low only.
  • Fixable: Show issues which have a proposed corrections (the fix).
    • Online: A fix that can be applied via the online Osmose tags editor.
    • JOSM: Are fixes that requires a more advanced editor as JOSM. These fixes need a manual repositioning of object in addition of modifying tags.
  • Topic: Show only issues marked with a specific theme, such as roads, cycling, sport.
  • Country: Show only issues in a specific country or region of country.
  • OSM Username: Show only issues last modified by this username.

The list of categories

The list on the left groups issues into several categories, each with a specific colour. Each item has its own glyph and can be activated or deactivated to filter the contents of the map. Some items may combine several analyses of the same kind.

The slippy map

Osmose is build upon a Slippy Map. The number of displayed issue markers is limited. Zoom to show more. The markers on the map will open a popup on click.

The information popup

Example of an information popup when clicking an issue marker.

When you click on one of the issue markers, a popup appears. It contains:

  1. The type of issue
    and a brief description of the issue, with optionally some focused problematic details about this object.
  2. The identification of the problematic element(s), with for each one:
    • type and identifier of the element ; the link goes to OpenStreetMap.org to show the default map centered on that element, opening the element browse panel (with all details for their actual current tags, version, links to show its history, and options to edit it there with the default editor...) and marking its geometry on the map.
    • josm – a link to the JOSM editor to load this element; JOSM must be running and the JOSM remote control must be enabled.
    • iD – a link to the iD editor to load this element.
    • edit – a link to the internal Osmose tags editor for this element ; this opens a panel on the right of the map
  3. The list of all tags for this element when the issue was detected.
  4. Some links to display the area containing the element (as it is currently in the database, without any fix):
    • osm-show – display the area on ​​the default map of OpenStreetMap.org
    • osm-edit – edit the area in the default editor of OpenStreetMap.org (iD or Potlatch, depending on local user preferences)
    • josm-zone – load data in the area into the external JOSM editor; JOSM must be running and the JOSM remote control must be enabled.
    • details – open a new browser tab with full details about this detected issue.
  5. The last date this issue was detected (and still not closed).
  6. Buttons:
    • ℹ Help on OSM wiki: open a new browser tab with help about this specific type of issue on the wiki and tips on how to fix it (see the list of issues). In the help window you can also hide all errors if this type from the map.
    • ✘ false positive: if the issue reported by Osmose is not an error in fact, then it is a false positive (except spelling errors). An issue marked as a false positive will be masked by Osmose (as long as the element version is not changed). It is not possible, so far, to write descriptive information on about false positive cases, but the list of false positives can help improve the analysis performed by Osmose. If the element is later modified, it may reappear on the next analysis.
    • ✔ corrected: issue will no longer be shown on the Osmose map; if the applied correction is not sufficient in the current version of the element, it will reappear again latter. Remember to mark as corrected the issue you corrected by your edits. This allows everyone to follow the correction progress. If you don't click this link, but the issue has been worked on and corrected, this issue will disappear automatically on the next analysis.
  7. The suggested tags correction (with + for adding a missing tag, − for deleting this tag, or ~ for modifying it).
    • The fix-josm link applies this correction in JOSM; JOSM must be running and the JOSM remote control must be enabled. Only that element will be loaded with the corrections applied.
    • The fix-edit link applies this correction in the internal Osmose tags editor
  8. ✖ close the popup

Layers

Heatmap, issues density

Among the map layers available (under the upper right icon), most are standard basemap choices. There are however to some layers generated by the tool:

  • Osmose issues: markers layer
  • Osmose Issues Heatmap: density of issues, the intensity of red depends on the number of issues present and the items filtered.

Several other layers are generated by separate tools but may be also displayed on top of the selected basemap, including

  • layers showing administrative boundaries for the most common levels;
  • coverage by Mapillary street-level photos along major road axis in rural areas, and streets in cities, to help mapping complex intersections, lanes, signals, equipment, and POIs, when aerial photography is not sufficient.

Help and issues description

An information panel is available on the right. It can be open and close with "i" button. It content documentation about current selected marker the section are Titles, Details, Correction aids, pitfalls to Avoid, source code, etc.

This content is included in the source code of the analyses. You can contribute to the improvement of the documentation by proposing changes to the source on Github. Online on the Github website just use the pencil icon and let Github guide you to make a change request (pull request).

Osmose integrated tags editor

Osmose tags Editor with a deleted tag, a modified one and three added

Osmose allows direct correction of issues through an integrated tags editor. It allows you to view erroneous elements edit their tags. To use the editor you must be logged in Osmose with your OSM account.

  • In red, overstriked and preceded by a − sign, the existing tags that will be removed when saving; the cross × button on the right will restore them to preserve their original value.
  • In black and preceded by a white = sign, the existing tags that will be kept unchanged when saving; the cross × button on the right will remove them when saving.
  • In orange and preceded by a ~ symbol, the existing tags whose value will be changed when saving; the cross × button on the right will restore them to preserve their original value.
  • In green and preceded by a + sign, the new tags that will be added when saving; the cross × button on the right will remove them, these tags won't be saved.

Tags are free fields. They must all contain a key and a value separated by an "=". The last line is always empty after the + sign, and allows adding a new tag.

When you press the Valid button, changes go into a queue. They are sent to the OSM server when you follow the link "Save" on the main menu. The Valid button automatically set the issue marker status as corrected. Note that the queued data is still not applied to the OSM database, it is kept only with your personal online session, even if the marker is no longer visible on the "Osmose issues" layer.

Osmose tags editor before sending to OSM servers

In the Save window, the changeset input fields (Comment, Source, Type) will be stored in the changeset's tags. The "Reuse changeset" checkbox allows to reuse, if possible, the last changeset that Osmose has created rather than creating a new one.

Other features and tools

Testmode by username

This mode is an interesting feature, which is list of issues per user. This can be useful if you want to just correct your own mistakes, for example, or to race with your co-worker ;). It is accessible via the URL: http://osmose.openstreetmap.fr/en/byuser or from your user name once logged into Osmose. From the list of issues you can return to the map and display only issues affected to you.

Analyzer relationship

See Servers/analyser.openstreetmap.fr (disfunctional)

administrative border drawn by the parser

This tool is an addition to the issues "open relationship", "self-intersection" and "too many ways" in "administrative" in the slippy map. It allows you to check after sending a modification to the server, if the errors have been corrected.

Featurerequests and Bugreports

→ ahead to frontend ticket system ←
→ ahead to backend ticket system ← (Using for validation rule issues)

How does it work?

One frontend and multiple backends

The tool comes in two parts:

  • The frontend: visible part for displaying issues on a slippy map,
  • The backends: hidden part for OpenStreetMap data analysis and issues detection.

The coverage is worldwide and a complete and updated list of areas is available on Osmose's API. The areas are not all analysed in the same way, the detection of certain types of defects depends of particularities like language, availability of third-party data (e.g. Open Data)...

Contact the project to offer resources to support for country-specific settings, or new languages:

The source code is available on GitHub: backend and frontend under GPL.

  • Analysis: for each type of issue detected by Osmose there is an analysis. They are written in python, SQL or MapCSS language. Details on OpenData merge analyser.
  • Update are made each day.

Statistics update are here: update and here update matrix. Analyse scheduler of OSM-Fr server can be found on Buildbot.

API documentation

Osmose provides an API for external tools to get analysis results.

Issues reporting API

Issues can be reported by programs to Osmose-frontend.

Majority of issues are reported by Osmose-Backend, but some are reported by other tools :

Analyser send XML issues list to the frontend by HTTP POST to https://osmose.openstreetmap.fr/control/send-update, with the following parameters:

  • analyser: id of source used by the frontend.
  • code: password associated with the source.
  • country: name of country (see osmose config: [1] )
  • content: multipart/form-data encoding of the issue file, described below.

With cURL this gives something like: 

   curl --request POST --form analyser='mysource' --form code='mypassword' --form country='anndora' --form content=@myfile.xml https://osmose.openstreetmap.fr/control/send-update

To obtain a source id and code contact the Osmose team.

The issue file can erase and replace all issues on the frontend from this analyser or be diff file and only update frontend issue database.

Issues file format

DOC : https://github.com/osm-fr/osmose-backend/blob/master/doc/0-Index.md#classes-of-issues 

<?xml version="1.0" encoding="UTF-8"?>
<analysers timestamp="2014-10-17T09:52:58Z">
  <!-- Result of an analyser run (many) -->
  <analyser timestamp="2014-10-17T09:52:58Z">
    <!-- Class definition (many)
      id: referenced by <error class="" />
      item: osmose frontend target item
      level: matter of issue in OSM data, from "1" as major to "3" as minor
      tag: key-words of issue (not a set of OSM tags)
      -->
    <class item="3210" tag="highway,tag,fix:chair" id="1" level="2">
      <!-- title of class, one line per language -->
      <classtext lang="en" title="noexit on way with multiple exits" />
    </class>
    <!-- Issue entry (many)
      class: refer to <class id="" />
      subclass: group the same kind of issue (optional)
      -->
    <error class="1" subclass="1">
      <!-- Issue location marker -->
      <location lat="43.2899342" lon="5.3618622" />
      <!-- erroneous object (none or many), all details are optional -->
      <node version="3" user="FrançoiseR" lat="43.2899342" lon="5.3618622" id="33806225">
        <tag k="noexit" v="yes" />
      </node>
      <!-- optional subtitle, one line per language -->
      <text lang="en" value="Tag power_source is deprecated: generator:source" />
      <!-- Optional fixes -->
      <fixes>
        <!-- One way of fix the issue (many) -->
        <fix>
          <!-- Objects to patch (many)
            id: OSM object id, 0 to create a new one (node only)
            -->
          <node id="33806225">
            <!-- Tags patch (many)
              action: one of "delete", "create", "modify"
              k: tag key
              v: tag value
              -->
            <tag action="delete" k="noexit" />
          </node>
        </fix>
      </fixes>
    </error>
  </analyser>
</analysers>

Writting a new check

A check can be written as an internal analyser or an external tool uploading results to Osmose-QA Frontend.

Internal analysers are run by Osmose Backends and can be:

  • A mono OSM object check: written in Python or MapCSS.
  • A multi object check involving spatial computation: written in SQL with PostGIS.
  • An OpenData conflation: written by configuring a column to OSM tag mapping.

Documentation

The documentation of each check (named "class") consists of a set of strings defined in the source code. Translation of these strings is handled on Transifex.

Each instance of a class can also have an optional "subtitle" - which is a free text providing details specific to the instance (for example, which Key the issue pertains to). Ideally this should be written as a valid sentence.

Data Key Required Markdown
Enabled		 Description
title ✔ Short string which describes the issue (answers "what is the issue?").
Ideally titles should not contain any OSM tagging.
Only the first word of the tile should be capitalised (and any proper nouns or acronyms) and there should be no end period.
detail ✔ ✔ Longer string which explains the issue (answers "why is this an issue?"). Is constructed with valid sentences and ends in a period.
fix ✔ General guide on how to fix the issue. Constructed with valid sentences and ends in a period.
trap ✔ Common mistakes and complications you may make while fixing the issue. Constructed with valid sentences and ends in a period.
example ✔ An example of the issue. Constructed with valid sentences and ends in a period.
Images are commonly found here (markdown ![](imgURL)).
source ✔ URL to the source code of this class.
resource URL to any data used to produce this issue.

An example of how these are served by the API.

See also

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Osmose&oldid=2706772"