Xapi

This page describes a historic artifact in the history of OpenStreetMap. It does not reflect the current situation, but instead documents the historical concepts, issues, or ideas.

The OSM Extended API (or xapi, pronounced zappy) is a read-only API protocol, based on a modified version of the OSM main API, that provides enhanced search and querying capabilities. It offers search queries for some common simple use cases and helps to take load off the main API. In particular, it reimplements the standard map request such that it can be performed much faster. Xapi uses a REST style interface with X-path flavouring. More sophisticated queries can be handled with other read-only mirrors such as Overpass API which support a more powerful but more complex syntax

Xapi only ever deals with elements that are current and does not return any elements that are historical or deleted. The source database is a mirror of the main OSM database and is updated via the per-minute diff dumps. The data is normally no more than about 10 minutes behind the main database.

All responses are in the same format as the standard protocol, but with the addition of some namespaced extensions. Tag queries use an X-path like syntax to specify search terms. The visible attribute is excluded from the response because it always has a true value.

Web services available

Known public running deployments :

http://www.overpass-api.de/api/xapi_meta?... Running normally Overpass API based

Former public instances (no longer available):

http://overpass.openstreetmap.ru/cgi/xapi_meta?...
http://open.mapquestapi.com/xapi/api/0.6/... JXAPI based provided by Open Mapquest. Decomissioned since October 31, 2016
https://jxapi.openstreetmap.org/xapi/api/0.6/... JXAPI based by Ian Dees
http://jxapi.openstreetmap.ru/xapi/api/0.6/... JXAPI based provided by Rambler
http://www.informationfreeway.org/api/0.6/... XAPI instance by The Information Freeway
http://api.openstreetmap.fr/xapi?... Overpass API based Provided by OpenStreetMap_France more info at Servers/api.openstreetmap.fr

Implementations

There are several code/database implementation of the XAPI idea. These are listed below.

Overpass API

Overpass API now has a XAPI Compatibility Layer. It supports most XAPI queries, as well as its own, more extensible query language.

jXAPI

In January 2011 Ian Dees created a new Java version of XAPI backed onto postGIS. See his diary entry for details. Several people are now running this on different servers.

pnorman's blog post on jXAPI setup has some information on tuning postgresql for a large dataset and speeding up the initial load of data. No known public instances

Xappy.js

Xappy.js is a reimplementation of the OSM XAPI in node.js (Java Script). Currently using Postgres and the PostGIS extension with the option to implement other backends. No known public instances

The original XAPI

The original XAPI was created using GT.M (formerly MUMPS) by User:80n. There were multiple instances of the xapi service running on several different servers. Each server could be accessed directly or via a redirect which was used to direct the request to a suitable server of its choosing. Source code is here https://github.com/fosm/xapi

No known public instances

Query Map

The Map query is identical to the main API map query and returns:

All nodes that are inside a given bounding box and any relations that reference them.
All ways that reference at least one node that is inside a given bounding box, any relations that reference them [the ways], and any nodes outside the bounding box that the ways may reference.

GET /api/0.6/map?bbox=left,bottom,right,top

where:

left is the longitude of the left (westernmost) side of the bounding box, or minlon.
bottom is the latitude of the bottom (southernmost) side of the bounding box, or minlat.
right is the longitude of the right (easternmost) side of the bounding box, or maxlon.
top is the latitude of the top (northernmost) side of the bounding box, or maxlat.

Example

http://www.informationfreeway.org/api/0.6/map?bbox=11.54,48.14,11.543,48.145

Notes

Unlike the main API which restricts queries to 0.25 square degrees, XAPI allows much larger requests, up to 10,000,000 elements (this is still less than a third of California) or 100 square degrees, whichever is smaller.
The map query is functionally identical to the tag query /api/0.6/*[bbox=left,bottom,right,top]
There must not be spaces in the string of values defining the box.

Query Tags

Xapi implements queries for all three kinds of elements; nodes, ways and relations.

Nodes

A node request will return an xml document containing nodes and their associated tags. The URL for a node request takes the following form:

http://www.informationfreeway.org/api/0.6/node[...]

This will return a standard OSM API response containing nodes and tags. For example:

  <?xml version='1.0' standalone='no'?>
  <osm version='0.6' generator='xapi: OSM Extended API' 
   xmlns:xapi='http://www.informationfreeway.org/xapi/0.6'
   xapi:uri='/api/0.6/node[amenity=hospital]'
   xapi:planetDate='200803150826' 
   xapi:copyright='2008 OpenStreetMap contributors' 
   xapi:instance='zappy2'>
    <node id='672180' lat='48.2111685091189' lon='16.3035366605548' timestamp='2006-09-11T16:28:25+01:00' version='1' changeset='10968'>
      <tag  k='amenity' v='hospital'/>
      <tag  k='name' v='Wilhelminenspital'/>
    </node>
    <node id='3596186' lat='53.4633699598014' lon='-2.22667910006381' timestamp='2007-06-21T17:10:58+01:00' version='2' changeset='2213'>
      <tag  k='amenity' v='hospital'/>
      <tag  k='name' v='Manchester Royal Infirmary'/>
    </node>
    ...
  </osm>

Ways

The URL for way requests has the following form:

http://www.informationfreeway.org/api/0.6/way[...]

This returns an xml document containing ways that match the search terms. For each matching way the nodes referenced by that way are also returned. An example of the response from a way query looks like this:

  <?xml version='1.0' standalone='no'?>
  <osm version='0.6' generator='xapi: OSM Extended API' 
   xmlns:xapi='http://www.informationfreeway.org/xapi/0.6'
   xapi:uri='/api/0.6/way[landuse=residential]'
   xapi:planetDate='200803150826' 
   xapi:copyright='2008 OpenStreetMap contributors' 
   xapi:instance='zappy2'>
    <node id='218963' lat='52.5611324692581' lon='-1.79024812573334' timestamp='2006-03-22T16:47:48+00:00' version='1' changeset='2211'>
    </node>
    <node id='331193' lat='53.7091237972264' lon='-1.50282510180841' timestamp='2007-03-31T00:09:22+01:00' version='1' changeset='2211'>
      <tag  k='highway' v='traffic_signals'/>
      <tag  k='source' v='Yahoo'/>
    </node>
    ...
    <way id='4958218' timestamp='2007-07-25T01:55:35+01:00' version='3' changeset='2211'>
      <nd ref='218963'/>
      <nd ref='331193'/>
      ...
      <tag  k='landuse' v='residential'/>
      <tag  k='source' v='landsat'/>
    </way>
  </osm>

Relations

The URL for relation requests has the following form:

http://www.informationfreeway.org/api/0.6/relation[...]

This returns an xml document containing relations that match the search terms. For each matching relation the nodes and ways referenced by that relation are also returned. An example of the response from a relation query looks like this:

  <?xml version='1.0' standalone='no'?>
  <osm version='0.6' generator='xapi: OSM Extended API' 
   xmlns:xapi='http://www.informationfreeway.org/xapi/0.6'
   xapi:uri='/api/0.6/way[landuse=residential]'
   xapi:planetDate='200803150826' 
   xapi:copyright='2008 OpenStreetMap contributors' 
   xapi:instance='zappy2'>
    <node ...
    <way ...
    <relation id='2670' timestamp='2007-10-25T03:05:34Z' version='32' changeset='2211'>
      <member type='way' ref='3992472' role=''/>
      <member type='way' ref='3992524' role=''/>
      <member type='way' ref='4253050' role=''/>
      <member type='way' ref='4253053' role=''/>
      <member type='way' ref='4266813' role=''/>
      <member type='way' ref='10285106' role=''/>
      <tag k='name' v='Fonnereau Way'/>
      <tag k='network' v='Ipswich footpaths'/>
      <tag k='type' v='route'/>
    </relation>    
  </osm>

All Elements

All elements (nodes, ways and relations) that match the given filter predicates can be requested using the following URL:

http://www.informationfreeway.org/api/0.6/*[...]

This returns an xml document containing nodes, ways and relations that match the search terms. For each matching way the nodes and referenced by that way are also returned. Likewise, for each matching relation the ways and nodes referenced by that relation are also returned. An example of the response from this type of query looks like this:

  <?xml version='1.0' standalone='no'?>
  <osm version='0.6' generator='xapi: OSM Extended API' 
   xmlns:xapi='http://www.informationfreeway.org/xapi/0.6'
   xapi:uri='/api/0.6/*[amenity=hotel]'
   xapi:planetDate='200803150826' 
   xapi:copyright='2008 OpenStreetMap contributors' 
   xapi:instance='zappy2'>
    <node id='218963' lat='52.5611324692581' lon='-1.79024812573334' timestamp='2006-03-22T16:47:48+00:00' version='1' changeset='2211'>
    </node>
    <node id='331193' lat='53.7091237972264' lon='-1.50282510180841' timestamp='2007-03-31T00:09:22+01:00' version='1' changeset='2211'>
      <tag k='amenity' v='hotel'/>
    </node>
    ... 
    <way id='4958218' timestamp='2007-07-25T01:55:35+01:00' version='1' changeset='2211'>
      <nd ref='218963'/>
      <nd ref='331193'/>
      ...
      <tag k='amenity' v='hotel'/>
      <tag k='building' v='hotel'/>
    </way>
    <relation id='123456' timestamp='2007-10-25T03:05:34Z' version='32' changeset='2211'>
      <member type='node' ref='331193' role=''/>
      <member type='node' ref='331194' role=''/>
      ...
      <tag k='amenity' v='hotel'/>
      <tag k='operator' v='Premier Inns'/>
      <tag k='type' v='operators'/>
    </relation>    
  </osm>

Predicates

Each API request can be suffixed with selection predicates which determine which elements to select. For example [amenity=hospital] will select all elements that have an amenity tag with a value of hospital. The full url to select all nodes that are tagged as hospitals would be:

http://www.informationfreeway.org/api/0.6/node[amenity=hospital]

A request can be suffixed with multiple predicates, each of which further constrains the results (currently limited to one tag predicate and one bbox predicate, though jXAPI and Overpass servers support multiple tag predicates). For example:

http://www.informationfreeway.org/api/0.6/node[amenity=hospital][bbox=-6,50,2,61]

this request will select and return all nodes with a tag of amenity=hospital within the bounding box that covers the whole of the England, Wales and Scotland.

Tag Predicates

A selection predicate must be of the form [key=value]. For example: node[amenity=hospital] will match any node that has an amenity key with a value of hospital, ie

<tag k="amenity" v="hospital"/>

.

A union operator may be used to select multiple values. For example to select all major roads:

way[highway=motorway|motorway_link|trunk|primary]

The union operator can also be used with the key. For example to select golf courses:

node[amenity|leisure=golf_course]

A wildcard can be used for the value (but not for the key), so predicates similar to the following examples are possible:

way[construction=*]
way[highway=*]

Key wildcards were supported in the past, but this is no longer maintained.

Note: The URL needs to be UTF-8 url encoded in case of non-ascii characters.

BBox Predicates

Another form of selection predicate is the bbox pseudo key. A selection predicate of the form [bbox=left,bottom,right,top] defines a bounding box that is used to limit the extent of the result document. Only elements that have some part within the bbox will be included in the result document.

Unlike the standard OSM API, the bbox predicate is not limited in size unless it is used without a tag predicate, in which case it is limited to 100 square degrees.

The default extent, if a bbox predicate is not specified, is the whole planet. This is equivalent to specifying [bbox=-180,-90,180,90].

Child Element Predicates

Items can be selected based on whether or not they have child elements. For example, ways can be selected that do not have any tags or do not have any nodes. This is achieved by using an XPath-like element test. Specifically the following are implemented:

/api/0.6/way[nd] - selects ways that have at least one node
/api/0.6/way[tag] - selects ways that have at least one tag
/api/0.6/way[not(nd)] - only selects ways that do not have any nodes
/api/0.6/way[not(tag)] - only selects ways that do not have any tags
/api/0.6/node[way] - selects nodes that belong to at least one way
/api/0.6/node[not(way)] - selects nodes that do not belong to any way
/api/0.6/node[not(tag)] - selects nodes that do not have any tags
/api/0.6/relation[node] - selects relations that have at least one node member
/api/0.6/relation[way] - selects relations that have at least one way member
/api/0.6/relation[relation] - selects relations that have at least one relation member
/api/0.6/relation[not(node)] - selects relations that do not have any node members
/api/0.6/relation[not(way)] - selects relations that do not have any way members
/api/0.6/relation[not(relation)] - selects relations that do not have any relation members

Notes:

If using wget then parenthesis characters need to be escaped, so for negation predicates, use [not\(way\)] rather than [not(way)].
In the OSM xml schema nodes do not really have ways as child elements. However, there is, logically, a many-to-many relationship between nodes and ways, and so xapi implements this relationship and enables nodes to be selected based on whether or not they are members of a way. Of course, the output xml is the same as normal and does not include way elements within a node element.

Escaping

Values within predicates can be escaped by prefixing a backslash character. For example to retrieve elements with a tag value that contains a pipe character such as [route=46|46A] you would need to use [route=46\|46A].

The following characters can be escaped in this way: | [ ] * / = ( ) \ and space

Spaces in tag values do not normally need to be escaped but this is dependent on the client. In this case you need to use URL escaping which requires that each space be replaced by %20. So [route=46 46A] would become [route=46%2046A].

Usage

Xapi URLs can be used from within any web-browser, however a simple query can return several Mb of data which can overwhelm some browsers. It is recommended that a tool such as wget or curl be used with the output directed to a file. For example, the following command uses wget to select all hospitals and store the result document in a file named data.osm. Note that wget has a default timeout of 15 minutes which may be too short for some queries so be sure to add the --timeout=0 option to turn this off.

wget --timeout=0 http://www.informationfreeway.org/api/0.6/node[amenity=hospital] -O data.osm

Note: If you use curl, it will handle redirections automatically, so you need to query the host server directly or use the "--location" option. Also you need to use the "--globoff" option, otherwise it will try to interpret the brackets ("[", "]") as part of the command line:

curl --location --globoff "http://www.informationfreeway.org/api/0.6/node[amenity=hospital]" -o data.osm

If you use a script regularly to fetch data from Xapi it would be helpful to set the user agent string to something meaningful. This will help us understand more about how Xapi is being used.

Limitations

Predicates. Currently each request is limited to one tag predicate and one bbox predicate. In the future it is hoped that more sophisticated conditions will be implemented. Multiple bboxes would be great for applications like GpsMid - they already implemented data retrieval over xapi in Osm2GpsMid for single regions which is very helpful.
Request grammar. The request grammar is not formally defined and is not processed using a proper parser, so malformed requests may have unpredictable results. Caveat Emptor.

Proposed Future Extensions

"Search by region": the ability to limit queries not by a bounding box, but by recognized regions (e.g. Countries, Counties, continents). Proposed Jan 2011.

json output: We have this feature in xappy.js it might be used by setting the http content-type header to "application/json". Here some format infos: Xappy.js#JSON output

See Overpass API/Overpass QL instead.

Implementation-specific differences

Some xapi implementations differ from the specification set out in this page.

Overpass XAPI wrapper

Request Metadata

The request metadata returned from overpass differs from the xapi metadata. Todo: list how it differs

Metadata

Overpass does not return the metadata (version number and changeset) by default. An xapi request against the overpass xapi compatibility layer will not result in a .osm file that can be used for editing. See @meta below.

@meta

As a work-around to the above, [@meta] can be added to a request to have the metadata returned and an editable file.