Talk:Xapi

From OpenStreetMap Wiki
Jump to navigation Jump to search
Logo.png
This page describes a historic artifact in the history of OpenStreetMap. It does not reflect the current situation, but instead documents the historical concepts, issues, or ideas.


Warning
 XAPI is no longer recommended as none of the original native installations are available anymore.
Legacy XAPI applications can still leverage the XAPI Compatibility Layer, new projects should use Overpass QL instead.


data as attachment (HELP!)

I am trying to use a PHP script (based on the ones used by the binary map API) to get data from XAPI for a specific bounding box and content filtering. XAPI appears suitable as it is not limited to 0.25 degrees for the BBox and the OSM API gives you ALL the data when I want very limited subsets. For the first tests I am retrieving just coastline ways for the UK.

The script is working so far but expects the data to be bundled with the HTTP response whereas XAPI delivers it as an attachment in a separate file 'data.osm'. Unfortunately, being a complete PHP novice, I don't know if I can save and access this file using a PHP script.

Is there a way to either get the data included in the body of the response or to tell .../API/0.5/way[...] where the attachment should be saved? --Elvin 15:16, 7 May 2008 (UTC)


other output formats

Any chance of other output formats besides osm xml? Like GeoRSS, KML, GML,... SVG :) --Stefanb 08:25, 27 July 2007 (BST)

Output in something very simple like CSV could be quite useful to help get people outside the project using the data --Dankarran 01:59, 15 November 2009 (UTC)

Why not use XPath based protocol?

http://www.informationfreeway.org/api/0.5/node[@k='amenity']['@v='hospital']

Oracle has XML capabilities. It should be possible to store XML data directly to the DB. SELECT and INDEX are able to understand XPATH.

Here some examples for PHP: http://www.oracle.com/technology/oramag/oracle/05-jul/o45php.html

--User:Robotnik 22:15, 30 December 2007

Errors

Getting 501 internal server error for http://informationfreeway.org/api/0.5/way[railway=subway] Ojw 10:21, 24 December 2007 (UTC)

No value

How to get objects without a property? For example coutries without Polish name. 

http://smxapi.hypercube.telascience.org/api/0.5/node[place='country][name:pl=...

Steelman 19:19, 10 January 2008 (UTC)

Only ways

I think it might be useful to allow download only ways (not referenced nodes). I need it to get highway refs so I can compare them with the official list. Other use could be all kind of way related statistics. Downloaded file can get quite big when referenced nodes are included. --Jttt 19:34, 16th May 2008 (UTC)

I would like to save bandwidth by downloading only the ways for some specific tag processing too. Is is possible to just return ways? --Pinkduck 12:46, 7 November 2009 (UTC)

UTF8 encoding bug?

Seems that for usernames containing accented characters (like what I got from http://www.informationfreeway.org/api/0.5/way%5bosm:timestamp=2008-01-15T20:52:15Z%5d ), the accents are not properly converted to utf-8, they seem to remain in latin-1 or similar charset.

This of course breaks many software trying to process the data, as they'll choke on the character and bail out ...

--Bilbo 00:29, 8 June 2008 (UTC)

Another one: I tried to search for strings containing a plus ("+") character. No chance with the current query code, since it translates all plus character to space, even URL-encoded strings. No way to get the correct string to the interpreter. Solutions? Since Xapi is actually a brilliant tool do do things like OSM database cleanup (if curious, see my current effort to clean misspellings and more) I hope to get this working.

Maximum Response Size

Is there a maximum response size? If so could it be mentioned in the article, perhaps, with some suggestions for how to download multiple sets and merge them into one. It seems that now that New Zealand exceeds 60,000,000 bytes I have no luck downloading it. Perhaps I'm just unlucky, or perhaps there is a limit in place. Karora 22:43, 13 June 2008 (UTC)

Hmmm... looks like something's broken on the server, since I get a ERROR 501: Internal Server Error back. Karora 04:58, 14 June 2008 (UTC)
Yes, I'm getting a 501 error too, and only trying to download a relatively small area. jamjar2906 21:41, 20 July 2008 (UTC)

incomplete data

I downloaded postal_codes of Germany with this url: http://www.informationfreeway.org/api/0.5/*[postal_code=*][bbox=6,47,15,55]. And now I see, data I get is not all that is in database. Data provided here is based on the planet.osm file. I also download Berlin from download.geofabrik.de based also on planet file. Here the data is included (north-east of Berlin). So the error is not on the file. All data which is missing in osmxapi-stream was created before a specific (unknown) date. Any idea why this happen (501 internal server error?) and how this can be fixed in future. --Bahnpirat 14:10, 27 June 2008 (UTC)

wildcards in the value?

hi, it could be userfull to enable the use of wildcards in the value part of the query
eg:

wget http://www.informationfreeway.org/api/0.5/way[name=via*][bbox=9.0556,45.3782,9.2777,45.5585] -O osm_milano_via.osm

in this way i could select just the ways with the name starting with "via" (so i can convert them to "Via") --EdoM (lets talk about it) 08:55, 1 August 2008 (UTC)

Feature-Request: Relation as Predicate

Would be nice to select data inside a relation: E.g. getting street-names inside a town border for statistics could be handy... kannix 2008-12-04

Child Element Predicates

Could I get a example how it work? Could I make complex queries with this new feature? I.E.: I want to get all train_stops as child's off a special railway? I want in this case only the train_stops and not the other points on the railway. Would this be possible? --Kolossos 18:46, 18 December 2008 (UTC)

slashes e.g. [source=http://wiki.openstreetmap.org/wiki/....]

Slashes seem to break xapi requests. Any solution? --kannix

Reported as #2831. --Scytale 22:04, 26 March 2010 (UTC)

API 0.6

What is the state of api0.6-support on xapi? (versions, ordered relations) --MarcusWolschon 11:33, 6 February 2009 (UTC)


Semicolons

Should XAPI unescape semicolons in the data it returns? I'm seeing \s in data where a semicolon should be, which then, if re-uploaded, comes back as a \\s next time, and so on... Southglos 00:08, 16 March 2009 (UTC)

How to use Xapi in a practical way?

Is it a simple change in my OSM editor? I don't think it's a serious issue about the 10-minute delay Logictheo 09:42, 15 April 2009 (UTC)

Wrong behavior of XAPI 0.6

The Bugreport is here

The docu Osmxapi#Relations says: 

This returns an xml document containing relations that match the search terms. 
For each matching relation the nodes and ways referenced by that relation are
also returned.

The output do not contain the referenced nodes and ways! for example: http://osmxapi.hypercube.telascience.org/api/0.6/relation[bbox=9.5,52,9.6,52.1][type=restriction] --Langläufer 14:27, 4 May 2009 (UTC)

I like that behaviour in some cases. When I want to change tags on some relations, I don't need to download all the related objects (it could be imposible to open that large file in JOSM). Could this be kept as an option?
There is probably another bug in xapi. I am not quite sure if it is bug of Xapi or bug in JOSM. There is problem when I upload changes in files from Xapi in JOSM. I reported this bug in JOSM track, but it coud be bug in Xapi.--Petr Dlouhý 21:30, 8 May 2009 (UTC)
Yes, I've noticed this as well. For one thing I'm doing, the new behaviour is actually useful, since it makes the file smaller and easier to work with. However, for other applications, I can see it would be highly desirable to include the referenced objects to save users having to make multiple requests. Might I suggest that the XAPI could provide both types of output, with an option to allow users to choose which they'd prefer for a given application. Maybe /relation and /relation_with_members ? -- Rjw62 12:14, 11 May 2009 (UTC)
So how it is in the moment it is a problem for my Query-to-map project. Please fix it. For some things it would be really nice to get only the IDs or only the names of the elements(e.g. for streetlists) but this shouldn't be the standard behaviour. --Kolossos 08:40, 12 May 2009 (UTC)

is fixed now.

Bugs

The http://www.informationfreeway.org/api/0.6/ seems to delivering in the moment no answer. --Kolossos 10:25, 31 May 2009 (UTC) It run's again. --Kolossos 11:37, 31 May 2009 (UTC)

The http://www.informationfreeway.org/api/0.5/ and 0.6 seem to be down. added a ticket to track. --mikecanann 16:25, 8 Jun 2009 (UTC)

Servers down?

Sorry that I post my question here, but it seems that no mirror server is currently running. http://www.informationfreeway.org/api/0.6/ directs to http://osm.bearstech.com/osmxapi/api/0.6/ and bearstech returns "ERROR 501: Internal Server Error." The other mirrors also return error codes. Using the openstreetmap server directly does not work, because my request is too big. So how do can I access the API? See also my posting at: http://forum.openstreetmap.org/viewtopic.php?id=3809 --Flomigulau 19:06, 29 June 2009 (UTC)

I notice that when I pull from a server, it often bounces me to another server which may or may not be overloaded. Let's face it, "www.informationfreeway.org" is easier for most people to remember than "osmxapi.hypercube.telascience.org" or "osm.bearstech.com". Servers change, and a server may be coded into JOSM or Merkaartor, which suddenly stops working, causes panic and millions of new downloads of the gui client.
Instead of the current system, maybe someone with more technical expertise than I can figure out logistically a way of setting up xapi.openstreetmap.org as a "Tier 1 server," and the other listed servers as "Tier 2 servers," both pingable with their name and a round-robin DNS name (2.xapi.openstreetmap.org or something). Anyone pledging enough bandwith and storage space can become a Tier 2 server, and from that, it may be easier to have some continent- or country-specific servers, like the pool.NTP.org server hierarchy.
The main XAPI address for most users would be 2.xapi.openstreetmap.org, or a more specific server if needed, and server load would be automatically distributed more evenly across the various servers, and even make it easier on most of the developers. Tahongawaka 01:51, 28 October 2010 (BST)

New Functions

I would like to see the XAPI serving requests like:

All results as API in OSM format. This could take load off of the API. Thanks. --Gary68 07:21, 26 August 2009 (UTC)

Zip Compression

I would like to see the XAPI service provide compressed OSM files. The current raw .osm is heavily compressible and this would mean users get the data quicker, using less bandwidth. Admittedly the extra compression would require processing resources at the server, though this could be cached to an extent to avoid unnecessary repeats. --Pinkduck 16:23, 7 November 2009 (UTC)

Me too. I like this idea. Could it also be possible to create the osm-file, compress it and email this file? Or store the file on a server (for 6 hours) and mail me if ready to download. So discontinued downloads can be startet again if fail. Bandwith is not my problem. But I have to wait/stay online during download. --Bahnpirat 17:04, 9 November 2009 (UTC)
If a webserver is configured correctly, and the sysadmin knows what he's doing, it should be able to compress data on the fly. Currently only http://osmxapi.telascience.org and http://open.mapquestapi.com/ send a compressed file if your client asks for it. I use wget to retrieve my osm data, so my command looks like this: 
wget --header="accept-encoding: gzip" -O MyData.osm.gz 'http://open.mapquestapi.com/xapi/api/0.6/map?bbox=-122.25,37.31,-121.85,37.77'
Without the compression, the server is serving up a 130+MB file. With server side compression, the file is less than 12MB. On the fly compression uses very limited cpu power on modern hardware, and the reduction in bandwidth usage combined with getting rid of the client faster more than makes up for compression. --Tahongawaka 16:33, 18 April 2011 (BST)

Revision updating sucks

I just happened to find that my POI control program [1] did not work anymore because it called rev 0.5 which was meanwhile updated.

OK, I could implement some commands which first load this Xapi page, search for the current link to informationfreeway.org/api/???/ and use the result for the xapi call, but IMHO it would be much better if the xapi call would work without the knowledge of the current revision number. --Plenz 04:30, 18 November 2009 (UTC)

Revision-numbers are the only thing that saves you from breaking the map due to concurrent edits. It's a feature, not a bug. --MarcusWolschon 07:09, 18 November 2009 (UTC)
OK, I see the point regarding editing. But I am talking about reading from the database.
I am used to develop my programs until they work perfectly and then I like to forget them completely. But if I understand this subject correctly, I am forced to care forever that my program always calls the correct xapi version. And I hate to do such stupid things which a simple program can do as well. Probably also the guy feels stupid who has to update this wiki page after every xapi update.
Anyway, it would help me a lot if there would be a "versionless" call which provides the current version number. --Plenz 14:42, 18 November 2009 (UTC)
I don't think Plenz is referring to the revision numbers of objects in the database, but the version of the API (currently 0.6). This version number is good practice since the format of API requests and responses may change from one version to another. The change in URL forces tool authors to update their tools, providing a useful reminder that they should also check that their tools still work properly with the new API version. Rjw62 14:10, 30 November 2009 (UTC)

Well, meanwhile I found by myself what I need. http://api.openstreetmap.org/api/capabilities gives me the current API version number which is hopefully also always valid for XAPI. --Plenz 14:55, 7 December 2009 (UTC)

Add links to deal with limitations

I would like to see links on this page to show up method how you can deal if you need more predicates or need a "*" in the key.

--Lulu-Ann 08:21, 19 December 2009 (UTC)

Relation search is not working

It seems, that relation search is not working. The file it produces is not ended, and is not containing any relation.--Petr Dlouhý 16:36, 20th December 2009 (UTC)

For me it works again --Langläufer 10:35, 21st December 2009 (UTC)
Yes, it works now. Isn't possible to download only relations (not ways and nodes) as it was before?
It is not possible, before it was a bug. --Langläufer 22:21, 24th December 2009 (UTC)
But it could be useful in some cases - save my time and Xapi's traffic (when no need for ways and nodes).--Petr Dlouhý 08:47, 27th December 2009 (UTC)
I am downloading nodes, ways and relations - when all I want to do is fix the relation tagging. Please add an option to support returning only the requested element type. --Pink Duck 11:29, 16 February 2010 (UTC)
Oh yes, that would be really helpfull. --MarcusWolschon 05:33, 17 February 2010 (UTC)
+1 from me! Lulu-Ann

Functions to GET number of objects

Is there a way to get number of objects (polylines, polygons, points) located within given bounds without actually downloading XML document with all these objects? My application has limit on number of objects within single map. I would need API function to get number of objects within given bounds, to decide whether to download map or to divide it into several smaller maps. If there is no such API function, could it be added, please? --Dusan January 2010

Variable output precision

for details and work-around scripts see http://josm.openstreetmap.de/ticket/5031

when opening .osm files downloaded then exported using JOSM, Merkaartor, and Xapi I have noticed some differences that, when mixed, cause the JOSM Validator to fail to detect duplicate nodes. It's an ascii representation + floating point precision issue.

  • JOSM seems to export with %.9g (~1cm)
  • Merkaartor seems to save with %.8f (~1mm)
  • Xapi seems to export with %.7f (~1cm)

I would argue that %.7f is no good for Xapi as it loses data from survey grade RTK GPSs. (~2-3mm, which needs %.8f)

I suggest for JOSM, Merkaartor, and Xapi to all agree on a single print format, and for that format to be %.8f (slightly beyond what survey grade RTK GPS will give you, but no more as to avoid cluttering everyone's disks with geo-noise).

--thanks, Hamish 02:53, 16 May 2010 (UTC)
The database currently stores coordinates as integers (real value multiplied by a million): that means that there isn't anything more to return. Editors need to work with better precision (for extrude etc. to work correctly), but the validator should probably take the db resolution into account separately - IIRC it's even a constant somewhere in the JOSM source. Alv 07:13, 16 May 2010 (UTC)
fwiw API_v0.7#Better_precision says 10m * coord then rounded, which if I did my sums correctly means Xapi's '%.7f' is already correct to represent the full precision stored in the back-end DB (10m == 1e7), and JOSM+Merkaartor need to be brought into sync. --cheers, Hamish 02:52, 17 May 2010 (UTC)

lossy compression

It would be cool to have lossy compression. If I want to display the highway network of this planet XAPI will give me the data. But there are too much data and the query will not work.

An easy lossy compression for ways would be nice. The easyest way would be to remove every second node of a way. At the highest compression level only the start and the end point would describe a way.

--?

I think this is more normally described as "Simplification" of geo data rather than compression. I wrote a SimplifyPlugin for osmosis which does it.
Could be a nice feature for XAPI. Actually PostGIS has simplification functions, so there may be a way of implementing this within jXAPI. Probably is quite a resource intensive process though
-- Harry Wood 01:30, 15 November 2011 (UTC)

multiple tag-values

Multiple values for tags are given in the form of 

tag=value1;value2

using XAPI with node[tag=value1] don't give me those nodes. Actually there seems to be no way to get all nodes which have 'value1' as an value of 'tag'. Requests like node[tag=*value1*] dont't work. So you should ad any possibility how to get all nodes witch have 'value1' as an value of 'tag'. One possibility would be node[tag=~value1]. -- MichaelSchoenitzer 16:11, 28 August 2010 (BST)

A way to down relations without its members

Is there a way to download relations without getting all its members also? If not, I think it would be a good thing to have such an option.

there isn't, and yes would be nice, see former talks. --Langläufer 16:38, 16 November 2010 (UTC)

improvement requests

based on my blog post here are a couple of suggestion how i would like to see xapi improved:

  • possibility to combine filters like modules. for example:
    • [bbox=…]
    • [elementtype=node|way|relation] --> so i can, for example, query nodes and ways but not relations, in one request
    • multiple use of [property=value1|value2|valueN] --> for example: …[amenity=bar|pub|fast_food|restaurant][cuisine=italian][payment:notes=yes]… would select all ways or nodes taggted with (amenity=bar OR pub OR fast_food OR restaurant) AND cuisine=italian AND payment:notes=yes
    • [last_edited_by=…], [last_edited_by_id=…]
    • [created_by=…], [created_by_id=…]
    • [changeset=…]
    • [created_at=YYYY-mm-ddTH:i:s], [created_from=YYYY-mm-ddTH:i:s], [created_until=YYYY-mm-ddTH:i:s] (if time is set to "00:00:00" all elements are returned from that day)
    • [last_changed_at=YYYY-mm-ddTH:i:s], [last_changed_from=YYYY-mm-ddTH:i:s], [last_changed_until=YYYY-mm-ddTH:i:s]
    • [limit=…] limit the result to the first N elements
  • [count=true]: if this filter is used i don't get the actual result, but the sum of all elements for the query. this way clients can see if a query would return to much information (for example in highly mapped areas) and could further fine tune the query before fetching the actual result
  • [order_by=…]: certain properties like "last_changed_at" should be usable for ordering the result
  • in addition to a bbox value, an alternative notation should be possible where i send latitude and longitude and a radius (in m). the result set could then include the calculated distance from the coordinates to each element in the result
  • possibility to exclude certain tags and properties from the result to further reduce the size of the xml
  • [property LIKE value1]: for example [note LIKE reviewed] would return all elements with a note tag which has the word "reviewed" somewhere in it
  • in addition to XML the result should also be offered in JSON

i know, certainly a lot of this filtering and ordering could be done on the client, but mobile devices, for example, have limited resources and battery power. therefore as much load as possible should be handled by the server. further a optimized and reduced XML means less wasted bandwidth for the user and the OSM servers.

Flaimo 19:53, 14 January 2011 (UTC)

JXAPI - Bug with relations query

JXAPI gives no results where it should. I think there is a bug in indexing relations.

but mostly no answer!

  • Same for mapquestapi

--Langläufer 18:41, 12 April 2011 (BST)

JSON Output

JSON output sure would be nice for those of us doing web/browser apps Brycenesbitt 23:03, 27 May 2011 (BST)

Changes to syntax

Please don't change the documented output of Xapi without first discussing it. If an implementation isn't correctly working as described then it isn't compatible with the Xapi syntax. Something like @meta is not part of the Xapi syntax. Pnorman 08:39, 9 November 2012 (UTC)

I have created a section for implementation-specific details for where an implementation differs from the Xapi spec listed here. I'm more familiar with the jxapi quirks and bugs so I'm sure the overpass section is not complete. Please add any bugs or added features. Pnorman 09:24, 9 November 2012 (UTC)

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Talk:Xapi&oldid=1977885"