Osmosis

From OpenStreetMap Wiki
Jump to navigation Jump to search
Osmosis - Other languages
Sub pages
Installation
Quick install (Windows)
Detailed usage
Examples
Benchmarking / tuning
Development
Writing plugins
Notice

Osmosis is in light maintenance mode with Osmium as a recommended alternative.

See the announcement 1 and announcement 2 on the mailing list.

Osmosis is a command line Java application for processing OSM data. The tool consists of pluggable components that can be chained to perform a larger operation. For example, it has components for reading/writing databases and files, deriving/applying changes to data sources, and sorting data, (etc.). It has been written to easily add new features without re-writing common tasks such as file and database handling.

Some examples of the things it can currently do are:

Osmosis can also be included as a library in other Java applications (including Android apps).[1]

Current status

Osmosis is in a state where the majority of the codebase is unlikely to change drastically. New functionality will be introduced over time in the form of new tasks and plugins but existing functionality is unlikely to be removed. All bug reports welcome.

The current released version only supports the API 0.6 OSM data model.

Downloading

Latest stable version

The stable release below should be used whenever possible.

Older versions

API 0.5 support

The 0.35 release was the last release to support API 0.5.

DB schema

Main article: Osmosis/PostGIS Setup

Database schema creation scripts matching the production API are bundled with the Osmosis distribution within the script/contrib directory.

Database schema creation scripts for the Osmosis PostGIS "simple" schema and "snapshot" schema (osmosis 0.39 ff) are bundled with the Osmosis distribution within the script directory.

Source code

The Git repository is hosted on GitHub at: https://github.com/openstreetmap/osmosis

(More details can be found on the Osmosis/Installation instructions sub-page.)

How to install

Linux

Follow the instructions in Osmosis/Installation#Linux

OS X

Follow the instructions in Osmosis/Installation#OS X

Windows

Follow the instructions in Osmosis/Quick Install (Windows). Note that since October 2015, you need to manually fix the osmosis.bat file, instructions are given.

Usage

Beginner's guide

A few important things to note for the newbies:

  • Osmosis is a Java tool, so it may be important to know about java environmental settings,
  • keep in mind that argument order matters,
  • to avoid trouble don't process compressed files,
  • check the examples first as the detailed usage guide is a bit rough on beginners.

Detailed usage

Full details of all tasks and their usage are available at the following links:

Usage details for older versions are available below:

To maximize performance, refer to Tuning and Benchmarking.

There is also a frontend for the use of Osmosis named OSMembrane.

Example usage

(More on the sub-page Osmosis/Examples.)

An example to extract the speed cameras from an area: 

osmosis --read-pbf \myregion.pbf --node-key-value keyValueList="highway.speed_camera" --write-xml radar.osm

What if you only want all highways ?

Use the tag-filter parameter the following manner: 

osmosis --read-xml city.osm --tf accept-ways highway=* --used-node --write-xml highways.osm

Extract only those ways that have certain tags. Specifying --used-node only extracts those nodes that are used in the filtered ways. Relations stay untouched.

In this example, our output will be a file with ways tagged with railway=tram or railway=tram_stop. 

osmosis --read-xml city.osm --way-key-value keyValueList="railway.tram,railway.tram_stop" --used-node --write-xml city_tram.osm

On larger files (e.g. a planet file), this may result in a segfault due to memory limitations. In this case, passing idTrackerType=BitSet as a parameter to --used-node may help: 

osmosis --read-xml city.osm --way-key-value keyValueList="railway.tram,railway.tram_stop" --used-node idTrackerType=BitSet --write-xml city_tram.osm

But what if you want to extract all highways except one or 2 types ? This command will output all highways except those tagged with motorway or motorway_link 

osmosis \
  --read-xml input.osm \
  --tf accept-ways highway=* \ 
  --tf reject-ways highway=motorway,motorway_link \
  --tf reject-relations \
  --used-node \
  --write-xml output.osm

Export a planet file from a local PostgreSQL rails port database. 

osmosis --read-apidb host="x" database="x" user="x" password="x" --write-xml file="planet.osm"

Derive a change set between two planet files. The first planet file is the file after changing, the second planet file is the file before changing. 

osmosis --read-xml file="planet1.osm" --read-xml file="planet2.osm" --derive-change --write-xml-change file="planetdiff-1-2.osc"

Derive a change set between a planet file and a local PostgreSQL rails port database. 

osmosis --read-xml file="planet1.osm" --read-apidb host="x" database="x" user="x" password="x" --derive-change --write-xml-change file="planetdiff-1-2.osc"

Apply a change set to a planet file. 

osmosis  --read-xml-change file="planetdiff-1-2.osc" --read-xml file="planet1.osm" --apply-change --write-xml file="planet2.osm" 

or

osmosis  --read-xml file="planet1.osm" --read-xml-change file="planetdiff-1-2.osc" --apply-change --write-xml file="planet2.osm" for the older version

Sort the contents of a planet file. 

osmosis --read-xml file="data.osm" --sort type="TypeThenId" --write-xml file="data-sorted.osm"

The above examples make use of the default pipe connection feature, however a simple read and write planet file command line could be written in two ways. The first example uses default pipe connection, the second explicitly connects the two components using a pipe named "mypipe". The default pipe connection will always work so long as each task is specified in the correct order. Each task pushes its output stream(s) onto a stack, and each task that requires input stream(s) takes them from the stack. This stack-based approach is the reason why the above "--apply-change" example first reads the change file, then the data file - the --apply-change task will then first retrieve the data file from the stack and then the change file. 

osmosis --read-xml file="planetin.osm" --write-xml file="planetout.osm"

osmosis --read-xml file="planetin.osm" outPipe.0="mypipe" --write-xml file="planetout.osm" inPipe.0="mypipe"

Extract an area based on a Osmosis/Polygon Filter File Format file: 

osmosis --read-xml file="planet-latest.osm" --bounding-polygon file="country2pts.txt" --write-xml file="germany.osm"

Note that --way-key-value and --node-key-value cannot be used in one run. Instead, create two (or more) files in several runs and merge them together afterwards (yes, for n files you need n-1 merge commands): 

osmosis --rx 1.osm --rx 2.osm --rx 3.osm --merge --merge --wx merged.osm

What if you want restaurants in a particular town or country ? Since restaurants can be mapped as nodes and ways, you'll have to run 3 commands: One to extract the nodes tagged as amenity=restaurant, one to extract the ways tagged amenity=restaurant, and third command will merge them together: 

osmosis --read-xml input.osm --tf accept-nodes amenity=restaurant --tf reject-ways --tf reject-relations --write-xml output-nodes.osm

osmosis --read-xml input.osm --tf accept-ways amenity=restaurant --tf reject-relations --used-node --write-xml output-ways.osm

osmosis --rx output-ways.osm --rx output-nodes.osm --merge --wx merged.osm

Extracting bounding boxes

One of the basic use cases is extracting a subset of OSM XML from a large downloaded file. You might have downloaded the world OSM (planet.osm) and want to extract some region, or maybe you have a country extract downloaded but only want one city. The simplest way to do that is by extracting a rectangle.

An example that extracts Nürnberg and some area around it from Germany - or maybe world - OSM: 

bzcat downloaded.osm.bz2 | osmosis  --read-xml enableDateParsing=no file=-  --bounding-box top=49.5138 left=10.9351 bottom=49.3866 right=11.201 --write-xml file=- | bzip2 > extracted.osm.bz2

This approach uses bzip2 directly for extracting and compressing, as it is claimed to be more effective than the built-in archiver.

The easiest way to get coordinates is to use the map on www.openstreetmap.org. Double click the spot you want coordinates for, then click "Permalink" and grab the data from the URL field.

Windows notes

bzcat is the tool to use even on Windows if you want to have good performance. However, /dev/stdin is not a valid path on Windows. Use CONIN$ instead.

Use in Android

Add following maven repository in your root build.gradle file:

allprojects {
		repositories {
			...
			maven { url 'https://jitpack.io' }
		}
	}

Add the dependency to your module-level build.gradle file (choose the required subproject according to your needs):

dependencies {
    implementation 'com.github.openstreetmap.osmosis:osmosis-apidb:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-areafilter:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-core:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-dataset:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-extract:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-hstore-jdbc:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-osm-binary:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-pbf:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-pbf2:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-pgsimple:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-pgsnapshot:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-replication:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-replication-http:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-set:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-tagfilter:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-tagtransform:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-testutil:0.48.0'
    implementation 'com.github.openstreetmap.osmosis:osmosis-xml:0.48.0'
}

Getting help and reporting bugs

The best place to get answers to questions on osmosis usage is the osmosis dev mailing list.

https://lists.openstreetmap.org/listinfo/osmosis-dev

Please discuss potential bugs on the dev mailing list before creating new tickets. If you're confident you have discovered a bug, it can be logged in osmosis project on github:

https://github.com/openstreetmap/osmosis/issues

See also

Development

Details on osmosis development are available at Osmosis/Development. Visit there for documentation on the internal structure of osmosis, design decisions, steps to configuring development environments, etc.

Notes

The minimum supported Java version is 1.6. Osmosis makes use of some java.awt.geom classes that only appeared in 1.6, if these are re-written to use 1.5 functionality it may be possible to return to the previous minimum of 1.5. Osmosis utilises generics and java.util.concurrent features requiring Java 1.5 as a minimum.

The built-in bzip2 (.bz2) performance is much slower than the gzip (.gz) built-in Java one (Java gzip utilizes native code to improve performance). However, the trade off is that bzip2 creates much smaller compressed files. Also recent planet downloads use features of bzip2 which are not supported by the built-in bzip2 library (streams). For working with these, and for faster processing of any larger files, it is suggested to use your platform's native bzip2 implementations and read and write files from /dev/stdin and /dev/stdout respectively. 

pbzip2 -d -c planet.osm.bz2 | bin/osmosis --rx /dev/stdin --wx /dev/stdout | gzip -c > planet-same.osm.gz

See also /Benchmarking

Under Win 7 64 bit it may be necessary to run the .bat file from a command prompt. The underlying osmosis.bat file may need to have the full path name for java.exe inserted.

Alternatives to osmosis

  • Imposm
  • OGR - OGR OSM driver with ogr2ogr
  • Osm2pgrouting
  • Osm2pgsql - can also do imports of osm file to postgres DB with postgis extension
  • Osmium, special fast C++ data processor can also do imports of osm file to postgres DB with postgis extension.
  • openstreetmap_h3 a high performance tool for importing OSM PBF files into PostGIS databases or into Big Data ecosystem via Apache Arrow data format. This project split planet dump Geo data by H3 indexes into many partitions to simplify world wide data Geo analysis/aggregation and routing tasks.

References

  1. ↑ https://lists.openstreetmap.org/pipermail/dev/2011-February/021804.html
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Osmosis&oldid=2523369"