User:Mmd/Planet import challenge 22
< User:Mmd(Redirected from User:Mmd/planet-220829)
Jump to navigation
Jump to search
Runtime comparison
Using lz4 map file compression
| Version | Overpass API 0.7.58.5 | Overpass API 0.7.59.120 (mmd) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Commit | 766266ec136f2e49ec5243f83d8741f3f6748bbb | 3b1e7b34b4c5b69918b9f61a3c5d642477db8c83 | a4479f84bb767813eacf6b3443e167af75e32eab | 465a3fef2f53ba559366fb62ace07aaa9cca8df4 | 9ef1e5b23a0f7ffcc1393a5498ccdb09318f1392 | 9ef1e5b23a0f7ffcc1393a5498ccdb09318f1392 | 9ef1e5b23a0f7ffcc1393a5498ccdb09318f1392 | ||
| flush size = 16 (default) | flush size = 64 | flush size = 16 (default) | flush size = 64 | flush size = 64 | flush size = 64 | flush size = 48 | flush size = 32 | flush size = 64 | |
| User time (seconds) | 99982.26 | 87259.37 | 55720.12 | 44965.89 | 34340.63 | 33374.36 | 34697.80 | 37377.67 | 33270.08 |
| System time (seconds) | 3753.62 | 3176.61 | 4088.99 | 3553.10 | 3382.84 | 3424.39 | 3366.12 | 3512.75 | 3375.83 |
| Percent of CPU this job got | 95% | 94% | 158% | 154% | 137% | 136% | 139% | 141% | 137% |
| Elapsed (wall clock) time (h:mm:ss or m:ss): | 30:12:17 | 26:32:17 | 10:29:47 | 8:42:09 | 7:36:42 | 7:28:18 | 7:35:50 | 8:03:07 | 7:22:40 |
| Maximum resident set size (kbytes) | 7958192 | 27284816 | 7849728 | 23197216 | 24712892 | 24766280 | 20856304 | 12998596 | 22568672 |
- Overpass API 0.7.58.5:
osmium cat ~/planet/planet-220829.osm.pbf -f xml | /usr/bin/time -v ./update_database --db-dir=/db --meta --compression-method=lz4 --map-compression-method=lz4 - Overpass API 0.7.59.120 (mmd):
/usr/bin/time -v src/update_database --db-dir=/db --use-osmium --meta --parallel=10 --initial-load < ~/planet/planet-220829.osm.pbf - Overpass API 0.7.59.120 (mmd) flush size 64:
/usr/bin/time -v src/update_database --db-dir=/db --use-osmium --meta --flush-size=64 --parallel=10 --initial-load < ~/planet/planet-220829.osm.pbf
Resulting database size: about 230G
With no map file compression
| Version | Overpass API 0.7.58.5 | Overpass API 0.7.59.120 (mmd) |
|---|---|---|
| Commit | 465a3fef2f53ba559366fb62ace07aaa9cca8df4 | |
| flush size = 16 (default) | flush size = 64 | |
| User time (seconds) | 95595.48 | 31566.81 |
| System time (seconds) | 9238.92 | 8212.37 |
| Percent of CPU this job got | 86% | 107% |
| Elapsed (wall clock) time (h:mm:ss or m:ss): | 33:32:58 | 10:14:22 |
| Maximum resident set size (kbytes) | 7490992 | 26060352 |
- Overpass API 0.7.58.5:
osmium cat ~/planet/planet-220829.osm.pbf -f xml | /usr/bin/time -v ./update_database --db-dir=/db --meta --compression-method=lz4 --map-compression-method=no - Overpass API 0.7.59.120 (mmd) flush size 64:
/usr/bin/time -v src/update_database --db-dir=/db --use-osmium --meta --flush-size=64 --parallel=10 --initial-load --map-compression-method=no < ~/planet/planet-220829.osm.pbf
Resulting database size: 258G - 263G
CMake options
- CMAKE_CXX_FLAGS_RELWITHDEBINFO:
-O2 -g -DNDEBUG -ftree-vectorize -march=native -flto - CMAKE_CXX_COMPILER:
/usr/bin/g++-12 - CMAKE_EXE_LINKER_FLAGS:
-flto -fwhole-program - USE_UNITY_BUILD: ON
- Compiler: g++-12 (Ubuntu 12.1.0-2ubuntu1~22.04) 12.1.0
- Running on Ubuntu 22.04
Hardware
- Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz, 4 cores, 8 threads
- 64 GB RAM (4x RAM 16384 MB DDR4 ECC)
- 2x SSD SATA 480 GB Datacenter (Micron 5100 MTFDDAK480TBY), RAID 0
- Cost: 40€/month
Planet file stats
Name: planet-220829.osm.pbf Format: PBF Compression: none Size: 69869926964 Timestamps: First: 2005-05-21T21:03:22Z Last: 2022-08-28T23:59:56Z Number of changesets: 0 Number of nodes: 7882127434 Number of ways: 883613025 Number of relations: 10176596 Smallest changeset ID: 0 Smallest node ID: 1 Smallest way ID: 37 Smallest relation ID: 11 Largest changeset ID: 0 Largest node ID: 9984322083 Largest way ID: 1089962152 Largest relation ID: 14514635