Tiles@home/Dev/Client
Development base for the tiles@home project
Better PNG compression
The clients should return fully compressed PNG images. The tiles covering the capitol of Norway, Oslo, is 23,6% smaller when fully compressed.
3x3 12z tiles 12-17 zoom-level:
Total : 20,8MB -> 15,9MB : -23,6%
Smallest: 908B -> 719B : -20,8%
Largest: 61592B -> 41382B : -38,8%
The reason for the smaller files, it that a lot of program have sloppy PNG implementation. The standard specifies that a program tries multiple ways of storing the pixels, while a lot of program only use method NR 0.
PNGcrush is a program that re-compresses PNG files. In my opinion pngcrush should bee run in the upload.pl.
Below is the cange to the perl code inside the while loop that moves tiles in the gather dir.
From:
my $Filename1 = "$TileDir/$file";
my $Filename2 = "$TempDir/$file";
if($file =~ /tile_\d+_\d+_\d+\.png$/i){
$Size += -s $Filename1;
$Count++;
To:
my $Filename1 = "$TileDir/$file";
my $Filename2 = "$TempDir/$file";
my $FilenameC = "$TileDir/pngc/$file";
if($file =~ /tile_\d+_\d+_\d+\.png$/i){
if ( -e "/usr/bin/pngcrush" ) {
my $CommandC = sprintf("%s %s %s",
"pngcrush -q -d ",
"$TileDir/pngc",
$Filename1);
`$CommandC`;
rename($FilenameC, $Filename1);
}
$Size += -s $Filename1;
$Count++;
The changes will only have an affect if you have the file /usr/bin/pngcrush. So this code can be put into the svn, without breaking the installation for the people that do not have pngcrush installed.
--Mikemenk 21:55, 25 February 2007 (UTC)
Pngcrush implemented in Frederikssund version 13 April 2007, in tilesGen.pl during splitting of tiles. -- Dutch Dutch 12:39, 13 April 2007 (BST)
Map projection code needs extensive peer review
- Mercator projection is used for the slippy map, see slippy map tilenames for details
- Trig functions are used in the client to decide where to split one tile into its 4 components
Issues
- Tiles generated at different zoom levels don't match-up exactly in y-coordinates, this is causing visible kinks in a road as it crosses border between some tiles
- Osmarender itself only does an approximation of mercator, so is stretching osmarendered images over a mercator tile valid?
- Do we actually need to use mercator when splitting tiles, rather than dividing image into exact quarters?
Rendered area
When rendering, the SVG map data does not completely match the area being exported as PNG, leaving a white border at the top or bottom of each tileset
Places to look (in the tilesgen script):
- addbounds() puts the lat/long limits of the tile into the osmarender file, so that osmarender only outputs the exact area - if that failed in such a way as to make the north boundary too low, it would cause it
- getsize() finds the size of the SVG file in mm, and uses that to divide up the image into PNG outputs. If that failed in such a way as to overestimate the height of the tile, or to incorrectly determine the tile's top/bottom edge, that could also cause it
Config file
- Put all the config options in a hash instead of named variables, and have a better detection program for invalid configs
- Add port (range), where is client listening. Its for PCs behind firewall, proxy
Library
Functions used by both tilesgen and upload need to be shared
Filenames
Rename stuff so that tab-completion works (don't let everything start with "tiles")
Error checking
More error-checking required at every stage, to detect common errors
Download osmarender
Check for an updated version of osmarender on SVN and download it
Configurable maximum zoom
"Stop at zoom 17" is currently in the config file - that should really be controlled by the server...
Request server in config file
URL for tile requests is hardcoded - put it in config file
Separate password and general config files
The file with usernames/passwords should be separate, so that new versions of a config file with URLs etc. can be downloaded from the server
A separate authentication.conf exists for quite some time
Rendering on n-core CPU
- Detect number of cores
- Create "slot" folders in TEMP. For example folder_for_rendering_core1 and render there.
- No more than one running Inkscape instance. This should remove Out of memory issue.
Fit work for CPU
- reverse rendering order of layers (17,16,...). Simple to implement, better workload for CPUs on multicore and fork>0. Earlier detection of render-problems.
some cpus are slower then any other. This can cause delays for rendering tiles.
- Benchmark CPU and save result to file or server ?!
- Then request more or less complex tiles to render.
Reduce downloaded osm size
Can you download gziped or ? 7ziped ? osmfile like JOSM ? then unpack and render ?
Showing timestamps in screen output
It would be nice to see the timestamp of an operation in screen output auf tilesGen.pl.
Example:
[#9 100% default 08-03-26 10:39:00] Finished 995,513 for layer default [#9 100% maplint 08-03-26 10:39:12] Finished 995,513 for layer maplint
80n's work In Progress
user:80n is working on the following:
Allow multiple instances of tiles@home to be run on one machine- implementedSkip the render step for blank tiles, upload a standard empty tile- implementedUse different osm-map-features.xml files for each zoom level- implementedReduce edge effects on tile boundaries- implementedHandle tiles that are too complex to download by downloading a set of smaller .osm files- implemented- Make easier to use on Windows - thinking about this
Fred's Work In Progress
user:Frederik Ramm is working on the following:
Fix projection issues
- Idea: Fix projection problems by applying projection to OSM file before stuffing it into osmarender.
- Status: Perl projection script working, osmarender4 works with modified OSM input
- Next: on hold; not terribly urgent since deelkar has improved tielsGen.pl (rendering stripes & cutting from them)
- Pitfalls: Using this method we will achieve a correct projection even within a tile which is not the case at present. This may lead to problems where old and new tiles meet. Level-11 and smaller will not have correct projection unless some image distortion is applied when creating lowzoom tiles.
Use osmarender4 with tiles@home
done
Get tilesPipe@home to work
- Idea: Ultimately, with osmarender4 and proper projection, switch over to tilesPipe@home which should be orders of magnitude faster than the current solution. Current solution calls inkscape about 60 times during rendering of a dense tileset. tilesPipe@home runs inkscape only once per zoom level.
- Excellent idea, but my concerns: (1) Would making one pass through inkscape mean extra memory overhead, limiting the number of client PCs that could otherwise participate? (2) Does that mean we'll have to change the name of the whole project from Tiles@Home to TilesPipe@Home? --Larry Gilbert 19:19, 23 December 2008 (UTC)
Beziercurve hinting of ways
In progress, testing at the moment. Example of output :
Dutch 20:41, 2 March 2007 (UTC)
Minor Bug
I get this Warning consistantly with version 20055 (Ulm):
Warning: used uninitialized fault inkscape
This appears to come from a missing line
initFault("inkscape")
in the main code. At least I notice that adding this as line 247 in tilesGen.pl solves the issue. Apparently there is no real fault, but the warning is only triggered by an unmatched resetFault("inkscape") and not by an addFault("inkscape"); so as an additional feature request: It might be useful to have different warning messages in line 178 and line 198 of lib/tahlib.pm.--hagman 10:48, 21 February 2010 (UTC)
- the fault was renamed as "rasterizer" so any reference to "inkscape" should be changed. --Deelkar (talk) 15:47, 21 February 2010 (UTC)
What's my fault?
Currently, there appears to be something very wrong: "More OSM Coming soon" on a proviously working area ([1]) - and it appears to be my fault: [2]
How can such be corrected? (And I also noticed a giant "Unknown Type" patch at z=6 south of Berlin today [3], though at least that was not my fault(?))--hagman 20:57, 21 February 2010 (UTC)