Redaction bot progress map
The redaction bot progress map showed the progress of the redaction bot. It was live updating during the redaction bot run. It is now showing as complete.
Display success regions except if there was no redactions - This display will make your browser struggle.
Display success regions including no redactions - This display will probably break your browser. Too many regions to show
Redaction bot?
The redaction bot was an automated license change data processing operation which was run in 2012 to rebuild the OpenStreetMap data in a "clean" license state ready for transition to the Open Database License. See the licence change codebase for details of how it worked.
What the map shows
Nowadays it's all looking green but originally the map showed...
- where the bot is currently (in yellow) - Meaning a log file was initiated and appears to not be concluded yet.
- where it hit some errors (in pink) - These are often timeout errors or changeset failures where some individual objects require more processing.
- where it failed (in red) - This means the bot explicitly labelled the region as failed. The whole thing needs doing again.
- where it is still left to do (in blue) - calculated by looking at all possible regions, and subtracting those which seem to be done.
- where it has run (in green) - But only if you select one of the alternate views linked above. Click to see the summary information spat out at the end of the log file
- where it was re-run (in darker green) - This means it was successfully and completely processed after earlier problems
How is this useful?
It's not useful any more, other than as an interesting historical record of one of the biggest technical challenges OpenStreetMap's core developers overcame.
At the time it was useful to see where the bot was processing in a live-updated fashion because the community was generally asking and answering a lot of questions about what these edits meant on the rebuild mailing list and in local community discussions around the world. Also we were asking people to avoid editing in the areas where it might clash with the redactions bot. It also helped the tech team who were running the bot see what was happening, and how quickly it was progressing (It was decided to run several bots in parallel which showed up as multiple yellow squares on the map)
Development
Mostly developed by Harry Wood but input from others is very welcome (or alternative views on the same data).
Some tweaks so far from Jongleur, TomH, and Cobra.
...and it's using OpenLayers.
How it works
It's all based on the bot log files...
http://gravitystorm.dev.openstreetmap.org/redactions/logs-live/
...but there's a ruby script running every minute which "distills" these into a JSON file:
http://harrywood.dev.openstreetmap.org/license-change/distilled-logs.json
It then aggregates them by region, to give the information the display needs. There's a new smaller file not containing all the 'success' region info:
http://harrywood.dev.openstreetmap.org/license-change/nosuccess.json
And the original json file with all success info is here (and still powers the alternate views):
http://harrywood.dev.openstreetmap.org/license-change/botprocessing.json
The map display loads this in as a script include and processes the json in javascript. Although a .php file, there's very little php logic involved.
See also
- User:OSMF Redaction Account - The User Account against which edits are made by the bot
- OSM Inspector Redaction Bot view by geofabrik
- ToeBee's redaction map - showing nodes touched by the bot