Humanitarian OSM Team/Haiti Strategy And Proposal

From OpenStreetMap Wiki
Jump to navigation Jump to search

OpenStreetMap has become an essential mapping resource for the response in the Haiti earthquake. OSM consists of a global volunteer community willing to dedicate substantial time and experience, a platform and technical approach that allows for wide contribution from individuals and large organizations, and a license that permits reuse and distribution of the data, in a variety of formats, map products (static and dynamic maps) and services, to responders that need maps most. Following the onset of the Haiti crisis, the Humanitarian OpenStreetMap Team (H.O.T) activated, and focused the community on acting for Haiti. The CrisisMappers community, and the CrisisCamp events globally, have fostered an open community that connects everyone involved, and particularly helped by facilitating the release of key data sources for OSM work. So far, the existing OSM channels and tools have responded well remotely. As the crisis in Haiti continues to develop, OSM must remain relevant to field responders needs, and the culture of openness must persist. Work is necessary to thoroughly articulate OSM capabilities to the on-the-ground mapping community, through capacity building with field workers and technical improvements in the mapping and sharing coordination of OSM.

Work on this effort began with Mikel Maron and Nicolas Chavent in an intensive and overdue two day meeting in Vannes, Brittany, France, and is now open to all for comment and improvement. We are especially eager to open discussions and collaborations with humanitarian response agencies and individuals, as the intention here is to make the work of OSM as relevant and easy as possible for their needs. Likewise, we seek dialogue with community groups and government in Haiti to ensure long term capacity for, and good relationships with, OpenStreetMap. The overall aim of H.O.T is to instutionalize response capacity within OSM, and build capacity to use and interact with OSM within response institutions.

This strategy document and proposal outlines a number of activities, each grouping addressed to a particular need that has been exposed or reemphasized by the experience so far in mapping Haiti. Each need has a proposed set of actions, some urgent and well defined, some longer term or more open, identifying key partners in reaching each goal. These activities have been laid out as needs from the OSM perspective, but does not mean that OSM is the primary or only actor; for instance, imagery is certainly something in which OSM has a stake, but not core responsibility. Each item is separately fund-able, or within an entire package of support for H.O.T. Needs to support these efforts are crucial and current; details on budget for each is available on request when not already made available here.

HOT and OSM Coordination

OpenStreetMap performed extremely well in the Haiti response, and the community surprisingly coordinated intensive work by large numbers of people in a small area. Certainly it wasn't always easy, and mistakes were made, and areas of improvement exposed. The wiki grew rapidly, sometimes out of control, but has stabilized into something near an ideal structure for OSM crisis response. Conflicts in the map did temporarily arise. Likewise, the response community adapted quickly to the OSM way of doing things, but there was a common complaint of confusion over where, and to whom, to address requests and find resources. H.O.T is undertaking a review of the Haiti response in order to identify key improvements to the OpenStreetMap process and tool set. In other words, how can we template the response, and improve. This section is divided into social process, and tool improvements.

Social Processes

  • Define clear roles for volunteers. These can include map editors, wiki gardeners, educators, data compilers, tool builders, community connectors, administrator/coordinators, among others. There's no need to directly enforce any kind of role for any volunteer, but with a well defined obligation, and clear hand off of responsibilities, we can insure that gaps do not develop in the response, and that incoming new volunteers have a clear way to contribute.
  • Take the existing Haiti wiki structure, improve it, and devise a system for rapid deploy of a set of subpages for any WikiProject in the event of a crisis.
  • Enable dedicated communication channels for those involved in OSM crisis response, to insure that OSM does not overburden other channels.
  • Ticketing system: Establish a clear channel for requests from the field, request to integrate incoming data sets, that can be checked out, fulfilled, and communicated back by the OSM community.
  • Identify a core set of tools for developers to stand up in time of crisis.
  • Have a simplified, low bandwidth web presence template, outside the wiki, targeted for time strapped responders (ala http://imaphaiti.com/)
  • Educational material and clear guides for new users to OSM, especially in the field.

Tools

  • Tools for ongoing coordination and identifying needs, addressing the problem of what to map now? which thematic layer? what is complete? what is stale? OSMatrix is one approach that uses heat-map like visualizations.
  • Improved in-situ coordination. Show in real time which users are editing where, and provide means for geo-oriented chat to talk to nearby editors.
  • Measuring completeness and activity is a research area in itself. There is a pressing need to have meta-data describing the state of the map in any place, prior to an emergency. This will take a combination of data and community metrics.
  • Interfaces for quick contribution from the field. Email address to receive GPS tracks.
  • Imports: A number of classes of tools for integrating new data sets are needed. Humanitarian Data Model synchronization. Bulk upload. Individual mediated upload. Upload into OSM, w/ tag for human check need.
    • Need for better import tools that perform automatic rule-based duplicate detection (e.g.) "do not upload if there is another feature with this tag closer than X meters". Tools for semi-automatic road conflation.
  • Mechanical turk style process for working through and importing individual features from large imports. Especially necessary for updates received on key data sets. For things like identifying damaged buildings, make sure educational materials clearly show examples. Allow process for duplicate checking of other works, to increase confidence level by responders. Notify editors of errors in judgment. Set reputation by overall veracity. The stand out example here are camp assessments, which were undertaken by several entities, and in some cases too much enthusiasm by OSM editors.
  • Build tools for participation by other entities. ESRI adaptor for direct participation in OSM by traditional GIS pros. Encourage other entities to issue changesets of their data, in addition to complete data dumps.
  • Improve KML and Shapefile export. Allow for selection of particular thematic layers in exports.
  • Offline visualization, geocoding and editing tools for the field. USB sticks for rapid deploy of data and tools. Disk images.
  • Build facility for other response apps, like Ushahidi and Sahana, to transmit particular updates relevant to the base map from their information streams.

More lists

Camp Roberts planning session from WhereCamp

Interoperability and Technical Infrastructure

OpenStreetMap has historically been geared for mapping in developed countries, in non-emergency situations. In the urgent response to the Haiti earthquake, the traditional OSM approach was very useful in itself, but as field work becomes more established in Haiti, the relative chaos and standard feature set of the OSM tagging becomes less useful to organizations working in traditional GIS modes in crisis situations. Response agencies need to easily know how OSM tags relate to their existing data models, and how to include new types of things in OSM - features which may only be relevant in disaster response.

The Humanitarian Data Model seeks to reconcile schemas from many humanitarian response agencies, all of which are based on field requirements. When possible, it matches Humanitarian attributes to existing Map Features in OSM. Once these mappings are established, imports to and exports from OSM to response agencies becomes more straightforward, and could lead to direct participation of OSM by responders. Extract-Transform-Load (ETL) tools will smooth this effort. This work is ongoing with the Humanitarian Data Model, led and designed by Nicolas Chavent, and needs support now.

Further tools need to be designed and implemented for managing both the process of mapping and distributing schema to different editing and rendering systems, within and without OSM, and synchronizing and merging the data itself. In merging data from diverse sources, notions of trust and community become important, and the analysis and visualization tools for this process become paramount. Essentially, this work leads to OpenStreetMap as a full participant, and community broker, within ad-hoc and formal Spatial Data Infrastructures (SDI). This complements World Bank work on SDI with OpenGeo. This ties as well to the United Nations (UN) efforts towards the building of a UNSDI happening within the UN Geographical Information Working Group (UNGIWG) in which HOT is participating since 2007.

This activity is part based in need right now, and part implementation based research in some of the most open questions facing geographic data management right now. Some questions articulated by Jo Walsh are: Can we merge versions of the same data source that have been corrected in different places? Can we federate search across different registries or catalogues, or do we still need a "master"? Can we treat data like software, with packages and release cycles... or is data more of a stream, a trickle of updates negating the idea of "version"?

Activities

  • Gather requirements and data models from partners in relevant thematic areas (transport, WASH, health, populations, etc.), both in the current Haiti crisis and other projects such as Map Kibera. Reconcile into a superset Humanitarian Data Model, with mapping to existing and new OSM map features, all documented in the OSM wiki. Currently in discussion with groups from IFRC, MapAction, iMMAP, UNGIWG/UNSDI, UNOCHA, WHO-PAHO, Sahana, CCCM and Shelter Cluster, ITHACA. (list more as relevant). Iterate as needed in new thematic areas, and with new requirements.
  • Document and lead discussion in the OSM community on endorsement of the scheme. Push a pilot ETL process in conjunction with GeoCommons, for distributing data to partners. Design OSM configuration files for editors (JOSM, Potlatch) and rendering (Mapnik) that match Humanitarian Data Model tags.
  • From this pilot, lead design and implement of tools to handle the uncovered requirements of managing schema, ETL processes and configuration files editing and symbolic rendering.
  • Lead effort to establish these processes and tools as part of SDI.
  • Participate in the design and implementation of tools to manage synchronization and merging of different data sources. This would involve components to
    • Make trust networks within OSM explicitly actionable in tools.
    • Give agencies the ability to authoritatively sign particular revisions of objects in OSM.
    • Analysis to identify the same feature in different data sets, and conclude the particular differences which need manual work.
    • Visualization and workflow tools to highlight trust and authority within OSM (or any geographic data set), and tools to effectively manage merging of what could be overlapping or out of sync data sources.

On the Ground Support

In the coming phases, we've identified a direct role on the ground as a crucial need to keep up the momentum and utility of OSM in response. As responders are cycled through, and the mechanics of the response drift into recovery and reconstruction, the advantages of, and necessity to, work in the commons and with the community in the field requires high visibility and support. This individual, or 2-person team, would provide direct on the ground support, training and advocacy for using and contributing to OSM. They'd bring a set of GPS units, run OSM trainings, do small bits of technical OSM work as needed, and funnel field requests back to the OSM community.

In the coming recovery phase, we have lined up objectives and candidates, and in the longer reconstruction, we'd hope to again identify all the needs to continue direct involvement, dependent on need. We are not in a position to make all logistic arrangements ourselves, and will rely on a hosting organization relevant to the phase of the response. We definitely don't want to be a burden on any operation, but feel if some organization can take on additional people, and give them the freedom to carry out this work, the benefit to the entire operation will be huge. We hope this can set a precedent for future phases of the crisis in Haiti, and in other crisis events, so that direct OSM representation can be part of the immediate response.

We are now working to deploy to Haiti an individual, or 2-person team, for three weeks starting in the end of February. On arrival in Port-au-Prince, their remit will be to directly connect with H.O.T partners in the mapping and data field from the UN agencies and NGOs with whom we have been cooperating throughout the response, as well as outreach to newcomers among UN, NGOs and local entities. Training workshops on GPS data collection and editing in OSM tools would be organized for this group. They will especially work closely with colleagues in the hosting organization to build capacity. They'll serve in a direct coordination role between the response community in Haiti and the remote OSM community. Specific data collection missions will be organized to fill gaps in areas that have not been adequately covered by remote mapping and existing data sets, particularly outside the capital.

Imagery

The imagery sharing during the Haiti response has been another watershed event. In no other crisis have satellite companies like GeoEye and Digital Globe, and users and distributors of imagery products like UNOSAT, UN-SPIDER and Google, made imagery data so freely available. The World Bank arranged new aerial flights for free distribution to the CrisisCommons community. In most cases, the imagery was completely public domain, or non-commercial, with explicit provisions for tracing in OSM.

For best use on the web and in OSM, the raw imagery was prepared for use via WMS or Spherical Mercator Tiles. Chris Schmidt in particular volunteered to monitor new releases, process that imagery, and then distribute it both as tiles for OSM and in aggregated forms such http://haiticrisismap.org/. The catalog of imagery was maintained through OSM wiki pages [1], and text on the Haiti Crisis Map site [url]. He made use of computing, storage, and bandwidth donated by TelaScience. Jeff Johnson, Schuyler Erle and Tim Waters also contributed to all of this process, especially in ensuring proper rectification of early misaligned imagery. Additionally, Schuyler and Tim helped coordinate distribution of archived raster maps from NYPL, through Map Warper, which along with other vector data sets, complimented the aerial imagery with road names and identified features.

Besides tracing into OSM, imagery was used by expert groups at UNOSAT, JRC, World Banck, ITHACA, SERTIT, and elsewhere for producing specialized analysis products like building damage reports, road obstructions, and location of IDP camps. It was seen that many groups have similar directives in times of crisis, and some duplication is inherent in response agencies work. Other groups have expressed interest in other types of analysis, for instance KeyObs could make population estimates of IDP camps and informal areas from recent imagery.

The Haiti response at this moment is unique, so H.O.T is especially motivated to work from this demonstrated effectiveness of the open approach. Pieces are missing in the long term. On the one hand, there is no established relationship or agreement with companies and agencies for imagery to be made available to OSM and the CrisisCommons community in the next crisis. On the other hand, the processing and distribution of imagery is ad-hoc, and no established community and resources exist to take on this role.

Our recommendation is therefore twofold.

  • First, OpenAerialMap [2] has recently been restarted organizationally, and can act as the focal point for community handling of imagery. The experience of Haiti should make clear requirements for the community, and funding allocated towards software development and computing resources are necessary.
  • Second, a convening of all the core actors in imagery acquisition and distribution in crisis should happen asap, perhaps at upcoming CrisisCamps, or Where 2.0, in order to hash out standing agreements for access, possibly in line with the Disaster Charter.

Grassroots Mapping

OpenStreetMap is strongest when led by people local to the map. They are most familiar with their own areas, and have the most motivation to keep the map current, up to date, and in relevant applications. So far, the OpenStreetMap contribution and use in Haiti has been overwhelming from remote locations, or from agencies responding to the disaster. Even in prior normal times, the challenges of access, poverty and education has not allowed a local OSM group to form. In other places with similar challenges, OSM and other digital media projects have been designed to explicitly build this capacity in the community, by focused training, networking, and high profile outputs. Map Kibera and Palestine mapping are two examples where the accessible and inexpensive tools of OSM have been introduced into marginalized communities.

Many slum areas lie within Port au Prince, and other parts of Haiti, and were already facing extreme challenges, in a country of extreme challenges. Slums typically harbor some of the most vulnerable and marginalized communities. How can the work of OSM and CrisisCommons be directly relevant to the residents of Haiti now? There are already indications that due to undermapping by MINUSTAH, slum areas are less fully represented in OSM, which is causing difficulties for Ushahidi geocoding reports. There is every possibility that slum areas are being underserved by the humanitarian response, but due to lack of information flow here, there's no complete picture available. And in the later recovery and reconstruction phases, we see capacity to use these tools as crucial for the citizens of Haiti to have an empowered voice and full participation in decision making. One of Ushahidi's principle goals in Haiti is to ensure that residents of slums have the ability to text in location and needs to the Ushahidi-Haiti platform and to receive customized alerts if they so choose.

Thus, there's been call to reproduce the success of Map Kibera in Haiti, by a network of people from Ushahidi and Google. UNICEF have also expressed interest. Digital Democracy had been implementing Project Einstein prior to the quake, and are interested to continue that work, with an added mapping component, once Haiti is ready.

At the moment, we are eager to assess conditions in the slums, and the feasibility of such a mapping activity in the next few months. MINUSTAH is being consulted. An OpenStreetMap person on the ground would be most effective in answering our questions:

  • Are there contacts/lists of civil society groups, religious groups, academic groups, or anyone else who have been active working in the slums prior to the earthquake? Would especially be interested in community based organizations and NGOs run by residents of these areas.
  • What is the current social and security situation like in the slums? What is the devastation like in comparison to other areas? Are similar levels of aid reaching the slums?
  • Are there well defined geographic boundaries for slum areas in Port au Prince, and other cities in Haiti?
  • Are there any existing data sets that cover these slum areas well, in transport and other thematic areas? (We do have access to the MINUSTAH data sets released so far, and largely incorporated into OpenStreetMap)
  • Looking at the Map Kibera website [3], is there anything else that strikes you that we should know for such a project in Haiti, when the time is right?

Once deemed feasible, the template and toolkit of Map Kibera would be applied in Haiti, in chosen slum areas and wherever deemed beneficial. Especially crucial is to identify trusted community based partners, and university and government bodies who can help sustain the effort and act as trusted points of introduction to the community. Individuals recruited to lead grassroots mapping efforts on the ground would benefit greatly from initial training in Nairobi, to experience first hand the unexpected challenges of this work.

Data Sources and Links

Paper

This section is not completed, write up welcome.'


Need: Printed paper maps still needed. A big drag on GIS resources on the ground to supply these. A3 Map Book printing and distribution is an immediate need. Underlying geometries won't change much. Provide information for online resources. Distribute to GIS folks on the ground. See Delta State map book file production.

Longer term, work towards integration with Walking Papers for flow of information back to OSM. Can also promote W-P immediately.

Actors: UNHCR interested from Eduoardo. UN-OCHA. Star-Tides/USMIL interested in Walking Papers.

Walking Papers modifications to easily support alternate rendering back ends, other improvements.

HOT Establishment

Humanitarian OpenStreetMap Team itself now only exists as an ad-hoc group within OpenStreetMap, with several early leaders. In order to fulfill its role as the interface where volunteers and the humanitarian community meet, H.O.T needs to get its own house in order. This mean proper organizational establishment, either as an independent entity or under the umbrella of a response NGO, agency, or university. It will certainly have relationship to the OpenStreetMap Foundation, through the in-development Local Chapters arrangement. Long term relationships will be established with the responding agencies, interagency groups such as UNGIWG, ability to submit to ReliefWeb directly, and with academics, especially through the development of curriculum and research programs, like work undertaken by Muki Hakley, like the OpenRouteService for Haiti maintained by the University of Heidelberg. Hired positions within H.O.T would be responsible for implementing and managing programs outlined in this document.

We are therefore seeking advising, guidance and funding to establish H.O.T.

Haiti Assessment

It is critical that the response community as a whole assess the Haiti response. What role does each GIS responder play? What is the OSM role systematically? How can we reduce duplication and work better together in the future?

There will be a number of conferences in the next 6 months to convene all relevant parties. H.O.T strongly recommends full participation, wherever possible, with funding agencies helping to ensure a critical mass of attendance.

  • Where 2.0 / WhereCamp: late March 2010
  • UNITAR/UNOSAT technical workshop on damage assessment methodologies from remotely sensed imagery over Haiti: 27th-28th April 2010.
  • DC CrisisCamp: June 2010
  • State of the Map crisis sessions: July 2010
  • [Crowd Sourcing for Updating National Databases (CSND) 2010], Swisstopo - EuroSDR - EuroGeographics, 8-9th September, 2010, Bern, Switzerland.
  • ICCM, October 2010
  • http://crisismappers.net
Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=Humanitarian_OSM_Team/Haiti_Strategy_And_Proposal&oldid=1586531"