Two of my close friends, Gil Weiss and Ben Horne, were identified as missing last month while climbing a new route in the Cordillera Blanca region in Peru.
Once friends sounded the alarms that Gil and Ben were overdue we all sprung into action. One friend was charged with calling the parents, one was charged with calling the embassy, another the airline, etc… My role was to deploy satellites to try to get a visual on where the guys were climbing (That's right, luckily I have access to a platform that uses satellites).
We contacted our colleagues at DigitalGlobe and GeoEye and they agreed to divert their satellite resources to snap a picture of the search region. Satellite imagery is beautiful because it can give you a very fast response time when you need to see an updated image of the terrain. However, high-resolution satellite imagery is very heavy and requires movement and processing of very large files, not to mention the difficulty in actually analyzing the imagery. Imagine scanning a 5,000 piece puzzle of snow for a white flower petal.
This is where crowdsourcing comes in. Since we consistently work closely with DigitalGlobe, we had hooks into their data and were able to get it online in short order. We then placed a call for help from friends, family, and other climbers on social media channels & setup a platform to view the images. We had 800 people log on to the site and look through the imagery for subtle clues. Since each person was only able to view a small region of the search site we were able to identify small disturbances in the snow that wouldn’t have been detected had they been exposed to the full imagery set. The human visual system is incredibly good at detecting anomalies.
Within 15 minutes of going online we started detecting anomalies in the snow and for the following 4 hours we collected over 5,000 annotations. Of course, the power is in the wisdom of the crowd and we were able to hone in areas of consensus among people.
Backend to the image annotation showing all of the annotations made by members of our crowd.
4 hours after launching the platform we sent a document with the 4 highest probability locations to the search coordinator in Peru who then briefed the search team. The image below shows what ended up being the last tracks of Ben and Gil before their fatal fall. The tracks are incredibly subtle but through the power of the crowd we were able to pick them out of the imagery.
What we learned:
- Many people were incredibly happy to be contributing to the search. Normally all you can do is hope and pray but we were told that it was very therapeutic to actually do something that could have an impact on the outcome of the search.
- Automated algorithms for crowdsourcing reliability are absolutely critical in order to actually extract information reliably from the imagery. This is especially relevant when working with a serious time constraint as was in our case (we had to send the document to Peru before the air support left the ground).
- Long tail – A relatively small percentage of people were responsible for most of the annotations. This is something we see in many of our deployments.
Tags per user showing the long tail nature
- When your crowd actually cares about the task they need a voice. Upon identifying a compelling clue in the imagery, many of the members of the crowd were not content with just tagging it and moving on. Rather, they took screen shots of the clue and would forward them to us or post them on a message board. They needed their voice to be heard and wanted to discuss their findings.
Although in this case we weren’t able to help Ben and Gil in time, I feel fortunate that our team was in the right place at the right time to try to help. We gave it our all and I feel content that if the guys were stuck out there alive we would have found them. We are now more determined than ever to set up this sort of capability to support future search and rescue missions. Imagine if this process is already established and can be deployed within minutes after an explorer is deemed missing.
Thank you to DigitalGlobe and to everyone who helped us search for them.