iPhone App Uses Crowdsourcing to Help the Blind
A team of innovative thinkers from several universities have been working together on a joint project to merge existing technology (the iPhone) with a real-world crowdsourcing application to help blind people make decisions about their environment. The result is VizWiz, an iPhone/Mechanical Turk application that allows a blind user to snap a photo of something, pose a question about something shown in the photograph, and then receive an answer back within seconds.
Winner of the best paper prize last year at ACM’s 23rd symposium on User Interface Software and Technology, VizWiz sidesteps the problem of having a computer try to decipher such problems as reading can labels for blind people. Instead the chore is handed over to a crowd of awaiting workers who look at a photograph the user has taken, listen to a question about it from them, then respond by recording an answer. This information is then relayed back to the original user.
Mechanical Turk is an Amazon web site that pays workers (or turkers as they’re called) to perform oftentimes routine or mundane tasks that are easy for humans to do, but difficult for computers. Tasks are generally short and very low paying, but turkers make up for that by carrying them out in large volumes.
The downside to Mechanical Turk though, is that it is not very user friendly for those that want the answers from the system; this is where scientists from MIT came in, and developed a toolkit called Turkit. This application works as a go-between, allowing other programmers to create applications that utilize Mechanical Turk, without having to actually visit the site. This is what the developers of VizWiz (led by the University of Rochester’s, Jeffrey Bigham) have done. Instead of forcing a blind client to have to visit and use Mechanical Turk directly, they have created an iPhoneapp that works in conjunction with a Turkit application, which in turn sends requests to Mechanical Turk.
In addition, Turkit solves the problem of users having to wait a long time for an answer, by designing the application to contact the Mechanical Turk workers as soon as the iPhone app is launched. This prepares the worker the assignments that will soon be heading their way, which means they will be in a position to answer the question sent, as soon as the picture and audio recording arrive.
VizWiz is still under development, so it’s not yet available for general use. The day of its launch will arrive soon enough, and when it does, is likely to prove a boon to blind and vision impaired people who might find it, as one of the test volunteers described, “…very useful, because I get so frustrated when I need sighted help and no one is there.“