University team competition focuses on crowdsourcing

Put simply, crowdsourcing occurs when a task is assigned to a wide audience rather than a specific expert or group of experts. The online encyclopedia Wikipedia is one of the most prominent examples—anyone can write or edit an entry. Over time, the crowds refine and improve the product. Crowdsourcing can range from a simple question blasted to a person’s Twitter followers to amateur programmers fine-tuning open-source software.

IARPA spokeswoman Cherreka Montgomery said her project’s goal is to develop methods to refine and improve on crowdsourcing in a way that would be useful to intelligence analysts.

“It’s all about strengthening the capabilities of our intelligence analysts,” Montgomery said.

And if analysts can use crowdsourcing to better determine the likelihood of seemingly unpredictable world events, those analysts can help policy makers be prepared and develop smarter responses. In a hypothetical example, a crowd-powered prediction about the breakout of popular uprisings in the Middle East could influence what goes in a dossier given to decision-makers at the highest levels.

The program at George Mason is called DAGGRE, short for Decomposition-based Aggregation. The researchers have used blog postings, Twitter, and other means to get the word out about their project to potential participants. No specialized background is required, though a college degree is preferred.

The project seeks to break down various world events into their component parts. The stability of Kim Jong Il’s regime in North Korea provides an example. One forecaster might base his prediction based solely on political factors. But what if the political experts could be guided by health experts, who might observe that Kim’s medical condition is flagging?

The DAGGRE participants key their answers into forms on the project’s website, and also supply information at the outset about their education and what areas they have expertise in. The scholars overseeing the project will then seek to break down the variables that influence a forecaster’s prediction, and use the data in a way that people with disparate knowledge bases can help guide each other to the most accurate forecast.

Military and intelligence researchers have long studied ways to improve the ability to predict the future. In 2003, the Defense Advanced Research Projects Agency (DARPA) launched research to see whether a terrorist attack could be predicted by allowing speculative trading in a financial market, in which people would make money on a futures contract if they bet on a terrorist attack occurring within a designated time frame. The theory was that a spike in the market could serve as a trip wire that an attack was under way. But some found the idea ghoulish, and others objected to the notion that a terrorist could conceivably profit by carrying out an attack, and the research was halted.

Laskey said George Mason’s research bears some fundamental similarities with the discontinued DARPA research, with the crucial difference that nobody participating in George Mason’s project can profit from making accurate predictions. But participants who make accurate predictions are rewarded with a point system, and there is a leaderboard of sorts for participants to measure their success. Some can also choose to receive a small stipend for their time, but it’s not tied to how they answer questions.