Wolf's tarot-reading post seems a good take-off for this...
David Brooks has finally written a rather intellectual editorial
... he's not talking about Fox News.
NY Times
DAVID BROOKS
Published: March 21, 2013
Forecasting Fox
Quote:
In 2006, Philip E. Tetlock published a landmark book called “Expert Political Judgment.”<snip>.
His subsequent work helped prompt people at one of the government’s most creative agencies,
the Intelligence Advanced Research Projects Agency, to hold a forecasting tournament
to see if competition could spur better predictions.
<snip>
Five teams entered the tournament, from places like M.I.T., Michigan and Maryland.
Tetlock and his wife, the decision scientist Barbara Mellers, helped form a Penn/Berkeley team,
which bested the competition and surpassed the benchmarks by 60 percent in Year 1.
How did they make such accurate predictions?
In the first place, they identified better forecasters.
It turns out you can give people tests that usefully measure
how open-minded they are. <snip>
In the second year of the tournament, Tetlock and collaborators
skimmed off the top 2 percent of forecasters across experimental conditions,
identifying 60 top performers and randomly assigning them into five teams of 12 each.
These “super forecasters” also delivered a far-above-average performance in Year 2.
Apparently, forecasting skill cannot only be taught, it can be replicated.
<snip>
|
Although a somewhat different methodological approach was used,
it looks to me much like a modified "Delphi" method.
I once used the Delphi mehto to gather consensus on a public health policy
that was not being resolved by usual means and staff meetings.
It made me a believer because Delphi neutralized the "self-appointed experts" who tended to dominate/intimidate discussions.