Together with our dear colleague Thomas Donath from Nordlight Research I had the opportunity to present a study we conducted to find out, if a wisdom of crowds-approach to election prognosis could potentially compete with or even outperform traditional methods for election research.

With this short blog-entry I`d like to give you a short summary on the speech. Hope you enjoy it and of course your comments are welcome!

“The Wisdom of Crowds”, a book written by James Surowiecki in 2004 about the aggregation of information in groups, claims the many to be smarter than the few. According to Surowiecki, four key criteria have to be met to enable a large group (crowd) to make accurate predictions and the right decisions in terms of reaching common goals.

1) Diversity of opinion: a wide range of points of view in the group
2) Independence of opinion: individuals making guesses do not influence each other
3) Decentralization: local and specialised knowledge
4) Aggregation: a mechanism to reach a collective guess or decision

The goal of this study was to obtain a first indication on whether a “wisdom of crowds” approach to election prognosis could potentially compete with or even outperform traditional means of prognosis.

Methods & Data
We asked 2,323 Germans, registered in an online-access-panel, to predict the official outcome of the German “Bundestagswahl 2009” one week in advance. There were three major differences to surveys executed by institutes like “Forschungsgruppe Wahlen” or “Infratest dimap”. Firstly, the sample was not representative. Secondly, no weighting of raw data was employed. Thirdly and most importantly, participants were asked to give their own prognosis for all major parties rather than state which party they would vote for the following Sunday.

Results
The main outcome of the study is that – in this single trial – the wisdom of crowds approach yielded a slightly less precise prognosis compared to results of traditional surveys just before the election (sum of absolute difference in percentage points: 10.3 for Wisdom of Crowds vs. 8.4 for e.g. Forschungsgruppe Wahlen and Infratest dimap).
However the precision of the wisdom of crowds approach is still high enough to justify further interest.
Also, a large number of moderator variables such as interest in politics, gender, age, or the knowledge of other prognoses correlate with individual forecast performance.

Based on these results, we shed additional light on the idea of the wisdom of crowds – a method which may be an interesting methodological addition to various research purposes which require prognosis. You might find our charts here:

For more information on this case-study and on our findings, please don’t hesitate to contact either Thomas (thomas.donath [at] nordlight-research.com) or me (ci [at] odc-services.com), or simply use the comment-function!

Wie in vorherigen Blog-Einträgen bereits angedeutet, beschäftigen wir uns momentan verstärkt damit, wie es gelingen kann die Qualität der in Online Panels durchgeführten Umfragen zu verbessern und diese für den einzelnen Teilnehmer spannender zu gestalten.

Denn jeder, der bereits einmal an einer Online-Umfrage teilgenommen hat, wird festgestellt haben, dass diese in den seltensten Fällen spannend gestaltet sind. Vielmehr zeichnen sich die derzeit online durchgeführten Umfragen oftmals durch z.B. eine Abfolge von Matrixfragen aus und wirken entsprechend monoton und langweilig. Um dem entgegen zu wirken, haben wir bei ODC Services verschiedene neue Fragetypen entwickelt, die auf die Interaktivität mit den Usern abzielen.

Um beim Beispiel der Matrixfragen zu bleiben, haben wir als neues Tool unsere sogenannte Flash-Skalenklick-Matrix entwickelt, bei der die Matrixattribute dem User nacheinander präsentiert werden. Sobald die Bewertung durch Klicken auf die Skala erfolgt ist, verschwindet das aktuelle Attribut und das nächste rückt zur Bewertung in den Vordergrund. Wie genau dieser Fragetyp und die vielen neuen anderen Features funktionieren, können Sie sich in der nachfolgenden Musterpräsentation ansehen.

Viel Spaß!