View publication

Title Hybrid Forecasting of Geopolitical Events
Authors Daniel Benjamin, Fred Morstatter, Ali Abbas, Andrés Abeliuk, Pavel Atanasov, Stephen Bennett, Andreas Beger
Publication date March 2023
Abstract Sound decision-making relies on accurate prediction for
outcomes ranging from military conflict to disease outbreaks. To improve
crowdsourced forecasting accuracy, we developed SAGE, a hybrid forecasting
system that combines human and machine generated forecasts. The system
provides a platform where users can interact with machine models and thus
anchor their judgments on an objective benchmark. The system also aggregates
human and machine forecasts weighting both for propinquity and based on
assessed skill while adjusting for overconfidence. We present results from
the Hybrid Forecasting Competition (HFC) larger than comparable
forecasting tournaments including 1085 users forecasting 398 real-world
forecasting problems over 8 months. Our main result is that the hybrid
system generated more accurate forecasts compared to a human-only baseline,
which had no machine generated predictions. We found that skilled
forecasters who had access to machine-generated forecasts outperformed those
who only viewed historical data. We also demonstrated the inclusion of
machine-generated forecasts in our aggregation algorithms improved
performance, both in terms of accuracy and scalability. This suggests that
hybrid forecasting systems, which potentially require fewer human resources,
can be a viable approach for maintaining a competitive level of accuracy
over a larger number of forecasting questions.
Pages 112-128
Volume 44
Journal name AI Magazine
Publisher AAAI Press
Reference URL View reference page