Bad Calls
Verbal Aggressions Against Major League Baseball Umpires Affect Their Decision Making
Joël Guérette, Caroline Blais & Daniel Fiset
Psychological Science, forthcoming
Abstract:
Excessively criticizing a perceived unfair decision is considered to be common behavior among people seeking to restore fairness. However, the effectiveness of this strategy remains unclear. Using an ecological environment where excessive criticism is rampant -- Major League Baseball -- we assess the impact of verbal aggression on subsequent home-plate umpire decision making during the 2010 to 2019 seasons (N = 153,255 pitches). Results suggest a two-sided benefit of resorting to verbal abuse. After being excessively criticized, home-plate umpires (N = 110 adults, employed in the United States) were less likely to call strikes to batters from the complaining team and more prone to call strikes to batters on the opposing team. A series of additional analyses lead us to reject an alternative hypothesis, namely that umpires, after ejecting the aggressor, seek to compensate for the negative consequences brought on by the loss of a teammate. Rather, our findings support the hypothesis that, under certain conditions, verbal aggression may offer an advantage to complainants.
---------------------
Premature predictions: Accurate forecasters are not viewed as more competent for earlier predictions
Robert Mislavsky & Celia Gaertig
Journal of Experimental Psychology: General, January 2024, Pages 159–170
Abstract:
How does the timing of a prediction influence how a forecaster is perceived? Many people believe that they will be seen as more competent if they make accurate predictions far in advance of an event. However, we find that forecasters are not seen as more competent -- and are sometimes seen as less competent -- when they make predictions far in advance of an event occurring. Furthermore, we find that this is because observers recognize that events far in the future are less knowable, suggesting that they may attribute accurate but premature forecasts more to luck than to skill. Forecasters would benefit from knowing this when considering making predictions. They are not penalized for waiting until more information is known but may lose credibility if they make a prediction too early.
---------------------
Approaching Human-Level Forecasting with Language Models
Danny Halawi et al.
University of California Working Paper, February 2024
Abstract:
Forecasting future events is important for policy and decision making. In this work, we study whether language models (LMs) can forecast at the level of competitive human forecasters. Towards this goal, we develop a retrieval-augmented LM system designed to automatically search for relevant information, generate forecasts, and aggregate predictions. To facilitate our study, we collect a large dataset of questions from competitive forecasting platforms. Under a test set published after the knowledge cut-offs of our LMs, we evaluate the end-to-end performance of our system against the aggregates of human forecasts. On average, the system nears the crowd aggregate of competitive forecasters, and in some settings surpasses it. Our work suggests that using LMs to forecast the future could provide accurate predictions at scale and help to inform institutional decision making.
---------------------
AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy
Philipp Schoenegger et al.
London School of Economics Working Paper, February 2024
Abstract:
Large language models (LLMs) show impressive capabilities, matching and sometimes exceeding human performance in many domains. This study explores the potential of LLMs to augment judgement in forecasting tasks. We evaluated the impact on forecasting accuracy of two GPT-4-Turbo assistants: one designed to provide high-quality advice ('superforecasting'), and the other designed to be overconfident and base-rate-neglecting. Participants (N = 991) had the option to consult their assigned LLM assistant throughout the study, in contrast to a control group that used a less advanced model (DaVinci-003) without direct forecasting support. Our preregistered analyses reveal that LLM augmentation significantly enhances forecasting accuracy by 23% across both types of assistants, compared to the control group. This improvement occurs despite the superforecasting assistant's higher accuracy in predictions, indicating the augmentation's benefit is not solely due to model prediction accuracy. Exploratory analyses showed a pronounced effect in one forecasting item, without which we find that the superforecasting assistant increased accuracy by 43%, compared with 28% for the biased assistant. We further examine whether LLM augmentation disproportionately benefits less skilled forecasters, degrades the wisdom-of-the-crowd by reducing prediction diversity, or varies in effectiveness with question difficulty. Our findings do not consistently support these hypotheses. Our results suggest that access to an LLM assistant, even a biased one, can be a helpful decision aid in cognitively demanding tasks where the answer is not known at the time of interaction.
---------------------
Deep distortions in everyday memory: Fact memory is illogical, too
Charles Brainerd et al.
Journal of Experimental Psychology: General, forthcoming
Abstract:
A distinction has recently been drawn between surface distortions and deep distortions in false memory, where the former are conventional errors of commission and the latter are illogical relations among multiple memories of items. The deep distortions that have been studied to date are violations of the logical rules that govern incompatibility relations, such as additivity and countable additivity. Because that work is confined to laboratory word-list tasks, it is subject to the ecological validity criticism that memory for everyday facts may not exhibit such phenomena. We report evidence that memory for everyday facts displays the same deep distortions as laboratory tasks. We developed a version of the conjoint-recognition paradigm that measures memory for incompatible general knowledge facts, similar to those found on the quiz program Jeopardy! In experiments with university participants, four deep distortions were detected (violations of the additivity, countable additivity, universal set, and compensation rules), with participants consistently remembering more than what is logically possible. The distortions were more robust than in laboratory experiments, and memories of incompatible facts (e.g., Jupiter and Saturn cannot both be the largest planet in the solar system) did not suppress each other. These patterns were replicated in subsequent experiments with older and more diverse participant samples. Consistent with the notion that deep distortions are by-products of gist memory, conjoint-recognition modeling analyses revealed that memory for everyday facts was even more reliant on gist than memory for word lists, and that verbatim memory was near-floor.
---------------------
Choosing not to get anchored: A choice mindset reduces the anchoring bias
Krishna Savani & Monica Wadhwa
Journal of Experimental Social Psychology, May 2024
Abstract:
In negotiations, first offers serve as potent anchors. After receiving a first offer, although people clearly have a choice about what amount to counteroffer, they often fail to adjust away from the first offer. We identify a simple nudge, a reminder that people have a choice, that can reduce the anchoring bias. We argue that a choice nudge leads people to think of more potential counteroffers that they can make, which reduces the extent to which they are anchored to the first offer. Seven studies conducted with US residents recruited from online research platforms tested this hypothesis. We found that merely reminding buyers that they have a choice led them to anchor away from sellers' first offers in a painting buying task (Studies 1 and 2) and a used car negotiation (Study 3). A choice reminder nudged people to consider more counteroffers (Study 4a) and asking people to consider more counteroffers reduced the anchoring bias (Study 4b). Consistent with the idea that thinking of counteroffers requires cognitive resources, we found that the effect of a choice nudge is attenuated under high cognitive load (Study 5). Study 6 ruled out an alternative motivational account for the choice nudge effect. This research contributes to the choice mindset literature by showing that highlighting the semantic concept of choice can help correct a pervasive decision-making bias, and to the anchoring literature by showing that thinking of more counteroffers can reduce the anchoring bias, at least in contexts in which the direction of adjustment from the anchor is known.
---------------------
The Prediction Order Effect: People Are More Likely to Choose Improbable Outcomes in Later Predictions
Jackie Silverman & Uri Barnea
Management Science, forthcoming
Abstract:
People often need to predict the outcomes of future events. We investigate the influence of order on such forecasts. Six preregistered studies (n = 7,955) show that people are more likely to forecast improbable outcomes (e.g., that an “underdog” will win a game) for predictions they make later versus earlier within a sequence of multiple predictions. This effect generalizes across several contexts and persists when participants are able to revise their predictions as well as when they are incentivized to make correct predictions. We propose that this effect is driven by people’s assumption that improbable outcomes are bound to occur at some point within small sets of independent events (i.e., “belief in the law of small numbers”). Accordingly, we find that the effect is attenuated when the statistical independence of events is made salient to forecasters both through the nature of the predictions themselves (i.e., when the events are from distinct domains) and through directly informing them about statistical independence. These findings have notable practical implications, as policy makers and businesses have the ability to control the order in which people evaluate and predict future events.
---------------------
Evidence from the future
Tianwei Gong & Neil Bramley
Journal of Experimental Psychology: General, March 2024, Pages 864–872
Abstract:
The outcome of any scientific experiment or intervention will naturally unfold over time. How then should individuals make causal inferences from measurements over time? Across three experiments, we had participants observe experimental and control groups over several days posttreatment in a fictional biological research setting. We identify competing perspectives in the literature: contingency-driven accounts predict no effect of the outcome timing while the contiguity principle suggests people will view a treatment as more harmful to the extent that bad treatment outcomes occur earlier rather than later. In contrast, inference of the functional form of a treatment effect can license extrapolation beyond the measurements and lead to different causal inferences. We find participants’ causal strength and direction judgments in temporal settings vary with minimal manipulations of instruction framing. When it is implied that the observations are made over a preplanned number of days, causal judgments depend strongly on contiguity. When it is implied that the observation may be ongoing, participants extrapolate current trends into the future and adapt their causal judgments accordingly. When data are revealed sequentially, participants rely on extrapolation regardless of instruction framing. Our results demonstrate human flexibility in interpreting temporal evidence for causal reasoning and emphasize human tendency to generalize from evidence in ways that are acutely sensitive to task framing.