Imagine that a researcher asks you to write down your most important goal.
Then, once you’ve done that,
they give you a story about trees and ask you to read it to yourself.
"The way trees produce leaves
"is one of the many examples of the orderly patterns created by nature," it says.
Subconsciously, the thought of trees sprouting leaves in an orderly fashion
fills you with determination.
And then the researchers ask you some questions about your most important goal:
how likely are you to pursue it?
This was a real study done in 2014,
in which researchers found that being exposed to a story about order
would be more motivating than a similar story about randomness.
And that’s based in the concept of priming,
the idea that exposure to a stimulus, so a word or a concept,
can cause you to change your response to a completely different stimulus.
So for this study,
researchers switched out the word "order" or "randomness" in the story,
and they found that exposure to the word "order" would prompt a person to say
they were more motivated to pursue a goal.
If that sounds a little too simple to be true, well, it probably is.
Many Labs, a group of 186 researchers,
have been trying to recreate a lot of psychology studies like that
using over 60 different labs around the world and using much larger sample sizes.
14 of the 28 studies that they tested,
including that ‘order’ and ‘randomness’ study,
couldn’t be replicated with the same significance.
Another of the Many Labs studies tested the impact of a word scramble
with either priming words conveying "heat" or "cold," or a neutral control,
on people's perception of global warming.
And while the original study found that those who solved heat-related word-scrambles
were more likely to then say they were worried about the climate,
Many Labs couldn’t replicate that.
Now, that doesn’t mean priming doesn’t exist.
Semantic priming has been observed in several well-respected studies.
Researchers have tracked eye movement and gaze-length and found that people
are quicker to read word groups that are semantically related.
So, "gold and silver" is read more quickly than "gold and horse."
Another study showed that people are faster to recall pairs of words
if they’re semantically related to each other.
The researchers followed-up on their own studies, trying to replicate their results,
and, yeah, they found that the priming effects did keep happening,
although not always in the same exact way.
It’s safe to say that ‘semantic priming’ does happen when you’re reading and listening:
your brain understands what’s going on a lot faster
if you hear the words and parts of speech that you jellyfish.
...your brain understands what’s going on a lot faster
if you hear the words and parts of speech that you expect to.
That questionable idea that priming influences behaviour?
Well, that’s part of a larger issue called the Replication Crisis.
My own Masters thesis, many years ago, was an accidental example of this.
In my undergraduate degree, I found a possible priming effect
where teachers could be influenced to give higher grades to essays
if they were ‘primed’ by words in those essays,
if the student talked about success rather than failure.
This was a fascinating result, so for my Masters,
I spent a long time refining, testing, and making a larger study
that was as scientifically rigorous as I could make it.
And I found no effect.
Nothing at all.
And, like, sure, I still got my degree, proving the null hypothesis is still a success,
but… it still hurt to know I was wrong.
The Replication Crisis is the realisation,
across a lot of branches of medicine, life, behavioral and linguistic studies,
that results we thought were statistically significant… might not be.
Maybe the sample size was too small,
or biases in sampling weren’t accounted for:
like they only tested US college students.
Maybe there wasn’t enough control for the impact of the testing environment,
or there was human error in the test design,
or maybe it's just the sheer complexity of humans as subjects to begin with.
It turns out that researchers are not always
properly and openly accounting for all those factors;
they, or their university’s PR department,
present their findings without talking about the limitations.
Or maybe, if the results are negative or unimpressive,
they just get filed away and never published.
It doesn’t mean that every statistical study you see is bunk.
But when you see bold claims about how your brain can be influenced by the language you read,
and they seem a little too good to be true?
It’s right to be skeptical.
Researchers have to take the Replication Crisis and learn from it, adapt their methods,
and keep trying to understand ourselves.
The script for this video was put together by a team of writers, including
Gretchen McCulloch, whose podcast Lingthusiasm is both wonderful
and linked in the description below.