Had a very good conversation here, and I think I explained the problem with saying that (even many) perfectly failed direct replications mean that the original was probably and/or that the theory needs to be reevaluated quite well. They are, essentially: Context, Theory, and Expertise.
This is just the same as in the picture, just a little more worked out:
- is again context stuff. You can use Literally exactly same material but mean different things. Having someone make the exact same hand gesture across cultures can mean different things.
- is the difference between a specific manipulation and the theory. Strack study not replicating does not mean that smiling when we are unhappy will not make us happier (even if it is just placebo or experimenter effects in world). It is clear, I think, that even if holding a pen in your mouth doesn’t make you happier or cartoons funnier, that doesn’t mean that every instance of facial feedback doesn’t exist, or even more generally that there is no such thing as facial feedback.
- is that even kobe bryant and michael jordan could not hit 80 points on demand, and certainly many other people can not even do it once! 😀 even though we have video of them doing it and so it doubtlessly happened. MAybe the team was different, maybe they didn’t feel as good. Sure, we can agree it is not generalizable then, but it also doesn’t mean it didn’t happen or that it is no longer evidence for the theory. If you don’t believe in expertise in psychology, you should my last blog post, which was exactly on that subject (tldr; its a real thing).
and I find those three reasons convincing enough to at least not dismiss the original study and theory outright. Especially number 2 seems strong to me, what do you think?