The replication crisis in social research is ongoing and there was an interesting paper mentioned last week on some related issues. The paper is The replication crisis as mere indicator of two fundamental misalignments and it is concerned with both the methodology issues and the anthropological issues.
It is the latter that interest me. The academic framing of what proper research should look and act like has been problematic for years. That framing has itself then become the model for corporate research and so the problem has been extended.
I remember once listening to a conference talk about Social Psychology and methods for eliminating bias. The descriptions of control techniques, both physical and psychological, to control bias. I spent the whole talk quietly muttering ‘Oh god, that’s bad’ as the speaker listed off mechanisms that seemed founded in the power and privilege of the organisation with little regard for the comfort or autonomy of the participants.
The de-bias mechanisms noted were all about control. Control of where participants were, control of what they did and control of how their reactions were queried and recorded.
None of this sounded good.
This is abusing power to minimise the agency and autonomy of individuals to provide data and meaning that is entirely focused around the organisation’s goals.
This approach minimises the humanity of participants in order to provide greater clarity (though not necessarily greater truth) in the research reports.
This bias-free approach is utterly contaminated by bias. Not the bias of participants, so often criticised and demeaned, but of researchers and their organisations.
The power and privilege of organisations is the greatest bias in research. The desire to control, to pull people out of their communities and, in commercial settings, to test the user not the product usability.
This is the bias that concerns me most. The replication crisis has mostly been talked of in terms of how the control mechanisms are hard to replicate in different places. However, it is the those mechanisms themselves that are the bias problem.