The decision of the British government to require large employers to publish their gender pay gap, which happened for the first time last August, both shines a light onto some uncomfortable truths about the divide between men and women in the workplace, and affirmed the government’s commitment to narrowing those divides through social pressure, as well as through legislation like the equal pay act.

We know that these social influences are powerful – in fact, it’s the topic of a book that one of us has just written – but influence isn’t enough, it needs to be channelled. What concrete behaviour is it that the government, in requiring firms to publish their gender pay gaps, is trying to elicit?

One common solution that firms seem to be adopting is to change their hiring and promotion processes. Blinding the application process – so that hiring managers can’t tell whether a candidate is male or female – or their ethnicity, or even where they went to university, is based on sound research over the last 20 years showing both that tacit discrimination during CV sifts is rife, and that blinding can substantially mitigate the problem. Digital services, like Applied, make the process of CV blinding straightforward, as well as helping employers avoid discrimination in their job adverts, by helping to cull words that disproportionately attract male – and discourage female – candidates.

Other solutions that policymakers and firms have latched onto are much less likely to have a positive effect. For example, Goldman Sachs announced this week that they were to require hiring managers to interview at least two “diverse candidates” for every role, in a bid to diversify who gets hired and promoted. This is a purportedly “evidence-based” way of increasing diversity. Unfortunately, not all evidence is created equal. In this case, the finding comes from a study in the Harvard Business Review, which compares hiring outcomes across roles where different numbers of women were interviewed. The study concludes that when there is only one woman interviewed for a role, they have a “statistically 0%” chance of being hired – compared to 50% if there are two women interviewed. This seems great – a simple nudge and Goldman Sachs, or the government, can effectively eliminate biases in hiring.

If only it were so easy. There are two flaws with this interpretation. First, “statistically 0” does not mean 0 – and, depending on your data, can be a pretty long way from 0. The authors of the HBR article are careful in their use of the term, but others have not been. Second, and more importantly, causal claims are being made from data that are not causal. These findings aren’t the product of a carefully conducted experiment, but instead emerge from data that exists in the messiness of the real world. So, people who choose to interview more women are more likely to hire women than people who choose to interview fewer men. When we take a cold hard look at what we can really learn from the data, this is about as exciting as learning that people who go to McDonalds a lot eat more Big Macs than people who hardly visit.

But, if more diverse candidates are being interviewed, surely that can’t be a bad thing? Maybe, but it’s not clear. Daniel Effron, a Behavioural Scientist at London Business School, studies moral licensing – where doing one “good” thing leaves us feeling like it’s OK to do something bad later on – even when the bad thing is larger, or even cancels out, the good thing. Having invited two diverse candidates to interview, hiring managers could feel more justified in appointing the straight white male candidate.

Even if there isn’t such an obvious backlash, it’s not clear that this approach does anything to change the attitudes which are driving the problem. Katherine Milkman, a professor at the Wharton School, looks at the makeup of company boards and finds an interesting pattern – boards are much more likely to have two female members than we’d expect by chance alone, and are pretty unlikely to have just one. Milkman calls this “twokenism” – firms have faced a backlash against having only a single “token” female board member, so have responded by adding one more – but this doesn’t mean that the underlying tokenism has been resolved. We could be seeing the same thing in hiring lower down the firm.

Narrowing the gender pay gap is a vital frontier in the fight for gender equality, and it’s encouraging to see employers taking action to try and achieve that, and trying to look at the evidence. But if their efforts are based on a misreading of the statistics, these good intentions could actually see our progress slow.

Dr Michael Sanders is Reader in Public Policy at the Policy Institute, and Rosie Campbell is Director of the Global Institute for Women’s Leadership, both at King's College London.