HR professionals should base decisions on the best available evidence.

A beautiful study by Greer K. Gosnell, John A. List, and Robert D. Metcalfe, with the help of Virgin Atlantic, provides solid facts on a difficult topic: the productivity of highly-skilled workers. What makes this study so impressive is that it is based on objective measures and well-designed control groups so we can have a high degree of confidence in the findings.

How The Virgin Atlantic Study Was Done

The study looked at the productivity of commercial airline captains. The productivity metrics were fuel use, time delays, and safety.

They tested four management practices:

• Monitoring Performance: Captains got detailed reports on their performance.
• Providing Performance Feedback: Captains were told how often they hit the targeted level of the performance metrics.
• Setting Personalized Targets: Captains were encouraged to achieve personalized targets.
• Offering Pro-social incentives for Performance: Captains were told that if they achieved their targets the company would donate to a charity.

In each case, there was an experimental group and a control group. The researchers gathered data from 40,000 flights over a 27-month period for 335 captains.

What They Learned From Their HR Metrics

One might think that a motivated and skilled professional in a tightly regulated industry would already be operating at their best. If that were the case, these management practices wouldn’t have an impact. Let’s see what happened!

Here are the results of the study:

• Result 1. The performance of captains improved considerably upon announcement of monitoring performance metrics.
• Result 2. Providing captains with information on recent performance moderately improved their fuel efficiency.
• Result 3. The inclusion of personalized targets significantly increases the captains’ performance.
• Result 4. Adding a charitable component to the personalized targets intervention did not lead to better results than providing targets alone.
• Result 5. The effects of the management practices attenuate or disappear after the practices are is removed.
• Result 6. Largely due to fuel savings from the monitoring effect, the airline saved about $6 million throughout the eight-month experimental period.

Yes, But Will It Work For Us?

One of the questions we need to ask in assessing the relevance of research is how close the context of the research is to the context of our own situation. You probably have highly skilled workers in your company, but it’s unlikely they’re pilots. You probably have a complex operation, but you are probably not an airline. Does it make sense to generalize these findings to your situation?

I believe you can generalize for a couple of reasons. One is that the apparent core causal mechanism that providing data on performance improves performance makes a lot of sense, and that casual mechanism will apply in many situations. The other reason is that in the absence of other evidence you have to go with what you’ve got. Remember, the rule in evidence-based management it “the best available evidence” not “perfect evidence”. This is a good finding, use it.

What To Tell Your Co-Workers

Managers may believe one or more of the following:

• What HR says are good people management practices don’t really have enough of an impact to be worth spending time on;
• The skilled and motivated professionals they have on their team don’t need to be managed; and/or
• Simple, obvious management interventions won’t make a difference, they need something sophisticated.

This study shows those beliefs are, at least in the case of airline pilots, false. Managers may not notice the impact of simple good people management practices if they are not doing a scientific experiment because the improvements are hidden in the noise of all the other things going on. The lesson is that the disciplined practices of good management pay off and that it’s a mistake to accept excuses that they are not really necessary. It would be have been a reasonable guess that the simple management practices used in this study wouldn’t make a difference with these highly trains professional airline captains—and yet the evidence shows they do matter. That’s a finding worth remembering.

You can find the working paper here. Thanks to Lorenzo Galli of ScienceForWork.com for bringing this to my attention.

David Creelman is CEO of Creelman Research. If you need help elevating the analytics and business savvy of HRBPs then get in touch. You can connect to Mr. Creelman on LinkedIn or email him at dcreelman@creelmanresearch.com

For the latest HR and business articles, check out our main page.