I recently wrote about how the "talent wars" are likely to be returning soon in the U.S. (and indeed, there are signs they have already returned in places like China and Singapore), and how companies that have treated people well during the downturn will have an advantage in keeping and retaining the best people –and those that have not damn well better change their ways or will face the prospect of their best people running for the exits in concert with the inability to attract the best people. A related question has to do with the problem of determining who the best people might be — what does the best evidence say about the best way to pick new people?
Its is always dangerous to say there is one definitive paper or study on any subject, but in this case there is candidate — a paper I have blogged about before when taking on graphology (handwriting analysis). But there is one article that just might qualify. It was published by Frank Schmidt and the late John Hunter in the Psychological Bulletin in 1998. These two very skilled researchers
analyzed the pattern of relationships observed in peer reviewed journals during
the prior 85 years to identify which employee selection methods were best and
worst as predictors of job performance. They used a method called "meta-analysis" to do this, which they helped to develop and spread. The advantage of this method is — in the hands of skilled researchers like Schmidt and Hunter — is it reveals the overall patterns revealed by the weight of evidence, rather than the particular quirks of any single study.
The upshot of this research is that work sample tests (e.g., seeing if people can
actually do key elements of a job — if a secretary can type or a programmer can write code ), general mental ability (IQ and related tests), and structured interviews had the highest validity of all methods examined (Arun, thanks for the corrections). As Arun also suggests, Schmidt and Hunter point out that three combinations of methods that were the most powerful predictors of job performance were GMA plus a work sample test (in other words, hiring someone smart and seeing if they could do the work), GMA plus an integrity test, and GMA plus a structured interview (but note that unstructured interviews, the way they are usually done, are weaker).
Note that this information about combinations is probably more important than the pure rank ordering, as it shows what blend of methods works best, but here is also the
rank order of the 19 predictors examined, rank ordered by the validity coefficient, an indicator of how strongly the individual method is linked to performance:
1. Work sample tests (.54)
2. GMA tests …"General mental ability" (.51)
3. Employment interviews — structured (.51)
4. Peer ratings (.49)
5. Job knowledge tests (.48) Test to assess how much employees know about specific aspects of the job.
6. T & E behavioral
consistency method (.45) "Based
on the principle that past behavior is the best predictor of future
behavior. In practice, the method involves describing previous
accomplishments gained through work, training, or other experience
(e.g., school, community service, hobbies) and matching those
accomplishments to the competencies required by the job. a method were past achievements that are thought to be important to behavior on the job are weighted and score
7. Job tryout procedure (.44) Where employees go through a trial period of doing the entire job.
8. Integrity tests (.41) Designed to assess honesty … I don't like them but they do appear to work.
9. Employment interviews — unstructured (.38)
10. Assessment centers (.37)
11. Biographical data measures(.35)
12. Conscientiousness tests (.31) Essentially do people follow through on their promises, do what they say, and work doggedly and reliably to finish their work.
13. Reference checks (.26)
14. Job experience –years (.18)
15. T & E point
method (.11)
16. Years of education (.10)
17. Interests (.10)
18. Graphology (.02) e.g., handwriting analysis.
19. Age (-01)
Certainly, this rank-ordering does not apply in every setting. It is also important to recall that there is a lot of controversy about IQ, with many researchers now arguing that it is more malleable than previously thought. But I find it interesting to see what doesn't work very well — years of education and age in particular. And note that unstructured interviews, although of some value, are not an especially powerful method, despite their widespread use. Interviews are strange in that people have excessive confidence in them, especially in their own abilities to pick winners and losers — when in fact the real explanation is that most of us have poor and extremely self-serving memories.
Many of these methods are described in more detail here by the Society for Industrial and Organizational Psychology. Also note that I am not proposing that any boss or company just mindlessly apply this rank ordering, but I think it is useful to see the research.
The reference for this article is:
Schmidt, F.L.
& Hunter, J.E. (1998) The validity and utility of selection methods in
personnel psychology: Practical and theoretical implications of 85 years of
research findings,” Psychological Bulletin, 124, 262–274.
P.S. Note the corrections, thanks Arun!
Leave a Reply