Evidence-based Management: Blog

Predicting Employee Performance: Schmidt and Hunter's Masterpiece

Industrial psychologists Frank L. Schmidt and his close colleague, the late John E. Hunter did an impressive series of research papers on employee selection and performance, and are also widely admired for popularizing meta-analysis, in large part, because their writings are so approachable and they were so willing to help any scholar to learn the method. In brief, meta-analysis is a method that enables researchers to summarize quantitative findings across larger numbers of studies. If you want to learn a bit more about what meta-analysis is and its strengths and weaknesses, check out the discussion and links on meta-analysis on Wikipedia, and if you are really curious about the kinds of findings this method can yield, check out the meta-analysis blog (where you can learn things like, yes, there is strong evidence that eating SPAM increases your risk of stomach cancer).

Their most amazing – and perhaps most useful – meta-analysis was “The Validity and Utility of Selection Methods in Personnel Psychology Practical and Theoretical Implications of 85 Years of Research Findings,” which was published in Psychological Bulletin in 1998. Schmidt and Hunter used correlations from every study they could find that was conducted over the past 85 YEARS to determine which methods of selecting new employees were best – and worst – for predicting subsequent performance.
They looked at 19 different selection methods. The findings?

The top 5: general mental ability (e.g., IQ tests), work sample tests, integrity tests, conscientious, and structured job interviews

The bottom 5: the T& E point method (a scoring system reflecting how much experience, education, and training employees have had in the past), years of education, interests, and by far the worst predictors, graphology (handwriting analysis) and age.

Certainly, these findings don’t comfort opponents of IQ tests. Yet is interesting to see the power of a simply asking a person to do the job – it is a lot easier to fool a potential employer in an interview. As Jeff Pfeffer and I discuss in Hard Facts, we keep finding that the best companies and managers are masters of simple and obvious knowledge, rather than complex and obscure solutions.

Although few U.S. companies use handwriting analysis, my students from other countries — including France and Israel – tell me that this is a common method, and that they often find themselves worrying about what their handwriting will reveal to the graphologist. Finally, I was delighted (as I am getting older by the minute) to see that the age of the applicant was – across these studies – the most useless of all predictors of job performance

....................................................................................................................................................
Posted in Academic research by Bob Sutton | | Permalink | Comments (1)

Creativity and Constraint, Part 1

Many managers and academics believe that creativity thrives in work environments that are relatively unconstrained compared to those found in traditional businesses. In this view excessive bureaucracy, tiny cubicles, strict P&L requirements and TPS reports are anathema to new ideas. Bell Labs physicist Sharad Raanathan expressed this in a recent (8/21) Wall Street Journal article on Bell Lab’s newfound market orientation, saying that “it’s important to have some people released from the constraints of immediate or even remote applicability of their research” in order to create groundbreaking innovations.

Conversely, many creative professionals claim that creativity needs constraint. In this view, people are more creative when they have roadblocks that must be worked around. Too little constraint is paralyzing and constraint helps people focus on the problem. Google’s Vice President of Search Products, Marissa Mayer, makes just such a claim in this Stanford Technology Ventures Program video where she declares that, “creativity loves constraint.”

So which is it? Does constraint hinder or help creativity? As with many topics involving creative work, the answer is that it’s probably a good deal more complex than that. Constraints take many forms, and managers must judiciously apply and remove various kinds and levels of constraint in order to encourage creativity. Fortunately, there is some excellent theory and evidence available (which will be covered in this series of blog entries) that can help make the issue more manageable.

Jon Elster’s Theory of Constraint

The best work, to my mind, on the subject of creativity and constraint comes from Columbia University political scientist Jon Elster. In 2000 Elster wrote a book titled Ulysses Unbound (Amazon link) that draws on evidence from a variety of domains to form what he calls ‘constraint theory’. Constraint theory has many facets, but in it’s most basic form holds that people ‘self-bind’ themselves to arbitrary sets of constraints when there isn’t sufficient constraint present in either their environment or the problem at hand.

Elster’s book didn’t receive much attention from the management community largely because it is a book ostensibly outside the field of management with a broad, multi-disciplinary scope. This shouldn’t be a turn-off though. Nearly all of the best management theories originated in other fields (e.g. sociology, economics, psychology). Good social theory can be a wonderful source of knowledge about management once it is tailored to the particular problems and phenomena of the workplace.

Elster seeks to explain situations where more options do not equate to better decisions or creativity. In these situations, there are two types of constraints that are beneficial. The first type of beneficial constraint occurs when it is desirable for specific options to be “unavailable, available only with a delay, or at greater cost.” An example of this would be the situation where the lack of an exit option for a project forces a team to do better work than if they believed it was possible to cancel the project (assuming they went ahead with it). The other type of beneficial constraint occurs when it is desireable for fewer options to be available, but there is no desire to exclude specific options. Elster provides a nice example of this in the, “film director (who) decides to shoot in black and white so as not to be tempted by the facile charms of color photography.”

Both of these types of beneficial constraint (specific and arbitrary) are important to creativity and innovation, but can be difficult to for managers to get a handle on. In my next blog entries I will discuss the role of specific and arbitrary constraint in more detail.

| Main

August 29, 2006

Predicting Employee Performance: Schmidt and Hunter's Masterpiece

Creativity and Constraint, Part 1