Briner, Rob B. & Denise M. Rousseau. Evidence-Based I-O Psychology: Not There Yet. Industrial & Organizational Psychology, Mar. 2011, 4:1, p. 3-22

Abstract: Evidence-based practice is now well established in several fields including medicine, nursing, and social policy. This article seeks to promote discussion of whether the practice of industrial-organizational (I-O) psychologists is evidence based and what is needed to make I-O psychology an evidence-based discipline. It first reviews the emergence of the concept of evidence-based practice. Second, it considers the definitions and features of evidence-based practice, including evidence-based management. It then assesses whether I-O psychology is itself an evidence-based discipline by identifying key characteristics of evidence-based practice and judging the extent these characterize I-O psychology. Fourth, some key strategies for promoting the use of evidence in I-O psychology are considered: practice-oriented research and systematic reviews. Fifth, barriers to practicing evidence-based I-O psychology are identified along with suggestions for overcoming them. Last is a look to the future of an evidence-based I-O psychology that plays an important role in helping consultants, in-house I-O psychologists, managers, and organizations become more evidence based.

Commentaries:

• Evidence-Based I–O Psychology: What Do We Lose on the Way? Catherine Cassell (p. 23-26)
• The Neglect of the Political: An Alternative Evidence-Based Practice for I-O Psychology. Dean Bartlett (p27-31)
• I–O Psychology: We Have the Evidence; We Just Don’t Use It (or Care To) Amanda L. Thayer, Jessica L. Wildman and Eduardo Salas (p. 32–35)
• Is There a Fly in the “Systematic Review” Ointment? Michael J. Burke (p. 36–39)
• The Kryptonite of Evidence-Based I–O Psychology. George C. Banks and Michael A. McDaniel (p. 40–44)
• You have free access to this contentEvidence-Based I–O Psychology: Lessons From Clinical Psychology. Victor M. Catano (p. 45–48)
• Why Evidence-Based Practice in I–O Psychology Is Not There Yet: Going Beyond Systematic Reviews. Gerard P. Hodgkinson (p. 49–53)
• Assessing the Uptake of Evidence-Based Management: A Systems Approach. Georges Potworowski and Lee A. Green (p. 54–56)
• Broadening the View of What Constitutes “Evidence”. Matthew A. Cronin and Richard Klimoski (p. 57–61)
• Putting Evidence in Its Place: A Means Not an End. Wayne A. Baughman, David W. Dorsey and David Zarefsky (p. 62–64)
• The Universe of Evidence-Based I–O Psychology Is Expanding. Richard A. Guzzo (p. 65–67)
• The Path Forward to Meaningful Evidence. Jazmine Espejo Boatman and Evan F. Sinar. (p. 68–71)
• Evidence-Based Approaches in I–O Psychology Should Address Worse Grumbles. Jean M. Bartunek (p. 72–75)

Response from Authors:

• Evidence-Based I-O Psychology: Not There Yet but Now a Little Nearer? By: Briner, Rob B.; Rousseau, Denise M. (p76-82)
  Abstract: Our focal article sought to promote discussion of evidence-based approaches to practice in industrial-organizational (I-O) psychology. It did so by describing the meanings and origins of evidence-based practice, evaluating the extent to which I-O psychology practice is currently evidence-based, and considering the role of systematic reviews in promoting evidence-based practice. The commentaries on our focal article raised many interesting and important points. In our response, we divide them into two broad categories. The first category consists of comments and objections that arise from what we believe to be misinterpretations of evidence-based practice and our focal article. The second category contains those comments that in various ways extend and elaborate the issues raised in our focal article. Although we are not there yet, we hope that these commentaries will take us a little nearer to an evidence-based approach to I-O psychology.

Academy of Management Journal editors’ forum on the research-practice gap in human resource management. Academy of Management Journal, 50:5, October 2007.  [full-text of this issue is available to subscribers to EBSCO's Business Source Complete]
The forum includes ten short articles on the topic: why doesn’t human resource management use evidence?

1. Editor’s Foreword: tackling the “great divide” between research production and dissemination in human resource management. p. 985-986 [full-text in Business Source Complete]
2. Rynes, Sara L., Tamara L. Giluk, & Kenneth G. Brown. The very separate worlds of academic and practitioner periodicals in human resource management: implications for evidence-based management. p. 987-1008 [full-text in Business Source Complete]
3. Cascio, Wayne F. Evidence-based management and the marketplace for ideas. p. 1009-1012 [full-text in Business Source Complete]
4. Cohen, Debra J. The very separate worlds of academic and practitioner publications in human resource management: reasons for the divide and concrete solutions for bridging the gap. p. 1013-1019 [full-text in Business Source Complete]
5. Guest, David E. Don’t shoot the messenger: a wake-up call for academics. p. 1020-1026 [full-text in Business Source Complete]
6. Latham, Gary P. A speculative perspective on the transfer of behavioral science findings to the workplace: “The times they are a-changin’”. p. 1027-1032  [full-text in Business Source Complete]
7. Lawler III, Edward E. Why HR practices are not evidence-based. p. 1033-1036 [full-text in Business Source Complete]
8. Rousseau, Denise M. A sticky, leveraging, and scalable strategy for high-quality connections between organizational practice and science. p. 1037-1042 [full-text in Business Source Complete]
9. Saari, Lise. Bridging the worlds. p. 1043-1045. [full-text in Business Source Complete]
10. Rynes, Sara. Let’s create a tipping point: what academics and practitioners can do, alone and together. p. 1046-1054 [full-text in Business Source Complete]

Armstrong, J. Scott. Findings from Evidence-Based Forecasting: Methods for Reducing. International Journal of Forecasting, 22:3, 2006, p. 583-598 [full-text available to subscribers to Elsevier's ScienceDirect]

Banerjee, Abhijit Vinayak. Making Aid Work: How to Fight Global Poverty – Effectively.  Boston Review, July/August 2006.

Briner, Rob B. Is HRM evidence-based and does it matter? IES Opinion, April 2007.

Cowen, Amanda, Boris Groysberg, & Paul Healy. Which Types of Analyst Firms Are More Optimistic? Journal of Accounting & Economics, 41:1-2, April 2006, p. 119-146 [full-text available to subscribers to Elsevier's ScienceDirect]

Denyer, David. Evidence-informed Management [slides ]. Presented at the 2007 Evidence-Based Management Conference at Carnegie Mellon University.

Dixon-Woods, Mary, Shona Agarwal, Bridget Young, David Jones, & Alex Sutton. Integrative Approaches to Qualitative and Quantitative Evidence. Health Development Agency (HDA), 2004

Dvorak, Phred. Why Management Trends Quickly Fade Away… Wall Street Journal, June 26, 2006

Evidence-Based Management. FCW.com’s Culture & Context [blog]. June 16, 2006

Evidence-Based Management . United Nations Online Network in Public Administration and Finance. [A PowerPoint Slide presentation based on the article by Pfeffer and Sutton in Harvard Business Review, January 2006, p. 63-74

Evidence-Based Management and the Doing-Knowing Problem. Strategy Central  [blog]. June 25, 2006

Fisher, Thomas. Architects Behaving Badly: Ignoring Environmental Behavior Research, Harvard Design Magazine, Fall 2004/Winter 2005, Number 21

“At a talk a few months ago I learned that there is a view amongst some architects and certainly among most environmental psychologists interested in buildings that architects do not pay sufficient attention to social science evidence in their work – and it sounds like it has quite a few parallels with EBMgt. Here’s the original article that I believe got debate going within architecture. Also, like so many areas where evidence-based ideas have been discussed, it appears that this is most developed in health-care.” (Rob B. Briner, Professor of Organizational Psychology, University of London)

Gray, Bradford H., Michael K. Gusmano, & Sara R. Collins. AHCPR and the Changing Politics Of Health Services Research: Lessons from the Falling and Rising Political Fortunes of the
Nation’s Leading Health Services Research Agency. Health Services Research: AHCPR Politics, June 25, 2003

“It is a nice story about how difficult it has been to get research done on the ‘system’ of health care, rather than on specific diseases” (Jeffrey Pfeffer)

Hansen, Fay. Special Report: Compensation & Salary Forecast—Where’s the Merit-Pay Payoff? Workforce Management, November 3, 2008, p. 33-39

Hofmann, Paul B. The Ethics of Evidence-Based Management, Healthcare Executive, Jan/Feb2010, Vol. 25, No. 1, p. 48-51 [full-text available to subscribers to Business Source Complete]

HSE Contract Research Report 356/2001: A critical review of psychosocial hazard measures . Prepared by The Institute for Employment Studies for the Health and Safety Executive, 2001.

HSE Research Report 024: Review of existing supporting scientific knowledge to underpin standards of good practice for key work-related stressors- Phase 1 [slides ]. Prepared by The Institute for Employment Studies for the Health and Safety Executive, 2002

Hymowitz, Carol. Executives Must Stop Jumping Fad to Fad And Learn to Manage. Wall Street Journal, May 15, 2006

Joyce, Kerry, Roman Pabayo, Julia A Critchley, & Clare Bambra. Flexible working conditions and their effects on employee health and wellbeing, Cochrane Database of Systematic Reviews 2010, Issue 2 [full-text available to subscribers to Wiley InterScience]

“As we know there a few systematic reviews in management but this recently published review relates to an important management topic and also gives a good feel for how systematic reviews are very different to the usual reviews we produce in management. If you have access to Wiley full text you should be able to get the PDF from here – but this link also shows the basic structure of the review.” (Rob B. Briner, Professor of Organizational Psychology, University of London)

Kearns, Paul. Defining and measuring the value of leadership . The original version of this article was published by the Leadership Trust in 2005 (www.leadership.org.uk) in “Leadership under the microscope” for their 8th annual conference at which Kearns delivered a keynote speech entitled “Real Leaders Get Real Results”.

Kovner, Anthony R., Jeffrey Pfeffer, & David Fine. What More Evidence Do You Need? Harvard Business Review, May 2010, 88:5, p. 123-127 [full-text available to subscribers to Business Source Complete]

Leslie, Keith, Mark A. Loch, & William Schaninger. Managing your organization by the evidence. The McKinsey Quarterly, no. 3, 2006 [requires free registration]

Madden, Bartley J.  Applying a Systems Mindset to Stock Valuation . Working Paper, Revised on July 16, 2008 [also available at SSRN]

Madden, Bartley J. Guidepost to Wealth Creation: Value-Relevant Track Records .  Journal of Applied Finance, Fall/Winter 2007

Management “Science” and the PMBOK. Project: Project Management Theories, Techniques and Tools [blog]. January 16, 2006

Morrel-Samuels, Palmer, Ed Francis, & Steve Shucard. Merged Datasets: an Analytical Tool for Evidence-Based Management. California Management Review, Fall 2009, Vol. 52, No. 1, p. 120-139 [full-text available to subscribers to Business Source Complete]

Outcomes Evidence & Monitoring Discussion Papers . OEM Project – Second Kete January 2007. “The Outcomes Evidence and Monitoring project team is leading the work on six actions from Better Outcomes For Children. We want to ensure that this work progresses guided by the thinking of staff at all levels of the organisation and by key stakeholders. In this kete we share our first thinking and seek your input to the development of both the evidence policy, and the selection of key national indicators.”

Pfeffer, Jeffrey. Don’t Dismiss Office Politics—Teach It. Wall Street Journal, October 24, 2011

Pfeffer, Jeffrey. Evidence-Based Management for Entrepreneurial Environments: Faster and Better Decisions with Less (March 2010). Rock Center for Corporate Governance at Stanford University Working Paper No. 75; Stanford University Graduate School of Business Research Paper No. 2051

Pfeffer, Jeffrey. Evidence of Profit . Human Resource Executive, July 2006

“Conference speaker claims better decision-making can boost the bottom line when those decisions are tied to evidence-based management.” This article consists of excerpts from Pfeffer and Sutton’s book: Hard Facts, Dangerous Half-Truths and Total Nonsense: Profiting from Evidence-Based Management.

Pfeffer, Jeffrey. Jeffrey Pfeffer über Evidenzbasiertes Management und OE: Ein Gespräch via E-Mail [in German and English] (copy edit version ). OrganisationsEntwicklung, Issue 1, 2008

Pfeffer, Jeffrey. Management a Profession? Where’s the Proof? Harvard Business Review, Sept/Oct 2011, 89:9/10, p. 38 [full text in EBSCO Business Source Complete database]

The article discusses the lack of professionalism in business management and what can be done to repair that deficit. In the author’s view, teachers at business schools need to embrace the importance of evidence-based practices. He also cites the need for management journals to pay greater heed to previous research when selecting articles to publish, and for managers to demand that decision-making be based on relevant data.

Pfeffer, Jeffrey. Management a Profession? Where’s the Proof? Harvard Business Review, Sept/Oct 2011, 89:9/10, p. 38

Pfeffer, Jeffrey & Robert I. Sutton. Act on Facts, Not Faith. Stanford Social Innovation Review, Spring 2006

Pfeffer, Jeffrey & Robert I. Sutton. Benchmarking: Dangerous Half Truths . CriticalEYE REVIEW: The Journal of Europe’s Centre for Business Leaders, issue 15, 2006

Pfeffer, Jeffrey & Robert I. Sutton. Demanding Proof. Industrial Engineer, 38:6, June 2006  [full-text available to subscribers to ABI/INFORM or EBSCO's Business Source Complete database]

Abstract: Engineering is well positioned to be one of the primary leaders for an evidence-based movement. The modern evidence-based medicine movement was founded by David Sackett, and his colleagues at McMaster University in Canada. Studies suggest that physicians trained in evidence-based techniques are better informed than their peers even 15 years after graduating from medical school. Leaders and organizations that practice evidence-based management follow three basic guidelines: 1. They put aside beliefs, ideologies, conventional wisdom, and dangerous half-truths about organizations and management. Instead, they seek, face, and act on the facts. 2. They are committed to gathering the facts and information required to make more informed and smarter decisions. 3. They adopt an attitude of wisdom, which permits them to act on what they know even as they use the results of their actions to learn new things and update future actions.

Pfeffer, Jeffrey & Robert I. Sutton. Evidence-based Management. Harvard Business Review, 84:1, January 2006 [full-text available to subscribers to EBSCO's Business Source Complete database]

Abstract: The article discusses evidence-based management and creating effective medical organizations. The same care that teaching hospitals take to implement evidence-based medicine should be used to implement evidence-based management. An example of evidence-based management is provided by a discussion of Kent Thiry’s turnaround efforts at Da Vita dialysis centers, which are headquartered in California. An example is given showing the decision-making process used at Oxford’s Centre for Evidence-Based Medicine, which employs the first step of defining the situation in the form of an answerable question.

Pfeffer, Jeffrey & Robert I. Sutton. A matter of fact . First published in People Management, September 28, 2006

Pfeffer, Jeffrey & Robert I. Sutton. The pitfalls of imitating ‘best practices’. Canadian HR Reporter, 19:7, April 10, 2006 [full-text available to subscribers to ABI/INFORM database]

Abstract: There are many problems with seeking to copy “best practices” that ought to give pause to anyone thinking this is the path to success. The first, and possibly most serious problem, is that a lot of “best practices” are based on success stories and anecdotal evidence rather than the best evidence. In medicine, there is a growing evidence-based medicine movement.  This same sort of evidence-based movement needs to be embraced in HR and in management. Evidence-based management is pretty much the opposite of benchmarking and seeking out best practices from others that seems to characterize so much of contemporary HR management.

Pfeffer, Jeffrey & Robert I. Sutton. Profiting from evidence-based management. Strategy & Leadership, 34:2, 2006 [full-text available to subscribers to ABI/INFORM database]

Abstract: Evidence-based management is not just a list of techniques that you can memorize, mimic, and install. It is a perspective for traveling through organizational life, a way of thinking about what you and your company know and what you do not know, what is working and is not, and what to try next. Here are seven implementation principles to help people and companies that are committed to doing what it takes to profit from evidence-based management: 1. Treat your organization as an unfinished prototype. 2. Don’t brag, just use the facts. 3. See yourself and your organization as outsiders do. 4. Evidence-based management is not just for senior executives. 5. Like everything else, you still need to sell it. 6. If all else fails, slow the spread of bad practices. 7. The best diagnostic question is what happens when people fail.

Pfeffer, Jeffrey & Robert I. Sutton. The Real Brain Teaser. People Management, 12:8, April 20, 2006.  [full-text available to subscribers to ABI/INFORM database]

Abstract: The most fundamental assumption we have is that talent is a reasonably fixed characteristic and it is therefore the job of companies and their HR staff to identify, recruit and retain these high potentials: “A” players, stars, or whatever else they are called. The same belief continues in what we do in our workplaces, beginning with hiring for skills and abilities rather than aptitude and attitude, and continuing through investments in career development, which mostly get lavished on those who have been selected to reach higher-level positions, while front-line employees and people with less perceived potential are largely ignored. This idea, that talent is a fixed, identifiable characteristic and that those firms with best people do the best is both partly wrong – a dangerous half-truth – and harmful to people and companies.

Pfeffer, Jeffrey & Robert I. Sutton. Sometimes Less Is More. Leadership Excellence, 23:3, March 2006 [full-text available to subscribers to ABI/INFORM database]

Pfeffer, Jeffrey & Robert I. Sutton. Three Myths of Management. HBS Working Knowledge, March 27, 2006

Pfeffer, Jeffrey & Robert I. Sutton. Trust the Evidence, Not Your Instincts. New York Times, September 3, 2011

Pfeffer, Jeffrey & Robert I. Sutton. Why Managing by Facts Works. Strategy + Business, June 29, 2006

Pritchard, Robert D., Melissa M. Harrell, Deborah DiazGranados, & Melissa J. Guzman. The Productivity Measurement and Enhancement System: A Meta-Analysis [manuscript ]. In press, Journal of Applied Psychology.

Lipshitz, Raanan. Paradigms and Mindfulness in Decision Making: Why the Israel Defense Force (I.D.F.) failed in the Second Lebanon War . October 2007

“An amazing article by an Israeli professor that has some fascinating material on how things went so wrong for the Israeli army in its recent struggles in Lebanon. The article highlights the importance of assumptions, mental models, and mind sets as crucial to making better and better informed decisions.” (Jeff Pfeffer)

Reay, Trish, Whitney Berta, & Melanie Kazman Kohn. What’s the Evidence on Evidence-Based Management? Academy of Management Perspectives, Nov. 2009, Vol. 23, No. 4, p. 5-18  [full-text available to subscribers to Business Source Complete]

Rousseau, Denise M. Is There Such a Thing as “Evidence-Based Management”? Academy of Management Review, 31:2, April 2006, p. 256-269

Rousseau, Denise M. & Sharon McCarthy. Educating Managers from an Evidence-Based Perspective. Academy of Management Learning & Education, 6:1, March 2007, p. 84-101
[full-text available to subscribers to EBSCO's Business Source Complete]

Rousseau, Denise M., Joshua Manning, & David Denyer. Evidence in Management and Organizational Science: Assembling the Field’s Full Weight of Scientific Knowledge Through Syntheses . Prepared for the Annals of the Academy of Management, v. 2 (Rev. Draft)

Signs of the emerging knowledge economy: Part Four. Eclectic Bill  [blog]. June 28, 2006

Schmidt, Frank L. & John E. Hunter. The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings. Psychological Bulletin, 124:2, September 1998, p. 262-274 [full-text available to subscribers to PsycARTICLES]

Shortell, Stephen M., Thomas G. Rundall, & John Hsu. Improving Patient Care by Linking Evidence-Based Medicine and Evidence Based Management. Journal of the American Medical Association, 298, 2007, p. 673-676 [full-text available to subscribers at the JAMA site]

Sortino, Frank, Mark Kordonsky, & Hal Forsey. Evidenced – Based Portfolio Management . A working paper. 2006.

Sutton, Robert I. How a Few Bad Apples Ruin Everything. Wall Street Journal, October 24, 2011

Tilley, Nick. Realistic Evaluation: An Overview . Presented at the Founding Conference of the Danish Evaluation Society, September 2000

“a newly uncovered, very interesting and relevant, article about evaluating interventions, with particular relevance to social policy in general and criminology in particular.” (Jeff Pfeffer)

When Fashion Is Fleeting: Transitory Collective Beliefs and The Dynamics of TQM Consulting. Academy of Management Journal, April 2006, 49:2, p. 215-233  [full-text available to subscribers to ABI/INFORM database]

Young, Robert A. Implementing Evidence-Based Management for Improved Mission Assurance. Journal of Homeland Security, August 2010