The mismeasure of I/O psychology programs

Each year the number of graduate programs in Industrial-Organizational psychology increases and the choice of which graduate programs to apply to becomes more difficult for prospective students. For an unconscionably long time students have coveted a resource to aid them in their search for the program befitting their interests and experience. Winter, Healy & Syvantek (1995) may be commended for addressing the need for an objective measure of program quality. However, Winter et al..s contribution cannot be regarded as a panacea for the ills of survey-like measures (e.g., U.S. News & World Report, March 1995) of graduate program quality until some important concerns have been considered.

First, in the multidimensional space of performance/productivity, it is important to note that Winter et al. (1995) only consider one dimension. Despite the fact that research in I-O psychology has provided ample evidence of the multidimensional nature of performance (e.g., Campbell, McCloy, Oppler, & Sager, 1993), Winter et al..s methodology is based on an obsolete theory that the best possible measure of performance is an objective measure of individual accomplishment, such as number of publications. While we concur with Winter et al. (1995) in their critique of the U.S. News and World Report (March, 1995) methodology, and in their assessment of "subjective" criteria, we disagree that number of publications is an absolute or even adequate proxy for program quality.

Second, the results of Winter et al. (1995) are based on the number of publications found in five journals "unanimously chosen by a group of I-O psychologists" as likely places for I-O psychologists to publish their research findings. It is arguable that this list of journals is limiting in that it neglects research contributions that I-O psychologists make in domains of psychology rarely addressed in these five publications. For example, I-O psychologists have made several important contributions in the areas of item response theory, structural equation modeling, and meta-analysis. These methods for analyzing psychological data represent significant developments for our field despite the fact that they do not always receive treatment in the more mainstream I-O journals. Further, the source for Winter et al.'s justification to use the chosen five journals (Howard, Maxwell, Berra, & Sternitzke, 1985) was published over a decade ago. Few can deny that the research pursuits of I-O psychologists have, since then, diversified. To be sure, Katzell and Austin (1992) list a total of 38 areas of research in which I-O psychologists have become immersed.

Third, and perhaps most importantly, advocating that a graduate program's reputation be determined solely on the basis of the number of publications is, in essence, stating that I-O programs that train students to apply I-O interventions in the work place (rather than to conduct basic research) are without merit. This declaration exacerbates the already substantial separation between the I-O scientist and practitioner. By advocating a uni-dimensional measure of program quality, Winter et al. disregard other important criteria such as number of internships received by graduate students, job placements, or even number of technical reports written. This would indicate that a good measure of program quality is much more difficult to derive than a convenience variable such as number of publications.

The changing face of I-O psychology is reasonably well understood (Dunnette, 1990). The fact that large numbers of graduate students are being trained primarily for applied positions rather than academic-research positions does not mean that I-O psychology will be less able to develop as a scientific discipline. The fact is that graduate programs are currently producing more academic-research Ph.D.s than there are positions to be filled. This changing face does mean, however, that those who discourage the application of I-O psychology to the workplace, by filtering information about graduate program reputations, will be doing a great disservice to the advancement of the field. As we have seen so resplendently, the intrinsic differences between science and practice often result in discord. What our field needs now is not individuals bent on proving that science overshadows practice, or practice overshadows science, but rather individuals who affirm a cooperative and synergistic relationship. Instead of fragmenting we can unite and make our combined knowledge base work for the benefit of humankind. We submit that a good first step towards this goal is recognizing the contributions of all I-O programs, not just those that publish the most.

References

America's Best Graduate Schools. (1995, March 20). U.S. News and World Report, pp.108-109.

Campbell, J.P., McCloy, R.A., Oppler, S.H., & Sager, C.E. (1993). A theory of performance. In N. Schmitt & W.C. Borman (Eds.), Personnel selection. Jossey-Bass: San Francisco.

Dunnette, M.D. (1990). Blending the science and practice of Industrial and Organizational psychology: Where are we and where are we going? In M.D. Dunnette & L.M. Hough (Eds.), Handbook of Industrial and Organizational Psychology (Vol. 1). Palo Alto, CA: Consulting Psychologists Press, Inc.

Howard, G.S., Maxwell, S.E., Berra, S.M., & Sternitzke, M.E. (1985). Institutional research productivity in industrial/organizational psychology. Journal of Applied Psychology, 70, 233-236.

Katzell, R.A., & Austin, J.T. (1992). From then to now: The development of industrial-organizational psychology in the United States. Journal of Applied Psychology, 77, 803-827.

Winter, J.L., Healy, M.C., & Svyantek, D.J. (1995). North America's top I-O psychology doctoral programs: U.S. News and World Report revisited. The Industrial-Organizational Psychologist, 33(1), 54-58.