Peerless Science

Peer Review and U. S. Science Policy

By Daryl E. Chubin & Edward J. Hackett

Subjects: Science And Technology, Public Policy
Series: SUNY series in Science, Technology, and Society
Paperback : 9780791403105, 267 pages, July 1990
Hardcover : 9780791403099, 267 pages, July 1990

Alternative formats available from:

Table of contents

Preface and Acknowledgments

1. The Centrality of Peer Review


What Is "Peer Review"?
Symbolism and Chauvinism
Approach to a Dialogue
Five Axioms about the Culture of Science
The Policy Context
Studying Grants Peer Review
Summary: A Study of Policy and Practice


2. Peer Review in Theory and Practice


Peer Review: Origins and Current Practices
Peer Review at the National Institutes of Health
Peer Review at the National Science Foundation
Trends in Proposal Volume and Award Rates

Criticisms of Peer Review: More Than Sour Grapes
Rustum Roy's Critique
Roy Redux

Criteria for Evaluating Peer Review


3. Funding Success and Failure


Entering the Black Box
Agency-Sponsored Studies
Agency-Funded Studies
Independent Studies

Litigation and Science: The Story of a Social Survey
Obtaining the Sample of Scientists
The Appeal
Lessons from Litigation

Empirical Studies of Grants Peer Review
Surveys of Scientists
Success in the Pursuit of Research Support
The Consequences of Failure
Attitudes toward Peer Review

Competition for Research Support
Upward Creep and Resubmission: A Protracted Process
Anomie and Reform

Inferences: Peer Review and Conflicts of Interest


4. Peer Review and the Printed Word


The Purposes of Scientific Publication and Journal Peer Review
New Burdens
"Truth" as Consequence

Principles and Practices: The Tensions of Peer Review
Journal Practices
Studies of Journal Peer Review
The Dangers of Studying Journal Practices
What the Studies Tell Us

An Analysis of Referee Commentary
A Sample of Referee-Editor Discourse
The Rhetoric of Referees
Toward a Theory of Refereeing

Vignette" The Sanctity of Journal Peer Review and a Conspiracy of Ignorance

Sober Considerations


5. Scientific Malpractice and the Politics of Knowledge


Malpractice Defined
Can Peer Review Help?

The Emergence of Public Science
Malpractice as a Grappling with Norms
Fraud in Research: The Social Structure of Scapegoating

Misconduct and Public Science
The Mainstreaming of Dispute
Out of the Nursery, Into the Night

Earmarking and the Pork Barrel
Autonomy, Accountability, and the Politics That Intervene


6. Augmenting Peer Review: The Place of Research Evaluation


Bibliometrics as Research Evaluation: The Promise
The First Generation of Bibliometrics (1961-1974)
The Second Generation (1975-Present)
Lessons Learned
Lessons Applied

Evaluative Bibliometrics and Beyond
Converging Partial Indicators
Qualitative Scientometrics

Challenges and New Directions
Science Push-Application Pull
Broker Roles



7. Peer Review and Unauthorized Science Policy


Impediments and Nagging Issues

Reforming Peer Review: Slouching Toward Iconoclasm
Modest Improvements at the Margin
Changing Roles and Rules
Toward a Process of Reform

Meta-Analysis, The Science Critic, and Science Policy
The Science Critic in Action
Policy Research and the Public Interest

Science Policy and the Flywheel of Peer Review
Inventing Tools for Perception and Foresight


Appendix: Survey of NCI Applicants





This book examines the structure and operation of peer review as a family of quality control mechanisms and looks at the burdens placed on the various forms of peer review. Assuming that peer review is central to the functioning of U. S. science policy, Chubin and Hackett explore the symbolic and practical value of peer review in the making, implementing, and analysis of this policy.

Daryl E. Chubin is Senior Analyst in the Science, Education, and Transportation Program, Office of Technology Assessment, U. S. Congress. Edward J. Hackett is Assistant Professor in the Department of Science and Technology Studies at Rensselaer Polytechnic Institute.


"Peer review is the principle on which the internal governance system of science has traditionally depended. During the past ten years there has been a great deal of evidence suggesting that that assumption is no longer valid (if it ever was), and that many of the strains in the science-government relationship in the US are traceable either to the assumption itself or to the ways it is implemented. Chubin and Hackett examine the assumption and its implementation from different perspectives and explore its role in the science-government relationship.

"The authors display the ability to take peer review and use it as a springboard to illuminate a central issue of our time: the relationship between the conduct of science and the larger society in the US in the waning years of this century. The writing itself — colorful, interesting, heartfelt — is surprising and refreshing. " — William A. Blanpied, National Science Foundation