A column in last week’s SCIENCE described a study by Michael
Lauer of the NIH who compared NIH peer review of the merits of grant proposals
to the publication record and impact factors of the investigators who submitted
those proposals. He found essentially no correlation between high evaluation by
the NIH study section and future publication productivity. This result has
apparently triggered some hand wringing at the NIH and calls to alter the way
in which study sections work. However, this concern may be misplaced.
Every scientist who is worth
his/her salt can recount experiences where an NIH study section trashed a grant
on a topic that later turned out to be quite important. It has been pointed out
many times that NIH study sections are inherently conservative and tend to
reflect the existing consensus in a field. Major breakthroughs, however, come
from science that disrupts consensus. This has been understood ever since
Thomas Kuhn and The Structure of
Scientific Revolutions. The issue is how to identify breakthrough science
and differentiate it from misguided error. To give the NIH some credit, in
recent years it has implemented a number of granting programs that try to stress
innovation (e.g. the Pioneer awards) as well as funding for beginning
investigators who may have some new ideas.
Almost the converse
situation prevails in publication where the ilk of Nature, CELL and other high profile journals try to focus
exclusively on ‘hot’ topics. Over time
some of the designated hot areas prove to be not so hot and dwindle away, but
nonetheless the ‘hot’ articles will have generated high impact factors. No
wonder there is a discrepancy between NIH funding and publication impact!
Although there is no need to
obsess about the grant/publication disparity, it is clear that the NIH peer
review system could use some renovation (as well as more money to dispense).
The titles and functions of study sections still primarily reflect a
disease/organ system specific orientation. However, current biomedical science
is developing information and insights that cut across traditional boundaries.
If the study section system were more reflective of the thrust of current
research there would probably be fewer misfires on grant funding.
No comments:
Post a Comment