Interesting post on the entrenchment of impact factors in small countries over at A Blog Around the Clock. The third comment down, by Comrade PhysioProf, is particularly troubling, because it's almost certainly true:
So we have what you might call the Impact Factor Paradox: getting a real handle on something as slippery as the value of someone's scientific contributions is inevitably going to be time-consuming and hard; metrics that allegedly measure that value are going to fall along a spectrum from "time-consuming but accurate" to "quick, easy, and horribly flawed"; in a system where time is the limiting resource, there will always be a sort of grim undertow toward the quick-'n-greasy metrics.The bottom line for impact factor--or whatever other quantitative metric might replace or supplement it--is that people are lazy and time is highly rate limiting in the professional lives of academics. People are always gonna rely on something fast and easy to obtain--like a journal impact factor--than they are on something difficult and time consuming to obtain--like a developed opinion about the solidity and importance of a particular published paper.
Anything that is gonna replace impact factor of journals in which scientists publish papers as a metric for comparative assessment of scientific productivity is gonna have to be as braindead easy to deploy as impact factor.
Your thoughts on how to avoid this trend are welcome.
I don't have an answer to this. But while reading it, I was reminded of this most recent entry from Paul Graham. These problems are very similar, leading me to believe it's a Human Organization problem, not anything intrinsic to either Academia or the Business World.
ReplyDeleteThe Other Half Of "Artists Ship"
Hmmmm...good question. How to present a complex subject worthy of further study in a way that "gets to the point" and, based on its brevity, be subject to a misled public?
ReplyDeletePublish in Nature or Science!