Dangers of fracking prepublication publicity
This short post from Andy Revkin combines several of my interests: fracking, peer review, and scholarly communication.
Tag Archives: peer review
Blue skies, impacts, and peer review | RT. A Journal on Research Policy and Evaluation
This paper describes the results of a survey regarding the incorporation of societal impacts considerations into the peer review of grant proposals submitted to public science funding bodies. The survey investigated perceptions regarding the use of scientific peers to judge not only the intrinsic scientific value of proposed research, but also its instrumental value to society. Members of the scientific community have expressed – some more stridently than others – resistance to the use of such societal impact considerations. We sought to understand why. Results of the survey suggest that such resistance may be due to a lack of desire rather than a lack of confidence where judging impacts is concerned. In other words, it may be less that scientists feel unable to judge broader societal impacts and more that they are unwilling to do so.
Blue skies, impacts, and peer review | Holbrook | RT. A Journal on Research Policy and Evaluation.
Book Review: Peer Review, Research Integrity, and the Governance of Science: Practice, Theory, and Current Discussions | LSE Review of Books
On Learning from Peer Review | Extending ‘Peers’ to Include non-Academics
Absent from the many analyses and discussions of scientific peer review are two intangible but very important byproducts: 1) feedback to the applicant and 2) exposure of the reviewers to new hypotheses, techniques, and approaches. Both of these phenomena have a virtual mentoring effect that helps move science forward. Such learning can occur as a consequence of both manuscript review and grant application review, but the review of grant applications, by its very nature, is more iterative and impacts the direction in which research moves very early in the investigation.
Opinion: Learning from Peer Review | The Scientist Magazine®.
There are at least two funding agencies that recognize this phenomenon in the actual design of their peer review processes, so they deserve mention. The idea is to include non-academics as peer reviewers precisely to effect the sort of co-production of knowledge the article above suggests.
The first is STW, the Dutch Technology Foundation. I outline their peer review process in this article, available Open Access.
The second is the US Congressionally Directed Medical Research Program. Details of their peer review process are available on their website here.
What does it take to be ‘liked’ by scientists?
Scientists don’t like me. Or, at least, they don’t show any evidence of liking what I have to say about NSF’s Broader Impacts Merit Review Criterion. Last week, I blogged this ScienceInsider interview (here and on the CSID blog) with an unnamed congressional aide connected with Rep. Lamar Smith and his efforts to add “an extra layer of accountability” to NSF’s Merit Review Process.
I also left a couple of comments in the comments section under the article itself. It’s possible for readers of ScienceInsider to press buttons to indicate their agreement — or not — with comments. The site then tracks the number of likes or dislikes (expressed by pressing up or down carrots), displays them with each comment, and moves those comments with the most likes up to the top.
Guess whose comments are dead last in line?
Here are the two most-liked comments:
lollardy • 3 days ago Studying dairy production in China is a very poor choice for an example of what constitutes a bad grant. It has direct relevance to something most people in America consume every day. It could reduce cost for millions, increase food safety, improve the quality or nutrient density of a commonly consumed item, etc. Every time I hear a story on Fox about a “wasteful” study, I can usually think of ten ways it could benefit people and industry here. Somehow I think the time would be better spent putting in an “additional layer” to cover pentagon spending.
Kenneth DeBacker • 4 days ago A lot of smoke is being blown by Rep. Lamar Smith’s aide. The aide’s answers are slick and cover’s the real intent of the bill- to politicize the sciences through selective funding or defunding of areas of study Republicans do not like. The most egregious example would be the ban on studying gun violence in America.
Each of them has received twelve likes.
I suppose if I were simply to say that Congress is out to politicize science or that Smith is out of his depth or that scientists should be left alone to pursue research however they wish, scientists might like that. But I’m willing to give Smith the benefit of the doubt, at this point. My contention is that he (or his aide) doesn’t yet understand the revisions to NSF’s Merit Review Process. If he did, then I think he’d see that accountability is already built into the process. I think Smith should not introduce the High Quality Research Act, but instead should seek to monitor how scientists respond to the new Broader Impacts Criterion.
But there’s a real problem with what I’m suggesting. And it’s not that Smith is a Republican out to get science. The problem is that scientists themselves don’t understand the Broader Impacts Criterion. They don’t understand that this is their last, best hope to preserve their academic autonomy while meeting accountability demands. And they don’t want to hear it, either.
To see my comments on the ScienceInsider interview, simply follow this link and scroll to the bottom of the page.
ScienceInsider’s latest on NSF Merit Review and the draft ‘High Quality Research Act’
Obama Promises to Protect Peer Review in Salute to NAS – ScienceInsider
ScienceInsider‘s take on Obama’s speech at the National Academy of Sciences:
Obama Promises to Protect Peer Review in Salute to NAS – ScienceInsider.
Autonomy and accountability — old and new thinking
OLD THINKING
Academics have tended to view autonomy as freedom from constraint by the state. We want to be able to go wherever our thoughts lead us, without ‘outside’ interference. This interpretation of autonomy as freedom from constraint is a negative definition, appealing to what is absent. It says, ‘Hands off!’
Politicians have tended to view accountability only in economic terms — that is, accountability has been reduced to accounting and conceived narrowly as something like return on investment (ROI).
Under such ‘old thinking’, autonomy is opposed to accountability, since accountability conceived as demonstrable ROI puts constraints on the autonomy of researchers conceived as freedom from such demands.
NEW THINKING
Autonomy means self-legislation, rather than freedom from constraint. Under this ‘new thinking’ on autonomy, the point is not for academics to be free from all constraint (the negative definition rooted in old thinking), but rather for academics to give themselves whatever constraints they are subject to.
Accountability means being able to give an account, in the Socratic sense of the term. This is by no means limited to a notion of ROI, though such may be included in the account one is expected to give.
Under this ‘new thinking’ on autonomy and accountability, the accountability demand is expressed as the formal demand that one give an account. There is no reduction of that account to economic concepts. One is free to offer whatever justification one sees fit. In other words, one is able to exercise one’s autonomy to respond to the generic accountability demand: account for yourself! Under this ‘new thinking’, then, accountability and autonomy are compatible.
Someone may be quick to point out that these are not really new definitions of the terms. So much the better!
jbrittholbrook 