Reinventing the Wheel, Again

What happens when someone who reads Nietzsche also reads science and technology policy documents? Click on the link to see one answer.

Negative Results – Not Just For Journal Articles?

David Bruggeman offers another twist on turning negatives into positives here. I’d like to add to this that it’s part of an ethos of not being afraid to make mistakes, even of valuing them. Some might refer to this as an entrepreneurial attitude.

Pasco Phronesis

There is a strong positive bias in how scientific knowledge is generated, written about, and measured.  It is easier to find research proving a hypothesis than replication studies that fail to confirm earlier findings.  It is easier to access explanations of why certain technologies came to be than studies about why we don’t have flying cars, or some other breakthrough promised to us through the magnificence of science and technology.  It’s an enormous hole in our understanding of the world, facilitated by the mores of the scientific reward system.

The same is true for metrics.  While the number of ways one can assess the impact of a particular paper is changing, many of the ‘alt’ metrics emerging are still thinking primarily in positive terms.  At least that’s the proposition of J. Britt Holbrook and some of his colleagues at the University of North Texas.  In a letter to Nature

View original post 185 more words

San Francisco Declaration on Research Assessment — Well done, DORA

Anyone interested in research assessment should read this with care.

DORA.

It’s been presented in the media as an insurrection against the use of the Journal Impact Factor — and the Declaration certainly does … ehr … declare that the JIF shouldn’t be used to assess individual researchers or individual research articles. But this soundbite shouldn’t be used to characterize the totality of DORA, which is much broader than that.

Honestly, it took me a few days to go read it. After all, it’s uncontroversial in my mind that the JIF shouldn’t be used in this way. So, an insurrection against it didn’t strike me as all that interesting. I’m all for the use of altmetrics and — obviously, given our recent Nature correspondence (free to read here) — other inventive ways to tell the story of our impact.

But, and I cannot stress this enough, everyone should give DORA a careful read. I’m against jumping uncritically on the bandwagon in favor of Openness in all its forms. But I could find little reason not to sign, and myriad reasons to do so.

Well done, DORA.

‘Pure hype of pure research helps no one’ says Sarewitz; what this says about freedom

I tend to agree with a lot of what Dan Sarewitz argues here:

Pure hype of pure research helps no one : Nature News & Comment.

But I also want to suggest that there’s an argument to be made against the High Quality Research Act that goes beyond Sarewitz’s claim that it helps no one.

To be fair, that’s just the headline. Sarewitz also claims something I think is a bit more controversial — that the HQRA is really nothing to get too worried about. Not only does it help no one, but also it doesn’t hurt anyone.

This strikes me as mistaken. I’ll try to articulate why in terms of the distinction between negative and positive freedom I’ve been exploring. Here goes.

First, I agree that the HQRA helps no one.  But it’s not just that the HQRA is redundant — though this is certainly true. It’s also that it doesn’t allow us to do anything more to demonstrate our accountability, as I think the Broader Impacts Criterion does. In other words, it doesn’t increase anyone’s positive freedom.

Second, it actually decreases our negative freedom. By requiring NSF to re-certify what the merit review process already certifies (at least when it’s working as designed), this ‘added layer of accountability’ actually just increases the kind of bureaucratic red tape we should be trying to decrease if we’re interested in an efficient government. This makes about as much sense as the Florida Blue Ribbon Task Force’s suggestion to charge more for classes in majors that supposedly won’t result in better jobs for graduates. Majors that result in higher paying jobs actually should be in greater demand, and so should cost more, not less. But not according to the Blue Ribbon Task Force (see pp 22-3).

Finally, I think the HQRA might be a case study in how to reconcile notions of positive and negative freedom — or at least how to think of both ideas of liberty as possibly working together. It’s sort of a test. Sometimes, a policy that might increase our positive freedom can be seen as decreasing our negative freedom. I think the NSF’s Broader Impacts Criterion is a case in point. Yes, it places an additional burden on researchers, and so in this sense it is a limitation on their negative freedom. But it also increases their positive freedom, so that a trade-off is possible. The HQRA, on the other hand, decreases our negative freedom without also increasing our positive freedom. Any policy that doesn’t increase either our positive or our negative freedom is highly questionable — or so I am suggesting.

Will The High Quality Research Act Diminish Our Collective Cognitive Dissonance?

I plan to follow up on David’s post and Dan’s argument in Nature, soon. Until then, enjoy this!

Pasco Phronesis

The High Quality Research Act is a draft bill from Representative Lamar Smith, Chair of the House Science, Space and Technology Committee.  Still not officially introduced, it has prompted a fair amount of teeth gnashing and garment rending over what it might mean.  The bill would require the Director of the National Science Foundation (NSF) to certify that the research it funds would: serve the national interests, be of the highest quality, and is not duplicative of other research projects being funded by the federal government.  The bill would also prompt a study to see how such requirements could be implemented in other federal science agencies.

There’s a lot there to explore, including how the bill fits into recent inquiries about specific research grants made by the National Institutes of Health (NIH) and the NSF.  (One nice place to check on this is the AmericanScience team blog.)

But…

View original post 278 more words

Altmetrics for the Nature correspondence on negative metrics of impact

Fascinating.

Article details.

We need negative metrics, too / Nature

Keith Brown, Kelli Barr, and I have a short piece published in the new issue of Nature.

The correspondence also contains a link to a slightly revised version of our original submission. Since Nature keeps everything behind a pay wall, here is that link.

Very interested in hearing everyone’s thoughts on the idea that seemingly negative events could be turned to indicate positive impact.