The demand for measures of individual performance in the management of universities and research institutes has been growing, in particular since the early 2000s. The publication of the Hirsch Index in 2005 (Hirsch, 2005) and its popularisation by the journal Nature (Ball, 2005) has given this a strong stimulus. According to Hirsch, his index seemed the perfect indicator to assess the scientific performance of an individual author because “it is transparent, unbiased and very hard to rig”. The h-index balances productivity with citation impact. An author with a h-index of 14 has created 14 publications that each have been cited at least 14 times. So neither authors with a long list of mediocre publications, nor an author with 1 wonder hit are rewarded by this indicator. Nevertheless, the h-index turned out to have too many disadvantages to be wearing the crown of “the perfect indicator”. As Hirsch acknowledged himself, it…
This paper describes the results of a survey regarding the incorporation of societal impacts considerations into the peer review of grant proposals submitted to public science funding bodies. The survey investigated perceptions regarding the use of scientific peers to judge not only the intrinsic scientific value of proposed research, but also its instrumental value to society. Members of the scientific community have expressed – some more stridently than others – resistance to the use of such societal impact considerations. We sought to understand why. Results of the survey suggest that such resistance may be due to a lack of desire rather than a lack of confidence where judging impacts is concerned. In other words, it may be less that scientists feel unable to judge broader societal impacts and more that they are unwilling to do so.
There is something very appealing about the simplicity of using a single number to indicate the worth of a scientific paper.
But a growing group of scientists, publishers, funders, and research organizations are increasingly opposed to the broad use of the Journal Impact Factor (JIF) as the sole measure used to assess research and researchers.
I don’t have time to be doing this, but it’s important. Making time is a state of mind — as, claims Cameron Neylon, is ‘Open’:
Being open as opposed to making open resources (or making resources open) is about embracing a particular form of humility. For the creator it is about embracing the idea that – despite knowing more about what you have done than any other person – the use and application of your work is something that you cannot predict.
There’s a lot to unpack, even in this short excerpt from Neylon’s post. Whether, for instance, the idea of ‘humility’ is captured by being open to unintended applications of ones work — surely that’s part, but only part, of being open — deserves further thought. But I really do think Cameron is on to something with the idea that being open entails a sort of humility.
For those who wish to have access, there is an admission cost: they must invest in the education prerequisite to enable them to understand the language used. Current publication practices work to ensure that the entry threshold for understanding my language is as low as possible. Open access will raise that entry threshold. Much more will be downloaded; much less will be understood.
There’s a lot to unpack here, as well. There’s a sort of jiujitsu going on in this excerpt that requires that one is at least familiar with — if it is not one’s characteristic feeling — the feeling that no one will ever understand. What is obvious, however, is Osborne’s arrogance: there is a price to be paid to understand me, and open access will actually raise that price.
In my original talk on “Open Access and Its Enemies” I traced one source of disagreement about open access to different conceptions of freedom. Those with a negative concept of freedom are opposed to any sort of open access mandates, for instance, while those appealing to a positive concept of freedom might accept certain mandates as not necessarily opposed to their freedom. There may be exceptions, of course, but those with a positive concept of freedom tend to accept open access, while those with a negative view of freedom tend to oppose it. The two posts from Neylon and Osborne reveal another aspect of what divides academics on the question of open access — a different sense of self.
For advocates of humility, seeing our selves as individuals interferes with openness. In fact, it is only in contrast to those who view the self as an individual that the appeal to humility makes sense. The plea is that they temper their individualistic tendencies, to humble their individual selves in the service of our corporate self. For advocates of openness, the self is something that really comes about only through interaction with others.
Advocates of elitism acknowledge that the social bond is important. But it is not, in itself, constitutive of the self. On the contrary, the self is what persists independently of others, whether anyone else understands us or not. Moreover, understanding me — qua individual — requires that you — qua individual — discipline yourself, learn something, be educated. Indeed, to become a self in good standing with the elite requires a certain self-abnegation — but only for a time, and only until one can re-assert oneself as an elite individual. Importantly, self-abnegation is a temporary stop on the way to full self-realization.
Self-sacrifice is foreign to both of the advocate of humility and the advocate of elitism, I fear. Yet it is only through self-sacrifice that communication is possible. Self-sacrifice doesn’t dissolve the individual self completely into the corporate self. Nor does self-sacrifice recognize temporary self-abnegation on the road to self-assertion as the path to communication. Self-sacrifice takes us beyond both, in that it requires that we admit that content is never what’s communicated. A self with a truly open mindset would have to be able to experience this. Alas, no one will ever understand me!
I noticed two articles on sports concerned with officials’ decisions today, and their juxtaposition raises more questions than either, alone.
The first was an article in The Guardian questioning Hawk-Eye, the technology used in tennis to determine whether balls landed in our out of bounds, thus usually to determine points won or lost. Anyone who watched the Wimbledon Gentlemen’s Final yesterday witnessed Novak Djokavic yelling “What is going on?!” at the umpire when calls — including challenges that were resolved by appealing to Hawk-Eye — seemed continually to go against him.The article describes the results of a paper that critiques Hawk-Eye’s accuracy and ends with an interesting question:
The paper concludes that Hawk-Eye should be used as an aid to human judgement (their italics), and that, if used with a little more nuance, it could provide added enjoyment of the games involved and public understanding of technology, its uses and its limitations. What do you think? Do you want a simple binary decision in your sports, or would you rather know the accuracy of Hawk-Eye’s output?
I’m old enough to remember a time before Hawk-Eye, and I recall when it was introduced. I remember being skeptical at the time. Why think that a model of what happened would be better than human judgment? And, really, is the issue which is more accurate? Part of sports is overcoming bad calls. Our collective mania for objectivity borders on madness.
To answer the author’s question at the end of the article: neither. I’d prefer if we took such technologies, including instant replay, completely out of all game-time decisions in sporting events. If leagues want to review officials’ decisions later, after the game has been decided, fine. But this rush to judgment, as if we have to have the ‘objectively’ correct answer right then and there, is a bane to sports.
Do we really enjoy Djokavic’s comparatively mild protests more than John McEnroe’s? You cannot be serious!
On the same page as the Hawk-Eye article is a link to another article in The Guardian, this one on a recent local soccer match in Brazil during which a referee stabbed a player, then was mobbed, stoned, and decapitated by the angry crowd. Say what?! Look, I’m from Alabama, so I’m well aware of people who do really stupid things in the name of sports. As an Auburn fan, I’m glad no one has decapitated Harvey Updyke, yet. But it’s interesting how the story of the Brazilian double murder (yes, the player stabbed by the ref also died) is treated — as an image problem:
Brazil faces mounting pressure to show it is a safe place for tourists before 12 cities host the 2014 World Cup and Rio de Janeiro the Olympics in 2016. The Confederations Cup in June was marked by violence as anti-government protestors angered by the amount of money being spent on the events clashed with police.
So, unless Brazil can clean up its act and tone down the violence to a level that’s acceptable to tourists, the World Cup and the Olympics are in trouble? Someone must be Djoking!
When did we lose all perspective about what’s important? Sports are a form of entertainment, one that’s more entertaining when we take it seriously. But it’s possible to take sports too seriously. Killing someone is an extreme example, obviously. But treating murder as an image problem reveals that we take sports too seriously in other ways, as well. As if the real problem is whether Brazil will respond to mounting pressure to show it is a safe place for tourists in time to save the World Cup. Imagine the economic fall-out were people to stay away in droves! As if that were the problem, rather than the problem being our thinking of sports in economic — or technoscientific — terms. Or our thinking of the protests as a problem for sports, rather than an expression of a cultural moment.
So, to rephrase the question raised by the initial article: Do you want simple, binary decisions rendered by someone — or something — else, or would you rather do the hard work of thinking?
SPRU Professor Andy Stirling is beginning a series in The Guardian on the precautionary principle. Stirling’s first article paints an optimistic picture:
Far from the pessimistic caricature, precaution actually celebrates the full depth and potential for human agency in knowledge and innovation. Blinkered risk assessment ignores both positive and negative implications of uncertainty. Though politically inconvenient for some, precaution simply acknowledges this scope and choice. So, while mistaken rhetorical rejections of precaution add further poison to current political tensions around technology, precaution itself offers an antidote – one that is in the best traditions of rationality. By upholding both scientific rigour and democratic accountability under uncertainty, precaution offers a means to help reconcile these increasingly sundered Enlightenment cultures.
Stirling’s work on the precautionary principle is some of the best out there, and Adam Briggle and I cite him in our working paper on the topic. I look forward to reading the rest of Stirling’s series. Although I’m a critic of the Enlightenment, I don’t reject it wholesale. In fact, I think rational engagement with the thinkers of the Enlightenment — and some of its most interesting heirs, including Stirling and Steve Fuller, who’s a proponent of proaction over precaution — is important. So, stay tuned for more!
jbrittholbrook 