On Rubrics

Faculty Development

This semester I’m attending a series of Faculty Development Workshops at NJIT designed to assist new faculty with such essentials as teaching, grant writing, publishing, and tenure & promotion.

I’m posting here now in hopes of getting some feedback on a couple of rubrics I developed after attending the second such workshop.

I’m having students give group presentations in my course on Sports, Technology, and Society, and I was searching for ways to help ensure that all members contributed to the group presentation, as well as to differentiate among varying degrees of contribution. Last Tuesday’s workshop focused on assessment, with some treatment of the use of rubrics for both formative and summative assessment. I did a bit more research on my own, and here’s what I’ve come up with.

First, I developed a two-pronged approach. I want to be able to grade the presentation as a whole, as well as each individual’s contribution to that presentation. I decided to make the group presentation grade worth 60% and the individual contribution grade worth 40% of the overall presentation grade.

Second, I developed the group presentation rubric. For this, I owe a debt to several of the rubrics posted by the Eberly Center at Carnegie Mellon University. I found the rubrics for the philosophy paper and the oral presentation particularly helpful. I am thinking about using this rubric both for formative evaluation (to show the students what I expect), as well as for summative evaluation (actually grading the presentations).

Third, I developed the individual peer assessment rubric. I would actually have the students anonymously fill out one of these for each of their fellow group members. For this rubric, I found a publication from the University of New South Wales to be quite helpful (especially Table 2).

I’d be quite interested in constructive feedback on this approach.

Free Online Education: Time to Borrow From the Commercial Open Source Playbook | Innovation Insights | Wired.com

Let’s imagine that the top tier of higher education is actually not in the business of selling education. Instead, they are in what I would term the “talent identification” business. The real payoff for universities comes not from selling courses but rather from finding and nurturing talent and then waiting for payback in the form of contributions to their endowments.

Free Online Education: Time to Borrow From the Commercial Open Source Playbook | Innovation Insights | Wired.com.

Thanks to @pahndeepah for pointing this out to me. It’s definitely worth reading. I doubt too many of the business-types we now have running out universities will be able to stretch their imaginations this far.

Even if they can, however, they’ll be up against the habit of thought that sees students as consumers. The sort of loyalty needed to inspire former students to give back to their universities doesn’t mesh well with the idea that students are customers. At most, loyal customers may return to buy something else. But once you ‘buy’ the university degree, there’s not much incentive to go back and buy another. So why should students — seen as consumers — give back to their universities? They’ve already paid for their degree, after all.

Two Watersheds for Open Access?

This past week I taught “Two Watersheds,” a chapter from Ivan Illich’s Tools for Conviviality. I got some interesting reactions from my students, most of whom are budding engineers. But that’s not what this post is about.

I do want to talk a bit about Illich’s notion of the two watersheds, however. Illich illustrates the idea with reference to medicine. Illich claims that 1913 marks the first watershed in medicine. This is so because in 1913, one finally had a greater than 50% chance that someone educated in medical school (i.e., a doctor) would be able to prescribe an effective treatment for one’s ailment. At that point, modern medicine had caught up with shamans and witch doctors. It rapidly began to outperform them, however. And people became healthier as a result.

By the mid 1950s, however, something changed. Medicine had begun to treat people as patients, and more and more resources were devoted to extending unhealthy life than to helping keep people healthy or to restoring health. Medicine became an institutionalized bureaucracy rather than a calling. Illich picks (admittedly arbitrarily) 1955 to mark this second watershed.

Illich’s account of the two watersheds in medicine is applicable to other technological developments as well.

A couple of weeks ago, Richard Van Noorden published a piece in Nature the headline of which reads “Half of 2011 papers now free to read.” Van Noorden does a good job of laying out the complexities of this claim (‘free’ is not necessarily equivalent to ‘open access’, the robot used to gather the data may not be accurate, and so on), which was made in a report to the European Commission. But the most interesting question raised in the piece is whether the 50% figure represents a “tipping point” for open access.

The report, which was not peer reviewed, calls the 50% figure for 2011 a “tipping point”, a rhetorical flourish that [Peter] Suber is not sure is justified. “The real tipping point is not a number, but whether scientists make open access a habit,” he says.

I’m guessing that Illich might agree both with the report and with Suber’s criticism, but that he might also disagree with both. But let’s not kid ourselves, here. I’m talking more about myself than I am about Illich — just using his idea of the two watersheds to make a point.

The report simply defines the tipping point as more than 50% of papers available for free. This is close enough to the way Illich defines the first watershed in medicine. So, let’s suppose, for the sake of argument, that what the report claims is true. Then we can say that 2011 marks the first watershed of open access publishing.

What should we expect? There’s a lot of hand wringing from traditional scholarly publishers about what open access will do to their business model (blow it up, basically). But many of the claims that the strongest advocates of open access are making in order to suggest that we ought to make open access a habit will likely come to pass. Research will become more efficient. Non-researchers will be able to read the research without restriction (no subscription required, no paywall encountered). If they can’t understand a piece of research, they’ll be able to sign up for a MOOC offered by Harvard or MIT or Stanford and figure it out. Openness in general will increase, along with scientific and technological (and maybe even artistic and philosophical) literacy.

Yes, for profit scholarly publishers and most colleges and universities will end up in the same boat as the shamans and witch doctors once medicine took over in 1913. But aren’t we better off now than when one had only folk remedies and faith to rely on when one got sick?

Perhaps during this time, after the first watershed and before the second, open access can become a habit for researchers, much like getting regular exercise and eating right became habits after medicine’s first watershed. Illich’s claim is that the good times following the first watershed really are good for most of us … for a while.

Of course, there are exceptions. Shamans and witch doctors had their business models disrupted. Open access is likely to do the same for scholarly publishers. MOOCs may do the same for many universities. But universities and publishers will not go away overnight. In fact, we still have witch doctors these days.

The real question is not whether a number or a behavior marks the tipping point — crossing the first watershed. Nor is the question what scholarly publishers and universities will do if 2011 indeed marks the first watershed of openness. The real question is whether we can design policies for openness that prevent us from reaching the second watershed, when openness goes beyond a healthy habit and becomes a bane. Because once openness becomes an institutionalized bureaucracy, we won’t be talking only about peer reviewed journal articles being openly, easily, and freely accessible to anyone for use and reuse.

First days teaching at Georgia Tech

So, I walked into my first class on Monday morning around 8:30am. The class doesn’t start until 9:05am; but it’s in the same building as my office, and I wanted to figure out the computer, screen, lights, etc. before students arrived.

I’m going to have to start getting up earlier if I’m going to beat them into class. There were already 7-10 students present. More than half an hour early. For a philosophy class.

“Y’all are here early,” I remarked casually, trying to hide my surprise. “Yes sir,” they replied, very nearly in unison.

Think I’m exaggerating? Take a look at this from freshman convocation (thanks to @ReplyCollective for pointing this out to me).

One hell of an engineer, indeed. I’m looking forward to this year. Should be fun.