lott v. levitt and the meaning of replication

via volokh: the seventh circuit u.s. court of appeals has affirmed the dismissal of john lott's libel suit against steven levitt. the case hinges on the following critique of mr. lott's more guns, less crime in freakonomics:

Then there was the troubling allegation that Lott actually invented some of the survey data that support his more-guns/less-crime theory. Regardless of whether the data were faked, Lott’s admittedly intriguing hypothesis doesn’t seem to be true. When other scholars have tried to replicate his results, they found that right-to-carry laws simply don’t bring down crime.

reading this, i figured the case might hinge on the allegation of falsified data in the first sentence. instead, it came down to the meaning of replication in the last sentence. in truth, that last sentence is very close to my own reading of the literature -- and my lecture descriptions of mr. lott's (admittedly intriguing) more guns/less crime hypothesis. i'd joke that i'm tempted to sue professor levitt and mr. dubner for plagiarizing my lecture notes, but i suspect they're in no mood for jokes about lawsuits.

in my view, we social scientists have dodged a real bullet on this one. were mr. lott to prevail in this case, the chilling effect on faculty speech would do the impossible: it would add a whole new level of weasel words to our literature reviews, rendering them even more boring and hesitant. they'd look something like the following:

"When certain other scholars may have tried to partially replicate his results, some found that in some cases right-to-carry laws sort of don't significantly bring down some forms of crime. Or not."

in any case, justice evans' brief and well-written opinion might make fine reading in a proseminar class on professional ethics. a few passages:

John Lott, an academic and economist, believes that his reputation was sullied by Freakonomics, the popular and off-beat book written by Steven Levitt and Stephen Dubner. Lott’s name was mentioned in one paragraph of the 200-page book, and understood this passage to be an accusation of scholarly dishonesty. Offended, he filed suit against Levitt and HarperCollins, the publisher of the book, claiming that he had been defamed. The district court dismissed this claim after concluding that the passage could reasonably be read as a refutation of Lott’s controversial theories and not a swipe at his integrity. Lott now appeals...

Lott contends that Levitt’s refutation of his more-guns/less-crime hypothesis can be read only as a smear of his professional reputation and is therefore defamatory per se. Using an academic definition of “replicate,” Lott maintains that the passage means that others repeated, to a tee, his technical analysis but were unable to duplicate his results, suggesting that he either faked his data or performed his analysis incompetently...

The book relies on anecdotal evidence and describes with only the broadest strokes the statistical methodologies used. In this context, it is reasonable to read “replicate” in more generic terms. That is, the sentence could mean that scholars tried to reach the same conclusion as Lott, using different models, data, and assumptions, but could not do so. This reading does not imply that Lott falsified his results or was incompetent; instead, it suggests only that scholars have disagreed with Lott’s findings about the controversial relationship between guns and crime. By concluding that this more generic definition of “replicate” is reasonable, we are not assuming that the reader is a simpleton. After all, econometrics is far from conventional wisdom. We are, however, taking into account the context of the statement and acknowledging that the natural and obvious meaning of “replicate” can lie outside the realm of academia for this broadly appealing book.