The Dissemination of Scholarly Information: A Question of (In-)Sufficient Instruments
July 16th, 2008
Sören Auer posted today to the okfn-discuss lists about plans for Open Participatory Research. Reading this I was particularly struck by his mention of ‘open peer review’ as this seemed directly related to some recent ideas of my own. Specifically I’ve been working on an economics paper with an academic colleague on the subject of dissemination of scholarly information. This is still at an early stage but the basic ideas in it are quite simple — as set out in the current introduction which can be found below.
Introduction
It is well known that in order to (completely) address a given number of (independent) goals one needs an equal number of instruments. For example, if one is seeking to address both congestion and pollution in relation to road-traffic, a single instrument, for example petrol taxes, will be insufficient to address both goals exactly (of course it will allow one to address both goals partially). The same issues arise in relation to the dissemination of scholarly information.
Here too there are multiple independent goals. Traditional academic publishing provides but a single instrument. Originally there was nothing that could be done (for reasons discussed further below), but changes in technology render this restriction to a single instrument unnecessary. Unfortunately, the two-sided nature of the journal market (based on expectations), combined with the current evaluation structure of academia, continue to lock society into this inefficient restriction. Open-access journals provides one, though as we shall argue, not the only, or even most efficient, way to improve the current situation.
Goals and Instruments
Crudely put, the two main goals (or tasks), in relation to the dissemination of scholarly information are:
- Distribution (transmission of the data/information) — `Making material available for Reading’
- Filtering/Recommendation — `Deciding what to Read’
It seems clear that these are distinct and hence require distinct instruments for their achievement. Journals can be seen as a single instrument which traditionally have tried to address both ends simultaneously. The deficiency of academic publishing can then be seen as one of insufficient instruments. Initially, because of the limitations of reproduction and distribution technologies, there was little that could be done about this. Today with the advent of the computer and the Internet this is no longer the case and it is possible to these two distinct goals with two distinct instruments.
Why then did restricted-access Journals originally come about? The answer lies in technology, in particular the nature of the technology available in earlier periods to manage distribution (printing and transmission). When many journals were originally started the cost of transmitting information was very high. Journals essentially acted as a club good by which the costs of reproduction and distribution could be (efficiently) shared (the efficiency arising here from economies of scale).
At the same time, given the limited ‘bandwidth’ it was natural for Journals to take on some filtering role in order to economize on the scarce transmission capacity. In this situation, dissemination is limited and with only one instrument available (Journals) and it is natural to tie dissemination and filtering together (with filtering in many ways secondary). Once filtering is being done it is natural for journals to `tie’ material to the journal explicitly via copyright — though at an early stage given the scale economies of journals this explicit tying was not actually necessary and was probably done for simple legal convenience.
With the advent of digital communications, in particular the Internet, bandwidth is no longer scarce. What is now scarce is attention. In this setup the importance of a journal is not its role in efficiently sharing reproduction and distribution costs but its role as a filtering mechanism. However, while when distribution is central it is natural to `add-in’ filtering, it is not natural, or necessary, to tie distribution in to filtering when filtering is central. In fact it seems clear that distribution and filtering can be done entirely separately (i.e. one can have two instruments focused on distribution and filtering respectively). The Open Access movement can be seen as largely about achieving this separation: with open access there is no longer a connection between access/distribution (which would be free) and the filtering mechanism (the choice of which articles go in a particular journal).
That said the `Open Access’ movement still has a large focus on journals — albeit open-access ones. This, in our view, is a mistake. Technology has also affected possibilities for filtering. In particular it is no longer clear why the centralized mechanism of official peer-review and journals is superior to alternative decentralized options. The last decade, has witnessed widespread, and often successful, experimentation with distributed voting and evaluation mechanisms (for example Slashdot’s story-ratings and Google’s link-based site rankings).
Thus, to be more radical, it may make sense not only to remove centralized control of distribution but also centralized control of filtering. A more distributed (market-like?) filtering mechanism would permit the same freedom (and same status?) to participate in reviewing and recommendation as it does in the production of scholarly information. At the same time it would deliver greater transparency, and by permitting `free-entry’ in filtering, would allow greater specialization, greater diversity, increased participation and greater competition.
As such, the gains from going ‘open’ are not simply wider access, but a reduction in the time and energy scholars spend finding and processing research information. Significantly, this second item, which is less frequently mentioned in discussions of ‘Open Access‘, may well be the most significant.
July 17th, 2008 at 6:33 pm
Hi Rufus,
I don’t know anyone in the OA movement who believes that OA intrinsically favors traditional peer review. On the contrary, the prevailing view (and certainly my view) is that OA is compatible with every kind of peer review, from the most conservative to the most innovative. For that reason, I think it’s best to treat the achievement of OA and the reform of peer review as independent projects. If they’re both important, and I think they are, then they should proceed in parallel. We have to link them if our preferred form of peer review depends on OA, which is the case for most forms of retroactive, distributed forms of peer review. But if we link them when we don’t have to, then the OA movement loses potential allies or delays progress while we try to achieve some (unlikely) consensus on the best form of peer review.
July 18th, 2008 at 9:34 am
Hi Peter,
Thanks for your thoughtful comments on those ideas about scholarly communication.
I hope it was clear we weren’t suggesting that OA itself favours traditional peer review. Furthermore, as I hope is clear from what I wrote, making material OA, whether in journals or not, is clearly essential to permitting the kind of separation of distribution and filtering that we think is valuable. As you say OA is compatible with any kind of review and filtering process.
However, and this was the major point we were trying to make, the major benefits of OA may lie, not in the freer access per se, but in the innovative filtering and reviewing processes these make possible. Thus, while the two topics of OA and filtering/reviewing are not coupled I do think they are linked. In particular, changes in filtering/reviewing depend on OA (though not vice-versa I would emphasize) and many of the benefits of OA are only realized by changes in the filtering/reviewing process.
July 23rd, 2008 at 10:21 am
The strength of journals lies in a third function of academic publication, namely “Boosting scholarly reputation”, that is partly served by having one’s article read, but is not reducible to it. This is as important a function of publication as gaining readers, and at certain points in an academic’s career it is the most important.
A rival to the current journal model needs to solve the problem of how to give robust indicators of significance to publications or it will not gain traction.
October 2nd, 2008 at 1:08 pm
Surely, if scholarly knowledge is as objective as it is supposed to be -then the quality of any scholarly research should be apparent from studying the material presented.
So what exactly is the function of “boosting scholarly reputations”?
Within the career and publications marketing system -the function is obvious: Respected people, publications and institutions attract money, readers, writers, students -and thus eventually (unless unmasked) even more prestige in a self-reinforcing system.
However, the more important question remains, How does scholarly reputation relate to the validity of the scholarly knowledge produced or reviewed by the scholar(s) concerned?
Unfortunately, here the tautological weakness of academic knowledge itself becomes frighteningly visible. Who and how are scholarly reputations built and how can they be questioned -when within the rules of the community involved-it is exactly those with the reputations who are the most qualified (simply on the grounds of their reputation) to judge their own reputations.
This is not a purely academic social-ritual based issue: It also relates to the whole nexus of belief systems upon which any single academic “fact” can be tested. At college, I was once told the charming story that Helmholtz claimed that Marconi was a fool -because radio waves traveled in straight lines and so without a mirror in the sky, or the earth being flat, radio waves could never reach America from the UK. Logically, Marconi’s succesful experiment proved the earth was flat. Although it seems that in fact, the even less plausible explanation for his contra-factual experiment seems now more acceptable: There really is a big mirror in the sky!
I’m sure that there are many other examples of peer review (based on widely held misconceptions) that have “disproved” facts that were later proven to be correct. I understand the tektonic plate theory in geology is an example. Frenology and the Ether were also widely accepted theories of phenomena supported by reputable academics. Who knows how many unknown examples are still haunting our scientific institutions.
As a pioneer artist/programmer I seriously suspect that In my own field a whole scala of both scientific and artistic information and propaganda organisations -inside and outside educational institutions, aided by government and commercial interests have developed a complete industry of professional reputations -that, in my view, are built entirely on sand. The sociological and economic implications of this are enourmous -although completely invisible, simply because of the tautology of the process that has been (perhaps carefully and deliberately) built up over the years. The power of Microsoft (and other companies) for example, seems closely related to the lack of information under the public regarding the nature of the commercial computing industry and any alternatives that might be potentially or actually available. This “digital feudalism” is supported by mainstream newsmedia -and by academics and educational institutions who are less than critical regarding the social effects of commercial exploitation of digital technology -and the way the public are educated to use it. It is also supported by deeply rooted (western) cultural habits (and an educational system that still preserves the “two cultures” system -while pretending to transcend it).
Surely, one ot the most potentially valuable functions that a trully “open access” can have is to break though the dangerous tautology of “peer review” and “scholarly reputation”. In a world in which so called “global communication media” are increasingly used to create a universal intelectual and cultural monoculture -the problem is far from academic.
Since WWII the American socio-economic system has been increasingly seen as the only “authority” around. It’s “scholarly reputation” was invincible. Now it is proving to the world that this invincibility can only be maintained by openly breaking all its own rules and using taxpayers money to keep it alive. One can only wonder how many hidden subsidies helped to create the solid reputaion of the system in the first place -and all the scholarly reputations that supported it.