Developing Peer Review
On interdisciplinarity in academic publications and some policy changes of Philosophical Transactions
This is the third of a six-part series on the establishment of the modern scientific academy. With each part of the series, we will discuss different topics pertaining to the ethics of research. The present post touches on the interdisciplinarity of academic journals, and proposes four stages for the analysis of the development of peer review in the Royal Society. Each part will soon appear in A History of Research (Ethics) and your comments are most welcome below and here.
Al-Ruhawi’s 9th century “Al-Tabib” introduced us to a governance structure for the review of medical notes. He argued for a “local committee of physicians” who would ensure that notes followed appropriate procedure; the latest medical standards, if you will. But how could we ensure that the local council itself knew what the best practices in medicine were? The world had already seen that the library could help store and access knowledge — it could prove useful for the members of the council. We also had scientific institutes to bring groups of researchers together — again, communities of physicians could work in tandem to produce treatises for their field and peers. With the Royal Society, we see England follow in the footsteps of Italy and Germany. However, this story will focus on the history of the main publication of the Royal Society. I will first introduce the evolution of interdisciplinarity in academic journals, in light of the two oldest journals currently in circulation, and then introduce four stages that we see in the Royal Society’s development of its modern peer review process.
Institutionalising Discipline
To see how interdisciplinarity has evolved — even if only in rather coarse terms — I will compare the world’s two oldest academic journals: the French Journal des Sçavans, published since 1st January, 1665; and the Royal Society’s Philosophical Transactions, first published 6th March 1665.
The French Journal had already set the stage for future journals to learn from, as Vittu tells us (2005: 530; rather Potts paraphrases in English):
“Included among those that imitated [the French Journal’s] model of organization, production, and dissemination are Philosophical Transactions (London, established 1665), Acta eruditorum (Leipzig, 1682), Nouvelles de la république des lettres (Amsterdam, 1684), Giornale de’letterati (Ferrara, 1688), De Boekzaal van Europe (Rotterdam, 1691–4), and Galeria di Minerva (Venice, 1691–4)” (Potts, 2011: 69).
Unfortunately, of the above, only the French Journal and the British Transactions continue to be published today, hence my focusing on them. Their long history allows for their analyses and the conclusion that, over the past three centuries, academic journals have become a great deal more specialised, or less interdisciplinary. On the one hand, the French Journal shifted from a vast array of disciplines (see more here) toward “belles-lettres” in the 19th century (Potts, 2011: 70). This move was consolidated by the Journal coming under the Académie des Inscriptions et Belle-Lettres at the start of the twentieth century (Contamine, 2020: 6).
On the other, the British publications was less clearly interdisciplinary in its beginnings. Banks’ (2009) analysis shows that the Transactions did touch on many varied topics, but concludes that “the Philosophical Transactions is mainly restricted to what would now be thought of as scientific subjects” (ibid.: 18). Anecdotally, the variety seems to have reached its peak when the founder and sole editor of the Transactions — Oldenburg — lifted the obituary for the influential mathematician, Pierre de Fermat, from the French Journal without crediting them (Moxham, 2016: 472).
Setting aside controversies as to which publication influenced which most (McCutcheon, 1924, seems quite bitter about the French influencing the British), specialisation in the Royal Society’s publication becomes clear from 1887 onward, as the Transactions have since been divided into two series: Series A — covering mathematics and physical sciences — and Series B — covering biology (Royal Society, n.d.). The increased specialisation of journals is a well-documented phenomenon and is a concern raised further below when discussing peer review.
Valuing Interdisciplinarity
Despite this historical trend, in recent years, a great deal of funding has been directed towards interdisciplinary research (Trussell et al., 2017). But the “interdisciplinary ‘buzz’” (ibid.) often clashes with the institutions that employ researchers and train them in individual disciplines (Poteete et al., 2010). We must question, then, how researchers are to conduct their work with integrity when they are expected to perform across numerous disciplinary boundaries. Might there be a decrease in the quality of research if scientists engage in “epistemic trespassing” (Ballantyne, 2019)? Furthermore, is there a problem if there is a gap between the cognitive resources that academic institutions offer and what research funders look for? I do not intend to respond to such complex issues here, but I hope it is clear that these are important questions in the realm of research ethics.
Peer Review at the Royal Society
The second major topic we can ponder on in light of the Royal Society’s history is peer review. Recall, once again, Al-Ruhawi’s “local council.” It was very explicitly a council and not any one person’s job to review medical notes. In the case of the French and British publications, it took some years to figure this out. The exact way peer review emerged for the French Journal will not be discussed, but note that they had already established an editorial board by 1699 (Vittu, 2002: 186; Banks, 2018). Conversely, the Royal Society took until 1752 to even begin publishing the Transactions as an institution. Prior to this, there was what I call the Null Period — to give it a statistically significant name. There are three phases that followed. Below, I briefly each of the four phases of the Royal Society’s establishment of modern-day peer review.
The first phase for the Royal Society’s peer review process was the Null Period, between the 1665 founding of the Transactions and 1752 (Banks, 2018: 4), when publications were a private enterprise (Moxham & Fyfe, 2018). At the start, the Royal Society’s Secretary, Henry Oldenberg, had taken on the task of compiling the letters sent to him for their discussion in Society meetings but also to ensure that members who missed meetings would have a copy (Banks, 2018: 2). Oldenberg made every effort to show that his publications were a private enterprise (@AHRCPhilTrans,
n.d.), but they were still often perceived as a product of the Royal Society (Moxham & Fyfe, 2018: 870).
In the 1750s, this confused attribution of the Transactions was exploited by a failed candidate for the Society’s fellowship. In 1751, John Hill published “A Review of the Works of the Royal Society of London; containing animadversions on such of the papers as deserve particular observation” (1751). The critique contained rather bitter remarks about the Society’s president but, more importantly, Hill critiqued the weaker papers that had appeared in Philosophical Transactions, some of which dated all the way back to 1665. Hill blamed the Society, as an institution, for this low quality of the publication (Moxham & Fyfe, 2017: 870).
The response of the Society’s leadership was precisely to implement the collective editorial responsibility that Hill had insinuated — something like what the Journal des Sçavans had implemented over half a century earlier. This was partly facilitated by the recent passing of the Society’s editor, with which the institution’s president regained control. Thus, from 1752, Philosophical Transactions came under the Society’s financial and editorial management (ibid.). I call the period from 1752 to 1832 the Referring Period, based on the new “referring process” whereby a member of the newly formed “Committee of Papers” would occasionally be asked to scrutinise a paper and refer it to pre-planned committee meetings (Fyfe et al., 2017: §2).
The third stage of the development of peer review at the Royal Society can be called the Refereeing Period. It emerged as a response to two publications. The first was mathematician Charles Babbage’s “Reflections on the Decline of Science in England” (1830). Babbage offered a pessimistic view of the state of science in England, particularly compared to France and Germany. More pertinently, Babbage produced the report that followed the Society’s 1827 committee, where “reformers” sought to change the focus of Philosophical Transactions. They saw the publication as a way to indicate the Society’s “positive recommendation of [a published paper’s] author as a man of science” (Moxham & Fyfe, 2017: 874).
The second critical publication was from the physician Augustus Bozz Granville. “Science without a Head; or the Royal Society Dissected, by one of the 687 F.R.S.” (1830) agreed with Babbage insofar that it viewed the Society’s approach to reviewing as inadequate. This was on the basis of both (i) insufficient numbers of members on the Committee of Papers, and (ii) a lack of qualification amongst reviewers given the increasingly specialised nature of papers received by the Society (ibid.: 122; Moxham & Fyfe, 2017: 874).
The response to these critiques was announced by the duke of Sussex in 1832, when the “referring process” in place since 1752 would be used for all — not only some — manuscripts for the Committee of Papers’ consideration. The change might not seem too important, it might even be seen as paying lip-service to Babbage and Granville’s critiques. The duke of Sussex was even quite frank about the new process being inspired by what was being done already at the Parisian Académie des Sciences. However, the change was significant in light of the 1831 creation of a second periodical, Proceedings (ibid: 877). This second publication, unlike the much better established Transactions, was not subject to peer review. The Society’s approach was much lighter-touch, so the publication could be more frequent and reach a much wider public. In the meantime, the Committee of Papers became more selective with regards to manuscripts for Transactions. This meant an increase in quality of content of the traditional publication, which entailed a greater prestige, both for the publication and the Society, which had been more clearly linked since 1752. Furthermore, the reformers criticised by Babbage and Granville did get their way, after all, as the increasing prestige of the Transactions meant that getting published would mean a discount on the membership fee for fellows; publishing in Proceedings was not compensated.
However, refereeing itself came under disrepute over the coming decades. As Moxham & Fyfe (ibid.) explain, refereeing came to be seen as a slow and cumbersome editorial process that was out of touch with the needs of researchers in otherwise fast-moving fields. To increase efficiency, the bottle-neck caused by having only one editor was relieved by transferring editorial tasks to a wider group of associate editors. However, I am taking the liberty to mark 1990 as the start of the fourth stage of the Society’s development of peer review: the Board Period, which continues to this date. This is because it was in 1990 when the Committee of Papers, established in 1752, was finally disbanded. After over two centuries, there was once again an editorial function (ibid.: 885).
Concluding
The above outline is very coarse, and the referenced papers by Fyfe provide a much more nuanced analysis. There are also questions about the emergence of the internet, although I briefly discuss these when telling the history of the Académie des Sciences.
What I would like to take away from the evolution of the Society’s peer review process relates with the challenges that the process itself faces. The 1751 exchange with Hill, for example, showed us that peer review processes can be established for the purpose of protecting an institution’s reputation, with which one might wonder “is this good enough? Is the point of peer review not to ensure the highest standards of published research?” The Refereeing Period then began in light of major concerns raised by Babbage and Grenville, including the preparedness of the Committee of Papers to even review the manuscripts they received. Finally, we have the Board Period beginning with the abolishment of the Committee of Papers and the establishment of clear editorial roles for fellows. The question here relates to the debate started in the 1960s, where they questioned the appropriateness of having editorial structures that slowed down the publication of quality research. Conversely, one might ask for the editorial skills of the Society’s fellows who have been trained as expert researchers, not necessarily journal editors. More on the topic of the relationship between researchers and journalism will be discussed in the story of l’Académie des Sciences (part V of this series); and an introduction to the problematic of groups working as one is offered when discussing Durkheim — a story still under review.
NB: I am very much #NotAHistorian! I have tried my best to draw on credible sources but very sincerely apologise for any errors I may have committed in the above text.
Once again, if you would like to help improve this story — or have thoughts on A History of Research Ethics — I would love to hear from you!
