Sunday 7 June 2015

My collapse of confidence in Frontiers journals



Frontiers journals have become a conspicuous presence in academic publishing since they started in 2007 with the advent of Frontiers in Neuroscience. When they were first launched, I, like many people, was suspicious. This was an Open Access (OA) online journal where authors paid to publish, raising questions about the academic rigour of the process. However, it was clear that the publishers had a number of innovative ideas that were attractive to authors, with a nice online interface and a collaborative review process that made engagement with reviewers more of a discussion than a battle with anonymous critics. Like many other online OA journals, the editorial decision to publish was based purely on an objective appraisal of the soundness of the study, not on a subjective evaluation of importance, novelty or interest. As word got round that respectable scientists were acting as editors, reviewers and authors of paper in Frontiers, people started to view it as a good way of achieving fast and relatively painless publication, with all the benefits of having the work openly available and accessible to all.
The publishing model has been highly successful. In 2007, there were 45 papers published in Frontiers in Neuroscience, whereas in 2014 it was 3,012 (data from Scopus search for source title Frontiers in Neuroscience, which includes Frontiers journals in Human Neuroscience, Cellular Neuroscience, Molecular Neuroscience, Behavioral Neuroscience, Systems Neuroscience, Integrative Neuroscience, Synaptic Neuroscience, Aging Neuroscience, Evolutionary Neuroscience and Computational Neuroscience). If all papers attracted the author fee of US$1900 (£1243) for a regular article, this would bring in £3.7 million pounds in 2014: the actual income would be less than this because some articles are cheaper, but it's clear that the income is any in case substantial, especially since the journal is online and there are no print costs. But this is just the tip of the iceberg. Frontiers has expanded massively since 2007 to include a wide range of disciplines.  A Scopus search for articles with journal title that includes "Frontiers in" found over 54,000 articles since 2006, with 10,555 published in 2014.
With success, however, have come growing rumbles of discontent. Questions are being raised about the quality of editing and reviewing in Frontiers.  My first inkling of this was a colleague told me he would not review for Frontiers because his name was published with the article. This wasn't because he wanted confidentiality; rather he was concerned that it would appear he had given approval for the article, when in fact he had major reservations.
Then, there have been some very public criticisms of editorial practices at Frontiers. The first was associated with the retraction of a paper that claimed climate denialism was associated with a more general tendency to advocate conspiracy theories. Papers on this subject are always controversial and this one was no exception, attracting complaints to the editor. The overall impression from the account in Retraction Watch was that the editor caved in to legal threats, thereby letting critics of climate change muzzle academic freedom of speech. This led to the resignation of one Frontiers editor**.
Next, there was a case that posed the opposite problem: the scientific establishment were outraged that a paper on HIV denial had been published, and argued that it should be retracted. The journal editor decided that the paper should not be retracted, but instead rebranded it as Opinion – see Retraction Watch account here.
Most recently, in May 2015 there was a massive upset when editors of the journals Frontiers in Medicine and Frontiers in Cardiovascular Medicine mounted a protest at the way the publisher was bypassing their editorial oversight and allocating papers to associate editors who could accept them without the knowledge of the editor in chief. The editors protested and published a manifesto of editorial independence, leading to 31 of them being sacked by the publisher.   
All of these events have chipped away at my confidence in Frontiers journals, but it was finally exploded completely when someone on Twitter pointed me to this article entitled "First time description of dismantling phenomenon" by Laurence Barrer and Guy Giminez from Aix Marseille Université, France. I had not realised that Frontiers in Psychology had a subsection on Psychoanalysis and Neuropsychoanalysis, but indeed it does, and here was a paper proposing a psychoanalytic account of autism. The abstract states: "The authors of this paper want to demonstrate that dismantling is the main defense mechanism in autism, bringing about de-consensus of senses." Although the authors claim to be adopting a scientific method for testing a hypothesis, it is unclear what would constitute disproof. Their evidence consists of interpreting known autistic characteristics, such as fascination with light, in psychoanalytic terms. The source of dismantling is attributed to the death drive. This reads like the worst kind of pseudoscience, with fancy terminology and concepts being used to provide evidence for a point of view which is more like a religious belief than a testable idea. I wondered who was responsible for accepting this paper.  The Editor was Valeria Vianello Dri, Head of Child and Adolescent Neuropsychiatry Units in Trento, Italy. No information on her biography is provided on the Frontiers website. She lists four publications: these are all on autism genetics. All are multi-authored and she is not first or last author on any of these*. A Google search confirmed she has an interest in psychoanalysis but I could find no further information to indicate that she had any real experience of publishing scientific papers. There were three reviewers: the first two had no publications listed on their Frontiers profiles; the third had a private profile, but a Google search on his name turned up a CV but it did not include any peer-reviewed publications.
So it seems that Frontiers has opened the door to a branch of pseudoscience to set up its own little circle of editors, reviewers and authors, who can play at publishing peer-reviewed science. I'm not saying all people with an interest in psychoanalysis should be banished: if they do proper science, they can publish that in regular journals without needing this kind of specialist outlet. But this section of Frontiers is a disastrous development; there is no evidence of scientific rigour, yet the journal gives credibility to a pernicious movement that is particularly strong in France and Argentina, which regards psychoanalysis as the preferred treatment for autism. Many experts have pointed out that this approach is not evidence-based, but worse still, in some of its manifestations it amounts to maltreatment.  What next, one wonders? Frontiers in homeopathy?
Like the protesting editors of Frontiers in Medicine, I think the combined evidence is that Frontiers has allowed the profit motive to dominate. They should be warned, however, that once they lose a reputation for publishing decent science, they are doomed. I've already heard it said that someone on a grants review panel commented that a candidate's articles in Frontiers should be disregarded. Unless these journals can recover a reputation for solid science with proper editing and peer review, they will find themselves shunned.


*The Frontiers biography suggests she is last author on a paper in 2008, but the author list proved to be incomplete.
** Correction: Shortly after I posted this, Stephan Lewandowsky wrote to say that there were 3 editors who resigned over the RF retraction, plus another one voicing intense criticism

56 comments:

  1. Perhaps I am reading too much into your initial comments. It sounds like you are not into Open Access author pays model in general? How do you feel about PLOS journals?

    ReplyDelete
    Replies
    1. I'm a staunch supporter, provided scientific quality remains high. Have recently published some of my best work in BMC Psychology, Peer J and PLOS One.

      Delete
    2. Hi Dorothy Bishop,
      I have read with interest your comments on my article and about Frontiers. I provide you below new researches we are doing in France mixing qualitative and quantitative methods to highlight how psychoanalysis process is working and how we can observe psychic mechanism describe by psychoanalysis. May be you have never heard about it because it’s rather new researches. My thesis is not yet publishing because I had the grade of Doctor (PhD) in November 2013. But you can find it in www.theses.fr . The Frontiers’ article is about one of the conclusions I have made and it is a mini review.

      First, I agree with you about Frontiers as being “a nice online interface and a collaborative review process that made engagement with reviewers more of a discussion than a battle with anonymous critics”. And I say more, I don’t know you as a researcher, you don’t know me either but we can speak together about my work. I find that great.
      The interest of Frontiers is obviously that the editorial decision to publish is based on objective appraisal of the researches. And my researches are based on objective appraisal because I follow a scientific method ant a quantitative method (statistics with “r Bravais Pearson” and I have worked with the laboratory University’s statistician). During 2 years, I take video of my sessions with 3 different children in autistic state. I collect 127 videos. Afterwards, I write hypothesis about a psychic process called dismantling by D. Meltzer (Yale University) in 1975. Neither psychoanalysts nor clinical psychologist wrote about it since Meltzer in 1975, but lot of them speak about it. So I decide to observe it, quantify it and describe it from a clinical point of view. I have read L. Mottron’s articles too but it’s not really the same process even if it’s talking about senses.
      What is a scientific method? Like you, I look in Google and I found: “The scientific method is a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. To be termed scientific, a method of inquiry is commonly based on empirical or measurable evidence subject to specific principles of reasoning. The Oxford English Dictionary defines the scientific method as "a method or procedure that has characterized natural science since the 17th century, consisting in systematic observation, measurement, and experiment, and the formulation, testing, and modification of hypotheses."

      https://en.wikipedia.org/wiki/Scientific_method

      I follow this scientific method to highlight that dismantling is an observable phenomenon in nature (children with autism). In the hospital where I was working, I have seen this children having a special comportment describe by Meltzer as dismantling. He said that looks like a toy witch represente a dog made of wooden beads and held together by strings (1975, p.12). But his sayings aren’t scientific !!! It’s an analogy (“a pseudo-science and religious belief” you say, I agree with you because nobody had made my kind of researches for such psychic process and lot of my collegues speak instead of elaborate a scientific protocole). So I have decide to look for standards of proof to make real observations (pink pointer on the diagram). I thought about “why does that pattern occur” and “formulate hypotheses” (salmon pointer) and “develop testable predictions” (blue pointer). And I gathered data to test predictions (green pointer). You allready have read a part of the development of my general theory in Frontiers. The other parts are still in my thesis. Today, I’m working with a neuroscientific, Katia Dauchot, for understanding where are the neuronic ways of this psychic mechanisms that psychoanalysis describe for years (white pointer).
      (sorry I have to continue on an other page)

      Delete
    3. We are looking for that you say (“The source of dismantling is attributed to the death drive”). What is the death drive? How can we make it observal? What and where is the energy of the drive? Action potential? Na+ Cl- K+? Or something else? Freud was looking for this energy too, but he never find it (I remind you that he was a neurologist and his first work was on crawfish’s neuron (ref: Neurosciences” 3e ed Mark F. Bear, Barry W. Connors, Michael A. Paradiso)). Freud is beyond us, he is overtake by many, psychoanalysis is not.
      For instance, I give you an extract of my thesis below. You can see the development over two years of dismantling’s defence mechanism for the child Case 1 (“H1” hypothesis 1, “demant” is dismantling, “%” is the duration percent of dismantling phenomenon in a session):

      http://www.theses.fr/?q=laurence+barrer (Barrer, 2013, p. 174) Fig. 1 Cas 1 H1% de démantèlement.

      Anyone who works with a person in an autistic state can make observation of this defence mechanism as I have done myself, so my method is reproducible. For the qualitative method of my researches, I use the IPA (Interpretative Phenomenological Analysis, Smith and Osborn 2003) for my fourth hypothesis to highlight why this special persons use it.

      Secondly, when you say “Their evidence consists of interpreting known autistic characteristics, such as fascination with light, in psychoanalytic terms”, it’s false because I don’t interprete what I see in videos. I count up the duration with a timer when the different factors I have determine appear only simultaneously. These are the standard of proof describe in the article: I put on the timer when they appear on video and I put the timer off when they disappear. I make no interpretation, I watch and make diagram. Afterwards, I take my sessions and try to understand why the child use the dismantling process at that specific time. I use here the IPA which is a clinical observation method. So I can make hypothesis on why the process appears in that condition. Of course I know for a long time that persons with autism are fascinating by light (Meltzer, 1975 and Mottron in 2000s) but not only by light. In my researches fascination with light is only a component.

      Delete
  2. All good points, but I think that shunning all Frontiers is kind of throwing out the baby with the bathwater. I see things like Frontiers as a bit like twitter; it's an excellent forum for finding good, interesting, open research, but it's also inherently open to abuse and for the proliferation of shite. We just have to find ways of filtering it out.

    ReplyDelete
    Replies
    1. I don't see why it should be 'inherently open to abuse'. All that is needed is proper editorial controls, plus ensuring that editors are people with reasonable experience of publishing papers.
      One way they might rescue a reputation would be to evaluate all their editors and require that they meet some minimal standard.
      Another way would be to publish all the reviews with the paper - this is what they do at PeerJ and BMC Psychology, and it helps counteract those who say that there isn't adequate peer review.

      Delete
    2. BMC is also happy to publish pseudo-science alongside their more respectable journals:

      http://www.biomedcentral.com/bmccomplementalternmed

      http://www.cmjournal.org/

      http://www.chiromt.com/

      Delete
    3. Aargh! Truly depressing that the complaints I raise re Frontiers has led to a revelation that all publishers are as bad as each other...

      Delete
    4. yes, singling out Frontiers seems naive. parallels get even more specific: http://onlinelibrary.wiley.com/doi/10.1111/j.1745-8315.2011.00507.x/abstract

      Delete
    5. But question is what do we do about it?
      Just accept that life is imperfect, or put pressure on publishers to do a better job?
      Or , as I have suggested previously, take control of the process?

      Delete
    6. i honestly fail to see why life would be more perfect when things you and i consider pseudoscience would be published by a different publisher. what's the issue here? someone might read The Journal of Bollocks without being prepared to find bollocks in it? because it was published by BMC/Elsevier/Wiley/Whatever?

      Delete
    7. Perhaps. But you don't think the fact that Wiley, etc., is publishing this stuff casts a shadow on everything they publish?

      Delete
    8. I'm guessing you know about the lists of 'predatory publishers'. I think these are very important in keeping us up to date, and perhaps keeping some publishers honest.

      http://scholarlyoa.com/2014/01/02/list-of-predatory-publishers-2014/

      Delete
    9. I agree with you Kane, they play some good role to the public.

      Delete
  3. Agree with this [0]: lots of temptation, and therefore lots of need for safe-guards.

    Linking back to one of your previous posts on the temptations of editors in their own journals, Frontiers founder (and recipeint of €1billion (with a "B") EU grant) Henry Markram's google scholar is worth reading - quite a few in Frontiers [1]. Also interesting to see how/if things changed with 2013 Nature Publishing Group majority ownership [2]. They're having interesting editorial struggles with their fast-track system, prompting 150 editors to threaten resignation [3]. Editorial independence, fact-checking ethic, and peer-review time-tested across multiple media. Have you seen the the idea of peer-review by endorsement? [4]: Be great to see this at PeerJ, and integrated with scholarly societies.

    [0] https://twitter.com/timothycbates/status/604924375205335040

    [1] https://scholar.google.co.uk/scholar?q=h+markram+frontiers+in&hl=en

    [2] http://www.frontiersin.org/news/NPG_and_Frontiers_form_alliance_to_further_open_science/266

    [3] https://www.timeshighereducation.co.uk/news/nature-journal-playing-with-fire-over-fee-for-fast-track-review/2019810.article

    [4] http://blog.scienceopen.com/2015/04/welcome-jan-velterop-peer-review-by-endorsement/

    ReplyDelete
    Replies
    1. Thanks Tim .

      A subsequent commenter has queried whether Frontiers is a for-profit publisher: Would be good to establish what happens to all those millions of dollars, so if anyone has hard evidence, let me know.

      Peer-review-by-endorsement is, in my view, much less satisfactory than using reviewers prior to doing the research. That gives you the benefit of peer review input in designing the study, and also helps guard against P-hacking, HARKing and other questionable practices. See my previous blogpost.

      Delete
    2. What are your views on compulsory study registration? I think this is really the only way to prevent the above.

      Delete
    3. Hi Dorothy,

      Frontiers Media SA is a privately-held swiss corporation. [1]. They sold a controlling stake to Nature Publishing, a division of Macmillan, who are a subsidiary of Holtzbrinck, one of the largest media-holding companies.

      Anyone can use a ".org" url, it is an unrestricted domain.

      The goal for academia, IMHO, must be to return to society-based publishing, with expert partners like PeerJ contacted to handle function and innovation in production.

      Then we might not see sentences as bloviating and yet utterly empty as this: "Frontiers pursues its mission with conviction, using all resources at its disposal to improve and develop services for communities across academia."

      :-(

      [1] http://www.bloomberg.com/research/stocks/private/people.asp?privcapId=216839056

      Delete
  4. This is very disappointing. I co-authored an article in a Frontiers journal a few years ago and have reviewed a few papers for them over the last few years -- all of which went through the standard peer review process with experienced editors and reviewers who had a track record of (relevant) peer-reviewed research. I think the Frontiers system has a lot to offer: emphasis on scientific rigor over novelty, interactive peer review format, fast turn-around times. Unfortunately, this is exactly the kind of situation where a few rotten apples can spoil the whole bunch. Even if the majority of Frontiers papers go through a high quality editorial process, a critical mass of these bad ones could undermine the credibility of all Frontiers journals.

    ReplyDelete
    Replies
    1. Thanks Dan. See my reply below to Colin Phillips.
      Agree it would be good to retain Frontiers if the problems could be ironed out.

      Delete
  5. The examples you give are truly worrying. Having said this (and having published in Frontiers Journals), I feel I need to point out that this is by no means standard practice in Frontiers.
    How many articles does Frontiers publish each year, and how many are problematic? How does this ratio compare to other journals: How many articles that have been 'kindly' reviewed are published in established journals?
    Thinking about this, these examples actually speak in favour of Frontiers: As Frontiers names the reviewers, it is possible to check their 'academic competence' (as has happened in the example above), something that is impossible in any other journal I am aware of.

    ReplyDelete
    Replies
    1. I am not saying all papers in Frontiers are bad: on the contrary, I'm aware of excellent work published there. And of course, poor quality work gets into other journals. There have been such seismic changes in the publishing industry in recent years, that we are still working out feasible models. Of course, publishers are in the business to make money. Since money is associated with every published paper, there is bound to be pressure on the publisher to publish as much as possible. This is the business model of the so-called predatory publishers. However, another pressure on the publisher is to retain a good reputation so that people want to publish there and do not feel their work will be devalued by association with the journal. The point of my post is to point out that Frontiers seems perilously close to the point where they are putting profit before reputation and behaving like a predatory publisher. This is bad news for the authors who publish there, bad news for science, but also ultimately bad news for the publisher themselves.
      Any publisher who wants to stay in the business will need to demonstrate that they take scientific standards seriously, that their editors are competent and that a proper review process is adopted.

      Delete
  6. Frontiers published 54,000 articles since 2006? That's an awful lot. Isn't it rather unsurprising that there has been a number of problematic cases? I'd really like to see how well more traditional journals do before I conclude that the Frontiers approach is flawed.

    ReplyDelete
    Replies
    1. the concern is that the problematic cases aren't just the odd dodgy article, but reflect some systemic problems, ie lack of proper editorial oversight.

      Delete
  7. I have just had a commentary accepted at Frontiers and I discovered they have added a new layer to the acceptance process, presumably in response to recent events. The paper is in a 'final validation stage' at the Frontiers Editorial Office while they verify that it can be published. No information on what that verification involves but I'm not 'in press' till it's done.

    It seems that this extra layer is designed as an opportunity for the Chief Editor to catch problems without changing the policy that Associate Editors can accept papers. So perhaps some progress?

    ReplyDelete
    Replies
    1. Yes - this is a new and additional step in the quality control process.

      Delete
  8. "My first inkling of this was a colleague told me he would not review for Frontiers because his name was published with the article. This wasn't because he wanted confidentiality; rather he was concerned that it would appear he had given approval for the article, when in fact he had major reservations."

    Your colleague is exactly right. The blurb below is taken from http://www.frontiersin.org/about/reviewsystem. How about putting reviews up on PubPeer or, since anonymity is already compromised, on PubMedCommons? I understand the sentiment in publishing reviewers' names, but my personal choice would be to have the reviews published anonymously. The review is intrinsically useful. A name? Not so much.

    ----
    Transparent Review

    All Associate Editor and reviewers' names are made public upon the publication of articles, acknowledging their contribution. As a result reviewers are constructive, but also accountable and responsible for the paper and provide rigorous feedback that delivers the highest possible quality publication.

    To guarantee the most rigorous and objective reviews, the identities of reviewers remain anonymous during the review period. If for any reasons a reviewer withdraws during any stage of the review process, his/her name will not be disclosed. In case a manuscript is accepted that they have endorsed, their names appear on the published article, without exceptions.

    ReplyDelete
    Replies
    1. (Just to say for some reason this comment disappeared into spam folder, hence its late appearance!)

      Delete
    2. Thx! I had assumed I'd had an "all thumbs Sunday" experience! Relief.

      Delete
    3. The one time I reviewed for Frontiers (a few years ago), the paper I was assigned was terrible . The author refused to make any changes. The editor told me that there was no mechanism for rejection, and the paper would be published regardless of what I or any other reviewer said. I could have my name associated as a reviewer or I could remove myself from the review process. I did the latter and the paper was promptly published.

      Delete
    4. This part of the process may have changed. I was asked to make an editorial decision on a paper just a couple of hours ago, and "reject" was certainly among the options.

      Delete
  9. This comment has been removed by the author.

    ReplyDelete
  10. I agree with the comments about babies and bathwater. For the past couple of years I have been co-editing a research topic within Frontiers in Psychology. I was initially skeptical, but I’ve been impressed with many aspects of the process. It’s not perfect, it’s not a solution for all publishing, but it has a lot to recommend it.

    A Frontiers research topic is kind of like a traditional journal special issue, except that it’s not size-limited and it can extend over a period of time. By the time that we’re done, we’ll have published 30-40 papers. Our review process has brought in highly qualified reviewers, who have been admirably prompt. With the equivalent of the typical two rounds of review, our time from submission to on-line publishing has averaged 100 days, which is off-the-charts fast relative to traditional journals in our area. We have required authors to submit abstracts prior to submission, and that has helped us to weed out some non-viable submissions. Authors and reviewers have been initially skeptical of the non-traditional review model, but we’ve had a lot of positive feedback on both sides. As an editor, I’ve been terribly impressed with how the Frontiers backend staff makes things easier for me as an editor.

    Yes, it costs money. But all publishing costs somebody a lot of money. It’s just that normally we have the option of averting our eyes. And we shouldn’t overlook the huge cost of the inefficiency of the regular process — the biggest cost in the traditional model is the human time cost, which somebody is paying for. Of course, a Frontiers publication does not confer the cachet of publishing in one of the fancier traditional journals. Nor does it aim to do that. We know that some people are inclined to dismiss Frontiers publications as a consequence. But that’s like dismissing a student’s abilities because she didn’t attend a fancy university. The alternative, in both cases, is to judge the student or the research on their merits.

    I’m surprised at the references to profit motive. Isn’t Frontiers a non-profit publisher (and that’s why they have the .org address)?

    I wrote about initial experiences last year: A Grading Method from Hell (sorry, couldn’t get the external link to work).
    http://ling.umd.edu/~colin/wordpress/?p=2523

    ReplyDelete
    Replies
    1. Thanks Colin
      As in replies to previous commenters, I think the response is that it all comes down to editors. We can be confident that you will be a good editor and will select good reviewers and so make the most of what Frontiers has to offer. Which I agree in principle is a good model that overcomes many of the drawbacks of the traditional model.
      But let's suppose that instead of you, we have someone who does not do good work, is not conscientious, uses a special topic to just get a load of dross by self and cronies published with minimal reviewing. Should we care?
      One answer is that it doesn't matter: As Antonia Hamilton commented on Twitter, we can all read papers and make our own judgements.I'm sympathetic to that view, and agree that in the traditional model - as your blogpost so clearly describes - who gets published where is far too arbitrary. (I'll tweet the link : it's a well-made argument).
      But should we have a free-for-all? I don't think so. If you do, then you really have a problem. It is hard for those in other cultures to appreciate the extent to which psychoanalysts have held sway in treatment of autism in France (also Switzerland and Italy). They are challenged for not having any scientific credibility. Now they have an outlet where they can publish 'peer reviewed' 'scientific' papers. The other cases I mention seem to indicate this is the thin end of a large wedge, and that in general there is a lax approach to editorial matters at Frontiers.
      So I hope this debate will be noticed at Frontiers and will be taken as a wake-up call. They need to have some vetting of editors, and it would help if they also published peer reviews.



      Delete
    2. Thanks Dorothy. Again, my experience as an editor watching the review process has been generally quite positive, and better than I expected.
      The "twenty questions" approach to the reviews feels a bit constraining at first, but it seems to help a lot in keeping reviewers and authors focused. It makes it quite easy for authors, reviewers, and editors to see the points of disagreement, and hence easier to see how to respond.
      The fact that "impact" is explicitly not a review criterion removes a big challenge in the standard approach. And it makes it easier for authors to be candid about flaws or anomalies in their findings.
      Reviewers have told us that the speed of the process makes their job easier, because they don't have to waste time reminding themselves what the paper is about at each new submission. The back-and-forth of the interactive review stage seems a bit odd at first, but it seems to foster more constructive engagement between participants.
      The fact that reviewers' names are published with the article seems to help with the constructive tone of reviews also.
      I've found the Frontiers editorial management software to be extremely effective. It makes it super easy for me to know what to do when, and to identify what aspects of an article need attention. By making it much easier to be an editor (without local editorial support staff) it should hopefully make it easier to enlist qualified editors who have a lot on their plate. I've done other more traditional editing (not a whole lot), and it's much more time consuming.
      Would it help to publish the reviews, too? I'm not sure, based on the many reviews that I've now seen. The interactive review process that is useful for participants would also likely make the reviews less digestible for other readers. I wonder if some of the harsh-but-fair comments from reviewers would be less likely if the reviewers knew that their comments would be published. The fact that the system requires that all comments are satisfactorily addressed before endorsement would seem to take some of the place of publishing the reviews. But since I have no experience of the alternative model, I'm reluctant to pass judgment.
      The main annoyance has been the eagerness with which the Frontiers system tries to automate reviewer assignments if the editors are too slow to line up hand-picked reviewers. Their auto matching of candidates is a rather blunt instrument. At least it motivates me to contact reviewers quickly, to get ahead of the auto-spamming.

      Delete
  11. I'm one of the Frontiers editors who resigned over the recursiv fury debacle:

    http://bjoern.brembs.net/2014/04/resigning-from-frontiers/

    I also usually tend to find that your assessments are spot on.

    So it does feel somewhat weird to end up disagreeing with you and defnding Frontiers in this exceptional instance.

    You write, in the title, no less:
    "My collapse of confidence in Frontiers journals"

    ALL journals? Because of the incidents below?

    further down:
    "the scientific establishment were outraged that a paper on HIV denial had been published, and argued that it should be retracted. The journal editor decided that the paper should not be retracted"

    #arseniclife is not retracted either.

    "So it seems that Frontiers has opened the door to a branch of pseudoscience to set up its own little circle of editors, reviewers and authors, who can play at publishing peer-reviewed science."

    You mean like the editors at this Elsevier journal:
    http://www.journals.elsevier.com/homeopathy/
    Or did you refer to El Naschie's "Chaos, Fractals and Solitons" also at Elsevier?

    "once they lose a reputation for publishing decent science, they are doomed"

    I don't know if I should agree or disagree with you here: for one, this latest statement appears superficially correct. However, the kinds of incidences you mention don't seem to have any effect on publisher reputation at all: the publishers with the worst track records for sloppy scince, fraud and rtractions are also the ones with the highest repoutation. Even the publisher with the worst reputation, Elsevier, is doing just fine, thank you very much, with over a billion in annual profits and a margin of almost 40%.

    So here, at long last, my three points of criticism:

    1) If these incidents are shattering your confidence in all Frontiers journals, you must already have lost confidence in all journals from all other publishers long ago, with Frontiers being just the latest one.
    Which is a good thing, in a way :-)
    2) I cannot see any evidence that bad apples have ever had any lasting influence on the reputation of any publisher, ever. Scientists other than you don't seem to care all that much.
    3) Even publishers where there is a segment of scientists who openly declare issues with the reputation, do just fine - at least it spells far less than doom.

    Cheers,

    Bjoern

    P.S.: "editors of the journals Frontiers in Medicine and Frontiers in Cardiovascular Medicine mounted a protest at the way the publisher was bypassing their editorial oversight"

    I don't have any inside knowledge about Frontiers any more, but this incident reeked like a club of old boys upset they weren't in charge of their outdated club any more and had to make room for a more equitable way of publishing scientific results. No idea how accurate this impression is, though.

    ReplyDelete
  12. Thanks Bjoern
    You are right in thinking I don't have much confidence in any publishers! See previous blogpost.
    I didn't know about the Elsevier homeopathy stuff but I lost confidence in them very long ago.
    Just disappointed here in Frontiers because, as I intimated, the new OA journals had an uphill battle to fight initial prejudice when they started out, and I thought Frontiers was winning it.

    ReplyDelete
  13. This was predictable. Open access with authors' fees reverses the economics so that before a publisher had a cost to accept a paper now it has a cost in rejecting a paper. To expect that this would not create problems as long as editor/reviewers do their work was naive. The money pressure will find its way into the system and that is exactly what has happened. It is actually a triple whammy because, in addition to the lack of quality control, the inflation in number of publications is also a problem as it becomes impossible to read/browse this mountain of papers, and because the author's fee means poor authors/institutions have a disadvantage. The solution was also something that everybody could have seen in advance: Keep the publishing of journals in the hands of Academic Institutions and without author's fees. The fact that there is a cost in accepting a paper is a good thing, as it forces journals to be selective. Libraries should have led the way on this, they are the ones that benefit most from open access, they have the staff, and they are located next to the academics who do all the work. I fear it may all be too late, once a system develops with so many people milking it for profit it may be impossible to reverse the process.

    ReplyDelete
    Replies
    1. Nice economic analysis!

      Delete
    2. "The fact that there is a cost in accepting a paper is a good thing, as it forces journals to be selective."

      Even if there is a cost in accepting a paper, there are ways to recoup these costs after the paper's publication. Subscription journal publishers generally require copyright to be transfered to them so they have an incentive to have a not-so-strict editorial processes. Even if it costs them upfront to publish papers, they're left with plenty of intellectual property in their hands, which they can then license practically in perpetuity.

      Massive bundle subscriptions are another way of recouping these costs. Publishers can develop and grow a handful of really good and relevant journals and bundle them with many more mediocre journals, putting a high subscription cost to the entire bundle. Academic libraries are thus forced to pay up a lot more if they want access to the few quality titles that they really want and need.

      That's all assuming you believe subscription journals are better equipped to facilitate scientific communication.

      On the other hand, if you're suggesting a free-for-all (except for the publisher) open access venue would be a better solution, I'm afraid you're in for a long and ultimately fruitless wait. Publishing done right costs money, and someone's gotta pay up one way or another if you want it to be done right. Although one could argue that it's ultimately paid from the same pocket either way.

      Delete
    3. The point was in the context of how open access could be achieved without authors' fees.

      I also made the point that although possible it will be hard to get there. As you say publishers make money from copyright. Indeed the margins are astonishing, as high as 50% (van Noorden, 2013), and out of line of most 'normal' business activities. In other words even starting from the view that "publishing done right costs money", it does not cost anywhere near as much as we are paying at the moment. Because of this, publishers will fight it all the way against any change.

      Open access has much support, including from the RCUK and the Welcome Trust in Britain (they expect CC-BY for what they fund). The problem is the way they have gone about trying to achieve open access.

      The savings that libraries, Universities, and funding bodies will make from a switch to open access with no fees is big enough to invest some of that in the cost of publishing. Professional publishers have fewer of the skills relevant to run a scientific journal than libraries, charities, or academic societies.

      Delete
  14. I have seen things from several perspectives. I have been involved in writing papers in Frontiers. I have reviewed for the platform. I have read a number of colleagues' papers. I write this simply to show that I have no particular axe to grind with Frontiers per se.

    But I doubt I'll be contributing to the Journals in future. For some time I've had concerns about their aggressive 'why not create a research topic?' emails sent to myself and others on a pretty flimsy basis. And I've had comments from colleagues about the 'war of attrition' between authors and reviewers, with little or no editorial steer (reviewers eventually endorse a paper because they can't stomach further rounds of comments as much as because they really believe in the value of the paper).

    But mostly, Marco has pretty much nailed it. The shift in the revenue streams has not solved the role of commercial input, it has just put the onus on authors (or the blind funders of the research who apparently cough up the APCs without regards to value for money) and led such journals to potentially gobble up papers (more papers, more income). Are we trying to free ourselves for one set of shackles (subscription barriers) only to end up with another (APCs)?

    Another commentator suggested "it all costs money". Well, not exactly. Elsevier might say it all costs money but in response many people say their take is not proportionate. And there do exist models of OA journals with no charge. No subscriptions, no APCs, no copyright retention.
    For example, www.psychopen.eu. Using free, open-source workflow management systems, and using small central funding for the very cheap DOI assignment, along with copyediting and production and IT staff costs, the emphasis can be on scholarly communication and dissemination, not on commercial interest.

    Yes, for disclosure I do have a stake in one of PsychOpen's journals. But the focus here should be on the portfolio of journals and the underlying principle: is it possible and is it preferable to do without fees altogether? What might that look like? This is just one example of where the answer is maybe yes, and surely worth exploring. And as academics, we can choose (to some extent) what we wish to support

    ReplyDelete
  15. Isn't it a strange coincidence that the people who started Frontiers are the same as those at the head of the Human Brain Project? Scam artists, that have a good sense for finding the money where it is.

    ReplyDelete
  16. Dear Dr Bishop,

    First, I would like to express my deep appreciation for your devotion to science and scientific publishing – we need voices of concern and vigilance to keep both researchers and publishers aware of all that is at stake in insuring quality in scientific communication.

    Please allow me to address some of the points raised in your post.

    Background about Frontiers and the founding principles:
    Like you, as a researcher, I care about science (my latest article on autism was published in Frontiers in Neuroscience and a lay version is available at goo.gl/mrwMGU), as it is the foundation of our modern society.

    And like you I am also concerned about scholarly publishing. Back in the early 2000s, whenever the subject of publishing came up at conferences, people would lament on the state of affairs: bias in peer-review, lengthy peer-review cascades from one journal to another, lack of transparency, no access to journal content and, above all, no control for researchers over publishing. Out of these discussions emerged the basic idea behind Frontiers to use internet technology to give control of publishing directly to researchers, whose leaders would manage journals for the community. We felt that many benefits would be had by taking a fresh approach, including: restoring integrity in peer-review by bringing transparency to the process; focusing peer-review on the more objective scientific soundness of content rather than the far more subjective novelty or impact; building an IT platform to reinforce collaboration between authors and reviewers to obtain consensus; and providing quantitative impact metrics that measure use and reach of each article (rather than leave this as a judgment for reviewers and chief editors during the review process). As part of our efforts to support open science, we built a networking platform that not only makes authors, reviewers, associate and chief editors visible, but is also designed to increase readership for their articles. We are pursuing our vision by constantly improving our platform to bring the latest IT to publishing, to make peer-review even more efficient and collaborative in spirit, provide novel perspectives on performance that is quantitative and objective, and build bridges between communities and to the public. This is what we hope Frontiers can do. You can find more explanations about our publishing principles at goo.gl/mZXQbW and our history at goo.gl/Mmn8AY.

    In 7 short years, more than over 50,000 researchers from the world’s top universities (see infographic at goo.gl/MREYpM) across more than 400 academic specialties have joined the editorial boards of Frontiers. They decide how the journals are subdivided into sections, and they decide which articles should be accepted or rejected. In the current case, our team of editors in Frontiers in Psychology unambiguously recognize the need for a Specialty Section dedicated to Psychoanalysis. We have published over 33,000 papers in Frontiers, which have received more than 74M views and downloads. Yearly progress reports with all the numbers are available on the website (see goo.gl/IIgXgb for the 2014 report).

    Continued in the next comment

    ReplyDelete
  17. Part 2 of 3: Continuation from previous comment

    Open Access and the profit motive:
    You imply in your blog post that the series of complicated situations that you mention suggest a pattern that Frontiers is driven foremost by a profit motive. Let me first point out that the information you count on is heavily biased against the facts and that the situation is in fact far more nuanced than you suggest (please read the following: response to Climate article at goo.gl/whZ10r and response to the medical editors at goo.gl/u7MYPV). Even so, I think you might agree that these decisions are more related by a theme of editorial independence than profit motive. Taking a difficult and publicly controversial position is not typically made by an organization seeking a short-term gain.

    Frontiers pursues its mission with conviction, using all resources at its disposal to improve and develop services for communities across academia. Frontiers is a recognized leader in publishing technology, as recognized just last fall with the attribution of the Gold Award from ALPSP (see details at goo.gl/bXufld). Our managers are regularly invited to international conferences to talk about our vision and products. With our 200 employees in 4 different sites (of which well over 100 are IT developers), you can see where the income from article publishing charges is going — into the next generation of services.

    Frontiers chose the open-access model for its publication program because it is the only ethical option in this 21st century. It no longer makes sense to have selective or delayed access to the world’s research output. In fact, the service provided by an open-access publisher is priced in a perfectly transparent fashion and its full implementation would significantly reduce the global cost for publications to institutions universally.

    Yes, Frontiers was founded by a small group of neuroscientists with idealistic tendencies — myself included, who worked several years getting this program up and running without a salary. This is why we have no hesitation about re-investing all revenues back into the building Frontiers as we envisioned it. We want to make a difference and not a quick profit.

    Having said this, the financial sustainability of Frontiers is of course a responsibility we take very seriously. We are responsible to the community, to the more than 50,000 editors who have joined our editorial boards, to the over 100,000 authors who published with us, and to our employees. Frontiers is the fruit of these people’s efforts, and we need to ensure that their work will not have been in vain, that we will still be active 10 years down the road, that each of the research articles published lives on forever, that we have enough server capacity to store content indefinitely, and that we can disseminate these articles to all relevant repositories.

    Our program has been put into place with the premise that researchers shall be controlling the direction of science, not the publisher. For this reason, we introduced a strong separation between the business side and the editorial decision making process. As you have discovered, all decisions about article acceptances and rejections lie in the hands of the external editorial boards, all of whom are active researchers.

    Continued in the next comment

    ReplyDelete
  18. Part 3 of 3: Continuation from previous comment

    Quality standards
    Although researchers make decisions on acceptance and rejection in Frontiers, we have put many safeguards into place - at least 28 that are explicitly stated at goo.gl/rkT0FH. Many of these safeguards were integrated from the start, others were introduced – and continue to be developed – based on our experiences and on the continuous feedback from the community. Since the articles of concern mentioned in your post, we have added a pre-screening stage to flag suspicious or potentially contentious submissions to the attention of Associate and Chief Editors. And we have introduced a final validation stage to our process to allow all members of the editorial team to take a final look at the review dossier of each article after a provisional acceptance decision was made to add an additional stage of scrutiny.

    In this current case, it appears that our process failed to prevent a paper from being published that should have been better challenged in the peer-review stage – as a publisher, it is not in my remit to intervene directly, given the full independence of the external editorial boards (even if, as a researcher in autism and the step-parent of a child with autism, I can only agree with your observations). But we have raised this issue with the responsible Chief Editors of the Journal, who will now take the steps that they deem necessary. Since the beginning, we try to recruit only renowned researchers in each field and specialty to the editorial boards, because they are entrusted with operating the peer-review process. They certify acceptances with their name, which brings recognition of a job - usually very well - done, but also brings much-needed accountability and transparency to peer-review.

    Let me reassure you that the Frontiers process works extremely well for the vast majority of papers that are peer-reviewed. In fact, the paper in question is a good example of how our policy of transparency works. Readers with doubts about a published article, or about potential conflicts of interest, can scrutinize the qualifications of the handling editor and reviewers, as you did, and this will provide useful information for their assessment of the article. At other publishers, this information is unfortunately not available. We have a protocol for handling all complaints of this nature, and so if, in your opinion, this article requires some official qualification, then we would be pleased to accept your statement to this effect and begin the process.

    Sincerely,

    Kamila Markram, Co-founder & CEO Frontiers

    ReplyDelete
    Replies
    1. Jeroen Raaijmakers12 June 2015 at 01:39

      A lengthy (and self-serving) response that isn't really a response to the issues raised by Dorothy. I have always been a bit skeptical of the Frontiers philosophy to only judge a paper in terms of its technical merits without any consideration of its theoretical relevance. In such a view, a paper that reports e.g. a technically flawless experiment that looks at the height of a pile of blocks that two-year olds can build before it tumbles over, and tests whether there is a difference in the height between the morning and the afternoon, would be accepted for publication, even though it doesn't make any contact with existing theories in the area of (say) development of motor behavior and does not have any theoretical relevance. In my opinion, publishing such papers is basically littering the literature. This is not an outrageous example, I have had discussions with postdocs in my department that asked permission to publish a paper in Frontiers that had been rejected by no less than seven regular journals on the grounds that the paper did not have any relevance. I sometimes have the feeling that Frontiers' mission is just to make a researcher's CV more impressive. But I admit that I am a bit biased. In my view, the current system is already producing far too many junk articles, even in regular journals. What I would like to see is a open-access journal that has a strict "page" limit that forces the editor and reviewers to take scientific relevance into account.

      Delete
    2. Thanks, Kamila, for your considered reply. I greatly appreciate your taking the time to explain the background from the publisher's perspective.
      As you may be aware, I am a strong advocate of open science and I think that much of what Frontiers has done has been extremely positive.
      The additional checks you mention are a step in the right direction, but I do not think they go far enough.
      One question is how someone with no real publishing experience got to be an editor. Perhaps some minimum standard could be introduced - a minimum number of first- or last-authored publications in peer-reviewed journals and/or some evidence of acting as a reviewer for reputable journals other than Frontiers. The figures would be arbitrary but I'd suggest for someone to have the expertise to be an editor they should have at least 5 papers where they'd been corresponding author, and have reviewed at least 10 papers. That is, I suspect, a much lower bar than many journals would have. I wonder how many Frontiers editors would meet it.
      Second, how about my suggestion that reviews should be published, preferably non-anonymised? I realise this makes life more difficult for the editors, as reviewers may be reluctant to review under that circumstance. It would, however, allow anyone to scrutinise the quality of review as well as the paper itself, which would help give confidence in papers in the journal.

      Delete
    3. It is appreciated to hear a detailed and articulate reply from the publisher. But we are talking about fixes rather than considering the bigger picture here.

      The inflation in number of papers is linked to the business model of these publishers. I believe this is a problem. For instance, a recent study (papers.ssrn.com/sol3/papers.cfm?abstract_id=2575225) has analysed citation rates which normally increase and then decrease rapidly. This decay is becoming faster, signaling that nowadays papers are forgotten more quickly. When time is counted in terms of the number of published papers, the rate of decay is independent of the period considered, suggesting that the attention of scholars depends on the number of published items, and not on time.

      Given how much gets published and the limited quality control, and the fact that many of the safeguards rely on the work of the community, then platforms such as Frontiers are not fundamentally different from a glorified blog. We can publish our studies on a blog and get comments, citations, and quality checks directly from the community, saving millions.

      Another recent study has found that there is a trend to concentrate papers in the hands of a small number of publishers (journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0127502). Here is a sentence form their conclusions, which I share: "While one could argue that their role of typesetting, printing, and diffusion were central in the print world, the ease with which these function can be fulfilled—or are no longer necessary—in the electronic world makes one wonder: what do we need publishers for? What is it that they provide that is so essential to the scientific community that we collectively agree to devote an increasingly large proportion of our universities budgets to them?"

      Delete
  19. The way I see it, Frontiers is a private school. If you pay your fees, you will pass. The thing is, though, brilliant students would still pass without the fee (some fantastic papers published in Frontiers) and other not-so-bright students will still pass THANKS to the fee (terrible papers that get accepted because of the way this system is set up, with annoying reminders, predatory "special issue" emails, endorsement policies, etc). With this idea in mind, I try not to look down on papers published in Frontiers (even those I have published myself) but to keep an open mind and judge on a per-article basis, just like I would try to judge on a per-student basis if interviewing students from X private school for a job.

    ReplyDelete
  20. Ref:
    "The overall impression from the account in Retraction Watch was that the editor caved in to legal threats, thereby letting critics of climate change muzzle academic freedom of speech."

    sorry no... there was a clear case of conflict of interest, and the authors breaking the ethics approval for the 'research' paper..

    I was one of the peole that pointed out, is it normal for researchers/co-authors to be publicly attacking, the people they are researching, before, during and after the research period? After all this is psychology, not climate science.

    ReplyDelete
  21. http://www.frontiersin.org/blog/Rights_of_Human_Subjects_in_Scientific_Papers/830

    Prof Henry Markram (co-founder Frontiers)

    "....For Frontiers, publishing the identities of human subjects without consent cannot be justified in a scientific paper. Some have argued that the subjects and their statements were in the public domain and hence it was acceptable to identify them in a scientific paper, but accepting this will set a dangerous precedent. With so much information of each of us in the public domain, think of a situation where scientists use, for example, machine learning to cluster your public statements and attribute to you personality characteristics, and then name you on the cluster and publish it as a scientific fact in a reputable journal. While the subjects and their statements were public, they did not give their consent to a public psychological diagnosis in a scientific study. Science cannot be abused to specifically label and point out individuals in the public domain."

    ReplyDelete
    Replies
    1. Has anyone taken this to an ethics committee?
      As someone who has served on an ethics committee, I'd be surprised if it was deemed unacceptable to analyse material in the public domain. Human subjects protections are mostly about treatment of people who are recruited to a study and who provide information that would not normally be publicly available.

      Delete
    2. Analysing public domain material is absolutely fine..

      You seem to miss Masrkram's point..

      Identifying named individuals, based on this and assigning psychopathological labels, based on selected comments, is not fine..

      It is doubly not fine, when the researchers are not neutral ,inbderpendasnt observers. But opponents of the people named, and active and vocasl participants in the area they are researching..

      They also bvroke erthics approval of observing only. When one or more researchers were actually publicly challenging,goadng and interacting, with people.. About the issue they were researching, whilst performing the reseasrch.. Stirring the pot

      One of the authors writing a dozen blog posts during the research period, attacking, the people being researched,by name, probably wouldn't usually go down to well with a psychology ethics committee either..?

      Delete
  22. I'm not sure offhand just who all three editors who supposedly resigned in response to the Recursive Fury paper being retracted are. I do know who two of them are though, and I say good riddance to both of them. one of them, who has commented on this very page, labeled people who complained about the paper delusional with absolutely no basis. He links to his post which begins:

    "Last month, I was alerted to an outrageous act of a scientific journal caving in to pressure from delusionals demanding the science about their publicly displayed delusions be hidden from the world"

    As one of the people who filed a complaint about the paper, I'm of the opinion it's a good thing Björn Brembs resigned. He shouldn't be an editor for a journal if he feels comfortable publicly labeling people being studied delusional without the slightest shred of evidence. That's slander.

    Speaking of which, Björn Brembs pushes the same bogus notion this post seems to have fallen for - that the paper was retracted due to legal threats. That's bogus. The reality is the paper was retracted for ethical violations. The journal tried to downplay that at first, but when people like Brembs condemned it, it spoke more plainly.

    Besides which, there were no legal threats which could have caused Frontiers to retract the paper. Brembs claims there was an explicit threat of a defamation suit, but he bases that entirely upon this sentence:

    “You recently published an article by Lewandowsky et al, which, with malice, made a variety of defamatory and untrue allegations against me.”

    In no world is that an explicit threat of a defamation lawsuit. It's no more a threat than saying, "You committed murder" is threatening to arrest a person. But sadly, nobody reporting on this story seems to have bothered to actually read what the people who filed complaints about the paper actually said so details like that are conveniently overlooked.

    I don't know about the other examples in this post, but in the case of Recursive Fury, Frontiers misbehaved not by muzzling anyone, but by underreacting. They never should have accepted the paper given its ethical lapses, and after they received complaints, they were far more lenient and forgiving on Lewandowsky et al than they should have been.

    ReplyDelete