My first #AAASmtg: on Reproducibility and Open Access

At first I was planning to write two separate posts on each one of those issues. Reproducibility and Open Access were extensively discussed during the #AAASmtg, and even if I wanted it would be impossible to share all with you. Also, they were discussed in totally separated sessions, and they apparently are two totally separate issues, I can’t help to see them as two big problems mainly caused by our system.

Yes, reproducibility is a real problem. According to a Nature’s survey, more than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments.There has been a scary big number of published articles being retracted due to reproducibility problems. Not only experiments can give us different results when repeated, but sometimes the same data can be interpreted differently among researchers. The attempt to replicate key cancer studies with the “Reproducibility Project: Cancer Biology” is bringing more questions than answers.

Yes, science should be open and accessible to all. Most scientific research is funded by federal grants, which is supported largely by our taxes. It is logically expected to be open to the public that is actually paying for it. Open Access brings more transparency to research and can also stimulate curiosity and interest – bringing more attention to science! Openness not only regarding published articles, but also Open Data – where other researchers can access, analyze, and collaborate.

But both reproducibility and open access fall under the problem of how our system works.

Scientists are mainly judged by the number of publications, and the impact factor of the journals where they publish. One of the main requirements for publication of a scientific article is NOVELTY. Neither replication nor negative studies are encouraged to be published. Even though I totally agree when Jessica Polka said at the #AAASmtg that “We need a culture where people read papers and not the name of the journal”, unfortunately both funding and hiring committees still care about journal titles and metrics when judging scientist’s achievements.

The “publish or perish” culture not only incentive publishing unreliable data, but also decreases the quality of science. Tight funding and the increasing competition may encourage falsehood and misconduct in academia. Probably not deliberately, but who has time (and money) to repeat that experiment with those two outliers? There is a lot of cherry-picking and p-hacking that can be easily performed by the analyzer. If you don’t believe me, you should try Nate Silver’s example of how to Hack Your Way To Scientific Glory.

Landing an academic job, getting funding, and publishing science is tough business! As professor Sydney Brenner points out in this interview:

Even God wouldn’t get a grant today because somebody on the committee would say, oh those were very interesting experiments (creating the universe), but they’ve never been repeated. And then someone else would say, yes and he did it a long time ago, what’s he done recently?  And a third would say, to top it all, he published it all in an un-refereed journal (The Bible).

 

Who’s to blame? Open Access defenders blame publishers. Publishers blame academics. Academics blame funding agencies. Potential solutions? Besides a cultural change in the system, where science would be more important than metrics, reproducibility and negative results should be supported and encouraged. Some publishers are contributing to this, as Nature published a manifesto for reproducible science and Elsevier creating developing a new article type especially for replication studies.  A new kind of paper combining the flexibility of basic research with the rigour of clinical trials was recently proposed by some researchers. Data sharing and the recent incentive to use preprints in biological science can also help with this, as data and research drafts are open to other researchers feedback, and possible problems with replications.

The scientific world is paying more attention to quality of research, replication, data analysis. Change seems to be coming. Hopefully funders, publishers, and hiring committees will change along as well.

PS: This is my fourth post within the series of “My first #AAASmtg”. Previous posts include expectationspersonal impressions, and public engagement.

On Elsevier and #OpenAccess

By the end of 2016 Nature.com published this article stating that negotiations between Elsevier and  Universities in Germany, Taiwan, and Peru didn’t reach an agreement and those countries were suspending their subscriptions to Elsevier journals. This is sad, because although some may still have some access to the journals illegally through SciHub, it is very likely that this will have a negative Science impact in these countries.

In an ideal world all science would be free, and everybody would have OA to every research published. But our world is far from ideal, and, although we would like this to happen eventually, how likely is this to happen?

Ok. So what’s the data? We live in a world where everything costs money. When I think about publishing, I immediately think about paper, ink, and printing actual magazines. When I started doing research as an undergrad I remember going to the library to look for articles and take photocopies of them to read later (yes, I’m old like that). But besides for old articles, what’s the point to print science papers anymore, if they can be easily accessed through PDF and saved (or printed)? But even if you don’t print Science magazines, there’s still the cost of personnel. You need to pay editors and people to review those papers and choose which ones are going to be published, right? Wait, no, professors do it… for free! So what are exactly the publishing costs? I tried to look for this information on the internet, but wasn’t very successful…

Elsevier has been the most hated publisher for several years. But there are others big publishers out there. Actually this article from 2015 states that Reed-Elsevier, Springer, Wiley-Blackwell, Taylor & Francis, and Sage published more than half of all academic papers in the peer-reviewed literature in 2013. Elsevier doesn’t disclose the price of their publications, but do other publishers do? Not really. Also, it seems that neither PLoS nor BioMed Central also discuss actual costs of publication. A few years ago, the true cost of science publishing was discussed in this article:

cost_of_publishing2

The subscription prices are also not disclosed by the publishers. Libraries are also not allowed to release their costs. It’s tough to judge if prices are fair or not if you don’t know the numbers! I mean, how do you set the price for a journal subscription to begin with? Number of students and professors, kind like the electoral vote? The larger the number of people in a particular University, the higher the price? But what about number of journals? I mean, in a particular university there might be a lot of people, but they might access more other publisher’s journals. Maybe they have an average of downloads and then set the price?

So as the majority of research around the world is still published in journals that require subscriptions, one would think those publishers are the most profitable. Not exactly, at least according to this blog post from 2013, proclaiming that the OA Hindawi Publishing Corporation “has a impressive profit margin of 52%. Much better than Elsevier (36% profit margin on revenue in 2010)”. Of course, this is a single article and things may have changed since then.

One thing we do know is the publication fees for scientific journals. And we all know that publication charges are higher in OA journals. Regarding scientific access to everybody, one cannot argue with the premise of OA journals. But are we just going from a “pay to read” to a “pay to publish” model? That doesn’t seem to help researchers and institutions, specially on times of tight research budget.

The internet turned the world into a much accessible place to everybody. People said newspapers were going to die, but even with fewer and fewer people actually buying printed newspapers, they are still standing. With facilities and paid personnel. Newspapers evolved to stay alive. Academic publishing need to evolve as well. Can we reach a middle point where we can have OA to all, at reasonable prices for researchers?