When science meets freakonomics

I subscribe to a couple of podcasts, and usually use my long runs to catch up with some longer episodes. As I am a slow runner, yesterday I was able to listen to two new episodes of different podcasts, and listening to them in sequence made me think about the connection between them.

The first podcast was Science Disrupt. This particular episode I’ve listened was about building the science startups of tomorrow (very interesting one). But a peculiar thing about this podcast is that after every interview, the hosts always ask the same question to all guests: “Thinking about the science ecosystem as a whole, what else do you think in science still needs to be disrupted?” – and they always get very interesting answers! The other podcast was the second episode of the Earth 2.0 series from Freakonomics Radio. This series is supposed to answer the question “What if we could remake earth? What would you change?

As I was listening to this Freakonomics episode, my mind just wandered around replacing the word “economy” by “science“, and I was amazed how the discussion still seemed pretty pertinent (in my mind). They asked A LOT of questions along the podcast, but I’m just going to go over a few of them. I’m going to try to describe the parallel here, and hopefully it will make sense on paper as much as it did while I was running…

  • “If we had the chance to totally reboot our economic system, what would that new system look like?. That seems like an unanswerable question – both economic and science-wise – and why we bother to discuss this?

Abhijit BANERJEE: I think that doing things consciously with thought, asking lots of questions; not doing them because this is how we’ve always done things, this is our tradition, this is the normal in the world … Asking questions: “Why do we do these things?” “Is this the right thing to do?” “What is the actual evidence for it?” That’s key. We won’t have a blueprint for the world. But we will have a better way of building a better world.

Applying this to science, it is easy to say that a lot of things need to change in our science system, regarding the way scientists are hired, funded, evaluated. Along those lines, our publishing and peer-review system also needs some evaluation. However, can you propose a real and feasible strategy that will make our scientific system work better? Paraphrasing Paul Ryan re: health care reform, “it’s a lot easier to oppose something than to be for something.”

Still, scientists should still reflect about the system and not simply do things automatically only because it’s how things are done. Are you happy with the current system? No, so what can you do to change it? If you don’t know where to start, the article How Scientists Can Influence Policy has some interesting suggestions. Figure out what is already happening. Expand your readings. Write letters, emails. Pick up the phone!

  • “What should be done about income inequality?”

KANTER: To make things work well, inequality doesn’t help. If you have a lot of people who feel left out of the system, well, they do get angry, and they sometimes surprise you with their feelings. But also, they often go passive. They think nothing could be done to change anything. And because of that, they’re not very motivated, and nothing does change.

Yes. Inequality doesn’t help in science either. Every time that I see a list of grants /prizes awarded, there’s a pattern there. You always see a lot of money going towards the big institutions and less or even no money awarded to small Universities. You can say that money comes to big institutions because they do better research, but how can small institutions do better research if they are underfunded?

I believe some researchers may become angry when thinking about the current funding climate, but most of them fall into the later category: go passive. As a salesperson that sells expensive laboratory equipment, I talk to many professors on a daily basis. Many times, when I suggest that they could apply for a equipment grant, or group with other researchers to get a multi-user equipment, they just sound so… unmotivated. Almost like “why bother to write a grant for that, if I know that it will be rejected?”.

I know government funding for research has never been that low, but there are alternatives. Ryan Bethencourt (Program Director and Venture Partner at IndieBio) suggested on the ScienceDisrupt podcast that researchers should start looking at partnerships with the private sector – partner with biotech companies to raise additional money and bring your research forward.

  • “Why North America has been more successful economically than South America?”

Tyler COWEN: Whereas, say, the Spanish colonies were more likely based on the idea of extracting wealth from other people, or taking a lot of the resources out of the ground and not investing as much in human capital.

Why some research groups are more successful than others? Is because of the institution that they are located? Or it’s because they research a topic that’s more appealing to the grant funding agencies? Also, this question made me think about all the team supporting a research group. Graduate students, postdocs, technicians. When a PI is training a GS or a PD, does it make a difference if they are just interested in their skilled labor, or if they are really training them to be better scientists? I truly believe so.

As much as a lab PI is supposed to be the leader, the mentor of its group, a successful lab still needs to be a group. And it’s the PI’s job to make this group cohesive and motivated to science! It is important to take your time and teach your group what takes to be a successful scientist. Also, keep in mind that not all GS (or even PD) will necessarily go towards an academic position. Knowing your group, understanding the goals and limitations of each one of its members can really make a difference!

  • “If you want to think about building the perfect economic system, there are so so so many elements to consider. Money, for instance. What is the optimal form of money?”

SURI: The biggest two findings are that mobile money improves financial resilience, which is the ability to deal with bad events. Basically, we find that it has an effect on poverty; it’s going to reduce poverty. (…) You’re saving more because you have the ability to save in your phone, and so people are able to do these things.

What is the currency of academia? Publications and grants. Not only number of publications, but also the impact factor of the journals on which they are published. Publications are part of the measurements of a researcher’s success, determining if they are going to get hired, funded, and be successful in their careers. The podcast suggested that the use of mobile money could decrease poverty, and I wonder if academia could somewhat change its currency to be more fair to all researchers.

  • Is the World Ready for a Guaranteed Basic Income?

CAPLAN: Saying, “Let’s give everybody free money no matter what. People perfectly able-bodied. People perfectly able to take care of themselves.” That seems crazy at the outset. But more importantly, if you do a small amount of math and realize how much would it cost, the cost is enormous. Right now, the welfare state — we’re able to keep the cost down because we don’t give money to everybody.

So, with all bias and problems regarding grant funding, what about if we just gave some funding to all researchers to do their research? I certainly don’t believe this would be a good idea, as there are so many researchers from different fields, all requiring different amounts of money to perform their research. Along those lines, a new grant funding system was proposed by Johan Bollen at Indiana University where scientists would just give each other money: “Self-organized fund allocation” (SOFA). Unfortunately, I don’t believe this would work either, as the bias (and pressure) towards funding your own friends would be enormous, and also making things even harder for junior researchers.

I don’t have an answer to most of these questions. But it was nice to reflect on those issues, economy and science-wise. One of the conclusions of the Freakonomics podcast was that it’s hard to build a good economic infrastructure without a good political infrastructure. I do truly believe we have a lot to learn from economics!

Advertisements

Breaking up is hard to do

I have been busy lately. Busy times, life-wise and mind-wise. This past weekend, I would have had enough things to do to keep me entertained the whole time, but despite that I decided to go into a short trip. One big meeting was being held nearby and I knew my former PI and my former collaborator were attending to it – I decided it was a good opportunity to catch up. When I was in academia, my collaborator and I used to have weekly lab meetings through Skype. Before (and after) that we were always good friends, but after I left academia our conversations became scarce, and I miss her a lot.

Initially I was not attending to the meeting. I was only going to be there for one whole day, and the registration was far too expensive (I checked). But then the miracle of the multiplication of the badges happened and before I could notice there I was, in the conference. I attended to a couple of talks, what was very interesting at first, but brought me a certain feeling of nostalgia. In the end of the day, even though new data could bring me new ideas, I am not doing in academia anymore and those new ideas could never be put into practice.

Then I went to the poster session. There were only a couple of rows with posters about my previous research, so it was easy to go through all of them. I stopped by a poster that was being presented by a student. I listened to presentation, made comments, asked questions. By the end, the student tried to read my badge (that was strategically hidden) and asked where I was, what my research was about. I froze from a second, told her that I was not in academia anymore, but used to work with that topic. Mentioned the last paper I published as a first author and she immediately recognized it. Bittersweet feeling again, as she suggested a possible follow up to the paper.

While wandering around the poster session, I saw a lot of the researchers I knew there. They were all busy talking to poster presenters, and normally I’d just stay around, until I’d talk to them. Some of them saw me and waved. Some of them I didn’t really wanted them to see me. I started to feel extremely uncomfortable. I couldn’t really understand what was going on at that moment, but I just wanted to get the hell out of there. Someone mentioned an interesting talk was going to happen after the poster session. The topic was delightful, and the speaker was an old friend of mine. But at some point I simply realized that this was not my life anymore, I had no reason for being there. That was part of my past, a past I left behind and that I don’t want to come back. So I just picked up my stuff and walked back to the hotel.

You know how they say that after you break up with someone, you need to meet that person again to see if you’re fully over it? That’s how it felt like. Two years ago I broke up with academia, ending a relationship of almost 20 years. It was tough in the beginning, but after a while you don’t think that much about your ex, and you end up forgetting your feelings about it. This past weekend I met my lover again. I realized that, although I still have feelings for it, breaking up was the right thing to do. But as every long term relationship breakup, it still hurts when you meet.

I feel stronger now. I feel I gave one more step leaving the past where it belongs and looking forward my future. Will we meet again? Certainly. But I know next time it will be different. A lot less painful. And easier.

PS> the lover break up analogy was probably used by several people before. But the first that comes to my mind is this post from Lenny Teytelman “Dear Academia, I loved you, but I’m leaving you. This relationship is hurting me.” It is worth a reading!

My first #AAASmtg: on Reproducibility and Open Access

At first I was planning to write two separate posts on each one of those issues. Reproducibility and Open Access were extensively discussed during the #AAASmtg, and even if I wanted it would be impossible to share all with you. Also, they were discussed in totally separated sessions, and they apparently are two totally separate issues, I can’t help to see them as two big problems mainly caused by our system.

Yes, reproducibility is a real problem. According to a Nature’s survey, more than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments.There has been a scary big number of published articles being retracted due to reproducibility problems. Not only experiments can give us different results when repeated, but sometimes the same data can be interpreted differently among researchers. The attempt to replicate key cancer studies with the “Reproducibility Project: Cancer Biology” is bringing more questions than answers.

Yes, science should be open and accessible to all. Most scientific research is funded by federal grants, which is supported largely by our taxes. It is logically expected to be open to the public that is actually paying for it. Open Access brings more transparency to research and can also stimulate curiosity and interest – bringing more attention to science! Openness not only regarding published articles, but also Open Data – where other researchers can access, analyze, and collaborate.

But both reproducibility and open access fall under the problem of how our system works.

Scientists are mainly judged by the number of publications, and the impact factor of the journals where they publish. One of the main requirements for publication of a scientific article is NOVELTY. Neither replication nor negative studies are encouraged to be published. Even though I totally agree when Jessica Polka said at the #AAASmtg that “We need a culture where people read papers and not the name of the journal”, unfortunately both funding and hiring committees still care about journal titles and metrics when judging scientist’s achievements.

The “publish or perish” culture not only incentive publishing unreliable data, but also decreases the quality of science. Tight funding and the increasing competition may encourage falsehood and misconduct in academia. Probably not deliberately, but who has time (and money) to repeat that experiment with those two outliers? There is a lot of cherry-picking and p-hacking that can be easily performed by the analyzer. If you don’t believe me, you should try Nate Silver’s example of how to Hack Your Way To Scientific Glory.

Landing an academic job, getting funding, and publishing science is tough business! As professor Sydney Brenner points out in this interview:

Even God wouldn’t get a grant today because somebody on the committee would say, oh those were very interesting experiments (creating the universe), but they’ve never been repeated. And then someone else would say, yes and he did it a long time ago, what’s he done recently?  And a third would say, to top it all, he published it all in an un-refereed journal (The Bible).

 

Who’s to blame? Open Access defenders blame publishers. Publishers blame academics. Academics blame funding agencies. Potential solutions? Besides a cultural change in the system, where science would be more important than metrics, reproducibility and negative results should be supported and encouraged. Some publishers are contributing to this, as Nature published a manifesto for reproducible science and Elsevier creating developing a new article type especially for replication studies.  A new kind of paper combining the flexibility of basic research with the rigour of clinical trials was recently proposed by some researchers. Data sharing and the recent incentive to use preprints in biological science can also help with this, as data and research drafts are open to other researchers feedback, and possible problems with replications.

The scientific world is paying more attention to quality of research, replication, data analysis. Change seems to be coming. Hopefully funders, publishers, and hiring committees will change along as well.

PS: This is my fourth post within the series of “My first #AAASmtg”. Previous posts include expectationspersonal impressions, and public engagement.

On Elsevier and #OpenAccess

By the end of 2016 Nature.com published this article stating that negotiations between Elsevier and  Universities in Germany, Taiwan, and Peru didn’t reach an agreement and those countries were suspending their subscriptions to Elsevier journals. This is sad, because although some may still have some access to the journals illegally through SciHub, it is very likely that this will have a negative Science impact in these countries.

In an ideal world all science would be free, and everybody would have OA to every research published. But our world is far from ideal, and, although we would like this to happen eventually, how likely is this to happen?

Ok. So what’s the data? We live in a world where everything costs money. When I think about publishing, I immediately think about paper, ink, and printing actual magazines. When I started doing research as an undergrad I remember going to the library to look for articles and take photocopies of them to read later (yes, I’m old like that). But besides for old articles, what’s the point to print science papers anymore, if they can be easily accessed through PDF and saved (or printed)? But even if you don’t print Science magazines, there’s still the cost of personnel. You need to pay editors and people to review those papers and choose which ones are going to be published, right? Wait, no, professors do it… for free! So what are exactly the publishing costs? I tried to look for this information on the internet, but wasn’t very successful…

Elsevier has been the most hated publisher for several years. But there are others big publishers out there. Actually this article from 2015 states that Reed-Elsevier, Springer, Wiley-Blackwell, Taylor & Francis, and Sage published more than half of all academic papers in the peer-reviewed literature in 2013. Elsevier doesn’t disclose the price of their publications, but do other publishers do? Not really. Also, it seems that neither PLoS nor BioMed Central also discuss actual costs of publication. A few years ago, the true cost of science publishing was discussed in this article:

cost_of_publishing2

The subscription prices are also not disclosed by the publishers. Libraries are also not allowed to release their costs. It’s tough to judge if prices are fair or not if you don’t know the numbers! I mean, how do you set the price for a journal subscription to begin with? Number of students and professors, kind like the electoral vote? The larger the number of people in a particular University, the higher the price? But what about number of journals? I mean, in a particular university there might be a lot of people, but they might access more other publisher’s journals. Maybe they have an average of downloads and then set the price?

So as the majority of research around the world is still published in journals that require subscriptions, one would think those publishers are the most profitable. Not exactly, at least according to this blog post from 2013, proclaiming that the OA Hindawi Publishing Corporation “has a impressive profit margin of 52%. Much better than Elsevier (36% profit margin on revenue in 2010)”. Of course, this is a single article and things may have changed since then.

One thing we do know is the publication fees for scientific journals. And we all know that publication charges are higher in OA journals. Regarding scientific access to everybody, one cannot argue with the premise of OA journals. But are we just going from a “pay to read” to a “pay to publish” model? That doesn’t seem to help researchers and institutions, specially on times of tight research budget.

The internet turned the world into a much accessible place to everybody. People said newspapers were going to die, but even with fewer and fewer people actually buying printed newspapers, they are still standing. With facilities and paid personnel. Newspapers evolved to stay alive. Academic publishing need to evolve as well. Can we reach a middle point where we can have OA to all, at reasonable prices for researchers?