“Is the Scientific Paper a Fraud?”

“First ethical rule: If you see fraud and do not say fraud, you are a fraud.”

~ Nassim Taleb, Antifragile.

I’m in the midst of writing up a long overdue research paper. As you know, the standards and format required of a research study follow a convention that comprises the following sequence: Literature review, Methods, Results, and a Discussion Section. Actually, it useful to have a format. It makes writing—and reading the work of others—much easier.

However, the convention of a scientific paper is misleading. As I try to give the flesh and bones of the study based on the conventions of a scientific paper, It gives a false impression that one starts the process by scouring past research, built a hypothesis around a gap they think can be filled, set up the experiment, lay down the results, and discuss its implications. Turns out, more often than not, the truth is much messier than this. The process of discovery and insight is typically more non-linear than we are made to think when we read a peer-reviewed article. The scientific paper is more of an “after the fact” discourse, or even a marketing of convictions, as opposed to a communication of the scientific process.

In 1964, Brazilian-born British biologist Sir Peter Medawar (28 February 1915 – 2 October 1987) gave a talk addressing this very issue. Medawar described the underlying style of scientific writing gives the impression that scientific discovery is an inductive process. “It start with simple observation—simple unbiased unprejudiced, naive or innocent observation—and out of this sensory evidence, embodied in the form of simple propositions or declarations of fact, generalizations will grow up and take shape, almost as if some process of crystallization or condensation were taking place… this is a mere philosophic fiction. There is no such thing as unprejudiced observation. Every act of observation we make is biased. What we see or otherwise sense is a function of what we have or sensed in the past.”

A case in point. Susan Hewitt and Anna Wilson points out that the discovery of the double helical structure of DNA by James Watson and Francis Crick was nothing like how it was stated in the research paper. Their study, published in prestigious journal Nature, is famous for its elegance and brevity, citing that they use models to generate the double helix structure accommodating complementary base pairs. It usually also mentions in brief about the X-ray data of Rosalind Franklin and Maurice Wilkins.

Turn outs, the actual process of discovery of the double helix structure was anything but straightforward. In James Watson’s book, The Double Helix, makes clear at the preface that his reason for writing this book was because he was concerned of a false impression of their scientific discovery. He says, “There remains general ignorance about how science is ‘done’.” Not reported in their journal article were two crucial mistakes during the scientific process. First, he misunderstood the X-ray data, with the bases on the outside, which he described as humiliating. Second, another colleague mistakenly published a triple helix structure with the bases on the outcomes, neglecting some basic chemistry. In both instances, formal and informal feedback from the scientific community helped correct their errors. Hewitt and Wilson said, “This presents a completely different view than that presented in either a standard text book or Watson and Crick’s Nature paper. It shows science as an activity whereby ideas are tried out, discussed, and modified and where mistakes are all too common…”

“Science is a social activity (emphasis mine) and we do students a disservice if we maintain that it is only about facts and objective truths… (we) need to learn that mistakes or false starts are not time wasted, but are an essential part of making progress.”

~ Susan Hewitt and Anna Wilson.

Medawar puts it succinctly, “The scientific paper is a fraud in the sense that it does give a totally misleading narrative of the processes of thought that go into the making of scientific discoveries.”
As reality would have it, the process of science is typically more deductive than inductive. (Click here for a usual visual to understanding the difference between deductive and inductive reasoning).

If we have a chance to re-imagine how a scientific paper would be, how would it look like?

I have 8 ideas of how we can re-envision the scientific paper[1]:

1. Thought Process: For the introduction to include the thought process that led to the interest and development of the ideas that the researchers set out to test, followed by the usual literature overview.
2. Publishing Hypotheses Beforehand: For research hunches to be published and made available to the public, prior to the study being carried out. (e.g., See Center for Open Science) . This way, all positive and null findings are published and not cherry-picked (Then we wouldnt need someone like Irving Kirsch to wave his freedom of information rights and demand to reanalyze all the published and unpublished results of anti-depressants, which was a kick in the gut to Big Pharma[2])
3. Open Access: When the research is completed, it should be common practice—not an exception—for the raw data to be made available. (See What is Open Data?)[3]
4. No Paywall: When the paper is published, this should not be hidden behind a paywall that is only accessible to academics or people who are willing to pay $30 or more to access a single journal article (Do should people exist?). This should be made accessible to all [4]
5. Visuals: The use of intelligent and thoughtful visuals is not a dump-down. It’s good communication of key and complex messages that researchers need to exercise using more of.
6. Humanise: Heck, what’s stopping us from having a URL link in the article to a video of the researchers explaining their process, thinking, blunders and reflection of their study?
7. A New (and Real) Discussion Section: For a centralized location for a moderated intellectual dialogue from all interested readers and the scientific committee to enter into a discussion about the study. In fact, this is probably the right place to call it a “Discussion Section.” I would imagine such discussions to spark further thinking, replication studies or even reanalysis to take place, at a momentum that can generate engagement, and hopefully, better ideas.
8. Clarity: Finally, can we aim to write in simple English? Clear communication is a sign of wit, not a lack of. Using complex language is likely more of a sign of not taking the extra effort to think clearer.[5]

Can we demand the process of science to be more of a social activity? Can we make this happen?

Notes:
[1] Some of these ideas aren’t new, like points #2, #3, and #4.
[2] Read Irvin Kirsch’s brilliant book, The Emperor’s New Drugs
[3] My colleagues and I were able to re-analyze a meta-analysis study regarding the influence of deliberate practice on performance. We were able to do so because the authors of the original meta-analysis made their data available.
[4] Watch this heart-wrenching biographical documentary of the activist Aaron Schwartz, The Internet’s Own Boy .
[5] I’m currently listening to a wonderful NPR podcast with my daughter called But Why? The host interviews leading experts around the world about all sorts of topics. It’s so impressive when these experts figure out a way to communicate their ideas thoughtfully and clearly, in a way that even a 6-year-old child—and a naïve adult—can understand. That’s a skill.)