Your favorite podcast mentions a new study about the health risks of seed oils. A viral social media post says that new research reveals the reason for rising cervical cancer rates. How can you be sure what they’re saying is accurate? One way is to look at the study yourself.
Scientific publications—also called research articles, journal articles, or scientific papers—are written for an audience of fellow researchers, not a general audience. Because of this, when they are cited in media intended for the broader public—podcasts, YouTube videos, and news articles, for example—their findings can be misinterpreted, often unintentionally, or dramatized for clicks. Some findings are based on narrow studies that were not designed to be applied to a broader group. Study findings may also be cherry-picked or misrepresented to perpetuate a narrative—or even to stoke fear.
Emily Gurley, PhD ’12, MPH, distinguished professor of the practice in Epidemiology, has reviewed tens of thousands of papers throughout her career, including for the Novel Coronavirus Research Compendium during the height of the pandemic. While being able to read and fully understand a study is a skill researchers learn over years of practice, Gurley explains that it can be useful for the average reader to know what makes a good study, what a single study can and can’t conclude, how to understand the findings, and how to spot red flags.
How to access a study
To look more deeply at a finding or statistic, you first need to identify the study it comes from:
- News articles will often include a hyperlink to the study or to a press release announcing its publication.
- Check a podcast episode’s show notes for a list of studies referenced. If they don’t include the one you’re looking for, use the podcast’s contact information to ask for the study title or link.
- On social media posts and videos, look for a list of links in the caption, or look to see if the source material is referenced visually in the content. You can also comment on the post to ask for the study name and link.
Once you know what study you want to read, accessing the full text can be a hurdle in and of itself. Many journals do not make papers publicly available; nonsubscribers can only see the authors and their affiliations and the abstract—a summary of the study—and not much more.
University students and employees can often access these journals through their school’s library. For others, PubMed is a good place to start, as it offers both open access papers and paywalled studies. If a study on PubMed only shows the abstract, look in the top right for links to access the full text elsewhere.
If you’re unable to access the full text of a study, try looking for an official press release from the authors’ affiliated institutions—these will often cover the major findings and data points. In the future, more studies will be accessible for free under a new federal requirement that NIH-funded authors share their papers on PubMed Central.
What to read and in what order
Reading a scientific paper for the first time can feel intimidating: They’re long and may use terminology unfamiliar to general readers. But because most papers follow the same format, understanding the purpose of specific sections makes it easier to find the information you’re looking for.
Gurley recommends approaching a paper using the following guide:
1. Skim the Abstract.
This section summarizes the study, though exactly what the authors choose to highlight here can vary widely. Gurley says many Abstracts are too high-level or technical for nonscientists to glean meaningful information from.
2. Read the Introduction.
This section puts the study into context. This is where the researchers share what’s already known about the topic and other relevant research into it—and the reasons the researchers wanted to pursue the topic.
In particular, “the last couple sentences of the introduction will tell you the objective of the study,” Gurley notes. “A paper title may give you the general topic, but you need to know the exact question the investigators were trying to answer.”
3. Review the Results and Discussion sections.
The Results section details the study’s findings and presents the data in informative tables and figures. To a new reader, the numbers and statistics in the results section may feel overwhelming, so reading through the Discussion section will be especially helpful.
“The first and last paragraphs of the Discussion section will generally provide the highlights,” Gurley says. The rest of the Discussion will explain the data points listed in the Results section—how they compare and relate to one another, where they fit into the context of other relevant research, and what they may suggest. Additional explanation of findings may also be presented in the Conclusion or Analysis sections.
4. Understand the limitations of the study.
This information is generally included in the next-to-last paragraphs of the Discussion section, though some studies may present it in a dedicated section. “This is where the researchers will explain any caveats and how someone should interpret the findings based on limitations,” Gurley says. These limitations can include things like study design and duration, the methods used to collect and analyze data, the sample size or participants’ demographics, and more. They often note what further research should be undertaken.
Gurley recommends the average reader skip the Methods section, unless you’re looking for a specific piece of information, like who was enrolled or the exact drug that was involved. This section tends to be very technical and is intended more for fellow scientists and researchers who may want to replicate the study.
Distinguishing correlation from causation
Two things can be related without one directly causing the other. A common example is that ice cream sales and drownings both increase in the summer, but ice cream sales don’t cause drownings—the two are correlated. On the other hand, smoking cigarettes has been shown to increase a person’s likelihood of developing cancer—studies show that smoking causes cancer.
When reading about research, look closely at the language used. If an article states that X is associated with Y, or A is linked to B, it is describing a correlation. That study has not shown X to cause Y or A to result in B. Correlation and causation are two separate conclusions, each requiring a specific type of study—often multiple studies. “Ability to derive causation is determined by the study design and the totality of the evidence, not just one observational study,” says Gurley.
For example, a 2019 study found that people with high blood levels of linoleic acid, an omega-6 fatty acid, had a lower incidence of cardiovascular disease and stroke. The finding was that there was an association between high levels of linoleic acid and lower risk of cardiovascular disease and stroke—not that consuming linoleic acid will directly reduce your risk of those outcomes. Separately, a controlled trial in which some participants consumed specific amounts of linoleic acid and others did not showed a causal relationship between linoleic acid and cholesterol levels.
Other important details to review in a study
To better understand a study’s results or significance, also consider the following questions:
What journal is it published in? Where an article is published can tell you whether it has gone through rigorous quality review—an essential process for findings that could be used to inform health and policy decisions. Reputable journals follow ethical publishing standards and have transparent peer review processes and an editorial board of relevant experts.
Illegitimate and predatory journals, on the other hand, are designed to collect fees from authors without reviewing submissions for quality, plagiarism, or ethics. Gurley recommends checking journal names against Beall’s List of Potential Predatory Journals and Publishers before reading or citing research they’ve published.
When was it published? If it’s been a few years since the study was published, it’s worth checking if the authors have published on the same topic more recently. Reading an author’s related articles—from before or after the one you started with—can give you an idea of how knowledge in the field has evolved. To find earlier research on the topic, look at the study’s references, which appear at the end of the paper. For related research done since, look for a section titled “Cited by.”
How many people participated in the study? A study may include anywhere from a few participants to thousands or millions. That number is essential to contextualizing the results as percentages and rates. “Denominators matter,” Gurley says. For example, 10 people experiencing a negative effect from a drug will imply much greater risk in a study of 100 participants compared to one with 100,000 participants. Additionally, some outcomes observed will happen by chance alone—unrelated to what’s being studied—and in a small study group, these outcomes can skew the results.
Can the findings be generalized to an entire population? Look closely at who was enrolled in the study. “For example, if a study enrolls people over the age of 65, it may or may not be appropriate to generalize the results to young adults,” explains Gurley. Similarly, a study that enrolls only healthy individuals, or only men, may warrant additional research into how the same exposure or intervention affects individuals with underlying conditions, or women, respectively. This type of information is often noted in the study’s limitations but may not make it into how the findings are shared by news media or influencers.
Do the authors have any conflicts of interest? Reputable journals require authors to disclose information that could be perceived as influencing their objectivity or integrity and will publish that information with the article. For example, a researcher having financial or personal ties to an organization that might be impacted by their study’s results could affect how they design a study and report their findings. “Reputable journals will include a declaration of conflicts, including if there are no conflicts,” Gurley says. “If there are no declarations in the paper, you should assume that there could be conflicts among authors.”
How journals determine what to publish
When researchers submit an article to a journal, it goes through the peer review process: In addition to review by the journal’s editors, scientists with expertise in the same field of study volunteer to read the article, give feedback, and advise on whether it should be published in that journal.
The purpose of peer review is to ensure quality and trustworthiness by confirming that a study meets high scientific standards and adds to the existing body of research. In addition to weighing whether the article will be of interest to the journal’s audience, reviewers look for potential errors or gaps in reasoning; they may recommend that the authors make edits or complete new analyses to improve the study’s clarity or data presentation before it can be published. Some journals will even publish the peer review alongside the paper, offering additional transparency around the critique the article received prior to publication.
“Peer review isn’t perfect,” says Gurley, noting that a peer-reviewed paper may still contain errors that require later corrections or may even result in the paper being retracted. “But it is an important quality check process. … If something hasn’t been peer reviewed, it’s not a scientific publication.”
The purpose of preprints
You may come across a full research study labeled as a “preprint”—this is a scientific manuscript that has not yet gone through the peer review process. Preprints are a way for scientists to share new findings quickly with other researchers, says Gurley, but are not intended for a general audience and may contain errors.
Trustworthy preprint servers—including arXiv, bioRxiv, medRxiv, and SSRN—will screen submissions for things like plagiarism, personally identifiable information, and confirmation of participant consent. But because the submissions have not been scrutinized and validated through peer review, readers should be mindful that preprints are not the final version of the study. Preprints should not be used to support clinical practice or individual health behavior.
The bigger picture
Any one study—or scientist, or health influencer—will not have all the answers. Findings from individual studies may contradict each other. Many studies can suggest an association or link between two things but are not designed to prove causality. It’s important to view a study’s findings as part of a larger body of evidence and to recognize that scientific understanding evolves over time—often slowly and incrementally.
It’s good to be curious and wary of dramatic headlines, Gurley says. She encourages people to ask deeper questions on topics they want to understand and know where to go to find the best answers. That’s how science progresses, after all—it starts with asking questions.