Science – Part 2: What type of study are we dealing with?

In the second part of our blog series on science, we explain why the nature of the study makes a difference

We often come across news on science and health. The media tend to present certain scientific studies as “wonder”. This wakes the interest of many readers but also creates big expectations, which, in reality, are not always justified.

In this blog series on science, we present three basic questions that we should ask ourselves when reading news on science in the media. Answering these questions may help us to create expectations that are more realistic.

In the first part of our blog series, we have discussed the question: What is the current stage of the research? Knowing the timeline of research can help understand whether the results of a study can be applied in clinical practice in the near future, or whether it will still take years before the drug will be available on the market.

Part 2 gives answers to the question: What type of study are we dealing with?

Individual studies or a synthesis of many studies?

When we read about a study in the media, the focus often lies on the results. However, to assess the results correctly it is fundamental to know how the study was carried out, i.e. how the study design was laid out. Why? Simply because the study design affects the validity of the evidence found by conducting the study. In other words, it influences the conclusions that can be drawn from the results.

First, in the study design it is determined whether it is an individual study or a synthesis of studies. Individual studies can – independently of their nature and methodical stringency with which they were conducted – deliver interesting or even revolutionary results. The limitation is, however, that they are only individual studies.

One swallow does not make a summer, and a single study does not produce a meaningful result.

Syntheses, on the other hand, summarize the results of all studies available on a certain topic and often also assess the stringency applied when conducting the individual studies. The more stringent the studies included in the synthesis are, the more significant are the recommendations derived from the analysis. These recommendations influence health policy and medical practice.

What is an intervention study?

Second, it is important to differentiate between intervention and observational studies. The purpose of intervention studies is to test something, usually a new product such as a drug or a medical device. The goal of these studies is to understand whether better results can be achieved with the new product than with familiar products or a placebo.

The purpose of intervention studies is to test a new product, like the Lokomat here.

In comparison to observational studies, intervention studies provide clear correlations since all variables are controlled. This method allows to determine with certainty the cause-and-effect relationship between two factors while excluding other explanations.

Intervention studies in the health care sector are the so-called randomized controlled trials. This type of study is often chosen to assess the efficacy of a new drug. During these studies, a group of persons (the so-called study group) receives a specific new drug; another group (the so-called control group), on the other hand, receives a drug already available on the market or a placebo. By comparing the two groups it is assessed if the new drug achieves better or only similar results to the known drug or placebo.

What is an observational study?

In observational studies, on the other hand – as the name already reveals – a situation is observed without intervention. In cohort studies, for example, the occurrence of a health condition is observed, and its development is monitored over time. In Switzerland, the Swiss Spinal Cord Injury Cohort Study (SwiSCI Study) is a good example for a cohort study.

The SwiSCI Study logo

The 2011 initiated SwiSCI Study is the largest study ever conducted in Switzerland to describe the lives of para- and tetraplegics living in Switzerland and to monitor their development. Every five years, persons with a spinal cord injury are asked to fill out a questionnaire; in this questionnaire, they give information about their state of health and quality of life, participation in social activities, integration in the world of work, challenges due to aging, etc.

While a cohort study delivers fewer clear correlations than an intervention study, its advantage is that long-term changes can be observed under real-world conditions. With the collected information, scientists identify problem areas and are able to make recommendations. In the case of the SwiSCI Study, the scientists can, for example, make recommendations on measures for the improvement of the Swiss health and social system to better fulfill the needs of persons with spinal cord injury.

Which types of study has the highest validity?

The following graph shows, which types of studies deliver the most reliable evidence:

(Source: adapted from https://toolbox.eupati.eu)

• In this graph, the syntheses rank first: so-called meta-analyses and systematic reviews.
• Second are the intervention studies, as e.g. the randomized controlled trials or RCTs
• Third are the observational studies of which the cohort studies have the most informative value, followed by case-control studies, case studies and finally case series. From a case series, for example, we can learn about the characteristics of a new or rare condition or about unusual symptoms some patients experience that are observed in clinical practice. However, no conclusions can be drawn that can be directly applied to other people.
• On the lowest level are editorials and expert opinions. These are just opinions and not results achieved through scientific approaches. Therefore, they are not considered scientific proof.

Based on this knowledge, we can now better categorize news that present the results of a study as “revolutionary”. One of many examples was a 2014 study in which a person with spinal cord injury was able to walk again. Due to a cell therapy and following intensive rehabilitation period for more than two years, the person regained a certain sensitivity and was able to walk with support.

This was indeed a very positive outcome; however, when confirming the type of study, we will find that it was a case study. Knowing this, we can evaluate the outcome more carefully and not get our hopes up too high with respect to this therapy: An individual study with a single patient is only the beginning of a research project, not the end. Further studies are necessary to check if this therapy may bring the same benefit to other patients.

In this blog, we have learned how to recognize the type of study and how to assess the validity of the gained knowledge. Now we only need to find out whether the study results are also relevant and applicable for us. Don’t miss out on part 3!