Science – Part 3: Do the results apply to me?

In the third part of the blog series, we explain why it is important to take a look at who participated in the study

Often we come across news about science and health. The media tends to present certain scientific studies as “miracles”. This catches the attention of many readers but, on the other hand, it also creates great expectations which are not always justified in reality.

This blog series about science presents three basic questions which we should ask ourselves when we read about science in the media. Answering these questions can help us create more realistic expectations.

The results of a study are what we are most interested in. However, as we have seen in part 1 of our blog series, the results of a study carried out with mice in a lab cannot be directly applied in medical practice. A few years will probably pass before the drug is sold on the market since its safety and effectiveness must be tested in humans before.

Furthermore it is important to know the study method, i.e. how it was carried out, since it relates directly to the degree of conclusiveness of a study. What should be assessed are the study design – at which we took a closer look in part 2 – and the stringency that was applied to the study. As we know today also the characteristics of the participants influence the results of a study – and therefore the question: Do the results apply to me?

If a study is in its lab phase, it usually takes a long time before we can benefit from the results.

Is the study representative or not?

Let’s imagine we want to scientifically research, for example, the quality of life of persons with spinal cord injury in Switzerland. In an ideal world we would ask all affected persons. In reality, however, it is impossible to collect data from everyone because we have neither the time nor the resources. Research therefore uses a sample of participants.

The sample can be representative for the so called population – in our example all persons with spinal cord injury in Switzerland – or the sample is not representative. In the first case the sample is a group of people whose characteristics, e.g. sex and age, reflect those of the population. In this case, the conclusion that can be drawn from the study can be applied to the entire population.

If the sample, however, is not representative of the population, the study results cannot be transferred to the population but only to persons who are comparable to the study participants (e.g. men with spinal cord injury aged 40-60). Knowing the characteristics of the study participants may therefore help to understand in how far the results are relevant and can be applied to us.

It is essential to know the characteristics of the study participants to determine for which people the results are relevant.

Have people with conditions comparable to mine been excluded from the study?

Experimental studies often aim at minimizing the number of included variables in order to identify the cause-and-effect relationship more easily. Then, for example, only healthy people are recruited or only men or persons of a specific age group. Furthermore, for ethical reasons, certain groups that are considered vulnerable cannot participate in the study. Pregnant women belong in this category.

The same applies to patients who suffer from various chronic diseases such as diabetes or high blood pressure and therefore take medication: They are often excluded from studies because it is unknown whether the drug to be tested in the study can have interactions with the medication taken and therefore pose a health risk. The study results can therefore not be applied universally or only to a smaller population.

For example: To examine the effectiveness of a new drug against spasticity, a study with paraplegics with no secondary conditions or complications between 18 and 65 years is carried out. The effectiveness of the drug can then be determined for persons with exactly these conditions. But how about a quadriplegic who takes drugs for chronic pain and bowel disorder? Will the new drug also work for her? Maybe but we don’t know for sure. We also don’t know how this new drug interacts with the medication she is already on.

Real-life examples

In 2014, the paraplegic Darek Fidyka was able to walk again with a walking aid. This was possible due to a therapy that was presented as “revolutionary” and which gave hope to many people with spinal cord injury worldwide. During this therapy, cells were transplanted into the spinal marrow combined with a two-year intensive neurorehabilitation program.

Today Darek Fidyka is able to walk again with a walking aid. (Source: AFP/BBC)

However, if you take a closer look, you will notice that Darek had a rather rare form of spinal cord injury: a clean cut caused by a knife. The question therefore is whether the therapy that helped Darek will also help people with a different type of injury than him – e. g. an injury of the cervical vertebra caused by crushing and not a sharp cut. Currently there is a study ongoing of which the results will be published in 2022.

Other studies examine interventions for patients who have suffered a spinal cord injury only recently. The results of these studies can therefore be hardly applied to people who have lived with a spinal cord injury for a while. An example is the international NISCI study; it examines the effectiveness of anti-Nogo-A antibodies to determine whether nerve fibers can regenerate more easily and reconnect.

Nogo-A antibodies help reconnect the nerves. (Source: https://jahresbericht.balgrist.ch/)

There are also studies which include only people with an incomplete spinal cord injury – the results are therefore especially promising for those people. This includes the Swiss study on epidural electrical stimulation (EES). The goal of using this method is to reactivate nerve fibers with electricity while bypassing the injury, which has interrupted communication between the brain, nerves and muscles. The less severe the injury is and the more intact the nerve fibers are, the more likely it is that the patients can benefit from this approach.

Gender discrimination in medical research

A big problem in medical research is that for decades only or almost only men participated in studies while the results also applied to women. It was, however, noted that there are limits to this approach. In fact, we know today that symptoms of a heart attack are not the same in women vs. men. In lack of specific studies with women, the first symptoms of a heart attack in women had not been noticed for years which reduced their chances of receiving timely optimal care.

Also, the ideal dosage of drugs can differ for men and women. Dosages that were recommended on the basis of studies with men have lead to problems in women or even fatalities. While there have been improvements since the 1990s, women participating in medical research studies today are still fewer compared to men.

Women are still underrepresented in medical studies.

In summary: If we read the results of a scientific study, we should watch out for two factors: 1. the characteristics of the study participants, above all age and sex; 2. health conditions of the participants, i.e. whether they had the exact same problems as we. If the characteristics and clinical situation of the participants is similar to ours, then we can correctly assume that the study results can be applied to us. But let’s not forget that each case is unique. It is therefore recommended to assess, together with your physician, whether the results can be applied to our specific situation.