RCTs – GPS or Compass?

A heavy emphasis on Evidenced Based Practice combined with an increase in online journals and an emphasis on publication for many teaching posts has seen an explosion in RCTs of varying quality making their way into the hands of ever increasing numbers of clinicians thanks to the world wide web.

 

This must be a good thing, surely?

If properly critiqued and integrated appropriately into clinical practice then absolutely. Is this occurring with the level of scrutiny required? In many cases possibly not.

During my teaching duties in many countries, I am often confronted with discussions on evidence. The presumption is that there must be an RCT before something can be indicated but even more worrying is the notion that RCTs are required to guide and negotiate every twist and turn in a client’s rehab, just like a GPS, delivering our clients and clinicians safely to destination cured. The use of the intervention from the RCT seemingly based more on the diagnosis and less on a thorough assessment and sound clinical reasoning process, the assessment and reasoning process when it does exist resembling inclusion and exclusion criteria for the RCT rather than a thorough exploration of the patient themselves.

In my clinical practice, I consider the literature to be more like a compass than a GPS for a number of reasons:

 

Reason 1: Not everyone in the cohort responded in the way that the RCT may conclude is statistically significant.

In other words, some people will have improved, some will have stayed much the same while others may have in fact gotten worse with the intervention. Which category does the patient in front of you slot into? A thorough and ongoing assessment and reassessment is required to determine the response to the intervention but determining the various influences at the outset might avoid inappropriate use of the intervention in the first place.

 

Reason 2:The patient in front of you might not have been appropriate for the study.

How much does the study actually relate to this person? My caseload has largely consisted of those with persistent pain who have had any number of procedures, surgeries and treatments by a variety of clinicians or they are elite athletes. They would either have been excluded from the majority of studies due to the complexity of their presentation or there is no study that reflects my athlete’s situation as it is simply not possible to replicate all the biopsychosocial variables in a controlled environment.
For example, one of my athletes won the bronze medal in beach volleyball at the Commonwealth Games in 2018 nursing an infant. I am yet to see a study on female beach volleyball players in a final at a major multisport event, nursing infants, let alone with the same clinical condition and contributing factors.

An element of interpretation, extrapolation, integration and evaluation drawing on research across a range of subject matter from pathology, pathophysiology, pathomechanics, pain physiology, aetiology and endocrinology is likely warranted to develop and test appropriate hypotheses in the individual.

 

Reason 3:The outcome measures of the study are not consistent or commensurate with the client’s goals.

At the Commonwealth Games, my volleyballer’s goal was to get through the next match and take home the bronze medal. It was not to have a moderate reduction in pain in four weeks or to be in the 90% of people whose problem has resolved in 12 months. Similarly, I once had a gentleman ask me how he could use Dynamic Tape for his footdrop as his daughter was getting married on the weekend and he preferred not to walk her down the aisle using an AFO. He was not looking for a cure. He was not concerned with changes in EMG or nerve conduction tests. I was able to provide some direction and he could quickly determine whether it achieved his goal or not. No RCT required. N = 1 when working clinically.

 

Reason 4: Subjects are often not classified sufficiently creating significant washout in the results.

This may in part be responsible for the outcomes observed in point 1. A study may consist of 149 low back pain patients who have all had pain for greater than three months and it does not extend beyond their buttock.  That does not make this group homogenous. Why do we think a single intervention will work for all of them? That is a search for a panacea rather than evaluating a clinically reasoned and indicated intervention. We observe this time and again with research into manual therapy.

Certainly, some in the group may respond well and if identified and evaluated show a strong effect. I relate this to measuring the effect of firefighters on putting out fires by measuring the temperature of all the houses in the city pre and post being doused with water. If only ten homes out of 50,000 are on fire we are not likely to see much statistical difference in temperature across the group due to the washout (pardon the pun) effect. If however we correctly identify those on fire and in need of the intervention and study them in isolation, the effect would be wildly significant.

 

Reason 5: Conclusions are often very broad beyond what can deduced from the small amount of information gleaned from the study.

For example, a study on Dynamic Taping may show no change in 3D kinematics and therefore conclude that the tape is ineffective (this has not been shown by the way). Examining kinematics only provides part of the picture. It does not provide any information on kinetics. Position or movement does not equate to load. Consider a person’s foot and ankle when standing flat on a step in plantargrade and then imagine the person moving backwards so that they are fully supported on the balls of their feet with their heels suspended off the back of the step but still in the plantargrade position. A measure of 3D kinematics would show no difference – roughly 90º. A force platform would show a big difference as would EMG of the calf for example. If the foot is then taped into full plantar flexion using a Dynamic Tape Powerband application, the tape will provide considerable resistance to the foot moving into the plantargrade position on the back of the step thereby reducing the work that the calf must do to maintain that position. You can see some videos of the difference that Dynamic Tape can make here – https://www.youtube.com/watch?v=D_zkHEDmadI&t=27s

Reason 6: One paper is preliminary evidence only

We generally see that research can take several years to be incorporated into clinical practice but we also see the opposite where one paper has completely changed the way that people treat and dismiss anything that has gone before this. Often papers can be misrepresented through media, social media, blogs etc. and be rapidly adopted. Isometric exercises suddenly being used as a magic fix for all tendinopathies based on one paper on the patellar tendon. We have seen phases or trends from eccentrics to isometrics to heavy slow resistance, time under load and so on. Again, this often dismisses the patient’s goal. Do any of these get a tennis player through a four hour final at Wimbledon? Can managing some load via an external application like in the above video? Pain in tendinopathy is generally load dependent. Can reducing the workload requirements of the calf-achilles complex reduce stimulation of load dependent peripherally sensitized nociceptors – almost certainly.

In this age of information, clinicians often only scan the abstract and do not identify methodological flaws or overreaching conclusions as described above. Case studies which can in fact be high quality evidence and serve to guide larger studies are often dismissed despite being incredibly valuable. The patient gives way to the paper.

The literature is important but must be scrutinized and then interpreted against a backdrop of patient goals and situation specific factors. Assessment must examine biomedical and psychosocial aspects, their inter-relationship and how this impacts our aims. The most sensitive and specific evaluation procedure is of little value if applied incorrectly or used inappropriately. Clinical expertise is essential.

To me, the literature acts more like a map and compass. Road blocks often pop up on the most direct route and our original heading, the one that the RCT suggests is no longer useful. Using our map and compass (broad knowledge of related literature and fundamental sciences), we may be able to take on other landmarks and information to work our way around the block and find our way to the destination on a new heading. Our destination may change mid way through the journey based on the client (e.g. they may decide to play a different tournament giving more or less time) or the vehicle we have been using to get there might not be suitable for the journey or might simply not be working the way it should so we need to abandon it and its GPS and go on foot.