Abstract
Clinical decisions are increasingly driven by evidence-based recommendations of guideline groups, which aim to be based on the highest quality knowledge—randomized clinical trials (RCTs) and meta-analyses. Although RCTs provide the best assessment of the overall value of a therapy, high-quality evidence from RCTs is often incomplete, contradictory, or absent even in areas that have been most exhaustively studied. Moreover, the likelihood of the success or failure of a therapy is not identical in all the individuals treated in any trial because therapy is not the only determinant of outcome. Therefore, the overall results of a trial cannot be assumed to apply to any particular individual, not even someone who corresponds to all the entry criteria for the trial. In addition, the potential for bias due to financial conflicts remains in many guideline groups. Guidelines are key sources of knowledge. Nevertheless, limitations in the extent, quality, generalizability, and transferability of evidence mean that we clinicians must still reason through the best choices for an individual because even in the absence of full and secure knowledge, clinical decisions must still be made. Clinical reasoning is the pragmatic, tried-and-true process of expert clinical problem solving that does value mechanistic reasoning and clinical experience as well as RCTs and observational studies. Clinicians must continue to value clinical reasoning if our aim is the best clinical care for all the individuals we treat.
Abbreviations and Acronyms:
EBM (evidence-based medicine), LDL (low-density lipoprotein), RCT (randomized clinical trial)Evidence-based medicine (EBM) was originally defined as the process of “integrating individual clinical expertise and the best available external clinical evidence from systematic research.”
1
The patient and the physician remained the central decision makers, but the physician was alerted to the randomized clinical trial (RCT) as the best experimental model to determine the potential benefits of new therapies. Evidence-based medicine, so defined, was unquestionably a major advance and an overdue corrective to medical opinions and decisions that too often were not based on rigorously obtained evidence, but rather were the products of inappropriate extensions of mechanistic reasoning and selective recall of personal experience. Beliefs, sometimes, became evidence. Patients with coronary disease died of ventricular arrhythmias. Therefore, suppressing ventricular extrasystoles with antiarrhythmic agents should save lives. However, this is not true. Indeed, the opposite was found in an RCT,2
and the death of that unfounded belief gave birth to EBM.3
Randomized clinical trials are essential to evaluate therapies that reduce rather than eliminate a complication of a disease. However, EBM in its first formulation failed because most physicians did not think that they were sufficiently expert or had enough time to evaluate research publications that appeared with exponentially increasing frequency and often reported conflicting results that were described in a highly technical and recondite language, language that sometimes seemed designed to intimidate, not inform. In response, the guideline process was created. Experts were gathered and charged to evaluate the evidence and to issue recommendations regarding actions that should or should not be taken. The decisional process became codified and displaced from the individual patient and physician.
Much good has been achieved from the guideline process. As the guideline process has evolved, the categorization and weighting of evidence has been progressively formalized such that the “best” evidence is separated from the rest, and recommendations are to be based on the best. Thus, RCTs, and, even better, meta-analyses of RCTs, are the highest quality evidence, whereas clinical experience and mechanistic reasoning are not accepted either as evidence or as tests of evidence.
3
Guideline recommendations have become the standard of care, and quality of care is increasingly assessed on the basis of adherence to these recommendations. To be sure, the recommendations of guideline groups are always accompanied by a brief statement that they can be modified on the basis of individual patient circumstances and preferences. But just how this is to be done and what role clinical expertise and mechanistic reasoning should play in this process are not formally addressed. Indeed, if clinical expertise and mechanistic reasoning are disallowed any role in framing guideline recommendations about care, how can they be legitimate tools to modify such recommendations?
Unfortunately, the insights gained in every age may become its blind spots, just as its advances may become its excesses. Guidelines and EBM are no exception. We accept that RCTs are the best method to test therapies, and we salute the efforts our colleagues make in serving on guideline groups. However, we believe that the evidence RCTs provide is too often incomplete, inconclusive, absent, or outdated. Indeed, guideline writers themselves have documented that most recommendations are not based on RCTs and meta-analyses, but rather are increasingly supported only by lower levels of evidence.
4
Therefore, there is an inherent limit as to how definitive guideline recommendations can be.Finally, and a point not widely understood or commented on, there is, at least for the moment, a critical limitation in our ability to meaningfully translate the overall results of a trial into the likelihood that an individual patient will respond to, or be injured by, the therapy that was tested.
Notwithstanding our colleagues’ best efforts, guidelines have not overcome these limitations in the evidence and its transferability, nor indeed have they generally acknowledged them. Thus, we argue that clinical reasoning—the expert, pragmatic, complex, subtle, analytical process that physicians and surgeons have routinely used to solve clinical problems in individual patients—continues to play an indispensable role in clinical decision making because the individual patient, who is real, and not the “average” patient, who is not, needs to remain the focus of our attention. We do not argue against guidelines being major determinants of care. We do argue against them being the only determinants of care.
Limitations in the Evidence in EBM
No Evidence, Incomplete Evidence, or Conflicting Evidence
We contend that limitations in the evidence are a major limitation of EBM. For many clinical problems, there is simply no RCT evidence to apply. Consider antibiotic prophylaxis for endocarditis. The most recent guidelines
5
differ dramatically from those that preceded them.- Wilson W.
- Taubert K.A.
- Gewitz M.
- et al.
Prevention of infective endocarditis: guidelines from the American Heart Association: a guideline from the American Heart Association Rheumatic Fever, Endocarditis, and Kawasaki Disease Committee, Council on Cardiovascular Disease in the Young, and the Council on Clinical Cardiology, Council on Cardiovascular Surgery and Anesthesia, and the Quality of Care and Outcomes Research Interdisciplinary Working Group.
Circulation. 2007; 116: 1736-1754
6
However, the evidence, or what there is of it, did not change. Time and many of the evaluators did. For many other issues, multiple RCTs have been performed, but the outcomes and the conclusions may conflict,7
with the result that meta-analyses are now considered the highest form of evidence. However, even the results of meta-analyses may sometimes conflict8
and, even in the most intensively studied areas, knowledge remains incomplete.For example, multiple RCTs have established unequivocally the benefit of statin therapy in reducing cardiovascular risk. Indeed, statin therapy must be among the therapies most intensely studied and a therapy whose value has been confirmed repeatedly and resoundingly by RCTs. Thus, one might assume that we know all we need to know about statin therapy. We do not.
Sharp public debate on the role of statins in the primary prevention of cardiovascular disease remains.
9
In contrast, substantially on the basis of a meta-analysis of 5 dose comparison studies,Should healthy people take cholesterol drugs to prevent heart disease? Wall Street Journal. http://online.wsj.com/article/SB10001424052970203471004577145053566185694.html. Accessed January 11, 2013.
10
the conclusion that the highest doses of statins are the best choice has been widely accepted, as evidenced by the most recent European11
and Canadian12
guidelines. However, in 2 of the 5 studies that make up the meta-analysis, a dose of 80 mg, simvastatin once a day was used; unfortunately, this dose is not acceptably safe.13
In the 3 other studies, 80 mg of atorvastatin was superior to 10 mg in one study and inferior to 20 mg of pravastatin in another, and 40/80 mg of atorvastatin was superior to 20/40 mg of simvastatin in the third study—not a compelling body of evidence on which to base public policy.US Food and Drug Administration website. FDA Drug Safety Communication: New restrictions, contraindications, and dose limitations for Zocor (simvastatin) to reduce the risk of muscle injury. http://www.fda.gov/Drugs/DrugSafety/ucm256581.htm. Accessed April 1, 2013
Moreover, the evidence as to the best dose of a statin is reportedly incomplete because, for practical purposes, only the highest and lowest doses of statins have been compared.
10
Indeed, there is no evidence that the highest dose of any statin is superior to an intermediate dose: 80 mg, atorvastatin vs either 40 mg, atorvastatin once a day or 20 mg, atorvastatin once a day. Although not always appreciated, most of the benefit in terms of low-density lipoprotein (LDL) cholesterol lowering is achieved with the lowest dose of statins. Doubling the dose produces only limited further lowering of LDL cholesterol, whereas adverse effects include myalgia,14
risk of diabetes,15
and perhaps an increase in acute renal injury16
as the dose increases. Thus, no recommendation of the highest dose as the “best” dose can be evidence based because the necessary evidence to do so is absent.10
Nevertheless, the physician must make a decision as to which dose to prescribe to a particular patient.There is also the reality that critical evidence will continue to appear in the not inconsiderable interval between editions of many guidelines. For example, the Cochrane Report
17
that the treatment of mild hypertension—blood pressures in the range of 150/90 to 159/99 mm Hg—has not been found to be of clinical benefit appeared in 2012, challenging the validity of recommendations made almost a decade before.18
Publications of this potential import from sources with such authority cannot be just set aside until the next guideline group meets to consider its significance. Nor can their conclusions be simply accepted on blind faith. We need to reconsider the options with each patient we treat because we are always deciding in the interludes between the latest and the next recommendations.- Chobanian A.V.
- Bakris G.L.
- Black H.R.
- et al.
National Heart, Lung, and Blood Institute Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure; National High Blood Pressure Education Program Coordinating Committee
The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation and Treatment of High Blood Pressure: the JNC 7 Report.
The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation and Treatment of High Blood Pressure: the JNC 7 Report.
JAMA. 2003; 289: 2560-2572
Finally, randomization and blinded assignment, the hallmarks of an RCT, are not possible for many therapies, including most surgical interventions.
19
Indeed, with many medical acts, not only is there a learning curve but skill and judgment remain essential determinants of the value of the intervention. What is of value elsewhere may not be of value everywhere. Just as in the end, all politics is local, so is all medical care.Limitations in the Generalizability of RCTs
By generalizability, we mean whether the overall conclusions of the study population apply to other groups of patients with the same clinical problem. Limitations in the generalizability of their results are the second fundamental limitation of RCTs.
20
The large clinical trials, which should yield the most secure results, require the participation of hundreds of sites, generally in multiple countries. The aggregate sample may be large, but inclusion criteria almost always ensure that it will not be representative of the overall population with the clinical problem. Age and gender restrictions are common. Comorbid conditions can be excluded from trials but not from clinical practice.But the challenges go deeper. Even within the stated criteria, the sample selected may not be equivalent to the whole to which it is said to be representative. Usually, only a handful of patients are recruited from each center. How typical are they of the local experience? Patients recruited from specialty centers may not resemble those seen in general practice and vice versa. Patients selected for clinical trials differ by definition from those who were ineligible and indeed often differ from those who were eligible but not recruited, and these differences can affect the outcome.
21
, 22
Trials with run-in periods select patients (1) who can tolerate the therapy and (2) who respond to the therapy. The results of such trials should not be extrapolated to all those patients in whom such therapy might be initiated. And yet they are.Limitations in the Transferability of Group Results to the Individual
Randomized clinical trials determine the net outcomes in groups of individuals, but those probabilities are not simply and directly transferable to all individuals within the groups.
23
, 24
, 25
This is the third fundamental methodological limitation of RCTs and therefore of present-day EBM, a methodological limitation that has received little attention and one that most physicians are not aware of. The overall probability of benefit documented in the treated group in an RCT would apply only to individuals with characteristics identical to those studied if the likelihood of drug-delivered benefit and drug-produced adverse events was equally probable in all individuals in the group that had been studied. This is almost never the case. Within any clinical trial, individuals are recruited with various characteristics, many of which, even if not specified, will affect outcome. These effects are balanced by randomization into groups in the RCT but come into full play in the clinical outcome of the individual. Moreover, there is frequently substantial heterogeneity in the individual responses to therapy, such as the wide variance in lowering of LDL cholesterol by rosuvastatin,26
a phenomenon that should raise doubt as to the validity of any recommendation of a standard one-size-fits-all regimen–based strategy.27
An even more dramatic reality is that the same therapy may benefit some and injure others, but the beneficiary and the victim cannot be confidently distinguished by conventional analysis of an RCT. We need to learn how this and identifying the clinical exceptions to the guideline rules and learning from them could provide important new insights into who is particularly likely to benefit from a therapy and who is particularly likely to be harmed.Limitations in Meta-analyses
Meta-analyses are considered the highest level of evidence. Meta-analyses integrate the results of a number of studies to calculate more precisely what the average result of an intervention is likely to be. However, meta-analyses select the studies they include on the basis of certain criteria. These restrictions may be reasonable to ensure that all studies are comparable in certain important aspects but will, unfortunately, from time to time, eliminate a particular trial whose design might mirror exactly a specific clinical problem. Moreover, in meta-analyses, the range of possible results is not presented, only the range of the average result. The trials included in a meta-analysis almost inevitably differ somewhat in precisely what treatment was given and to whom it was given. This may increase the robustness of the calculation of the average effect, but it makes less certain to whom this effect applies.
Although the relative benefit of a therapy may be similar in multiple patient groups, the actual benefit is almost always a function of specific patient characteristics. Thus, the older the patient, the higher the blood pressure, and the higher the cholesterol, the greater the absolute benefit of lowering blood pressure or cholesterol.
28
So, the irony is that as our knowledge about the outcome in groups becomes stronger, the relation of that outcome to individuals becomes weaker.Limitations in the Guideline Process
Guidelines have become the forum in which the evidence is evaluated. Guidelines acquire their authority from multiple sources: from the perceived expertise and integrity of those who participate in them, from the prestige of the societies that sponsor them, from the renown of the journals in which they appear, and, increasingly, from the perceived consequences of not adhering to them. However, there are failings that have not been adequately addressed, which include the persistent challenge of financial conflicts of interest among those who participate,
29
the failure to ensure that reasonable dissenting viewpoints in scientific debate are heard and that minority reports are provided when reasonable differences in view persist,30
and, finally, the reality that there is no effective process to challenge the validity of specific conclusions that guidelines reach. For science to advance, challenge must be possible.The guideline process is too important to fail, and we must work together to improve its performance. Nevertheless, even the powerful guidance to care provided by the guidelines does not relieve physicians of their responsibility to determine the most appropriate therapy for the specific patient they are treating.
What is Clinical Reasoning?
If the highest-quality evidence is incomplete, as it so often is and as guideline writers themselves admit,
4
how can the gap to best care be bridged? We argue that clinical reasoning is the best tool the physician can use to do so. Of course, our reasoning is imperfect, but, as Croskerry31
has recently highlighted, not all forms of decision making are equally fragile. There is the intuitive mode of problem solving, the rapid, generally subconscious approach, driven by experience—an approach that, although subject to considerable error because of all our cognitive biases, is nevertheless indispensable given the number of decisions a clinician must make every day.Clinical reasoning stands in contrast to intuitive decision making. Clinical reasoning is the disciplined, analytical, scientific approach that integrates all the relevant information in the search for the best approach to diagnosis and therapy for individual patients.
32
It does not supply generic answers for groups and is, therefore, not the same as expert opinion, which proposes general approaches on the basis of personal experience or personal analysis and which, deservedly, has been much criticized. Clinical reasoning is certainly not a license to guess or to ignore RCTs. Guidelines, which summarize evidence and offer recommendations, are the starting point of clinical reasoning. However, clinical reasoning accepts the probabilistic nature of decisions and acknowledges that all decisions are provisional. Clinical reasoning is pragmatic to its core because it declares that clinical decisions should be based on an analysis of the consequences of acting or not acting on what we think will happen to the patient immediately in front of us if we do this rather than that.33
Complexity and imagination are defining features of clinical reasoning. Indeed, it is our individual abilities to imagine all that is possible, to grade the likelihood and the significance of one outcome vs another, to understand the limitations as well as the strengths in our analysis of the evidence, to evaluate accurately the condition and circumstances of the individual patient, to know in detail the multiplicity of possible responses to the therapies we are considering; it is all these skills and reasoning wrapped together that distinguishes the clinical reasoning of the expert clinician.
An example may be helpful. A postoperative patient was not doing well after mitral valve replacement, but there were, as is so often the case, many competing explanations without clear evidence for one as opposed to another. Routine investigations included echocardiography, which documented normal left ventricular systolic function and only mild aortic regurgitation. Nevertheless, the pulmonary hydrostatic pressures were markedly elevated, and no acceptable explanation was apparent. Thus, a decision was made to perform an aortogram; it revealed severe aortic insufficiency, which, by producing high left ventricular diastolic pressures, resulted in high pulmonary pressures. The severe aortic insufficiency was a complication of the insertion of the mitral valve, an unusual complication to be sure, but one that was hemodynamically devastating. Aortic valve replacement was followed by rapid and complete recovery. In retrospect, the echocardiogram apparently underestimated the degree of regurgitation because of the rapid decay in aortic pressure and the rapid increase in left ventricular pressure produced by the severe acute regurgitation.
Clinical reasoning is the problem-solving process that first determined the severity of the clinical problem (the patient might well not survive), then identified a specific feature that needed to be explained (the elevated pulmonary hydrostatic pressures), and finally determined that the evidence that had been gathered by routine processes (the echocardiogram) was inadequate and needed to be set aside. Indeed, our ability to recognize that the evidence may be misleading or inadequate even when we do not know the specific flaw at that moment is a hallmark of clinical reasoning. And so is the confidence to proceed when necessary with incomplete information and incomplete understanding. Only after the aortogram was obtained did the facts that seemed discordant and contradictory fall into place. This example illustrates why we characterize clinical reasoning as a complex, pragmatic, and imaginative reasoning process.
We stand with Bradford Hill, one of the fathers of RCTs, who said that “Any belief that the controlled trial is the only way would mean not that the pendulum had swung too far but that it had come right off the hook.”
Conclusion
Clinical reasoning remains integral to clinical care because the evidence from RCTs remains incomplete and the generalizability and transferability of the results of RCTs remain challenging. Few physicians, if any, have the time or the expertise to review all the relevant knowledge, and therefore, the guideline process remains essential to sort out and evaluate what is known. However, the evaluation of evidence is only one step in the process of care. To be sure, guidelines frequently state, at one point or another, that their recommendations are just that—recommendations—and they should be modified on the basis of the particular wishes and circumstances of the patient. They do not, however, identify this additional step as an integral part of the process, and they do not, specifically and unequivocally, validate clinical reasoning as a copartner in care. On the basis of the actual medical circumstances of individuals and their personal preferences, reasoned and reasonable extrapolations from recommendations remain essential, and guidelines and organizations that oversee care must leave room for this to occur.
Clinical reasoning is essential to integrate the knowledge from RCTs into a specific clinical context. We need to acknowledge this and value and teach clinical reasoning if we are to appropriately value ourselves as caregivers and meet our responsibilities to patients who have placed their trust in us.
Supplemental Online Material
References
- Evidence based medicine: what it is and what it isn’t.BMJ. 1996; 312: 71-72
- Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction.N Engl J Med. 1989; 321: 406-412
- The Philosophy of Evidence-Based Medicine.Wiley-Blackwell/BMJ Books, Chichester, United Kingdom2011
- Scientific evidence underlying the ACC/AHA clinical practice guidelines.JAMA. 2009; 301: 831-841
- Prevention of infective endocarditis: guidelines from the American Heart Association: a guideline from the American Heart Association Rheumatic Fever, Endocarditis, and Kawasaki Disease Committee, Council on Cardiovascular Disease in the Young, and the Council on Clinical Cardiology, Council on Cardiovascular Surgery and Anesthesia, and the Quality of Care and Outcomes Research Interdisciplinary Working Group.Circulation. 2007; 116: 1736-1754
- Prevention of bacterial endocarditis: recommendations by the American Heart Association.JAMA. 1997; 277: 1794-1801
- Contradicted and initially stronger effects in highly cited clinical research.JAMA. 2005; 294: 218-228
- Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects.J Clin Epidemiol. 2011; 64: 1060-1069
Should healthy people take cholesterol drugs to prevent heart disease? Wall Street Journal. http://online.wsj.com/article/SB10001424052970203471004577145053566185694.html. Accessed January 11, 2013.
- Is lower and lower better and better? a re-evaluation of the evidence from the Cholesterol Treatment Trialists’ Collaboration meta-analysis for low-density lipoprotein lowering.J Clin Lipidol. 2012; 6: 303-309
- ESC/EAS Guidelines for the management of dyslipidaemias: the Task Force for the management of dyslipidaemias of the European Society of Cardiology (ESC) and the European Atherosclerosis Society (EAS).Atherosclerosis. 2011; 217: 3-46
- 2012 Update of the Canadian Cardiovascular Society guidelines for the diagnosis and treatment of dyslipidemia for the prevention of cardiovascular disease in the adult.Can J Cardiol. 2013; 29: 151-167
US Food and Drug Administration website. FDA Drug Safety Communication: New restrictions, contraindications, and dose limitations for Zocor (simvastatin) to reduce the risk of muscle injury. http://www.fda.gov/Drugs/DrugSafety/ucm256581.htm. Accessed April 1, 2013
- Narrative review: statin-related myopathy.Ann Intern Med. 2009; 150: 858-868
- Risk of incident diabetes with intensive-dose compared with moderate-dose statin therapy: a meta-analysis.JAMA. 2011; 305: 2556-2564
- Use of high potency statins and rates of admission for acute kidney injury: multicenter, retrospective observational analysis of administrative databases.BMJ. 2013; 346: f880
- Pharmacotherapy for mild hypertension.Cochrane Database Syst Rev. 2012; : CD006742
- The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation and Treatment of High Blood Pressure: the JNC 7 Report.JAMA. 2003; 289: 2560-2572
- Challenges in evaluating surgical innovation.Lancet. 2009; 374: 1097-1104
- External validity of randomized controlled trials: “to whom do the results of this trial apply”?.Lancet. 2005; 365: 82-93
- What fraction of hospital ulcer patients is eligible for prospective trials?.Scand J Gastroenterol Suppl. 1991; 186: 73-76
- Prediction of benefit from carotid endarterectomy in individual patients: a risk modeling study.Lancet. 1999; 353: 2105-2110
- An Introduction to the Philosophy of Induction and Probability.Oxford University Press, New York, NY1989
- Beyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapies.Circulation. 2008; 118: 1294-1303
- Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages.Milbank Q. 2004; 82: 661-687
- Rosuvastatin, proprotein convertase subtilisin/kexin type 9 concentrations, and LDL cholesterol response: the JUPITER trial.Clin Chem. 2012; 58: 183-189
- Three reasons to abandon low-density lipoprotein targets: an open letter to the adult treatment panel IV of the National Institutes of Health.Circ Cardiovasc Qual Outcomes. 2012; 5: 2-5
- Efficacy and safety of more intensive lowering of LDL cholesterol: a meta-analysis of data from 170,000 participants in 26 randomized trials.Lancet. 2010; 376: 1670-1681
Lenzer J, Hoffman JR, Furberg CD, Ioannidis JPA. Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ. In press.
- Why guideline-making requires reform.JAMA. 2009; 301: 429-431
- From mindless to mindful practice—cognitive bias and clinical decision making.N Engl J Med. 2013; 368: 2445-2448
- Learning Clinical Reasoning.2nd ed. Lippincott, Williams & Wilkins, Baltimore, MD2010
- Meaning and Action: A Critical History of Pragmatism.2nd ed. Hackett Publishing Company, Indianapolis, IN1984: 431
- Reflections on the controlled trial.Ann Rheum Dis. 1966; 25: 107-113
Article info
Identification
Copyright
© 2013 Mayo Foundation for Medical Education and Research. Published by Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- Two Medical WorldviewsMayo Clinic ProceedingsVol. 89Issue 1
- PreviewI read with interest the articles and editorial in the October 2013 issue of Mayo Clinic Proceedings on empiricism and rationalism in medicine,1-3 a subject that has captured my attention for virtually my whole career. I agree with the editorial authors, Drs Lanier and Rajkumar,1 that these approaches are complementary and must both be utilized in the diagnosis and treatment of the individual patient.
- Full-Text
- Preview
