Levels of Evidence: What Should Ophthalmologists Know?

Níveis de evidência: O que os Oftalmologistas devem saber?

Abstract

Facing an enormous influx of information from medical research, clinicians need to differentiate robust study findings from spurious ones. The levels of evidence are an important component of Evidence-Based Medicine. Understanding the levels helps the Ophthalmologist to prioritize information and make right clinical decisions. The aim of this article is to describe the hierarchy of studies regarding their scientific evidence focusing on ophthalmology.

Descritores:
Níveis de evidência; Pesquisa clínica; Medicina Baseada em Evidências; Tipos de estudos

Resumo

Em face a um enorme influxo de informações de pesquisa médica, os clínicos precisam diferenciar os achados de estudos robusto dos espúrios. Os níveis de evidência são um componente importante da Medicina Baseada em Evidências. Compreender os níveis ajuda ooftalmologista a priorizar as informações e tomar decisões clínicas corretas. O objetivo deste artigo é descrever a hierarquia dosestudos em relação à evidência científica com enfoque na oftalmologia.

Keywords:
Levels of evidence; Medical research; Evidence-Based Medicine; Types of studies

Introduction

Facing an enormous influx of information from medical research, clinicians need differentiate robust study findings from spurious ones and to decide which results they can use with high confidence and which they should be more skeptical about.⁽¹⁾ The levels of evidence (LE) are an important component of Evidence-Based Medicine (EBM). Understanding the levels and why they are assigned to publications and abstracts helps the reader to prioritize information.⁽²⁾

The LE were originally described in a report by the Canadian Task Force on the Periodic Health Examination in 1979.⁽³⁾ The purpose of the report was to develop recommendations on the periodic health exam and base those recommendations on evidence in the medical literature. The authors developed a system of rating evidence when determining the effectiveness of a particular intervention.

Since the introduction of LE, several other organizations and journals have adopted variation of the classification system. Diverse specialties are often asking different questions and it was recognized that the type and level needed to be modified accordingly.⁽²⁾

Ophthalmologists are encouraged to find the highest evidence to answer clinical questions. The aim of this article is to describe the hierarchy of studies regarding their scientific evidence focusing on ophthalmology.

Types of Studies

Multiple LE rating scales exist, and although there are some differences among the various scales, most are very similar. A pyramid (Figure 1) has expressed the idea of hierarchy of medical evidence for so long that not all evidence is the same.

FIG 1
LEVELS OF EVIDENCE OR HIERARCHY OF EVIDENCE.

(The hierarchies rank studies according to the probability of bias)

Various versions of the evidence pyramid have been described, but all of them focused on showing weaker study designs in the bottom (basic science and case series), followed by case-control and cohort studies in the middle, then randomized controlled trials (RCTs), and at the very top, systematic reviews and meta-analysis. This description is intuitive and likely correct in many instances. The placement of systematic reviews at the top underwent several changes in interpretation, but still was thought of as an item in the hierarchy.⁽⁴⁾

In this paper we will describe the study types starting from the smallest to the highest LE as well as the best practice guides for conducting each type of research.

Animal Research / In Vitro Studies (Basic science studies).

Basic science studies investigate the cause-outcome relationships between a dependent variable and independent variables, such as animal experiment, genetic and cell studies. Also, method development studies investigate the development and improvement of biochemical (e.g., enzymes, markers or genes).⁽⁵⁾ These kinds of study have a low LE, being at the base of the pyramid

Several checklists have been developed to guide authors in the preparing, conducting and reporting stages of their studies.⁽⁶⁾ The ARRIVE checklist supplies transparency and accuracy in the animal experiments.⁽⁷⁾

Over the past decade, a new discipline in biomedical research has emerged. Translational science, as it has been termed, is concerned with the application of laboratory or “bench” science to the diagnosis and treatment of human diseases.⁽⁸⁾ Moreover; the role of well-informed clinicians, invested in promoting as well as utilizing new research, is clearly recognized as crucial for the communal advancement of medicine and continued support of basic science.⁽⁹⁾ The Translational Eye Research stimulates communication between basic scientists and clinicians.

Experts Opinion / Letters.

Letters to the editor of an academic journal are usually open post publication reviews of a paper, often critical of some aspect of the original paper. An expert opinion is often biased by the author’s experience or opinions and there is no control of confounding factors.⁽²⁾ They provide an important platform for comments on current approaches in ocular medicine, but present low level in the hierarchy of evidence.

Case Reports / Case Series

Patient and disease characteristics related to some interesting and remarkable type defined in a patient are called a “case report”. When the number of patients is more than one, this is called a “case series”. These are the simplest research types and do not contain a control group. Case series are usually starting points of the examined hypothesis in the Case-control, cross-sectional or cohort studies.⁽¹⁰⁾

Case reports present clinical observations customarily collected in healthcare delivery settings. They have proved helpful in the identification of adverse and beneficial effects, the recognition of new diseases, unusual forms of common diseases, and the presentation of rare diseases.⁽¹¹⁾

The CARE (CAse REport) guidelines include a reporting checklist, an international initiative aimed at promoting transparent and accurate reporting of health research studies to enhance the value and reliability of medical research literature. This 13-item checklist includes indications regarding the title, key words, abstract, introduction, patient information, clinical findings, timeline, diagnostic assessment, therapeutic interventions, follow-up and outcomes, discussion, patient perspective, and informed consent. The implementation of the CARE guidelines by medical journals improved the completeness and transparency of published case reports.⁽¹²⁾

Case-Control Studies

When studying rare diseases or diseases with long latency, it makes sense to start with groups who do (cases) and do not (controls) have the outcome of interest and to investigate the exposures retrospectively. The advantage of this design is its biggest drawback: in assessing exposures retrospectively, cases may overreport exposures relative to controls (recall bias). Where and how to select the appropriate control group for a series of cases also may affect the study findings (potential selection bias).⁽¹⁾

Case-control studies are observational by design, generally quick, cheap, easy to perform. Furthermore, case-control studies are particularly suitable for studying risk factors associated with rare diseases or conditions.⁽¹³⁾ The STROBE statement for case-control studies guides authors (checklist of items that should be included in reports of observational studies).⁽¹⁴⁾

Cohort Studies

A Cohort is a special group of people who have been selected according to some defining characteristics and they have certain disease risk factors or health outcome. Cohort Studies, also called follow-up studies, are generally prospective and enquire: “What will happen in the future?” Individuals are followed over time in cohort studies, and researchers assess exposure and outcome during follow-up.⁽⁶⁾

A cohort study analyzes two or more groups forward from exposure to outcome. This study can be done by going ahead in time from the present (prospective cohort study) or, alternatively, by going back in time to comprise the cohorts and following them up to the present (retrospective cohort study). A cohort study is the best way to identify incidence and natural history of a disease, and can be used to examine multiple outcomes after a single exposure. However, this type of study is less useful for rare events or those that take a long time to develop. A cohort study should provide specific definitions of exposures and outcomes: determination of both should be as objective as possible.⁽¹⁵⁾As well as in case-control studies, the STROBE statement for cohort studies helps authors.⁽¹⁴⁾

Cohort studies often are viewed as the gold standard in observational epidemiology. The available evidence generated from these studies for eye diseases has been promising with the capacity to evaluate multiple outcomes with a measure of absolute risk in addition to a measure of association. The effort and expense to conduct cohort studies can be justified with these high LE.⁽¹⁶⁾

Randomized Controlled Trials

In modern-day EBM, randomized controlled trials (RCTs) represent a cornerstone on which we base our clinical decision.⁽¹⁷⁾ A RCT is a type of scientific experiment which aims to reduce bias when testing a new treatment. People participating in the trial are randomly allocated to either the group receiving the treatment under investigation or to the group receiving standard treatment (or placebo treatment) as the control. Randomization minimizes selection bias and the different comparison groups allow the researchers to determine any effects of the treatment when compared with the no treatment (control) group, while other variables are kept constant. The RCT is often considered the gold standard for a clinical trial. RCTs are often used to test the efficacy or effectiveness of various types of medical intervention and may provide information about adverse effects, such as drug reactions⁽¹⁸⁾. RCTs are expensive and slow, however, their LE is higher due to the fact that randomization removes the allocation bias.⁽⁶⁾

Shen et al.⁽¹⁹⁾ reported a Fragility of Results in Ophthalmology Randomized Controlled Trials. According to the authors, statistically significant dichotomous results in ophthalmology RCTs are often fragile, meaning that a difference of only a few events can change the statistical significance of the result.

The Optic Neuritis Treatment Trials ⁽²⁰⁾ is an example of a RCT. Many respected journals endorse the CONSORT statement in order to improve the scientific quality and transparency of RCTs. Authors should be used to the CONSORT statement as a guideline in RCTs.⁽²¹⁾

Systematic Review and Meta-Analysis

Several clinical studies (RCTs or Cohort) may be conducted in a clinical area over a period of years in different parts of the world. The results may be different and there may be different properties such as sample size and multicenter. A Meta-Analysis combines the statistical results of different studies in a particular clinical area.⁽²²⁾ Systematic reviews and meta-analyses are essential to summarize evidence relating to efficacy and safety of health care interventions accurately and reliably and the PRISMA statement guides the authors in the preparation of a Meta-Analysis.⁽²³⁾ A Systematic Review evaluates and interprets the evidence of all studies conducted in a clinical area. The main difference from a Meta-Analysis is that former combines the evidence of different studies based on interpretation instead of combining statistical results.⁽⁶⁾

Chen et al .⁽²⁴⁾ in a survey of systematic reviews and meta-analyses published in ophthalmology until 2010 noted some interesting aspects: 1- The number of published systematic reviews and meta-analyses in ophthalmology has been increasing progressively over the past few years; 2- Retina and glaucoma were the two major subspecialties accounting for 35% and 21% of the published, respectively; 3- The major topics published in retina were age-related macular degeneration (37%), tumors (14%), and diabetic retinopathy (12%); 4- The author affiliations of these studies were largely from the USA (30%) and the UK (22%); 5- About 60% of the systematic reviews and meta-analyses were published in ophthalmology journals, followed by the Cochrane Library (15.75%) and other non-ophthalmic journals (25.14%), respectively.

SOME IMPORTANT CONCEPTS

The goal of public health is to decrease or prevent diseases in the population. Relative risks (RR) and odds ratios (OR) estimate the strength of association between diseases and risk factors. A risk factor may be strongly related to a disease, but may contribute less to the problem of that disease in the population if its prevalence is low. In epidemiology, RR or relative risk is the ratio of the probability of an outcome in an exposed group to the probability of an outcome in an unexposed group. An OR is a statistical tool defined as the ratio of the odds of A in the presence of B and the odds of A without the presence of B. This statistic attempts to quantify the strength of the association between A and B.⁽²⁵⁾

RCTs are the gold standard in the assessment of a treatment effect. The magnitude of this effect can be presented in various ways. In 1998, Laupacis et al.⁽²⁶⁾ reported the Number Needed to Treat (NNT), an expression of the number of patients who must be treated to prevent one adverse event. A statistical tool called NNT has been proposed, and is now included in some textbooks and used in research articles and guidelines. The NNT is the inverse of the difference in rates and is usually expressed as a whole number. If the difference between the infection rates on two treatments is 17%, then 100 ‚ 17 = 6 is the NNT.⁽²⁷⁾

EVIDENCE IN OPHTHALMOLOGY: ARE WE DOING BETTER?

Ang et al. in 2001⁽²⁸⁾ analyzed publishing trends in two internationally renowned ophthalmology journals. In conclusion, it was suggested that the standard of publications has improved in the British Journal of Ophthalmology and the American Journal of Ophthalmology, with an increasing international contribution over the past two decades.

Siddiqui et al.⁽²⁹⁾ evaluated the quality of reporting of all diagnostic studies published in five major ophthalmic journals in the year 2002 using the Standards for Reporting of Diagnostic Accuracy (STARD) initiative parameters. According to the authors, the standards of that time of reporting of diagnostic accuracy tests are highly variable. The STARD initiative may be a useful tool for appraising the strengths and weaknesses of diagnostic accuracy studies.

Lai et al.⁽³⁰⁾ evaluated the proportion of interventions that are evidence based in the acute care unit of a regional eye hospital (Hong Kong Eye Hospital in July 2002). This study demonstrated that the majority of interventions in the ophthalmic unit were evidence based and comparable to the experience of other specialties.

Bojikian et al.⁽¹⁷⁾ in 2015 carried out a research to determine whether the LE of papers published in 4 major ophthalmology journals have improved over a decade. They identified all articles from American Journal of Ophthalmology, Archives of Ophthalmology (now JAMA Ophthalmology), British Journal of Ophthalmology, and Ophthalmology published from January 1, 1997, to December 31, 1997, and from January 1, 2007, to December 31, 2007. The articles were then screened to include only clinical articles. Each manuscript was assigned a LE using the criteria from the Oxford Centre for EBM. It was assessed whether citation frequency was associated with its levels of evidence. They concluded that, given the cost and difficulty associated with performing a iirge, prospective trial to answer any given question, it is realistic to assume that the lower LE publications will continue to play a large role in guiding our field. It seems that not only the absolute number, but also the proportion of high-level publications, has increased over the decade studied (all journals showed an improvement in their mean LE over this decade). Additionally, authors seem to be less frequently citing weakest LE publications, which should indicate improved critical evaluation.

RESEARCH FUNDING AND CONFLICTS OF INTEREST IN MEDICAL RESEARCH

Lexchin et al.⁽³¹⁾, in a systematic review on sponsorship and pharmaceutical research results and quality, noted that: Research sponsored by the drug industry was more likely to produce results favouring the product made by the company sponsoring the research than studies funded by other sources. Thus, when a pharmaceutical company funds research into drugs, studies are likely to produce results favourable to the sponsoring company’s product. This cannot be explained by the reported quality of the methods in research sponsored by industry. The result may be due to inappropriate comparators or to publication bias.

Bero et al. ⁽³²⁾ in a review published by Cochrane states that: Sponsorship of drug and device studies by the manufacturing company leads to more favorable efficacy results and conclusions than sponsorship by other sources.

According to Bekelman et al.,⁽³³⁾ financial relationships among industry, scientific investigators, and academic institutions are widespread. Conflicts of interest arising from these ties can influence biomedical research in important ways.

Conclusion

Not all therapeutic recommendations are based on evidence of equal quality. EBM is about finding evidence and using that evidence to make clinical decisions. A cornerstone of EBM is the hierarchical system of classifying evidence. However, this does not mean that this hierarchy should be adopted blindly; there is now increasing recognition that not all randomized, controlled trials are equal. A badly performed randomized, controlled trial may rank lower than a well-conducted cohort or case-control study. Furthermore, there is also increasing recognition that even a well-conducted randomized, controlled trial does not mean that an intervention is adopted automatically; translating a result into clinical practice depends on a consideration of local circumstances, patient values, and resource availability. The proportion of publications in ophthalmology has increased over time, therefore, understanding the types of scientific evidence as well as its hierarchy is fundamental to guide research without wasting time prioritizing information for correct clinical decision making.

• Funding sources: No specific financial support was provided for this study.

References

• ¹
  Wang JJ, Attia J. Study designs in epidemiology and levels of evidence. Am J Ophthalmol. 2010;149(3):367-70.
• ²
  Burns PB, Rohrich RJ, Chung KC. The levels of evidence and their role in evidence-based medicine. Plast Reconstr Surg. 2011;128(1):305-10.
• ³
  The periodic health examination. Canadian Task Force on the Periodic Health Examination. Can Med Assoc J. 1979; 121(9): 1193-254.
• ⁴
  Paul M, Leibovici L. Systematic review or meta-analysis? Their place in the evidence hierarchy. Clin Microbiol Infect. 2014;20(2):97-100.
• ⁵
  Röhrig B, du Prel JB, Wachtlin D, Blettner M. Types of study in medical research: part 3 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2009;106(15):262-8.
• ⁶
  SSüt N. Study designs in medicine. Balkan Med J. 2014; 31(4): 273-277.
• ⁷
  Carol Kilkenny, William J. Browne, Innes C. Cuthill, Michael Emerson, Douglas G. Altman Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biol. 2010 Jun; 8(6): e1000412.
• ⁸
  Knox BE. Translational science in ophthalmology. J Ophthalmic Vis Res. 2012;7(1):1.
• ⁹
  Fang FC, Casadevall A. Lost in translation-basic science in the era of translational research. Infect Immun. 2010;78(2):563-6.
• ¹⁰
  Rosenbaum SE. Basic and Clinical Biostatistics. JAMA. 1991;265(5):652.
• ¹¹
  Hauben M, Aronson JK. Gold standards in pharmacovigilance: the use of definitive anecdotal reports of adverse drug reactions as pure gold and high-grade ore. Drug Saf. 2007;30(8):645-55.
• ¹²
  Gagnier JJ, Kienle G, Altman DG, Moher D, Sox H, Riley D; CARE Group. The CARE guidelines: consensus-based clinical case reporting guideline development. Headache. 2013;53(10):1541-7.
• ¹³
  Sedgwick P. Case-control studies: advantages and disadvantages. MJ 2013;348:f7707 doi: 10.1136/bmj.f7707 (Published 3 January 2014).
  » https://doi.org/10.1136/bmj.f7707
• ¹⁴
  von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. G?tzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-9.
• ¹⁵
  Grimes DA, Schulz KF. Cohort studies: marching towards outcomes. Lancet. 2002;359(9303):341-5.
• ¹⁶
  Soh SE, Saw SM. Cohort studies: design and pitfalls. Am J Ophthalmol. 2010;150(1):3-5.
• ¹⁷
  Bojikian KD, Gupta D, Dettori JM, Dettori NJ, Dettori JR, Chang P, et al. Evidence in ophthalmology: are we doing better? Ophthalmology. 2015;122(12):2584-6.
• ¹⁸
  Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, et al. A method for assessing the quality of a randomized control trial. Control Clin Trials. 1981;2(1):31-49.
• ¹⁹
  Shen C, Shamsudeen I, Farrokhyar F, Sabri K. Fragility of Results in Ophthalmology Randomized Controlled Trials: A Systematic Review. Ophthalmology. 2018;125(5):642-8.
• ²⁰
  Beck RW, Cleary PA, Anderson MM Jr, Keltner JL, Shults WT, Kaufman DI, et al.; The Optic Neuritis Study Group. A randomized, controlled trial of corticosteroids in the treatment of acute optic neuritis. N Engl J Med. 1992;326(9):581-8.
• ²¹
  Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al.; CONSORT. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg. 2012;10(1):28-55.
• ²²
  Friedman L M, Furberg C D, DeMets D L: Fundamentals of Clinical Trials. 4rd edition. Springer-Verlag, New York; 2004.
• ²³
  Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.
• ²⁴
  Chen H, Jhanji V. Survey of systematic reviews and meta-analyses published in ophthalmology. Br J Ophthalmol. 2012;96(6):896-9.
• ²⁵
  Nakayama T, Zaman MM, Tanaka H. Reporting of attributable and relative risks, 1966-97. Lancet. 1998;351(9110):1179.
• ²⁶
  Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988;318(26):1728-33.
• ²⁷
  Hutton JL. Misleading statistics: the problems surrounding number needed to treat and number needed to harm. Pharmaceut Med. 2010;24(3):145-9.
• ²⁸
  Ang A, Tong L, Bhan A. Analysis of publication trends in two internationally renowned ophthalmology journals. Br J Ophthalmol. 2001;85(12):1497-8.
• ²⁹
  Siddiqui MA, Azuara-Blanco A, Burr J. The quality of reporting of diagnostic accuracy studies published in ophthalmic journals. Br J Ophthalmol. 2005;89(3):261-5.
• ³⁰
  Lai TY, Wong VW, Leung GM. Is ophthalmology evidence based? A clinical audit of the emergency unit of a regional eye hospital. Br J Ophthalmol. 2003;87(4):385-90.
• ³¹
  Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326(7400):1167-70.
• ³²
  Bero L. Industry sponsorship and research outcome: a Cochrane review. JAMA Intern Med. 2013;173(7):580-1.
• ³³
  Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454-65.
• ³⁴
  Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, Wilson MC, Richardson WS. Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group. JAMA. 2000;284(10):1290-6.

Publication Dates

• Publication in this collection
  02 Dec 2019
• Date of issue
  Nov-Dec 2019

History

• Received
  27 Jan 2019
• Accepted
  24 Mar 2019