SHORT COMMUNICATION
Systematic reviews of medical literature and metanalyses
Álvaro Nagib Atallah
MD, PhD, MSc Department of Internal Medicine, Escola Paulista de Medicina Chairman, Editor, São Paulo Medical Journal
Systematic reviews of medical literature constitute a modern methodology with which to evaluate sets of medical data. Although they can be applied in various fields of medicine or biology, systematic reviews are used more often to obtain scientific proof of medical therapies.
As the name suggests, the researcher who is interested in discovering if one therapeutic method is better than another must always begin his own search of an already completed systematic review (SR) of the literature, and if one is not found, do it himself if possible.
Using this procedure, he will come across various possibilities requiring different reactions:
1. If a well done SR already exists and provides evidence confirming the best therapeutic treatment, the researcher can make decisions based on the same.
2. If controlled clinical trials on the subject exist, but the results are inconsistent, a metanalysis should be performed. At the end of a systematic review, a metanalysis is a summary statistic of the data from controlled studies of similar treatments and objectives.
This summary will yield final results that could be statistically significant - in favor or against the determined treatment - or be inconclusive. When results are inconclusive, or rather, do not show statistically significant differences, there are two possible conclusions:
1. One treatment is not better than another, in the case that the total number of cases studies yields a sufficient sample size and statistical power which would permit the detection of clinically relevant effects.
2. That the number of cases or events studied is insufficient to respond to the study question, and more clinical trials need to be performed.
It is not infrequent to, after exhaustively searching the medical literature and not finding even one clinical trial, to support a determined therapy that has been in pratice for decades. However, there will not be sufficient data to perform a metanalysis (there will be no data to combine); in this case, the result would be just a systematic review.
Faced with this situation, the researcher, instead of feeling frustrated, should be excited about discovering a rich new line of research, consisting of the realization of clinical trials on the subject.
SRs thus permit the researcher, in an effective manner, to distinguish efficient treatments from those that are not, to resolve treatment controversies, and to determine when treatments need updating. They also identify areas in which it is necessary to performe controlled clinical trials, the raw material of SRs, and a reference point in scientifically-founded treatment decisions.
On the other hand, for the researcher who does not want to do a review in the systematic manner, with adequate methodology, the other option is to do it the old-fashioned way, without systemization. This review would then be, very probably, incomplete, not reproducible, imprecise, and, therefore, scientifically inconclusive. An alternative to doing a SR is to undertake a great randomized clinical trial, which would cost much more and take much more time to complete.
A following issue will bring a description of how to do a systematic literature review and complete it with a metanalysis. The accompanying graph, Figure 1, is an example incorporating figures resulting from SRs, completed with metanalyses, and how to interpret them. Such graphs are the most practical manner to present and read a metanalysis.
The horizontal line represents treatment effects and is divided by a vertical line or axis that marks the zero effect, or rather, the odds or relative risk that equals the unit. Sometimes, the result of the difference in the proportion of the events in the two groups is placed on the graph; in this case; the vertical axis marks the zero effect (the clinical interpretation is the same in both cases).
To the left of the vertical axis are the results that signify a reduction in risk, or beneficial effect, of the treatment in relation to the control. To the right are the results that signify an increase in risk caused by the treatment (that is to say, the placebo group or control group had better results than the group submitted to the treatment being studied).
Each clinical trial selected to be included in the metanalysis may be placed from the top down, represented by a straight line whose length represents the confidence interval (the interval to which there is 95 percent probability that the true effect is circumscribed). In general, the study represented by the top line is of the greatest methodological quality.
When a line crosses the vertical axis, it signifies that the result is not statistically significant (although it could be clinically significant: an analysis across the line has the advantage of showing the tendency of the effect). The greater the sample size, the shorter the confidence level.
In Figure 1, Line A represents the result of the treatment whose effect is not statistically significant, notwithstanding the tendency towards the reduction in the risk of the undesirable event occurring. The dot, that is the point stimated, represents a 25 percent reduction in the number of events, but there was no statistical significance (P > 0.05).
Line B represents a study with a reduction in the number of events of 50 percent, which is statistically significant. B does not cross the vertical, consequently, it does not include 1.0.
Line C represents the results a study with a lesser sample size (notice that C is the longest of all), whose therapeutic effect was increased 25 percent in proportion to adverse events. However, this results are not statistically significant.
The symbol R represents the sum of the results of the three studies; there was a approximately a 25 percent reduction in proportion of events (beneficial effect) that is statistically significant. This result is called typical and synthesizes the results of the cases studies in the three studies, which, for being homogenous and of good quality, were included in the metanalysis.
As can be seen, this research methodology, widely accepted and implemented in the best medical schools and literature worldwide, has numerous advantages, included that it can:
1. Be reproduced as it uses clear methodology.
2. Prevent unnecessary duplication of efforts, being that, once completed, it does not need to be repeated by other groups.
3. Be quickly updated each time new clinical trials on the subject are published; if the same are of good quality, they are included in the analysis and reflected in the graph.
4. Nullify controversies in the literature, as it is not the number of favorable studies on one treatment or another that counts, but the sum of all the cases adequately studied, published or not.
5. Anticipate by various decades the results of very large studies (which still have not been done).
6. Detect harmful treatments in the early stages of their use, thus saving a great number of patients.
7. Be used in itself to improve the precision of results, by decreasing the confidence interval of the results.
8. Determine in which areas more clinical trials are needed.
With the aim of avoiding unnecessary duplication of efforts, Dr. Iain Chalmers of England, working at Oxford, has created the Cochrane Collaboration, whose objective is to perform, aid, and disseminate systematic reviews of medical treatments in many areas. The Cochrane Collaboration has been establishing Cochrane Centers in various First World countries, uniting forces so that medical decisions can be based on the best existing scientific evidence.
- 1. Chalmers I & Altman DG. Systematic Reviews. BMJ Publishy Groups. London, 1995.
- 2. Cochrane Collaboration. The Cochrane Library. BMJ Publisching Group & Update Software. London, 1996.
- 3. Antiplatelet Trialists' Collaboration. Collaborative overview of randomized therapy - I: Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients. BMJ 1994; 308;81-106.
Publication Dates
-
Publication in this collection
07 Nov 2008 -
Date of issue
Oct 1996
