Systematic review versus internet search: considerations about availability and reliability of medical information regarding pregnancy in women with multiple sclerosis

Revisão sistemática versus busca na internet: considerações sobre a disponibilidade e acurácia das informações médicas referentes à gravidez em portadoras de esclerose múltipla

ORIGINAL ARTICLE

Systematic review versus internet search: considerations about availability and reliability of medical information regarding pregnancy in women with multiple sclerosis

Revisão sistemática versus busca na internet: considerações sobre a disponibilidade e acurácia das informações médicas referentes à gravidez em portadoras de esclerose múltipla

Yara Dadalti Fragoso^I; Suely Dadalti Fragoso^II; Alessandro Finkelsztejn^III; Joseph Bruno Bidin Brooks^IV; Rebeca Recupero Rebs^V

^IDepartment of Neurology, Universidade Metropolitana de Santos, Santos, SP, Brazil

^IIDepartments of Communication and Design, Universidade Federal do Rio Grande do Sul, Porto Alegre, RS, Brazil; Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq, Brasilia, DF, Brazil

^IIIDepartment of Neurology, Hospital de Clinicas de Porto Alegre, Porto Alegre, RS, Brazil

^IVDepartments of Physiology and of Neurology, Universidade Metropolitana de Santos, SP, Brazil

^VDepartment of Communication, Universidade do Vale do Rio dos Sinos, São Leopoldo, RS, Brazil

^{Correspondence to}

ABSTRACT

OBJECTIVE: To compare the results from a systematic medical review and meta-analysis on the subject of pregnancy and multiple sclerosis (MS) with an internet review based on lay sites conducted by experts in computer-mediated communication, on the same subject.

METHODS: The two reviews were carried out independently in English. Three medical doctors with experience in systematic reviews and two communication experts with experience in internet searching tools performed the study.

RESULTS: Data from the medical systematic review showed some differences from that of highly accessed websites.

CONCLUSION: Evidence-based studies conducted under the strictest rules for careful systematic reviews and meta-analyses should be available with open access, i.e. accessible without payment of a fee, thereby enabling worldwide knowledge on matters of great interest to healthcare providers and patients.

Keywords: Multiple sclerosis. Pregnancy. Systematic review. Meta-analysis. Internet.

RESUMO

OBJETIVO: Comparar os resultados de uma revisão sistemática médica com meta-análise sobre o tema gravidez em esclerose múltipla (EM) aos resultados de uma revisão leiga, do mesmo assunto, baseada na Internet e conduzida por experts em comunicação mediada por computador.

MÉTODOS: As duas revisões foram realizadas independentemente em inglês. Três médicos com experiência em revisões sistemáticas e duas experts em comunicação com experiência na busca de dados nas ferramentas da Internet realizaram este estudo.

RESULTADOS: Os dados da revisão sistemática mostraram algumas diferenças daqueles que podem ser obtidos em websites de alto acesso.

CONCLUSÃO: Estudos com dados baseado em evidência, realizados sob criteriosas regras válidas para revisão sistemática e meta-análise devem ser disponibilizados de forma aberta, ou seja, permitindo acesso sem pagamento de taxas, fazendo com que o conhecimento de assuntos de grande interesse a profissionais de saúde e pacientes se globalize.

Palavras-chave: Esclerose múltipla. Gravidez. Revisão sistemática. Meta-análise. Internet.

Introduction

Life in the 21^st century gives doctors and patients an entirely new perspective regarding medical information: while extensive medical databases are continuously updated with published papers, an immense amount of unfiltered information is also displayed on the internet. More than 80% of American internet users have been reported to search for health information online, most often searching for information about a specific disease or health problem¹. Doctor-patient discussions about the latest web trend for a diagnosis or treatment often consume a lot of time in medical consultations², and some patients are now recognized as having the "internet printout syndrome" or "cyberchondria"³.

The concept of evidence-based medicine took shape thirty years ago, when Cochrane criticized the lack of organized and periodically updated summaries of randomized clinical trials⁴. Evidence-based medicine is defined as the integration of best research evidence with clinical expertise and patient value, and it is the state of the art regarding medical recommendations⁵. Few medical doctors engage in systematically reviewing published papers on a given subject, with the aim of producing solid evidence on which to base medical recommendations⁶. Those who do this know how much time and expense is involved in the task: the recommendations are very strict^7-9, demanding months of work, and access to many crucial papers is often only available on a charged-for basis. When researchers do not have a grant to carry out the task, or do not work at an institution with full access to all medical papers, they are likely to spend a considerable sum of money during the process of properly conducting a review. To add to the burden, if researchers want the systematic review and meta-analysis published with open access on the internet, another few thousand dollars will be required for this privilege, for papers published in many of the leading high-impact journals. If a systematic review and meta-analysis is published without open access, it will not be easily available to those who want to read it without payment. When doctors try to access the best evidence-based data from the internet, it is frustrating to find that this usually requires a subscription to a journal or at least the payment of a fee to read a certain paper. Ultimately, it may be easier just to select a few freely accessible medical papers and take some conclusions from them.

The aim of the present work was to compare the results from a systematic review and meta-analysis of the medical literature with those from an extensive and comprehensive internet review. The chosen subject was "pregnancy in multiple sclerosis" (MS).

Methods

Published and online data were used in preparing this paper; hence, no ethical approval was required.

The outcomes independently assessed by the medical systematic review and by the internet search were:

• the effect of pregnancy on the MS relapse rate;

• pregnancy complications and delivery mode;

• the rate of prematurity and low birth weight;

• the prevalence of malformations; and

• the breastfeeding index.

Search strategy

Medical systematic review

The present work followed the recommendations of the PRISMA Statement⁷. Using the PICO framework⁸, the authors independently searched for the terms "pregnancy" OR "gestation" OR "pregnant" AND "multiple sclerosis" OR "MS" in the following databases: EMBASE/Excerpta Medica, Medline, Pubmed, Scopus, Index Medicus, Biomed Central, Ebsco Fulltext, LILACS, Scielo and the Cochrane Database of Systematic Reviews. Abstracts of articles in any language that contained these words in English (in the title, key words or abstract) were independently reviewed by the authors. The inclusion criterion was to evaluate only papers presenting original work with analysis of at least one of the aforementioned outcomes among women with MS. Reference lists from these papers were read to search for other possible relevant publications. Abstracts from scientific meetings, review papers, anecdotal case reports, duplicate papers and editorials were excluded. The period of the literature search started in 1983 (the year when Poser's criteria were established for the diagnosis of MS¹⁰) and finished in December 2009. A full paper with the systematic review and meta-analysis presentation and discussion has been the subject of another submission¹¹. This paper was considered the Editor's choice and will be freely available from the journal website from June 2011.

The articles selected for the systematic review were individually read by the three medical authors, who summarized the results in an Excel^TM file for subsequent analysis using the CMA (Comprehensive Meta Analysis) software, version 2 (V.2). Since the time period of the relapses reported by different authors was variable, the final result considered in the meta-analysis was that of an average for the whole studied period of each paper.

Internet search

Internet searches were individually conducted by the two authors specializing in computer-mediated communication on the two main commercial search engines that operate independent databases, i.e. Google and Yahoo!¹², using the words "pregnancy' and "multiple sclerosis" in English. The top 100 hits of each search were taken into consideration (400 hits in total), a number that corresponds to the maximum number of hits each search engine displays on a single webpage but is much higher than the number of hits that are normally checked by most internet users¹³. Each list was normalized by eliminating clustered results (46 in total) and the first 50 unique results from each list (200 total) were individually analyzed.

Results

The diagrams of search strategies are presented in the Figure. A summary of the comparison between the results obtained from the two methods (medical review and internet search) is presented in the Table.

Medical systematic review

The active search resulted in 83 studies. Six papers with no abstract were excluded after several unsuccessful attempts to contact the journals and the authors. Out of the 77 potentially eligible studies identified by the search strategy, papers that did not conform with the inclusion criteria used for this systematic review were excluded (for example, single case reports, reviews or journal editorials). Twenty-two papers were selected for evaluation, since they fulfilled all the inclusion criteria.

Relapse rate

The risk of a relapse was assessed in the meta-analysis using data from 1,221 pregnancies. Prior to pregnancy, the risk was 0.56 relapse/year (95% confidence interval (CI): 0.40 to 0.72). During pregnancy, the yearly risk of a relapse decreased to 0.26 (95% CI: 0.19 to 0.32). After delivery, the risk increased to 0.758 (CI: 0.64 to 0.87). The significance of these changes reached p < 0.001.

Pregnancy complications

The abortion rate was found to be 27.9% (95% CI: 23.3% to 32.7%), including both miscarriages and induced abortions. Other pregnancy complications were not fully reported and consisted of anecdotal data that could not be considered for meta-analysis.

Delivery mode

The caesarean section rate was 21.4% (95% CI: 11.2% to 36.9%). None of the studies specified the reasons for caesarean section in these women.

Rate of prematurity

The prematurity rate was found to be 10.2% (95% CI: 8.80% to 11.40%). The results were very consistent in all studies.

Rate of low birth weight

The rate of low birth weight was found to be 5.7% (95% CI: 4.2% to 7.8%). The results were consistent throughout the studies.

Prevalence of malformations and neonatal death

The reported rates of unspecified malformations and neonatal deaths ranged from 1.13% to 6.25% of all births, with an average of 3.03%. Four neonatal deaths were reported among 1,081 deliveries (0.37%). Malformations and neonatal deaths were not clearly defined as being disease-related or drug-related in any of the papers.

Breastfeeding index

The event rate was 54.5% (95% CI: 38.1% to 70.0%).

Internet searches

None of the full medical papers or their summaries were read by the researchers, in order to avoid contamination or bias of the results obtained from non-medical sources. Internet searches led to 100 pages in English that were individually analyzed. Forty-six of these pages contained lay information about Multiple Sclerosis and Pregnancy that the reviewers considered relevant. The following results were obtained from pooling data from these webpages.

Relapse rate

Reduction of relapse during pregnancy was mentioned by most webpages. Internet pages in English did not include quantitative data and mentioned a "slight reduction" or "small decrease". The reviewers agreed that in most cases the texts appeared to be trying to be supportive to women with MS who were already pregnant. The results also mentioned an increase of 20 to 40% in the relapse rate during puerperium.

Pregnancy complications

In general, the webpages in English indicated that MS had no effect on pregnancy complications.

Delivery mode

According to the accessed webpages, the incidence of cesarean section was not affected by MS. One page said that women with MS were "only a little more likely to deliver babies by cesarean section" (http://www.webmd.com/multiple-sclerosis/news).

Rate of prematurity

All pages that mentioned risk of prematurity said that it was not affected by MS.

Rate of low birth weight

In general, the webpages indicated that MS did not affect birth weight.

Prevalence of malformations and neonatal death

Pages in English were divided into two groups: some affirmed that MS itself did not influence the rate of malformations, while others concentrated on drugs and recommended avoidance of corticosteroids, azathioprine, cyclophosphamide, diazepam, phenytoin, carbamazepine and baclofen (these drugs were mentioned by their commercial names).

Breastfeeding

Pages in English mentioned that some MS mothers avoided breastfeeding due to excessive tiredness. Nearly all webpages that contained information on this subject indicated the necessity to avoid certain drugs. These were most often referred to as "disease-modifying drugs", and included all of the following (brand names as cited): Avonex, Azathioprine, Baclofen, Betaseron, Carbamazepine, Corticotropin, Cyclophosphamide, Copaxone, Diazepam, Novantrone, Phenytoin, Prednisone, Rebif and Tysabri.

Discussion

Medical systematic review

Several aspects of the assessed outcomes may have been subject to geographical and/or temporal influence. For example, it is possible that abortions were more recommended at the time of the earlier stages of immunomodulatory drugs, when doctors might have been more worried about possible side effects. It is also possible that abortions in the medical review included both spontaneous and induced abortions, while the internet review may have considered only spontaneous abortions. The indication of cesarean sections and breastfeeding habits may still be subject to cultural influences. The literature on the subject does not typically include information from national birth databases, and when some information was provided, it was limited to a specific outcome for a specific country. This made it difficult to establish overall comparisons, although cesarean sections and abortions seemed to be overrepresented in the study, while prematurity, low birth weight, malformations and neonatal deaths did not seem to be excessive.

Despite all the limitations of the analysis, it is possible to conclude that there is a significant decrease in the relapse rate during pregnancy and a significant increase in the relapse rate after delivery. Likewise, it is possible to conclude that there is no alarming risk of prematurity, low birth weight, malformations or neonatal death among children born from mothers with MS. With regard to the effect of drugs on the pregnancy and/or on the child, the risks are still being assessed and no conclusions can be drawn^14-16.

Internet review

Online information gathering is always imprecise. Criteria behind major search engines' hierarchization algorithms are commercial secrets and they are known to employ exclusion protocols to avoid the inclusion of certain sites or types of content in their databases. Internet search results necessarily vary over time and can be influenced by the geographical location of the user. The searches in Google and Yahoo! were based on the same computers and separated by only a few minutes, but generated significantly different hit lists. Most internet users search with a single engine and do not check more than one to three result pages ¹³, but the most informative webpages were generally not among the highest classified. The reliability of online information sources is very difficult to judge. In the present work, the evaluation of reliability of information about MS and pregnancy was carried out by comparison. Identification of the credibility and possible biases of the social players responsible for creating and maintaining websites was also taken into consideration. Differentiation of scientific, personal and commercial sites is not straightforward even for communication experts, and proved particularly complicated in relation to medical content. Analyses of pages under the same domain revealed several cases of sponsorship by companies that produce and commercialize drugs for MS treatment. The present search and analysis procedures allow speculation that more (and probably better) results were obtained by the communication experts than would have been obtained by typical internet users performing similar searches.

Overall, the association between pregnancy and MS is approached in a very positive tone on internet pages. The most widespread information is about the reduction of relapses during pregnancy. The possibility of an increase in relapses during the puerperium is mentioned on a number of pages. Pregnancy complications, risk of premature birth, mode of delivery and risk of low birth weight were mentioned on some websites.

Several sites used strong terms to warn against the use of certain drugs during pregnancy and breastfeeding, often referring to specific drugs by their commercial names. However, recent work from different groups appears to indicate that, in most cases, drug exposure during pregnancy does not seem to negatively affect obstetric or neonatal outcomes^14-18. The absence of good quality data indicating that a drug can certainly cause malformation in children born from MS women, does not mean that it is safe for MS pregnant women to use this drug.

Conclusion

The professional power of medicine is being challenged by the public with access to information on healthcare, and investigations on the impact of this evolving technology must continue ³. Systematic reviews and meta-analyses carried out with the utmost scientific rigor should be freely available to the community, physicians and patients.

Acknowledgment: YD Fragoso and SD Fragoso are currently holders of post-doctoral research support from the Conselho Nacional de Desenvolvimento Científico e Tecnologico, CNPq, Brasilia, DF, Brazil.

Conflicts of interest: All authors have received financial support from pharmaceutical companies, private institutions and/or governmental agencies to attend conferences and congresses.

Recebido em: 05/08/11

Versão final apresentada em: 07/06/12

Aprovado em: 12/07/12

This study was short-listed for a poster-prize during the 26^th European Multiple Sclerosis Congress (ECTRIMS), in Gothenburg, Sweden; 2010.

• 1. Fox, S. and Jones, S. American's pursuit of health takes place within a widening network of both online and offline sources Pew Internet & American Life Project, June 2009. Available in http://www.pewinternet.org/~/media//Files/Reports/2009/PIP_Health_2009.pdf [Accessed on May, 20th, 2011]
• 2. Kivits J. Informed patients and the internet: a mediated context for consultations with health professionals. J Health Psychol 2006; 11: 269-82.
• 3. Powell JA, Darvell M, Gray JA. The doctor, the patient and the world-wide web: how the internet is changing healthcare. J R Soc Med 2003; 96: 74-6.
• 4. Cochrane AL. 1931 - 1971: a critical review, with particular reference to the medical profession. In Medicines for the Year 2000 London: Office of Health Economics; 1979: 1-11.
• 5. Sackett D. Evidence Based Medicine: What it is and what it is not. BMJ 1996; 312: 71.
• 6. Kaczorowski J. Standing on the shoulders of giants: Introduction to systematic reviews and meta-analyses. Can Fam Physician 2009; 55: 1155-6.
• 7. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group - Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Open Med 2009; 3: 123-30.
• 8. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak 2007; 7: 16.
• 9. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000; 283: 2008-12.
• 10. Poser CM, Paty DW, Scheinberg L, McDonald WI, Davis FA, Ebers GC et al. New diagnostic criteria for multiple sclerosis: guidelines for research protocols. Ann Neurol 1983; 13: 227-31.
• 11. Finkelsztejn A, Brooks JBB, Paschoal Jr FM, Fragoso YD. What can we really tell women with multiple sclerosis regarding pregnancy? A systematic review and meta-analysis of the literature. BJOG 2011; 118: 790-7.
• 12. Bruce Clay, Inc. The search engine relationship chart 2007. Available in http://www.bruceclay.com/searchenginerelationshipchart.htm [Accessed in Decembre 22, 2010]
• 13. Fragoso, S. Seek and Ye Shall Find. In Kelsey, S and St. Amant K, Handbook of Research on Computer Mediated Communication Vol. I. London: IGI Global; 2008. p. 740-54.
• 14. Sandberg-Wollheim M, Frank D, Goodwin TM, Giesser B, Lopez-Bresnahan M, Stam-Moraga M et al. Pregnancy outcomes during treatment with interferon beta-1a in patients with multiple sclerosis. Neurology 2005; 65: 802-6.
• 15. Waubant E, Sadovnick AD. Interferon beta babies. Neurology 2005; 65: 788-9.
• 16. Fragoso YD, Finkelsztejn A, Kaimen-Maciel DR, Grzesiuk AK, Gallina AS, Lopes J et al. Long-term use of glatiramer acetate by 11 pregnant women with multiple sclerosis: a retrospective, multicentre case series. CNS Drugs 2010; 24: 969-76.
• 17. Salminen HJ, Leggett H, Boggild M. Glatiramer acetate exposure in pregnancy: preliminary safety and birth outcomes. J Neurol 2010; 257: 2020-3.
• 18. Amato MP, Portaccio E, Ghezzi A, Hakiki B, Zipoli V, Martinelli V et al. Pregnancy and fetal outcomes after interferon-β exposure n multiple sclerosis. Neurology 2010; 75: 1794-802.

Publication Dates

• Publication in this collection
  20 Feb 2013
• Date of issue
  Dec 2012

History

• Received
  05 Aug 2011
• Accepted
  12 July 2012
• Reviewed
  07 June 2012