The quality of guidelines for treatment of carotid artery disease: a critical appraisal using the AGREE II instrument

Abstract

Clinical Practice Guidelines (CPG) are structured recommendations based on systematic reviews of the available evidence and are useful tools to support clinical decision-making. However, studies have raised concerns about the methodological and scientific quality of several CPG, which can affect their application in clinical practice. The objective of this study was to perform a systematic appraisal of the methodological quality of carotid atherosclerotic disease clinical guidelines, published from 2000 to 2019, using the AGREE II instrument (Appraisal of Guidelines for Research and Evaluation Instrument II). The appraisers independently assessed the quality of the CPG included in the study for each of the 6 domains of the AGREE II tool. The CPG were rated as high, moderate, or low quality using a points scale. A total of 9 CPGs were selected for appraisal. Except for domain 2 (kappa=0.715), excellent agreement was observed between the appraisers (kappa>0.75). Five of the CPGs were rated as high overall methodological quality rating, 5 were rated as moderate overall methodological quality, and 2 were rated low overall methodological quality. The authors conclude that: (1) appraisal of carotid atherosclerotic disease clinical guidelines using the AGREE II instrument is feasible, with a high degree of agreement among appraisers; and (2) that most CPGs on the management of atherosclerotic carotid disease have high methodological quality.

Keywords:
practice guideline; carotid artery diseases; atherosclerosis; carotid stenosis

Resumo

Diretrizes clínicas (DCs) são recomendações estruturadas baseadas na revisão sistemática da evidência disponível, sendo ferramentas úteis na tomada de decisões clínicas. Entretanto, estudos têm levantado preocupação quanto à qualidade metodológica e científica de várias DCs, que podem afetar sua aplicação na prática clínica. O objetivo do presente estudo foi fazer uma avaliação sistemática da qualidade metodológica das DCs que abordam o tratamento da doença arterial obstrutiva carotídea, publicadas entre 2000 e 2019, utilizando a ferramenta AGREE II (Appraisal of Guidelines Research and Evaluation Instrument II). Os pesquisadores avaliaram independentemente a qualidade das DCs incluídas no estudo em cada um dos seis domínios da ferramenta AGREE II. Por meio de um sistema de pontuação, as DCs foram classificadas em alta, moderada e baixa qualidade. Um total de nove DCs foram selecionadas. Exceto pelo domínio dois (Kappa = 0,715), houve concordância excelente entre os três avaliadores (Kappa > 0,75). Considerando-se a avaliação global da qualidade metodológica das DCs, cinco foram consideradas de alta qualidade (55%), duas foram consideradas de qualidade moderada e duas foram consideradas de baixa qualidade. Concluímos que (1) foi factível a utilização da AGREE II para a avaliação de DCs sobre o tratamento da doença arterial obstrutiva carotídea com alto grau de concordância inter-avaliadores; e que (2) a maioria das DCs disponíveis sobre o tratamento da doença arterial obstrutiva carotídea tem alta qualidade metodológica.

Palavras-chave:
guia de prática clínica; doenças das artérias carótidas; aterosclerose; estenose das carótidas

INTRODUCTION

Clinical practice guidelines (CPGs) are structured recommendations developed to support healthcare professionals in decision-making for individual patients in specific circumstances, based on a systematic review of the available evidence and on the risks and benefits of the available therapeutic options.^1,2 Healthcare professionals, managers, and healthcare financers see CPGs as tools that can close the gap between healthcare practice and the scientific evidence yielded by clinical trials conducted in controlled settings.^3,4 Therefore, CPGs based on the best available scientific evidence provide a basis for clinical decision-making taking into account each patient’s individual clinical characteristics and also support healthcare managers tasked with regulating healthcare systems.⁵ However, several different studies have identified CPGs of low to moderate scientific methodological quality, raising concerns among the healthcare professionals who apply them in their decision-making.^6,7

Several tools for assessment of CPGs quality have been developed.^8,9 The Appraisal of Guidelines for Research and Evaluation Instrument (AGREE)¹⁰ and its second version (AGREE II), published in 2009,¹¹ have gained recognition and have been validated and are widely used in several different languages,^12,13,14 including Portuguese.¹⁴ The AGREE II¹¹ instrument has been used in Brazil with increasing frequency over recent years and is used by the country’s Ministry of Health as part of its process of CPGs development,¹⁵ for assessment of CPGs for treatment of non-transmissible diseases¹⁶ and to support CPGs development by specialist physicians from a variety of specialties.^17,18

The AGREE II tool¹¹ is designed for: (1) healthcare professionals, who can use it to assess a guideline before adopting its recommendations in clinical practice; (2) for guideline writers, so that they can employ a structured and rigorous development methodology; (3) for those responsible for managing healthcare policies, to enable them to decide which CPGs can be used to support healthcare policy decision-making; and (4) for educators, to help them improve critical assessment skills and emphasize which competencies are essential to CPGs development to ensure that they can be used to support clinical decision-making. A total of 33 official translations of the AGREE¹⁰ and AGREE II instruments¹¹ are available for use by the international community.¹¹

Extracardiac vascular diseases have a high prevalence in the global population over the age of 60 years. Extracranial obstructive carotid artery disease (CAD) is a condition in which the great majority of surgical interventions are conducted in individuals who are entirely asymptomatic.¹⁹ Considering that this condition has zero impact on the lives of asymptomatic individuals and that there are no interventions, whether surgical or drug-based, that involve zero risk (or cost), a systematic review of the quality of the CPGs that guide treatment of this disease is justified, with the objective of supporting both physicians in their decision-making and healthcare system managers and financers who need to assess whether available resources are being employed rationally.

Anatomically, CAD is characterized by stenosis or occlusion of the carotid artery that, in the majority of cases, is secondary to atherosclerotic processes that primarily affect the carotid bifurcation, and it is responsible for approximately 20% of cases of ischemic stroke (IS).²⁰ Treatment of CAD consists of drug-based clinical treatment, with rigorous clinical control of associated diseases (systemic arterial hypertension, diabetes mellitus, dyslipidemia, and smoking) and of surgical interventions in selected cases (carotid endarterectomy or angioplasty with stenting).^19,20 For symptomatic patients, there is consensus on surgical treatment for secondary prevention for a repeat IS; whereas surgical treatment of stenosis remains controversial in cases in which patients diagnosed with CAD are asymptomatic.²⁰

This study is a systematic appraisal of the methodological quality of CPGs that cover treatment of CAD, using the AGREE II tool.

METHODS

Searches were run on two electronic bibliographic databases (PubMed/MEDLINE and SciELO) and on Google Scholar. The following keywords were used: “carotid artery disease”, “atherosclerotic”, and “practice guideline”. Publications classified as CPG or as consensus statements covering treatment of CAD and published from 2000 to 2019 were included. Publications in languages other than English, Portuguese, or Spanish were excluded. Two appraisers conducted the initial screening of studies for inclusion and selection of the CPGs for analysis by reading titles and abstracts.

Three appraisers (SGBC, JCR, and LPC) independently assessed the CPGs included in the review scoring them from 1 (low quality) to 7 (high quality), for each of the 23 items across the six domains of the AGREE II tool: 1) scope and purpose; 2) stakeholder involvement; 3) rigor of development; 4) clarity of presentation; 5 applicability; and 6) editorial independence; (Table 1). The following formula¹² was used to generate a weighted score (as a percentage) for each domain: (score awarded - minimum possible score)/(maximum possible score - minimum possible score) × 100. All assessments and ratings were performed as described in the AGREE II user’s manual, available on the AGREE Research Trust website.¹¹

Table 1
Items and domains of the AGREE II instrument (Appraisal of Guidelines for Research and Evaluation Instrument II)^*.

Item	Content	Domain
1	The overall objective(s) of the guideline is (are) specifically described.	Scope and purpose
2	The health question(s) covered by the guideline is (are) specifically described.
3	The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.
4	The guideline development group includes individuals from all the relevant professional groups.	Stakeholder involvement
5	The views and preferences of the target population (patients, public, etc.) have been sought.
6	The target users of the guideline are clearly defined.
7	Systematic methods were used to search for evidence.	Rigor of development
8	The criteria for selecting the evidence are clearly described.
9	The strengths and limitations of the body of evidence are clearly described.
10	The methods for formulating the recommendations are clearly described.
11	The health benefits, side effects, and risks have been considered in formulating the recommendations.
12	There is an explicit link between the recommendations and the supporting evidence.
13	The guideline has been externally reviewed by experts
14	A procedure for updating the guideline is provided.
15	The recommendations are specific and unambiguous.	Clarity of presentation
16	The different options for management of the condition or health issue are clearly presented.
17	Key recommendations are easily identifiable
18	The guideline describes facilitators and barriers to its application.	Applicability
19	The guideline provides advice and/or tools on how the recommendations can be put into practice.
20	The potential resource implications of applying the recommendations have been considered.
21	The guideline presents monitoring and/ or auditing criteria.
22	The views of the funding body have not influenced the content of the guideline.	Editorial independence
23	Competing interests of guideline development group members have been recorded and addressed.

• ^*
  Extracted from: AGREE Next Steps Consortium.11

The degree of agreement between appraisers was calculated using the Fleiss Kappa coefficient of agreement.²¹ As recommended by Landis and Koch,²² Fleiss Kappa values ≥ 0.75 were defined as representation excellent agreement between appraisers. Domains for which the Fleiss Kappa value was < 0.75 would be discussed and reviewed in a meeting of all three appraisers, regardless of individual scores.

The overall methodological quality of each CPG was rated using the metric proposed by Molino et al.,¹⁶ which prioritizes domain 3, since this is domain assessing the CPG’s methodological rigor of development. The result for domain 3 was therefore used to categorize the CPG as “high”, “moderate”, or “low” quality. They were then subclassified as A, B, or C by analysis of the next two best performing domains according to the AGREE II tool (Figure 1).

Figure 1
Metric used for methodological quality rating of each guideline (extracted from Molino et al.¹⁶). AGREE II: Appraisal of Guidelines for Research and Evaluation Instrument II.

RESULTS

Nine CPGs^23-31 were identified and listed by year of publication (Table 2). Eight CPGs were published in English, found in the PubMed/MEDLINE search, and one was written in Portuguese, identified by Google Scholar. The majority of the CPGs identified were published since 2010 (66%), i.e., in the second half of the period stipulated for the search.

Table 2
General information on the nine clinical guidelines.

NO.	Country	Title	Year of publication
1	International (European)	2017 ESC Guidelines on the diagnosis and treatment of peripheral arterial diseases, in collaboration with the European Society for Vascular Surgery.	201723
2	International (European)	Management of atherosclerotic carotid and vertebral artery disease: 2017 clinical practice guidelines of the European Society for Vascular Surgery.	201724
3	Brazil	Doença carotídea extracraniana. Diagnóstico e tratamento.	201525
4	United States	ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS. Guideline on the Management of Patients With Extracranial Carotid and Vertebral Artery Disease: Executive Summary.	201126
5	United States	Updated Society for Vascular Surgery guidelines for management of extracranial carotid disease.	201127
6	United Kingdom	Carotid artery stent placement for asymptomatic extracranial carotid stenosis.	201128
7	United States	Management of carotid atherosclerotic disease: Clinical practice guidelines of the Society for Vascular Surgery.	200829
8	United States	Primary prevention of ischemic stroke.	200630
9	United Kingdom	Guidelines on the management of secondary prophylaxis of vascular events in stable patients in primary care.	200431

The initial scores for each domain and the initial overall assessment for the nine CPGs assessed are shown in Table 3. Except for domain 2 (stakeholder involvement), there was excellent agreement between appraisers. The three appraisers therefore held a meeting to discuss and possibly revise the scores awarded to the CPG for domain 2. After this discussion meeting, each appraiser was completely free to revise (or not) their original scores, and then the coefficient of agreement was recalculated. The final result was then rated as excellent inter-appraiser agreement (excellent or total agreement) (Table 4).

Table 3
Initial scores in the six domains for the nine clinical guidelines assessed with the AGREE II tool (Appraisal of Guidelines for Research and Evaluation Instrument II).

CPG (year)	Domain 1 Scope and purpose*	Domain 2 Stakeholder involvement*	Domain 3 Rigor of development*	Domain 4 Clarity of presentation*	Domain 5 Applicability*	Domain 6 Editorial independence*
1 (2017)	100.00	80.00	98.61	100.00	84.72	100.00
2 (2017)	100.00	100.00	91.60	100.00	91.60	100.00
3 (2015)	72.22	3.70	16.66	78.18	19.44	19.44
4 (2011)	98.14	96.29	96.52	100.00	80.00	80.00
5 (2011)	77.77	88.88	77.77	100.00	68.05	72.22
6 (2011)	100.00	96.29	92.36	98.14	72.22	88.88
7 (2008)	100.00	77.77	83.33	100.00	58.33	94.44
8 (2006)	83.33	75.92	75.69	92.59	66.66	91.66
9 (2004)	98.14	79.62	56.94	96.29	58.33	69.44
Fleiss Kappa **	0.962	0.715	0.793	0.928	0.889	0.803

• CPG: clinical practice guideline.
• ^*
  Scores as percentages;
• ^**
  Fleiss Kappa coefficient of agreement.

Table 4
Final scores (after meeting of appraisers) for the six domains and overall methodological quality rating for the nine clinical guidelines, using the AGREE II tool (Appraisal of Guidelines for Research and Evaluation Instrument II).

CPG (year)	Domain 1 Scope and purpose*	Domain 2 Stakeholder involvement*	Domain 3 Rigor of development*	Domain 4 Clarity of presentation*	Domain 5 Applicability*	Domain 6 Editorial independence*	Overall methodological quality rating **
1 (2017)	100.00	80.00	98.61	100.00	84.72	100.00	HIGH - A
2 (2017)	100.00	100.00	91.60	100.00	91.60	100.00	HIGH - A
3 (2015)	72.22	1.80	16.66	78.18	19.44	19.44	LOW - A
4 (2011)	98.14	96.29	96.52	100.00	80.00	80.00	HIGH - A
5 (2011)	77.77	88.88	77.77	100.00	68.05	72.22	MODERATE - A
6 (2011)	100.00	96.29	92.36	98.14	72.22	88.88	HIGH - A
7 (2008)	100.00	77.77	83.33	100.00	58.33	94.44	HIGH - A
8 (2006)	83.33	57.40	75.69	92.59	66.66	91.66	MODERATE - A
9 (2004)	98.14	74.10	56.94	96.29	58.33	69.44	LOW - A
kappa ***	0.962	0.813	0.793	0.928	0.889	0.803	-

• CPG: clinical practice guideline.
• ^*
  Scores as percentages;
• ^**
  Overall methodological quality rating using metric proposed by Molino et al.16;
• ^***
  Fleiss Kappa coefficient of agreement.

According to the overall methodological quality assessment for the CPGs using the metric proposed by Molino et al.,¹⁶ five guidelines were rated high quality (HIGH A), which corresponds to 55% of the CPGs assessed; two guidelines were rated moderate quality (MODERATE A); and two guidelines were rated low quality (LOW A) (Table 4).

DISCUSSION

With regard to the overall methodological quality ratings for these CPGs, if we only consider those published in English and available on PubMed/MEDLINE, 62.5% were rated high quality (all HIGH A). With regard to the negative assessment of the overall methodological quality of the only CPG available in Portuguese, as was also found in quality assessments of CPGs on cardiac diseases³² and other chronic diseases,^16,33 there is ample room for improving methodological quality using instruments such as the AGREE II tool during the design phase. Equally important is the evident need for those reading CPGs and treating physicians to also employ easy-to-use tools such as the AGREE II during their critical reading of the CPGs that they use to support their patient management decisions.³²

Hoffmann-Eßer et al.³⁴ conducted an online survey of 376 researchers, finding that the domains that had greatest impact on the overall methodological quality rating of a CPG are domains 3 (rigor of development) and six (editorial independence). In our study, the two CPG that were rated low quality also had the worst scores in these two domains. These findings underscore the importance of domain 3, which has the highest number of items (eight), and of domain 6 (despite only having two items).

It is presumed that the year of publication of the ninth CPG (2004), 1 year after the first version of the AGREE instrument was published,¹⁰ is a relevant factor in its low overall methodological quality, since the AGREE tool was still not widely used then. Although the third CPG is more recent, its design may not have taken the AGREE II tool¹¹ or other similar instruments^8,9 into consideration.

One drawback with regard to use of the AGREE II tool is the wide range of variation (1 to 7) in the scale appraisers use to rate each of the 23 items, which can lead to some degree of subjectivity on the part of each appraiser.³² However, the high degree of inter-appraiser agreement we observed shows that the AGREE II instrument can overcome this possible bias. Even in the only domain in which discussion and re-rating were conducted (domain 2), the degree of initial agreement was already close to the cutoff for excellent (an initial Fleiss Kappa value of 0.715). This is consistent with studies that have confirmed the validity of the AGREE II as a tool for CPGs assessment in a variety of medical specialties, even when there is a degree of heterogeneity among appraisers.³⁵

Certain criticisms of the applicability of the AGREE II instrument to CPGs for surgical diseases were made in a recent protocol proposal,³⁶ which highlighted the difficulties of dealing with cost-effectiveness due to several different reasons: because it is an item rarely covered in the surgical literature, because of the variability of surgical experience in different countries, because of the need for application of the instrument in different healthcare settings, and because of the need for common comparators to cover the details of more complex interventions. This protocol therefore proposed an extension of the AGREE instrument for evaluation of CPGs for surgical diseases, with inclusion of a checklist of specific items related to their specific characteristics. The objective of developing this extension would therefore be to expand the instrument’s applicability and increase its value for CPGs for clinical practice in surgery.

One limitation of the present study was the participation of only a specific subset of appraisers. The three appraisers were physicians, vascular surgery specialists, whose main activity is patient care, with no participation by other appraisers, such as management, educators, and/or CPG writers. There was also no participation by treating physicians from other specialties who take part in the decision-making process for patients with CAD (p. ex., neurologists and cardiologists). Another drawback is that the AGREE II instrument was developed to assess the methodological quality of CPGs development and is not appropriate for analysis specifically of the merit of their content, as already pointed out by other authors who have used the instrument to assess the quality of CPGs focused on other vascular diseases.³⁷

CONCLUSIONS

On the basis of our findings, the following can be concluded: (1) it proved feasible to employ the AGREE II to assess CPGs on treatment of CAD with a high degree of agreement between appraisers; and (2) the majority of CPGs on treatment of CAD published in English are of high overall methodological quality.

• How to cite: Coutinho SGB, Ricardo JC, Coutinho AIM, Cavalcante LP. The quality of guidelines for treatment of carotid artery disease: a critical appraisal using the AGREE II instrument. J Vasc Bras. 2022;21:e20220032. https://doi.org/10.1590/1677-5449.202200322
• Financial support: None.
• The study was carried out at [Programa de Pós-Graduação em Cirurgia (PPGRACI), Universidade Federal do Amazonas (FM/UFAM), Manaus, AM, Brazil.]

REFERÊNCIAS

• ¹
  Field MJ, Lohr KN. Clinical practice guidelines: directions for a new program. Washington: National Academies Press; 1990. [citado 2022 maio 10]. https://nap.nationalacademies.org/catalog/1626/clinical-practice-guidelines-directions-for-a-new-program
  » https://nap.nationalacademies.org/catalog/1626/clinical-practice-guidelines-directions-for-a-new-program
• ²
  Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Clinical practice guidelines we can trust. Washington: National Academies Press; 2011. http://dx.doi.org/10.17226/13058
  » http://dx.doi.org/10.17226/13058
• ³
  Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527-30. http://dx.doi.org/10.1136/bmj.318.7182.527 PMid:10024268.
  » http://dx.doi.org/10.1136/bmj.318.7182.527
• ⁴
  Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001;39(8):II46-54. http://dx.doi.org/10.1097/00005650-200108002-00003 PMid:11583121.
  » http://dx.doi.org/10.1097/00005650-200108002-00003
• ⁵
  Reis ECD, Passos SRL, Santos MABD. Quality assessment of clinical guidelines for the treatment of obesity in adults: application of the AGREE II instrument. Cad Saúde Pública. 2018;34(6):e00050517. http://dx.doi.org/10.1590/0102-311x00050517 PMid:29952398.
  » http://dx.doi.org/10.1590/0102-311x00050517
• ⁶
  Al-Ansary LA, Tricco AC, Adi Y, et al. A systematic review of recent clinical practice guidelines on the diagnosis, assessment and management of hypertension. PLoS One. 2013;8(1):e53744. http://dx.doi.org/10.1371/journal.pone.0053744 PMid:23349738.
  » http://dx.doi.org/10.1371/journal.pone.0053744
• ⁷
  Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet institute of medicine standards: two more decades of little, if any, progress. Arch Intern Med. 2012;172(21):1628-33. http://dx.doi.org/10.1001/2013.jamainternmed.56 PMid:23089902.
  » http://dx.doi.org/10.1001/2013.jamainternmed.56
• ⁸
  Vlayen J, Aertgeerts B, Hannes K, Sermeus W, Ramaekers D. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. Int J Qual Health Care. 2005;17(3):235-42. http://dx.doi.org/10.1093/intqhc/mzi027 PMid:15743883.
  » http://dx.doi.org/10.1093/intqhc/mzi027
• ⁹
  Siering U, Eikermann M, Hausner E, Hoffmann-Eßer W, Neugebauer EA. Appraisal tools for clinical practice guidelines: a systematic review. PLoS One. 2013;8(12):e82915. http://dx.doi.org/10.1371/journal.pone.0082915 PMid:24349397.
  » http://dx.doi.org/10.1371/journal.pone.0082915
• ¹⁰
  AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1):18-23. http://dx.doi.org/10.1136/qhc.12.1.18 PMid:12571340.
  » http://dx.doi.org/10.1136/qhc.12.1.18
• ¹¹
  AGREE Next Steps Consortium [site na Internet]. Ottawa: Canadian Institutes of Health Research; 2009. [citado 2022 jan 02]. http://www.agreetrust.org
  » http://www.agreetrust.org
• ¹²
  Alonso-Coello P, Irfan A, Solà I, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):e58. PMid:21127089.
• ¹³
  Legido-Quigley H, Panteli D, Brusamento S, et al. Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health Policy. 2012;107(2-3):146-56. http://dx.doi.org/10.1016/j.healthpol.2012.08.004 PMid:22939646.
  » http://dx.doi.org/10.1016/j.healthpol.2012.08.004
• ¹⁴
  Khan GS, Stein AT. Adaptação transcultural do instrumento Appraisal of Guidelines for Research & Evaluation II (AGREE II) para avaliação de diretrizes clínicas. Cad Saúde Pública. 2014;30(5):1111-4. http://dx.doi.org/10.1590/0102-311X00174912 PMid:24936826.
  » http://dx.doi.org/10.1590/0102-311X00174912
• ¹⁵
  Ronsoni RM, Pereira CC, Stein AT, Osanai MH, Machado CJ. Avaliação de oito Protocolos Clínicos e Diretrizes Terapêuticas (PCDT) do Ministério da Saúde por meio do instrumento AGREE II: um estudo piloto. Cad Saúde Pública. 2015;31(6):1157-62. http://dx.doi.org/10.1590/0102-311X00118814 PMid:26200364.
  » http://dx.doi.org/10.1590/0102-311X00118814
• ¹⁶
  Molino CG, Romano-Lieber NS, Ribeiro E, Melo DO. Non-communicable disease clinical practice guidelines in Brazil: a systematic assessment of methodological quality and transparency. PLoS One. 2016;11(11):e0166367. http://dx.doi.org/10.1371/journal.pone.0166367 PMid:27846245.
  » http://dx.doi.org/10.1371/journal.pone.0166367
• ¹⁷
  Cotrim HP, Parise ER, Figueiredo-Mendes C, Galizzi-Filho J, Porta G, Oliveira CP. Nonalcoholic fatty liver disease Brazilian Society of Hepatology Consensus. Arq Gastroenterol. 2016;53(2):118-22. http://dx.doi.org/10.1590/S0004-28032016000200013 PMid:27305420.
  » http://dx.doi.org/10.1590/S0004-28032016000200013
• ¹⁸
  Westphal GA, Garcia VD, Souza RL, et al. Guidelines for the assessment and acceptance of potential brain-dead organ donors. Rev Bras Ter Intensiva. 2016;28(3):220-55. http://dx.doi.org/10.5935/0103-507X.20160049 PMid:27737418.
  » http://dx.doi.org/10.5935/0103-507X.20160049
• ¹⁹
  Abbott AL. Medical (nonsurgical) intervention alone is now best for prevention of stroke associated with asymptomatic severe carotid stenosis: results of a systematic review and analysis. Stroke. 2009;40(10):e573-83. http://dx.doi.org/10.1161/STROKEAHA.109.556068 PMid:19696421.
  » http://dx.doi.org/10.1161/STROKEAHA.109.556068
• ²⁰
  Sitrângulo CJ, Silva ES. Carotid atherosclerotic disease. J Vasc Bras. 2018;17(3):179-83. PMid:30643502.
• ²¹
  Fleiss JL, Levin B, Paik MC. Statistical method for rates and proportions. 3a ed. Hoboken: John Wiley & Sons; 2003. The measurement of interrater agreement, p. 598-626. http://dx.doi.org/10.1002/0471445428.ch18
  » http://dx.doi.org/10.1002/0471445428.ch18
• ²²
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-74. http://dx.doi.org/10.2307/2529310 PMid:843571.
  » http://dx.doi.org/10.2307/2529310
• ²³
  Aboyans V, Ricco JB, Bartelink MEL, et al. 2017 ESC Guidelines on the Diagnosis and Treatment of Peripheral Arterial Diseases, in collaboration with the European Society for Vascular Surgery (ESVS): document covering atherosclerotic disease of extracranial carotid and vertebral, mesenteric, renal, upper and lower extremity arteries. Endorsed by: the European Stroke Organization (ESO) the Task Force for the Diagnosis and Treatment of Peripheral Arterial Diseases of the European Society of Cardiology (ESC) and of the European Society for Vascular Surgery (ESVS). Eur Heart J. 2018;39(9):763-816. http://dx.doi.org/10.1093/eurheartj/ehx095 PMid:28886620.
  » http://dx.doi.org/10.1093/eurheartj/ehx095
• ²⁴
  Naylor AR, Ricco JB, Borst GJ, et al. Management of atherosclerotic carotid and vertebral artery disease: 2017 clinical practice guidelines of the European Society for Vascular Surgery (ESVS). Eur J Vasc Endovasc Surg. 2018;55(1):3-81. http://dx.doi.org/10.1016/j.ejvs.2017.06.021 PMid:28851594.
  » http://dx.doi.org/10.1016/j.ejvs.2017.06.021
• ²⁵
  Presti C, Miranda FM Jr, Silva JCCB, et al. Doença carotídea extracraniana - diagnóstico e tratamento. São Paulo: Sociedade Brasileira de Angiologia e de Cirurgia Vascular; 2015. [citado 2022 julho 12]. https://sbacvsp.com.br/wp-content/uploads/2016/05/doenca-carotidea-extracraniana.pdf
  » https://sbacvsp.com.br/wp-content/uploads/2016/05/doenca-carotidea-extracraniana.pdf
• ²⁶
  Brott TG, Halperin JL, Abbara S, et al. 2011ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease: executive summary. Stroke. 2011;42(8):e420-63. PMid:21282494.
• ²⁷
  Ricotta JJ, Aburahma A, Ascher E, Eskandari M, Faries P, Lal BK. Updated Society for Vascular Surgery guidelines for management of extracranial carotid disease. J Vasc Surg. 2011;54(3):e1-31. http://dx.doi.org/10.1016/j.jvs.2011.07.031 PMid:21889701.
  » http://dx.doi.org/10.1016/j.jvs.2011.07.031
• ²⁸
  National Institute for Health and Clinical Excellence - NICE. Carotid artery stent placement for asymptomatic extracranial carotid stenosis. Londres: NICE; 2011. [citado 2022 julho 12]. https://www.nice.org.uk/guidance/ipg388
  » https://www.nice.org.uk/guidance/ipg388
• ²⁹
  Hobson RW 2nd, Mackey WC, Ascher E, et al. Management of atherosclerotic carotid artery disease: clinical practice guidelines of the Society for Vascular Surgery. J Vasc Surg. 2008;48(2):480-6. http://dx.doi.org/10.1016/j.jvs.2008.05.036 PMid:18644494.
  » http://dx.doi.org/10.1016/j.jvs.2008.05.036
• ³⁰
  Goldstein LB, Adams R, Alberts MJ, et al. Primary prevention of ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council: cosponsored by the Atherosclerotic Peripheral Vascular Disease Interdisciplinary Working Group; Cardiovascular Nursing Council; Clinical Cardiology Council; Nutrition, Physical Activity, and Metabolism Council; and the Quality of Care and Outcomes Research Interdisciplinary Working Group: the American Academy of Neurology affirms the value of this guideline. Stroke. 2006;37(6):1583-633. http://dx.doi.org/10.1161/01.STR.0000223048.70103.F1 PMid:16675728.
  » http://dx.doi.org/10.1161/01.STR.0000223048.70103.F1
• ³¹
  Betteridge DJ, Belch J, Brown MM, et al. Guidelines on the management of secondary prophylaxis of vascular events in stable patients in primary care. Int J Clin Pract. 2004;58(2):153-68. http://dx.doi.org/10.1111/j.1368-5031.2004.0109.x PMid:15055864.
  » http://dx.doi.org/10.1111/j.1368-5031.2004.0109.x
• ³²
  Sabharwal S, Patel V, Nijjer SS, et al. Guidelines in cardiac clinical practice: evaluation of their methodological quality using the AGREE II instrument. J R Soc Med. 2013;106(8):315-22. http://dx.doi.org/10.1177/0141076813486261 PMid:23759888.
  » http://dx.doi.org/10.1177/0141076813486261
• ³³
  Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999;281(20):1900-5. PMid:10349893.
• ³⁴
  Hoffmann-Eßer W, Siering U, Neugebauer EAM, Brockhaus AC, McGauran N, Eikermann M. Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use. BMC Health Serv Res. 2018;18(1):143. http://dx.doi.org/10.1186/s12913-018-2954-8 PMid:29482555.
  » http://dx.doi.org/10.1186/s12913-018-2954-8
• ³⁵
  Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010;182(10):E472-8. http://dx.doi.org/10.1503/cmaj.091716 PMid:20513779.
  » http://dx.doi.org/10.1503/cmaj.091716
• ³⁶
  Antoniou GA, Mavridis D, Tsokani S, et al. Protocol of an interdisciplinary consensus project aiming to develop an AGREE II extension for guidelines in surgery. BMJ Open. 2020;10(8):e037107. http://dx.doi.org/10.1136/bmjopen-2020-037107 PMid:32784259.
  » http://dx.doi.org/10.1136/bmjopen-2020-037107
• ³⁷
  Zhang P, Lu Q, Li H, et al. The quality of guidelines for diabetic foot ulcers: A critical appraisal using the AGREE II instrument. PLoS One. 2019;14(9):e0217555. http://dx.doi.org/10.1371/journal.pone.0217555 PMid:31545806.
  » http://dx.doi.org/10.1371/journal.pone.0217555

Publication Dates

• Publication in this collection
  25 Nov 2022
• Date of issue
  2022

History

• Received
  09 Mar 2022
• Accepted
  12 July 2022