A Bayesian approach to estimate the accuracy of "in-house" ELISA assay to measure rabies antibodies from compulsory vaccinated dogs and cattle

SHORT COMMUNICATION

A Bayesian approach to estimate the accuracy of "in-house" ELISA assay to measure rabies antibodies from compulsory vaccinated dogs and cattle

Nogueira AHC^I; Silva C^II; Gomes DE^II; Rosa ACG^II; Luvizotto MCR^III; Cardoso TC^II

^ISão Paulo Agency of Agribusiness Technology, APTA, Araçatuba, São Paulo State, Brazil

^IILaboratory of Virology, Department of Animal Health and Production, School of Dentistry, São Paulo State University, UNESP, Araçatuba, São Paulo State, Brazil

^IIILaboratory of Veterinary Pathology, Department of Animal Health and Production, School of Dentistry, São Paulo State University, UNESP, Araçatuba, São Paulo State, Brazil

^{Correspondence to}

ABSTRACT

Rabies is a vaccine-preventable disease that causes acute encephalitis in mammals, and it is still a significant public health problem in numerous countries. Infected dogs represent the main vectors involved in human rabies. Additionally, cattle rearing close to geographic areas where vampire bats are found presents an important connection with rural epidemiology. We applied two "in-house" enzyme-linked immunosorbent assay (ELISA) methodologies, considered alternatives to measure antibodies from vaccinated dogs and cattle, without employing the gold standard approach. The ELISA assays were performed on individual serum samples taken from domestic adult dogs and cows compulsory vaccinated against rabies (147 urban dogs and 64 cows; n = 211). The sandwich and liquid-phase competitive ELISA (scELISA and lpcELISA), considered "in-house" assays, were performed according to previous works. The only statistical methodology that allows this study is the Bayesian approach, developed to replace the conventional Hui-Walter paradigm. For conditional independent Bayesian model (one population, two tests and no gold standard) the prior information for sensitivity and specificity of each test, mode, prevalence and transformed (á, â) were submitted to Bayesian inference. The "in-house" lpcELISA revealed 16 - out of 261 serum samples - negative results, whereas in scELISA all results were positive. The Bayesian approach showed that prior information was specified for all parameters; posterior medians were Se_scELISA 89%, Sp_scELISA 88%, Sp_lpcELISA 95% Se_lpcELISA 98%, and prevalence (pi) of 99%, without the use of gold standard analysis to measure specific anti-rabies antibodies.

Key words: rabies antibody, Bayesian analysis, ELISA assay.

INTRODUCTION

Rabies is a lethal zoonotic viral infection of the central nervous system that is transmitted by the bite of a rabid animal and is capable of infecting all mammal species. In most modern laboratories, this disease is usually diagnosed through detection of viral antigens in the brain by means of a direct fluorescent antibody test (FAT) (1). Considerable progress has been achieved in rabies prophylaxis in Brazil, mainly by screening a large number of clinical samples by FAT and by increasing the vaccination programs, resulting in a marked decrease in cases of urban and rural rabies (2, 3). Tissue culture techniques have long been applied in studies related to rabies virus, and there are now a number of continuous cell lines employed in research on pathogenesis, vaccine production, rabies diagnosis, and measurement of the respective immune response (4-9). Post-assessment of the efficacy of rabies vaccination campaigns requires blood sampling of vaccinated animals to determine the respective immune response through virus neutralization (VN) antibodies detection (10). The most commonly used techniques for this purpose are serum neutralizing methods performed on mice or cell cultures (11).

Virus neutralization tests - rapid fluorescent focus inhibition assay (RFFIT) and fluorescent antibody virus neutralization (FAVN) - are the current gold standard methods, prescribed by the WHO and OIE (2, 3). RFFIT test is a sensitive and highly specific test for VN antibody response to rabies virus in serum collected from vaccinated animals (3). However, these tests are tedious and complicated to perform, making them unsuitable for large-scale epidemiologic surveillance studies (12, 13). Furthermore, these techniques require very well-trained technicians, appropriate laboratory facilities - in accordance with protocols of live rabies virus procedures to create monoclonal antibodies - and specialized microscopes to reveal signals produced by fluorescein labels (2, 3).

Initially, to overcome disadvantages of VN tests, several types of ELISA have been developed for detecting antibodies against rabies virus, using monoclonal based techniques directed to the whole viral particle or specific proteins, as glycoprotein (12-16). Usually the availability of a suitable ELISA test is an important requirement for the performance evaluation study (13). But it is difficult or sometimes next to impossible to perform a VN test that can truly identify neutralization antibodies, due to the complex laboratory structure required for cell culture (12). However, when gold standard test is not possible, the performance of two tests can be estimated using latent-class approaches, usually provided by error probability of the reference test (17, 18).

Serological evaluation is particularly suited to Bayesian framework applied in a wide spectrum of infectious diseases (18). Moreover, Bayesian analysis uses prior information knowledge about parameters of the test under study either from other similar studies or expert's best guess. In addition, it has the advantage of providing more stable point estimates and intervals without the necessity of large sample sizes (18). In conclusion, Bayesian inference is the combination of â distribution of the prior information and maximum likelihood estimates of observed data.

In this study, we analyzed results from two unofficial serological techniques described as "in-house ELISA" competitive methods - sandwich competitive ELISA (scELISA) and liquid-phase competitive ELISA (lpcELISA) - in order to measured antibodies against rabies virus after compulsory vaccination. The Bayesian model was employed to analyze test performance in the absence of gold standard assay. The parameters used to measure the accuracy were sensitivity, specificity, positive and negative predictive values.

The assays were carried out in individual serum samples obtained from domestic adult dogs and cows compulsory vaccinated against rabies (147 urban dogs aged one year old or more and 64 cows; sera were collected six months after vaccination, n = 211). Serum samples were stored at 5 ± 3°C for no more than five days. For prolonged storage, the samples were frozen at -20°C. Serum samples were inactivated at 56°C for 30 minutes. A positive control serum of dog origin titrated to 132 IU/mL, purchased from the US Centers for Disease Control (USA), stored at -20°C and diluted to 5 IU/mL, was used for all assays.

The sandwich and liquid-phase competitive ELISA (scELISA and lpcELISA), considered "in-house" assays, were performed according to previous works (14, 16). All the conditions and techniques were repeated by the same technician and the optical densities read at 490 nm on an ELISA plate reader. In Bayesian analysis prior information are often specified for the unknown parameters either from published papers or expert's best guess. The prior information of disease prevalence was conduct according to expert's best guess recommendation from Institute Pasteur, Paris, France (Table 1). Uncertain prior information is often reconstructed with â distributions. The modal value of prior information was transformed into â distribution model using Betabuster 1.0 free software from the website (http://www.epi.ucdavis.edu/diagnostictests/ ). For conditional independent Bayesian model (one population, two tests and no gold standard) the prior information for sensitivity and specificity of each test, mode, prevalence and á, â values were submitted to Bayesian inference as described in the following model:

model;

{

x[1:4] ~ dmulti(p[1:4], n)

p[1] <- pi*(Se_scELISA*Se_lpcELISA+covDp) + (1-pi)*((1-Sp_scELISA)*(1-Sp_LPCELISA )+covDn)

p[2] <- pi*(Se_scELISA*(1-Se_lpcELISA)-covDp) + (1-pi)*((1-Sp_scELISA)*Sp_lpcELISA -covDn)

p[3] <- pi*((1-Se_scELISA)*Se_lpcELISA-covDp) + (1-pi)*(Sp_scELISA*(1-Sp_lpcELISA )-covDn)

p[4] <- pi*((1-Se_scELISA)*(1-Se_lpcELISA )+covDp) + (1-pi)*(Sp_scELISA*Sp_lpcELISA+covDn)

ls <- (Se_scELISA-1)*(1-Se_lpcELISA)

us <- min(Se_scELISA,Se_lpcELISA) - Se_scELISA*Se_lpcELISA

lc <- (Sp_scELISA-1)*(1-Sp_lpcELISA)

uc <- min(Sp_scELISA,Sp_lpcELISA) - Sp_scELISA*Sp_lpcELISA

pi ~ dbeta(36.70, 2.88) ### Mode=0.95, 95% sure > 0.85

Se_scELISA ~ dbeta(48.28,12.82) ### Mode=0.85, 95% sure > 0.80

Sp_scELISA ~ dbeta(48.28, 12.82) ### Mode=0.85, 95% sure > 0.80

Se_lpcELISA ~ dbeta(42.57, 5.62) ### Mode=0.95, 95% sure > 0.80

Sp_lpcELISA ~ dbeta(42.57, 5.62) ### Mode=0.95, 95% sure > 0.80

covDn ~ dunif(lc, uc)

covDp ~ dunif(ls, us)

rhoD <- covDp / sqrt(Se_scELISA*(1-Se_scELISA)*Se_lpcELISA*(1-Se _lpcELISA))

rhoDc <- covDn / sqrt(Spfat1*(1-Sp_scELISA)*Sp_lpcELISA *(1-Sp_lpcELISA))

}

list(n=261, x=c(241,0,16,4))

list(pi=0.95, Se_scELISA=0.85, Sp_scELISA=0.85, Se_lpcELISA=0.95, Sp_lpcELISA=0.95)

The analysis was performed in WinBUGS version 1.4 (http://www.epi.ucdavis.edu/diagnostictests/) and 10,000 iterations were made for the Bayesian inference. The model was run successively and managed in batch-runs with R software to access the convergence and storage of the Markov chain Monte Carlo objects for further analysis.

Both α and β values were calculated by Betabuster 1.0 software. The Sp_scELISA and Sp_lpcELISA analysis resulted in an equal value for α and β, 48.28, with prior information of Sp 85%. In the same way, Sp_lpcELISA prior information of 95% resulted in 42.57 and 5.62 respectively for α and β values, as shown in Figure 1. The cross-classification of the results of the scELISA and lpcELISA for both populations is shown (Table 1). All scELISA positive results were also positive in lpcELISA analysis. However, 16 out of 261 samples were negative in lpcELISA and positive in scELISA assay. The posterior medians obtained with the conditional dependence model were calculated (Table 2). The model appeared to have reached convergence after 9,500 iterations according to visual checking of the Kernel density for the prevalence (pi) (Figure 2). When priors were specified for all parameters posterior medians were Se_scELISA 89%, Sp_scELISA 88%, Sp_lpcELISA 95% Se_lpcELISA 98%, and prevalence (pi) of 99% (Table 2).

Attempts were made to connect the obtained results, in particular those from RFFIT, with the protective immune response of serum-donating animals (13). In the present study, antibodies from dogs (older than one year old) and cattle (sera collected six months after first vaccination), compulsory vaccinated against rabies, were analyzed using two different "in-house" ELISA methods of which lpcELISA demonstrated close relationship with previous studies (12-16).

In fact, if quality control measures are maintained for all reagents used in a serological test for rabies, many different methods can accurately determine an immunological response to rabies vaccine (15). It has been proved that rapid fluorescent focus inhibition test can adequately evaluate antibody levels in vaccinated animals and represents the gold standard assay (11). However, it is difficult to estimate its costs when a large number of samples must be tested in vaccination campaigns. Moreover, ELISA methods do not assess protection against in vivo rabies infection, since they are incapable of measuring the immune response required for in vivo protection (10). The protective immune response against rabies virus relies on the interaction of virus-specific antibody with the virus glycoprotein (10).

Most statistical approaches applied on biological studies are based on Hui-Walter paradigm, and restrictions on the parameters need to be imposed to estimate the data performance, especially from animals of unknown infectious status (19). In recent decades, there have been enormous advances in the use of Bayesian methodology for the analysis of epidemiological data, and there are now many practical advantages in this approach. Bayesian methodology was never applied for rabies static analysis. Indeed, the specificity and sensitivity showed in the current study revealed accuracy close to 100% for lpcELISA, higher than results described before (20). Moreover, the use of prior probability distribution represents a powerful mechanism for incorporating information from previous studies that must be explored for tropical diseases in Brazil.

Furthermore, the findings of the present study indicated that:

• "In-house" lpcELISA revealed 16 negative results out of 261 serum samples, whereas all scELISA results were positive. This is significant, especially when unprotected animals may be involved in virus dissemination among homeless dogs or adult cattle.

• The model herein used appeared to have reached convergence after 9,500 iterations according to visual checking of the Kernel density for the prevalence (pi), indicating that high levels of antibodies were present among compulsory vaccinated animals.

• The Bayesian approach, applied for the first time in this study, demonstrated that prior information was specified for all parameters and posterior medians were Se

  _scELISA 89%, Sp

  _scELISA 88%, Sp

  _lpcELISA 95% Se

  _lpcELISA 98%, and prevalence (pi) of 99%, without utilization of the gold standard analysis to measure specific antibodies.

Results suggested that "in-house" ELISA can be an alternative technique to control unprotected animals by rapid measurement of antibodies with reliable results based on Bayesian analysis.

ACKNOWLEGEMENTS

This study was supported by FAPESP, CNPq and by the laboratories of pathology and virology, São Paulo State University, Brazil. The authors would like to thank the São Paulo Agency of Agribusiness Technology, APTA, for its assistance with serum samples.

Received: December 1, 2008

Accepted: December 9, 2008

Abstract published online: January 12, 2009

Full paper published online: February 28, 2009

Conflicts Of Interest: There is no conflict

Financial Source: CNPq, FAPESP.

• 1. Bingham J, Merwe MV. Distribution of rabies antigen in infected brain material: determining the reliability of different regions of the brain for the rabies fluorescent antibody test. J Virol Meth. 2002;101(1):85-94.
• ²
  World Health Organization (WHO). WHO Expert Consultation on Rabies. First Report. Geneva: WHO Press; 2004. 88 p. WHO Technical Report Series, 931. Available from: http://www.who.int/rabies/trs931_%2006_05.pdf
• ³
  World Organisation for Animal Health (OIE),  Vallat B. Manual de Pruebas de Diagnostico y Vacunas para Animales Terrestres. 5th ed. Paris: OIE; 2004.
• 4. Habel K. Habel test for potency. In: Meslin FX, Kaplan NM, Koprowski H, editors. Laboratory techniques in rabies. 4th ed. Geneva: World Health Organization; 1996. p. 369-73.
• 5. Cardoso TC, Pilz D. Wild rabies virus detection by plaque assay from naturally infected brains in different species. Vet Microbiol. 2004;103(3-4):161-7.
• 6. Cardoso TC, Queiroz da Silva LH, Albas A, Ferreira HL, Perri SHV. Rabies neutralizing antibody detection by indirect immunoperoxidase serum neutralization assay performed on chicken embryo related cell line. Mem Inst Oswaldo Cruz. 2004;99(5):531-4.
• 7. King AA. Cell culture of rabies virus. In: Meslin FX, Kaplan NM, Koprowski H, editors. Laboratory techniques in rabies. 4th ed. Geneva: World Health Organization; 1996. p. 114-30.
• 8. Bussereau F, Flamand A, Pese-Part D. Reproducible plaquing system for rabies virus in CER cells. J Virol Meth. 1982;4(4-5):277-82.
• 9. Cardoso TC, Queiroz da Silva LH, Da Silva SEL, Albas A, Pardo PE, Tanaka AH, Cossy LB, Perri SHV. Chicken embryo related (CER) cell line for quantification of rabies neutralizing antibody by fluorescent focus inhibition test. Biologicals. 2006;34(1):29-32.
• 10. Flamand A, Raux H, Gaudin Y, Ruigrok RWH. Mechanisms of rabies virus neutralization. Virology. 1993;194(1):302-13.
• 11. Cliquet F, Aubert M, Sagne L. Development of a fluorescent antibody virus neutralization test (FAVN test) for the quantification of rabies-neutralizing antibody. J Immunol Meth. 1998;212(1):79-87.
• 12. Servat A, Cliquet F. Collaborative study to evaluate a new ELISA test to monitor the effectiveness of rabies vaccination in domestic carnivores. Virus Res. 2006;120(1-2):17-27.
• 13. Briggs DJ, Smith JS, Mueller FL, Schwenke J, Davis RD, Gordon CR, Schweitzer K, Orciari LA, Yager PA, Rupprecht CE. A comparison of two serological methods detecting the immune response after rabies vaccination in dogs and cats being exported to rabies-free areas. Biologicals. 1998;26(4):347-55.
• 14. Cleaveland S, Barrat J, Barrat MJ, Selve M, Kaare M, Esterhuysen J. A rabies serosurvey of domestic dogs in rural Tanzania: results of a rapid fluorescent focus inhibition test (RFFIT) and a liquid-phase blocking ELISA used in parallel. Epidemiol Infect. 1999;123(1):157-64.
• 15. Servat A, Feyssaguet M, Blanchard I, Morize JL, Schereffer JL, Boue F, Cliquet F. A quantitative indirect ELISA to monitor the effectiveness of rabies vaccination in domestic and wild carnivores. J Immunol Meth. 2007;318(1-2):1-10.
• 16. Esterhuysen JJ, Prehaud C, Thompson GR. A liquid-phase blocking ELISA for the detection of antibodies to rabies virus. J Virol Meth. 1995; 51(1):31-42.
• 17. Johnson WO, Gastwirth JL, Pearson LM. Screening without a "Gold Standard": The Hui-Walter paradigm revisited. Am J Epidemiol. 2001;153(9):921-4.
• 18. Joseph L, Gyorkos TW, Coupal L. Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. Am J Epidemiol. 1995;141(3):263-72.
• 19. Hui SL, Walter SD. Estimating the error rates of diagnostic tests. Biometrics. 1980;36(1):167-71.
• 20. Fachin N, Carvalho BA, Cardoso TC. A comparison of serological methods for detecting the immune response after rabies vaccination in dogs and cows from rabies-endemic areas in Brazil. Intern J Appl Res Vet Med. 2005;3(3):199-206.

Publication Dates

• Publication in this collection
  19 Mar 2009
• Date of issue
  2009

History

• Accepted
  09 Dec 2008
• Received
  01 Dec 2008