Quality control assessment for the PCR diagnosis of tick-borne encephalitis virus infections

Tick-borne encephalitis virus (TBEV), the most important flaviviral infection of the central nervous system (CNS) in Europe and Russia with approximately 12,000 people affected yearly, has a significant impact on public health (Gunther and Haglund, 2005). The epidemiology of TBE is closely related to the ecology and biology of ticks. TBEV is distributed in an endemic pattern over a wide area in Europe and Asia, and is classified taxonomically into three subtypes: European, Siberian and Far Eastern (Kallio-Kokko et al., 2005, Charrel et al., 2004). The co-circulation of all three subtypes of TBEV in the same geographical region could be shown for Europe (Lundkvist et al., 2001).

In two-thirds of patients who develop CNS involvement, the disease typically takes a biphasic course (Holzmann, 2003). After a short incubation period (4–14 days), the first phase presents as an influenza-like illness often followed by a symptom-free interval of about 1 week. Within this early viraemic phase, during which no TBE antibodies are detectable, directly detecting virus by reverse transcriptase-polymerase chain reaction (RT-PCR) can be valuable for the early differential diagnosis of TBE infection in patients who present with a febrile illness following a tick bite. This is particularly true for those living in or coming from regions where more than one tick-transmitted disease is endemic (Saksida et al., 2005). The second phase of TBE is marked by signs of meningitis, meningo-encephalitis or meningoencephalomyelitis and the appearance of specific antibodies in the serum and cerebrospinal fluid (CSF). As the majority of patients come for medical attention when neurologic symptoms develop, the diagnosis of TBE is based on the detection of these specific antibodies, usually by EIA. During this second phase, TBEV itself is only rarely detectable in the blood and CSF, but TBE RT-PCR can be of great diagnostic help when the patient has not developed antibodies at this stage, has a severe case of TBE or has died after a relatively short course of infection (Gelpi et al., 2005, Schwaiger and Cassinotti, 2003).

To assess the quality of TBE diagnostics for Europe, the European Network for Diagnostics of ‘Imported’ Viral Diseases (ENIVD) [www.enivd.org] distributed 12 prepared human plasma samples to be tested for the presence of TBEV-specific RNA. The positive samples had been spiked with cell-culture derived and sequence-confirmed strains of TBEV previously described by Niedrig et al. (1994) and Ternovoi et al. (2003): K23 and Absettarov (European subtype), Aina (Siberian subtype) and Sofjin (Far Eastern subtype). As a control for sensitivity, a 10-fold dilution series of 500,000–50 RNA copies/ml was prepared using TBEV strain K23 (samples #3, #1, #4, #6 and #5, respectively). For specificity controls plasma samples were spiked with Louping ill virus (LIV) and a pool of four other flaviviruses [dengue virus (DENV), yellow fever virus (YFV), St. Louis encephalitis virus (SLEV) and Japanese encephalitis virus (JEV)] and two additional plasma samples served as negative controls. The samples were inactivated by heat and gamma irradiation, aliquoted, freeze dried and distributed as described previously (Drosten et al., 2004). Before shipping, the test panel was approved by three expert laboratories. The participants were asked to analyse the samples using the molecular methods they routinely applied to suspected human cases. Assay details, such as the RT-PCR protocols and source of primers, the type of extraction method used and suppliers and types of commercial kits used (if any), were requested. Participants were not obliged to use a particular diagnostic procedure. The results were scored for sensitivity and specificity, giving one point for a correct result independent of whether the type of virus was differentiated by sequencing or not. As nucleic acid amplification techniques (NAT) do not usually involve indeterminate endpoints and laboratories should be able to resolve unclear results by repeated testing, indeterminate results were treated as negative in positive samples and as positive in negative samples.

We first determined how many participants were able to identify each of the samples correctly (Table 1, last row). The ability to detect was diminished with decreasing concentrations of TBEV (strain K23) and many laboratories failed to detect strains of the Far Eastern subtype, the Siberian subtype and LIV. The concentration-dependent, cumulative positivity rates per sample for the 10-fold dilution series of TBEV strain K23 corresponded closely to the rates calculated by probit regression analysis, equivalent to a dose–response model (Fig. 1). This model predicts that only 50% of all test results would be correctly positive with 80 copies of virus RNA/ml of sample, and 95% with more than 350,000 copies/ml. These data clearly show a need to improve the sensitivity of the tests used. The early detection of TBEV viraemia will facilitate the rapid differentiation between serious specific infection and non-infectious illnesses in febrile patients with thrombo- and leucocytopenia (Pantanowitz, 2003, Schlossberg et al., 1996, Lotrič-Furlan and Strle, 1995). Table 1 further summarises the results obtained and gives details on the extraction and diagnostic methods used by the participating laboratories. Only 2 (9%) of 23 laboratories correctly identified all samples. Nine laboratories (39%) correctly identified between 75.0% and 91.7%, seven (30%) between 54.5% and 66.7%, and five (22%) identified 50% or less. The failure in 1 laboratory (4%) was due to a lack of sensitivity, in 7 laboratories (30%) was due to a lack of specificity (false positive and/or false negative results), and in 13 laboratories (57%) was due to a combination of both. Ten of 14 laboratories (71%) lacking sensitive procedures failed to identify the two samples weakly positive for TBEV strain K23 (#6 and #5), while 4 laboratories (29%) had serious problems with sensitivity, being unable to identify samples with >500 RNA copies/ml, well above the detection limit of published NATs. Only 7 laboratories (30%) genotyped their positive samples (Labs. 2, 8, 10, 17, 18, 25 and 34), and only laboratory 18 correctly identified the LIV-positive sample (#9). LIV is a member of the genus Flavivirus that predominantly causes encephalitis in sheep, but which, in rare cases, can also cause a disease in man that has symptoms similar to the biphasic meningitis typical of western European TBEV. From a practical standpoint it is important to know the aetiology of an infection because therapies may differ and could be better adapted depending on the nature of the disease. We therefore recommend genotyping of positive samples for improved differentiation of viruses and strains.

Table 1. Results of the EQA for the molecular detection of TBEV

	Lab. no.	Sample no.
		#3	#1	#4	#6	#5	#10	#8	#11	#9	#12	#7	#2
		TBEV European subtype1	TBEV European subtype1	TBEV European subtype1	TBEV European subtype1	TBEV European subtype1	TBEV European subtype2	TBEV Far Eastern subtype3	TBEV Siberian subtype4	Louping ill virus	DENV/YFV/SLEV/JEV	Neg.	Neg.	Correct results (%)
		500,000*	50,000*	5000*	500*	50*
	7a,B	+	+	+	+	+	+	+	+	+	−	−	−	100.0
	32a,B	+	+	+	+	+	+	+	+	+	−	−	−	100.0
	17a,A	+	+	+	+	+	+	(−)	+	+	−	−	−	91.7
	33a,A,B	+	+	+	+	+	+	+	+	+	−	−	(+)	91.7
	9a,A	+	+	+	+	+	+	(−)	(−)	+	−	−	−	83.3
	18a,A	+	+	+	+	+	+	(−)	+	+ (LIV)	(+)	−	−	83.3
	25a,A	+	+	+	+	(−)	+	(−)	+	+	−	−	−	83.3
	38a,B	+	+	+	+	+	(−)	+	+	+	(+)	−	−	83.3
	40a,B	+	+	+	(−)	(−)	+	+	+	+	−	−	−	83.3
	45a,A	+	+	+	+	+	+	+	(−)	(−)	−	−	−	83.3
	43a,A	+	+	+	+	(−)	+	(−)	+	(−)	−	−	−	75.0
	1a,B	+	+	+	(−/+)	(−)	+	(−/+)	+	+	−	−	(−/+)	66.7
	2a,A,B	+	+	+	+	(−)	+	(−)	(−)	(−)	−	−	−	66.7
	13a,B	+	+	+	+	(−)	+	(−)	(−)	(−)	−	−	−	66.7
	46a,A	+	+	+	+	(−)	+	(−)	(−)	(−)	−	−	−	66.7
	4a,B	+	+	+	+	(−)	+	(−)	(−)	(−)	−	(+)	−	58.3
	34a,A	+	+	+	(−)	(−)	+	(−)	(−)	(−)	−	−	−	58.3
	8a,A	+	+	(−)	(−)	inc.	+	(−)	(−)	(−)	−#	−	−	54.5
	10a,A	+	+	+	(−)	(−)	(−)	(−)	(−)	(−)	−#	−	−	50.0
	11a,A	+	(−)	+	(−)	(−)	+	(−)	(−)	(−)	−	−	−	50.0
	15b,A	+	+	+	+	+	(−)	+	(−)	(−)	(+)	(+)	(+)	50.0
	41b,A	(−)	+	(−)	(−)	(−)	(−)	+	(−)	(−)	−	−	(+)	33.3
	44a,A	(−/+)	(−)	(−)	(−)	(−)	(−)	(−)	(−)	(−)	−	−	−	25.0
	Correct results (%)	91.3	91.3	86.9	65.2	40.9	78.2	34.8	43.5	43.5	86.9	91.3	82.6

Virus strains: 1TBEV strain K23; 2TBEV strain Absettarov; 3TBEV strain Sofjin; 4TBEV strain Aina. *Dilution series with TBEV strain K23: 500,000 RNA copies/ml to 50 RNA copies/ml. +: correct positive result; −: correct negative result; (−/+): equivocal, indefinite, borderline result; (+): false positive result; (−): false negative result; inc.: inconclusive due to inhibition; #other flavivirus positive. RNA extraction and preparation: ause of a commercial extraction kit; buse of an in-house extraction method. Diagnostic test procedure: Ause of a nested RT-PCR; Buse of a real time RT-PCR. Three laboratories (Labs. 2, 33 and 44) used two different NATs.

View Large Image Download to PowerPoint

Fig. 1. Probit analysis of the fraction of laboratories achieving a positive result in relation to the virus RNA concentration in a given positive sample (10-fold dilution series of TBEV strain K23). Data points represent the fraction of laboratories with a positive result. The regression line calculated on the basis of a probit model (dose–response curve) is shown; the bars represent the 95% confidence intervals (p=0.021).

We also assessed whether common technical factors influenced the accuracy of the participating laboratories (Table 2). Using multivariate logistic regression we found that both factors, extraction method and NAT, were significantly associated with correct classifications of the samples (p=0.001 and 0.005, respectively). Eight different extraction methods and four different NATs were analysed as technical factors to characterise the test procedures each laboratory was using. Compared to in-house extraction techniques, four extraction methods showed significantly higher likelihood of correct classification (approximately 3- to 9-fold). Furthermore, compared to the real time RT-PCR assay by Schwaiger and Cassinotti (2003) the other NATs had a significantly lower probability of correct classification. Indeed, the only two participants (Labs. 7 and 32) that correctly identified all samples used this method. However, it must be pointed out that some differences could also be due to improper handling of the assays and/or the samples, since laboratories using the same assays showed large variations in diagnostic values (e.g. due to differences in the parameters used for indeterminate results). This is, for example, obvious for the assays described by Schwaiger and Cassinotti (2003) (Labs. 1, 7, 32, 33, 38 and 40) and Puchhammer-Stockl et al. (1995) (Labs. 2, 15, 18, 33, 44 and 46). We previously observed similar discrepancies between real time and nested RT-PCR assays in our external quality assurance (EQA) for PCR diagnostics of dengue virus infection (Lemmer et al., 2004). Laboratories which used nested RT-PCR produced twice the number of false positive results than laboratories using real time RT-PCR, probably because the high sensitivity and steps involved in nested RT-PCR renders it more susceptible to contamination.

Table 2. Factors influencing the performance of laboratories (analyses by multivariate logistic regression)

	Possible technical influence factors	No. of laboratories	p-Value	Odds-ratio	95% confidence interval
	Extraction method (overall)		0.001†
	In-house extraction method	2		1
	QIAamp Viral RNA Extraction Kit (Qiagen)	10	0.013†	3.319	1.294–8.515
	QIAamp RNA Blood Mini Kit (Qiagen)	1	0.305	2.140	0.501–9.146
	RNeasy Kit (Qiagen)	2	0.526	0.700	0.232–2.109
	RNA Mini Kit (Qiagen)	2	0.075	3.057	0.894–10.459
	Trizol-LS (Invitrogen)	1	0.013†	8.734	1.582–48.235
	MagNA Pure LC Total Nucleic Acid Isolation Kit (Roche)	4	0.027†	3.449	1.152–10.326
	High Pure Viral RNA Kit (Roche)	1	0.024†	7.643	1.315–44.422
	PCR method (overall)		0.005†
	Real-time PCR (Schwaiger and Cassinotti, 2003)	6		1
	Nested PCR (Puchhammer-Stockl et al., 1995)	6	0.006†	0.272	0.108–0.686
	Any other nested PCR assay	11	0.001†	0.229	0.094–0.560
	Any other real-time PCR assay	3	0.002†	0.184	0.064–0.527

Three laboratories (Labs. 2, 33 and 44) used two different NATs for detection.

† Statistically significant; p≤0.05 (for details see text).

This study demonstrates the importance of quality control measures for the molecular detection of TBE infections. The results clearly indicate a need for certain laboratories to improve their test systems with regard to sensitivity and specificity, particularly those achieving levels of accuracy less than 75%.