Materials and methods
Study area and populations
The study was done in eight MHFs located in six malaria endemic regions across Tanzania. The MHFs included Bulombora, Chita, Kaboya, Maramba, Mgambo, Msange, Ruvu and Rwamkoma.
The study population included patients admitted (inpatients) and not admitted (outpatients) at MHFs.
Study design and period
A cross-sectional study was conducted in selected facilities from January 2016 to December 2018.
Sample size and sampling
A total of 16 professional laboratory staff were purposively recruited in QIPs at their respective MHFs. A total of 16,000 slides with thick blood smears from all the laboratories at the MHFs formed a sampling frame. Of these, 20% (n = 3200) were positive of which 65% (n = 2080) were found and collected for assessment. A total of 1920 negative slides were also collected by selecting 10 slides each month using a random sampling method from each MHF in 10 rounds done on quarterly visits. For proficient testing, 2240 slides with standardized expert validated blood smears were available.
Training of microscopists
Prior to commencement of the microscopy QIPs at their MHFs and after competency assessment test, 16 microscopists trained as laboratory technicians and assistants who attended standard 2 weeks’ microscopy course at Malaria Diagnostic Centre as per WHO curriculum participated in the study. The microscopists were assessed by three expert microscopists who were taking part in annual proficiency testing.
Supply of laboratory equipment, reagents and consumables
Unlike before implementation of QIP, during quarterly site visits when EQA started, as part of the quality improvement program, MHFs were supplied sustainably with high quality malaria diagnostic equipments (microscopes, slide warmers, etc.), reagents (stains and fixatives) and laboratory consumables (slide preparation template, distilled water etc.).
Quality control
The smears were read for the first time by microscopists at MHFs. Two experienced expert microscopists performed the second reading as another microscopist performed the third read to break-tie the discordant results of the two assessors.
Data collection
Slide crosschecking Each quarter, assessors collected from storage boxes, slides that were 2–3 months old. Assessors selected in situ from the laboratory register, all reported positive and 10 negative blood slides each month. Negative slides were selected for re-reading at the reference laboratory, using a representative sampling protocol (Additional file 1) to ensure random distribution from the start, middle and end of working day and from variable days distributed across the month. Thick blood smears were assessed for quality of preparation, staining and reading accuracy. Slides were assessed macroscopically and microscopically on 12 key parameters namely, labelling, not cracked, not fixed, no fungus, no bacteria, no wash, size of smear, thickness of smear, uniformity of smear, correct staining and reading the results (Additional file 2). The assessment aimed at scoring total quality index (TQI) of laboratory as quality performance. The TQI is a combined score (%) from all microscopists for the macroscopic and microscopic 12 parameters of blood smears at MHFs. It is the average score of all microscopists for a month, quarter and year under review.
Proficiency testing It was done using malaria slides provided by National Malaria Slide Bank (NMSB). The slides had been read and validated by four blinded National and International experts with WHO certification. During supervision visits, each laboratory staff was assigned to read 20 slides with known parasitaemia of which nine were negatives and 11 were positives with low (< 100 asexual parasites/µl) or high (≥ 100 asexual parasites/µl) parasitaemia. The results for each laboratory staff was compared against known results of the samples then aggregated quarterly for each MHF. The quarterly average performance of proficiency testing was calculated for each participating MHF.
Data analysis
The data was managed using Microsoft Excel software and analyzed using STATA software (STATA Inc, TX, USA). Statistical significance was measured at P-value < 0.05. The results of the EQA was reported for 12 parameters using quality index indicators of well performed (over 75%) for more than 9 parameters, fairly performed (50 to 75%) between 6 and 9 parameters, and poorly performed (< 50%) for less than 6 parameters.
Results
Characteristics of items for external quality assessment
Of the 16 laboratory staff assessed, 13 were assistant health laboratory technologists and two were health laboratory technologists. A total of 4000 slides (1920 negative and 2080 positive) were selected for cross-checking while 2240 validated slides were given to laboratory staff for proficiency testing. Proficiency testing was done three quarters later compared to crosschecking that started earlier in quarter one of 2016.
Slide crosschecking
At the start of the EQA, the baseline data showed that preparation and staining of blood smears were well performed (> 75%) for more than half of parameters assessed. About one quarter of all parameters were performed fairly (between 50 and 75%) while two parameters were performed poorly (< 50%). The smears read by facility staff, 61.5% were correctly diagnosed and 38.5% were incorrectly diagnosed (Fig. 1). For the smears reported as positive, 53.3% were true positive while 46.7% were false positive. With respect to smears reported as negative, 83.6% were true negative while only 16.4% were false negative. The rate of false positive over time decreased with an increase in the TQI of slide preparation; likewise, false negatives increased with a decrease in quality of preparation (Additional files 3 and 4).
Slide cross-checking revealed gradual increase of TQI from 52% in quarter one to 58%, 59% and 72% in quarters two, three and four respectively. Annual mean combined TQI for all studied laboratories increased from 60% in the first year (2016) to 78% in the second year (2017) and 90% in the third year (2018) (Fig. 2).
Proficiency testing
After 1 year, the average of proficiency test scores increased from 75% in quarter four in 2006 to 82% in quarter four in 2017. Generally, the mean proficiency testing performance scores showed an increasing trend from 75% in the first (2016) to 82% in the second (2017) and to 90% in the third year (2018) (Fig. 3).
Discussion
The findings from ten quarterly visits to MHFs demonstrated that supportive supervision contributed to identifying gaps, and through on-site training, a remarkable improvement in the quality of malaria diagnosis by microscopy was achieved. The increased performance was noticed among individual microscopists and the average aggregated results for each of the MHF as described previously [12, 13]. In our view, the reported improvement could also be attributed to provision, and sustaining availability of quality laboratory equipment, reagents and consumables [12]. The observed improvements were apparent in the second quarter after implementing the QIPs during the first quarter, and was maintained in the subsequent quarters and over the three years.
Despite the observed increasing trend of overall laboratory performance for TQI (slide crosschecking) and combined scores (proficiency testing), few MHFs such as Ruvu did not show steady quarterly increasing trends. The reason behind this is the average performance in the final score of the facility being pulled down by low performers among introduced new laboratory staff with less experience in microscopy in the reporting quarters compared to the previous ones when only trained experienced staffs performed well. This was confirmed when newly introduced laboratory staff were re-trained and started to improve their individual scores, which ultimately raised the overall performance of the respective facility [14].
During the first quarter of assessment, the baseline data of combined TQI (from Crosschecking) was found to be 52% lower by 23% compared to 75% of a combined score (for proficiency testing) in the first quarter as well for all MHF. Similarly, the TQI in the first year (2016) was lower by 15% compared to combined proficiency testing scores for all MHFs. The possible explanation for the difference could be due to the effects of on-site retraining of laboratory staff that was conducted after the baseline assessment in the first quarter and before initiation of proficiency testing which started later in the fourth quarter of 2016. That is, by the time laboratory staff were assigned validated slides for proficiency testing, they had acquired reading skills during the initial training and re-training in the previous three quarterly visits.
Generally, a lower (52%) mean preparation TQI for blood smear and high false positive rate (46.7%) at the baseline, suggested that detection of malaria parasites in a poorly prepared and stained smear becomes more difficult and significantly reduces the sensitivity of microscopy. This was verified by re-training laboratory staff and improvements made thereafter. Unlike before implementation of EQA when the false test results were high, we observed improvement in total quality index that was associated with decrease of false positivity rates (graph S3) and vice versa. Likewise, false negatives increased with a decrease in quality of preparation (graph S4). That is, slides which had high quality with respect to preparation with contributed to increased accuracy in detection of the parasite and vice versa. Therefore, the results strongly suggest that poor preparation and staining of blood smear contribute significantly to the high rates of false results. Therefore, EQA programs should address quality of preparations and staining rather than focusing only on microscopists reading skills to ensure high quality of malaria diagnosis and case management. Thus, the overall improvement in the diagnostic services observed in this study was potentially attributed to increased site assessment scores as a result of implemented EQA at MHFs.