Human resource requirements for quality-assured electronic data capture of the tuberculosis case register

The recording and reporting system assists national programs in the management of tuberculosis. [1] The tuberculosis case register is the data source for the reports submitted by basic management units to the national tuberculosis program. All essential information on tuberculosis patients to produce the two types of quarterly reports on case finding and treatment outcome is recorded in the standard tuberculosis case register. [1]

The development of computerized implementation of tuberculosis recording and reporting to generate quarterly reports requires data entry into some kind of electronic data entry screen before various analyses or reports can be generated. [2] While analysis of an electronic data base is efficient and very flexible, data entry might be time-consuming and prone to errors which can only be appreciated and corrected with double-entry and validation. This requires additional human resources on top of the basic and indispensable requirement of a physical paper record.

Our objective was to measure the data entry time required to complete and double-enter one record, and to estimate the time for the correction of errors in the captured information from tuberculosis case registers in Cambodia and Viet Nam. This should assist in quantifying the additional requirements in human resources for national programs moving towards electronic recording and reporting.

Each of the 30 pre-determined randomly selected case registers from each jurisdiction was successfully obtained. Each country collected information on patients registered during a 2-year period, Cambodia for the years 2003 and 2004, and Viet Nam for the years 2004 and 2005. There were 4,215 patients from Cambodia and 7,163 patients from Viet Nam. Each of the patient records from Cambodia had two records with the number of seconds recorded, thus 8,430 records were available. For the 7,163 patients from Viet Nam, only 14,302 records had the number of seconds recorded, 7,153 (10 missing) in the first, and 7,149 (14 missing) in the second set. Thus, a total of 22,732 records were available for the analysis of data entry time, 8,430 (37.1%) from Cambodia and 14,302 (62.9%) from Viet Nam.

Data entry time

The results for data entry time are summarized in Table 1 and the distributions shown in Figures 1 and 2. The results demonstrate that the mean data entry time required in Cambodia was approximately 50% higher per record with 97.5 s than in Viet Nam with 66.2 s. The respective medians were 90 and 31 s. The distribution shows that very few records in Cambodia required fewer than 40 s, but in Viet Nam, the mode was actually at 30 to 40 s.

Characteristic	Cambodia	Viet Nam	Total
Number of patients	4,215	7,163	11,378
Number of records with data entry time	8,430	14,302	22,732
Data entry time (seconds)
Mean	97.5	66.2	77.8
95% confidence interval of mean	96.2 - 98.8	59.5 - 73.0	73.5 - 82.1
Median	90	31	52
Error assessment
Number of common records	4,215	7,146	11,361
Number (%) of records with errors	251 (6.0)	2,787 (39.0)	3,038 (26.7)
Number of common fields *	240,255	426,950	667,205
Number (%) of fields with errors	433 (0.2)	8,920 (2.1)	9,353 (1.4)

*Due to small differences in the questionnaire design adapted to country-specific requirements, 57 fields per record were compared in Cambodia and 60 fields per record in Viet Nam (except for 58 fields in 5 units of the latter)

Validating double data entry

The measure of data entry quality is the proportion of fields with an error. (Figure 3) It must be noted, however, that a discordance for instance in a day field will show up as three errors as the automatically calculated exact and approximate dates will also be discordant. Thus correcting the error in the day in such an instance will automatically resolve all three discordances. The measure for the amount of work it will take to resolve all discordances will principally depend on the number of records, rather than the number of fields, that have to be revisited. The summary in Table 1 shows that in Cambodia only 6.0% of records, but in Viet Nam 39.0% of records showed any discordance between the pair of records. The proportion of records with any discordance depends on the number of fields that need to be entered per record as the probability of a data entry error in a given record increases with the number of fields.

The time required for checking the reported discordance in each record was not measured electronically. It had been done with a stopwatch in a pilot study of a tuberculosis laboratory register (Rieder H L, Gafner Zwahlen H, Spörry D, May 2006, unpublished). In that database with 1,620 common records, 105 (6.5%) had an error, a similar proportion as in Cambodia, but there were only 10 fields per record. The two data entry persons in the unpublished pilot study required 1.5 h to identify the discordant of records, check whether there was an error, and correct it if necessary. The total average data entry time for one set had been 9.47 h. In other words, the time required for record identification and correction required much more time than entering an entire new record. For the study here with the longer data entry forms, it appeared thus to be justified to assume that finding a record, checking the discordance(s), correcting them if necessary, and saving the revised record to disk would take the same time as entering a new record. The results of the calculations are summarized in Table 2.

Characteristic	Cambodia	Viet Nam	Total
Total number of records times two for double entry	8,430	14,302	22,732
Number of seconds per new record	97.5	66.2	77.8
Number of hours for all records	228.3	263.0	491.3
Number of records to check	251	2,787	3,038
Number of seconds for correction per record *	97.5	66.2	77.8
Total number of hours for correction	6.8	51.2	65.7
Total number of hours for validated records	235.1	314.2	556.9
Number of hours per 1,000 records	27.9	22.0	24.5
Number of hours per 1,000 validated records	55.8	43.9	49.0
Adjusted number of hours by 10-minute break hour per 1,000 records	66.9	52.7	58.8
Number of hours for 2 data entry persons per 1,000 records	133.9	105.5	117.6

* Identifying a record with a discordance, checking for the discordance, correcting an error if necessary, and saving the revised record to disk was assumed to take the same amount of time as entering a new record

As the computer-calculated number of seconds is pure data entry time, adjustment must be made for breaks to provide fair remuneration for work. For this study the recommendation was that 1 h should comprise 50 min work and 10 min break time. Furthermore, the data were always entered by a pair of workers. A total of 117.6 person-hours were thus required to obtain a final validated data set of 1,000 records, with a fairly small difference between Cambodia and Viet Nam.

In Viet Nam a professional team was employed for data entry, while in Cambodia the investigator took this task upon himself with colleagues and friends. It would appear that the professional team in Viet Nam worked much faster but also made considerably more errors that in themselves then required substantially more time for identifying and correcting. In contrast, the team in Cambodia worked slower but made substantially fewer errors, thus shortening considerably the time to make corrections. As a net result, there was only a small difference in the overall time required to produce a validated record.

In this study, we found that the median data entry time per record was three times larger in Cambodia than in Viet Nam. However, data entry time was inversely associated with error frequency: only six per cent of records in Cambodia had at least one data entry error as compared to almost 40% in Viet Nam.

We used freely available software for our study. Keeping up with proprietary software can accumulate to large costs that might be prohibitive for many low-income countries. The use of proprietary software will also prevent many researchers in such countries to carry out their research. Most importantly, for both affluent and low-income countries, proprietary software is mostly specified for analysis but not for quality-assured data entry. EpiData (EpiData Association, Denmark) and Epi Info (US Centers for Disease Control and Prevention) are the most important software packages that address both needs, they are free and allow efficient data entry and validation. [3]

With the spread of personal computers to intermediate and increasingly peripheral levels, an increasing number of countries has been initiating or is planning using electronic recording and reporting systems, referred to by the World Health Organization (WHO) as "e-R&R software" (http://www.who.int/tb/country/recording_reporting/en/index.html, accessed 1 December 2010). The advantages for analysis are apparent, yet any analysis is only as good as the quality of the input data.

At the above "TB e-Recording and Reporting Portal" of the WHO various systems are reviewed, most of which are based on proprietary software, and many depend on continuous and secure internet access. In the discussions of the Expert Group, the role of data quality is mentioned but no specific recommendations are made except for the suggestion that samples are checked for error frequency. Requirements of additional human resources for electronic data capture find no mentioning. In fact, an extensive evaluation of the experience with an electronic tuberculosis surveillance system in a low-income country, its pitfalls and successes alike, has, to our knowledge, only been published from one country. [2, 4]

We conducted a country-wide representative study of tuberculosis registers in Cambodia, two provinces in China, and Viet Nam. [5] Because of the research nature of the study we required that the electronically captured data had to be an accurate reflection of the actual physical case registers. The thus chosen approach with double data entry and validation also provided an opportunity to quantify precisely the extent of errors made during data entry. Coupled with an inbuilt measurement of data entry time in Cambodia and Viet Nam, the dataset permitted determination of the cost in human resources to establish an as accurate as possible electronic dataset. Our analysis shows that it required 118 person-hours to obtain 1,000 validated patient records.

Importantly, the study demonstrates that the quality of primary data capture differed considerably between Cambodia and Viet Nam. This is likely the result of the actual differences in the arrangements made in the two countries: in Cambodia, the responsible researcher entered the data personally together with friends while in Viet Nam the task of data entry was outsourced but remained nevertheless under close supervision of the responsible researcher. The resulting differences are telling in that the quality of data entry was much superior where the researcher took personal responsibility than where the task was outsourced. Although the professional team worked considerably faster, this seeming advantage was nevertheless almost entirely lost by the longer time required to make all the necessary corrections. It is likely that in a routine setting supervision of data entry personnel with often little stake in quality-assured data is likely to be less tight than in this study setting. Because the extent of transcription errors will remain unpredictable and may vary greatly between settings, it would appear that quality assurance is an indispensable component of any research project specifically [6] and any routine electronic surveillance in general.

The system we employed was based on non-proprietary software and did not depend on continuous and high-speed internet access. Because EpiData software is text-based (http://www.epidata.dk), 1,000 records with 60 fields took less than 300 kilobytes, a file size that is easily transmitted as an E-mail attachment even with a slow and irregular internet connection.

This study has one main limitation. While the software allows measurement of cumulative time spent on each record, we did not foresee this in the design of our data entry form and recorded only the time needed for a first entry. We had thus to make the assumption (derived from external data) that finding a record, checking the discordance(s), correcting them if necessary, and saving the revised record to disk would take the same time as entering a new record.

While this study was done for research purposes, we believe that it has wider and generic implications for planning and budgeting in national tuberculosis programs. This is exemplified in this study, where the information of the recording time was necessary to budget payment for data entry persons based on an objective measure of delivery of validated records.

Approximately 118 person-hours were required on average to produce 1,000 validated records of the tuberculosis case register. While professional data entry teams appear to work faster, they also make more errors. Our study seems to suggest that validating electronically captured data through double-entry and subsequent correction is an indispensable prerequisite for any epidemiologic research. The results demonstrate differences in quality of data entry, using identical data entry forms but different types of data entry teams. This suggests that our observations have important repercussions on electronic surveillance of tuberculosis in general if the objective is ensuring the quality of any subsequent analysis. The frequency of data entry errors varies greatly and cannot be known without actual data validation and analyses might thus be wrong. National tuberculosis programs need to pay special attention to this fact. They must take the necessary inbuilt precautionary measures in electronic database systems to reduce data entry errors. Barring double-entry and validation, a minimum requirement is ascertainment of error frequency in representative samples from the database.