Back to: Extract Data

Let’s start with how you train and monitor your extractors:

Ideal: Train all of your data extractors in advance

Practical: Train your data extractors as you go

Even if you carefully train your team far in advance, you’re going to find some inconsistencies in how each person fills out the data extraction form. You’re working with people after all, not machines.

Take a look at this side by side comparison of how two research assistants extracted data from the same study:

 

You’ll notice that reviewer A’s form includes just the basic information, while reviewer B’s form includes a lot of details, especially about depression incidence. Reviewer A remembered to write “NR” or “Not Reported” for fields where the study didn’t have related information. Reviewer B just left those fields blank.

The reviewers also disagree on a few fields: check out their answers for Health Related QoL and depression incidence.

So which reviewer is better? Neither one. Both filed out the form to the best of their ability; they just interpreted what they were supposed to do differently.