Dual independent screening in your systematic review process is undoubtedly considered a best practice. But for some research teams, a lack of resources precludes this from being a standard operating procedure.
New evidence reinforces the value in dual screening, and why it’s essential to have at least some kind of process in place to mitigate the risk of using only one screener. Let’s take a closer look.
The Evidence
The major risk that research teams take when only using one screener is that relevant studies will be missed, potentially invalidating the review.
Siw Waffenschmidt, Marco Knelangen, Wiebke Sieben, Stefanie Bühn, and Dawid Pieper published a methodological systematic review, which included 4 studies comparing single versus dual independent screening. They found that single screeners missed a median of 5% of studies that would have been included by two independent reviewers reaching consensus. Although reviewer experience may affect this error rate, they also report that even reviewers with more experience missed a median of 3% of relevant studies.
A major question, of course, is: does this matter? Are the added resource needs required to conduct the screening process fully in duplicate going to affect the results or conclusions of your review? The answer is not yet clear. Not only did the error rate vary widely between comparisons (0-58%), the authors report that, of 7 meta-analyses examining the effect of missed studies, the findings of 3 would have been “substantially” changed, while missed studies had little or no impact on the remaining four.
Another recent study by Gartlehner G, Affengruber L, Titscher V, Noel-Storr A, Dooley G, Ballarini N, König F suggests a similar pattern and reports that single reviewer screening does not “fulfill the high methodological standards that decision-makers expect from systematic reviews.”
A study published by Zhen Wang, Tarek Nayfeh, Jennifer Tetzlaff, Peter O’Blenis, and Mohammad Hassan Murad suggests that both single and double reviewer methodologies produce errors. They state that it’s important to note that the gold standard (independent dual screening) is not perfect, and therefore automating systematic review processes to the degree of human accuracy is sufficient when it comes to taking measures to save time and resources in a review.
Balancing Your Resources and Risk Tolerance
What does this mean for your systematic review process? Dual independent screening is still supported as the gold standard. Individual reviewers make mistakes and these mistakes may affect your ability to draw valid conclusions. However, time and resources may still make this a challenge.
The good news is that additional resource requirements for dual screening can be minimized and efficiencies gained by using systematic review software to facilitate this process.
How Systematic Review Software Makes Screening Easier
For research teams looking to make their systematic review process more efficient without sacrificing validity, systematic review software offers many advantages. Using technology frees up more resources and makes the review process faster and more cost-effective.
Some of the main ways systematic review tools help research teams are:
- Automatic tracking of inclusions/exclusions for improved transparency and simplified management of records throughout your workflow
- Editing, time stamping, and version control to maintain a clear audit trail
- Instant generation of exclusion lists and PRISMA Diagrams
- Rapid title screening, deduplication, and artificial intelligence
Dual Screening Alternatives
Still limited to one screener? Does your particular topic or process have a higher risk-threshold for missed studies? Most of the above options help minimize the risk and increase efficiencies. Teams will also occasionally employ alternate options such as the following:
- One Screener with Verification: This method consists of one person doing the screening, and a second person double-checking the first person’s work.
- Quality control: Teams will occasionally conduct dual review for a sub-sample of records (for instance a random sample) to estimate if error rates are within acceptable bounds for their specific review.
- Dual Screen Exclusions Only: In this case, preliminary screening levels (e.g. title/abstract only) require dual review only for exclusions. A record labelled for inclusion by either reviewer would be passed on to the next level of screening without a second review as would records with conflicting screening decisions. This method helps avoid and potential lengthy processes of conflict resolution based on limited information, however, the relative time and financial costs of such methods are unclear.
Does your systematic review process employ one screener or two? If you’re already employing dual screening, is your process as efficient as possible? Or does it need work? Systematic review software could be the answer.
