Systematic reviews are viewed as the gold standard for evidence-based research. Used in a wide variety of healthcare fields including health economics outcomes research (HEOR), regulatory compliance, guidelines development, and evidence-based medicine, they are also gaining traction as a policy development tool in social science fields such as education, crime and justice, environmental studies, and social welfare.
Regardless of why you’re conducting a systematic review, chances are there are some parts of the review process that aren’t as efficient as you’d like them to be. This guide presents five pragmatic and easy-to-use best practices for screening and data extraction to help to make these processes faster and easier while producing cleaner, better structured, and easier to analyze data.
Tip #1: Use Closed Ended Questions
Often you will see free-form text fields used to capture things like study type, publication language, or country. Generally speaking, if the set of all possible answers to these questions can be determined in advance, the practice of reviewers typing in text answers is suboptimal.
Why Closed Ended Questions Are Better:
- Consistency (must choose from available answers)
- Accuracy (errors introduced when you type manually)
- Speed (much faster to click on an answer than type one)
Learn more: Two Simple Tips for Better Data Extraction Questions
Tip #2: Validate Data at the Point of Entry
Of course, there will always be questions that can’t be closed-ended, for which reviewers must type in answers. For these cases, there are a few things you can do to guide reviewers to give you the data you want, in the format you need.
4 Ways To Validate Data
- Restrict the field to only accept only numeric characters.
- Enforce minimum and maximum values.
- Put character limits on text boxes to keep answers concise.
- Use a date picker to collect date information in a consistent format.
Tip #3: Manage hierarchical repeating datasets efficiently
A long-standing challenge in the systematic review community has been how to design a data extraction form, or forms, to capture related, repeating blocks of information.
Some extracted data is simple and only occurs once in a paper – the study type or setting, for example.
Other types of data, however, may be repeated throughout a paper. Studies may examine multiple outcomes, and each outcome may have multiple study arms, with results for each study arm measured across multiple time periods. The number of each of these data subsets can vary widely from study to study.
The simplest and most common approach has traditionally been to extract the data directly into a spreadsheet, inserting new rows and columns as relevant data is found in the papers.
However, as you can imagine – and may have experienced yourself – with a column for each possible combination of repeating data, your spreadsheet can quickly become monolithic and impossible to analyze. We call this a data management nightmare!
The solution? For each recurring data type, create a separate form or table. Reviewers can complete as many forms or tables as they need for a given study, instead of trying to anticipate the right number from study to study.
When using this approach, remember to link the child dataset with its parent (e.g. by having a question in the child form that names its parent) to retain the context of the information. As long as the hierarchy is maintained, you’ll end up with a database-style table that will be dramatically easier to manipulate and analyze.
Tip #4: Don’t ask for more than you need
Clutter has a way of creeping into our lives, and systematic reviews are far from immune. One way to reduce clutter in your final data set is to design your data collection forms after the tables are mapped out for your final output.
It may seem counterintuitive, but crafting your forms last will ensure you know exactly what information you need so you can put the right questions, with the right wording and the right format, on your forms.
Your systematic review software may allow you to include dynamic forms with conditional hide/show logic, which can be extremely useful for ensuring you get all the required data without any unnecessary information. Based on previous answers, dynamic forms show you only the questions that are applicable – you can even hide a whole form if it doesn’t apply.
Tip #5: Standardize for reuse
Chances are, if you’re working on multiple reviews, your study instruments (i.e. forms) are the same or similar for each one. If not, here’s why you should consider standardizing them.
Standard forms – even if they require minor tweaks from project to project – can reduce the errors and increase the speed of your review team. Because you’re not reading a brand new form every time, you will quickly become much more efficient when filling them out.
On a broader scale, by standardizing your forms, you’re also introducing a repeatable process across your organization, reducing setup time and the learning curve for reviewers. Some systematic review software comes with standard forms for the most common processes, such as quality assessment and risk of bias, to get you started.
You can even take this one step further and create entire project templates. If you’re doing a lot of similar research over and over again, why not build a standard project template that, with minor adjustments from one project to the next, will reduce your setup time and reinforce a prescriptive process?
Bonus tip: Tag excluded references so you can find them later
So far, we’ve shared five tips to help improve the speed and quality of your screening and data extraction. But what happens to the references that didn’t make the cut?
If you’re like most review groups, excluded references are quickly forgotten once it has been determined that they aren’t a good t for your project. But during the screening process, it’s common to come across references that aren’t relevant to your current review project, but could be useful in another review or are of general interest to the screener.
Rather than just tossing these references into the “excluded” pile, try adding a question or two to your screening form to allow reviewers to tag them for future use.
Learn more: 3 Tags for Tracking Excluded References in Your Systematic Reviews
Conclusion
Implementing these five tips won’t make your systematic reviews completely painless, but they should improve your screening and data extraction processes.
All of these tips have one thing in common: they will help you to improve the efficiency and consistency of your review process. Completing a review faster, with fewer errors…who doesn’t want that?
