How to write a systematic literature review
Table of Contents
7.0 DECIDING ON QUANTITATIVE OR QUALITATIVE RESEARCH.. 10
8.0 PRESENTING A SYSTEMATIC REVIEW… 11
1.0 INTRODUCTION
Volumes and volumes of studies are produced yearly, usually with conflicting results. The difference in results could be because of difference in methodology, sampling and flaws. In these circumstances, it’s not possible to determine the general outcome or the most consistent results that ought to be used in developing policies or practices. As such, a systematic review aims at addressing these issues by establishing, and undertaking a critical evaluation that integrates the findings of related studies. As explained by Cooper (2003), a good systematic literature review should achieve most if not all of the following aspects, determine to what level present studies have progressed towards explaining a certain issue, establish relations, differences, gaps and discrepancies in the literature, develop general statements, comment on, assess, extend or formulate theory, and in underlining these things offer implications for policy and practice. Accordingly, the following guide provides the guidelines for writing a systematic literature review. “How to write a systematic literature review” is a comprehensive guide provided by our experts.
2.0 SCOPING
The first step is to develop one or several research questions? The research questions should seek to establish what you seek to know, and the subject under investigation. The research question should be formulated bearing in mind the type of audience. A successful systematic review requires clear and specific research questions. For instance, “is behavioral therapy an effective intervention for youth engaged in drug abuse? Will provide more specific answers than “how do you assist youth with drug problems? Certainly, you will need to undertake some “scoping” of available literature to establish what has been established earlier and what might be new, important and exciting scientific contribution to the exciting literature. Focusing on a narrow research area or research question simplifies task ahead.
Under scoping, the next thing is to clarify if the planned systematic review is new or was already carried before. This will help you to avoid wasting your time and resources. Similarly, researching introduce you to the exiting literature on your subject. Supposing there is already existing systematic review that does not require an update, save time by conducting a different one. Still, it is possible to conduct an updated systematic review owing to different reasons. For example, the last review could be older (over 10 years), meaning new studies needs to be incorporated. The previous review was used a flawed methodology, which the present review seeks to address. Lastly, it could be because the last review focused on a different issue than that you seek to review. The important thing at this point is ensuring that you have formulated a good research question that will contribute to the exiting literature.
3.0 PLANNING
The second stage of undertaking systematic review is planning. This entails breaking down the research questions into specific individual concepts to form search terms. The search terms will be used to operationalize the research questions that have been formulated and assist you in getting suitable articles that you will use. The aim should be to carry out a search that is exhaustive, which represents all previous studies carried out on the subject/issue of interest. Reviewing the available literature and consulting your supervisor will provide you with an idea on how to derive search terms from the research question. It is also important to think of alternative concepts and terms that address the present questions since it is common to have different terms describing the same occurrence or research subject.
After formulating the search terms, you need now to develop initial inclusion and exclusion standards, and review these standards at the start of literature search and sieving process. Basing on your knowledge and advice from your supervisor on the literature, develop a list of objectives that will determine the literature to include and exclude. The literature selected should enable you to address your research question. It is necessary to ensure that you select quality articles and determine the similarities of the studies that should be included. More so, define the limits of the review, and apply the exclusion and inclusion criteria all through the review.
The specific inclusion and exclusion method applied depends on the subject of the systematic review being carried out. It will as well depend on empirical, methodological and theoretical issues related to the systematic literature. As observed by Lipsey & Wilson (2001), if you are dealing with a wider research subject, then a broader inclusion and exclusion method will be used. It is important to justify why you have adopted a certain inclusion and exclusion process ( for example could be implication, generalizability or practical). This is important because your readers will have to interpret the final results and conclusion based on the background of inclusion and exclusion method. Some of the common concerns when undertaking inclusion and exclusion include:
- Research question (the topic and the scope)
- Conceptualization (concepts and terms are usually defined differently by authors)
- Measures (variable used may differ)
- Research design (different designs are used for example qualitative studies, quantitative research, observations, self-reporting or case studies)
- Participants of the study (could be children, or adults)
- Data (data size could affect the final results)
Planning also involves developing a clear record keeping system where you will have to keep a thorough record before undertaking any search or sieving of the literature. You should use one or several record keeping systems to keep the records, and this could assist you in decision-making. Though this may seem has unnecessary, it is helpful where massive literature is involved. Some of the things to record include searches that you have made studies that have been excluded and the reasons and the unpublished work searched. There are several guidelines that can be followed in reporting systematic reviews, they include:
CONSORT guidelines for reporting randomized controlled trials)
AMSTAR (Assessment of Multiple Systematic Reviews)
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)
4.0 IDENTIFICATION
This is the third stage in undertaking the system literature review. You should now use the search terms or concepts to search for two or more appropriate electronic databases. The objective is to find all existing published and unpublished literature that answers your research question. Baumeister (2013) points out that the best approach of finding most of published studies is through searching in at least two varying electronic databases.
- Choose databases that match your subject area (for example, Medline)
- Decide which sections of the articles you wish to search (for example, title, abstract or full article)
- Consider applying limits and filters in the databases when searching the articles (for example, reviewed articles, empirical articles or by date published)
- Consider applying Boolean search operators (and/or) to narrow or widen your search
- Consider applying a wildcard symbol to fill in for character (for example ‘#’ and this depends on the database “wom#n” =women/woman).
Best practices recommendations for undertaking systematic review recommends two different reviewers should carry out the search and sieving process. However, they both should agree on what to be included. Where there is a disagreement on what to include, both should refer to the inclusion and exclusion criteria established earlier. Though in some cases, the student or researcher undertaking the systematic review is expected to carry out the whole processes by him/herself.
You need to carefully scrutinize the search results by scanning through the articles to check the quality, relevance and date of publication. Determine if the research results show the inclusion and exclusion criteria being used is effective in providing possible relevant articles. If it does not, then you need to revise the inclusion and exclusion criteria being used, or the search terms and concepts that you had developed. Alternatively, the search results could be revealing new search terms that need to be included in your current search terms. Carryout extra searches to make sure that you have located every potential and suitable published and unpublished work. It is possible to miss out on potentially suitable articles in the databases. Therefore, extra searchers are necessary for both published and unpublished articles.
Published articles: there are several methods that could discover potentially suitable published (and even published) articles that you could have missed out during the searching phase. Get to read the reference part of the article that is appropriate for inclusion that was obtained in the database. This will provide you with potential relevant articles, and a list of articles that can have relevant studies that later can be researched. If there is need, a manual search (hand-search) can be carried out.
Supposing the past database search (for example Medline) had not included non-English language works, and conference papers, your additional search could include these. More so, you need to search for suitable book chapter maybe for data, but most likely as references of suitable article that you need to follow-up. If the information contained in the article seems to be inadequate to reach a decision regarding inclusion, you may try to contacting the author for clarification.
Unpublished articles: As noted by Cooper (2003). Systematic literature review should be wide-ranging and representative of the past literature as much as possible. Owing to this, one important element of systematic reviews is putting an attempt to search for and use suitable unpublished articles. This assists in reducing the impact of publication bias. Vevea & Woods (2005) observes that in the past, studies have selectively been published based on statistical significance of results, since statistical significance is viewed as a good, while statistical non-significance is viewed as bad. Nonetheless, two studies with similar effect size could have different significance level. Therefore, publication bias happens where studies that are considered statistically non-significant are not submitted for publication. According to Vevea & Woods (2005) publication bias creates a possible serious risk to the validity and reliability of the conclusions of a systematic literature review.
5.0 SCREENING
Screening involves exporting references used to a citation manger with the objective of collating the search results. Certainly, the research showed a huge number of results. When you export the research results to an online citation manger such as RefWorks comes with the following benefits:
- Saves you a lot of time since it is an electronic process and not a manual process
- You will be able to save your search results and backed up implying that your valuable information is not lost
- The citation manager is able to detect and remove duplicates
- The citation manager allows you to share the articles in full-text versions
- The citation manager is able to compile a systematic reference list that is well formatted in any reference style
You should read the title of the articles or their abstract that have been obtained through your search. Many articles will not meet the inclusion and exclusion criteria that you had set earlier. Supposing the title/abstract indicates that article is possibly eligible for inclusion, then you will have to get the full-text version and go through it very carefully. During this phase, continue to focus on sensitivity and locate and sieve as many past articles as possible to avoid missing anything. For the purpose of record keeping, it is enough to only keep the list of rejected articles at this point and include reasons later.
6.0 ELIGIBILTY
Sieve the full-text versions to establish those that are eligible and obtain suitable information that should be included in the literature review. The focus need to change from sensitivity of the article to specificity ensuring that the potential eligible studies are actually relevant and suitable for inclusion. Even at this point, it is possible to quickly reduce the number of potential studies by centering your reading on if the published or unpublished article meets the inclusion and exclusion criteria set (for example, examine the method and findings section, and not introduction on discussion).
Record the reason why particular articles have failed to meet the inclusion and exclusion criteria. When you get an article for inclusion, you should carefully and methodically obtain all relevant information for possible inclusion (this will depend on the research topic/question). The information you obtain will mainly relate to the inclusion and exclusion process, and possibly cover elements in established in the planning section. Remember that the inclusion and exclusion criteria is meant to ensure that you get high-quality and appropriate articles. Nonetheless, you could add notes regarding the quality of each article which will helpful when detailing the limitations of methodology when writing the limitations.
7.0 DECIDING ON QUANTITATIVE OR QUALITATIVE RESEARCH
The best methodology to use between qualitative and qualitative will depend on the type of data that you have accessed the research question, the objective of the review, and the empirical and theoretical issues surrounding the review. In some cases, the systematic review could entail a meta-analysis.
As explained by Lipsey & Wilson (2001) meta-analysis systematic literature review is quantitative system review carried out by performing an analysis that is concerned with establishing estimations. For instance “how big is the effect? (The effect of water pollution in marine animals). Meta-analysis could be applied in other circumstances for example in measurement research, group contrasts and in individual difference studies. Meta-analysis that involves numerical synthesis would be suitable in cases where you need to report quantitative results, or examine similar constructs or relationships. It could as well be used in cases where results are bivariate or results could be presented as standardized (Borenstein et al., 2009).
Baumeister (2013) explains that qualitative research synthesis becomes suitable where a group of researches or studies have different methodologies, where the conceptual and methodological methods to research have changed with time. Qualitative research can as well be used when formulating new theories and when reviewing measures methods in a certain literature.
8.0 PRESENTING A SYSTEMATIC REVIEW
As noted by Cooper (2003), there is no specific way of presenting your systematic literature review. However, what you cover in the review and the way you organize the review ought to be determined by the objectives of the review. For instance, the systematic review could be arranged in conceptual based on the ideas of the researcher or the methodology they used, or could be arranged historically based how years of publication.
However, in generally, a systematic review ought to include all if not most of the underlined items:
Introduction: presents a theoretical and empirical context of the literature that explains key concepts and terms. The introduction as well underlines the aim of the systematic review, and clearly states the goals, audiences and perspectives.
Method: This section should comprehensively the objective and how the systematic literature search will be carried out. The methodology should as well describe how and when specific databases searched, the years of the articles used and the inclusion and exclusion criteria used.
Results: should include the attributes of studies in a table format, describing the quality of the studies used. Articles used should be integrated in unbiased and systematic way.
Discussion: This section includes a summary of the findings that are discussed and conclusions made. The conclusions should be linked to the evidence and in the literature and the strengths and limitations of the studies used underlined. In addition, the implications of the literature review, scientific quality and the vigor of methodology should be discussed. Determine the extent to which current literature has progressed in trying to clarify the particular issue under investigation, and determine if the findings have proved your hypothesis or not. Finally, conclude and make recommendations.
Appendix: you should consider including a list of studies that have been excluded but are possibly eligible.
REFERENCES
Baumeister, R. F. (2013). Writing a literature review. In M. J. Prinstein & M. D. Patterson (Eds.), The portable mentor: Expert guide to a successful career in psychology (pp. 119-132; 2nd ed.). New York: Springer Science+ Business Media.
Borenstein, M., Hedges, L. V., Higgins, J., & Rothstein, H. R. (2009). Introduction to meta-analysis. New York, NY: Wiley.
Cooper, H. M. (2003). Editorial. Psychological Bulletin, 129, 3-9.
Lipsey, M. W., & Wilson, D. (2001). Practical meta-analysis. London: Sage Publications.
Vevea, J. L., & Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10, 428−443.