what is an systematic literature review

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

• 1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
• 2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
• 3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
• PMID: 30089228
• DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

• Guidelines as Topic
• Meta-Analysis as Topic*
• Publication Bias
• Review Literature as Topic
• Systematic Reviews as Topic*

Library Services

UCL LIBRARY SERVICES

• Guides and databases
• Library skills
• Systematic reviews

What are systematic reviews?

• Types of systematic reviews
• Formulating a research question
• Identifying studies
• Searching databases
• Describing and appraising studies
• Synthesis and systematic maps
• Software for systematic reviews
• Online training and support
• Live and face to face training
• Individual support
• Further help

Systematic reviews are a type of literature review of research which require equivalent standards of rigour as primary research. They have a clear, logical rationale that is reported to the reader of the review. They are used in research and policymaking to inform evidence-based decisions and practice. They differ from traditional literature reviews particularly in the following elements of conduct and reporting.

Systematic reviews: 

• use explicit and transparent methods
• are a piece of research following a standard set of stages
• are accountable, replicable and updateable
• involve users to ensure a review is relevant and useful.

For example, systematic reviews (like all research) should have a clear research question, and the perspective of the authors in their approach to addressing the question is described. There are clearly described methods on how each study in a review was identified, how that study was appraised for quality and relevance and how it is combined with other studies in order to address the review question. A systematic review usually involves more than one person in order to increase the objectivity and trustworthiness of the reviews methods and findings.

Research protocols for systematic reviews may be peer-reviewed and published or registered in a suitable repository to help avoid duplication of reviews and for comparisons to be made with the final review and the planned review.

• History of systematic reviews to inform policy (EPPI-Centre)
• Six reasons why it is important to be systematic (EPPI-Centre)
• Evidence Synthesis International (ESI): Position Statement Describes the issues, principles and goals in synthesising research evidence to inform policy, practice and decisions

On this page

Should all literature reviews be 'systematic reviews', different methods for systematic reviews, reporting standards for systematic reviews.

Literature reviews provide a more complete picture of research knowledge than is possible from individual pieces of research. This can be used to: clarify what is known from research, provide new perspectives, build theory, test theory, identify research gaps or inform research agendas.

A systematic review requires a considerable amount of time and resources, and is one type of literature review.

If the purpose of a review is to make justifiable evidence claims, then it should be systematic, as a systematic review uses rigorous explicit methods. The methods used can depend on the purpose of the review, and the time and resources available.

A 'non-systematic review' might use some of the same methods as systematic reviews, such as systematic approaches to identify studies or quality appraise the literature. There may be times when this approach can be useful. In a student dissertation, for example, there may not be the time to be fully systematic in a review of the literature if this is only one small part of the thesis. In other types of research, there may also be a need to obtain a quick and not necessarily thorough overview of a literature to inform some other work (including a systematic review). Another example, is where policymakers, or other people using research findings, want to make quick decisions and there is no systematic review available to help them. They have a choice of gaining a rapid overview of the research literature or not having any research evidence to help their decision-making. 

Just like any other piece of research, the methods used to undertake any literature review should be carefully planned to justify the conclusions made. 

Finding out about different types of systematic reviews and the methods used for systematic reviews, and reading both systematic and other types of review will help to understand some of the differences. 

Typically, a systematic review addresses a focussed, structured research question in order to inform understanding and decisions on an area. (see the  Formulating a research question  section for examples). 

Sometimes systematic reviews ask a broad research question, and one strategy to achieve this is the use of several focussed sub-questions each addressed by sub-components of the review.  

Another strategy is to develop a map to describe the type of research that has been undertaken in relation to a research question. Some maps even describe over 2,000 papers, while others are much smaller. One purpose of a map is to help choose a sub-set of studies to explore more fully in a synthesis. There are also other purposes of maps: see the box on  systematic evidence maps  for further information. 

Reporting standards specify minimum elements that need to go into the reporting of a review. The reporting standards refer mainly to methodological issues but they are not as detailed or specific as critical appraisal for the methodological standards of conduct of a review.

A number of organisations have developed specific guidelines and standards for both the conducting and reporting on systematic reviews in different topic areas.  

• PRISMA PRISMA is a reporting standard and is an acronym for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. The Key Documents section of the PRISMA website links to a checklist, flow diagram and explanatory notes. PRISMA is less useful for certain types of reviews, including those that are iterative.
• eMERGe eMERGe is a reporting standard that has been developed for meta-ethnographies, a qualitative synthesis method.
• ROSES: RepOrting standards for Systematic Evidence Syntheses Reporting standards, including forms and flow diagram, designed specifically for systematic reviews and maps in the field of conservation and environmental management.

Useful books about systematic reviews

Systematic approaches to a successful literature review

An introduction to systematic reviews

Cochrane handbook for systematic reviews of interventions

Systematic reviews: crd's guidance for undertaking reviews in health care.

Finding what works in health care: Standards for systematic reviews

Systematic Reviews in the Social Sciences

Meta-analysis and research synthesis.

Research Synthesis and Meta-Analysis

Doing a Systematic Review

Literature reviews.

• What is a literature review?
• Why are literature reviews important?
• << Previous: Systematic reviews
• Next: Types of systematic reviews >>
• Last Updated: Apr 4, 2024 10:09 AM
• URL: https://library-guides.ucl.ac.uk/systematic-reviews

Literature Reviews

• Getting started

What is a literature review?

Why conduct a literature review, stages of a literature review, lit reviews: an overview (video), check out these books.

• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject.

Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field.

Analysis: Involves critically evaluating and summarizing key findings, methodologies, and debates found in academic literature.

Identifying Gaps: Aims to pinpoint areas where there is a lack of research or unresolved questions, highlighting opportunities for further investigation.

Contextualization: Enables researchers to understand how their work fits into the broader academic conversation and contributes to the existing body of knowledge.

tl;dr  A literature review critically examines and synthesizes existing scholarly research and publications on a specific topic to provide a comprehensive understanding of the current state of knowledge in the field.

What is a literature review NOT?

❌ An annotated bibliography

❌ Original research

❌ A summary

❌ Something to be conducted at the end of your research

❌ An opinion piece

❌ A chronological compilation of studies

The reason for conducting a literature review is to:

Literature Reviews: An Overview for Graduate Students

While this 9-minute video from NCSU is geared toward graduate students, it is useful for anyone conducting a literature review.

Writing the literature review: A practical guide

Available 3rd floor of Perkins

Writing literature reviews: A guide for students of the social and behavioral sciences

Available online!

So, you have to write a literature review: A guided workbook for engineers

Telling a research story: Writing a literature review

The literature review: Six steps to success

Systematic approaches to a successful literature review

Request from Duke Medical Center Library

Doing a systematic review: A student's guide

• Next: Types of reviews >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Systematic Reviews

Describes what is involved with conducting a systematic review of the literature for evidence-based public health and how the librarian is a partner in the process.

Several CDC librarians have special training in conducting literature searches for systematic reviews.  Literature searches for systematic reviews can take a few weeks to several months from planning to delivery.

Fill out a search request form here  or contact the Stephen B. Thacker CDC Library by email  [email protected] or telephone 404-639-1717.

Campbell Collaboration

Cochrane Collaboration

Eppi Centre

Joanna Briggs Institute

McMaster University

PRISMA Statement

Systematic Reviews – CRD’s Guide

Systematic Reviews of Health Promotion and Public Health Interventions

The Guide to Community Preventive Services

Look for systematic reviews that have already been published. 

• To ensure that the work has not already been done.
• To provides examples of search strategies for your topic

Look in PROSPERO for registered systematic reviews.

Search Cochrane and CRD-York for systematic reviews.

Search filter for finding systematic reviews in PubMed

Other search filters to locate systematic reviews

A systematic review attempts to collect and analyze all evidence that answers a specific question.  The question must be clearly defined and have inclusion and exclusion criteria. A broad and thorough search of the literature is performed and a critical analysis of the search results is reported and ultimately provides a current evidence-based answer  to the specific question.

Time:  According to Cochrane , it takes 18 months on average to complete a Systematic Review.

The average systematic review from beginning to end requires 18 months of work. “…to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months…” ( Cochrane Collaboration )

Review Team: Team Members at minimum…

• Content expert
• 2 reviewers
• 1 tie breaker
• 1 statistician (meta-analysis)
• 1 economist if conducting an economic analysis
• *1 librarian (expert searcher) trained in systematic reviews

“Expert searchers are an important part of the systematic review team, crucial throughout the review process-from the development of the proposal and research question to publication.” ( McGowan & Sampson, 2005 )

*Ask your librarian to write a methods section regarding the search methods and to give them co-authorship. You may also want to consider providing a copy of one or all of the search strategies used in an appendix.

The Question to Be Answered: A clearly defined and specific question or questions with inclusion and exclusion criteria.

Written Protocol: Outline the study method, rationale, key questions, inclusion and exclusion criteria, literature searches, data abstraction and data management, analysis of quality of the individual studies, synthesis of data, and grading of the evidience for each key question.

Literature Searches:  Search for any systematic reviews that may already answer the key question(s).  Next, choose appropriate databases and conduct very broad, comprehensive searches.  Search strategies must be documented so that they can be duplicated.  The librarian is integral to this step of the process. Before your librarian creates a search strategy and starts searching in earnest you should write a detailed PICO question , determine the inclusion and exclusion criteria for your study, run a preliminary search, and have 2-4 articles that already fit the criteria for your review.

What is searched depends on the topic of the review but should include…

• At least 3 standard medical databases like PubMed/Medline, CINAHL, Embase, etc..
• At least 2 grey literature resources like Clinicaltrials.gov, COS Conference Papers Index, Grey Literature Report,  etc…

Citation Management: EndNote is a bibliographic management tools that assist researchers in managing citations.  The Stephen B. Thacker CDC Library oversees the site license for EndNote.

To request installation:   The library provides EndNote  to CDC staff under a site-wide license. Please use the ITSO Software Request Tool (SRT) and submit a request for the latest version (or upgraded version) of EndNote. Please be sure to include the computer name for the workstation where you would like to have the software installed.

EndNote Training:   CDC Library offers training on EndNote on a regular basis – both a basic and advanced course. To view the course descriptions and upcoming training dates, please visit the CDC Library training page .

For assistance with EndNote software, please contact [email protected]

Vendor Support and Services:   EndNote – Support and Services (Thomson Reuters)  EndNote – Tutorials and Live Online Classes (Thomson Reuters)

Getting Articles:

Articles can be obtained using DocExpress or by searching the electronic journals at the Stephen B. Thacker CDC Library.

IOM Standards for Systematic Reviews: Standard 3.1: Conduct a comprehensive systematic search for evidence

The goal of a systematic review search is to maximize recall and precision while keeping results manageable. Recall (sensitivity) is defined as the number of relevant reports identified divided by the total number of relevant reports in existence. Precision (specificity) is defined as the number of relevant reports identified divided by the total number of reports identified.

Issues to consider when creating a systematic review search:   

• All concepts are included in the strategy
• All appropriate subject headings are used
• Appropriate use of explosion
• Appropriate use of subheadings and floating subheadings
• Use of natural language (text words) in addition to controlled vocabulary terms
• Use of appropriate synonyms, acronyms, etc.
• Truncation and spelling variation as appropriate
• Appropriate use of limits such as language, years, etc.
• Field searching, publication type, author, etc.
• Boolean operators used appropriately
• Line errors: when searches are combined using line numbers, be sure the numbers refer to the searches intended
• Check indexing of relevant articles
• Search strategy adapted as needed for multiple databases
• Cochrane Handbook: Searching for Studies See Part 2, Chapter 6

A step-by-step guide to systematically identify all relevant animal studies

Materials listed in these guides are selected to provide awareness of quality public health literature and resources. A material’s inclusion does not necessarily represent the views of the U.S. Department of Health and Human Services (HHS), the Public Health Service (PHS), or the Centers for Disease Control and Prevention (CDC), nor does it imply endorsement of the material’s methods or findings. HHS, PHS, and CDC assume no responsibility for the factual accuracy of the items presented. The selection, omission, or content of items does not imply any endorsement or other position taken by HHS, PHS, and CDC. Opinion, findings, and conclusions expressed by the original authors of items included in these materials, or persons quoted therein, are strictly their own and are in no way meant to represent the opinion or views of HHS, PHS, or CDC. References to publications, news sources, and non-CDC Websites are provided solely for informational purposes and do not imply endorsement by HHS, PHS, or CDC.

Exit Notification / Disclaimer Policy

• The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
• Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
• You will be subject to the destination website's privacy policy when you follow the link.
• CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.

Literature Review: Systematic literature reviews

• Traditional or narrative literature reviews
• Scoping Reviews
• Systematic literature reviews
• Annotated bibliography
• Keeping up to date with literature
• Finding a thesis
• Evaluating sources and critical appraisal of literature
• Managing and analysing your literature
• Further reading and resources

Systematic reviews

Systematic and systematic-like reviews

Charles Sturt University library has produced a comprehensive guide for Systematic and systematic-like literature reviews. A comprehensive systematic literature review can often take a team of people up to a year to complete. This guide provides an overview of the steps required for systematic reviews:

• Identify your research question
• Develop your protocol
• Conduct systematic searches (including the search strategy, text mining, choosing databases, documenting and reviewing
• Critical appraisal
• Data extraction and synthesis
• Writing and publishing .
• Systematic and systematic-like reviews Library Resource Guide

Systematic literature review

A systematic literature review (SLR) identifies, selects and critically appraises research in order to answer a clearly formulated question (Dewey, A. & Drahota, A. 2016). The systematic review should follow a clearly defined protocol or plan where the criteria is clearly stated before the review is conducted. It is a comprehensive, transparent search conducted over multiple databases and grey literature that can be replicated and reproduced by other researchers. It involves planning a well thought out search strategy which has a specific focus or answers a defined question. The review identifies the type of information searched, critiqued and reported within known timeframes. The search terms, search strategies (including database names, platforms, dates of search) and limits all need to be included in the review.

Pittway (2008) outlines seven key principles behind systematic literature reviews

• Transparency
• Integration
• Accessibility

Systematic literature reviews originated in medicine and are linked to evidence based practice. According to Grant & Booth (p 91, 2009) "the expansion in evidence-based practice has lead to an increasing variety of review types". They compare and contrast 14 review types, listing the strengths and weaknesses of each review. 

Tranfield et al (2003) discusses the origins of the evidence-based approach to undertaking a literature review and its application to other disciplines including management and science.

References and additional resources

Dewey, A. & Drahota, A. (2016) Introduction to systematic reviews: online learning module Cochrane Training   https://training.cochrane.org/interactivelearning/module-1-introduction-conducting-systematic-reviews

Gough, David A., David Gough, Sandy Oliver, and James Thomas. An Introduction to Systematic Reviews. Systematic Reviews. London: SAGE, 2012.

Grant, M. J. & Booth, A. (2009) A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal 26(2), 91-108

Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol, 18(1), 143. https://doi.org/10.1186/s12874-018-0611-x 

Pittway, L. (2008) Systematic literature reviews. In Thorpe, R. & Holt, R. The SAGE dictionary of qualitative management research. SAGE Publications Ltd doi:10.4135/9780857020109

Tranfield, D., Denyer, D & Smart, P. (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review . British Journal of Management 14 (3), 207-222

Evidence based practice - an introduction : Literature reviews/systematic reviews

Evidence based practice - an introduction is a library guide produced at CSU Library for undergraduates. The information contained in the guide is also relevant for post graduate study and will help you to understand the types of research and levels of evidence required to conduct evidence based research.

• Evidence based practice an introduction
• << Previous: Scoping Reviews
• Next: Annotated bibliography >>
• Last Updated: May 12, 2024 12:18 PM
• URL: https://libguides.csu.edu.au/review

Charles Sturt University is an Australian University, TEQSA Provider Identification: PRV12018. CRICOS Provider: 00005F.

University Libraries      University of Nevada, Reno

• Skill Guides
• Subject Guides

Systematic, Scoping, and Other Literature Reviews: Overview

• Project Planning

What Is a Systematic Review?

Regular literature reviews are simply summaries of the literature on a particular topic. A systematic review, however, is a comprehensive literature review conducted to answer a specific research question. Authors of a systematic review aim to find, code, appraise, and synthesize all of the previous research on their question in an unbiased and well-documented manner. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) outline the minimum amount of information that needs to be reported at the conclusion of a systematic review project. 

Other types of what are known as "evidence syntheses," such as scoping, rapid, and integrative reviews, have varying methodologies. While systematic reviews originated with and continue to be a popular publication type in medicine and other health sciences fields, more and more researchers in other disciplines are choosing to conduct evidence syntheses. 

This guide will walk you through the major steps of a systematic review and point you to key resources including Covidence, a systematic review project management tool. For help with systematic reviews and other major literature review projects, please send us an email at  [email protected] .

Getting Help with Reviews

Organization such as the Institute of Medicine recommend that you consult a librarian when conducting a systematic review. Librarians at the University of Nevada, Reno can help you:

• Understand best practices for conducting systematic reviews and other evidence syntheses in your discipline
• Choose and formulate a research question
• Decide which review type (e.g., systematic, scoping, rapid, etc.) is the best fit for your project
• Determine what to include and where to register a systematic review protocol
• Select search terms and develop a search strategy
• Identify databases and platforms to search
• Find the full text of articles and other sources
• Become familiar with free citation management (e.g., EndNote, Zotero)
• Get access to you and help using Covidence, a systematic review project management tool

Doing a Systematic Review

• Plan - This is the project planning stage. You and your team will need to develop a good research question, determine the type of review you will conduct (systematic, scoping, rapid, etc.), and establish the inclusion and exclusion criteria (e.g., you're only going to look at studies that use a certain methodology). All of this information needs to be included in your protocol. You'll also need to ensure that the project is viable - has someone already done a systematic review on this topic? Do some searches and check the various protocol registries to find out. 
• Identify - Next, a comprehensive search of the literature is undertaken to ensure all studies that meet the predetermined criteria are identified. Each research question is different, so the number and types of databases you'll search - as well as other online publication venues - will vary. Some standards and guidelines specify that certain databases (e.g., MEDLINE, EMBASE) should be searched regardless. Your subject librarian can help you select appropriate databases to search and develop search strings for each of those databases.  
• Evaluate - In this step, retrieved articles are screened and sorted using the predetermined inclusion and exclusion criteria. The risk of bias for each included study is also assessed around this time. It's best if you import search results into a citation management tool (see below) to clean up the citations and remove any duplicates. You can then use a tool like Rayyan (see below) to screen the results. You should begin by screening titles and abstracts only, and then you'll examine the full text of any remaining articles. Each study should be reviewed by a minimum of two people on the project team. 
• Collect - Each included study is coded and the quantitative or qualitative data contained in these studies is then synthesized. You'll have to either find or develop a coding strategy or form that meets your needs. 
• Explain - The synthesized results are articulated and contextualized. What do the results mean? How have they answered your research question?
• Summarize - The final report provides a complete description of the methods and results in a clear, transparent fashion. 

Adapted from

Types of reviews, systematic review.

These types of studies employ a systematic method to analyze and synthesize the results of numerous studies. "Systematic" in this case means following a strict set of steps - as outlined by entities like PRISMA and the Institute of Medicine - so as to make the review more reproducible and less biased. Consistent, thorough documentation is also key. Reviews of this type are not meant to be conducted by an individual but rather a (small) team of researchers. Systematic reviews are widely used in the health sciences, often to find a generalized conclusion from multiple evidence-based studies. 

Meta-Analysis

A systematic method that uses statistics to analyze the data from numerous studies. The researchers combine the data from studies with similar data types and analyze them as a single, expanded dataset. Meta-analyses are a type of systematic review.

Scoping Review

A scoping review employs the systematic review methodology to explore a broader topic or question rather than a specific and answerable one, as is generally the case with a systematic review. Authors of these types of reviews seek to collect and categorize the existing literature so as to identify any gaps.

Rapid Review

Rapid reviews are systematic reviews conducted under a time constraint. Researchers make use of workarounds to complete the review quickly (e.g., only looking at English-language publications), which can lead to a less thorough and more biased review. 

Narrative Review

A traditional literature review that summarizes and synthesizes the findings of numerous original research articles. The purpose and scope of narrative literature reviews vary widely and do not follow a set protocol. Most literature reviews are narrative reviews. 

Umbrella Review

Umbrella reviews are, essentially, systematic reviews of systematic reviews. These compile evidence from multiple review studies into one usable document. 

Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal , vol. 26, no. 2, 2009, pp. 91-108. doi: 10.1111/j.1471-1842.2009.00848.x .

• Next: Project Planning >>

Jump to navigation

• Bahasa Malaysia

What are systematic reviews?

Watch this video from   Cochrane Consumers and Communication to learn what systematic reviews are, how researchers prepare them, and why they’re an important part of making informed decisions about health - for everyone. 

Cochrane evidence, including our systematic reviews, provides a powerful tool to enhance your healthcare knowledge and decision making. This video from Cochrane Sweden explains a bit about how we create health evidence and what Cochrane does. 

• Search our Plain Language Summaries of health evidence
• Learn more about Cochrane and our work

• Research Process

Systematic Literature Review or Literature Review?

• 3 minute read
• 46.8K views

Table of Contents

As a researcher, you may be required to conduct a literature review. But what kind of review do you need to complete? Is it a systematic literature review or a standard literature review? In this article, we’ll outline the purpose of a systematic literature review, the difference between literature review and systematic review, and other important aspects of systematic literature reviews.

What is a Systematic Literature Review?

The purpose of systematic literature reviews is simple. Essentially, it is to provide a high-level of a particular research question. This question, in and of itself, is highly focused to match the review of the literature related to the topic at hand. For example, a focused question related to medical or clinical outcomes.

The components of a systematic literature review are quite different from the standard literature review research theses that most of us are used to (more on this below). And because of the specificity of the research question, typically a systematic literature review involves more than one primary author. There’s more work related to a systematic literature review, so it makes sense to divide the work among two or three (or even more) researchers.

Your systematic literature review will follow very clear and defined protocols that are decided on prior to any review. This involves extensive planning, and a deliberately designed search strategy that is in tune with the specific research question. Every aspect of a systematic literature review, including the research protocols, which databases are used, and dates of each search, must be transparent so that other researchers can be assured that the systematic literature review is comprehensive and focused.

Most systematic literature reviews originated in the world of medicine science. Now, they also include any evidence-based research questions. In addition to the focus and transparency of these types of reviews, additional aspects of a quality systematic literature review includes:

• Clear and concise review and summary
• Comprehensive coverage of the topic
• Accessibility and equality of the research reviewed

Systematic Review vs Literature Review

The difference between literature review and systematic review comes back to the initial research question. Whereas the systematic review is very specific and focused, the standard literature review is much more general. The components of a literature review, for example, are similar to any other research paper. That is, it includes an introduction, description of the methods used, a discussion and conclusion, as well as a reference list or bibliography.

A systematic review, however, includes entirely different components that reflect the specificity of its research question, and the requirement for transparency and inclusion. For instance, the systematic review will include:

• Eligibility criteria for included research
• A description of the systematic research search strategy
• An assessment of the validity of reviewed research
• Interpretations of the results of research included in the review

As you can see, contrary to the general overview or summary of a topic, the systematic literature review includes much more detail and work to compile than a standard literature review. Indeed, it can take years to conduct and write a systematic literature review. But the information that practitioners and other researchers can glean from a systematic literature review is, by its very nature, exceptionally valuable.

This is not to diminish the value of the standard literature review. The importance of literature reviews in research writing is discussed in this article . It’s just that the two types of research reviews answer different questions, and, therefore, have different purposes and roles in the world of research and evidence-based writing.

Systematic Literature Review vs Meta Analysis

It would be understandable to think that a systematic literature review is similar to a meta analysis. But, whereas a systematic review can include several research studies to answer a specific question, typically a meta analysis includes a comparison of different studies to suss out any inconsistencies or discrepancies. For more about this topic, check out Systematic Review VS Meta-Analysis article.

Language Editing Plus

With Elsevier’s Language Editing Plus services , you can relax with our complete language review of your systematic literature review or literature review, or any other type of manuscript or scientific presentation. Our editors are PhD or PhD candidates, who are native-English speakers. Language Editing Plus includes checking the logic and flow of your manuscript, reference checks, formatting in accordance to your chosen journal and even a custom cover letter. Our most comprehensive editing package, Language Editing Plus also includes any English-editing needs for up to 180 days.

• Publication Recognition

How to Make a PowerPoint Presentation of Your Research Paper

• Manuscript Preparation

What is and How to Write a Good Hypothesis in Research?

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Eccles Health Sciences Library

Spencer s. eccles health sciences library.

• University of Utah
• ULibraries Research Guides
• * Eccles Health Sciences Library Research Guides

Systematic Reviews

• Systematic Review Process: At a Glance
• Learn Systematic Reviews
• Conducting Scoping Reviews
• Types of Evidence Reviews
• ULibraries Evidence Review Services

What are Systematic Reviews?

What is a systematic review?

"A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making." 

- About Cochrane Systematic Reviews , Cochrane Library.

Systematic Review Methodology Requires

Systematic reviews are not individual projects nor are they the purview of clinical experts alone. multiple contributors with varying skills are needed. "the number of individuals with a particular expertise needs to be carefully balanced so that one group of experts is not overly influential. for example, review teams that are too dominated by clinical content experts are more likely to hold preconceived opinions related to the topic of the sr, spend less time conducting the review, and produce lower quality srs (oxman and guyatt, 1993)." (finding what works in health care:standards for systematic reviews, iom.  2011), "the most critical part of a systematic review is asking the right question. if a systematic review is conducted in a methodologically flawless manner, but the clinical question(s) addressed are of little consequence to patients or providers, the review will have marginal clinical utility." umscheid , 2013. pmid: 23697745 .   |   ask a clear question with pico model, develop an  a priori protocol to specify the research methods. a protocol is the "plan or set of steps to be followed in a study. a protocol for a systematic review should describe the rationale for the review, the objectives, and the methods that will be used to locate, select, and critically appraise studies, and to collect and analyse data from the included studies."     cochrane community glossary,  " systematic reviews seek to collate evidence that fits pre-specified eligibility criteria in order to answer a specific research question. they aim to minimize bias by using explicit, systematic methods documented in advance with a protocol."       - cochrane handbook's, ch. 1: introduction, systematic reviews typically take 12-18 months to complete., in addition to assembling your team and developing your protocol, time is needed for:, identification of relevant studies from a number of different sources (including unpublished sources);, selection of studies for inclusion and evaluation of their strengths and limitations on the basis of clear, predefined criteria;, systematic collection of data;, appropriate synthesis of data. , manuscript completion for submission to publication. ​​       source: cochrane handbook    , what authors do - the evidence process.

• Next: Learn Systematic Reviews >>
• Last Updated: May 23, 2024 12:46 PM
• URL: https://campusguides.lib.utah.edu/SystematicReviews
• Youtube EHSL Digital Publishing
• Twitter @EHSLibrary
• Facebook @EcclesLibrary
• Instagram @EHSLibrary
• Email @Digital Publishing Group

Spencer S. Eccles Health Sciences Library, University of Utah, logo

• © 1993 Spencer S. Eccles Health Sciences Library
• The University of Utah
• 10 N. 1900 E. Salt Lake City, UT 84112
• Phone 801.581.5534

• EHSL Website
• UofU Health
• UofU Policies

• Library Hours
• Strategic Plan
• Giving to the Libraries
• Jobs at the Libraries
• Find Your Librarian
• View All →
• Google Scholar
• Research Guides
• Textbook/Reserves
• Government Documents
• Get It For Me
• Print/Copy/Scan
• Renew Materials
• Study Rooms
• Use a Computer
• Borrow Tech Gear
• Student Services
• Faculty Services
• Users with Disabilities
• Visitors & Alumni
• Special Collections
• Find Information

Basics of Systematic Reviews

• About Systematic Review

Types of Reviews

Literature review.

Collects key sources on a topic and discusses those sources in conversation with each other

• Standard for research articles in most disciplines
• Tells the reader what is known, or not known, about a particular issue, topic, or subject
• Demonstrates knowledge and understanding of a topic
• Establishes context or background for a case or argument
• Helps develop the author’s ideas and perspective

Rapid Review

Thorough methodology but with process limitations in place to expeditethe completion of a review.

• For questions that require timely answers
• 3-4 months vs. 12-24 months
• Limitations - scope, comprehensiveness bias, and quality of appraisal
• Discusses potential effects that the limited methods may have had on results

Scoping Review

Determine the scope or coverage of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview of its focus.

• Identify types of available evidence in a given field
• Clarify key concepts/definitions in the literature
• Examine how research is conducted on a certain topic or field
• Identify key factors related to a concept
• Key difference is focus
• Identify and analyze knowledge gaps

Systematic Review

Attempts to identify, appraise, and summarize all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question.

• clearly defined question with inclusion/exclusion criteria
• rigorous and systematic search of the literature
• thorough screening of results
• data extraction and management
• analysis and interpretation of results
• risk of bias assessment of included studies

Meta-Analysis

Used to systematically synthesize or merge the findings of single, independent studies, using statistical methods to calculate an overall or ‘absolute’ effect.

• Combines results from multiple empirical studies
• Requires systematic review first
• Use well recognized, systematic methods to account for differences in sample size, variability (heterogeneity) in study approach and findings (treatment effects)
• Test how sensitive their results are to their own systematic review protocol

For additional types of reviews please see these articles:

• Sutton, A., Clowes, M., Preston, L. and Booth, A. (2019), Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J, 36: 202-222. https://doi.org/10.1111/hir.12276
• Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
• << Previous: About Systematic Review
• Next: Sources >>
• Last Updated: May 17, 2024 10:04 AM
• URL: https://libguides.utsa.edu/systematicreview
• Library Locations
• Staff Directory
• 508 Compliance
• Site Search
• © The University of Texas at San Antonio
• Information: 210-458-4011
• Campus Alerts
• Required Links
• UTSA Policies
• Report Fraud

Indiana University Indianapolis Indiana University Indianapolis IU Indianapolis

• Herron School of Art
• Ruth Lilly Law
• Ruth Lilly Medical
• School of Dentistry

Literature Review - A Self-Guided Tutorial

• Literature Reviews: A Recap
• Reading Journal Articles
• Does it describe a Literature Review?
• 1. Identify the question
• 2. Review discipline styles
• Searching article databases - video
• Finding the article full-text
• Citation chaining
• When to stop searching
• 4. Manage your references
• 5. Critically analyze and evaluate
• 6. Synthesize
• 7. Write literature review

Who's My Librarian?

Locate your University Library's subject librarian  for personalized assistance.

Students doing research in specific areas may also request assistance at other IUPUI libraries:

• IU School of Dentistry Library
• Ruth Lilly Law Library
• Ruth Lilly Medical Library  

What's a Literature Review?

What's a literature review .

A literature review (or lit review, for short) is an in-depth critical analysis of published scholarly research related to a specific topic. Published scholarly research (the "literature") may include journal articles, books, book chapters, dissertations and thesis, or conference proceedings. 

A solid lit review must:

• be organized around and related directly to the thesis or research question you're developing
• synthesize results into a summary of what is and is not known
• identify areas of controversy in the literature
• formulate questions that need further research

• << Previous: Quiz
• Next: Literature Reviews: A Recap >>
• Last Updated: May 16, 2024 10:16 AM
• URL: https://iu.libguides.com/literaturereview

Systematic Literature Review of Cloud Computing Research Between 2010 and 2023

• Conference paper
• First Online: 21 May 2024
• Cite this conference paper

• Shailaja Jha 10 &
• Devina Chaturvedi   ORCID: orcid.org/0009-0004-1242-2099 11  

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 508))

Included in the following conference series:

• Workshop on e-Business

We present a meta-analysis of cloud computing research in information systems. The study includes 152 referenced journal articles published between January 2010 to June 2023. We take stock of the literature and the associated research themes, research frameworks, the employed research methodology, and the geographical distribution of the articles. This review provides holistic insights into trends in cloud computing research based on themes, frameworks, methodology, geographical focus, and future research directions. The results indicate that the extant literature tends to skew toward themes related to business issues, which is an indicator of the maturing and widespread use of cloud computing. This trend is evidenced in the more recent articles published between 2016 to 2023.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

The conference proceedings were primarily used to assess the year-on-year numerical trends in publications, and they have not been used for detailed analysis.

Abdalla Mikhaeil, C., James, T.L.: Examining the case of French hesitancy toward IDaaS solutions: technical and social contextual factors of the organizational IDaaS privacy calculus. Inform. Manage. 60 (4), 103779 (2023)

Google Scholar  

Allen, B., et al.: Software as a service for data scientists. Commun. ACM 55 (2), 81–88 (2012)

Andrade-Rojas, M.G., Kathuria, A., Lee, H.-H.: Multilevel synergy of IT operational integration: competition networks and operating performance. Prod. Oper. Manage. (forthcoming) (2024)

Andrade-Rojas, M.G., Saldanha, T., Kathuria, A., Khuntia, J., Boh, W.F.: How IT overcomes deficiencies for innovation in SMEs: closed innovation versus open innovation. Inform. Syst. Res. (forthcoming) (2024)

Anthes, G.: Security in the cloud. Commun. ACM 53 , 16–18 (2010)

Armbrust, M., et al.: A view of cloud computing. Commun. ACM 53 , 50–58 (2010)

August, T., Niculescu, M.F., Shin, H.: Cloud implications on software network structure and security risks. Inform. Syst. Res. 25 , 489–510 (2014)

Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., Beekhuyzen, J.: Achieving rigor in literature reviews: insights from qualitative data analysis and tool-support. Commun. Assoc. Inform. Syst. 37 (8), 154–204 (2015). http://aisel.aisnet.org/cais/vol37/iss1/8

Benlian, A.: Is traditional, open-source, or on-demand first choice? Developing an AHP-based framework for the comparison of different software models in office suites selection. Eur. J. Inform. Syst. 20 , 542–559 (2011)

Benlian, A., Kettinger, W.J., Sunyaev, A., Winkler, T.J.: Special section: the transformative value of cloud computing: a decoupling, platformization, and recombination theoretical framework. J. Manage. Inform. Syst. 35 , 719–739 (2018)

Benlian, A., Koufaris, M., Hess, T.: The role of SaaS service quality for continued SaaS use: Empirical insights from SaaS using firms (2010)

Bhattacherjee, A., Park, S.C.: Why end-users move to the cloud: a migration-theoretic analysis. Eur. J. Inform. Syst. 23, 357–372 (2014)

Chaturvedi, D., Kathuria, A., Andrade, M., Saldanha, T.: Navigating the Paradox of IT Novelty and Strategic Conformity: The Moderating Role of Industry Dynamism (2023)

Chen, F., Lu, A., Wu, H., Li, M.: Compensation and pricing strategies in cloud service SLAs: considering participants’ risk attitudes and consumer quality perception. Electron. Commerce Res. Appl. 56 , 101215 (2022)

Cheng, H.K., Li, Z., Naranjo, A.: Research note—cloud computing spot pricing dynamics: latency and limits to arbitrage. Inform. Syst. Res. 27 , 145–165 (2016)

Choudhary, V., Vithayathil, J.: The impact of cloud computing: should the IT department be organized as a cost center or a profit center? J. Manage. Inform. Syst. 30 , 67–100 (2013)

Choudhary, V., Zhang, Z.: Research note—patching the cloud: the impact of SaaS on patching strategy and the timing of software release. Inform. Syst. Res. 26 , 845–858 (2015)

Dasgupta, A., Karhade, P., Kathuria, A., Konsynski, B.: Holding space for voices that do not speak: design reform of rating systems for platforms in GREAT economies (2021)

Demirkan, H., Cheng, H.K., Bandyopadhyay, S.: Coordination strategies in an SaaS supply chain. J. Manage. Inform. Syst. 26 , 119–143 (2010)

Demirkan, H., Delen, D.: Leveraging the capabilities of service-oriented decision support systems: putting analytics and big data in cloud. Decis. Support Syst. 55 , 412–421 (2013)

Dierks, L., Seuken, S.: Cloud pricing: the spot market strikes back. Manage. Sci. 68 (1), 105–122 (2022)

Article   Google Scholar  

Ding, S., Xia, C., Wang, C., Desheng, Wu., Zhang, Y.: Multi-objective optimization based ranking prediction for cloud service recommendation. Decis. Support. Syst. 101 , 106–114 (2017)

Dong, L., Shu, W., Sun, D., Li, X., Zhang, L.: Pre-alarm system based on real-time monitoring and numerical simulation using internet of things and cloud computing for tailings dam in mines. IEEE Access 5 , 21080–21089 (2017)

Xin, Du., Tang, S., Zhihui, Lu., Gai, K., Jie, Wu., Hung, P.C.K.: Scientific workflows in IoT environments: a data placement strategy based on heterogeneous edge-cloud computing. ACM Trans. Manage. Inform. Syst. 13 (4), 1–26 (2022)

Ermakova, T., Fabian, B., Kornacka, M., Thiebes, S., Sunyaev, A.: Security and privacy requirements for cloud computing in healthcare: elicitation and prioritization from a patient perspective. ACM Trans. Manage. Inform. Syst. 11 (2), 1–29 (2020)

Garrison, G., Kim, S., Wakefield, R.L.: Success factors for deploying cloud computing. Commun. ACM 55 (9), 62–68 (2012)

Giessmann, A., Legner, C.: Designing business models for cloud platforms. Inf. Syst. J. 26 (5), 551–579 (2016). https://doi.org/10.1111/isj.12107

Gray, A.: Conflict of laws and the cloud. Comput. Law Secur. Rev. 29 (1), 58–65 (2013)

Hosseini, L., Tang, S., Mookerjee, V., Sriskandarajah, C.: A switch in time saves the dime: a model to reduce rental cost in cloud computing. Inform. Syst. Res. 31 (3), 753–775 (2020)

Huang, K.-W., Sundararajan, A.: Pricing digital goods: discontinuous costs and shared infrastructure. Inf. Syst. Res. 22 (4), 721–738 (2011)

Iosup, A., Ostermann, S., Yigitbasi, M.N., Prodan, R., Fahringer, T., Epema, D.H.J.: Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans. Parallel Distrib. Syst. 22 , 931–945 (2011)

Iyer, B., Henderson, J.C.: Preparing for the future: understanding the seven capabilities cloud computing. MIS Q. Exec. 9 , 2 (2010)

Jha, S. and Kathuria, A. Size Matters for Cloud Capability and Performance (2022)

Jha, S., Kathuria, A.: How firm age and size influence value creation from cloud computing (2023)

Joe-Wong, C., Sen, S.: Harnessing the power of the cloud: revenue, fairness, and cloud neutrality. J. Manage. Inf. Syst. 35 , 813–836 (2018)

Joint, A., Baker, E.: Knowing the past to understand the present–issues in the contracting for cloud based services. Comput. Law Secur. Rev. 27 (4), 407–415 (2011)

Karhade, P., Kathuria, A.: Missing impact of ratings on platform participation in India: a call for research in GREAT domains. Commun. Assoc. Inf. Syst. 47 (1), 19 (2020)

Karhade, P., Kathuria, A., Dasgupta, A., Malik, O., Konsynski, B.R.: Decolonization of digital platforms: a research agenda for GREAT domains. In: Garimella, A., Karhade, P., Kathuria, A., Liu, X., Xu, J., Zhao, K. (eds.) The Role of e-Business during the Time of Grand Challenges. LNBIP, vol. 418, pp. 51–58. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79454-5_5

Chapter   Google Scholar  

Karhade, P., Kathuria, A., Konsynski, B.: When choice matters: assortment and participation for performance on digital platforms (2021)

Kathuria, A., Karhade, P.P., Konsynski, B.R.: In the realm of hungry ghosts: multi-level theory for supplier participation on digital platforms. J. Manag. Inf. Syst. 37 (2), 396–430 (2020)

Kathuria, A., Mann, A., Khuntia, J., Saldanha, T.J.V., Kauffman, R.J.: A strategic value appropriation path for cloud computing. J. Manage. Inf. Syst. 35 (3), 740–775 (2018). https://doi.org/10.1080/07421222.2018.1481635

Kaur, J., Kaur, P.D.: CE-GMS: A cloud IoT-enabled grocery management system. Electron. Commer. Res. Appl. 28 , 63–72 (2018)

Kepes, B.: 30% of servers are sitting “Comatose” according to research. Forbes https://forbes.com/sites/benkepes/2015/06/03/30-of-servers-are-sitting-comatose-according-to-research (2015)

Khokhar, R.H., Fung, B.C.M., Iqbal, F., Alhadidi, D., Bentahar, J.: Privacy-preserving data mashup model for trading person-specific information. Electron. Commer. Res. Appl. 17 , 19–37 (2016)

Khuntia, J., Kathuria, A., Andrade-Rojas, M.G., Saldanha, T., Celly, N.: How foreign and domestic firms differ in leveraging IT-enabled supply chain information integration in BOP markets: the role of supplier and client business collaboration. J. Assoc. Inf. Syst. 22 (3), 6 (2021)

King, W.R., He, J.: Understanding the role and methods of meta-analysis in IS Research. Commun. Assoc. Inf. Syst. 16, 665–686 (2005)

Krancher, O., Luther, P., Jost, M.: Key affordances of Platform-as-a-Service: self-organization and continuous feedback. J. Manage. Inf. Syst. 35 , 776–812 (2018)

Kumar, C., Marston, S., Sen, R., Narisetty, A.: Greening the cloud: a load balancing mechanism to optimize cloud computing networks. J. Manage. Inf. Syst. 39 ,, 513–541 (2022)

Kung, L., Cegielski, C.G., Kung, H.-J.: An integrated environmental perspective on software as a service adoption in manufacturing and retail firms. J. Inf. Technol. 30 , 352–363 (2015)

Lansing, J., Benlian, A., Sunyaev, A.: Unblackboxing” decision makers’ interpretations of IS certifications in the context of cloud service certifications. J. Assoc. Inf. Syst. 19 (11), 1064–1096 (2018)

Lansing, J., Siegfried, N., Sunyaev, A., Benlian, A.: Strategic signaling through cloud service certifications: Comparing the relative importance of certifications’ assurances to companies and consumers. J. Strateg. Inf. Syst. 28 , 101579 (2019)

Lansing, J., Sunyaev, A.: Trust in cloud computing. ACM SIGMIS Database DATABASE Adv. Inform. Syst. 47 , 58–96 (2016)

Lee, J., Cho, D., Lim, G.: Design and validation of the bright internet. J. Assoc. Inform. Syst. 19 , 63–85 (2018)

Lee, M.H., Han, S.P., Park, S., Oh, W.: Positive demand spillover of popular app adoption: implications for platform owners’ management of complements. Inf. Syst. Res. 34 (3), 961–995 (2023)

Li, S., Chen, W., Chen, Y., Chen, C. and Zheng, Z.: Makespan-minimized computation offloading for smart toys in edge-cloud computing. Electron. Commerce Res. Appl. 37 , 100884 (2019)

Li, S., Cheng, H.K., Duan, Y., Yang, Y.-C.: A study of enterprise software licensing models. J. Manag. Inf. Syst. 34 (1), 177–205 (2017)

Lins, S., Schneider, S., Szefer, J., Ibraheem, S., Ali, A.: Designing monitoring systems for continuous certification of cloud services: deriving meta-requirements and design guidelines. Commun. Assoc. Inf. Syst. 44 (1), 460–510 (2019)

Liu, Y., Sheng, X., Marston, S.R.: The impact of client-side security restrictions on the competition of cloud computing services. Int. J. Electron. Comm. 19 (3), 90–117 (2015)

Ma, D., Seidmann, A.: Analyzing software as a service with per-transaction charges. Inf. Syst. Res. 26 , 360–378 (2015)

Malik, O., Jaiswal, A., Kathuria, A., Karhade, P.: Leveraging BI systems to overcome infobesity: a comparative analysis of incumbent and new entrant firms (2022)

Mani, D., Srikanth, K., Bharadwaj, A.: Efficacy of R&D work in offshore captive centers: an empirical study of task characteristics, coordination mechanisms, and performance. Inf. Syst. Res. 25 (4), 846–864 (2014)

Mann, A., Kathuria, A., Khuntia, J., Saldanha, T.: Cloud-integration and business flexibility: the mediating role of cloud functional capabilities (2016)

Marston, S., Li, Z., Bandyopadhyay, S., Zhang, J., Ghalsasi, A.: Cloud computing — the business perspective. Decis. Support. Syst. 51 (1), 176–189 (2011)

Mell, P.M., Grance, T.: The NIST definition of cloud computing. National Institute of Standards and Technology (2011)

Metz, C.: The epic story of dropboxs exodus from the amazon cloud empire (2016)

Mithas, R., Sambamurthy,: How information management capability influences firm performance. MIS Q. 35 (1), 237 (2011)

Mithas, T., Bardhan, G.: Information technology and firm profitability: mechanisms and empirical evidence. MIS Q. 36 (1), 205 (2012)

Muhic, M., Bengtsson, L., Holmström, J.: Barriers to continuance use of cloud computing: evidence from two case studies. Inf. Manage. 60 , 103792 (2023)

Mukherjee, A., Sundarraj, R.P., Dutta, K.: Time-preference-based on-spot bundled cloud-service provisioning. Decis. Support. Syst. 151 , 113607 (2021)

Müller, S.D., Holm, S.R., Søndergaard, J.: Benefits of cloud computing: literature review in a maturity model perspective. Commun. Assoc. Inform. Syst. 37 , 851–878 (2015)

Ojala, A.: Business models and opportunity creation: how IT entrepreneurs create and develop business models under uncertainty. Inf. Syst. J. 26 , 451–476 (2015)

Oliveira, T., Thomas, M., Espadanal, M.: Assessing the determinants of cloud computing adoption: An analysis of the manufacturing and services sectors. Inf. Manage. 51 , 497–510 (2014)

Owens, D. Securing elasticity in the cloud. Communications of the ACM , 53, 6 (2010/06 2010), 46–51 (2010)

Pang, M.-S., Tanriverdi, H.: Strategic roles of IT modernization and cloud migration in reducing cybersecurity risks of organizations: the case of U.S. federal government. J. Strat. Inf. Syst. 31 , 101707 (2022)

Park, J., Han, K., Lee, B.: Green cloud? An empirical analysis of cloud computing and energy efficiency. Manage. Sci. 69 , 1639–1664 (2023)

Parno, B., Howell, J., Gentry, C., Raykova, M.: Pinocchio. Commun. ACM 59 , 103–112 (2016)

Pye, J., Rai, A., Dong, J.Q.: Business value of information technology capabilities: an institutional governance perspective. Inf. Syst. Res. 35 , 28–44 (2023)

Ramakrishnan, T., Kathuria, A., Khuntia, J., Konsynski, B.: IoT value creation through supply chain analytics capability (2022)

Retana, G., Forman, C., Narasimhan, S., Niculescu, M.F., Wu, D.J.: Technical support, knowledge transfer, and service demand: evidence from the cloud. SSRN Electron. J. (2012)

Rodrigues, J., Ruivo, P., Oliveira, T.: Mediation role of business value and strategy in firm performance of organizations using software-as-a-service enterprise applications. Inf. Manag. 58 (1), 103289 (2021)

Saldanha, T.J., Andrade-Rojas, M.G., Kathuria, A., Khuntia, J., Krishnan, M.: How the locus of uncertainty shapes the influence of CEO long-term compensation on IT capital investments. MIS Q. (2023)

Sambhara, C., Rai, A., Xu, S.X.: Configuring the enterprise systems portfolio: the role of information risk. Inf. Syst. Res. 33 (2), 446–463 (2022)

Sarker, S., Chatterjee, S., Xiao, X., Elbanna, A.: The sociotechnical axis of cohesion for the IS discipline: its historical legacy and its continued relevance. MIS Q. 43 (3), 695–720 (2019)

Schlagwein, D., Thorogood, A., Willcocks, L.P.: How commonwealth bank of Australia gained benefits using a standards-based, multi-provider cloud model. MIS Q. Exec. 13 (4), 209–222 (2014)

Schneider, S., Sunyaev, A.: Determinant factors of cloud-sourcing decisions: reflecting on the IT outsourcing literature in the era of cloud computing. J. Inf. Technol. 31 (1), 1–31 (2016). https://doi.org/10.1057/jit.2014.25

Schneider, S., Wollersheim, J., Krcmar, H., Sunyaev, A.: How do Requirements evolve over Time? A case study investigating the role of context and experiences in the evolution of enterprise software requirements. J. Inf. Technol. 33 (2), 151–170 (2018)

Schniederjans, D.G., Hales, D.N.: Cloud computing and its impact on economic and environmental performance: a transaction cost economics perspective. Decis. Support. Syst. 86 , 73–82 (2016)

Schreieck, M., Wiesche, M., Krcmar, H.: Capabilities for value co-creation and value capture in emergent platform ecosystems: a longitudinal case study of SAP’s cloud platform. J. Inf. Technol. 36 (4), 365–390 (2021)

Shiau, W.-L., Chau, P.Y.K.: Understanding behavioral intention to use a cloud computing classroom: a multiple model comparison approach. Inf. Manag. 53 (3), 355–365 (2016)

Singh, V.K., Shivendu, S., Dutta, K.: Spot instance similarity and substitution effect in cloud spot market. Decis. Support. Syst. 159 , 113815 (2022)

Soh, F., Setia, P.: The impact of dominant IT infrastructure in multi-establishment firms: the moderating role of environmental dynamism. J. Assoc. Inf. Syst. 23 (6), 1603–1633 (2022)

Son, I., Lee, D., Lee, J.-N., Chang, Y.B.: Market perception on cloud computing initiatives in organizations: an extended resource-based view. Inf. Manag. 51 (6), 653–669 (2014)

Srinivasan, S.: Is security realistic in cloud computing? J. Int. Technol. Inf. Manag. 22 (4), 3 (2013). https://doi.org/10.58729/1941-6679.1020

Article   MathSciNet   Google Scholar  

Sun, T., Shi, L., Viswanathan, S., Zheleva, E.: Motivating effective mobile app adoptions: evidence from a large-scale randomized field experiment. Inf. Syst. Res. 30 (2), 523–539 (2019)

Templier, M., Paré, G.: Transparency in literature reviews: an assessment of reporting practices across review types and genres in top IS journals. Eur. J. Inf. Syst. 27 (5), 503–550 (2017). https://doi.org/10.1080/0960085X.2017.1398880

Trenz, M., Huntgeburth, J., Veit, D.: Uncertainty in cloud service relationships: uncovering the differential effect of three social influence processes on potential and current users. Inf. Manage. 55, 971–983 (2018)

van de Weerd, I., Mangula, I.S., Brinkkemper, S.: Adoption of software as a service in Indonesia: examining the influence of organizational factors. Inf. Manage. 53 (7), 915–928 (2016)

Venkatesh, V., Bala, H., Sambamurthy, V.: Implementation of an information and communication technology in a developing country: a multimethod longitudinal study in a Bank in India. Inf. Syst. Res. 27 (3), 558–579 (2016)

Venkatesh, V., Sykes, T.A.: Digital divide initiative success in developing countries: a longitudinal field study in a Village in India. Inf. Syst. Res. 24 (2), 239–260 (2013)

Venters, W., Whitley, E.A.: A critical review of cloud computing: researching desires and realities. J. Inf. Technol. 27 (3), 179–197 (2012)

Wang, N., Huigang Liang, Yu., Jia, S.G., Xue, Y., Wang, Z.: Cloud computing research in the IS discipline: a citation/co-citation analysis. Decis. Support. Syst. 86 , 35–47 (2016)

Wang, X., Wang, X.: Multimedia data delivery based on IoT clouds. Commun. ACM 64 (8), 80–86 (2021)

Winkler, T.J., Benlian, A., Piper, M., Hirsch, H.: Bayer healthcare delivers a dose of reality for cloud payoff mantras in multinationals. MIS Q. Exec. 13 , 4 (2014)

Winkler, T.J., Brown, C.V.: Horizontal allocation of decision rights for on-premise applications and Software-as-a-Service. J. Manage. Inf. Syst. 30 (3), 13–48 (2013)

Wright, R.T., Roberts, N., Wilson, D.: The role of context in IT assimilation: a multi-method study of a SaaS platform in the US nonprofit sector. Eur. J. Inf. Syst. 26 (5), 509–539 (2017). https://doi.org/10.1057/s41303-017-0053-2

Wulf, F., Lindner, T., Strahringer, S., Westner, M.: IaaS, PaaS, or SaaS? The why of cloud computing delivery model selection: vignettes on the post-adoption of cloud computing. In: The Proceedings of Proceedings of the 54th Hawaii International Conference on System Sciences, pp. 6285–6294 (2021)

Xiong, Hu., Wang, Yi., Li, W., Chen, C.-M.: Flexible, efficient, and secure access delegation in cloud computing. ACM Trans. Manage. Inf. Syst. 10 (1), 1–20 (2019)

Yang, H., Tate, M.: A descriptive literature review and classification of cloud computing research. Commun. Assoc. Inf. Syst. 31 (1), 2 (2012)

Yaraghi, N., Du, A.Y., Sharman, R., Gopal, R.D., Ramesh, R.: Health Information exchange as a multisided platform: adoption, usage, and practice involvement in service co-production. Inf. Syst. Res. 26 (1), 1–18 (2015)

Yuan, S., Sanjukta Das, R., Ramesh, C.Q.: Service agreement trifecta: backup resources, price and penalty in the availability-aware cloud. Inf. Syst. Res. 29 (4), 947–964 (2018)

Zhang, G., Ravishankar, M.N.: Exploring vendor capabilities in the cloud environment: a case study of Alibaba cloud computing. Inf. Manage. 56 , 343–355 (2019)

Zhang, X., Yue, W.: Integration of on-premises and cloud-based software: the product bundling perspective. J. Assoc. Inform. Syst. 21 , 1507–1551 (2020)

Zorrilla, M., García-Saiz, D.: A service oriented architecture to provide data mining services for non-expert data miners. Decis. Support. Syst.. Support. Syst. 55 (1), 399–411 (2013). https://doi.org/10.1016/j.dss.2012.05.045

Download references

Author information

Authors and affiliations.

SP Jain Institute of Management and Research, Mumbai, India

Shailaja Jha

Indian School of Business, Hyderabad, India

Devina Chaturvedi

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Devina Chaturvedi .

Editor information

Editors and affiliations.

#6104, Indian School of Business, Hyderabad, Telangana, India

Abhishek Kathuria

Chinese University of Hong Kong, Sha Tin District, Hong Kong

Prasanna P. Karhade

University of North Carolina at Charlotte, Charlotte, NC, USA

Indian School of Business, Hyderabad, Telangana, India

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Jha, S., Chaturvedi, D. (2024). Systematic Literature Review of Cloud Computing Research Between 2010 and 2023. In: Kathuria, A., Karhade, P.P., Zhao, K., Chaturvedi, D. (eds) Digital Transformation in the Viral Age. WeB 2022. Lecture Notes in Business Information Processing, vol 508. Springer, Cham. https://doi.org/10.1007/978-3-031-60003-6_5

Download citation

DOI : https://doi.org/10.1007/978-3-031-60003-6_5

Published : 21 May 2024

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-60002-9

Online ISBN : 978-3-031-60003-6

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Family Med Prim Care
• v.2(1); Jan-Mar 2013

Systematic Reviews and Meta-analysis: Understanding the Best Evidence in Primary Healthcare

S. gopalakrishnan.

Department of Community Medicine, SRM Medical College, Hospital and Research Centre, Kattankulathur, Tamil Nadu, India

P. Ganeshkumar

Healthcare decisions for individual patients and for public health policies should be informed by the best available research evidence. The practice of evidence-based medicine is the integration of individual clinical expertise with the best available external clinical evidence from systematic research and patient's values and expectations. Primary care physicians need evidence for both clinical practice and for public health decision making. The evidence comes from good reviews which is a state-of-the-art synthesis of current evidence on a given research question. Given the explosion of medical literature, and the fact that time is always scarce, review articles play a vital role in decision making in evidence-based medical practice. Given that most clinicians and public health professionals do not have the time to track down all the original articles, critically read them, and obtain the evidence they need for their questions, systematic reviews and clinical practice guidelines may be their best source of evidence. Systematic reviews aim to identify, evaluate, and summarize the findings of all relevant individual studies over a health-related issue, thereby making the available evidence more accessible to decision makers. The objective of this article is to introduce the primary care physicians about the concept of systematic reviews and meta-analysis, outlining why they are important, describing their methods and terminologies used, and thereby helping them with the skills to recognize and understand a reliable review which will be helpful for their day-to-day clinical practice and research activities.

Introduction

Evidence-based healthcare is the integration of best research evidence with clinical expertise and patient values. Green denotes, “Using evidence from reliable research, to inform healthcare decisions, has the potential to ensure best practice and reduce variations in healthcare delivery.” However, incorporating research into practice is time consuming, and so we need methods of facilitating easy access to evidence for busy clinicians.[ 1 ] Ganeshkumar et al . mentioned that nearly half of the private practitioners in India were consulting more than 4 h per day in a locality,[ 2 ] which explains the difficulty of them in spending time in searching evidence during consultation. Ideally, clinical decision making ought to be based on the latest evidence available. However, to keep abreast with the continuously increasing number of publications in health research, a primary healthcare professional would need to read an insurmountable number of articles every day, covered in more than 13 million references and over 4800 biomedical and health journals in Medline alone. With the view to address this challenge, the systematic review method was developed. Systematic reviews aim to inform and facilitate this process through research synthesis of multiple studies, enabling increased and efficient access to evidence.[ 1 , 3 , 4 ]

Systematic reviews and meta-analyses have become increasingly important in healthcare settings. Clinicians read them to keep up-to-date with their field and they are often used as a starting point for developing clinical practice guidelines. Granting agencies may require a systematic review to ensure there is justification for further research and some healthcare journals are moving in this direction.[ 5 ]

This article is intended to provide an easy guide to understand the concept of systematic reviews and meta-analysis, which has been prepared with the aim of capacity building for general practitioners and other primary healthcare professionals in research methodology and day-to-day clinical practice.

The purpose of this article is to introduce readers to:

• The two approaches of evaluating all the available evidence on an issue i.e., systematic reviews and meta-analysis,
• Discuss the steps in doing a systematic review,
• Introduce the terms used in systematic reviews and meta-analysis,
• Interpret results of a meta-analysis, and
• The advantages and disadvantages of systematic review and meta-analysis.

Application

What is the effect of antiviral treatment in dengue fever? Most often a primary care physician needs to know convincing answers to questions like this in a primary care setting.

To find out the solutions or answers to a clinical question like this, one has to refer textbooks, ask a colleague, or search electronic database for reports of clinical trials. Doctors need reliable information on such problems and on the effectiveness of large number of therapeutic interventions, but the information sources are too many, i.e., nearly 20,000 journals publishing 2 million articles per year with unclear or confusing results. Because no study, regardless of its type, should be interpreted in isolation, a systematic review is generally the best form of evidence.[ 6 ] So, the preferred method is a good summary of research reports, i.e., systematic reviews and meta-analysis, which will give evidence-based answers to clinical situations.

There are two fundamental categories of research: Primary research and secondary research. Primary research is collecting data directly from patients or population, while secondary research is the analysis of data already collected through primary research. A review is an article that summarizes a number of primary studies and may draw conclusions on the topic of interest which can be traditional (unsystematic) or systematic.

Terminologies

Systematic review.

A systematic review is a summary of the medical literature that uses explicit and reproducible methods to systematically search, critically appraise, and synthesize on a specific issue. It synthesizes the results of multiple primary studies related to each other by using strategies that reduce biases and random errors.[ 7 ] To this end, systematic reviews may or may not include a statistical synthesis called meta-analysis, depending on whether the studies are similar enough so that combining their results is meaningful.[ 8 ] Systematic reviews are often called overviews.

The evidence-based practitioner, David Sackett, defines the following terminologies.[ 3 ]

• Review: The general term for all attempts to synthesize the results and conclusions of two or more publications on a given topic.
• Overview: When a review strives to comprehensively identify and track down all the literature on a given topic (also called “systematic literature review”).
• Meta-analysis: A specific statistical strategy for assembling the results of several studies into a single estimate.

Systematic reviews adhere to a strict scientific design based on explicit, pre-specified, and reproducible methods. Because of this, when carried out well, they provide reliable estimates about the effects of interventions so that conclusions are defensible. Systematic reviews can also demonstrate where knowledge is lacking. This can then be used to guide future research. Systematic reviews are usually carried out in the areas of clinical tests (diagnostic, screening, and prognostic), public health interventions, adverse (harm) effects, economic (cost) evaluations, and how and why interventions work.[ 9 ]

Cochrane reviews

Cochrane reviews are systematic reviews undertaken by members of the Cochrane Collaboration which is an international not-for-profit organization that aims to help people to make well-informed decisions about healthcare by preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of healthcare interventions.

Cochrane Primary Health Care Field is a systematic review of primary healthcare research on prevention, treatment, rehabilitation, and diagnostic test accuracy. The overall aim and mission of the Primary Health Care Field is to promote the quality, quantity, dissemination, accessibility, applicability, and impact of Cochrane systematic reviews relevant to people who work in primary care and to ensure proper representation in the interests of primary care clinicians and consumers in Cochrane reviews and review groups, and in other entities. This field would serve to coordinate and promote the mission of the Cochrane Collaboration within the primary healthcare disciplines, as well as ensuring that primary care perspectives are adequately represented within the Collaboration.[ 10 ]

Meta-analysis

A meta-analysis is the combination of data from several independent primary studies that address the same question to produce a single estimate like the effect of treatment or risk factor. It is the statistical analysis of a large collection of analysis and results from individual studies for the purpose of integrating the findings.[ 11 ] The term meta-analysis has been used to denote the full range of quantitative methods for research reviews.[ 12 ] Meta-analyses are studies of studies.[ 13 ] Meta-analysis provides a logical framework to a research review where similar measures from comparable studies are listed systematically and the available effect measures are combined wherever possible.[ 14 ]

The fundamental rationale of meta-analysis is that it reduces the quantity of data by summarizing data from multiple resources and helps to plan research as well as to frame guidelines. It also helps to make efficient use of existing data, ensuring generalizability, helping to check consistency of relationships, explaining data inconsistency, and quantifies the data. It helps to improve the precision in estimating the risk by using explicit methods.

Therefore, “systematic review” will refer to the entire process of collecting, reviewing, and presenting all available evidence, while the term “meta-analysis” will refer to the statistical technique involved in extracting and combining data to produce a summary result.[ 15 ]

Steps in doing systematic reviews/meta-analysis

Following are the six fundamental essential steps while doing systematic review and meta-analysis.[ 16 ]

Define the question

This is the most important part of systematic reviews/meta-analysis. The research question for the systematic reviews may be related to a major public health problem or a controversial clinical situation which requires acceptable intervention as a possible solution to the present healthcare need of the community. This step is most important since the remaining steps will be based on this.

Reviewing the literature

This can be done by going through scientific resources such as electronic database, controlled clinical trials registers, other biomedical databases, non-English literatures, “gray literatures” (thesis, internal reports, non–peer-reviewed journals, pharmaceutical industry files), references listed in primary sources, raw data from published trials and other unpublished sources known to experts in the field. Among the available electronic scientific database, the popular ones are PUBMED, MEDLINE, and EMBASE.

Sift the studies to select relevant ones

To select the relevant studies from the searches, we need to sift through the studies thus identified. The first sift is pre-screening, i.e., to decide which studies to retrieve in full, and the second sift is selection which is to look again at these studies and decide which are to be included in the review. The next step is selecting the eligible studies based on similar study designs, year of publication, language, choice among multiple articles, sample size or follow-up issues, similarity of exposure, and or treatment and completeness of information.

It is necessary to ensure that the sifting includes all relevant studies like the unpublished studies (desk drawer problem), studies which came with negative conclusions or were published in non-English journals, and studies with small sample size.

Assess the quality of studies

The steps undertaken in evaluating the study quality are early definition of study quality and criteria, setting up a good scoring system, developing a standard form for assessment, calculating quality for each study, and finally using this for sensitivity analysis.

For example, the quality of a randomized controlled trial can be assessed by finding out the answers to the following questions:

• Was the assignment to the treatment groups really random?
• Was the treatment allocation concealed?
• Were the groups similar at baseline in terms of prognostic factors?
• Were the eligibility criteria specified?
• Were the assessors, the care provider, and the patient blinded?
• Were the point estimates and measure of variability presented for the primary outcome measure?
• Did the analyses include intention-to-treat analysis?

Calculate the outcome measures of each study and combine them

We need a standard measure of outcome which can be applied to each study on the basis of its effect size. Based on their type of outcome, following are the measures of outcome: Studies with binary outcomes (cured/not cured) have odds ratio, risk ratio; studies with continuous outcomes (blood pressure) have means, difference in means, standardized difference in means (effect sizes); and survival or time-to-event data have hazard ratios.

Combining studies

Homogeneity of different studies can be estimated at a glance from a forest plot (explained below). For example, if the lower confidence interval of every trial is below the upper of all the others, i.e., the lines all overlap to some extent, then the trials are homogeneous. If some lines do not overlap at all, these trials may be said to be heterogeneous.

The definitive test for assessing the heterogeneity of studies is a variant of Chi-square test (Mantel–Haenszel test). The final step is calculating the common estimate and its confidence interval with the original data or with the summary statistics from all the studies. The best estimate of treatment effect can be derived from the weighted summary statistics of all studies which will be based on weighting to sample size, standard errors, and other summary statistics. Log scale is used to combine the data to estimate the weighting.

Interpret results: Graph

The results of a meta-analysis are usually presented as a graph called forest plot because the typical forest plots appear as forest of lines. It provides a simple visual presentation of individual studies that went into the meta-analysis at a glance. It shows the variation between the studies and an estimate of the overall result of all the studies together.

Forest plot

Meta-analysis graphs can principally be divided into six columns [ Figure 1 ]. Individual study results are displayed in rows. The first column (“study”) lists the individual study IDs included in the meta-analysis; usually the first author and year are displayed. The second column relates to the intervention groups and the third column to the control groups. The fourth column visually displays the study results. The line in the middle is called “the line of no effect.” The weight (in %) in the fifth column indicates the weighting or influence of the study on the overall results of the meta-analysis of all included studies. The higher the percentage weight, the bigger the box, the more influence the study has on the overall results. The sixth column gives the numerical results for each study (e.g., odds ratio or relative risk and 95% confidence interval), which are identical to the graphical display in the fourth column. The diamond in the last row of the graph illustrates the overall result of the meta-analysis.[ 4 ]

Interpretation of meta-analysis[ 4 ]

Thus, the horizontal lines represent individual studies. Length of line is the confidence interval (usually 95%), squares on the line represent effect size (risk ratio) for the study, with area of the square being the study size (proportional to weight given) and position as point estimate (relative risk) of the study.[ 7 ]

For example, the forest plot of the effectiveness of dexamethasone compared with placebo in preventing the recurrence of acute severe migraine headache in adults is shown in Figure 2 .[ 17 ]

Forest plot of the effectiveness of dexamethasone compared with placebo in preventing the recurrence of acute severe migraine headache in adults[ 17 ]

The overall effect is shown as diamond where the position toward the center represents pooled point estimate, the width represents estimated 95% confidence interval for all studies, and the black plain line vertically in the middle of plot is the “line of no effect” (e.g., relative risk = 1).

Therefore, when examining the results of a systematic reviews/meta-analysis, the following questions should be kept in mind:

• Heterogeneity among studies may make any pooled estimate meaningless.
• The quality of a meta-analysis cannot be any better than the quality of the studies it is summarizing.
• An incomplete search of the literature can bias the findings of a meta-analysis.
• Make sure that the meta-analysis quantifies the size of the effect in units that you can understand.

Subgroup analysis and sensitivity analysis

Subgroup analysis looks at the results of different subgroups of trials, e.g., by considering trials on adults and children separately. This should be planned at the protocol stage itself which is based on good scientific reasoning and is to be kept to a minimum.

Sensitivity analysis is used to determine how results of a systematic review/meta-analysis change by fiddling with data, for example, what is the implication if the exclusion criteria or excluded unpublished studies or weightings are assigned differently. Thus, after the analysis, if changing makes little or no difference to the overall results, the reviewer's conclusions are robust. If the key findings disappear, then the conclusions need to be expressed more cautiously.

Advantages of Systematic Reviews

Systematic reviews have specific advantages because of using explicit methods which limit bias, draw reliable and accurate conclusions, easily deliver required information to healthcare providers, researchers, and policymakers, help to reduce the time delay in the research discoveries to implementation, improve the generalizability and consistency of results, generation of new hypotheses about subgroups of the study population, and overall they increase precision of the results.[ 18 ]

Limitations in Systematic Reviews/Meta-analysis

As with all research, the value of a systematic review depends on what was done, what was found, and the clarity of reporting. As with other publications, the reporting quality of systematic reviews varies, limiting readers’ ability to assess the strengths and weaknesses of those reviews.[ 5 ]

Even though systematic review and meta-analysis are considered the best evidence for getting a definitive answer to a research question, there are certain inherent flaws associated with it, such as the location and selection of studies, heterogeneity, loss of information on important outcomes, inappropriate subgroup analyses, conflict with new experimental data, and duplication of publication.

Publication Bias

Publication bias results in it being easier to find studies with a “positive” result.[ 19 ] This occurs particularly due to inappropriate sifting of the studies where there is always a tendency towards the studies with positive (significant) outcomes. This effect occurs more commonly in systematic reviews/meta-analysis which need to be eliminated.

The quality of reporting of systematic reviews is still not optimal. In a recent review of 300 systematic reviews, few authors reported assessing possible publication bias even though there is overwhelming evidence both for its existence and its impact on the results of systematic reviews. Even when the possibility of publication bias is assessed, there is no guarantee that systematic reviewers have assessed or interpreted it appropriately.[ 20 ]

To overcome certain limitations mentioned above, the Cochrane reviews are currently reported in a format where at the end of every review, findings are summarized in the author's point of view and also give an overall picture of the outcome by means of plain language summary. This is found to be much helpful to understand the existing evidence about the topic more easily by the reader.

A systematic review is an overview of primary studies which contains an explicit statement of objectives, materials, and methods, and has been conducted according to explicit and reproducible methodology. A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way. Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the reviews were valid and reliable.

High-quality systematic reviews and meta-analyses take great care to find all relevant studies, critically assess each study, synthesize the findings from individual studies in an unbiased manner, and present balanced important summary of findings with due consideration of any flaws in the evidence. Systematic review and meta-analysis is a way of summarizing research evidence, which is generally the best form of evidence, and hence positioned at the top of the hierarchy of evidence.

Systematic reviews can be very useful decision-making tools for primary care/family physicians. They objectively summarize large amounts of information, identifying gaps in medical research, and identifying beneficial or harmful interventions which will be useful for clinicians, researchers, and even for public and policymakers.

Source of Support: Nil

Conflict of Interest: None declared.

A systematic review of literature examining mediators and mechanisms of change in empirically supported treatments for posttraumatic stress disorder

This systematic review summarized studies published before October 2022 that examined mediators/mechanisms of change in empirically supported treatments (ESTs) for PTSD.

Article Abstract

“Despite the availability of empirically supported  treatments  (ESTs) for  posttraumatic stress disorder  (PTSD), relatively little is known regarding these treatments’ mechanisms of change. This  systematic review  moves beyond previous reviews by summarizing the findings and reviewing the methodological quality of literature that specifically examined mediators/mechanisms of change in ESTs for PTSD. Studies were included if they were written in English, empirical, peer-reviewed, claimed to study mediators/mechanisms of a recommended PTSD treatment, measured the mediator/mechanism during or before and after treatment, and included a posttreatment PTSD or global outcome (e.g., functioning). PsycINFO and PubMed were searched on October 7, 2022. Two coders screened and coded studies. Sixty-two eligible studies were identified. The most consistent mediator/mechanism was reduction in negative posttraumatic cognitions, followed by between-session extinction and decreased depression. Only 47% of studies measured the mediator/mechanism before the outcome and measured the mediator/mechanism and outcome at least three times, and 32% also used  growth curve  modeling to establish temporal precedence of change in the mediator/mechanism and outcome. Many of the mediators/mechanisms examined had weak or no empirical support. Results highlight the need for improved methodological rigor in treatment mediator and mechanism research. Implications for clinical care and research are discussed. PROSPERO ID: 248088.”

—Description from publisher

Article Access

Purchase/Subscription Required

Alpert, E., Shotwell Tabke, C., Cole, T. A., Lee, D. J., & Sloan, D. M. (2023). A systematic review of literature examining mediators and mechanisms of change in empirically supported treatments for posttraumatic stress disorder. Clinical Psychology Review, 103. https://doi.org/10.1016/j.cpr.2023.102300

Date July 24, 2023

Creator(s) Elizabeth Alpert , Chelsea Shotwell Tabke, Travis A. Cole

Contributor(s) Daniel J. Lee, Denise, M. Sloan

Topics PTSD

Publisher Elsevier

APA Citation Alpert, E., Shotwell Tabke, C., Cole, T. A., Lee, D. J., & Sloan, D. M. (2023). A systematic review of literature examining mediators and mechanisms of change in empirically supported treatments for posttraumatic stress disorder. Clinical Psychological Review, 103. https://doi.org/10.1016/j.cpr.2023.102300

Audience EMDR Therapists, Other Mental Health Professionals

Language English

Content Type Article, Meta-analyses/Systematic Reviews, Peer-Reviewed

Access Type External Resource

Related Resources

EMDR Therapy and Military Resource List

EMDR Beyond PTSD (Clinical Neuropsychiatry)

Childbirth-related posttraumatic stress disorder: definition, risk factors, pathophysiology, diagnosis, prevention, and treatment (american journal of obstetrics & gynecology), find an emdr therapist.

EMDRIA has more than 16,000 members trained to provide EMDR Therapy. Find a therapist near you.

• Your Name *
• Your Email Address *
• Your Phone Number
• Your Message *
• Hidden Recipient Name
• Hidden Recipient Email Address
• Hidden Recipient ID - Algolia
• Hidden Hidden Recipient Name - Algolia
• Hidden Hidden Recipient Email - Algolia

Please be aware that email is not a secure means of communication and spam filters may prevent your email from reaching the therapist. We recommend you follow up with a phone call. Sending an email using this page does not guarantee that the recipient will receive, read, or respond to your email.

If this is an emergency do not use this form. Call 911 or your nearest hospital.

• Email This field is for validation purposes and should be left unchanged.