Review articles: purpose, process, and structure

  • Published: 02 October 2017
  • Volume 46 , pages 1–5, ( 2018 )

Cite this article

  • Robert W. Palmatier 1 ,
  • Mark B. Houston 2 &
  • John Hulland 3  

227k Accesses

428 Citations

63 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

Many research disciplines feature high-impact journals that are dedicated outlets for review papers (or review–conceptual combinations) (e.g., Academy of Management Review , Psychology Bulletin , Medicinal Research Reviews ). The rationale for such outlets is the premise that research integration and synthesis provides an important, and possibly even a required, step in the scientific process. Review papers tend to include both quantitative (i.e., meta-analytic, systematic reviews) and narrative or more qualitative components; together, they provide platforms for new conceptual frameworks, reveal inconsistencies in the extant body of research, synthesize diverse results, and generally give other scholars a “state-of-the-art” snapshot of a domain, often written by topic experts (Bem 1995 ). Many premier marketing journals publish meta-analytic review papers too, though authors often must overcome reviewers’ concerns that their contributions are limited due to the absence of “new data.” Furthermore, relatively few non-meta-analysis review papers appear in marketing journals, probably due to researchers’ perceptions that such papers have limited publication opportunities or their beliefs that the field lacks a research tradition or “respect” for such papers. In many cases, an editor must provide strong support to help such review papers navigate the review process. Yet, once published, such papers tend to be widely cited, suggesting that members of the field find them useful (see Bettencourt and Houston 2001 ).

In this editorial, we seek to address three topics relevant to review papers. First, we outline a case for their importance to the scientific process, by describing the purpose of review papers . Second, we detail the review paper editorial initiative conducted over the past two years by the Journal of the Academy of Marketing Science ( JAMS ), focused on increasing the prevalence of review papers. Third, we describe a process and structure for systematic ( i.e. , non-meta-analytic) review papers , referring to Grewal et al. ( 2018 ) insights into parallel meta-analytic (effects estimation) review papers. (For some strong recent examples of marketing-related meta-analyses, see Knoll and Matthes 2017 ; Verma et al. 2016 ).

Purpose of review papers

In their most general form, review papers “are critical evaluations of material that has already been published,” some that include quantitative effects estimation (i.e., meta-analyses) and some that do not (i.e., systematic reviews) (Bem 1995 , p. 172). They carefully identify and synthesize relevant literature to evaluate a specific research question, substantive domain, theoretical approach, or methodology and thereby provide readers with a state-of-the-art understanding of the research topic. Many of these benefits are highlighted in Hanssens’ ( 2018 ) paper titled “The Value of Empirical Generalizations in Marketing,” published in this same issue of JAMS.

The purpose of and contributions associated with review papers can vary depending on their specific type and research question, but in general, they aim to

Resolve definitional ambiguities and outline the scope of the topic.

Provide an integrated, synthesized overview of the current state of knowledge.

Identify inconsistencies in prior results and potential explanations (e.g., moderators, mediators, measures, approaches).

Evaluate existing methodological approaches and unique insights.

Develop conceptual frameworks to reconcile and extend past research.

Describe research insights, existing gaps, and future research directions.

Not every review paper can offer all of these benefits, but this list represents their key contributions. To provide a sufficient contribution, a review paper needs to achieve three key standards. First, the research domain needs to be well suited for a review paper, such that a sufficient body of past research exists to make the integration and synthesis valuable—especially if extant research reveals theoretical inconsistences or heterogeneity in its effects. Second, the review paper must be well executed, with an appropriate literature collection and analysis techniques, sufficient breadth and depth of literature coverage, and a compelling writing style. Third, the manuscript must offer significant new insights based on its systematic comparison of multiple studies, rather than simply a “book report” that describes past research. This third, most critical standard is often the most difficult, especially for authors who have not “lived” with the research domain for many years, because achieving it requires drawing some non-obvious connections and insights from multiple studies and their many different aspects (e.g., context, method, measures). Typically, after the “review” portion of the paper has been completed, the authors must spend many more months identifying the connections to uncover incremental insights, each of which takes time to detail and explicate.

The increasing methodological rigor and technical sophistication of many marketing studies also means that they often focus on smaller problems with fewer constructs. By synthesizing these piecemeal findings, reconciling conflicting evidence, and drawing a “big picture,” meta-analyses and systematic review papers become indispensable to our comprehensive understanding of a phenomenon, among both academic and practitioner communities. Thus, good review papers provide a solid platform for future research, in the reviewed domain but also in other areas, in that researchers can use a good review paper to learn about and extend key insights to new areas.

This domain extension, outside of the core area being reviewed, is one of the key benefits of review papers that often gets overlooked. Yet it also is becoming ever more important with the expanding breadth of marketing (e.g., econometric modeling, finance, strategic management, applied psychology, sociology) and the increasing velocity in the accumulation of marketing knowledge (e.g., digital marketing, social media, big data). Against this backdrop, systematic review papers and meta-analyses help academics and interested managers keep track of research findings that fall outside their main area of specialization.

JAMS’ review paper editorial initiative

With a strong belief in the importance of review papers, the editorial team of JAMS has purposely sought out leading scholars to provide substantive review papers, both meta-analysis and systematic, for publication in JAMS . Many of the scholars approached have voiced concerns about the risk of such endeavors, due to the lack of alternative outlets for these types of papers. Therefore, we have instituted a unique process, in which the authors develop a detailed outline of their paper, key tables and figures, and a description of their literature review process. On the basis of this outline, we grant assurances that the contribution hurdle will not be an issue for publication in JAMS , as long as the authors execute the proposed outline as written. Each paper still goes through the normal review process and must meet all publication quality standards, of course. In many cases, an Area Editor takes an active role to help ensure that each paper provides sufficient insights, as required for a high-quality review paper. This process gives the author team confidence to invest effort in the process. An analysis of the marketing journals in the Financial Times (FT 50) journal list for the past five years (2012–2016) shows that JAMS has become the most common outlet for these papers, publishing 31% of all review papers that appeared in the top six marketing journals.

As a next step in positioning JAMS as a receptive marketing outlet for review papers, we are conducting a Thought Leaders Conference on Generalizations in Marketing: Systematic Reviews and Meta-Analyses , with a corresponding special issue (see www.springer.com/jams ). We will continue our process of seeking out review papers as an editorial strategy in areas that could be advanced by the integration and synthesis of extant research. We expect that, ultimately, such efforts will become unnecessary, as authors initiate review papers on topics of their own choosing to submit them to JAMS . In the past two years, JAMS already has increased the number of papers it publishes annually, from just over 40 to around 60 papers per year; this growth has provided “space” for 8–10 review papers per year, reflecting our editorial target.

Consistent with JAMS ’ overall focus on managerially relevant and strategy-focused topics, all review papers should reflect this emphasis. For example, the domains, theories, and methods reviewed need to have some application to past or emerging managerial research. A good rule of thumb is that the substantive domain, theory, or method should attract the attention of readers of JAMS .

The efforts of multiple editors and Area Editors in turn have generated a body of review papers that can serve as useful examples of the different types and approaches that JAMS has published.

Domain-based review papers

Domain-based review papers review, synthetize, and extend a body of literature in the same substantive domain. For example, in “The Role of Privacy in Marketing” (Martin and Murphy 2017 ), the authors identify and define various privacy-related constructs that have appeared in recent literature. Then they examine the different theoretical perspectives brought to bear on privacy topics related to consumers and organizations, including ethical and legal perspectives. These foundations lead in to their systematic review of privacy-related articles over a clearly defined date range, from which they extract key insights from each study. This exercise of synthesizing diverse perspectives allows these authors to describe state-of-the-art knowledge regarding privacy in marketing and identify useful paths for research. Similarly, a new paper by Cleeren et al. ( 2017 ), “Marketing Research on Product-Harm Crises: A Review, Managerial Implications, and an Agenda for Future Research,” provides a rich systematic review, synthesizes extant research, and points the way forward for scholars who are interested in issues related to defective or dangerous market offerings.

Theory-based review papers

Theory-based review papers review, synthetize, and extend a body of literature that uses the same underlying theory. For example, Rindfleisch and Heide’s ( 1997 ) classic review of research in marketing using transaction cost economics has been cited more than 2200 times, with a significant impact on applications of the theory to the discipline in the past 20 years. A recent paper in JAMS with similar intent, which could serve as a helpful model, focuses on “Resource-Based Theory in Marketing” (Kozlenkova et al. 2014 ). The article dives deeply into a description of the theory and its underlying assumptions, then organizes a systematic review of relevant literature according to various perspectives through which the theory has been applied in marketing. The authors conclude by identifying topical domains in marketing that might benefit from additional applications of the theory (e.g., marketing exchange), as well as related theories that could be integrated meaningfully with insights from the resource-based theory.

Method-based review papers

Method-based review papers review, synthetize, and extend a body of literature that uses the same underlying method. For example, in “Event Study Methodology in the Marketing Literature: An Overview” (Sorescu et al. 2017 ), the authors identify published studies in marketing that use an event study methodology. After a brief review of the theoretical foundations of event studies, they describe in detail the key design considerations associated with this method. The article then provides a roadmap for conducting event studies and compares this approach with a stock market returns analysis. The authors finish with a summary of the strengths and weaknesses of the event study method, which in turn suggests three main areas for further research. Similarly, “Discriminant Validity Testing in Marketing: An Analysis, Causes for Concern, and Proposed Remedies” (Voorhies et al. 2016 ) systematically reviews existing approaches for assessing discriminant validity in marketing contexts, then uses Monte Carlo simulation to determine which tests are most effective.

Our long-term editorial strategy is to make sure JAMS becomes and remains a well-recognized outlet for both meta-analysis and systematic managerial review papers in marketing. Ideally, review papers would come to represent 10%–20% of the papers published by the journal.

Process and structure for review papers

In this section, we review the process and typical structure of a systematic review paper, which lacks any long or established tradition in marketing research. The article by Grewal et al. ( 2018 ) provides a summary of effects-focused review papers (i.e., meta-analyses), so we do not discuss them in detail here.

Systematic literature review process

Some review papers submitted to journals take a “narrative” approach. They discuss current knowledge about a research domain, yet they often are flawed, in that they lack criteria for article inclusion (or, more accurately, article exclusion), fail to discuss the methodology used to evaluate included articles, and avoid critical assessment of the field (Barczak 2017 ). Such reviews tend to be purely descriptive, with little lasting impact.

In contrast, a systematic literature review aims to “comprehensively locate and synthesize research that bears on a particular question, using organized, transparent, and replicable procedures at each step in the process” (Littell et al. 2008 , p. 1). Littell et al. describe six key steps in the systematic review process. The extent to which each step is emphasized varies by paper, but all are important components of the review.

Topic formulation . The author sets out clear objectives for the review and articulates the specific research questions or hypotheses that will be investigated.

Study design . The author specifies relevant problems, populations, constructs, and settings of interest. The aim is to define explicit criteria that can be used to assess whether any particular study should be included in or excluded from the review. Furthermore, it is important to develop a protocol in advance that describes the procedures and methods to be used to evaluate published work.

Sampling . The aim in this third step is to identify all potentially relevant studies, including both published and unpublished research. To this end, the author must first define the sampling unit to be used in the review (e.g., individual, strategic business unit) and then develop an appropriate sampling plan.

Data collection . By retrieving the potentially relevant studies identified in the third step, the author can determine whether each study meets the eligibility requirements set out in the second step. For studies deemed acceptable, the data are extracted from each study and entered into standardized templates. These templates should be based on the protocols established in step 2.

Data analysis . The degree and nature of the analyses used to describe and examine the collected data vary widely by review. Purely descriptive analysis is useful as a starting point but rarely is sufficient on its own. The examination of trends, clusters of ideas, and multivariate relationships among constructs helps flesh out a deeper understanding of the domain. For example, both Hult ( 2015 ) and Huber et al. ( 2014 ) use bibliometric approaches (e.g., examine citation data using multidimensional scaling and cluster analysis techniques) to identify emerging versus declining themes in the broad field of marketing.

Reporting . Three key aspects of this final step are common across systematic reviews. First, the results from the fifth step need to be presented, clearly and compellingly, using narratives, tables, and figures. Second, core results that emerge from the review must be interpreted and discussed by the author. These revelatory insights should reflect a deeper understanding of the topic being investigated, not simply a regurgitation of well-established knowledge. Third, the author needs to describe the implications of these unique insights for both future research and managerial practice.

A new paper by Watson et al. ( 2017 ), “Harnessing Difference: A Capability-Based Framework for Stakeholder Engagement in Environmental Innovation,” provides a good example of a systematic review, starting with a cohesive conceptual framework that helps establish the boundaries of the review while also identifying core constructs and their relationships. The article then explicitly describes the procedures used to search for potentially relevant papers and clearly sets out criteria for study inclusion or exclusion. Next, a detailed discussion of core elements in the framework weaves published research findings into the exposition. The paper ends with a presentation of key implications and suggestions for the next steps. Similarly, “Marketing Survey Research Best Practices: Evidence and Recommendations from a Review of JAMS Articles” (Hulland et al. 2017 ) systematically reviews published marketing studies that use survey techniques, describes recent trends, and suggests best practices. In their review, Hulland et al. examine the entire population of survey papers published in JAMS over a ten-year span, relying on an extensive standardized data template to facilitate their subsequent data analysis.

Structure of systematic review papers

There is no cookie-cutter recipe for the exact structure of a useful systematic review paper; the final structure depends on the authors’ insights and intended points of emphasis. However, several key components are likely integral to a paper’s ability to contribute.

Depth and rigor

Systematic review papers must avoid falling in to two potential “ditches.” The first ditch threatens when the paper fails to demonstrate that a systematic approach was used for selecting articles for inclusion and capturing their insights. If a reader gets the impression that the author has cherry-picked only articles that fit some preset notion or failed to be thorough enough, without including articles that make significant contributions to the field, the paper will be consigned to the proverbial side of the road when it comes to the discipline’s attention.

Authors that fall into the other ditch present a thorough, complete overview that offers only a mind-numbing recitation, without evident organization, synthesis, or critical evaluation. Although comprehensive, such a paper is more of an index than a useful review. The reviewed articles must be grouped in a meaningful way to guide the reader toward a better understanding of the focal phenomenon and provide a foundation for insights about future research directions. Some scholars organize research by scholarly perspectives (e.g., the psychology of privacy, the economics of privacy; Martin and Murphy 2017 ); others classify the chosen articles by objective research aspects (e.g., empirical setting, research design, conceptual frameworks; Cleeren et al. 2017 ). The method of organization chosen must allow the author to capture the complexity of the underlying phenomenon (e.g., including temporal or evolutionary aspects, if relevant).

Replicability

Processes for the identification and inclusion of research articles should be described in sufficient detail, such that an interested reader could replicate the procedure. The procedures used to analyze chosen articles and extract their empirical findings and/or key takeaways should be described with similar specificity and detail.

We already have noted the potential usefulness of well-done review papers. Some scholars always are new to the field or domain in question, so review papers also need to help them gain foundational knowledge. Key constructs, definitions, assumptions, and theories should be laid out clearly (for which purpose summary tables are extremely helpful). An integrated conceptual model can be useful to organize cited works. Most scholars integrate the knowledge they gain from reading the review paper into their plans for future research, so it is also critical that review papers clearly lay out implications (and specific directions) for research. Ideally, readers will come away from a review article filled with enthusiasm about ways they might contribute to the ongoing development of the field.

Helpful format

Because such a large body of research is being synthesized in most review papers, simply reading through the list of included studies can be exhausting for readers. We cannot overstate the importance of tables and figures in review papers, used in conjunction with meaningful headings and subheadings. Vast literature review tables often are essential, but they must be organized in a way that makes their insights digestible to the reader; in some cases, a sequence of more focused tables may be better than a single, comprehensive table.

In summary, articles that review extant research in a domain (topic, theory, or method) can be incredibly useful to the scientific progress of our field. Whether integrating the insights from extant research through a meta-analysis or synthesizing them through a systematic assessment, the promised benefits are similar. Both formats provide readers with a useful overview of knowledge about the focal phenomenon, as well as insights on key dilemmas and conflicting findings that suggest future research directions. Thus, the editorial team at JAMS encourages scholars to continue to invest the time and effort to construct thoughtful review papers.

Barczak, G. (2017). From the editor: writing a review article. Journal of Product Innovation Management, 34 (2), 120–121.

Article   Google Scholar  

Bem, D. J. (1995). Writing a review article for psychological bulletin. Psychological Bulletin, 118 (2), 172–177.

Bettencourt, L. A., & Houston, M. B. (2001). Assessing the impact of article method type and subject area on citation frequency and reference diversity. Marketing Letters, 12 (4), 327–340.

Cleeren, K., Dekimpe, M. G., & van Heerde, H. J. (2017). Marketing research on product-harm crises: a review, managerial implications. Journal of the Academy of Marketing Science, 45 (5), 593–615.

Grewal, D., Puccinelli, N. M., & Monroe, K. B. (2018). Meta-analysis: error cancels and truth accrues. Journal of the Academy of Marketing Science, 46 (1).

Hanssens, D. M. (2018). The value of empirical generalizations in marketing. Journal of the Academy of Marketing Science, 46 (1).

Huber, J., Kamakura, W., & Mela, C. F. (2014). A topical history of JMR . Journal of Marketing Research, 51 (1), 84–91.

Hulland, J., Baumgartner, H., & Smith, K. M. (2017). Marketing survey research best practices: evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science. https://doi.org/10.1007/s11747-017-0532-y .

Hult, G. T. M. (2015). JAMS 2010—2015: literature themes and intellectual structure. Journal of the Academy of Marketing Science, 43 (6), 663–669.

Knoll, J., & Matthes, J. (2017). The effectiveness of celebrity endorsements: a meta-analysis. Journal of the Academy of Marketing Science, 45 (1), 55–75.

Kozlenkova, I. V., Samaha, S. A., & Palmatier, R. W. (2014). Resource-based theory in marketing. Journal of the Academy of Marketing Science, 42 (1), 1–21.

Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis . New York: Oxford University Press.

Book   Google Scholar  

Martin, K. D., & Murphy, P. E. (2017). The role of data privacy in marketing. Journal of the Academy of Marketing Science, 45 (2), 135–155.

Rindfleisch, A., & Heide, J. B. (1997). Transaction cost analysis: past, present, and future applications. Journal of Marketing, 61 (4), 30–54.

Sorescu, A., Warren, N. L., & Ertekin, L. (2017). Event study methodology in the marketing literature: an overview. Journal of the Academy of Marketing Science, 45 (2), 186–207.

Verma, V., Sharma, D., & Sheth, J. (2016). Does relationship marketing matter in online retailing? A meta-analytic approach. Journal of the Academy of Marketing Science, 44 (2), 206–217.

Voorhies, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016). Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies. Journal of the Academy of Marketing Science, 44 (1), 119–134.

Watson, R., Wilson, H. N., Smart, P., & Macdonald, E. K. (2017). Harnessing difference: a capability-based framework for stakeholder engagement in environmental innovation. Journal of Product Innovation Management. https://doi.org/10.1111/jpim.12394 .

Download references

Author information

Authors and affiliations.

Foster School of Business, University of Washington, Box: 353226, Seattle, WA, 98195-3226, USA

Robert W. Palmatier

Neeley School of Business, Texas Christian University, Fort Worth, TX, USA

Mark B. Houston

Terry College of Business, University of Georgia, Athens, GA, USA

John Hulland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Robert W. Palmatier .

Rights and permissions

Reprints and permissions

About this article

Palmatier, R.W., Houston, M.B. & Hulland, J. Review articles: purpose, process, and structure. J. of the Acad. Mark. Sci. 46 , 1–5 (2018). https://doi.org/10.1007/s11747-017-0563-4

Download citation

Published : 02 October 2017

Issue Date : January 2018

DOI : https://doi.org/10.1007/s11747-017-0563-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Find a journal
  • Publish with us
  • Track your research

How to write a good scientific review article

Affiliation.

  • 1 The FEBS Journal Editorial Office, Cambridge, UK.
  • PMID: 35792782
  • DOI: 10.1111/febs.16565

Literature reviews are valuable resources for the scientific community. With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up to date with developments in a particular area of research. A good review article provides readers with an in-depth understanding of a field and highlights key gaps and challenges to address with future research. Writing a review article also helps to expand the writer's knowledge of their specialist area and to develop their analytical and communication skills, amongst other benefits. Thus, the importance of building review-writing into a scientific career cannot be overstated. In this instalment of The FEBS Journal's Words of Advice series, I provide detailed guidance on planning and writing an informative and engaging literature review.

© 2022 Federation of European Biochemical Societies.

Publication types

  • Search Menu
  • Advance Articles
  • Editor's Choice
  • CME Reviews
  • Best of 2021 collection
  • Abbreviated Breast MRI Virtual Collection
  • Contrast-enhanced Mammography Collection
  • Author Guidelines
  • Submission Site
  • Open Access
  • Self-Archiving Policy
  • Accepted Papers Resource Guide
  • About Journal of Breast Imaging
  • About the Society of Breast Imaging
  • Guidelines for Reviewers
  • Resources for Reviewers and Authors
  • Editorial Board
  • Advertising Disclaimer
  • Advertising and Corporate Services
  • Journals on Oxford Academic
  • Books on Oxford Academic

Society of Breast Imaging

  • < Previous

A Step-by-Step Guide to Writing a Scientific Review Article

  • Article contents
  • Figures & tables
  • Supplementary Data

Manisha Bahl, A Step-by-Step Guide to Writing a Scientific Review Article, Journal of Breast Imaging , Volume 5, Issue 4, July/August 2023, Pages 480–485, https://doi.org/10.1093/jbi/wbad028

  • Permissions Icon Permissions

Scientific review articles are comprehensive, focused reviews of the scientific literature written by subject matter experts. The task of writing a scientific review article can seem overwhelming; however, it can be managed by using an organized approach and devoting sufficient time to the process. The process involves selecting a topic about which the authors are knowledgeable and enthusiastic, conducting a literature search and critical analysis of the literature, and writing the article, which is composed of an abstract, introduction, body, and conclusion, with accompanying tables and figures. This article, which focuses on the narrative or traditional literature review, is intended to serve as a guide with practical steps for new writers. Tips for success are also discussed, including selecting a focused topic, maintaining objectivity and balance while writing, avoiding tedious data presentation in a laundry list format, moving from descriptions of the literature to critical analysis, avoiding simplistic conclusions, and budgeting time for the overall process.

  • narrative discourse

Email alerts

Citing articles via.

  • Recommend to your Librarian
  • Journals Career Network

Affiliations

  • Online ISSN 2631-6129
  • Print ISSN 2631-6110
  • Copyright © 2024 Society of Breast Imaging
  • About Oxford Academic
  • Publish journals with us
  • University press partners
  • What we publish
  • New features  
  • Open access
  • Institutional account management
  • Rights and permissions
  • Get help with access
  • Accessibility
  • Advertising
  • Media enquiries
  • Oxford University Press
  • Oxford Languages
  • University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

  • Copyright © 2024 Oxford University Press
  • Cookie settings
  • Cookie policy
  • Privacy policy
  • Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

  • PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
  • EDIT Edit this Article
  • EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
  • Browse Articles
  • Learn Something New
  • Quizzes Hot
  • This Or That Game New
  • Train Your Brain
  • Explore More
  • Support wikiHow
  • About wikiHow
  • Log in / Sign up
  • Education and Communications
  • Critical Reviews

How to Write an Article Review

Last Updated: September 8, 2023 Fact Checked

This article was co-authored by Jake Adams . Jake Adams is an academic tutor and the owner of Simplifi EDU, a Santa Monica, California based online tutoring business offering learning resources and online tutors for academic subjects K-College, SAT & ACT prep, and college admissions applications. With over 14 years of professional tutoring experience, Jake is dedicated to providing his clients the very best online tutoring experience and access to a network of excellent undergraduate and graduate-level tutors from top colleges all over the nation. Jake holds a BS in International Business and Marketing from Pepperdine University. There are 13 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 3,084,872 times.

An article review is both a summary and an evaluation of another writer's article. Teachers often assign article reviews to introduce students to the work of experts in the field. Experts also are often asked to review the work of other professionals. Understanding the main points and arguments of the article is essential for an accurate summation. Logical evaluation of the article's main theme, supporting arguments, and implications for further research is an important element of a review . Here are a few guidelines for writing an article review.

Education specialist Alexander Peterman recommends: "In the case of a review, your objective should be to reflect on the effectiveness of what has already been written, rather than writing to inform your audience about a subject."

Things You Should Know

  • Read the article very closely, and then take time to reflect on your evaluation. Consider whether the article effectively achieves what it set out to.
  • Write out a full article review by completing your intro, summary, evaluation, and conclusion. Don't forget to add a title, too!
  • Proofread your review for mistakes (like grammar and usage), while also cutting down on needless information. [1] X Research source

Preparing to Write Your Review

Step 1 Understand what an article review is.

  • Article reviews present more than just an opinion. You will engage with the text to create a response to the scholarly writer's ideas. You will respond to and use ideas, theories, and research from your studies. Your critique of the article will be based on proof and your own thoughtful reasoning.
  • An article review only responds to the author's research. It typically does not provide any new research. However, if you are correcting misleading or otherwise incorrect points, some new data may be presented.
  • An article review both summarizes and evaluates the article.

Step 2 Think about the organization of the review article.

  • Summarize the article. Focus on the important points, claims, and information.
  • Discuss the positive aspects of the article. Think about what the author does well, good points she makes, and insightful observations.
  • Identify contradictions, gaps, and inconsistencies in the text. Determine if there is enough data or research included to support the author's claims. Find any unanswered questions left in the article.

Step 3 Preview the article.

  • Make note of words or issues you don't understand and questions you have.
  • Look up terms or concepts you are unfamiliar with, so you can fully understand the article. Read about concepts in-depth to make sure you understand their full context.

Step 4 Read the article closely.

  • Pay careful attention to the meaning of the article. Make sure you fully understand the article. The only way to write a good article review is to understand the article.

Step 5 Put the article into your words.

  • With either method, make an outline of the main points made in the article and the supporting research or arguments. It is strictly a restatement of the main points of the article and does not include your opinions.
  • After putting the article in your own words, decide which parts of the article you want to discuss in your review. You can focus on the theoretical approach, the content, the presentation or interpretation of evidence, or the style. You will always discuss the main issues of the article, but you can sometimes also focus on certain aspects. This comes in handy if you want to focus the review towards the content of a course.
  • Review the summary outline to eliminate unnecessary items. Erase or cross out the less important arguments or supplemental information. Your revised summary can serve as the basis for the summary you provide at the beginning of your review.

Step 6 Write an outline of your evaluation.

  • What does the article set out to do?
  • What is the theoretical framework or assumptions?
  • Are the central concepts clearly defined?
  • How adequate is the evidence?
  • How does the article fit into the literature and field?
  • Does it advance the knowledge of the subject?
  • How clear is the author's writing? Don't: include superficial opinions or your personal reaction. Do: pay attention to your biases, so you can overcome them.

Writing the Article Review

Step 1 Come up with...

  • For example, in MLA , a citation may look like: Duvall, John N. "The (Super)Marketplace of Images: Television as Unmediated Mediation in DeLillo's White Noise ." Arizona Quarterly 50.3 (1994): 127-53. Print. [10] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

Step 3 Identify the article.

  • For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest.

Step 4 Write the introduction....

  • Your introduction should only be 10-25% of your review.
  • End the introduction with your thesis. Your thesis should address the above issues. For example: Although the author has some good points, his article is biased and contains some misinterpretation of data from others’ analysis of the effectiveness of the condom.

Step 5 Summarize the article.

  • Use direct quotes from the author sparingly.
  • Review the summary you have written. Read over your summary many times to ensure that your words are an accurate description of the author's article.

Step 6 Write your critique.

  • Support your critique with evidence from the article or other texts.
  • The summary portion is very important for your critique. You must make the author's argument clear in the summary section for your evaluation to make sense.
  • Remember, this is not where you say if you liked the article or not. You are assessing the significance and relevance of the article.
  • Use a topic sentence and supportive arguments for each opinion. For example, you might address a particular strength in the first sentence of the opinion section, followed by several sentences elaborating on the significance of the point.

Step 7 Conclude the article review.

  • This should only be about 10% of your overall essay.
  • For example: This critical review has evaluated the article "Condom use will increase the spread of AIDS" by Anthony Zimmerman. The arguments in the article show the presence of bias, prejudice, argumentative writing without supporting details, and misinformation. These points weaken the author’s arguments and reduce his credibility.

Step 8 Proofread.

  • Make sure you have identified and discussed the 3-4 key issues in the article.

Sample Article Reviews

review article research paper

Expert Q&A

Jake Adams

You Might Also Like

Write a Feature Article

  • ↑ https://writing.wisc.edu/handbook/grammarpunct/proofreading/
  • ↑ https://libguides.cmich.edu/writinghelp/articlereview
  • ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/
  • ↑ Jake Adams. Academic Tutor & Test Prep Specialist. Expert Interview. 24 July 2020.
  • ↑ https://guides.library.queensu.ca/introduction-research/writing/critical
  • ↑ https://www.iup.edu/writingcenter/writing-resources/organization-and-structure/creating-an-outline.html
  • ↑ https://writing.umn.edu/sws/assets/pdf/quicktips/titles.pdf
  • ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_periodicals.html
  • ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548565/
  • ↑ https://writingcenter.uconn.edu/wp-content/uploads/sites/593/2014/06/How_to_Summarize_a_Research_Article1.pdf
  • ↑ https://www.uis.edu/learning-hub/writing-resources/handouts/learning-hub/how-to-review-a-journal-article
  • ↑ https://writingcenter.unc.edu/tips-and-tools/editing-and-proofreading/

About This Article

Jake Adams

If you have to write an article review, read through the original article closely, taking notes and highlighting important sections as you read. Next, rewrite the article in your own words, either in a long paragraph or as an outline. Open your article review by citing the article, then write an introduction which states the article’s thesis. Next, summarize the article, followed by your opinion about whether the article was clear, thorough, and useful. Finish with a paragraph that summarizes the main points of the article and your opinions. To learn more about what to include in your personal critique of the article, keep reading the article! Did this summary help you? Yes No

  • Send fan mail to authors

Reader Success Stories

Prince Asiedu-Gyan

Prince Asiedu-Gyan

Apr 22, 2022

Did this article help you?

Sammy James

Sammy James

Sep 12, 2017

Juabin Matey

Juabin Matey

Aug 30, 2017

Kristi N.

Oct 25, 2023

Vanita Meghrajani

Vanita Meghrajani

Jul 21, 2016

Am I a Narcissist or an Empath Quiz

Featured Articles

What Does a Forehead Kiss Mean? 10+ Reasons Behind This Personal Peck

Trending Articles

8 Reasons Why Life Sucks & 15 Ways to Deal With It

Watch Articles

Fold Boxer Briefs

  • Terms of Use
  • Privacy Policy
  • Do Not Sell or Share My Info
  • Not Selling Info

Get all the best how-tos!

Sign up for wikiHow's weekly email newsletter

Elsevier QRcode Wechat

  • Research Process

Writing a good review article

  • 3 minute read
  • 70.7K views

Table of Contents

As a young researcher, you might wonder how to start writing your first review article, and the extent of the information that it should contain. A review article is a comprehensive summary of the current understanding of a specific research topic and is based on previously published research. Unlike research papers, it does not contain new results, but can propose new inferences based on the combined findings of previous research.

Types of review articles

Review articles are typically of three types: literature reviews, systematic reviews, and meta-analyses.

A literature review is a general survey of the research topic and aims to provide a reliable and unbiased account of the current understanding of the topic.

A systematic review , in contrast, is more specific and attempts to address a highly focused research question. Its presentation is more detailed, with information on the search strategy used, the eligibility criteria for inclusion of studies, the methods utilized to review the collected information, and more.

A meta-analysis is similar to a systematic review in that both are systematically conducted with a properly defined research question. However, unlike the latter, a meta-analysis compares and evaluates a defined number of similar studies. It is quantitative in nature and can help assess contrasting study findings.

Tips for writing a good review article

Here are a few practices that can make the time-consuming process of writing a review article easier:

  • Define your question: Take your time to identify the research question and carefully articulate the topic of your review paper. A good review should also add something new to the field in terms of a hypothesis, inference, or conclusion. A carefully defined scientific question will give you more clarity in determining the novelty of your inferences.
  • Identify credible sources: Identify relevant as well as credible studies that you can base your review on, with the help of multiple databases or search engines. It is also a good idea to conduct another search once you have finished your article to avoid missing relevant studies published during the course of your writing.
  • Take notes: A literature search involves extensive reading, which can make it difficult to recall relevant information subsequently. Therefore, make notes while conducting the literature search and note down the source references. This will ensure that you have sufficient information to start with when you finally get to writing.
  • Describe the title, abstract, and introduction: A good starting point to begin structuring your review is by drafting the title, abstract, and introduction. Explicitly writing down what your review aims to address in the field will help shape the rest of your article.
  • Be unbiased and critical: Evaluate every piece of evidence in a critical but unbiased manner. This will help you present a proper assessment and a critical discussion in your article.
  • Include a good summary: End by stating the take-home message and identify the limitations of existing studies that need to be addressed through future studies.
  • Ask for feedback: Ask a colleague to provide feedback on both the content and the language or tone of your article before you submit it.
  • Check your journal’s guidelines: Some journals only publish reviews, while some only publish research articles. Further, all journals clearly indicate their aims and scope. Therefore, make sure to check the appropriateness of a journal before submitting your article.

Writing review articles, especially systematic reviews or meta-analyses, can seem like a daunting task. However, Elsevier Author Services can guide you by providing useful tips on how to write an impressive review article that stands out and gets published!

What are Implications in Research

  • Manuscript Preparation

What are Implications in Research?

how to write the results section of a research paper

How to write the results section of a research paper

You may also like.

what is a descriptive research design

Descriptive Research Design and Its Myriad Uses

Doctor doing a Biomedical Research Paper

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

review article research paper

Making Technical Writing in Environmental Engineering Accessible

Risks of AI-assisted Academic Writing

To Err is Not Human: The Dangers of AI-assisted Academic Writing

Importance-of-Data-Collection

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

choosing the Right Research Methodology

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research

Why is data validation important in research?

Scholarly Sources What are They and Where can You Find Them

Scholarly Sources: What are They and Where can You Find Them?

Input your search keywords and press Enter.

review article research paper

How to Write an Article Review: Tips and Examples

review article research paper

Did you know that article reviews are not just academic exercises but also a valuable skill in today's information age? In a world inundated with content, being able to dissect and evaluate articles critically can help you separate the wheat from the chaff. Whether you're a student aiming to excel in your coursework or a professional looking to stay well-informed, mastering the art of writing article reviews is an invaluable skill.

Short Description

In this article, our research paper writing service experts will start by unraveling the concept of article reviews and discussing the various types. You'll also gain insights into the art of formatting your review effectively. To ensure you're well-prepared, we'll take you through the pre-writing process, offering tips on setting the stage for your review. But it doesn't stop there. You'll find a practical example of an article review to help you grasp the concepts in action. To complete your journey, we'll guide you through the post-writing process, equipping you with essential proofreading techniques to ensure your work shines with clarity and precision!

What Is an Article Review: Grasping the Concept 

A review article is a type of professional paper writing that demands a high level of in-depth analysis and a well-structured presentation of arguments. It is a critical, constructive evaluation of literature in a particular field through summary, classification, analysis, and comparison.

If you write a scientific review, you have to use database searches to portray the research. Your primary goal is to summarize everything and present a clear understanding of the topic you've been working on.

Writing Involves:

  • Summarization, classification, analysis, critiques, and comparison.
  • The analysis, evaluation, and comparison require the use of theories, ideas, and research relevant to the subject area of the article.
  • It is also worth nothing if a review does not introduce new information, but instead presents a response to another writer's work.
  • Check out other samples to gain a better understanding of how to review the article.

Types of Review

When it comes to article reviews, there's more than one way to approach the task. Understanding the various types of reviews is like having a versatile toolkit at your disposal. In this section, we'll walk you through the different dimensions of review types, each offering a unique perspective and purpose. Whether you're dissecting a scholarly article, critiquing a piece of literature, or evaluating a product, you'll discover the diverse landscape of article reviews and how to navigate it effectively.

types of article review

Journal Article Review

Just like other types of reviews, a journal article review assesses the merits and shortcomings of a published work. To illustrate, consider a review of an academic paper on climate change, where the writer meticulously analyzes and interprets the article's significance within the context of environmental science.

Research Article Review

Distinguished by its focus on research methodologies, a research article review scrutinizes the techniques used in a study and evaluates them in light of the subsequent analysis and critique. For instance, when reviewing a research article on the effects of a new drug, the reviewer would delve into the methods employed to gather data and assess their reliability.

Science Article Review

In the realm of scientific literature, a science article review encompasses a wide array of subjects. Scientific publications often provide extensive background information, which can be instrumental in conducting a comprehensive analysis. For example, when reviewing an article about the latest breakthroughs in genetics, the reviewer may draw upon the background knowledge provided to facilitate a more in-depth evaluation of the publication.

Need a Hand From Professionals?

Address to Our Writers and Get Assistance in Any Questions!

Formatting an Article Review

The format of the article should always adhere to the citation style required by your professor. If you're not sure, seek clarification on the preferred format and ask him to clarify several other pointers to complete the formatting of an article review adequately.

How Many Publications Should You Review?

  • In what format should you cite your articles (MLA, APA, ASA, Chicago, etc.)?
  • What length should your review be?
  • Should you include a summary, critique, or personal opinion in your assignment?
  • Do you need to call attention to a theme or central idea within the articles?
  • Does your instructor require background information?

When you know the answers to these questions, you may start writing your assignment. Below are examples of MLA and APA formats, as those are the two most common citation styles.

Using the APA Format

Articles appear most commonly in academic journals, newspapers, and websites. If you write an article review in the APA format, you will need to write bibliographical entries for the sources you use:

  • Web : Author [last name], A.A [first and middle initial]. (Year, Month, Date of Publication). Title. Retrieved from {link}
  • Journal : Author [last name], A.A [first and middle initial]. (Publication Year). Publication Title. Periodical Title, Volume(Issue), pp.-pp.
  • Newspaper : Author [last name], A.A [first and middle initial]. (Year, Month, Date of Publication). Publication Title. Magazine Title, pp. xx-xx.

Using MLA Format

  • Web : Last, First Middle Initial. “Publication Title.” Website Title. Website Publisher, Date Month Year Published. Web. Date Month Year Accessed.
  • Newspaper : Last, First M. “Publication Title.” Newspaper Title [City] Date, Month, Year Published: Page(s). Print.
  • Journal : Last, First M. “Publication Title.” Journal Title Series Volume. Issue (Year Published): Page(s). Database Name. Web. Date Month Year Accessed.

Enhance your writing effortlessly with EssayPro.com , where you can order an article review or any other writing task. Our team of expert writers specializes in various fields, ensuring your work is not just summarized, but deeply analyzed and professionally presented. Ideal for students and professionals alike, EssayPro offers top-notch writing assistance tailored to your needs. Elevate your writing today with our skilled team at your article review writing service !

order review

The Pre-Writing Process

Facing this task for the first time can really get confusing and can leave you unsure of where to begin. To create a top-notch article review, start with a few preparatory steps. Here are the two main stages from our dissertation services to get you started:

Step 1: Define the right organization for your review. Knowing the future setup of your paper will help you define how you should read the article. Here are the steps to follow:

  • Summarize the article — seek out the main points, ideas, claims, and general information presented in the article.
  • Define the positive points — identify the strong aspects, ideas, and insightful observations the author has made.
  • Find the gaps —- determine whether or not the author has any contradictions, gaps, or inconsistencies in the article and evaluate whether or not he or she used a sufficient amount of arguments and information to support his or her ideas.
  • Identify unanswered questions — finally, identify if there are any questions left unanswered after reading the piece.

Step 2: Move on and review the article. Here is a small and simple guide to help you do it right:

  • Start off by looking at and assessing the title of the piece, its abstract, introductory part, headings and subheadings, opening sentences in its paragraphs, and its conclusion.
  • First, read only the beginning and the ending of the piece (introduction and conclusion). These are the parts where authors include all of their key arguments and points. Therefore, if you start with reading these parts, it will give you a good sense of the author's main points.
  • Finally, read the article fully.

These three steps make up most of the prewriting process. After you are done with them, you can move on to writing your own review—and we are going to guide you through the writing process as well.

Outline and Template

As you progress with reading your article, organize your thoughts into coherent sections in an outline. As you read, jot down important facts, contributions, or contradictions. Identify the shortcomings and strengths of your publication. Begin to map your outline accordingly.

If your professor does not want a summary section or a personal critique section, then you must alleviate those parts from your writing. Much like other assignments, an article review must contain an introduction, a body, and a conclusion. Thus, you might consider dividing your outline according to these sections as well as subheadings within the body. If you find yourself troubled with the pre-writing and the brainstorming process for this assignment, seek out a sample outline.

Your custom essay must contain these constituent parts:

  • Pre-Title Page - Before diving into your review, start with essential details: article type, publication title, and author names with affiliations (position, department, institution, location, and email). Include corresponding author info if needed.
  • Running Head - In APA format, use a concise title (under 40 characters) to ensure consistent formatting.
  • Summary Page - Optional but useful. Summarize the article in 800 words, covering background, purpose, results, and methodology, avoiding verbatim text or references.
  • Title Page - Include the full title, a 250-word abstract, and 4-6 keywords for discoverability.
  • Introduction - Set the stage with an engaging overview of the article.
  • Body - Organize your analysis with headings and subheadings.
  • Works Cited/References - Properly cite all sources used in your review.
  • Optional Suggested Reading Page - If permitted, suggest further readings for in-depth exploration.
  • Tables and Figure Legends (if instructed by the professor) - Include visuals when requested by your professor for clarity.

Example of an Article Review

You might wonder why we've dedicated a section of this article to discuss an article review sample. Not everyone may realize it, but examining multiple well-constructed examples of review articles is a crucial step in the writing process. In the following section, our essay writing service experts will explain why.

Looking through relevant article review examples can be beneficial for you in the following ways:

  • To get you introduced to the key works of experts in your field.
  • To help you identify the key people engaged in a particular field of science.
  • To help you define what significant discoveries and advances were made in your field.
  • To help you unveil the major gaps within the existing knowledge of your field—which contributes to finding fresh solutions.
  • To help you find solid references and arguments for your own review.
  • To help you generate some ideas about any further field of research.
  • To help you gain a better understanding of the area and become an expert in this specific field.
  • To get a clear idea of how to write a good review.

View Our Writer’s Sample Before Crafting Your Own!

Why Have There Been No Great Female Artists?

Steps for Writing an Article Review

Here is a guide with critique paper format on how to write a review paper:

steps for article review

Step 1: Write the Title

First of all, you need to write a title that reflects the main focus of your work. Respectively, the title can be either interrogative, descriptive, or declarative.

Step 2: Cite the Article

Next, create a proper citation for the reviewed article and input it following the title. At this step, the most important thing to keep in mind is the style of citation specified by your instructor in the requirements for the paper. For example, an article citation in the MLA style should look as follows:

Author's last and first name. "The title of the article." Journal's title and issue(publication date): page(s). Print

Abraham John. "The World of Dreams." Virginia Quarterly 60.2(1991): 125-67. Print.

Step 3: Article Identification

After your citation, you need to include the identification of your reviewed article:

  • Title of the article
  • Title of the journal
  • Year of publication

All of this information should be included in the first paragraph of your paper.

The report "Poverty increases school drop-outs" was written by Brian Faith – a Health officer – in 2000.

Step 4: Introduction

Your organization in an assignment like this is of the utmost importance. Before embarking on your writing process, you should outline your assignment or use an article review template to organize your thoughts coherently.

  • If you are wondering how to start an article review, begin with an introduction that mentions the article and your thesis for the review.
  • Follow up with a summary of the main points of the article.
  • Highlight the positive aspects and facts presented in the publication.
  • Critique the publication by identifying gaps, contradictions, disparities in the text, and unanswered questions.

Step 5: Summarize the Article

Make a summary of the article by revisiting what the author has written about. Note any relevant facts and findings from the article. Include the author's conclusions in this section.

Step 6: Critique It

Present the strengths and weaknesses you have found in the publication. Highlight the knowledge that the author has contributed to the field. Also, write about any gaps and/or contradictions you have found in the article. Take a standpoint of either supporting or not supporting the author's assertions, but back up your arguments with facts and relevant theories that are pertinent to that area of knowledge. Rubrics and templates can also be used to evaluate and grade the person who wrote the article.

Step 7: Craft a Conclusion

In this section, revisit the critical points of your piece, your findings in the article, and your critique. Also, write about the accuracy, validity, and relevance of the results of the article review. Present a way forward for future research in the field of study. Before submitting your article, keep these pointers in mind:

  • As you read the article, highlight the key points. This will help you pinpoint the article's main argument and the evidence that they used to support that argument.
  • While you write your review, use evidence from your sources to make a point. This is best done using direct quotations.
  • Select quotes and supporting evidence adequately and use direct quotations sparingly. Take time to analyze the article adequately.
  • Every time you reference a publication or use a direct quotation, use a parenthetical citation to avoid accidentally plagiarizing your article.
  • Re-read your piece a day after you finish writing it. This will help you to spot grammar mistakes and to notice any flaws in your organization.
  • Use a spell-checker and get a second opinion on your paper.

The Post-Writing Process: Proofread Your Work

Finally, when all of the parts of your article review are set and ready, you have one last thing to take care of — proofreading. Although students often neglect this step, proofreading is a vital part of the writing process and will help you polish your paper to ensure that there are no mistakes or inconsistencies.

To proofread your paper properly, start by reading it fully and checking the following points:

  • Punctuation
  • Other mistakes

Afterward, take a moment to check for any unnecessary information in your paper and, if found, consider removing it to streamline your content. Finally, double-check that you've covered at least 3-4 key points in your discussion.

And remember, if you ever need help with proofreading, rewriting your essay, or even want to buy essay , our friendly team is always here to assist you.

Need an Article REVIEW WRITTEN?

Just send us the requirements to your paper and watch one of our writers crafting an original paper for you.

Related Articles

 How to Write a Policy Analysis Paper Step-by-Step

How to Write an Article Review That Stands Out

blog image

An article review is a critical assessment of another writer’s  research paper  or scholarly article. Such an activity aims to expand one’s knowledge by evaluating the original author’s research.

Of course, writing an article review could be tricky. But a few expert tips and tricks can get you on the right track. That’s what this interesting blog post is all about. So, ensure you read it till the end to make the most out of it.

Table of Contents

A Step-by-step Guide on How to Write an Article Review

Master the art of writing an article review with this step-by-step guide from professional  paper help  providers. 

Step 1: Select the Right Article

The first step is to pick a suitable article for a review. Choose a scholarly source that’s connected to your area of study. You can look for pieces printed in trustworthy journals or by respected authors.

For Example:

For reviewing an article on climate change, consider selecting one from scientific journals like Nature or Science.

Step 2: Read and Understand the Article

It’s super important to read and understand the article before writing your review. Read the article a few times and jot down the notes as you go. Focus on the main arguments, major points, evidence, and how it’s structured. 

Let’s say you’re looking at an article on how social media affects mental health. Ensure to take note of the following: 

  • The number of people involved 
  • How the data is analyzed 
  • The Results 

Step 3: Structure and Introduction

To start a solid review, start with an introduction that gives readers the background info they need. Must include the article’s title, the author, and where it was published. Also, write a summary of the main point or argument in the article.

“In the article ‘The Impact of Social Media on Mental Health by John Smith, published in the Journal of Psychology: 

The author examines the correlation between excessive social media usage and adolescent mental health disorders.”

Step 4: Summarize the Article

In this part, you’ll need to quickly go over the main points and arguments from the article. Make it short but must cover the most important elements and the evidence that backs them up. Leave your opinions and analysis out of it for now. 

For instance, you could write:

“The author discusses various studies highlighting the negative effects of excessive social media usage on mental health.

Smith’s research reveals a significant correlation between 

Increased social media consumption and higher rates of anxiety, depression, and low self-esteem among teenagers. 

The article also explores the underlying mechanisms, such as social comparison and cyberbullying. All are contributing to the adverse mental health outcomes.”

Step 5: Critically Analyze and Evaluate

Now that you’ve given a rundown of the article, it’s time to take a closer look. Think about what the author did well and what could have been done better. 

Check out the proof they used and if it seems solid. Give a thorough assessment, and use examples from the text to support your thoughts. 

For Example

“While the article presents compelling evidence linking social media usage to mental health issues , it is important to acknowledge some limitations in Smith’s study. 

The sample size of the research was relatively small. It comprises only 100 participants, which may limit the generalizability of the findings. 

Additionally, the study primarily focused on one specific age group, namely adolescents. This way, there’s room for further research on other demographic groups.”

Step 6: Express Your Perspective

Here’s your chance to give your two cents and show off your smarts. Put your spin on the article by pointing out the pros, cons, and other potential improvements. Remember to back up your thoughts with facts and sound arguments.

Continuing with the Previous Example

Despite the limitations, Smith’s research offers valuable insights into the complex relationship between social media and mental health. 

Future studies could expand the sample size and include a more diverse range of age groups. It is better to understand the broader impact of social media on mental well-being. 

Furthermore, exploring strategies for developing digital literacy programs could be potential avenues for future research.

Step 7: Conclusion and Final Thoughts

At the end of your article review, wrap it up with a brief and powerful conclusion. Give a summary of your main points and overall thoughts about the article. 

Point out its importance to the field and the impact of the study. Finish off with a thought-provoking conclusion. Give the reader a sense of finality and emphasize the need for additional research or discussion.

For instance

“In conclusion, John Smith’s article provides valuable insights into the detrimental effects of excessive social media usage on adolescent mental health. 

While the research has limitations, it serves as a starting point for further investigation in this rapidly evolving field. 

By addressing the research gaps and implementing targeted interventions: 

We can strive to promote a healthier relationship between social media and mental well-being in our digitally connected society.”

Step 8: Editing and Proofreading

Before submission, set aside some time for editing and proofreading. 

Ensure everything makes sense and everything is correct. Check out how it reads and if your points come across clearly. Get feedback from other people to get a different point of view and make it even better.

Types of Article Reviews

In college, you might be asked to write different types of review articles, including: 

Narrative Review

This type of review needs you to look into the author’s background and experiences. You have to go through the specialist’s theories and practices and compare them. For the success of a narrative review, ensure that your arguments are qualitative and make sense.

Evidence Review

For a solid evidence paper, you got to put in the work and study the topic. You’ll need to research the facts, analyze the author’s ideas, their effects, and more. 

Systematic Review

This task involves reviewing a bunch of research papers and summarizing the existing knowledge about a certain subject. A systematic paper type uses an organized approach and expects you to answer questions linked to the research.

Tips for Writing a Great Article Review

Here are some expert tips you could use to write an exceptional article review:

1. Figure out the main points you want to cover and why they matter.

  • It will help you zero in on the key points.

2. Look for and assess pertinent sources, both from the past and present.

  • It will give you a better understanding of the article you’re looking at.

3. Come Up with a Catchy Title, Summarize Your Topic in an Abstract, and Select Keywords

  • It will help people read your review and get a good idea of what it’s about.

4. Write the main point of a review along with introducing the topic. 

  • It should help readers get a better grasp of the topic.

Outline for Writing a Good Article Review

Here’s an outline to write an excellent article review. 

Introduction

– Begin with a summary of the article 

– Put in background knowledge of the topic 

– State why you are writing the review 

– Give an overview of the article’s main points 

– Figure out why the author choose to write something 

– Look at the article and consider what it does well and what it could have done better.

– Highlight the shortcomings in the article

– Restate why you are writing the review 

– Sum up the main points in a few sentences 

– Suggest what could be achieved in the future research 

Review Article Example

Title: “The Power of Vulnerability: A Review of Brené Brown’s Daring Greatly”

Introduction:

In her revolutionary book “Daring Greatly,” 

Brené Brown, a renowned researcher and storyteller. Delves into vulnerability and how it can positively impact our lives, both professionally and personally. 

Brown’s work has gained lots of praise. Since it resonates with people looking to build real connections in a world that often feels isolated. 

This article looks to recap the main ideas and concepts from “Daring Greatly.” Also explains why it is such a captivating and insightful read.

Summary of Key Ideas:

“Daring Greatly” is all about how the vulnerability isn’t a sign of being weak. but it’s actually what it takes to be brave, strong and live a full life. 

Brene Brown examines how society and culture can make it hard to be vulnerable. And, how fear of being judged or shamed stops us from being our authentic selves.

The book puts a lot of emphasis on shame and how it affects us. 

Brown explains that shame thrives when it’s kept hidden away and can only be cured by being open, understanding, and compassionate. 

By admitting our weaknesses, we can create meaningful connections and a sense of community.

Brown looks into the connection between being open to vulnerability and unleashing creative leadership and innovation. 

She uses her own experiences and research to support her viewpoint. The book also gives useful advice on how to include vulnerability in different parts of life. Such as relationships, parenting, and the workplace.

Strengths of the Book:

Brown’s book is remarkable for her ability to mix her own experiences with comprehensive research. Combining her stories and evidence makes the material engaging and easy to understand. 

Plus, her writing style is so friendly that readers feel they’re being acknowledged and accepted.

There’s advice on how to be kind to yourself. Set your limits, and accept that things won’t always be perfect. It’s like a toolkit to help you build strength and make positive changes.

Final Verdict

This book is really helpful for everyone, no matter who you are. It can help you figure out how to grow in life, have better relationships, and become a better leader. Plus, since it applies to all kinds of people, everyone can get something out of it.

If you want to write a great article review, it’s important to pick the right article, understand and analyze it critically. Finally, express your thoughts on it clearly. Ensure to stay impartial, back up your points with evidence, and write clearly and coherently.

Still if you are having troubles writing an article review, don’t hesitate to count on the expertise of  our writers .

Order Original Papers & Essays

Your First Custom Paper Sample is on Us!

timely deliveries

Timely Deliveries

premium quality

No Plagiarism & AI

unlimited revisions

100% Refund

Calculate Your Order Price

Related blogs.

blog-img

Connections with Writers and support

safe service

Privacy and Confidentiality Guarantee

quality-score

Average Quality Score

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

OWL logo

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

  • An introductory paragraph that explains what your working topic and thesis is
  • A forecast of key topics or texts that will appear in the review
  • Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
  • Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
  • Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
  • Critically Evaluate: Mention the strengths and weaknesses of your sources
  • Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

  • Summarize the key findings you have taken from the literature and emphasize their significance
  • Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

  • Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
  • Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
  • Qualitative versus quantitative research
  • Empirical versus theoretical scholarship
  • Divide the research by sociological, historical, or cultural sources
  • Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

  • It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
  • Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
  • Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
  • Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Page Content

Overview of the review report format, the first read-through, first read considerations, spotting potential major flaws, concluding the first reading, rejection after the first reading, before starting the second read-through, doing the second read-through, the second read-through: section by section guidance, how to structure your report, on presentation and style, criticisms & confidential comments to editors, the recommendation, when recommending rejection, additional resources, step by step guide to reviewing a manuscript.

When you receive an invitation to peer review, you should be sent a copy of the paper's abstract to help you decide whether you wish to do the review. Try to respond to invitations promptly - it will prevent delays. It is also important at this stage to declare any potential Conflict of Interest.

The structure of the review report varies between journals. Some follow an informal structure, while others have a more formal approach.

" Number your comments!!! " (Jonathon Halbesleben, former Editor of Journal of Occupational and Organizational Psychology)

Informal Structure

Many journals don't provide criteria for reviews beyond asking for your 'analysis of merits'. In this case, you may wish to familiarize yourself with examples of other reviews done for the journal, which the editor should be able to provide or, as you gain experience, rely on your own evolving style.

Formal Structure

Other journals require a more formal approach. Sometimes they will ask you to address specific questions in your review via a questionnaire. Or they might want you to rate the manuscript on various attributes using a scorecard. Often you can't see these until you log in to submit your review. So when you agree to the work, it's worth checking for any journal-specific guidelines and requirements. If there are formal guidelines, let them direct the structure of your review.

In Both Cases

Whether specifically required by the reporting format or not, you should expect to compile comments to authors and possibly confidential ones to editors only.

Reviewing with Empathy

Following the invitation to review, when you'll have received the article abstract, you should already understand the aims, key data and conclusions of the manuscript. If you don't, make a note now that you need to feedback on how to improve those sections.

The first read-through is a skim-read. It will help you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper.

Keep a pen and paper handy when skim-reading.

Try to bear in mind the following questions - they'll help you form your overall impression:

  • What is the main question addressed by the research? Is it relevant and interesting?
  • How original is the topic? What does it add to the subject area compared with other published material?
  • Is the paper well written? Is the text clear and easy to read?
  • Are the conclusions consistent with the evidence and arguments presented? Do they address the main question posed?
  • If the author is disagreeing significantly with the current academic consensus, do they have a substantial case? If not, what would be required to make their case credible?
  • If the paper includes tables or figures, what do they add to the paper? Do they aid understanding or are they superfluous?

While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on.

Editors say, " Specific recommendations for remedying flaws are VERY welcome ."

Examples of possibly major flaws include:

  • Drawing a conclusion that is contradicted by the author's own statistical or qualitative evidence
  • The use of a discredited method
  • Ignoring a process that is known to have a strong influence on the area under study

If experimental design features prominently in the paper, first check that the methodology is sound - if not, this is likely to be a major flaw.

You might examine:

  • The sampling in analytical papers
  • The sufficient use of control experiments
  • The precision of process data
  • The regularity of sampling in time-dependent studies
  • The validity of questions, the use of a detailed methodology and the data analysis being done systematically (in qualitative research)
  • That qualitative research extends beyond the author's opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

Major Flaws in Information

If methodology is less of an issue, it's often a good idea to look at the data tables, figures or images first. Especially in science research, it's all about the information gathered. If there are critical flaws in this, it's very likely the manuscript will need to be rejected. Such issues include:

  • Insufficient data
  • Unclear data tables
  • Contradictory data that either are not self-consistent or disagree with the conclusions
  • Confirmatory data that adds little, if anything, to current understanding - unless strong arguments for such repetition are made

If you find a major problem, note your reasoning and clear supporting evidence (including citations).

After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first summarizing the research question addressed and the second the contribution of the work. If the journal has a prescribed reporting format, this draft will still help you compose your thoughts.

The First Paragraph

This should state the main question addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

  • Help the editor properly contextualize the research and add weight to your judgement
  • Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do
  • Focus on successful aspects of the paper so the author gets a sense of what they've done well

The Second Paragraph

This should provide a conceptual overview of the contribution of the research. So consider:

  • Is the paper's premise interesting and important?
  • Are the methods used appropriate?
  • Do the data support the conclusions?

After drafting these two paragraphs, you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). Or whether it is publishable in principle and merits a detailed, careful read through.

Even if you are coming to the opinion that an article has serious flaws, make sure you read the whole paper. This is very important because you may find some really positive aspects that can be communicated to the author. This could help them with future submissions.

A full read-through will also make sure that any initial concerns are indeed correct and fair. After all, you need the context of the whole paper before deciding to reject. If you still intend to recommend rejection, see the section "When recommending rejection."

Once the paper has passed your first read and you've decided the article is publishable in principle, one purpose of the second, detailed read-through is to help prepare the manuscript for publication. You may still decide to recommend rejection following a second reading.

" Offer clear suggestions for how the authors can address the concerns raised. In other words, if you're going to raise a problem, provide a solution ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Preparation

To save time and simplify the review:

  • Don't rely solely upon inserting comments on the manuscript document - make separate notes
  • Try to group similar concerns or praise together
  • If using a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes - it helps later
  • Note line numbers of text upon which your notes are based - this helps you find items again and also aids those reading your review

Now that you have completed your preparations, you're ready to spend an hour or so reading carefully through the manuscript.

As you're reading through the manuscript for a second time, you'll need to keep in mind the argument's construction, the clarity of the language and content.

With regard to the argument’s construction, you should identify:

  • Any places where the meaning is unclear or ambiguous
  • Any factual errors
  • Any invalid arguments

You may also wish to consider:

  • Does the title properly reflect the subject of the paper?
  • Does the abstract provide an accessible summary of the paper?
  • Do the keywords accurately reflect the content?
  • Is the paper an appropriate length?
  • Are the key messages short, accurate and clear?

Not every submission is well written. Part of your role is to make sure that the text’s meaning is clear.

Editors say, " If a manuscript has many English language and editing issues, please do not try and fix it. If it is too bad, note that in your review and it should be up to the authors to have the manuscript edited ."

If the article is difficult to understand, you should have rejected it already. However, if the language is poor but you understand the core message, see if you can suggest improvements to fix the problem:

  • Are there certain aspects that could be communicated better, such as parts of the discussion?
  • Should the authors consider resubmitting to the same journal after language improvements?
  • Would you consider looking at the paper again once these issues are dealt with?

On Grammar and Punctuation

Your primary role is judging the research content. Don't spend time polishing grammar or spelling. Editors will make sure that the text is at a high standard before publication. However, if you spot grammatical errors that affect clarity of meaning, then it's important to highlight these. Expect to suggest such amendments - it's rare for a manuscript to pass review with no corrections.

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

1. The Introduction

A well-written introduction:

  • Sets out the argument
  • Summarizes recent research related to the topic
  • Highlights gaps in current understanding or conflicts in current knowledge
  • Establishes the originality of the research aims by demonstrating the need for investigations in the topic area
  • Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript

Originality and Topicality

Originality and topicality can only be established in the light of recent authoritative research. For example, it's impossible to argue that there is a conflict in current understanding by referencing articles that are 10 years old.

Authors may make the case that a topic hasn't been investigated in several years and that new research is required. This point is only valid if researchers can point to recent developments in data gathering techniques or to research in indirectly related fields that suggest the topic needs revisiting. Clearly, authors can only do this by referencing recent literature. Obviously, where older research is seminal or where aspects of the methodology rely upon it, then it is perfectly appropriate for authors to cite some older papers.

Editors say, "Is the report providing new information; is it novel or just confirmatory of well-known outcomes ?"

It's common for the introduction to end by stating the research aims. By this point you should already have a good impression of them - if the explicit aims come as a surprise, then the introduction needs improvement.

2. Materials and Methods

Academic research should be replicable, repeatable and robust - and follow best practice.

Replicable Research

This makes sufficient use of:

  • Control experiments
  • Repeated analyses
  • Repeated experiments

These are used to make sure observed trends are not due to chance and that the same experiment could be repeated by other researchers - and result in the same outcome. Statistical analyses will not be sound if methods are not replicable. Where research is not replicable, the paper should be recommended for rejection.

Repeatable Methods

These give enough detail so that other researchers are able to carry out the same research. For example, equipment used or sampling methods should all be described in detail so that others could follow the same steps. Where methods are not detailed enough, it's usual to ask for the methods section to be revised.

Robust Research

This has enough data points to make sure the data are reliable. If there are insufficient data, it might be appropriate to recommend revision. You should also consider whether there is any in-built bias not nullified by the control experiments.

Best Practice

During these checks you should keep in mind best practice:

  • Standard guidelines were followed (e.g. the CONSORT Statement for reporting randomized trials)
  • The health and safety of all participants in the study was not compromised
  • Ethical standards were maintained

If the research fails to reach relevant best practice standards, it's usual to recommend rejection. What's more, you don't then need to read any further.

3. Results and Discussion

This section should tell a coherent story - What happened? What was discovered or confirmed?

Certain patterns of good reporting need to be followed by the author:

  • They should start by describing in simple terms what the data show
  • They should make reference to statistical analyses, such as significance or goodness of fit
  • Once described, they should evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research
  • The outcome should be a critical analysis of the data collected

Discussion should always, at some point, gather all the information together into a single whole. Authors should describe and discuss the overall story formed. If there are gaps or inconsistencies in the story, they should address these and suggest ways future research might confirm the findings or take the research forward.

4. Conclusions

This section is usually no more than a few paragraphs and may be presented as part of the results and discussion, or in a separate section. The conclusions should reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If the conclusions are not evidence-based, it's appropriate to ask for them to be re-written.

5. Information Gathered: Images, Graphs and Data Tables

If you find yourself looking at a piece of information from which you cannot discern a story, then you should ask for improvements in presentation. This could be an issue with titles, labels, statistical notation or image quality.

Where information is clear, you should check that:

  • The results seem plausible, in case there is an error in data gathering
  • The trends you can see support the paper's discussion and conclusions
  • There are sufficient data. For example, in studies carried out over time are there sufficient data points to support the trends described by the author?

You should also check whether images have been edited or manipulated to emphasize the story they tell. This may be appropriate but only if authors report on how the image has been edited (e.g. by highlighting certain parts of an image). Where you feel that an image has been edited or manipulated without explanation, you should highlight this in a confidential comment to the editor in your report.

6. List of References

You will need to check referencing for accuracy, adequacy and balance.

Where a cited article is central to the author's argument, you should check the accuracy and format of the reference - and bear in mind different subject areas may use citations differently. Otherwise, it's the editor’s role to exhaustively check the reference section for accuracy and format.

You should consider if the referencing is adequate:

  • Are important parts of the argument poorly supported?
  • Are there published studies that show similar or dissimilar trends that should be discussed?
  • If a manuscript only uses half the citations typical in its field, this may be an indicator that referencing should be improved - but don't be guided solely by quantity
  • References should be relevant, recent and readily retrievable

Check for a well-balanced list of references that is:

  • Helpful to the reader
  • Fair to competing authors
  • Not over-reliant on self-citation
  • Gives due recognition to the initial discoveries and related work that led to the work under assessment

You should be able to evaluate whether the article meets the criteria for balanced referencing without looking up every reference.

7. Plagiarism

By now you will have a deep understanding of the paper's content - and you may have some concerns about plagiarism.

Identified Concern

If you find - or already knew of - a very similar paper, this may be because the author overlooked it in their own literature search. Or it may be because it is very recent or published in a journal slightly outside their usual field.

You may feel you can advise the author how to emphasize the novel aspects of their own study, so as to better differentiate it from similar research. If so, you may ask the author to discuss their aims and results, or modify their conclusions, in light of the similar article. Of course, the research similarities may be so great that they render the work unoriginal and you have no choice but to recommend rejection.

"It's very helpful when a reviewer can point out recent similar publications on the same topic by other groups, or that the authors have already published some data elsewhere ." (Editor feedback)

Suspected Concern

If you suspect plagiarism, including self-plagiarism, but cannot recall or locate exactly what is being plagiarized, notify the editor of your suspicion and ask for guidance.

Most editors have access to software that can check for plagiarism.

Editors are not out to police every paper, but when plagiarism is discovered during peer review it can be properly addressed ahead of publication. If plagiarism is discovered only after publication, the consequences are worse for both authors and readers, because a retraction may be necessary.

For detailed guidelines see COPE's Ethical guidelines for reviewers and Wiley's Best Practice Guidelines on Publishing Ethics .

8. Search Engine Optimization (SEO)

After the detailed read-through, you will be in a position to advise whether the title, abstract and key words are optimized for search purposes. In order to be effective, good SEO terms will reflect the aims of the research.

A clear title and abstract will improve the paper's search engine rankings and will influence whether the user finds and then decides to navigate to the main article. The title should contain the relevant SEO terms early on. This has a major effect on the impact of a paper, since it helps it appear in search results. A poor abstract can then lose the reader's interest and undo the benefit of an effective title - whilst the paper's abstract may appear in search results, the potential reader may go no further.

So ask yourself, while the abstract may have seemed adequate during earlier checks, does it:

  • Do justice to the manuscript in this context?
  • Highlight important findings sufficiently?
  • Present the most interesting data?

Editors say, " Does the Abstract highlight the important findings of the study ?"

If there is a formal report format, remember to follow it. This will often comprise a range of questions followed by comment sections. Try to answer all the questions. They are there because the editor felt that they are important. If you're following an informal report format you could structure your report in three sections: summary, major issues, minor issues.

  • Give positive feedback first. Authors are more likely to read your review if you do so. But don't overdo it if you will be recommending rejection
  • Briefly summarize what the paper is about and what the findings are
  • Try to put the findings of the paper into the context of the existing literature and current knowledge
  • Indicate the significance of the work and if it is novel or mainly confirmatory
  • Indicate the work's strengths, its quality and completeness
  • State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked

Major Issues

  • Are there any major flaws? State what they are and what the severity of their impact is on the paper
  • Has similar work already been published without the authors acknowledging this?
  • Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?
  • If major revisions are required, try to indicate clearly what they are
  • Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?
  • Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section

Minor Issues

  • Are there places where meaning is ambiguous? How can this be corrected?
  • Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?
  • Are there any factual, numerical or unit errors? If so, what are they?
  • Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not

Your review should ultimately help the author improve their article. So be polite, honest and clear. You should also try to be objective and constructive, not subjective and destructive.

You should also:

  • Write clearly and so you can be understood by people whose first language is not English
  • Avoid complex or unusual words, especially ones that would even confuse native speakers
  • Number your points and refer to page and line numbers in the manuscript when making specific comments
  • If you have been asked to only comment on specific parts or aspects of the manuscript, you should indicate clearly which these are
  • Treat the author's work the way you would like your own to be treated

Most journals give reviewers the option to provide some confidential comments to editors. Often this is where editors will want reviewers to state their recommendation - see the next section - but otherwise this area is best reserved for communicating malpractice such as suspected plagiarism, fraud, unattributed work, unethical procedures, duplicate publication, bias or other conflicts of interest.

However, this doesn't give reviewers permission to 'backstab' the author. Authors can't see this feedback and are unable to give their side of the story unless the editor asks them to. So in the spirit of fairness, write comments to editors as though authors might read them too.

Reviewers should check the preferences of individual journals as to where they want review decisions to be stated. In particular, bear in mind that some journals will not want the recommendation included in any comments to authors, as this can cause editors difficulty later - see Section 11 for more advice about working with editors.

You will normally be asked to indicate your recommendation (e.g. accept, reject, revise and resubmit, etc.) from a fixed-choice list and then to enter your comments into a separate text box.

Recommending Acceptance

If you're recommending acceptance, give details outlining why, and if there are any areas that could be improved. Don't just give a short, cursory remark such as 'great, accept'. See Improving the Manuscript

Recommending Revision

Where improvements are needed, a recommendation for major or minor revision is typical. You may also choose to state whether you opt in or out of the post-revision review too. If recommending revision, state specific changes you feel need to be made. The author can then reply to each point in turn.

Some journals offer the option to recommend rejection with the possibility of resubmission – this is most relevant where substantial, major revision is necessary.

What can reviewers do to help? " Be clear in their comments to the author (or editor) which points are absolutely critical if the paper is given an opportunity for revisio n." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Recommending Rejection

If recommending rejection or major revision, state this clearly in your review (and see the next section, 'When recommending rejection').

Where manuscripts have serious flaws you should not spend any time polishing the review you've drafted or give detailed advice on presentation.

Editors say, " If a reviewer suggests a rejection, but her/his comments are not detailed or helpful, it does not help the editor in making a decision ."

In your recommendations for the author, you should:

  • Give constructive feedback describing ways that they could improve the research
  • Keep the focus on the research and not the author. This is an extremely important part of your job as a reviewer
  • Avoid making critical confidential comments to the editor while being polite and encouraging to the author - the latter may not understand why their manuscript has been rejected. Also, they won't get feedback on how to improve their research and it could trigger an appeal

Remember to give constructive criticism even if recommending rejection. This helps developing researchers improve their work and explains to the editor why you felt the manuscript should not be published.

" When the comments seem really positive, but the recommendation is rejection…it puts the editor in a tough position of having to reject a paper when the comments make it sound like a great paper ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Visit our Wiley Author Learning and Training Channel for expert advice on peer review.

Watch the video, Ethical considerations of Peer Review

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • My Account Login
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Open access
  • Published: 21 March 2024

Expert review of the science underlying nature-based climate solutions

  • B. Buma   ORCID: orcid.org/0000-0003-2402-7737 1 , 2   na1 ,
  • D. R. Gordon   ORCID: orcid.org/0000-0001-6398-2345 1 , 3   na1 ,
  • K. M. Kleisner 1 ,
  • A. Bartuska 1 , 4 ,
  • A. Bidlack 5 ,
  • R. DeFries   ORCID: orcid.org/0000-0002-3332-4621 6 ,
  • P. Ellis   ORCID: orcid.org/0000-0001-7933-8298 7 ,
  • P. Friedlingstein   ORCID: orcid.org/0000-0003-3309-4739 8 , 9 ,
  • S. Metzger 10   nAff15   nAff16 ,
  • G. Morgan 11 ,
  • K. Novick   ORCID: orcid.org/0000-0002-8431-0879 12 ,
  • J. N. Sanchirico 13 ,
  • J. R. Collins   ORCID: orcid.org/0000-0002-5705-9682 1 , 14 ,
  • A. J. Eagle   ORCID: orcid.org/0000-0003-0841-2379 1 ,
  • R. Fujita 1 ,
  • E. Holst 1 ,
  • J. M. Lavallee   ORCID: orcid.org/0000-0002-3028-7087 1 ,
  • R. N. Lubowski 1   nAff17 ,
  • C. Melikov 1   nAff18 ,
  • L. A. Moore   ORCID: orcid.org/0000-0003-0239-6080 1   nAff19 ,
  • E. E. Oldfield   ORCID: orcid.org/0000-0002-6181-1267 1 ,
  • J. Paltseva 1   nAff20 ,
  • A. M. Raffeld   ORCID: orcid.org/0000-0002-5036-6460 1 ,
  • N. A. Randazzo 1   nAff21   nAff22 ,
  • C. Schneider 1 ,
  • N. Uludere Aragon 1   nAff23 &
  • S. P. Hamburg 1  

Nature Climate Change ( 2024 ) Cite this article

10k Accesses

43 Altmetric

Metrics details

  • Climate-change ecology
  • Climate-change mitigation
  • Environmental impact

Viable nature-based climate solutions (NbCS) are needed to achieve climate goals expressed in international agreements like the Paris Accord. Many NbCS pathways have strong scientific foundations and can deliver meaningful climate benefits but effective mitigation is undermined by pathways with less scientific certainty. Here we couple an extensive literature review with an expert elicitation on 43 pathways and find that at present the most used pathways, such as tropical forest conservation, have a solid scientific basis for mitigation. However, the experts suggested that some pathways, many with carbon credit eligibility and market activity, remain uncertain in terms of their climate mitigation efficacy. Sources of uncertainty include incomplete GHG measurement and accounting. We recommend focusing on resolving those uncertainties before broadly scaling implementation of those pathways in quantitative emission or sequestration mitigation plans. If appropriate, those pathways should be supported for their cobenefits, such as biodiversity and food security.

Similar content being viewed by others

review article research paper

Australian human-induced native forest regeneration carbon offset projects have limited impact on changes in woody vegetation cover and carbon removals

Andrew Macintosh, Don Butler, … Paul Summerfield

review article research paper

Meta-analysis shows the impacts of ecological restoration on greenhouse gas emissions

Tiehu He, Weixin Ding, … Quanfa Zhang

review article research paper

Historical impacts of grazing on carbon stocks and climate mitigation opportunities

Shuai Ren, César Terrer, … Dan Liu

Nature-based climate solutions (NbCS) are conservation, restoration and improved management strategies (pathways) in natural and working ecosystems with the primary motivation to mitigate GHG emissions and remove CO 2 from the atmosphere 1 (similar to ecosystem-based mitigation 2 ). GHG mitigation through ecosystem stewardship is integral to meeting global climate goals, with the greatest benefit coming from near-term maximization of emission reductions, followed by CO 2 removal 3 . Many countries (for example, Indonesia, China and Colombia) use NbCS to demonstrate progress toward national climate commitments.

The scope of NbCS is narrower than that of nature-based solutions (NbS) which include interventions that prioritize non-climate benefits alongside climate (for example, biodiversity, food provisioning and water quality improvement) 4 . In many cases, GHG mitigation is considered a cobenefit that results from NbS actions focused on these other challenges 2 . In contrast, NbCS are broader than natural climate solutions, which are primarily focused on climate mitigation through conservation, restoration and improved land management, generally not moving ecosystems beyond their unmodified structure, function or composition 5 . NbCS may involve moving systems beyond their original function, for example by cultivating macroalgae in water deeper than their natural habitat.

The promise of NbCS has generated a proliferation of interest in using them in GHG mitigation plans 6 , 7 ; 104 of the 168 signatories to the Paris Accord included nature-based actions as part of their mitigation plans 8 . Success in long-term GHG management requires an accurate accounting of inputs and outputs to the atmosphere at scale, so NbCS credits must have robust, comprehensive and transparent scientific underpinnings 9 . Given the urgency of the climate problem, our goal is to identify NbCS pathways with a sufficient scientific foundation to provide broad confidence in their potential GHG mitigation impact, provide resources for confident implementation and identify priority research areas in more uncertain pathways. Evaluating implementation of mitigation projects is beyond our scope; this effort focuses on understanding the underlying science. The purpose is not evaluating any specific carbon crediting protocol or implementation framework but rather the current state of scientific understanding necessary to provide confidence in any NbCS.

In service of this goal, we first investigated nine biomes (boreal forests, coastal marine (salt marsh, mangrove, seagrass and coral reef), freshwater wetlands, grasslands, open ocean (large marine animal and mesopelagic zone biomass, seabed), peatlands, shrublands, temperate forests and tropical forests) and three cultivation types (agroforestry, croplands and macroalgae aquaculture); these were chosen because of their identified potential scale of global impact. In this context, impact is assessed as net GHG mitigation: the CO 2 sequestered or emissions reduced, for example, discounted by understood simultaneous emissions of other GHG (as when N 2 O is released simultaneously with carbon sequestration in cropland soils). From there, we identified 43 NbCS pathways which have been formally implemented (with or without market action) or informally proposed. We estimated the scale of mitigation impact for each pathway on the basis of this literature and, as a proxy measure of NbCS implementation, determined eligibility and activity under existing carbon crediting protocols. Eligibility means that the pathway is addressed by an existing GHG mitigation protocol; market activity means that credits are actively being bought under those eligibility requirements. We considered pathways across a spectrum from protection to improved management to restoration to manipulated systems, but some boundaries were necessary. We excluded primarily abiotically driven pathways (for example, ocean alkalinity enhancement) or where major land use or land-use trade-offs exist (for example, afforestation) 10 , 11 , 12 . Of the 43 pathways, 79% are at present eligible for carbon crediting (sometimes under several methodologies) and at least 65% of those have been implemented (Supplementary Table 1 ). This review was then appraised by 30 independent scholars (at least three per pathway; a complete review synthesis is given in the Supplementary Data ).

Consolidation of a broad body of scientific knowledge, with inherent variance, requires expert judgement. We used an expert elicitation process 13 , 14 , 15 with ten experts to place each proposed NbCS pathway into one of three readiness categories following their own assessment of the scientific literature, categorized by general sources of potential uncertainty: category 1, sufficient scientific basis to support a high-quality carbon accounting system or to support the development of such a system today; category 2, a >25% chance that focused research and reasonable funding would support development of high-quality carbon accounting (that is, move to category 1) within 5 years; or category 3, a <25% chance of development of high-quality carbon accounting within 5 years (for example, due to measurement challenges, unconstrained leakage, external factors which constrain viability).

If an expert ranked a pathway as category 2, they were also asked to rank general research needs to resolve: leakage/displacement (spillover to other areas), measuring, reporting and verification (the ability to quantify all salient stocks and fluxes), basic mechanisms of action (fundamental science), durability (ability to predict or compensate for uncertainty in timescale of effectiveness due to disturbances, climate change, human activity or other factors), geographic uncertainty (place-to-place variation), scaling potential (ability to estimate impact) and setting of a baseline (ability to estimate additionality over non-action; a counterfactual). To avoid biasing towards a particular a priori framework for evaluation of the scientific literature, reviewers could use their own framework for evaluating the NbCS literature about potential climate impact and so could choose to ignore or add relevant categorizations as well. Any pathway in category 1 would not need fundamental research for implementation; research gaps were considered too extensive for useful guidance on reducing uncertainty in category 3 pathways. Estimates of the global scale of likely potential impact (PgCO 2 e yr −1 ) and cobenefits were also collected from expert elicitors. See Methods and Supplementary Information for the survey instrument.

Four pathways with the highest current carbon market activity and high mitigation potential (tropical and temperate forest conservation and reforestation; Table 1 and Supplementary Data ), were consistently rated as high-confidence pathways in the expert elicitation survey. Other NbCS pathways, especially in the forestry sector, were rated relatively strongly by the experts for both confidence in scientific basis and scale of potential impact, with some spread across the experts (upper right quadrant, Fig. 1 ). Conversely, 13 pathways were consistently marked by experts as currently highly uncertain/low confidence (median score across experts: 2.5–3.0) and placed in category 3 (for example, cropland microbial amendments and coral reef restoration; Supplementary Tables 1 and 2 ). For the full review, including crediting protocols currently used, literature estimates of scale and details of sub-pathways, see Supplementary Data .

figure 1

Pathways in the upper right quadrant have both high confidence in the scientific foundations and the largest potential scale of global impact; pathways in the lower left have the lowest confidence in our present scientific body of knowledge and an estimated smaller potential scale of impact. Designations of carbon credit eligibility under existing protocols and market activity at the present time are noted. Grassland enhanced mineral weathering (EMW) is not shown (mean category rating 2.9) as no scale of impact was estimated. See Supplementary Table 1 for specific pathway data. Bars represent 20th to 80th percentiles of individual estimates, if there was variability in estimates. A small amount of random noise was added to avoid overlap.

The experts assessed 26 pathways as having average confidence scores between 1.5 and 2.4, suggesting the potential for near-term resolution of uncertainties. This categorization arose from either consensus amongst experts on the uncertain potential (for example, boreal forest reforestation consistently rated category 2, with primary concerns about durability) or because experts disagreed, with some ranking category 1 and others category 3 (for example, pasture management). We note that where expert disagreement exists (seen as the spread of responses in Fig. 1 and Supplementary Table 1 ; also see Data availability for link to original data), this suggests caution against overconfidence in statements about these pathways. These results also suggest that confidence may be increased by targeted research on the identified sources of uncertainty (Supplementary Table 3 ).

Sources of uncertainty

Durability and baseline-setting were rated as high sources of uncertainty across all pathways ranked as category 2 by the experts (mean ratings of 3.6 and 3.4 out of 5, respectively; Supplementary Table 3 ). Understanding of mechanisms and geographic spread had the lowest uncertainty ratings (2.1 and 2.3, respectively), showing confidence in the basic science. Different subsets of pathways had different prioritizations, however, suggesting different research needs: forest-centric pathways were most uncertain in their durability and additionality (3.8 and 3.4, respectively), suggesting concerns about long-term climate and disturbance trajectories. Agricultural and grassland systems, however, had higher uncertainty in measurement methods and additionality (3.9 and 3.5 respectively). Although there were concerns about durability from some experts (for example, due to sea-level rise), some coastal blue carbon pathways such as mangrove restoration (mean category ranking: 1.7 (20th to 80th percentile 1.0–2.0)) have higher confidence than others (for example, seagrass restoration: mean category ranking 2.8, 20th to 80th percentile 2.6–3.0)), which are relatively poorly constrained in terms of net radiative forcing potential despite a potentially large carbon impact (seagrass median: 1.60 PgCO 2 e yr −1 ; see Supplementary Data for more scientific literature estimates).

Scale of impact

For those pathways with lower categorization by the expert elicitation (category 2 or 3) at the present time, scale of global impact is a potential heuristic for prioritizing further research. High variability, often two orders of magnitude, was evident in the mean estimated potential PgCO 2 e yr −1 impacts for the different pathways (Fig. 1 and Supplementary Table 2 ) and the review of the literature found even larger ranges produced by individual studies (Supplementary Data ). A probable cause of this wide range was different constraints on the estimated potential, with some studies focusing on potential maximum impact and others on more constrained realizable impacts. Only avoided loss of tropical forest and cropland biochar amendment were consistently estimated as having the likely potential to mitigate >2 PgCO 2 e yr −1 , although biochar was considered more uncertain by experts due to other factors germane to its overall viability as a climate solution, averaging a categorization of 2.2. The next four highest potential impact pathways, ranging from 1.6 to 1.7 PgCO 2 e yr −1 , spanned the spectrum from high readiness (temperate forest restoration) to moderate (cropland conversion from annual to perennial vegetation and grassland restoration) to low (seagrass restoration, with main uncertainties around scale of potential impact and durability).

There was high variability in the elicitors’ estimated potential scale of impact, even in pathways with strong support, such as tropical forest avoided loss (20th to 80th percentile confidence interval: 1–8 PgCO 2 e yr −1 ), again emphasizing the importance of consistent definitions and constraints on how NbCS are measured, evaluated and then used in broad-scale climate change mitigation planning and budgeting. Generally, as pathway readiness decreased (moving from category 1 to 3), the elicitor-estimated estimates of GHG mitigation potential decreased (Supplementary Fig. 1 ). Note that individual studies from the scientific literature may have higher or lower estimates (Supplementary Data ).

Expert elicitation meta-analyses suggest that 6–12 responses are sufficient for a robust and stable quantification of responses 15 . We tested that assumption via a Monte Carlo-based sensitivity assessment. Readiness categorizations by the ten experts were robust to a Monte Carlo simulation test, where further samples were randomly drawn from the observed distribution of responses: mean difference between the original and the boot-strapped data was 0.02 (s.d. = 0.05) with an absolute difference average of 0.06 (s.d. = 0.06). The maximum difference in readiness categorization means across all pathways was 0.20 (s.d. = 0.20) (Supplementary Table 2 ). The full dataset of responses is available online (see ʻData availabilityʼ).

These results highlight opportunities to accelerate implementation of NbCS in well-supported pathways and identify critical research needs in others (Fig. 1 ). We suggest focusing future efforts on resolving identified uncertainties for pathways at the intersection between moderate average readiness (for example, mean categorizations between ~1.5 and 2.0) and high potential impact (for example, median >0.5 PgCO 2 e yr −1 ; Supplementary Table 1 ): agroforestry, improved tropical and temperate forest management, tropical and boreal peatlands avoided loss and peatland restoration. Many, although not all, experts identified durability and baseline/additionality as key concerns to resolve in those systems; research explicitly targeted at those specific uncertainties (Supplementary Table 3 ) could rapidly improve confidence in those pathways.

We recommend a secondary research focus on the lower ranked (mean category 2.0 to 3.0) pathways with estimated potential impacts >1 PgCO 2 e yr −1 (Supplementary Fig. 2 ). For these pathways, explicit, quantitative incorporation into broad-scale GHG management plans will require further focus on systems-level carbon/GHG understandings to inspire confidence at all stages of action and/or identifying locations likely to support durable GHG mitigation, for example ref. 16 . Examples of this group include avoided loss and degradation of boreal forests (for example, fire, pests and pathogens and albedo 16 ) and effective mesopelagic fishery management, which some individual studies estimate would avoid future reductions of the currently sequestered 1.5–2.0 PgC yr −1 (refs. 17 , 18 ). These pathways may turn out to have higher or lower potential than the expert review suggests, on the basis of individual studies (Supplementary Data ) but strong support will require further, independent verification of that potential.

We note that category 3 rankings by expert elicitation do not necessarily imply non-viability but simply that much more research is needed to confidently incorporate actions into quantitative GHG mitigation plans. We found an unsurprising trend of lower readiness categorization with lower pathway familiarity (Supplementary Fig. 3 ). This correlation may result from two, non-exclusive potential causes: (1) lower elicitor expertise in some pathways (inevitable, although the panel was explicitly chosen for global perspectives, connections and diverse specialties) and (2) an actual lack of scientific evidence in the literature, which leads to that self-reported lack of familiarity, a common finding in the literature review (Supplementary Data ). Both explanations suggest a need to better consolidate, develop and disseminate the science in each pathway for global utility and recognition.

Our focus on GHG-related benefits in no way diminishes the substantial conservation, environmental and social cobenefits of these pathways (Supplementary Table 4 ), which often exceed their perceived climate benefits 1 , 19 , 20 , 21 . Where experts found climate impacts to remain highly uncertain but other NbS benefits are clear (for example, biodiversity and water quality; Supplementary Table 4 ), other incentives or financing mechanisms independent of carbon crediting should be pursued. While the goals here directly relate to using NbCS as a reliably quantifiable part of global climate action planning and thus strong GHG-related scientific foundations, non-climate NbS projects may provide climate benefits that are less well constrained (and thus less useful from a GHG budgeting standpoint) but also valuable. Potential trade-offs, if any, between ecosystem services and management actions, such as biodiversity and positive GHG outcomes, should be explored to ensure the best realization of desired goals 2 .

Finally, our focus in this study was on broad-scale NbCS potential in quantitative mitigation planning because of the principal and necessary role of NbCS in overall global warming targets. We recognize the range of project conditions that may increase, or decrease, the rigour of any pathway outside the global-scale focus here. We did not specifically evaluate the large and increasing number of crediting concepts (by pathway: Supplementary Data ), focusing rather on the underlying scientific body of knowledge within those pathways. Some broad pathways may have better defined sub-pathways within them, with a smaller potential scale of impact but potentially lower uncertainty (for example, macroalgae harvest cycling). Poorly enacted NbCS actions and/or crediting methodologies at project scales may result in loss of benefits even from high-ranking pathways 22 , 23 , 24 and attention to implementation should be paramount. Conversely, strong, careful project-scale methodologies may make lower readiness pathways beneficial for a given site.

Viable NbCS are vital to global climate change mitigation but NbCS pathways that lack strong scientific underpinnings threaten global accounting by potentially overestimating future climate benefits and eroding public trust in rigorous natural solutions. Both the review of the scientific literature and the expert elicitation survey identified high potential ready-to-implement pathways (for example, tropical reforestation), reinforcing present use of NbCS in planning.

However, uncertainty remains about the quantifiable GHG mitigation of some active and nascent NbCS pathways. On the basis of the expert elicitation survey and review of the scientific literature, we are concerned that large-scale implementation of less scientifically well-founded NbCS pathways in mitigation plans may undermine net GHG budget planning; those pathways require more study before they can be confidently promoted at broad scales and life-cycle analyses to integrate system-level emissions when calculating totals. The expert elicitation judgements suggest a precautionary approach to scaling lower confidence pathways until the scientific foundations are strengthened, especially for NbCS pathways with insufficient measurement and monitoring 10 , 24 , 25 or poorly understood or measured net GHG mitigation potentials 16 , 26 , 27 , 28 . While the need to implement more NbCS pathways for reducing GHG emissions and removing carbon from the atmosphere is urgent, advancing the implementation of poorly quantified pathways (in relation to their GHG mitigation efficacy) could give the false impression that they can balance ongoing, fossil emissions, thereby undermining overall support for more viable NbCS pathways. Explicitly targeting research to resolve these uncertainties in the baseline science could greatly bolster confidence in the less-established NbCS pathways, benefiting efforts to reduce GHG concentrations 29 .

The results of this study should inform both market-based mechanisms and non-market approaches to NbCS pathway management. Research and action that elucidates and advances pathways to ensure a solid scientific basis will provide confidence in the foundation for successfully implementing NbCS as a core component of global GHG management.

NbCS pathway selection

We synthesized scientific publications for nine biomes (boreal forests, coastal blue carbon, freshwater wetlands, grasslands, open ocean blue carbon, peatlands, shrublands, temperate forests and tropical forests) and three cultivation types (agroforestry, croplands and macroalgae aquaculture) (hereafter, systems) and the different pathways through which they may be able to remove carbon or reduce GHG emissions. Shrublands and grasslands were considered as independent ecosystems; nonetheless, we acknowledge that there is overlap in the numbers presented here because shrublands are often included with grasslands 5 , 30 , 31 , 32 , 33 .

The 12 systems were chosen because they have each been identified as having potential for emissions reductions or carbon removal at globally relevant scales. Within these systems, we identified 43 pathways which either have carbon credit protocols formally established or informally proposed for review (non-carbon associated credits were not evaluated). We obtained data on carbon crediting protocols from international, national and regional organizations and registries, such as Verra, American Carbon Registry, Climate Action Reserve, Gold Standard, Clean Development Mechanism, FAO and Nori. We also obtained data from the Voluntary Registry Offsets Database developed by the Berkeley Carbon Trading Project and Carbon Direct company 34 . While we found evidence of more Chinese carbon crediting protocols, we were not able to review these because of limited publicly available information. To maintain clarity and avoid misrepresentation, we used the language as written in each protocol. A full list of the organizations and registries for each system can be found in the Supplementary Data .

Literature searches and synthesis

We reviewed scientific literature and reviews (for example, IPCC special reports) to identify studies reporting data on carbon stocks, GHG dynamics and sequestration potential of each system. Peer-reviewed studies and meta-analyses were identified on Scopus, Web of Science and Google Scholar using simple queries combining the specific practice or pathway names or synonyms (for example, no-tillage, soil amendments, reduced stocking rates, improved forest management, avoided forest conversion and degradation, avoided mangrove conversion and degradation) and the following search terms: ‘carbon storage’, ‘carbon stocks’, ‘carbon sequestration’, ‘carbon sequestration potential’, ‘additional carbon storage’, ‘carbon dynamics’, ‘areal extent’ or ‘global’.

The full literature review was conducted between January and October 2021. We solicited an independent, external review of the syntheses (obtaining from at least three external reviewers per natural or working system; see p. 2 of the Supplementary Data ) as a second check against missing key papers or misinterpretation of data. The review was generally completed in March 2022. Data from additional relevant citations were added through October 2022 as they were discovered. For a complete list of all literature cited, see pp. 217–249 of the Supplementary Data .

From candidate papers, the papers were considered if their results/data could be applied to the following central questions:

How much carbon is stored (globally) at present in the system (total and on average per hectare) and what is the confidence?

At the global level, is the system a carbon source or sink at this time? What is the business-as-usual projection for its carbon dynamics?

Is it possible, through active management, to either increase net carbon sequestration in the system or prevent carbon emissions from that system? (Note that other GHG emissions and forcings were included here as well.)

What is the range of estimates for how much extra carbon could be sequestered globally?

How much confidence do we have in the present methods to detect any net increases in carbon sequestration in a system or net changes in areal extent of that?

From each paper, quantitative estimates for the above questions were extracted for each pathway, including any descriptive information/metadata necessary to understand the estimate. In addition, information on sample size, sampling scheme, geographic coverage, timeline of study, timeline of projections (if applicable) and specific study contexts (for example, wind-break agroforestry) were recorded.

We also tracked where the literature identified trade-offs between carbon sequestered or CO 2 emissions reduced and emissions of other GHG (for example, N 2 O or methane) for questions three and five above. For example, wetland restoration can result in increased CO 2 uptake from the atmosphere. However, it can also increase methane and N 2 O emissions to the atmosphere. Experts were asked to consider the uncertainty in assessing net GHG mitigation as they categorized the NbCS pathways.

Inclusion of each pathway in mitigation protocols and the specific carbon registries involved were also identified. These results are reported (grouped or individually as appropriate) in the Supplementary Data , organized by the central questions and including textual information for interpretation. The data and protocol summaries for each of the 12 systems were reviewed by at least three scientists each and accordingly revised.

These summaries were provided to the expert elicitation group as optional background information.

Unit conversions

Since this synthesis draws on literature from several sources that use different methods and units, all carbon measurements were standardized to the International System of Units (SI units). When referring to total stocks for each system, numbers are reported in SI units of elemental carbon (that is, PgC). When referring to mitigation potential, elemental carbon was converted to CO 2 by multiplying by 3.67. Differences in methodology, such as soil sampling depth, make it difficult to standardize across studies. Where applicable, the specific measurement used to develop each stock estimate is reported.

Expert elicitation process

To assess conclusions brought about by the initial review process described above, we conducted an expert elicitation survey to consolidate and add further, independent assessments to the original literature review. The expert elicitation survey design followed best practice recommendations 14 , with a focus on participant selection, explicitly defining uncertainty, minimizing cognitive and overconfidence biases and clarity of focus. Research on expert elicitation suggests that 6–12 responses are sufficient for a stable quantification of responses 15 . We identified >40 potential experts via a broad survey of leading academics, science-oriented NGO and government agency publications and products. These individuals have published on several NbCS pathways or could represent larger research efforts that spanned the NbCS under consideration. Careful attention was paid to the gender and sectoral breakdown of respondents to ensure equitable representation. Of the invitees, ten completed the full elicitation effort. Experts were offered compensation for their time.

Implementation of the expert elicitation process followed the IDEA protocol 15 . Briefly, after a short introductory interview, the survey was sent to the participants. Results were anonymized and standardized (methods below) and a meeting held with the entire group to discuss the initial results and calibrate understanding of questions. The purpose of this meeting was not to develop consensus on a singular answer but to discuss and ensure that all questions are being considered in the same way (for example, clarifying any potentially confusing language, discussing any questions that emerged as part of the process). The experts then revisited their initial rankings to provide final, anonymous rankings which were compiled in the same way. These final rankings are the results presented here and may be the same or different from the initial rankings, which were discarded.

Survey questions

The expert elicitation survey comprised five questions for each pathway. The data were collected via Google Forms and collated anonymously at the level of pathways, with each respondent contributing one datapoint for each pathway. The experts reported their familiarity (or the familiarity of the organization whose work they were representing) with the pathway and other cobenefits for the pathways.

The initial question ranked the NbCS pathway by category, from one to three.

Category 1 was defined as a pathway with sufficient scientific knowledge to support a high-quality carbon accounting system today (for example, meets the scientific criteria identified in the WWF-EDF-Oeko Institut and ICAO TAB) or to support the development of such a system today. The intended interpretation is that sufficient science is available for quantifying and verifying net GHG mitigation. Note that experts were not required to reference any given ‘high-quality’ crediting framework, which were provided only as examples. In other words, the evaluation was not intended to rank a given framework (for example, ref. 35 ) but rather expert confidence in the fundamental scientific understandings that underpin potential for carbon accounting overall. To this end, no categorization of uncertainty was required (reviewers could skip categorizations they felt were not necessary) and space was available to fill in new categories by individual reviewers (if they felt a category was missing or needed). Uncertainties at this category 1 level are deemed ‘acceptable’, for example, not precluding accounting now, although more research may further substantiate high-quality credits.

Category 2 pathways have a good chance (>25%) that with more research and within the next 5 years, the pathway could be developed into a high-quality pathway for carbon accounting and as a nature-based climate solution pathway. For these pathways, further understanding is needed for factors such as baseline processes, long-term stability, unconstrained fluxes, possible leakage or other before labelling as category 1 but the expert is confident that information can be developed, in 5 years or less, with more work. The >25% chance threshold and 5-year timeframe were determined a priori to reflect and identify pathways that experts identified as having the potential to meet the Paris Accord 2030 goal. Other thresholds (for example, longer timeframes) could have been chosen, which would impact the relative distribution of pathways in categories 2 and 3 (for example, a longer timeframe allowed could move some pathways from category 3 into category 2, for some reviewers). We emphasize that category 3 pathways do not necessarily mean non-valuable approaches but longer timeframes required for research than the one set here.

Category 3 responses denoted pathways that the expert thought had little chance (<25%) that with more research and within the next 5 years, this pathway could be developed into a suitable pathway for managing as a natural solutions pathway, either because present evidence already suggests GHG reduction is not likely to be viable, co-emissions or other biophysical feedbacks may offset those gains or because understanding of key factors is lacking and unlikely to be developed within the next 5 years. Notably, the last does not mean that the NbCS pathway is not valid or viable in the long-term, simply that physical and biological understandings are probably not established enough to enable scientific rigorous and valid NbCS activity in the near term.

The second question asked the experts to identify research gaps associated with those that they ranked as category 2 pathways to determine focal areas for further research. The experts were asked to rank concerns about durability (ability to predict or compensate for uncertainty in timescale of effectiveness due to disturbances, climate change, human activity or other factors), geographic uncertainty (place-to-place variation), leakage or displacement (spillover of activities to other areas), measuring, reporting and verification (MRV, referring to the ability to quantify all salient stocks and fluxes to fully assess climate impacts), basic mechanisms of action (fundamental science), scaling potential (ability to estimate potential growth) and setting of a baseline (ability to reasonably quantify additionality over non-action, a counterfactual). Respondents could also enter a different category if desired. For complete definitions of these categories, see the survey instrument ( Supplementary Information ). This question was not asked if the expert ranked the pathway as category 1, as those were deemed acceptable, or for category 3, respecting the substantial uncertainty in that rating. Note that responses were individual and so the same NbCS pathway could receive (for example) several individual category 1 rankings, which would indicate reasonable confidence from those experts, and several category 2 rankings from others, which would indicate that those reviewers have lingering concerns about the scientific basis, along with their rankings of the remaining key uncertainties in those pathways. These are important considerations, as they reflect the diversity of opinions and research priorities; individual responses are publicly available (anonymized: https://doi.org/10.5281/zenodo.7859146 ).

The third question involved quantification of the potential for moving from category 2 to 1 explicitly. Following ref. 14 , the respondents first reported the lowest plausible value for the potential likelihood of movement (representing the lower end of a 95% confidence interval), then the upper likelihood and then their best guess for the median/most likely probability. They were also asked for the odds that their chosen interval contained the true value, which was used to scale responses to standard 80% credible intervals and limit overconfidence bias 13 , 15 . This question was not asked if the expert ranked the pathway as category 3, respecting the substantial uncertainty in that rating.

The fourth question involved the scale of potential impact from the NbCS, given the range of uncertainties associated with effectiveness, area of applicability and other factors. The question followed the same pattern as the third, first asking about lowest, then highest, then best estimate for potential scale of impact (in PgCO 2 e yr −1 ). Experts were again asked to express their confidence in their own range, which was used to scale to a standard 80% credible interval. This estimate represents a consolidation of the best-available science by the reviewers. For a complete review including individual studies and their respective findings, see the Supplementary Data . This question was not asked if the expert ranked the pathway as category 3, respecting the substantial uncertainty in that rating.

Final results

After collection of the final survey responses, results were anonymized and compiled by pathway. For overall visualization and discussion purposes, responses were combined into a mean and 20th to 80th percentile range. The strength of the expert elicitation process lies in the collection of several independent assessments. Those different responses represent real differences in data interpretation and synthesis ascribed by experts. This can have meaningful impacts on decision-making by different individuals and organizations (for example, those that are more optimistic or pessimistic about any given pathway). Therefore, individual anonymous responses were retained by pathway to show the diversity of responses for any given pathway. The experts surveyed, despite their broad range of expertise, ranked themselves as less familiar with category 3 pathways than category 1 or 2 (linear regression, P  < 0.001, F  = 59.6 2, 394 ); this could be because of a lack of appropriate experts—although they represented all principal fields—or simply because the data are limited in those areas.

Sensitivity

To check for robustness against sample size variation, we conducted a Monte Carlo sensitivity analysis of the data on each pathway to generate responses of a further ten hypothetical experts. Briefly, the extra samples were randomly drawn from the observed category ranking mean and standard deviations for each individual pathway and appended to the original list; values <1 or >3 were truncated to those values. This analysis resulted in only minor differences in the mean categorization across all pathways: the mean difference between the original and the boot-strapped data was 0.02 (s.d. = 0.05) with an absolute difference average of 0.06 (s.d. = 0.06). The maximum difference in means across all pathways was 0.20 (s.d. = 0.20) (Supplementary Table 2 ). The results suggest that the response values are stable to additional responses.

All processing was done in R 36 , with packages including fmsb 37 and forcats 38 .

Data availability

Anonymized expert elicitation responses are available on Zenodo 39 : https://doi.org/10.5281/zenodo.7859146 .

Code availability

R code for analysis available on Zenodo 39 : https://doi.org/10.5281/zenodo.7859146 .

Novick, K. A. et al. Informing nature‐based climate solutions for the United States with the best‐available science. Glob. Change Biol. 28 , 3778–3794 (2022).

Article   Google Scholar  

Cohen-Shacham, E., Walters, G., Janzen, C. & Maginnis, S. (eds) Nature-based Solutions to Address Global Societal Challenges (IUCN, 2016).

IPCC Climate Change 2021: The Physical Science Basis (eds Masson-Delmotte, V. et al.) (Cambridge Univ. Press, 2021).

Seddon, N. et al. Understanding the value and limits of nature-based solutions to climate change and other global challenges. Philos. Trans. R. Soc. B 375 , 20190120 (2020).

Griscom, B. W. et al. Natural climate solutions. Proc. Natl Acad. Sci. USA 114 , 11645–11650 (2017).

Article   CAS   PubMed   PubMed Central   ADS   Google Scholar  

Blaufelder, C., Levy, C., Mannion, P. & Pinner, D. A. Blueprint for Scaling Voluntary Carbon Markets to Meet the Climate Challenge (McKinsey & Company, 2021).

Arcusa, S. & Sprenkle-Hyppolite, S. Snapshot of the carbon dioxide removal certification and standards ecosystem (2021–2022). Clim. Policy 22 , 1319–1332 (2022).

Seddon, N. et al. Global recognition of the importance of nature-based solutions to the impacts of climate change Glob. Sustain. 3 , pe15 (2020).

Anderegg, W. R. Gambling with the climate: how risky of a bet are natural climate solutions? AGU Adv. 2 , e2021AV000490 (2021).

Article   ADS   Google Scholar  

Gattuso, J. P. et al. Ocean solutions to address climate change and its effects on marine ecosystems. Front. Mar. Sci. 5 , p337 (2018).

Bach, L. T., Gill, S. J., Rickaby, R. E., Gore, S. & Renforth, P. CO 2 removal with enhanced weathering and ocean alkalinity enhancement: potential risks and co-benefits for marine pelagic ecosystems. Front. Clim. 1 , 7 (2019).

Doelman, J. C. et al. Afforestation for climate change mitigation: potentials, risks and trade‐offs. Glob. Change Biol. 26 , 1576–1591 (2019).

Speirs-Bridge, A. et al. Reducing overconfidence in the interval judgments of experts. Risk Anal. 30 , 512–523 (2010).

Article   PubMed   Google Scholar  

Morgan, M. G. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl Acad. Sci. USA 111 , 7176–7184 (2014).

Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F. & Wintle, B. C. A practical guide to structured expert elicitation using the IDEA protocol. Methods Ecol. Evol. 9 , 169–180 (2018).

Anderegg, W. R. et al. Climate-driven risks to the climate mitigation potential of forests. Science 368 , eaaz7005 (2020).

Article   CAS   PubMed   Google Scholar  

Boyd, P. W., Claustre, H., Levy, M., Siegel, D. A. & Weber, T. Multi-faceted particle pumps drive carbon sequestration in the ocean. Nature 568 , 327–335 (2019).

Article   CAS   PubMed   ADS   Google Scholar  

Saba, G. K. et al. Toward a better understanding of fish-based contribution to ocean carbon flux. Limnol. Oceanogr. 66 , 1639–1664 (2021).

Article   CAS   ADS   Google Scholar  

Seddon, N., Turner, B., Berry, P., Chausson, A. & Girardin, C. A. Grounding nature-based climate solutions in sound biodiversity science. Nat. Clim. Change 9 , 84–87 (2019).

Soto-Navarro, C. et al. Mapping co-benefits for carbon storage and biodiversity to inform conservation policy and action. Philos. Trans. R. Soc. B 375 , 20190128 (2020).

Article   CAS   Google Scholar  

Schulte, I., Eggers, J., Nielsen, J. Ø. & Fuss, S. What influences the implementation of natural climate solutions? A systematic map and review of the evidence. Environ. Res. Lett. 17 , p013002 (2022).

West, T. A., Börner, J., Sills, E. O. & Kontoleon, A. Overstated carbon emission reductions from voluntary REDD+ projects in the Brazilian Amazon. Proc. Natl Acad. Sci. USA 117 , 24188–24194 (2020).

Di Sacco, A. et al. Ten golden rules for reforestation to optimize carbon sequestration, biodiversity recovery and livelihood benefits. Glob. Change Biol. 27 , 1328–1348 (2021).

López-Vallejo, M. in Towards an Emissions Trading System in Mexico: Rationale, Design and Connections with the Global Climate Agenda (ed. Lucatello, S.) 191–221 (Springer, 2022)

Oldfield, E. E. et al. Realizing the potential of agricultural soil carbon sequestration requires more effective accounting. Science 375 , 1222–1225 (2022).

Burkholz, C., Garcias-Bonet, N. & Duarte, C. M. Warming enhances carbon dioxide and methane fluxes from Red Sea seagrass ( Halophila stipulacea ) sediments. Biogeosciences 17 , 1717–1730 (2020).

Guenet, B. et al. Can N 2 O emissions offset the benefits from soil organic carbon storage? Glob. Change Biol. 27 , 237–256 (2021).

Rosentreter, J. A., Al‐Haj, A. N., Fulweiler, R. W. & Williamson, P. Methane and nitrous oxide emissions complicate coastal blue carbon assessments. Glob. Biogeochem. Cycles 35 , pe2020GB006858 (2021).

Schwartzman, S. et al. Environmental integrity of emissions reductions depends on scale and systemic changes, not sector of origin. Environ. Res. Lett. 16 , p091001 (2021).

Crop and Livestock Products Database (FAO, 2022); https://www.fao.org/faostat/en/#data/QCL

Fargione, J. E. et al. Natural climate solutions for the United States. Sci. Adv. 4 , eaat1869 (2018).

Article   PubMed   PubMed Central   ADS   Google Scholar  

Meyer, S. E. Is climate change mitigation the best use of desert shrublands? Nat. Resour. Environ. Issues 17 , 2 (2011).

Google Scholar  

Lorenz, K. & Lal, R. Carbon Sequestration in Agricultural Ecosystems (Springer Cham, 2018).

Haya, B., So, I. & Elias, M. The Voluntary Registry Offsets Database (Univ. California, 2021); https://gspp.berkeley.edu/faculty-and-impact/centers/cepp/projects/berkeley-carbon-trading-project/offsets-database

Core Carbon Principles; CCP Attributes; Assessment Framework for Programs; and Assessment Procedure (ICVCM, 2023); https://icvcm.org/the-core-carbon-principles/

R Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, 2022).

Nakazawa, M. fmsb: Functions for medical statistics book with some demographic data. R package version 0.7.4 https://CRAN.R-project.org/package=fmsb (2022).

Wickham, H. forcats: Tools for working with categorical variables (factors). R package version 0.5.2 https://CRAN.R-project.org/package=forcats (2022)

Buma, B. Nature-based climate solutions: expert elicitation data and analysis code. Zenodo https://doi.org/10.5281/zenodo.7859146 (2023).

Download references

Acknowledgements

This research was supported through gifts to the Environmental Defense Fund from the Bezos Earth Fund, King Philanthropies and Arcadia, a charitable fund of L. Rausing and P. Baldwin. We thank J. Rudek for help assembling the review and 30 experts who reviewed some or all of those data and protocol summaries (Supplementary Data ). S.M. was supported by a cooperative agreement between the National Science Foundation and Battelle that sponsors the National Ecological Observatory Network programme.

Author information

Present address: Department of Atmospheric and Oceanic Sciences, University of Wisconsin-Madison, Madison, WI, USA

Present address: AtmoFacts, Longmont, CO, USA

R. N. Lubowski

Present address: Lombard Odier Investment Managers, New York, NY, USA

Present address: Ecological Carbon Offset Partners LLC, dba EP Carbon, Minneapolis, MN, USA

L. A. Moore

Present address: , San Francisco, CA, USA

J. Paltseva

Present address: ART, Arlington, VA, USA

N. A. Randazzo

Present address: NASA/GSFC, Greenbelt, MD, USA

Present address: University of Maryland, College Park, MD, USA

N. Uludere Aragon

Present address: Numerical Terradynamic Simulation Group, University of Montana, Missoula, MT, USA

These authors contributed equally: B. Buma, D. R. Gordon.

Authors and Affiliations

Environmental Defense Fund, New York, NY, USA

B. Buma, D. R. Gordon, K. M. Kleisner, A. Bartuska, J. R. Collins, A. J. Eagle, R. Fujita, E. Holst, J. M. Lavallee, R. N. Lubowski, C. Melikov, L. A. Moore, E. E. Oldfield, J. Paltseva, A. M. Raffeld, N. A. Randazzo, C. Schneider, N. Uludere Aragon & S. P. Hamburg

Department of Integrative Biology, University of Colorado, Denver, CO, USA

Department of Biology, University of Florida, Gainesville, FL, USA

D. R. Gordon

Resources for the Future, Washington, DC, USA

A. Bartuska

International Arctic Research Center, University of Alaska, Fairbanks, AK, USA

Department of Ecology Evolution and Environmental Biology and the Climate School, Columbia University, New York, NY, USA

The Nature Conservancy, Arlington, VA, USA

Faculty of Environment, Science and Economy, University of Exeter, Exeter, UK

P. Friedlingstein

Laboratoire de Météorologie Dynamique/Institut Pierre-Simon Laplace, CNRS, Ecole Normale Supérieure/Université PSL, Sorbonne Université, Ecole Polytechnique, Palaiseau, France

National Ecological Observatory Network, Battelle, Boulder, CO, USA

Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA

O’Neill School of Public and Environmental Affairs, Indiana University, Bloomington, IN, USA

Department of Environmental Science and Policy, University of California, Davis, CA, USA

J. N. Sanchirico

Department of Marine Chemistry & Geochemistry, Woods Hole Oceanographic Institution, Woods Hole, MA, USA

J. R. Collins

You can also search for this author in PubMed   Google Scholar

Contributions

D.R.G. and B.B. conceived of and executed the study design. D.R.G., K.M.K., J.R.C., A.J.E., R.F., E.H., J.M.L., R.N.L., C.M., L.A.M., E.E.O., J.P., A.M.R., N.A.R., C.S. and N.U.A. coordinated and conducted the literature review. G.M. and B.B. primarily designed the survey. A. Bartuska, A. Bidlack, B.B., J.N.S., K.N., P.E., P.F., R.D. and S.M. contributed to the elicitation. B.B. conducted the analysis and coding. S.P.H. coordinated funding. B.B. and D.R.G. were primary writers; all authors were invited to contribute to the initial drafting.

Corresponding author

Correspondence to B. Buma .

Ethics declarations

Competing interests.

The authors declare no competing interests. In the interest of full transparency, we note that while B.B., D.R.G., K.M.K., A.B., J.R.C., A.J.E., R.F., E.H., J.M.L., R.N.L., C.M., L.A.M., E.E.O., J.P., A.M.R., N.A.R., C.S., N.U.A., S.P.H. and P.E. are employed by organizations that have taken positions on specific NbCS frameworks or carbon crediting pathways (not the focus of this work), none have financial or other competing interest in any of the pathways and all relied on independent science in their contributions to the work.

Peer review

Peer review information.

Nature Climate Change thanks Camila Donatti, Connor Nolan and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Tables 1–4, Figs. 1–3 and survey instrument.

Supplementary Data

Literature review and list of reviewers.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Buma, B., Gordon, D.R., Kleisner, K.M. et al. Expert review of the science underlying nature-based climate solutions. Nat. Clim. Chang. (2024). https://doi.org/10.1038/s41558-024-01960-0

Download citation

Received : 24 April 2023

Accepted : 20 February 2024

Published : 21 March 2024

DOI : https://doi.org/10.1038/s41558-024-01960-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

review article research paper

Disclaimer » Advertising

  • HealthyChildren.org

Selection Criteria

Search strategy, data extraction, risk of bias and applicability, data synthesis and analysis, parent ratings, teacher ratings, youth self-reports, combined rating scales, additional clinician tools, neuropsychological tests, biospecimen, neuroimaging, variation in diagnostic accuracy with clinical setting or patient subgroup, measures for diagnostic performance, available tools, importance of the comparator sample, clinical implications, future research, conclusions, acknowledgments, tools for the diagnosis of adhd in children and adolescents: a systematic review.

FUNDING: The work is based on research conducted by the Southern California Evidence-based Practice Center under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract 75Q80120D00009). The Patient-Centered Outcomes Research Institute (PCORI) funded the research (PCORI Publication No. 2023-SR-03). The findings and conclusions in this manuscript are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ or PCORI, its Board of Governors, or Methodology Committee. Therefore, no statement in this report should be construed as an official position of PCORI, AHRQ or of the US Department of Health and Human Services.

CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no conflicts of interest to disclose.

  • Split-Screen
  • Article contents
  • Figures & tables
  • Supplementary Data
  • Peer Review
  • CME Quiz Close Quiz
  • Open the PDF for in another window
  • Get Permissions
  • Cite Icon Cite
  • Search Site

Bradley S. Peterson , Joey Trampush , Morah Brown , Margaret Maglione , Maria Bolshakova , Mary Rozelle , Jeremy Miles , Sheila Pakdaman , Sachi Yagyu , Aneesa Motala , Susanne Hempel; Tools for the Diagnosis of ADHD in Children and Adolescents: A Systematic Review. Pediatrics 2024; e2024065854. 10.1542/peds.2024-065854

Download citation file:

  • Ris (Zotero)
  • Reference Manager

Correct diagnosis is essential for the appropriate clinical management of attention-deficit/hyperactivity disorder (ADHD) in children and adolescents.

This systematic review provides an overview of the available diagnostic tools.

We identified diagnostic accuracy studies in 12 databases published from 1980 through June 2023.

Any ADHD tool evaluation for the diagnosis of ADHD, requiring a reference standard of a clinical diagnosis by a mental health specialist.

Data were abstracted and critically appraised by 1 reviewer and checked by a methodologist. Strength of evidence and applicability assessments followed Evidence-based Practice Center standards.

In total, 231 studies met eligibility criteria. Studies evaluated parental ratings, teacher ratings, youth self-reports, clinician tools, neuropsychological tests, biospecimen, EEG, and neuroimaging. Multiple tools showed promising diagnostic performance, but estimates varied considerably across studies, with a generally low strength of evidence. Performance depended on whether ADHD youth were being differentiated from neurotypically developing children or from clinically referred children.

Studies used different components of available tools and did not report sufficient data for meta-analytic models.

A valid and reliable diagnosis of ADHD requires the judgment of a clinician who is experienced in the evaluation of youth with and without ADHD, along with the aid of standardized rating scales and input from multiple informants across multiple settings, including parents, teachers, and youth themselves.

Attention-deficit/hyperactivity disorder (ADHD) is one of the most prevalent neurodevelopmental conditions in youth. Its prevalence has remained constant at ∼5.3% worldwide over the years, and diagnostic criteria have remained constant when based on rigorous diagnostic procedures. 1 Clinical diagnoses, however, have increased steadily over time, 2 and currently, ∼10% of US children receive an ADHD diagnosis. 3 Higher rates of clinical compared with research-based diagnoses are because of an increasing clinician recognition of youth who have ADHD symptoms that are functionally impairing but do not fully meet formal diagnostic criteria. 4 The higher diagnostic rates over time in clinical samples also results from youth receiving a diagnosis incorrectly. Some youth, for example, are misdiagnosed as having ADHD when they have symptoms of other disorders that overlap with ADHD symptoms, such as difficulty concentrating, which occurs in many other conditions. 5 Moreover, ADHD is more than twice as likely to be diagnosed in boys than in girls, 3 in lower-income families, 6 and in white compared with nonwhite youth 7 ; differences that derive at least in part from diagnostic and cultural biases. 8 , – 11  

Improving clinical diagnostic accuracy is essential to ensure that youth who truly have ADHD benefit from receiving treatment without delay. Similarly, youth who do not have ADHD should not be diagnosed since an incorrect diagnosis risks exposing them to unbeneficial treatments. 12 , 13 Clinician judgement alone, however, especially by nonspecialist clinicians, is poor in diagnosing ADHD 14 compared with expert, research-grade diagnoses made by mental health clinicians. 15 Accurately diagnosing ADHD is difficult because diagnoses are often made using subjective clinical impressions, and putative diagnostic tools have a confusing, diverse, and poorly described evidence base that is not widely accessible. The availability of valid diagnostic tools would especially help to reduce misdiagnoses from cultural biases and symptom overlap with ADHD. 12 , 16 , – 19  

This review summarizes evidence for the performance of tools for children and adolescents with ADHD. We did not restrict to a set of known diagnostic tools but instead explored the range of available diagnostic tools, including machine-learning assisted and virtual reality-based tools. The review aimed to assess how diagnostic performance varies by clinical setting and patient characteristics.

The review aims were developed in consultation with the Agency for Healthcare Research and Quality (AHRQ), the Patient-Centered Outcomes Research Institute, the topic nominator American Academy of Pediatrics, key informants, a technical expert panel (TEP), and public input. The TEP reviewed the protocol and advised on key outcomes. Subgroup analyses and key outcomes were prespecified. The review is registered in PROSPERO (CRD42022312656) and the protocol is available on the AHRQ Web site as part of a larger evidence report on ADHD. The systematic review followed Methods of the (AHRQ) Evidence-based Practice Center Program. 20  

Population: age <18 years.

Interventions: any ADHD tool for the diagnosis of ADHD.

Comparators: diagnosis by a mental health specialist, such as a psychologist, psychiatrist, or other provider, who often used published scales or semistructured diagnostic interviews to ensure a reliable DSM-based diagnosis of ADHD.

Key outcomes: diagnostic accuracy (eg, sensitivity, specificity, area under the curve).

Setting: any.

Study design: diagnostic accuracy studies.

Other: English language, published from 1980 to June 2023.

We searched PubMed, Embase, PsycINFO, ERIC, and ClinicalTrials.gov. We identified reviews for reference-mining through PubMed, Cochrane Database of Systematic Reviews, Campbell Collaboration, What Works in Education, PROSPERO, ECRI Guidelines Trust, G-I-N, and ClinicalKey. The peer reviewed strategy is in the Supplemental Appendix . All citations were screened by trained literature reviewers supported by machine learning ( Fig 1 ). Two independent reviewers assessed full text studies for eligibility. The TEP reviewed studies to ensure all were captured. Publications reporting on the same participants were consolidated into 1 record.

Literature flow diagram.

Literature flow diagram.

The data abstraction form included extensive guidance to aid reproducibility and standardization in recording study details, results, risk of bias, and applicability. One reviewer abstracted data and a methodologist checked accuracy and completeness. Data are publicly available in the Systematic Review Data Repository.

We assessed characteristics pertaining to patient selection, index test, reference standard, flow and timing that may have introduced bias, and evaluated applicability of study results, such as whether the test, its conduct, or interpretation differed from how the test is used in clinical practice. 21 , 22  

We differentiated parent, teacher, and youth self-report ratings; tools for clinicians; neuropsychological tests; biospecimens; EEG; and neuroimaging. We organized analyses according to prespecified outcome measures. A narrative overview summarized the range of diagnostic performance for key outcomes. Because lack of reported detail in many individual studies hindered use of meta-analytic models, we created summary figures to document the diagnostic performance reported in each study. We used meta-regressions across studies to assess the effects of age, comorbidities, racial and ethnic composition, and diagnostic setting (differentiating primary care, specialty care, school settings, mixed settings, and not reported) on diagnostic performance. One researcher with experience in use of specified standardized criteria 23 initially assessed the overall strength of evidence (SoE) (see Supplemental Appendix ) for each study, then discussed it with the study team to communicate our confidence in each finding.

We screened 23 139 citations and 7534 publications retrieved as full text against the eligibility criteria. In total, 231 studies reported in 290 publications met the eligibility criteria (see Fig 1 ).

Methodological quality of the studies varied. Selection bias was likely in two-thirds of studies; several were determined to be problematic in terms of reported study flow and timing of assessments (eg, not stating whether diagnosis was known before the results of the index test); and several lacked details on diagnosticians or diagnostic procedures ( Supplemental Fig 1 ). Applicability concerns limited the generalizability of findings ( Supplemental Fig 2 ), usually because youth with comorbidities were excluded. Many different tools were assessed within the broader categories (eg, within neuropsychological tests), and even when reporting on the same diagnostic tool, studies often used different components of the tool (eg, different subscales of rating scales), or they combined components in a variety of ways (eg, across different neuropsychological test parameters).

The evidence table ( Supplemental Table 10 , Supplemental Appendix ) shows each study’s finding. The following highlights key findings across studies.

Fifty-nine studies used parent ratings to diagnose ADHD ( Fig 2 ). The most frequently evaluated tool was the CBCL (Child Behavior Checklist), alone or in combination with other tools, often using different score cutoffs for diagnosis, and evaluating different subscales (most frequently the attention deficit/hyperactivity problems subscale). Sensitivities ranged from 38% (corresponding specificity = 96%) to 100% (specificity = 4% to 92%). 24 , 25  

Diagnostic performance parent and teacher ratings. For a complete list of scales see Supplemental Appendix.

Diagnostic performance parent and teacher ratings. For a complete list of scales see Supplemental Appendix .

Area under the curve (AUC) for receiver operator characteristic curves ranged widely from 0.55 to 0.95 but 3 CBCL studies reported AUCs of 0.83 to 0.84. 26 , – 28 Few studies reported measurement of reliability. SoE was downgraded for study limitation (lack of detailed reporting), imprecision (large performance variability), and inconsistent findings ( Supplemental Table 1 ).

Twenty-three studies used teacher ratings to diagnose ADHD ( Fig 2 ). No 2 studies reported on rater agreement, internal consistency, or test-retest reliability for the same teacher rating scale. The highest sensitivity was 97% (specificity = 26%). 25 The Teacher Report Form, alone or in combination with Conners teacher rating scales, yielded sensitivities of 72% to 79% 29 and specificities of 64% to 76%. 30 , 32 reported AUCs ranged from 0.65 to 0.84. 32 SoE was downgraded to low for imprecision (large performance variability) and inconsistency (results for specific tools not replicated), see Supplemental Table 2 .

Six studies used youth self-reports to diagnose ADHD. No 2 studies used the same instrument. Sensitivities ranged from 53% (specificity = 98%) to 86% (specificity = 70%). 35 AUCs ranged from 0.56 to 0.85. 36 We downgraded SoE for domain inconsistency (only 1 study reported on a given tool and outcome), see Supplemental Table 3 .

Thirteen studies assessed diagnostic performance of ratings combined across informants, often using machine learning for variable selection. Only 1 study compared performance of combined data to performance from single informants, finding negligible improvement (AUC youth = 0.71; parent = 0.85; combined = 0.86). 37 Other studies reported on limited outcome measures and used ad hoc methods to combine information from multiple informants. The best AUC was reported by a machine learning supported study combining parent and teacher ratings (AUC = 0.98). 38  

Twenty-four studies assessed additional tools, such as interview guides, that can be used by clinicians to aid diagnosis of ADHD. Sensitivities varied, ranging from 67% (specificity = 65%) to 98% (specificity = 100%); specificities ranged from 36% (sensitivity = 89%) to 100% (sensitivity = 98%). 39 Some of the tools measured activity levels objectively using an actometer or commercially available activity tracker, either alone or as part of a diagnostic test battery. Reported performance was variable (sensitivity range 25% to 100%, 40 specificity range 66% to 100%, 40 AUCs range 0.75–0.9996 41 ). SoE was downgraded for imprecision (large performance variability) and inconsistency (outcomes and results not replicated), see Supplemental Table 4 .

Seventy-four studies used measures from various neuropsychological tests, including continuous performance tests (CPTs). Four of these included 3- and 4-year-old children. 42 , – 44 A large majority used a CPT, which assessed omission errors (reflecting inattention), commission errors (impulsivity), and reaction time SD (response time variability). Studies varied in use of traditional visual CPTs, such as the Test of Variables of Attention, more novel, multifaceted “hybrid” CPT paradigms, and virtual reality CPTs built upon environments designed to emulate real-world classroom distractibility. Studies used idiosyncratic combinations of individual cognitive measures to achieve the best performance, though many reported on CPT attention and impulsivity measures.

Sensitivity for all neuropsychological tests ranged from 22% (specificity = 96%) to 100% (specificity = 100%) 45 ( Fig 3 ), though the latter study reported performance for unique composite measures without replication. Specificities ranged from 22% (sensitivity = 91%) 46 to 100% (sensitivity = 100% to 75%). 45 , 47 AUCs ranged from 0.59 to 0.93. 48 Sensitivity for all CPT studies ranged from 22% ( specificity = 96) to 100% (specificity = 75%). 49 Specificities for CPTs ranged from 22% (sensitivity = 91%) to 100% (sensitivity = 89%) 47 ( Fig 3 ). AUCs ranged from 0.59 to 0.93. 50 , 51 SoE was deemed low for imprecise studies (large performance variability), see Supplemental Table 5.

Diagnostic performance neuropsychological tests, CPTs, activity monitors, biospecimen, EEG.

Diagnostic performance neuropsychological tests, CPTs, activity monitors, biospecimen, EEG.

Seven studies assessed blood or urine biomarkers to diagnose ADHD. These measured erythropoietin or erythropoietin receptor, membrane potential ratio, micro RNA levels, or urine metabolites. Sensitivities ranged from 56% (specificity = 95%) to 100% (specificity = 100% for erythropoietin and erythropoietin receptors levels). 52 Specificities ranged from 25% (sensitivity = 79%) to 100% (sensitivity = 100%). 52 AUCs ranged from 0.68 to 1.00. 52 Little information was provided on reliability of markers or their combinations. SoE was downgraded for inconsistent and imprecise studies ( Supplemental Table 6 ).

Forty-five studies used EEG markers to diagnose ADHD. EEG signals were obtained in a variety of patient states, even during neuropsychological test performance. Two-thirds used machine learning algorithms to select classification parameters. Several combined EEG with demographic variables or rating scales. Sensitivity ranged widely from 46% to 100% (corresponding specificities 74 and 71%). 53 , 54 One study that combined EEG with demographics data supported by machine learning reported perfect sensitivity and specificity. 54 Specificity was also variable and ranged from 38% (sensitivity = 95%) to 100% (specificities = 71% or 100%). 53 , – 56 Reported AUCs ranged from 0.63 to 1.0. 57 , 58 SoE was downgraded for study imprecision (large performance variability) and limitations (diagnostic approaches poorly described), see Supplemental Table 7 .

Nineteen studies used neuroimaging for diagnosis. One public data set (ADHD-200) produced several analyses. All but 2 used MRI: some functional MRI (fMRI), some structural, and some in combination, with or without magnetic resonance spectroscopy (2 used near-infrared spectroscopy). Most employed machine learning to detect markers that optimized diagnostic classifications. Some combined imaging measures with demographic or other clinical data in the prediction model. Sensitivities ranged from 42% (specificity = 95%) to 99% (specificity = 100%) using resting state fMRI and a complex machine learning algorithm 56 to differentiate ADHD from neurotypical youth. Specificities ranged from 55% (sensitivity = 95%) to 100% 56 using resting state fMRI data. AUCs ranged from 0.58 to over 0.99, 57 SoE was downgraded for imprecision (large performance variability) and study limitations (diagnostic models are often not well described, and the number and type of predictor variables entering the model were unclear). Studies generally did not validate diagnostic algorithms or assess performance measures in an independent sample ( Supplemental Table 8 ).

Regression analyses indicated that setting was associated with both sensitivity ( P = .03) and accuracy ( P = .006) but not specificity ( P = .68) or AUC ( P = .28), with sensitivities lowest in primary care ( Fig 4 ). Sensitivity, specificity, and accuracy were also lower when differentiating youth with ADHD from a clinical sample than from typically developing youth (sensitivity P = .04, specificity P < .001, AUC P < .001) ( Fig 4 ), suggesting that clinical population is a source of heterogeneity in diagnostic performance. Findings should be interpreted with caution, however, as they were not obtained in meta-analytic models and, consequently, do not take into account study size or quality.

Diagnostic performance by setting and population.

Diagnostic performance by setting and population.

Supplemental Figs 3–5 in the Supplemental Appendix document effects by age and gender. We did not detect statistically significant associations of age with sensitivity ( P = .54) or specificity ( P = .37), or associations of the proportion of girls with sensitivity ( P = .63), specificity ( P = .80), accuracy ( P = .34), or AUC ( P = .90).

We identified a large number of publications reporting on ADHD diagnostic tools. To our knowledge, no prior review of ADHD diagnostic tools has been as comprehensive in the range of tools, outcomes, participant ages, and publication years. Despite the large number of studies, we deemed the strength of evidence for the reported performance measures across all categories of diagnostic tools to be low because of large performance variability across studies and various limitations within and across studies.

We required that studies report diagnoses when using the tool compared with diagnoses made by expert mental health clinicians. Studies most commonly reported sensitivity (true-positive rate) and specificity (true-negative rate) when a study-specific diagnostic threshold was applied to measures from the tool being assessed. Sensitivity and specificity depend critically on that study-specific threshold, and their values are inherently a trade-off, such that varying the threshold to increase either sensitivity or specificity reduces the other. Interpreting diagnostic performance in terms of sensitivity and specificity, and comparing those performance measures across studies, is therefore challenging. Consequently, researchers more recently often report performance for sensitivity and specificity in terms of receiver operating characteristics (ROC) curves, a plot of sensitivity versus specificity across the entire range of possible diagnostic thresholds. The area under this ROC curve (AUC) provides an overall, single index of performance that ranges from 0.5 (indicating that the tool provides no information above chance for classification) to 1.0 (indicating a perfect test that can correctly classify all participants as having ADHD and all non-ADHD participants as not having it). AUC values of 90 to 100 are commonly classified as excellent performance; 80 to 90 as good; 70 to 80 as fair; 60 to 70 as poor; and 50 to 60 failed performance.

Most research is available on parental ratings. Overall, AUCs for parent rating scales ranged widely from “poor” 58 to “excellent.” 59 Analyses restricted to the CBCL, the most commonly evaluated scale, yielded more consistent “good” AUCs for differentiating youth with ADHD from others in clinical samples, but the number of studies contributing data were small. Internal consistency for rating scale items was generally high across most rating scales. Test-retest reliability was good, though only 2 studies reported it. One study reported moderate rater agreement between mothers and fathers for inattention, hyperactivity, and impulsivity symptoms. Few studies included youth under 7 years of age.

AUCs for teacher rating scales ranged from “failed” 33 to “good.” 34 Internal consistency for scale items was generally high. Teacher ratings demonstrated very low rater agreement with corresponding parent scales, suggesting either a problem with the instruments or a large variability in symptom presentation with environmental context (home or school).

Though data were limited, self-reports from youth seemed to perform less well than corresponding parent and teacher reports, with AUCs ranging from “failed” for CBCL or ASEBA when distinguishing ADHD from other patients 33 to “good” for the SWAN in distinguishing ADHD from neurotypical controls. 36 , 37  

Studies evaluating neuropsychological tests yielded AUCs ranging from “poor” 60 , 61 to “excellent.” 50 Many used idiosyncratic combinations of cognitive measures, which complicates interpretation of the results across studies. Nevertheless, extracting specific, comparable measures of inattention and impulsivity from CPTs yielded diagnostic performance ranging from “poor” to “excellent” in differentiating ADHD youth from neurotypical controls and “fair” in differentiating ADHD youth from other patients. 42 , 60 , 62 No studies provided an independent replication of diagnosis using the same measure.

Blood biomarkers yielded AUCs ranging from “poor” (serum miRNAs) 63 to “excellent” (erythropoietin and erythropoietin receptors levels) 52 in differentiating ADHD from neurotypical youth. None have been independently replicated, and test-retest reliability was not reported. Most EEG studies used machine learning for diagnostic classification. AUCs ranged from “poor” 64 to “excellent” when differentiating ADHD youth from neurotypical controls. 65 Diagnostic performance was not prospectively replicated in any independent samples.

Most neuroimaging studies relied on machine learning to develop diagnostic algorithms. AUCs ranged from “poor” 66 to “excellent” for distinguishing ADHD youth from neurotypically developing controls. 57 Most studies used pre-existing data sets or repositories to retrospectively discriminate youths with ADHD from neurotypical controls, not from other clinical populations and not prospectively, and none assessed test-retest reliability or the independent reproducibility of findings. Reporting of final mathematical models or algorithms for diagnosis was limited. Activity monitors have the advantage of providing inexpensive, objective, easily obtained, and quantified measures that can potentially be widely disseminated and scaled.

Studies of combined approaches, such as integrating diagnostic tools with clinician impressions, were limited. One study reported increased sensitivity and specificity when an initial clinician diagnosis combined EEG indicators (the reference standard was a consensus diagnosis from a panel of ADHD experts). 67 These findings were not independently replicated, however, and no test-retest reliability was reported.

Many studies aimed to distinguish ADHD youth from neurotypical controls, which is a distinction of limited clinical relevance. In clinically referred youth, most parents, teachers, and clinicians are reasonably confident that something is wrong, even if they are unsure whether the cause of their concern is ADHD. To be informed by a tool that the child is not typically developing is not particularly helpful. Moreover, we cannot know whether diagnostic performance for tools that discriminate ADHD youth only from neurotypical controls is determined by the presence of ADHD or by the presence of any other characteristics that accompany clinical “caseness,” such as the presence of comorbid illnesses or symptoms shared or easily confused with those of other conditions, or the effects of chronic stress or current or past treatment. The clinically more relevant and difficult question is, therefore, how well the tool distinguishes youth with ADHD from those who have other emotional and behavioral problems. Consistent with these conceptual considerations that argue for assessing diagnostic performance in differentiating youth with ADHD from those with other clinical conditions, we found significant evidence that, across all studies, sensitivity, specificity, and AUC were all lower when differentiating youth with ADHD from a clinical sample than when differentiating them from neurotypical youth. These findings also suggest that the comparison population was a significant source of heterogeneity in diagnostic performance.

Despite the large number of studies on diagnostic tools, a valid and reliable diagnosis of ADHD ultimately still requires the judgement of a clinician who is experienced in the evaluation of youth with and without ADHD, along with the aid of standardized rating scales and input from multiple informants across multiple settings, including parents, teachers, and youth themselves. Diagnostic tools perform best when the clinical question is whether a youth has ADHD or is healthy and typically developing, rather than when the clinical question is whether a youth has ADHD or another mental health or behavioral problem. Diagnostic tools yield more false-positive and false-negative diagnoses of ADHD when differentiating youth with ADHD from youth with another mental health problem than when differentiating them from neurotypically developing youth.

Scores for rating scales tended to correlate poorly across raters, and ADHD symptoms in the same child varied across settings, indicating that no single informant in a single setting is a gold-standard for diagnosis. Therefore, diagnosis using rating scales will likely benefit from a more complete representation of symptom expression across multiple informants (parents, school personnel, clinicians, and youth) across more than 1 setting (home, school, and clinic) to inform clinical judgement when making a diagnosis, thus, consistent with current guidelines. 68 , – 70 Unfortunately, methods for combining scores across raters and settings that improve diagnosis compared with scores from single raters have not been developed or prospectively replicated.

Despite the widespread use of neuropsychological testing to “diagnose” youth with ADHD, often at considerable expense, indirect comparisons of AUCs suggest that performance of neuropsychological test measures in diagnosing ADHD is comparable to the diagnostic performance of ADHD rating scales from a single informant. Moreover, the diagnostic accuracy of parent rating scales is typically better than neuropsychological test measures in head-to-head comparisons. 44 , 71 Furthermore, the overall SoE for estimates of diagnostic performance with neuropsychological testing is low. Use of neuropsychological test measures of executive functioning, such as the CPT, may help inform a clinical diagnosis, but they are not definitive either in ruling in or ruling out a diagnosis of ADHD. The sole use of CPTs and other neuropsychological tests to diagnose ADHD, therefore, cannot be recommended. We note that this conclusion regarding diagnostic value is not relevant to any other clinical utility that testing may have.

No independent replication studies have been conducted to validate EEG, neuroimaging, or biospecimen to diagnose ADHD, and no clinical effectiveness studies have been conducted using these tools to diagnose ADHD in the real world. Thus, these tools do not seem remotely close to being ready for clinical application to aid diagnosis, despite US Food and Drug Administration approval of 1 EEG measure as a purported diagnostic aid. 67 , 72  

All studies of diagnostic tools should report data in more detail (ie, clearly report false-positive and -negative rates, the diagnostic thresholds used, and any data manipulation undertaken to achieve the result) to support meta-analytic methods. Studies should include ROC analyses to support comparisons of test performance across studies that are independent of the diagnostic threshold applied to measures from the tool. They should also include assessment of test-retest reliability to help discern whether variability in measures and test performance is a function of setting or of measurement variability over time. Future studies should address the influence of co-occurring disorders on diagnostic performance and how well the tools distinguish youth with ADHD from youth with other emotional and behavioral problems, not simply from healthy controls. More studies should compare the diagnostic accuracy of different test modalities, head-to-head. Independent, prospective replication of performance measures of diagnostic tools in real-world settings is essential before US Food and Drug Administration approval and before recommendations for widespread clinical use.

Research is needed to identify consensus algorithms that combine rating scale data from multiple informants to improve the clinical diagnosis of ADHD, which at present is often unguided, ad hoc, and suboptimal. Diagnostic studies using EEG, neuroimaging, and neuropsychological tests should report precise operational definitions and measurements of the variable(s) used for diagnosis, any diagnostic algorithm employed, the selected statistical cut-offs, and the number of false-positives and false-negatives the diagnostic tool yields to support future efforts at synthetic analyses.

Objective, quantitative neuropsychological test measures of executive functioning correlate only weakly with the clinical symptoms that define ADHD. 73 Thus, many youth with ADHD have normal executive functioning profiles on neuropsychological testing, and many who have impaired executive functioning on testing do not have ADHD. 74 Future research is needed to understand how test measures of executive functioning and the real-world functional problems that define ADHD map on to one another and how that mapping can be improved.

One of the most important potential uses of systematic reviews and meta-analyses in improving the clinical diagnosis of ADHD and treatment planning would be identification of effect modifiers for the performance of diagnostic tools: determining, for example, whether tools perform better in patients who are younger or older, in ethnic minorities, or those experiencing material hardship, or who have a comorbid illness or specific ADHD presentation. Future studies of ADHD should more systematically address the modifier effects of these patient characteristics. They should make available in public repositories the raw, individual-level data and the algorithms or computer code that will aid future efforts at replication, synthesis, and new discovery for diagnostic tools across data sets and studies.

Finally, no studies meeting our inclusion criteria assessed the consequences of being misdiagnosed or labeled as either having or not having ADHD, the diagnosis of ADHD specifically in preschool-aged children, or the potential adverse consequences of youth being incorrectly diagnosed with or without ADHD. This work is urgently needed.

We thank Cynthia Ramirez, Erin Tokutomi, Jennifer Rivera, Coleman Schaefer, Jerusalem Belay, Anne Onyekwuluje, and Mario Gastelum for help with data acquisition. We thank Kymika Okechukwu, Lauren Pilcher, Joanna King, and Robyn Wheatley from the American Academy of Pediatrics (AAP), Jennie Dalton and Paula Eguino Medina from PCORI, Christine Chang and Kim Wittenberg from AHRQ, and Mary Butler from the Minnesota Evidence-based Practice Center. We thank Glendy Burnett, Eugenia Chan, MD, MPH, Matthew J. Gormley, PhD, Laurence Greenhill, MD, Joseph Hagan, Jr, MD, Cecil Reynolds, PhD, Le'Ann Solmonson, PhD, LPC-S, CSC, and Peter Ziemkowski, MD, FAAFP who served as key informants. We thank Angelika Claussen, PhD, Alysa Doyle, PhD, Tiffany Farchione, MD, Matthew J. Gormley, PhD, Laurence Greenhill, MD, Jeffrey M. Halperin, PhD, Marisa Perez-Martin, MS, LMFT, Russell Schachar, MD, Le'Ann Solmonson, PhD, LPC-S, CSC, and James Swanson, PhD who served as a technical expert panel. Finally, we thank Joel Nigg, PhD, and Peter S. Jensen, MD for their peer review of the data.

Drs Peterson and Hempel conceptualized and designed the study, collected data, conducted the analyses, drafted the initial manuscript, and critically reviewed and revised the manuscript; Dr Trampush conducted the critical appraisal; Ms Brown, Ms Maglione, Drs Bolshakova and Padkaman, and Ms Rozelle screened citations and abstracted the data; Dr Miles conducted the analyses; Ms Yagyu designed and executed the search strategy; Ms Motala served as data manager; and all authors provided critical input for the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.

This trial has been registered at PROSPERO (identifier CRD42022312656).

COMPANION PAPER: A companion to this article can be found online at https://www.pediatrics.org/cgi/doi/10.1542/peds.2024-065787 .

Data sharing statement: Data are available in SRDRPlus.

attention-deficit/hyperactivity disorder

area under the curve

Child Behavior Checklist

continuous performance test

functional magnetic resonance imaging

receiver operating characteristics

strength of evidence

technical expert panel

Competing Interests

Supplementary data.

Advertising Disclaimer »

Citing articles via

Email alerts.

review article research paper

Affiliations

  • Editorial Board
  • Editorial Policies
  • Journal Blogs
  • Pediatrics On Call
  • Online ISSN 1098-4275
  • Print ISSN 0031-4005
  • Pediatrics Open Science
  • Hospital Pediatrics
  • Pediatrics in Review
  • AAP Grand Rounds
  • Latest News
  • Pediatric Care Online
  • Red Book Online
  • Pediatric Patient Education
  • AAP Toolkits
  • AAP Pediatric Coding Newsletter

First 1,000 Days Knowledge Center

Institutions/librarians, group practices, licensing/permissions, integrations, advertising.

  • Privacy Statement | Accessibility Statement | Terms of Use | Support Center | Contact Us
  • © Copyright American Academy of Pediatrics

This Feature Is Available To Subscribers Only

Sign In or Create an Account

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

  • Advanced Search
  • Journal List
  • PLoS Comput Biol
  • v.9(7); 2013 Jul

Logo of ploscomp

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

  • interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
  • an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
  • a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

  • keep track of the search items you use (so that your search can be replicated [10] ),
  • keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
  • use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
  • define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
  • do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

An external file that holds a picture, illustration, etc.
Object name is pcbi.1003149.g001.jpg

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

  • discussing in your review the approaches, limitations, and conclusions of past reviews,
  • trying to find a new angle that has not been covered adequately in the previous reviews, and
  • incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

  • be thorough,
  • use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
  • look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

  • the major achievements in the reviewed field,
  • the main areas of debate, and
  • the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

Help | Advanced Search

Computer Science > Human-Computer Interaction

Title: llms in hci data work: bridging the gap between information retrieval and responsible research practices.

Abstract: Efficient and accurate information extraction from scientific papers is significant in the rapidly developing human-computer interaction research in the literature review process. Our paper introduces and analyses a new information retrieval system using state-of-the-art Large Language Models (LLMs) in combination with structured text analysis techniques to extract experimental data from HCI literature, emphasizing key elements. Then We analyze the challenges and risks of using LLMs in the world of research. We performed a comprehensive analysis on our conducted dataset, which contained the specified information of 300 CHI 2020-2022 papers, to evaluate the performance of the two large language models, GPT-3.5 (text-davinci-003) and Llama-2-70b, paired with structured text analysis techniques. The GPT-3.5 model gains an accuracy of 58\% and a mean absolute error of 7.00. In contrast, the Llama2 model indicates an accuracy of 56\% with a mean absolute error of 7.63. The ability to answer questions was also included in the system in order to work with streamlined data. By evaluating the risks and opportunities presented by LLMs, our work contributes to the ongoing dialogue on establishing methodological validity and ethical guidelines for LLM use in HCI data work.

Submission history

Access paper:.

  • HTML (experimental)
  • Other Formats

license icon

References & Citations

  • Google Scholar
  • Semantic Scholar

BibTeX formatted citation

BibSonomy logo

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

  • Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

  • Open access
  • Published: 21 March 2024

Recent research advances in pain mechanisms in McCune–Albright syndrome thinking about the pain mechanism of FD/MAS

  • Yong Wang   ORCID: orcid.org/0009-0004-8916-4315 1 &
  • Tao Jiang   ORCID: orcid.org/0009-0004-6873-0410 1  

Journal of Orthopaedic Surgery and Research volume  19 , Article number:  196 ( 2024 ) Cite this article

233 Accesses

Metrics details

The lack of effective understanding of the pain mechanism of McCune–Albright syndrome (MAS) has made the treatment of pain in this disease a difficult clinical challenge, and new therapeutic targets are urgently needed to address this dilemma.

This paper summarizes the novel mechanisms, targets, and treatments that may produce pain in MAS and fibrous dysplasia (polyfibrous dysplasia, or FD).

We conducted a systematic search in the PubMed database, Web of Science, China Knowledge Network (CNKI) with the following keywords: “McCune–Albright syndrome (MAS); polyfibrous dysplasia (FD); bone pain; bone remodeling; G protein coupled receptors; GDNF family receptors; purinergic receptors and glycogen synthase kinase”, as well as other keywords were systematically searched. Papers published between January 2018 and May 2023 were selected for finding. Initial screening was performed by reading the titles and abstracts, and available literature was screened against the inclusion and exclusion criteria.

In this review, we systematically analyzed the cutting-edge advances in this disease, synthesized the findings, and discussed the differences. With regard to the complete mechanistic understanding of the pain condition in FD/MAS, in particular, we collated new findings on new pathways, neurotrophic factor receptors, purinergic receptors, interferon-stimulating factors, potassium channels, protein kinases, and corresponding hormonal modulation and their respective strengths and weaknesses.

This paper focuses on basic research to explore FD/MAS pain mechanisms. New nonneuronal and molecular mechanisms, mechanically loaded responsive neurons, and new targets for potential clinical interventions are future research directions, and a large number of animal experiments, tissue engineering techniques, and clinical trials are still needed to verify the effectiveness of the targets in the future.

Introduction

McCune–Albright syndrome (MAS) is a genetic disorder with multiple systemic involvement, and it most often occurs in women. It is characterized by atypical symptoms, multiple fibrous dysplasia, cutaneous café au lait, and multiple endocrine disorders that can involve various organs and tissues, including the gonads, thyroid, pituitary, and adrenal glands. The disease has a prevalence of 1 in 100, 000 to 1 in a million and exists in both mono- and poly-osteogenic forms. The disease is characterized by the formation of cranial skeletal dysplasia or mesial-extremity skeletal lesions with intertwining fibrous tissue with normal bone, as well as the accumulation of nonmineralized bone and increased osteoclast activity. Somatic mutations resulting from activating mutations in the gene (GNAS1) encoding the α-subunit of the guanine nucleotide-binding protein (Gsα) during early embryonic life are the main cause of the disease. Because of the mosaic distribution of mutations, it results in a low rate of positive genetic tests leading to adenylate cyclase stimulation and cyclic AMP production [ 1 ], as well as the proliferation of bone progenitor cells and the lesion of poly-fibrous dysplasia (FD).

The disease has been reviewed by Tseng [ 2 ] and Jiang [ 3 ]; however, they only focused on the statement of existing symptoms and the summary of diagnostic methods, the most serious pain mechanism of this disease has not been discussed. The focus of this article was to review the pain mechanisms of FD/MAS in axial limb skeletal lesions. This disease can lead to bone pain, spinal deformities, and fractures; moreover, corrective surgery, although commonly used in FD/MAS, does not provide sustained pain relief. Currently, surgical procedures and existing drugs that alter bone turnover pathways also have obvious drawbacks, and the incomplete correlation between disease lesion load and patient-reported pain intensity adds to the complexity of FD/MAS pain management [ 4 ]. Due to the lack of effective international understanding of the pain mechanisms in this disease, the pain treatment of FD/MAS remains a difficult problem to overcome in clinical practice. Therefore, this paper reviews recent research on pain mechanisms in FD/MAS, such as mechanosensitive channels, redox, acidosis, specific hormone-regulated abnormal bone remodeling, signaling pathways regulating pain, interferon-stimulated factors, and key receptors; in addition, it describes new approaches to treat pain in MAS to explore new therapeutic targets for pain in this disease, as well as tissue engineering approaches.

Materials and methods

We performed a database search on PubMed, Web of Science, China Knowledge Network (CNKI) and selected papers that were published between January 2018 and May 2023 in the English language. PubMed was last accessed on May 30th, 2023. The following keywords and terms were used: (1) McCune–Albright syndrome (MAS); (2) poly-fibrous displasia (FD); (3) bone pain; (4) bone remodeling; (5) G protein coupled receptors; (6) GDNF family receptors; (7) purinergic receptors; and (8) glycogen synthase kinase. The following string was used:

(((((((("Fibrous Dysplasia, Polyostotic" [Mesh] or Dysplasia, Polyostotic Fibrous or Dysplasias, Polyostotic Fibrous or Fibrous Dysplasias, Polyostotic or Polyostotic Fibrous Dysplasias or Polyostotic Fibrous Dysplasia or Albright Syndrome or Syndrome, Albright or McCune–Albright Syndrome or McCune Albright Syndrome or Syndrome, McCune–Albright or Albright’s Disease or Fibrous Dysplasia with Pigmentary Skin Changes and Precocious Puberty or Albright’s Syndrome with Precocious Puberty or Albright-Mccune-Sternberg Syndrome or Albright-Sternberg Syndrome or Albright’s Syndrome or Syndrome, Albright’s or Albright’s Disease of Bone) OR ("Bone Remodeling" [Mesh] or Remodeling, Bone or Bone Turnover or Bone Turnovers or Turnover, Bone or Turnovers, Bone and pain)) OR ("Receptors, G-Protein-Coupled" [Mesh] or Receptors, G Protein Coupled or G-Protein-Coupled Receptors or G Protein Coupled Receptors or G-Protein-Coupled Receptor or Receptor, G-Protein-Coupled or G Protein Coupled Receptor and pain)) OR ("Glial Cell Line-Derived Neurotrophic Factor Receptors" [Mesh] or Glial Cell Line Derived Neurotrophic Factor Receptors or GDNF Receptors or GFRA4 Receptor or GDNF Family Receptor Alpha 4 or Glial Derived Neurotrophic Factor Receptor 4 or GDNF Family Receptor 4 or GFRA3 Receptor or GDNF Family Receptor Alpha 3 or Glial Derived Neurotrophic Factor Receptor 3 or GDNF Family Receptor 3 or GFRA1 Receptor or GDNF Family Receptor 1 or GDNF Family Receptor Alpha 1 or Glial Derived Neurotrophic Factor Receptor 1 or GFRA2 Receptor or GDNF Family Receptor Alpha 2 or Glial Derived Neurotrophic Factor Receptor 2 or GDNF Family Receptor 2 and pain)) OR ("Receptors, Purinergic" [Mesh] or Purine Receptor or Receptor, Purine or Purinoceptors or Purine Receptors or Purinergic Receptors or Purinoceptor or Purinergic Receptor or Receptor, Purinergic or Receptors, Purine or Methyladenine Receptors or Receptors, Methyladenine and pain)) OR ("Glycogen Synthase Kinases" [Mesh] or Kinases, Glycogen Synthase or Glycogen Synthase Kinase or Kinase, Glycogen Synthase or Synthase Kinase, Glycogen and pain)) OR ("Bites and Stings" [Mesh] or Stings and Bites or Stings or Sting or Bites or Bite and pain)) OR ("P2RX7 protein, human" [Supplementary Concept] and bone pain)) OR ("Cyclic GMP-Dependent Protein Kinases" [Mesh] and pain).

In addition, a manual search was performed of the cited references in the included studies. The reviewers (YW and TJ) retrieved the data and independently analyzed each selected study; instances of disagreement were resolved by the senior investigator (TJ).

A total of 1, 778 papers were identified with no duplicates; additionally, as a first step, 110 papers were excluded for other reasons, and 122 papers were excluded due to duplication (PRISMA flow diagram reported in Fig.  2 ). A total of 712 articles that did not meet the requirements by reading the abstracts and titles were excluded. However, there were 78 unresearched literature reports, which resulted in 988 papers for further evaluation. According to the selection criteria, out of the 988 qualified research results that were evaluated by PubMed, 15 articles were ultimately included in this analysis.

Mechanisms by which abnormal bone remodeling affects FD/MAS pain

Mechanically sensitive channel piezo family.

FD/MAS is a genetic skeletal disorder that is characterized by abnormal bone remodeling, and studies have found that bone remodeling and inflammatory processes are thought to produce bone pain in FD/MAS [ 5 ]. The Piezo family includes Piezo1 and Piezo2 subtypes, and the most relevant stressor regulating Piezo1 is mechanical force, which is a key factor in bone remodeling. Piezo1 ion channel mediatesosteoblasts’ mechanical force sensitivity and its important role is load-dependent bone formation, suggesting a new mechanism for mechanical force sensing in bone tissue. When sensory nerve fibers swell in response to mechanical stress, they cause the activation of mechanosensitive ion channels, which are recruited in osteoblasts and chondrocytes, thereby regulating osteogenesis and cartilage degradation in joints and promoting the process of bone remodeling [ 6 ]. Piezo1 is detected in osteoblasts (which are the most abundant bone cells), and Piezo1-Akt channels are key players in tissue remodeling, wherein they mainly act on osteoblasts. Among these signaling effects, Piezo1-Akt signaling and the inhibition of sclerostin (which is an important regulator of bone formation), as well as its downregulation, affect osteoblast function in FD/MAS patients, and the activation of the Piezo1-Akt pathway in osteoblasts is needed for mechanical stretch-induced downregulation of sclerostin (Sost) expression. Therefore, Piezo1 was identified as a major skeletal mechanosensor regulating skeletal homeostasis, which raises the possibility for new selective therapeutic approaches for FD/MAS [ 7 , 8 ].

FD/MAS generates intraosseous pressure in the bone microenvironment, whereby it sensitizes primary afferent nerves through the activation of mechanoreceptors and osteocyte mechanotransduction. Piezo2 has been found to be critical for transferring pain and touch sensation, as well as proprioception, in the nervous system, and irreversible in vivo Piezo2 microdamage may be an important contributor to FD/MAS pain [ 9 ]. It has been suggested that primary Piezo2 injury is the primary pain pathway in FD/MAS because of the loss of unbalanced subthreshold calcium currents and N-methyl-D-aspartate (NMDA) activation, as well as due to the dysregulation of the primary pain pathway in the dorsal horn of the spinal cord (due to NMDA activation) and the loss of l-type calcium currents, thus resulting in the loss of activation of wide dynamic range neurons. Animal studies have demonstrated that loss-of-function mutations in Piezo2 lead to loss of pain perception, whereas irreversible Piezo2 microdamage can continuously activate the transcriptional pathway [ 10 ].

Additional studies have suggested that Piezo2 plays a role in synchronizing supraspinal neural networks [ 11 ], and the loss of excitatory function of Piezo2 may theoretically lead to impaired spinal synchrony and loss of spinal function in the central pattern generator (CPG). Pathogenic variants of Cav1.3, which encode the CACNA1D gene, may play a role in the pain mechanisms of FD/MAS and have a specific relationship with dysregulation of pain pathways in the dorsal horn of the spinal cord [ 12 ]. Table 1 summarizes studies on the mechanisms by which the Piezo family affects pain through abnormal bone remodeling.

Role of redox homeostasis in abnormal bone remodeling pain

Oxidative stress (OS) is a potential injury resulting from the disruption of the balance of strong oxidants and antioxidants, which plays an important role in the physiopathological and aging processes of many tissues. Its broad role is in bone tissue, where reactive oxygen species (ROS) are involved in bone remodeling via the RANKL pathway, thus physiologically promoting osteoclast differentiation [ 14 ]. The redox state should be kept in a precise balance to maintain proper bone remodeling; however, due to increased osteoclast activity, FD/MAS predisposes to ROS accumulation and cellular stress; additionally, ROS promote osteoblast apoptosis and indirectly elicit osteoclast formation, and excessive mechanical forces not only affect bone mass and its microarchitecture but also promote inflammatory responses through redox affecting NF-κB factors. Thus, redox dysregulation affecting bone remodeling plays an important role in the pathogenesis of FD/MAS pain [ 14 ]. Epithelial sodium channels (ENaCs) are major contributors to intracellular sodium homeostasis. Specifically, αENaC subunits are present in skeletal cells, including articular chondrocytes and osteoblasts, and are thought to be involved in mechanotransduction, sodium transport, and extracellular sodium sensing. It was found that there is a correlation between bone sodium content and FD/MAS, as well as the fact that ENaC is sensitive not only to extracellular sodium and mechanical forces but also to oxidative stress. Moreover, it has been shown that ENaC is upregulated through ROS production and that a similar correlation between ENaC function or expression and ROS production in osteoblasts may provide a new research direction for FD/MAS [ 15 ].

The role of acidosis in the pain mechanism of FD/mas bone remodeling

Acid-sensing ion channel (asic).

Acid-sensing ion channels (ASICs) are a group of proton-gated cation-permeable channels, and reduced osteoblastogenesis was observed under acidic conditions when osteoblasts were cultured in medium with elevated pH, whereas ASIC2, ASIC3, and ASIC4 were progressively upregulated. The authors suggested that decreased pH activates acid-sensing ion channels and that acidic pH leads to impaired osteoblastogenesis, which represents a pH-dependent pattern that may support a role for ASIC in the painful mechanism of FD/MAS bone remodeling [ 16 ].

Transient receptor potential (TRP) family channels

Transient receptor potential (TRP) channels, which are a class of cation channels, are widely expressed cation channels that play an important role in mediating calcium homeostasis and are considered to be potential regulators of inflammatory pain. Skeletal disorders with increased bone resorption by osteoclasts are usually associated with pain, and they secrete large amounts of protons during bone resorption, thus resulting in an acidic local environment. Acidosis is a typical noxious stimulus that innervates injured skeletal nerves and can excite injurious sensory neurons by opening acid-sensing ion channels (ASIC) and transient receptor potential channels [ 17 ]. The transient receptor potential channel subfamily V (TRPV) channels TRPV1, TRPA1, and TRPV4, as well as the transient receptor potential cation channel subfamily M (TRPM7) channels, have been reported to be present in bone; however, only TRPV1 and TRPA1 respond to pH changes [ 18 ]. However, in a mildly acidotic environment, OC formation is also increased when the specific agonist 4-alpha PDD activates TRPV4, thus indirectly causing acidosis and increasing FD/MAS pain risk [ 19 ].

Altogether, TRPV4, TRPV1, and TRPV2 are involved in acidosis, as well as in promoting osteoclast formation and impairing bone formation by enhancing calcium ion excretion and osteoclast resorption. TRPA1 is distributed in a large number of cell types and is usually associated with TRPV1, which is sensitive to external stimuli (especially in nerve tissue). Tissue stiffness in FD/MAS patients activates TRPV4 and increases M1 macrophage infiltration, and TRPV1 has a prominent role with TRPV4 in external stimuli and inflammatory environmental episodes; thus, this ion channel is considered a potential therapeutic target for FD/MAS pain. TRPM7 is a TRP channel involved in bone metabolism, as well as a cation channel that is covalently linked to the structural domain of protein kinase; additionally, it has been found that TRPM7 acts as a cation channel in bone. TRPM7 has also been found to play a key role in bone metabolism as a cation channel.

In the development of FD/MAS, formaldehyde can be secreted by disease-involved tissues. In pain models, formaldehyde was found to upregulate TRPV1 expression and contribute to pain behavior. These facts suggest that the targeting of formaldehyde production may be a potential therapeutic approach for FD/MAS pain. A summary of studies on the mechanisms by which transient receptor potential channels (TRPs) influence abnormal bone remodeling is detailed in Table  2 .

Summary of hormones associated with painful abnormal bone remodeling

Current research suggests that abnormal bone growth in FD/MAS leads to structural and functional alterations in nerve fibers and increased sensitivity to pain. The overproduction of hormones acting on nociceptive nerves can also lead to pain.

Role of parathyroid hormone in abnormal bone remodeling

Parathyroid hormone (PTH) regulates the body’s calcium homeostasis. In vitro and in vivo studies have shown that PTH directly activates survival signaling in osteoblasts, thus leading to an increase in the number of osteoblasts through a delay in osteoblast apoptosis, which correspondingly affects abnormal bone remodeling [ 27 ]. In a rat model of PD (Parkinson’s disease), PTH was found to have anti-inflammatory and inhibitory effects on bone loss. Cortisol inhibits parathyroid hormone and stimulates the proliferation and differentiation of progenitor cells into osteoclasts. Administration of cortisol may attenuate bone remodeling in patients with MAS by inhibiting bone formation [ 28 ]. Therefore, cortisol may improve MAS pain as a therapy, but not optimally.

Oxytocin regulates bone formation

Oxytocin (OT) is produced by the hypothalamus, and osteoblasts expressing OT receptors can bind to it. It was found that OTs are involved in the bone remodeling process, which reduces bone resorption and leads to a relative increase in bone formation. The OT treatment process leads to increased intracellular calcium levels and modulates the stimulation of osteoblast formation, thus regulating bone formation [ 29 ].

Insulin is involved in activating abnormal bone remodeling

Insulin has been reported to activate osteoblast differentiation, reduce apoptosis of osteoblasts, and decrease osteoclast activity. Insulin-like growth factor-1 (IGF-1) can be activated by human growth hormone and can also be secreted by osteoblasts. IGF-1 is involved in abnormal bone remodeling during FD/MAS, and IGF-1 synthesis is regulated by parathyroid hormone (PTH) [ 30 ]. A low decrease in IGF-1 expression is thought to be needed for the onset of apoptosis and to promote osteoblast differentiation.

Sex hormones are involved in activating abnormal bone remodeling

Estrogen (ES) plays an important role in the regulation of skeletal maturation and bone remodeling. At the cellular level, ES inhibits osteoclast differentiation, and deficiency in ES leads to increased osteoclast formation, which reduces osteoclast number and decreases the number of active remodeling units [ 31 ]. Testosterone activates osteoblasts by activating steroid receptors (either directly or via aromatization to estradiol) to regulate FD/MAS abnormal bone remodeling [ 32 ].

Erythropoietin is involved in abnormal bone remodeling

Erythropoietin (EPO) can indirectly increase bone formation by increasing the expression of vascular endothelial growth factor (VEGF), and EPO usually indirectly activates osteoblast differentiation. These empirical findings are important for understanding bone remodeling and may be useful in treating bone defect growth issues [ 33 ]. EPO can promote bone formation directly and indirectly through VEGF.

Abnormal bone remodeling involving lipocalin

Lipocalin (AN) is a protein produced by adipose tissue, and studies have identified a potential role for AN as a positive bone mass regulator, angiogenesis stimulator, and osteoclast suppressor. Two AN receptors have been identified by the expression of osteoblasts, thus suggesting a role for AN in bone metabolism. Additionally, it has a direct function in bone metabolism by promoting proliferation and stimulating bone formation, as well as possessing the potential for bone regeneration [ 34 ]. AN and its agonists may be used to treat FD/MAS abnormal bone remodeling pain.

Nociceptive sensitization and the role of G protein-coupled receptors (GPCRs) in FD/MAS pain

In FD/MAS, the articular cartilage is not innervated and does not become a direct source of pain, whereas the rest of the tissue has nociceptive receptors and sympathetic nerve fibers. Nociceptive receptors that are continuously stimulated respond to harmless painful stimuli or respond strongly to harmful pain, which is a phenomenon known as nociceptive sensitization [ 35 ]. The spinal dorsal horn is the first transit point for pain signals, which are transmitted to the dorsal horn of the spinal cord and subsequently to the upper brain centers, thus creating conscious pain. Numerous studies have shown that considerable amounts of pain are mediated by both central and peripheral sensitization. Peripheral sensitization leads to abnormal hyperexcitability of primary sensory neurons, whereas central sensitization enhances injurious information from the dorsal horn of the spinal cord to the cerebral cortex, in regards to synaptic transmission [ 36 ].

G protein-coupled receptors (GPCRs) can sometimes indirectly sensitize various voltage-gated ion channels (VGICs) that are expressed on sensory neurons, thereby further enhancing nociceptive signals to the spinal cord. Studies have demonstrated that ovarian cancer G protein-coupled receptor 1 (OGR1) family subtypes are differentially expressed in different inflammatory pain states and that they are involved in short- and long-term chronic inflammatory pain. Alkaline pH promotes mineralization and osteoblast potential, whereas acidosis correspondingly stimulates osteoclast resorption. OC sensitivity to protons involves the early expression of ovarian cancer G protein-coupled receptor 1 (OGR1), and studies have suggested that OGR1 is involved in osteoclast formation and that OGR1 may be a central acid sensor in bone , which is important for studying FD/MAS-induced pain mechanisms [ 37 ].

Dual role of the NO/cGMP (cyclic guanosine monophosphate) signaling pathway

The main process of the NO/cGMP signaling pathway in cells involves NO activation of soluble guanylate cyclase, which leads to subsequent production of cGMP. The activation of NO/cGMP signaling in the spinal cord significantly induces upregulation of downstream effectors, as well as reactive astrogliosis and microglia polarization involved in the chronic pain process. In dorsal root ganglion neurons, natriuretic peptides bind to granular guanylate cyclase to produce and further activate the cGMP/PKG pathway, thus contributing to the development of FD/MAS pain. In addition, FD/MAS leads to the upregulation of multiple receptors that are involved in the activation of the NO/cGMP signaling pathway in various types of pain, and the NO/cGMP signaling pathway induces the expression of downstream effectors that exert analgesic effects in both neuropathic and inflammatory pain [ 38 ].

Lactoferrin can exert its antinociceptive sensitization effects by activating the NO-cGMP-ATP-sensitive K-channel signaling pathway, and this interaction is used in inflammatory and injurious pain treatment. The activation of NO/cGMP signaling was found to play an important role in the development of chronic pain, and this signaling pathway with dual roles is a promising target for FD/MAS pain treatment [ 39 ].

Effects of the ERK/CREB pathway on FD/MAS pain mechanisms

Extracellularly regulated protein kinases (ERKs) play important roles as cell signaling transducers in a variety of essential cellular functions, such as migration, differentiation, growth, and survival, whereas cyclic phospho-adenosine effector-binding proteins (CREBs) regulate the transcription of a variety of cellular genes, including dopaminergic neurons. Moreover, the activation of CREBs can improve muscle performance. It has been shown that recombinant Sirtuin 6 (SIRT6) downregulates Sox6 (which is a key blocker of slow fiber-specific genes) by increasing CREB transcription, which improves muscle performance; this effect represents another new direction in the study of FD/MAS pain mechanisms [ 40 ].

Oxidative stress and neuroinflammation

The BDNF-TrkB-ERK-CREB pathway is extensively involved in neurologically relevant processes, and the BDNF-TrkB-ERK-CREB pathway is closely associated with oxidative stress and neuroinflammatory processes. Ectopic ossification in FD/MAS patients can detect elevated sensory innervation, and activated sensory fibers can release neuropeptide P and calcitonin gene-related peptide (CGRP) via the induction of neuroinflammation [ 41 ]. It has been demonstrated that increased pain and pain aversion in bone pain rats is accompanied by the upregulation of brain-derived neurotrophic factor (BDNF) expression, and the phosphorylation of the ERK/CREB pathway (pERK/pCREB) is upregulated in response to increased pain and pain aversion (or the exogenous injection of BDNF). The decreased expression level of pERK/pCREB after blocking BDNF-TrkB signaling suggests that the ERK/CREB pathway is critical for BDNF-TrkB signaling-mediated pain and aversion pain, and this ERK/CREB pathway may induce pain in FD/MAS by mediating oxidative stress and neuroinflammatory processes [ 42 ].

miRNA-mediated activation of the ERK/CREB pathway

During pain, the activation of the ERK/CREB pathway is often mediated by microRNAs. In chronic inflammatory rats, miR-211 can target the 3’-UTR of the ERK gene, which greatly increases the expression levels of ERK and CREB, thus promoting hypersensitivity and consequently pain [ 43 ]. MicroRNA-365 decreases the expression of β-arrestin2, ERK, and CREB proteins, and β-arrestin2 negatively targets and inhibits ERK/CREB pathway activation to achieve better analgesic effects [ 44 ].

Modulation of FD/MAS pain by purinergic receptors (P2XRs)

Purinergic receptors (P2XRs) are ligand-gated cation channels that are mainly activated by extracellular ATP (eATP). Some P2XRs are found in osteoclast, and P2XR members regulate bone resorption, particularly through the proinflammatory microenvironment [ 45 ]. P2X3R was found to be distributed not only in the peripheral nervous system but also in the presynaptic portion of the spinal cord and in the periaqueductal gray matter (PAG) of the midbrain. Tian Shuxin et al. showed that both peripheral and central P2X3Rs are involved in analgesia [ 46 ]. Moreover, Liu Min reported that the functional upregulation of P2X3R is involved in the pathogenesis of pain, as well as the fact that P2X3 receptors are involved in injurious transmission and modulation in the peripheral and central nervous system and that P2X3R blocks mechanically abnormal pain and nociceptive hyperalgesia, thus exerting a significant analgesic effect [ 47 ]. P2X7R is the most abundant form of receptor in osteoclasts, and P2X7R mechanotransduction mediates not only osteoclast proliferation but also bone resorption, pain, and reduced mechanical sensitivity. The author suggests that blockage of the P2X3 receptor is a new therapeutic target for FD/MAS pain [ 48 ]. A summary of studies on purinergic receptors on pain mechanisms is detailed in Table  3 .

Important player involving GDNF family receptors in FD/MAS pain

Glial cell-derived neurotrophic factor (GDNF) has been shown to sensitize injury receptors and induce chronic inflammatory pain and muscle pain. GFRα is a high-affinity receptor for GDNF, and the knockdown of GFRα1 by intrathecal injection of antisense oligodeoxynucleotides, as well as mRNA encoding GFRα1, attenuates both GDNF-induced nociceptive hyperalgesia and chronic muscle pain. GDNF-GFRα1-Ret-ERK signaling has been reported to activate Runx2-mediated P3X1R gene transcription, which contributes to the sensitization of DRG neurons and induces bone pain; additionally, this study may provide a potential target mechanism leading to bone pain in FD/MAS patients. Inflammatory bone pain involves the activation of the GDNF-GFRα1 signaling pathway and the sensitization of nonpeptidergic bone afferent neurons, and neuroinflammation occurring in FD/MAS patients can be induced by GDNF expression in activated astrocytes, microglia, and infiltrating macrophages [ 51 ]. Studies have shown that bone afferent neurons of GFRα1 and GFRα2 are mostly nonpeptidergic; therefore, GDNF and neuroturin signaling through nonpeptidergic sensory neurons may be important for bone pain in FD/MAS. However, GDNF isolation does not prevent inflammation-induced pain behavior and may contribute to the maintenance of deep tissue pain [ 52 ]. Artemin, which is a member of the neurotrophic factor family derived from the glial cell lineage, is an important player in persistent pain, such as neuropathic cold pain and inflammatory bone pain [ 53 ]. It has been found that inflammatory bone pain in FD/MAS involves the activation of NGF-sensitive peptidergic neurons via the artemin/GFRα3 signaling pathway [ 54 ]. Moreover, the activation of the artemin/GFRα3 pathway can directly or indirectly activate TRP channel expression and activity and perpetuate pain; thus, it is known that artemin/GFRα3 plays an important role in FD/MAS pain [ 55 ]. Table 4 provides a detailed summary of the potential targeting mechanisms of the GDNF family of receptors for bone pain.

Interferon-stimulating factor (STING): a new target for FD/MAS pain

STING is an intracellular DNA sensor with an important role in chronic pain. DRG neurons are involved in interferon-stimulated factor (STING), which mediates the stimulus of the innate immune response and regulates injury perception. Moreover, upon binding to DNA, cyclic GMP-AMP synthase (cGAS) undergoes a conformational change to an active state and generates a second messenger cyclic GMP-AMP (cGAMP) [ 57 ]. cGAMP acts as a second messenger to activate STING and drives STING translocation from the endoplasmic reticulum to perinuclear microsomes via the Golgi apparatus [ 58 ]. Subsequently, activated STING recruits and phosphorylates TANK-binding kinase 1 (TBK1), which further activates interferon regulatory factor 1 (IRF1) and NF-κB factors. These factors enter into the nucleus to induce the expression of type I interferon (IFN-I) and proinflammatory factors, such as IL-6β, IL-7, and TNF-α [ 59 ]. Previous studies have demonstrated that the cGAS-STING pathway is a potential mechanism for many inflammation-mediated pathophysiological processes [ 60 ]. Although there is growing evidence that STING is involved in the generation and maintenance of neuroinflammation and inflammatory bone pain, a recent study in an alternate nerve injury (SNI) model demonstrated that STING activation in DRG neurons drives bone pain through the TBK1/IKK/NF-κB proinflammatory factor signaling pathway; thus, STING may be a new therapeutic FD/MAS pain target for the treatment of FD/MAS pain [ 61 ].

Potassium channel involvement in patients with FD/MAS

Patients with FD/MAS may evoke hypernormal activation of neuronal resonance and disturbances in neuroplasticity through enhanced responses to sensory input. These processes may involve imbalance at the neuronal level, thus leading to hyperexcitability and hypersensitivity of neurons in the CNS, which results in sustained activation of injury receptors; these findings are consistent with the development of central sensitization [ 62 ]. K channels are involved in fibromyalgia in FD/MAS, with channels including fast K + channels, potassium leak (KL) channels, KCHN2 voltage-gated potassium channels, and Kv channel autoantibodies. Additionally, it has been found that the water fraction of substance P (SP) is elevated in the cerebrospinal fluid of FD/MAS patients [ 63 ]. Substance P is a mediator of pain, is involved in pain production and transmission signaling, and can also exhibit anti-injury-sensing properties.

Potential new approaches to treat FD/MAS pain

Glycogen synthase kinase-3 (GSK-3) is a cytoplasmic serine/threonine protein kinase that is involved in a large number of key cellular processes, and kenpaullone (kp) is a glycogen synthase kinase-3 (GSK3)/cell cycle protein-dependent kinase (CDK) inhibitor [ 64 ]. In vitro and in vivo studies have found that kp acts as an analgesic in a preclinical mouse model of pathological pain and that its cellular mechanism of action in neurons is based on its GSK-3β inhibitory function, which enhances Kcc2/KCC2 gene expression. In turn, Kcc2 upregulation is dependent on the nuclear translocation of the neuronal connexin δ-connexin (δ-cat). The enhancement of Kcc2 gene expression by KAISO transcription factors and increased Kcc2 gene expression leads to increased KCC2 transporter proteins in neurons. Furthermore, kP exhibits a robust analgesic response in mice, and the defect in KCC2 expression in SCDH is repaired by KP [ 65 ].

Abnormal bone remodeling activates the GSK-3β/Drp1 pathway, thus leading to mitochondrial fission and dysfunction. fd/MAS can activate NLRP3 inflammatory vesicles, which then induce an injurious response. The intrathecal injection of the GSK-3β inhibitor TDZD-8 decreased Drp1 activity, maintained mitochondrial function, reduced the NLRP3 inflammatory vesicle cascade response, and ultimately attenuated FD/MAS pain behavior, as shown in the mechanism in Fig.  1 below (Fig.  2 ).

figure 1

The analgesic mechanism of kenpaulone and TDZD. Notes : ① NLRP3 inflammasome, which is a cellular structure composed of multiple proteins, is involved in immune and inflammatory responses; ② TDZD-8 is a non-ATP competitive GSK-3β inhibitor; ③ Kenpaulone (kp) is a glycogen synthase kinase 3 (GSK3); ④ cyclin dependent kinase (CDK) inhibitor. δ-catenin (δ-cat) is an intercellular junction protein, which is a member of the classical catenin family; ⑤ The KCC2 gene, which encodes a molecule known to help expel chloride ions from neurons; ⑥ Mitochondrial reactive oxygen species clusters (mtROS). The figure was drawn by first author Yong Wang using Biorender software

figure 2

PRISMA flow diagram

Randomized controlled trials are necessary for FD/MAS, especially given the significant placebo effect in clinical pain trials; however, they are difficult to perform, and this represents a gap in the field. Disease modifiers that have previously acted in some way can directly or indirectly affect osteoclast function. Pharmacological treatments using bisphosphonates, denosumab, and surgery also fail to provide sustained pain relief in FD/MAS patients and have significant side effects.

The development of novel mechanism-based therapies may clinically improve the treatment of FD/MAS pain; however, the fundamental improvement of the treatment of FD/MAS pain requires continued research efforts to develop new therapies with multimodal effects to target the underlying mechanisms of different pain conditions. The author believes that the neurochemical and electrophysiological properties of DRG neurons, especially those new nonneuronal and molecular mechanisms, mechanically loaded responsive neurons, and new targets for potential clinical interventions, are future research directions. A large number of animal experiments, tissue engineering techniques, and clinical trials are still needed to verify the effectiveness of the targets in the future.

In this review, we have focused on collating the clinical research advances in the last 5 years on the mechanisms of FD/MAS pain and the studies that may serve as new pathways in terms of basic experiments. In terms of a complete mechanistic understanding of FD/MAS pain conditions, new research results on new signaling pathways, neurotrophic factor receptors, purinergic receptors, interferon-stimulating factors, potassium channels, protein kinases, and corresponding hormonal modulation are organized, and their respective advantages and disadvantages are summarized separately to make the included articles more updated and complete and to summarize the therapeutic approaches that have been identified by the latest studies. The neuronal activity of FD/MAS patients is at a consistently high level, which leads to an increased sensitivity to painful stimuli. However, the treatment of pain due to FD/MAS also requires a comprehensive consideration of skeletal deformities. This article does not describe in detail the mechanical pain caused by skeletal deformities.

Abbreviations

Acid-sensing ion channels

Brain-derived neurotrophic factor

Central pattern generator

Cyclic phospho-adenosine effector-binding proteins

Calcitonin gene-related peptide

Cyclic GMP-AMP synthase

Dorsal root ganglia

Erythropoietin

Extracellularly regulated protein kinases

Fibrous dysplasia

  • G protein-coupled receptors

G protein-coupled receptor 1

Glycogen synthase kinase-3

Insulin-like growth factor-1

Interleukin-1β

  • McCune–Albright syndrome

Mitochondrial reactive oxygen species clusters

N-methyl-D-aspartate

Nerve growth factor

Oxidative stress

Parathyroid hormone

Parkinson’s disease

  • Purinergic receptors

Periaqueductal gray

P38 protein kinase

Promyosin receptor kinase A

Reactive oxygen species

Runt-related transcription Factor 2

Recombinant Sirtuin 6

Transient receptor potential

Transient receptor potential cation channel subfamily M

Transient receptor potential channel subfamily V

TANK-binding kinase 1

Vascular endothelial growth factor

Voltage-gated ion channels

Yes-associated protein

Tucker-Bartley A, Selen DJ, Golden E, van Gool R, Ebb D, Mannstadt M, Upadhyay J. Pharmacological interventions targeting pain in fibrous dysplasia/McCune-Albright syndrome. Int J Mol Sci. 2023;24(3):2550. https://doi.org/10.3390/ijms24032550.PMID:36768871;PMCID:PMC9916440 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Zeng BH. Current status of diagnosis and treatment of McCune-Albright syndrome. Med Rev. 2016;22(22):4475–9.

Google Scholar  

Jiang X, Li Y. Advances in the study of McCune-Albright syndrome in children. Chin J Pediatr Surg. 2015;36(12):956–60.

Golden E, Zhang F, Selen DJ, Ebb D, Romo L, Drubach LA, Shah N, O’Donnell LJ, Lemme JD, Myers R, Cay M, Kronenberg HM, Westin CF, Boyce AM, Kaban LB, Upadhyay J. Case report: the imperfect association between craniofacial lesion burden and pain in fibrous dysplasia. Front Neurol. 2022;16(13): 855157. https://doi.org/10.3389/fneur.2022.855157.PMID:35370900;PMCID:PMC8966612 .

Article   Google Scholar  

Chapurlat RD, Gensburger D, Jimenez-Andrade JM, Ghilardi JR, Kelly M, Mantyh P. Pathophysiology and medical treatment of pain in fibrous dysplasia of bone. Orphanet J Rare Dis. 2012;7(Suppl 1):S3. https://doi.org/10.1186/1750-1172-7-S1-S3 .

Article   PubMed   PubMed Central   Google Scholar  

Hendrickx G, Fischer V, Liedert A, von Kroge S, Haffner-Luntzer M, Brylka L, Pawlus E, Schweizer M, Yorgan T, Baranowsky A, Rolvien T, Neven M, Schumacher U, Beech DJ, Amling M, Ignatius A, Schinke T. Piezo1 inactivation in chondrocytes impairs trabecular bone formation. J Bone Miner Res. 2021;36(2):369–84. https://doi.org/10.1002/jbmr.4198 . ( Epub 2020 Nov 12 PMID: 33180356 ).

Article   CAS   PubMed   Google Scholar  

Wang L, You X, Lotinun S, Zhang L, Wu N, Zou W. Mechanical sensing protein PIEZO1 regulates bone homeostasis via osteoblast-osteoclast crosstalk. Nat Commun. 2020;11(1):282. https://doi.org/10.1038/s41467-019-14146-6.PMID:31941964;PMCID:PMC6962448 .

Sasaki F, Hayashi M, Mouri Y, Nakamura S, Adachi T, Nakashima T. Mechanotransduction via the Piezo1-Akt pathway underlies Sost suppression in osteocytes. Biochem Biophys Res Commun. 2020;521(3):806–13. https://doi.org/10.1016/j.bbrc.2019.10.174 . ( Epub 2019 Nov 8 PMID: 31708103 ).

Sonkodi B. Miswired proprioception in amyotrophic lateral sclerosis in relation to pain sensation (and in delayed onset muscle soreness)-Is Piezo2 channelopathy a principal transcription activator in proprioceptive terminals besides being the potential primary damage? Life. 2023;13(3):657. https://doi.org/10.3390/life13030657.PMID:36983813;PMCID:PMC10053310 .

Szczot M, Liljencrantz J, Ghitani N, Barik A, Lam R, Thompson JH, Bharucha-Goebel D, Saade D, Necaise A, Donkervoort S, Foley AR, Gordon T, Case L, Bushnell MC, Bönnemann CG, Chesler AT. PIEZO2 mediates injury-induced tactile pain in mice and humans. Sci Transl Med. 2018;10(462):9892. https://doi.org/10.1126/scitranslmed.aat9892 .

Article   CAS   Google Scholar  

Nagy ZF, Sonkodi B, Pál M, Klivényi P, Széll M. Likely pathogenic variants of Cav1. 3 and Nav1. 1 encoding genes in amyotrophic lateral sclerosis could elucidate the dysregulated pain pathways. Biomedicines. 2023;11(3):933. https://doi.org/10.3390/biomedicines11030933 .

Venugopal S, Ghulam-Jhelani Z, Ahn IS, Yang X, Wiedau M, Simmons D, Chandler SH. Early deficits in GABA inhibition parallels an increase in L-type Ca2+ currents in the jaw motor neurons of SOD1G93A mouse model for ALS. Neurobiol Dis. 2023;177: 105992. https://doi.org/10.1016/j.nbd.2023.105992 . ( Epub 2023 Jan 6 PMID: 36623607 ).

Perin M, Chinigò G, Genova T, Mussano F, Munaron L. The impact of plasma membrane ion channels on bone remodeling in response to mechanical stress, oxidative imbalance, and acidosis. Antioxidants. 2023;12(3):689. https://doi.org/10.3390/antiox12030689.PMID:36978936;PMCID:PMC10045377 .

Agidigbi TS, Kim C. Reactive oxygen species in osteoclast differentiation and possible pharmaceutical targets of ROS-mediated osteoclast diseases. Int J Mol Sci. 2019;20(14):3576. https://doi.org/10.3390/ijms20143576.PMID:31336616;PMCID:PMC6678498 .

Dizin E, Olivier V, Roth I, Sassi A, Arnoux G, Ramakrishnan S, Morel S, Kwak BR, Loffing J, Hummler E, Wenger RH, Frew IJ, Feraille E. Activation of the hypoxia-inducible factor pathway inhibits epithelial sodium channel-mediated sodium transport in collecting duct principal cells. J Am Soc Nephrol. 2021;32(12):3130–45. https://doi.org/10.1681/ASN.2021010046 .

Boyce AM, Kelly MH, Brillante BA, Kushner H, Wientroub S, Riminucci M, Bianco P, Robey PG, Collins MT. A randomized, double blind, placebo-controlled trial of alendronate treatment for fibrous dysplasia of bone. J Clin Endocrinol Metab. 2014;99(11):4133–40. https://doi.org/10.1210/jc.2014-1371 .

Zhen G, Fu Y, Zhang C, Ford NC, Wu X, Wu Q, Yan D, Chen X, Cao X, Guan Y. Mechanisms of bone pain: progress in research from bench to bedside. Bone Res. 2022;10(1):44. https://doi.org/10.1038/s41413-022-00217-w.PMID:35668080;PMCID:PMC9170780 .

Koivisto AP, Belvisi MG, Gaudet R, Szallasi A. Advances in TRP channel drug discovery: from target validation to clinical studies. Nat Rev Drug Discov. 2022;21(1):41–59. https://doi.org/10.1038/s41573-021-00268-4 .

Duitama M, Vargas-López V, Casas Z, Albarracin SL, Sutachan JJ, Torres YP. TRP channels role in pain associated with neurodegenerative diseases. Front Neurosci. 2020;4(14):782. https://doi.org/10.3389/fnins.2020.00782.PMID:32848557;PMCID:PMC7417429 .

Liu YS, Liu YA, Huang CJ, Yen MH, Tseng CT, Chien S, Lee OK. Mechanosensitive TRPM7 mediates shear stress and modulates osteogenic differentiation of mesenchymal stromal cells through osterix pathway. Sci Rep. 2015;12(5):16522. https://doi.org/10.1038/srep16522.PMID:26558702;PMCID:PMC4642269 .

Son A, Kang N, Kang JY, Kim KW, Yang YM, Shin DM. TRPM3/TRPV4 regulates Ca2+-mediated RANKL/NFATc1 expression in osteoblasts. J Mol Endocrinol. 2018;61(4):207–18. https://doi.org/10.1530/JME-18-0051 . ( PMID: 30328352 ).

Corrigan MA, Johnson GP, Stavenschi E, Riffault M, Labour MN, Hoey DA. TRPV4-mediates oscillatory fluid shear mechanotransduction in mesenchymal stem cells in part via the primary cilium. Sci Rep. 2018;8(1):3824. https://doi.org/10.1038/s41598-018-22174-3.PMID:29491434;PMCID:PMC5830574 .

Du G, Chen W, Li L, Zhang Q. The potential role of mechanosensitive ion channels in substrate stiffness-regulated Ca2+ response in chondrocytes. Connect Tissue Res. 2022;63(5):453–62. https://doi.org/10.1080/03008207.2021.2007902 . ( Epub 2021 Nov 23 PMID: 34814790 ).

Dutta B, Arya RK, Goswami R, Alharbi MO, Sharma S, Rahaman SO. Role of macrophage TRPV4 in inflammation. Lab Invest. 2020;100(2):178–85. https://doi.org/10.1038/s41374-019-0334-6 .

Liu BL, Cao QL, Zhao X, Liu HZ, Zhang YQ. Inhibition of TRPV1 by SHP-1 in nociceptive primary sensory neurons is critical in PD-L1 analgesia. JCI Insight. 2020;5(20): e137386. https://doi.org/10.1172/jci.insight.137386.PMID:32960817;PMCID:PMC7605531 .

Chen W, Li H, Hao X, Liu C. TRPV1 in dorsal root ganglion contributed to bone cancer pain. Front Pain Res. 2022;9(3):1022022. https://doi.org/10.3389/fpain.2022.1022022.PMID:36438444;PMCID:PMC9682177 .

Wein MN, Kronenberg HM. Regulation of bone remodeling by parathyroid hormone. Cold Spring Harb Perspect Med. 2018;8(8): a031237. https://doi.org/10.1101/cshperspect.a031237.PMID:29358318;PMCID:PMC6071549 .

Du L, Feng R, Ge S. PTH/SDF-1α cotherapy promotes proliferation, migration and osteogenic differentiation of human periodontal ligament stem cells. Cell Prolif. 2016;49(5):599–608. https://doi.org/10.1111/cpr.12286 .

Tamma R, Colaianni G, Zhu LL, DiBenedetto A, Greco G, Montemurro G, Patano N, Strippoli M, Vergari R, Mancini L, Colucci S, Grano M, Faccio R, Liu X, Li J, Usmani S, Bachar M, Bab I, Nishimori K, Young LJ, Buettner C, Iqbal J, Sun L, Zaidi M, Zallone A. Oxytocin is an anabolic bone hormone. Proc Natl Acad Sci USA. 2009;106(17):7149–54. https://doi.org/10.1073/pnas.0901890106 .

Klein GL. Insulin and bone: recent developments. World J Diabetes. 2014;5(1):14–6. https://doi.org/10.4239/wjd.v5.i1.14.PMID:24567798;PMCID:PMC3932424 .

Jackson E, Lara-Castillo N, Akhter MP, Dallas M, Scott JM, Ganesh T, Johnson ML. Osteocyte Wnt/β-catenin pathway activation upon mechanical loading is altered in ovariectomized mice. Bone Rep. 2021;14(15): 101129. https://doi.org/10.1016/j.bonr.2021.101129.PMID:34584905;PMCID:PMC8455641 .

Manolagas SC, O’Brien CA, Almeida M. The role of estrogen and androgen receptors in bone health and disease. Nat Rev Endocrinol. 2013;9(12):699–712. https://doi.org/10.1038/nrendo.2013.179 .

Tsiftsoglou AS. Erythropoietin (EPO) as a key regulator of erythropoiesis, bone remodeling and endothelial transdifferentiation of multipotent mesenchymal stem cells (MSCs): implications in regenerative medicine. Cells. 2021;10(8):2140. https://doi.org/10.3390/cells10082140.PMID:34440909;PMCID:PMC8391952 .

Tunheim EG, Skallevold HE, Rokaya D. Role of hormones in bone remodeling in the craniofacial complex: a review. J Oral Biol Craniofac Res. 2023;13(2):210–7. https://doi.org/10.1016/j.jobcr.2023.01.009 .

Xiaoyun Z, Hao Z, Lin M. Research progress on pain mechanism and treatment of knee osteoarthritis. Chin J Pain Med. 2023;29(01):50–8.

Zhang X, Zhu L, Wang X, Xia L, Zhang Y. Advances in the role and mechanism of miRNA in inflammatory pain. Biomed Pharmacother. 2023;161: 114463. https://doi.org/10.1016/j.biopha.2023.114463 . ( Epub 2023 Mar 1 PMID: 36868014 ).

Huang YH, Su YS, Chang CJ, Sun WH. Heteromerization of G2A and OGR1 enhances proton sensitivity and proton-induced calcium signals. J Recept Signal Transduct Res. 2016;36(6):633–44. https://doi.org/10.3109/10799893.2016.1155064 . ( Epub 2016 Apr 6 PMID: 27049592 ).

Li DY, Gao SJ, Sun J, Zhang LQ, Wu JY, Song FH, Liu DQ, Zhou YQ, Mei W. Targeting the nitric oxide/cGMP signaling pathway to treat chronic pain. Neural Regen Res. 2023;18(5):996–1003. https://doi.org/10.4103/1673-5374.355748.PMID:36254980;PMCID:PMC9827765 .

Guerrero-Solano JA, Bautista M, Velázquez-González C, De la O-Arciniega M, González-Olivares LG, Fernández-Moya M, Jaramillo-Morales OA. Antinociceptive synergism of pomegranate peel extract and acetylsalicylic acid in an animal pain model. Molecules. 2021; 26(18): 5434. doi: 10. 3390/molecules26185434.

Song MY, Han CY, Moon YJ, Lee JH, Bae EJ, Park BH. Sirt6 reprograms myofibers to oxidative type through CREB-dependent Sox6 suppression. Nat Commun. 2022;13(1):1808. https://doi.org/10.1038/s41467-022-29472-5.PMID:35379817;PMCID:PMC8980083 .

Zhang B, Zhao J, Wang Z, Xu L, Liu A, Du G. DL0410 attenuates oxidative stress and neuroinflammation via BDNF/TrkB/ERK/CREB and Nrf2/HO-1 activation. Int Immunopharmacol. 2020;86: 106729. https://doi.org/10.1016/j.intimp.2020.106729 . ( Epub 2020 Jul 6 PMID: 32645628 ).

Li J, Wang X, Wang H, Wang R, Guo Y, Xu L, Zhang G, Wu J, Wang G. The BDNF-TrkB signaling pathway in the rostral anterior cingulate cortex is involved in the development of pain aversion in rats with bone cancer via NR2B and ERK-CREB signaling. Brain Res Bull. 2022;185:18–27. https://doi.org/10.1016/j.brainresbull.2022.04.001 . ( Epub 2022 Apr 8 PMID: 35398541 ).

Sun L, Zhou J, Sun C. MicroRNA-211-5p enhances analgesic effect of dexmedetomidine on inflammatory visceral pain in rats by suppressing ERK signaling. J Mol Neurosci. 2019;68(1):19–28. https://doi.org/10.1007/s12031-019-01278-z . ( Epub 2019 Mar 14 PMID: 30874971 ).

Wu XP, She RX, Yang YP, Xing ZM, Chen HW, Zhang YW. MicroRNA-365 alleviates morphine analgesic tolerance via the inactivation of the ERK/CREB signaling pathway by negatively targeting β-arrestin2. J Biomed Sci. 2018;25(1):10. https://doi.org/10.1186/s12929-018-0405-9.PMID:29415719;PMCID:PMC5802062 .

Scarpellino G, Genova T, Quarta E, Distasi C, Dionisi M, Fiorio Pla A, Munaron L. P2X purinergic receptors are multisensory detectors for micro-environmental stimuli that control migration of tumoral endothelium. Cancers. 2022;14(11):2743. https://doi.org/10.3390/cancers14112743.PMID:35681724;PMCID:PMC9179260 .

Tian SX, Xu T, Shi RY, Cai YQ, Wu MH, Zhen SJ, Wang W, Zhou Y, Du JY, Fang JF, Shao XM, Liu BY, Jiang YL, He XF, Fang JQ, Liang Y. Analgesic effect of electroacupuncture on bone cancer pain in rat model: the role of peripheral P2X3 receptor. Purinergic Signal. 2023;19(1):13–27. https://doi.org/10.1007/s11302-022-09861-7 .

Yuan ZL, Liu XD, Zhang ZX, Li S, Tian Y, Xi K, Cai J, Yang XM, Liu M, Xing GG. Activation of GDNF-ERK-Runx1 signaling contributes to P2X3R gene transcription and bone cancer pain. iScience. 2022;25(9):104936. https://doi.org/10.1016/j.isci.2022.104936 .

Guedon JG, Longo G, Majuta LA, Thomspon ML, Fealk MN, Mantyh PW. Dissociation between the relief of skeletal pain behaviors and skin hypersensitivity in a model of bone cancer pain. Pain. 2016;157(6):1239–47. https://doi.org/10.1097/j.pain.0000000000000514.PMID:27186713;PMCID:PMC5142607 .

Falk S, Appel CK, Bennedbæk HB, Al-Dihaissy T, Unger A, Dinkel K, Heegaard AM. Chronic high dose P2X7 receptor inhibition exacerbates cancer-induced bone pain. Eur J Pharmacol. 2019;15(845):48–55. https://doi.org/10.1016/j.ejphar.2018.12.032 . ( Epub 2018 Dec 22 PMID: 30582910 ).

Sun BX, Peng AS, Liu PJ, Wang MJ, Ding HL, Hu YS, Kang L. Neuroprotection of exercise: P2X4R and P2X7R regulate BDNF actions. Purinergic Signal. 2023;19(1):297–303. https://doi.org/10.1007/s11302-022-09879-x .

Duarte Azevedo M, Sander S, Tenenbaum L. GDNF, a neuron-derived factor upregulated in glial cells during disease. J Clin Med. 2020;9(2):456. https://doi.org/10.3390/jcm9020456.PMID:32046031;PMCID:PMC7073520 .

Nencini S, Ringuet M, Kim DH, Greenhill C, Ivanusic JJ. GDNF, neurturin, and artemin activate and sensitize bone afferent neurons and contribute to inflammatory bone pain. J Neurosci. 2018;38(21):4899–911.

Gupta A, Chiavaccini L, Minnema LM, Chiu KW, Knazovicky D, Hash JA, Mishra SK, Lascelles BDX. Serum artemin is not correlated with sensitivity within dogs with naturally occurring osteoarthritis pain. Sci Rep. 2021;11(1):6682. https://doi.org/10.1038/s41598-021-85976-y.PMID:33758254;PMCID:PMC7988108 .

Messina DN, Peralta ED, Acosta CG. Glial-derived neurotrophic factor regulates the expression of TREK2 in rat primary sensory neurons leading to attenuation of axotomy-induced neuropathic pain. Exp Neurol. 2022;357: 114190. https://doi.org/10.1016/j.expneurol.2022.114190 . ( Epub 2022 Jul 27 PMID: 35907583 ).

Minnema L, Gupta A, Mishra SK, Lascelles BDX. Investigating the role of artemin and its cognate receptor, GFRα3, in osteoarthritis pain. Front Neurosci. 2022;27(16): 738976. https://doi.org/10.3389/fnins.2022.738976.PMID:35153665;PMCID:PMC8829392 .

O-Sullivan I, Kc R, Singh G, Das V, Ma K, Li X, Mwale F, Votta-Velis G, Bruce B, Natarajan Anbazhagan A, van Wijnen AJ. Sensory neuron-specific deletion of tropomyosin receptor kinase A (TrkA) in mice abolishes osteoarthritis (OA) pain via NGF/TrkA intervention of peripheral sensitization. Int J Mol Sci 2022; 23(20):12076

Hopfner KP, Hornung V. Molecular mechanisms and cellular functions of cGAS-STING signalling. Nat Rev Mol Cell Biol. 2020;21(9):501–21. https://doi.org/10.1038/s41580-020-0244-x . ( Epub 2020 May 18 PMID: 32424334 ).

Taguchi T, Mukai K, Takaya E, Shindo R. STING operation at the ER/Golgi interface. Front Immunol. 2021;3(12): 646304. https://doi.org/10.3389/fimmu.2021.646304.PMID:34012437;PMCID:PMC8126659 .

Decout A, Katz JD, Venkatraman S, Ablasser A. The cGAS-STING pathway as a therapeutic target in inflammatory diseases. Nat Rev Immunol. 2021;21(9):548–69. https://doi.org/10.1038/s41577-021-00524-z .

Liu M, Cheng X, Yan H, Chen J, Liu C, Chen Z. MiR-135-5p alleviates bone cancer pain by regulating astrocyte-mediated neuroinflammation in spinal cord through JAK2/STAT3 signaling pathway. Mol Neurobiol. 2021;58(10):4802–15. https://doi.org/10.1007/s12035-021-02458-y . ( Epub 2021 Jun 27 PMID: 34176097 ).

Zhang Y, Wang W, Gong Z, Peng Y, Li X, Zhang Z, Zhang X, You X, Wu J. Activation of the STING pathway induces peripheral sensitization via neuroinflammation in a rat model of bone cancer pain. Inflamm Res. 2023;72(1):117–32. https://doi.org/10.1007/s00011-022-01663-2 .

Lawson K. What is the progress of experimental drug development for fibromyalgia? Expert Opin Investig Drugs. 2023;32(7):563–5. https://doi.org/10.1080/13543784.2023.2230118 .

Lawson K. Identification of the involvement of potassium channels in fibromyalgia. Front Biosci. 2022;14(4):29. https://doi.org/10.31083/j.fbs1404029 . ( PMID: 36575839 ).

Turner TN, Sharma K, Oh EC, Liu YP, Collins RL, Sosa MX, Auer DR, Brand H, Sanders SJ, Moreno-De-Luca D, Pihur V, Plona T, Pike K, Soppet DR, Smith MW, Cheung SW, Martin CL, State MW, Talkowski ME, Cook E, Huganir R, Katsanis N, Chakravarti A. Loss of δ-catenin function in severe autism. Nature. 2015;520(7545):51–6. https://doi.org/10.1038/nature14186 .

Arioka M, Takahashi-Yanaga F. Glycogen synthase kinase-3 inhibitor as a multi-targeting anti-rheumatoid drug. Biochem Pharmacol. 2019;165:207–13. https://doi.org/10.1016/j.bcp.2019.02.020 . ( Epub 2019 Feb 15 PMID: 30776323 ).

Download references

1. Supported by the Project of Jiangsu Provincial Department of Science and Technology: Research on Biomechanical Monitoring and Digital Medical Diagnosis System for Osteoporosis Spinal Fractures, No. BRA2020149. 2. Supported by the Natural Science Foundation of Nanjing University of Chinese Medicine: Research on the mechanism of baicalein-mediated nonspecific signaling pathways regulating ferroptosis in the prevention and treatment of osteoporosis, No. ZXR2021066.

Author information

Authors and affiliations.

Orthopedics Department, Changzhou Traditional Chinese Medicine Hospital, Nanjing University of Chinese Medicine, Changzhou, 213000, Jangsu Province, China

Yong Wang & Tao Jiang

You can also search for this author in PubMed   Google Scholar

Contributions

Yong Wang designed the research; Yong Wang, and Tao Jiang performed the research; Yong Wang and Tao Jiang analyzed the data; Yong Wang wrote the paper. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Tao Jiang .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Related statements

The authors have read the PRISMA 2009 Checklist, and the manuscript was prepared and revised according to the PRISMA 2009 Checklist.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wang, Y., Jiang, T. Recent research advances in pain mechanisms in McCune–Albright syndrome thinking about the pain mechanism of FD/MAS. J Orthop Surg Res 19 , 196 (2024). https://doi.org/10.1186/s13018-024-04687-y

Download citation

Received : 31 July 2023

Accepted : 16 March 2024

Published : 21 March 2024

DOI : https://doi.org/10.1186/s13018-024-04687-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Bone remodeling
  • GDNF family receptors
  • Glycogen synthase kinase

Journal of Orthopaedic Surgery and Research

ISSN: 1749-799X

review article research paper

This paper is in the following e-collection/theme issue:

Published on 28.3.2024 in Vol 13 (2024)

Telehealth Evaluation in the United States: Protocol for a Scoping Review

Authors of this article:

Author Orcid Image

  • Yunxi Zhang 1, 2 , PhD   ; 
  • Yueh-Yun Lin 2 , PhD   ; 
  • Lincy S Lal 3 , PharmD, PhD   ; 
  • Jennifer C Reneker 4 , PT, MSPT, PhD   ; 
  • Elizabeth G Hinton 5 , MSIS, AHIP   ; 
  • Saurabh Chandra 2, 6 , MBBS, PhD   ; 
  • J Michael Swint 3, 7 , PhD  

1 Department of Data Science, John D Bower School of Population Health, University of Mississippi Medical Center, Jackson, MS, United States

2 Center for Telehealth, University of Mississippi Medical Center, Jackson, MS, United States

3 Department of Management, Policy and Community Health, The University of Texas School of Public Health, Houston, TX, United States

4 Department of Population Health Sciences, John D Bower School of Population Health, University of Mississippi Medical Center, Jackson, MS, United States

5 Rowland Medical Library, University of Mississippi Medical Center, Jackson, MS, United States

6 Department of Medicine, University of Mississippi Medical Center, Jackson, MS, United States

7 Institute for Clinical Research and Learning Healthcare, John P and Katherine G McGovern Medical School, The University of Texas Health Science Center at Houston, Houston, TX, United States

Corresponding Author:

Yunxi Zhang, PhD

Department of Data Science

John D Bower School of Population Health

University of Mississippi Medical Center

2500 North State Street

Jackson, MS, 39216-4505

United States

Phone: 1 6018153477

Email: [email protected]

Background: The rapid expansion of telehealth services, driven by the COVID-19 pandemic, necessitates systematic evaluation to guarantee the quality, effectiveness, and cost-effectiveness of telehealth services and programs in the United States. While numerous evaluation frameworks have emerged, crafted by various stakeholders, their comprehensiveness is limited, and the overall state of telehealth evaluation remains unclear.

Objective: The overarching goal of this scoping review is to create a comprehensive overview of telehealth evaluation, incorporating perspectives from multiple stakeholder categories. Specifically, we aim to (1) map the existing landscape of telehealth evaluation, (2) identify key concepts for evaluation, (3) synthesize existing evaluation frameworks, and (4) identify measurements and assessments considered in the United States.

Methods: We will conduct this scoping review in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews and in line with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). This scoping review will consider documents, including reviews, reports, and white papers, published since January 1, 2019. It will focus on evaluation frameworks and associated measurements of telehealth services and programs in the US health care system, developed by telehealth stakeholders, professional organizations, and authoritative sources, excluding those developed by individual researchers, to collect data that reflect the collective expertise and consensus of experts within the respective professional group.

Results: The data extracted from selected documents will be synthesized using tools such as tables and figures. Visual aids like Venn diagrams will be used to illustrate the relationships between the evaluation frameworks from various sources. A narrative summary will be crafted to further describe how the results align with the review objectives, facilitating a comprehensive overview of the findings. This scoping review is expected to conclude by August 2024.

Conclusions: By addressing critical gaps in telehealth evaluation, this scoping review protocol lays the foundation for a comprehensive and multistakeholder assessment of telehealth services and programs. Its findings will inform policy makers, health care providers, researchers, and other stakeholders in advancing the quality, effectiveness, and cost-effectiveness of telehealth in the US health care system.

Trial Registration: OSF Registries osf.io/aytus; https://osf.io/aytus

International Registered Report Identifier (IRRID): DERR1-10.2196/55209

Introduction

Telehealth has witnessed a remarkable transformation in recent years, propelled by the COVID-19 pandemic, which has greatly accelerated its adoption and expansion. The imperative for social distancing and the need to minimize in-person contact have made telehealth a pivotal tool at the forefront of health care delivery, helping to ensure accessibility and continuity of health care. This transformation has been further facilitated by significant expansions in telehealth service coverage, as well as regulatory requirements for telehealth visits, by the Centers for Medicare and Medicaid Services (CMS) and other payers, enhancing accessibility and affordability for patients across the nation [ 1 - 5 ].

The rapid expansion of telehealth services underscores the necessity for comprehensive guidance and evaluation of telehealth programs to guarantee the quality and effectiveness of health care delivery. The World Health Organization (WHO) has published a series of operational guidelines and evaluation frameworks to facilitate its global strategy on digital health development [ 6 - 9 ].

Recognizing the importance of maintaining service quality in practice, professional groups and telehealth stakeholders across the United States have actively contributed to the development of clinical guidelines. For instance, the American Telemedicine Association (ATA) published practice guidelines to enhance the technical quality and reliability of telemental health services for children and adolescents [ 10 ]. The ATA’s practice guidelines for ocular telehealth-diabetic retinopathy were updated to their third edition in 2020 to incorporate new evidence and technologies [ 11 ]. Furthermore, the American Heart Association (AHA) has advocated using remote patient monitoring technologies to improve cardiovascular disease outcomes [ 12 ], highlighting the ever-expanding role of telehealth in quality health care. The American Nurses Association (ANA) updated its core principles on telehealth in 2019 to provide guidance for health care professionals in delivering quality care using health technologies [ 13 ]. Similarly, the National Association of Social Workers (NASW) published guidance on legal considerations for telemental health, promoting adherence to state and federal practice guidelines and payer contract agreements among social workers [ 14 ].

Concurrently, in response to the rapidly evolving landscape of telehealth services, researchers, professional groups, and organizations have crafted telehealth evaluation frameworks. These frameworks have been designed to guide and facilitate the assessment of specific dimensions of telehealth programs. For example, Zhang et al [ 15 ] designed a framework to guide the development and evaluation of sustainable telehealth programs. Curfman et al [ 16 ] developed an economic framework focusing on measuring the value of pediatric telehealth. Moreover, a consortium of experts from the Kaiser Permanente (KP) Institute for Health Policy, AcademyHealth, the ATA, and the Physician Insurers Association of America (PIAA) collaborated on a telehealth research and policy framework, facilitating the assessment of health services and the quality of health care [ 17 ]. During the COVID-19 pandemic, the National Quality Forum (NQF) updated its telehealth measurement development framework, initially created in 2017, through an environmental scan conducted in 2021 [ 18 - 20 ].

Despite the wealth of telehealth evaluation frameworks available, several critical questions remain unanswered. First, the rapidly evolving state of telehealth programs, along with the emergence of innovative telehealth tools, has underscored the pressing need for a comprehensive grasp of the essential concepts that should be considered in their evaluation. For instance, the growing use of artificial intelligence in clinical assessments and the application of virtual reality among pediatric patients with autism spectrum disorder have garnered broad attention [ 21 , 22 ]. It is imperative to acquire a broader evaluation framework that accommodates these emerging technologies and provides a comprehensive understanding of integrating state-of-the-art technologies on telehealth platforms. As such, a scoping review that systematically identifies what telehealth services and programs are evaluated and how they are assessed is needed.

Additionally, while numerous frameworks have been proposed, many of them have been developed primarily from a narrow perspective or to address a specific need, potentially limiting their broader applicability and the comprehensiveness of the evaluation. For example, the framework developed by Zhang et al [ 15 ] focused on the sustainability of single telehealth programs, encompassing domains of program implementation, clinical effectiveness, and economic analysis. In contrast, the framework developed by KP, AcademyHealth, the ATA, and the PIAA considered five domains, including (1) policy context, (2) payment policy, (3) delivery, (4) modality, and (5) outcomes [ 17 ]. In the case of the NQF telehealth measurement development framework, it emphasizes assessing the impact of telehealth on health care system readiness and health outcomes in rural areas, spanning across another five domains: (1) access to care and technology; (2) costs, business model, and logistics; (3) experience; (4) effectiveness; and (5) equity [ 15 , 18 , 19 ]. Given the diverse array of telehealth stakeholders in the United States, including patients, providers, hospitals, payers, professional associations, federal agencies, policy makers, and legislators, it is necessary to consider the perspectives of multiple stakeholders and comprehensively evaluate telehealth services and programs.

Moreover, the measurements and assessments associated with telehealth evaluation domains and frameworks remain unclear, raising questions about how to effectively gauge the impact and outcomes of telehealth services and programs. While the frameworks developed by the WHO, Zhang et al [ 15 ], and NQF outline the measurements to be considered, this clarity is absent in the frameworks developed by KP, AcademyHealth, the ATA, and the PIAA [ 6 - 9 , 17 , 18 ].

Objective and Review Question

The broad objective of this scoping review is to answer the question of what is known about telehealth services and program evaluation in the United States. Specifically, this scoping review will be conducted to (1) map the existing landscape of telehealth evaluation, (2) identify key concepts for evaluation, (3) synthesize existing frameworks, and (4) identify measurements and assessments that have been considered.

A preliminary search of PROSPERO, MEDLINE, the Cochrane Database of Systematic Reviews, the Open Science Framework, and JBI Evidence Synthesis was conducted, and no current or in-progress scoping reviews or systematic reviews on the topic were identified.

Study Design

The scoping review will be conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews and in line with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) [ 23 - 25 ].

Protocol and Registration

The protocol for this scoping review is registered on the Open Science Framework (osf.io/aytus) [ 26 ]. The scoping review is expected to be completed in 6 months.

Eligibility Criteria

The eligibility for this scoping review is elaborated following the Population, Concept, and Context (PCC) framework.

Participants

This scoping review will consider a range of document types, including reviews, reports, and white papers, specifically related to telehealth evaluation frameworks and associated measurements. The evaluation frameworks to be included will focus on telehealth services and programs used for the provision of health services through well-established modalities, such as store-and-forward telemedicine, remote monitoring, real-time counseling, audio and video conferencing, and videotelephony, as well as emerging innovations integrated with telehealth platforms, such as virtual realities and artificial intelligence [ 27 ]. We will exclude manuscripts with data analyses only, case studies, project intervention reports (including scale-up and scale-down studies), and commentaries.

This review will examine concepts pertaining to the evaluation frameworks of telehealth services and programs. The concepts to be examined will encompass existing evaluation frameworks and associated measurements developed by telehealth stakeholders, professional organizations, and authoritative sources, excluding those developed by individual researchers. This approach will allow us to collect data that reflect the collective expertise and consensus of experts within the respective professional group. The aim is to create a comprehensive overview of telehealth evaluation, incorporating perspectives from multiple stakeholder categories, including but not limited to public and private payers, providers, and policy makers.

This review will consider documents published in English for various health care settings, for example, primary care, specialty care, and rural health care for adult and pediatric patients. Considering the diversity of the contexts of telehealth services and programs across different health care systems and the unique nature of the US health care system, publications reporting on evaluation frameworks developed for regions that do not include the United States will be excluded. Documents published by worldwide health organizations will be included, given their relevance and influence in US health care.

Search Strategy

As many evaluation frameworks are likely to be published on stakeholders’ websites, we will source data from peer-reviewed journals and gray literature, such as reports, white papers, policy documents, and guidelines.

The search strategy will aim to locate published reviews, reports, and white papers. An initial PubMed search was undertaken to identify articles on the topic. The text words contained in the titles and abstracts of relevant articles and the index terms used to describe the articles were used to develop a full search strategy for PubMed ( Multimedia Appendix 1 ). The search strategy, including all identified keywords and index terms, will be adapted for each included database and information source. The reference lists of all included studies will be screened for additional titles. The databases to be searched include PubMed (US National Library of Medicine), Health Technology Assessments (International Network of Agencies for Health Technology Assessment), and Web of Science Core Collection (Clarivate Analytics). The websites of telehealth stakeholders, professional organizations, and authoritative sources mentioned in the included articles will be screened for additional documents.

Documents published from January 1, 2019, to the present will be considered for inclusion. While we recognize the relevance of earlier publications, the rapid expansion of telehealth, particularly accelerated by the COVID-19 pandemic, has led to significant changes in telehealth services and programs. During and after the pandemic, not only did the volume of telehealth services increase, but there was also a diversification in the types of modalities considered under the telehealth umbrella. For example, before the pandemic, audio-only interactions were not widely regarded as telehealth, and various modalities like e-consults and e-visits were not as commonly used as they are now. Therefore, the inclusion of documents published from January 1, 2019 onward will allow us to focus on the most recent and relevant telehealth service and program evaluations.

Source of Evidence Selection

Following the search, all identified records will be collated and uploaded into EndNote (V21; Clarivate Analytics), with duplicates removed. A total of 2 reviewers will independently screen titles and abstracts against the inclusion criteria. Following a pilot test, titles and abstracts will then be screened by 2 independent reviewers for assessment against the inclusion criteria for the review. All screening will be completed through Rayyan, a web-based tool for evidence synthesis projects [ 28 ]. The full text of selected citations will be assessed in detail against the inclusion criteria by 2 or more independent reviewers. Reasons for the exclusion of full-text documents that do not meet the inclusion criteria will be recorded and reported in the scoping review. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion. The results of the search will be reported in full in the final scoping review and presented in a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram [ 29 ].

Data Extraction

Data will be extracted from documents selected for inclusion in the scoping review by 2 independent reviewers using a data extraction tool developed by the reviewers based on the JBI data extraction template for scoping review [ 24 ]. The data extracted will include specific details about the PCC methods and key findings relevant to the review question. A draft extraction tool is provided in Multimedia Appendix 2 . The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included document. Modifications will be detailed in the full scoping review. Any disagreements that arise between the reviewers will be resolved through discussion.

Reviewers will synthesize data across selected documents using tools such as tables and figures. Frequency counts of domains and measurements considered in frameworks and guidelines will be presented when applicable to highlight key patterns. Visual aids like Venn diagrams will be used to illustrate the relationships between the evaluation frameworks from various sources. Additionally, a narrative summary will be crafted to further describe how the results align with the review objectives, facilitating a comprehensive overview of the findings. This scoping review is expected to conclude by August 2024.

Implications

In light of the rapid expansion of telehealth services, this scoping review seeks to address critical gaps in the current understanding of telehealth evaluation. The urgency of this endeavor is underscored by the need to ensure the quality, effectiveness, and cost-effectiveness of telehealth programs, with their increasing significance to the US health care system.

This scoping review will be the first to synthesize existing evidence of telehealth services and program evaluation in the United States and to facilitate the future development of telehealth in the postpandemic era. It will provide insights into the evaluation of state-of-the-art technology integration into telehealth.

In addition, the synthesized evaluation concepts and frameworks crafted by multiple telehealth stakeholders will guide the development of a multistakeholder evaluation framework that allows the comprehensive assessment of telehealth services and programs. Specifically, a multistakeholder framework will offer an inclusive and adaptable approach to telehealth evaluation, making it relevant and valuable to a wide range of users. By accommodating the diversity of telehealth initiatives, technologies, and objectives that exist within the US health care system, it will provide flexibility for stakeholders to tailor their evaluations for specific needs and objectives. In the evolving landscape of telehealth, where innovations and changes occur regularly, new technologies, practices, and policies would also be considered in the multistakeholder framework. The consideration of measurements and assessments will further illuminate the path toward actionable telehealth evaluation by providing more context and details.

Limitations

While we aim to conduct a comprehensive scoping review, some limitations should be considered. First, to ensure the comprehensiveness and relevance of this study’s findings, we only consider frameworks developed for regions that encompass the United States. For instance, frameworks from organizations like the WHO are considered if they pertain to the US context. However, given the unique and complex nature of the US health care system, the results from this scoping review might not be directly applicable to other countries. In addition, given that the scoping review aims to provide a comprehensive overview of telehealth evaluation from a multistakeholder perspective, the scoping review will exclude documents carried out by individual researchers. Therefore, when evaluating local telehealth programs, it is advisable to include specific local considerations in addition to the results of this scoping review.

Conclusions

This scoping review will serve as a critical initial step in advancing the understanding of telehealth evaluation, given the current role of telehealth in the US health care system. Its findings will inform policy makers, health care providers, researchers, and other stakeholders involved in the rapidly evolving field of telehealth. By addressing these fundamental questions, this protocol lays the foundation for the scoping review for comprehensive telehealth evaluation and future telehealth advancement.

Acknowledgments

This work is supported by the Office for the Advancement of Telehealth, Health Resources and Services Administration, US Department of Health and Human Services, under a cooperative agreement award (2 U66RH31459‐04‐00). The information, conclusions, and opinions expressed are those of the authors, and no endorsement is intended or should be inferred.

Authors' Contributions

YZ, YYL, LSL, JCR, EGH, and JMS undertook the conceptualization and methodology of this study. YZ drafted the initial version of the manuscript, with all authors contributing to the investigation and subsequent review and editing. YZ and EGH managed the resources for the study, while YZ and SC handled project administration. SC secured the funding for the project. SC and JMS provided additional supervision.

Conflicts of Interest

LSL is an employee of ConcertAI. All other authors have no conflicts of interest to declare.

Search strategy.

Data extraction instrument.

  • Mehrotra A, Jena AB, Busch AB, Souza J, Uscher-Pines L, Landon BE. Utilization of telemedicine among rural medicare beneficiaries. JAMA. 2016;315(18):2015-2016. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Thomas KC, Ellis AR, Konrad TR, Holzer CE, Morrissey JP. County-level estimates of mental health professional shortage in the United States. Psychiatr Serv. 2009;60(10):1323-1328. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the United States: results from the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):629-640. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • McClellan M, Rajkumar R, Couch M, Holder D, Pham M, Long P, et al. Health care payers COVID-19 impact assessment: lessons learned and compelling needs. NAM Perspect. 2021;2021:10.31478/202105a. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Arora P, Mehta D, Ha J. Impact of telehealth on health care resource utilization during the COVID-19 pandemic. J Comp Eff Res. 2022;11(5):301-309. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Defining Evaluation Indicators for Telemedicine as a Tool for Reducing Health Inequities: Study and Results of a Community of Practice. Washinton, DC. World Health Organization; 2016.
  • Digital Implementation Investment Guide (DIIG): Integrating Digital Interventions into Health Programmes. Geneva, Switzerland. World Health Organization; 2020.
  • Global health strategy on digital health. World Health Organization. 2022. URL: https:/​/cdn.​who.int/​media/​docs/​default-source/​digital-health-documents/​en_who_digitalhealth_summary-(1).​pdf?sfvrsn=e491ac50_5 [accessed 2024-02-28]
  • WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening: Web Supplement 2: Summary of Findings and GRADE Tables. Geneva, Switzerland. World Health Organization; 2019.
  • Myers K, Nelson EL, Rabinowitz T, Hilty D, Baker D, Barnwell SS, et al. American telemedicine association practice guidelines for telemental health with children and adolescents. Telemed J E Health. 2017;23(10):779-804. [ CrossRef ] [ Medline ]
  • Horton MB, Brady CJ, Cavallerano J, Abramoff M, Barker G, Chiang MF, et al. Practice guidelines for ocular telehealth-diabetic retinopathy, third edition. Telemed J E Health. 2020;26(4):495-543. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Using remote patient monitoring technologies for better cardiovascular disease outcomes guidance. American Heart Association. 2019. URL: https:/​/www.​heart.org/​-/​media/​files/​about-us/​policy-research/​policy-positions/​clinical-care/​remote-patient-monitoring-guidance-2019.​pdf?la=en [accessed 2024-02-28]
  • ANA core principles on connected health. American Nurses Association. 2019. URL: https:/​/www.​nursingworld.org/​~4a9307/​globalassets/​docs/​ana/​practice/​ana-core-principles-on-connected-health.​pdf [accessed 2024-02-28]
  • Felton E. Telemental health. National Association of Social Workers. 2020. URL: https://www.socialworkers.org/LinkClick.aspx?fileticket=xmfNHyy8i8M%3D&portalid=0 [accessed 2024-02-28]
  • Zhang Y, Chandra S, Peña MT, Lal L, Summers RL, Swint JM. Framework for evaluating and developing sustainable telehealth programs. Telemed J E Health. 2023;29(9):1421-1425. [ CrossRef ] [ Medline ]
  • Curfman A, McSwain SD, Chuo J, Olson CA, Simpson K. An economic framework to measure value of pediatric telehealth. Telemed J E Health. 2021;27(12):1440-1442. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Edmunds M, Tuckson R, Lewis J, Atchinson B, Rheuban K, Fanberg H, et al. An emergent research and policy framework for telehealth. eGEMS. 2017;5(2):1. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Rural telehealth and healthcare system readiness measurement framework—final report. National Quality Forum. 2021. URL: https:/​/www.​qualityforum.org/​Publications/​2021/​11/​Rural_Telehealth_and_Healthcare_System_Readiness_Measurement_Framework_-_Final_Report.​aspx [accessed 2024-02-28]
  • Leveraging quality measurement to improve rural health, telehealth, and healthcare system readiness—final environmental scan report. National Quality Forum. 2021. URL: https://tinyurl.com/4nubb2sw [accessed 2024-02-24]
  • Creating a framework to support measure development for telehealth. National Quality Forum. 2017. URL: https:/​/www.​qualityforum.org/​Publications/​2017/​08/​Creating_a_Framework_to_Support_Measure_Development_for_Telehealth.​aspx [accessed 2024-02-28]
  • Tsoi KKF. Application of artificial intelligence on a symptom diagnostic platform for telemedicine a pilot case study. IEEE; 2019. Presented at: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC); October 6-9, 2019; Bari, Italy. [ CrossRef ]
  • Elkin TD, Zhang Y, Reneker JC. Gaze fixation and visual searching behaviors during an immersive virtual reality social skills training experience for children and youth with autism spectrum disorder: a pilot study. Brain Sci. 2022;12(11):1568. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Implement. 2021;19(1):3-10. [ CrossRef ] [ Medline ]
  • Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: Scoping reviews. In: JBI Manual for Evidence Synthesis. Adelaide, Australia. Joanna Briggs Institute; 2020.
  • Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-473. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Zhang Y, Lin YY, Lal LS, Reneker JC, Hinton EG, Chandra S, et al. Framework for telehealth evaluation: a scoping review protocol. OSF. 2023. URL: https://doi.org/10.17605/OSF.IO/AYTUS [accessed 2024-02-28]
  • Eswaran H, Dawson L. Telehealth: current definitions and future trends. The Rural Telehealth Evaluation Center (RTEC). 2022. URL: https:/​/idhi.​uams.edu/​rtec/​wp-content/​uploads/​sites/​4/​2022/​05/​Telehealth-Definitions-Paper-06MAY2022-1.​pdf [accessed 2024-02-28]
  • Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210. [ FREE Full text ] [ CrossRef ] [ Medline ]
  • Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int Surg J. 2021;88:105906. [ FREE Full text ] [ CrossRef ]

Abbreviations

Edited by A Mavragani; submitted 05.12.23; peer-reviewed by M Pena, K Drude; comments to author 01.02.24; revised version received 06.02.24; accepted 07.02.24; published 28.03.24.

©Yunxi Zhang, Yueh-Yun Lin, Lincy S Lal, Jennifer C Reneker, Elizabeth G Hinton, Saurabh Chandra, J Michael Swint. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 28.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.

COMMENTS

  1. What is a review article?

    A review article can also be called a literature review, or a review of literature. It is a survey of previously published research on a topic. It should give an overview of current thinking on the topic. And, unlike an original research article, it will not present new experimental results. Writing a review of literature is to provide a ...

  2. How to write a review article?

    The fundamental rationale of writing a review article is to make a readable synthesis of the best literature sources on an important research inquiry or a topic. This simple definition of a review article contains the following key elements: The question (s) to be dealt with.

  3. Review articles: purpose, process, and structure

    Many research disciplines feature high-impact journals that are dedicated outlets for review papers (or review-conceptual combinations) (e.g., Academy of Management Review, Psychology Bulletin, Medicinal Research Reviews).The rationale for such outlets is the premise that research integration and synthesis provides an important, and possibly even a required, step in the scientific process.

  4. Writing a Scientific Review Article: Comprehensive Insights for

    This makes review articles distinctly different from scientific research papers. While the primary aim of the latter is to develop new arguments by reporting original research, the former is focused on summarising and synthesising previous ideas, studies, and arguments, without adding new experimental contributions.

  5. How to write a good scientific review article

    With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up-to-date with developments in a particular area of research. A good review article provides readers with an in-depth understanding of a field and ...

  6. How to review a paper

    How to review a paper. A good peer review requires disciplinary expertise, a keen and critical eye, and a diplomatic and constructive approach. Credit: dmark/iStockphoto. As junior scientists develop their expertise and make names for themselves, they are increasingly likely to receive invitations to review research manuscripts.

  7. How to write a good scientific review article

    With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up to date with developments in a particular area of research. A good review article provides readers with an in-depth understanding of a field and ...

  8. A Step-by-Step Guide to Writing a Scientific Review Article

    Scientific review articles provide a focused and comprehensive review of the available evidence about a subject, explain the current state of knowledge, and identify gaps that could be topics for potential future research. ... One article on the topic of scientific reviews suggests that at least 15 to 20 relevant research papers published ...

  9. Basics of Writing Review Articles

    The title makes the first introductory and is the most important sentence of the review paper. Like research paper titles, it must be brief, informative, and interesting all at the same time. It must contain the key words or their derivatives to increase the discoverability of the article via search engines. In addition, the type of the review ...

  10. How to write a superb literature review

    One of my favourite review-style articles 3 presents a plot bringing together data from multiple research papers (many of which directly contradict each other). This is then used to identify broad ...

  11. How to Write an Article Review (with Sample Reviews)

    Identify the article. Start your review by referring to the title and author of the article, the title of the journal, and the year of publication in the first paragraph. For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest. 4.

  12. Writing a good review article

    A review article is a comprehensive summary of the current understanding of a specific research topic and is based on previously published research. Unlike research papers, it does not contain new results, but can propose new inferences based on the combined findings of previous research. Types of review articles. Review articles are typically ...

  13. Writing an impactful review article: What do we know and what do we

    Overall, successful review articles identify research gaps and set future research agenda. 1. Introduction and scope of literature reviews. Subject areas advance when studies are synthesized and research gaps are identified ( Kumar, Paul, & Unnithan, 2020 ). In this context, systematic literature reviews allow researchers to identify gaps in ...

  14. How to write a review paper

    Include this information when writing up the method for your review. 5 Look for previous reviews on the topic. Use them as a springboard for your own review, critiquing the earlier reviews, adding more recently published material, and pos-sibly exploring a different perspective. Exploit their refer-ences as another entry point into the literature.

  15. How to Write an Article Review: Tips and Examples

    Just like other types of reviews, a journal article review assesses the merits and shortcomings of a published work. To illustrate, consider a review of an academic paper on climate change, where the writer meticulously analyzes and interprets the article's significance within the context of environmental science. Research Article Review

  16. (PDF) Writing a review article in 7 steps

    Read at least five highquality chapters on a similar topic to make yours better. STEP 2. Gather and read about 50 -100 original articles on a topic within your scientific field. STEP 3. Write down ...

  17. 5 Differences between a research paper and a review paper

    Scholarly literature can be of different types; some of which require that researchers conduct an original study, whereas others can be based on existing research. One of the most popular Q&As led us to conclude that of all the types of scholarly literature, researchers are most confused by the differences between a research paper and a review paper. This infographic explains the five main ...

  18. How to Write an Article Review

    This task involves reviewing a bunch of research papers and summarizing the existing knowledge about a certain subject. A systematic paper type uses an organized approach and expects you to answer questions linked to the research. Tips for Writing a Great Article Review. Here are some expert tips you could use to write an exceptional article ...

  19. How to Write a Literature Review

    What is the purpose of a literature review? Examples of literature reviews. Step 1 - Search for relevant literature. Step 2 - Evaluate and select sources. Step 3 - Identify themes, debates, and gaps. Step 4 - Outline your literature review's structure. Step 5 - Write your literature review.

  20. Writing a Literature Review

    A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research ...

  21. Step by Step Guide to Reviewing a Manuscript

    Briefly summarize what the paper is about and what the findings are. Try to put the findings of the paper into the context of the existing literature and current knowledge. Indicate the significance of the work and if it is novel or mainly confirmatory. Indicate the work's strengths, its quality and completeness.

  22. 110553 PDFs

    Explore the latest full-text research PDFs, articles, conference papers, preprints and more on RESEARCH PAPERS. Find methods information, sources, references or conduct a literature review on ...

  23. Expert review of the science underlying nature-based climate solutions

    Four pathways with the highest current carbon market activity and high mitigation potential (tropical and temperate forest conservation and reforestation; Table 1 and Supplementary Data), were ...

  24. Tools for the Diagnosis of ADHD in Children and Adolescents: A

    The review aims were developed in consultation with the Agency for Healthcare Research and Quality (AHRQ), the Patient-Centered Outcomes Research Institute, the topic nominator American Academy of Pediatrics, key informants, a technical expert panel (TEP), and public input. The TEP reviewed the protocol and advised on key outcomes.

  25. Writing, reading, and critiquing reviews

    Scoping Review: Aims to quickly map a research area, documenting key concepts, sources of evidence, methodologies used. Typically, scoping reviews do not judge the quality of the papers included in the review. They tend to produce descriptive accounts of a topic area. Kalun P, Dunn K, Wagner N, Pulakunta T, Sonnadara R.

  26. Ten Simple Rules for Writing a Literature Review

    Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used.

  27. [2403.18173] LLMs in HCI Data Work: Bridging the Gap Between

    Efficient and accurate information extraction from scientific papers is significant in the rapidly developing human-computer interaction research in the literature review process. Our paper introduces and analyses a new information retrieval system using state-of-the-art Large Language Models (LLMs) in combination with structured text analysis techniques to extract experimental data from HCI ...

  28. HIV/AIDS

    A patient with a 33-year history of HIV-1 infection who underwent stem-cell transplantation for the treatment of acute myelogenous leukemia remained HIV-1 free nearly 5 years later.

  29. Recent research advances in pain mechanisms in McCune-Albright syndrome

    The lack of effective understanding of the pain mechanism of McCune-Albright syndrome (MAS) has made the treatment of pain in this disease a difficult clinical challenge, and new therapeutic targets are urgently needed to address this dilemma. This paper summarizes the novel mechanisms, targets, and treatments that may produce pain in MAS and fibrous dysplasia (polyfibrous dysplasia, or FD).

  30. JMIR Research Protocols

    This scoping review will consider documents, including reviews, reports, and white papers, published since January 1, 2019. It will focus on evaluation frameworks and associated measurements of telehealth services and programs in the US health care system, developed by telehealth stakeholders, professional organizations, and authoritative ...