peer reviewed article research methods

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on December 17, 2021 by Tegan George . Revised on June 22, 2023.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, other interesting articles, frequently asked questions about peer reviews.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymized) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymized comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymized) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymized) review —where the identities of the author, reviewers, and editors are all anonymized—does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimizes potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymize everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimize back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarize the argument in your own words

Summarizing the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organized. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

Tip: Try not to focus too much on the minor issues. If the manuscript has a lot of typos, consider making a note that the author should address spelling and grammar issues, rather than going through and fixing each one.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticized, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the “compliment sandwich,” where you “sandwich” your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Prevent plagiarism. Run a free check.

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarized or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published. There is also high risk of publication bias , where journals are more likely to publish studies with positive findings than studies with negative findings.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Thematic analysis
• Discourse analysis
• Cohort study
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Peer review is a process of evaluating submissions to an academic journal. Utilizing rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication. For this reason, academic journals are often considered among the most credible sources you can use in a research project– provided that the journal itself is trustworthy and well-regarded.

In general, the peer review process follows the following steps: 

• Reject the manuscript and send it back to author, or 
• Send it onward to the selected peer reviewer(s) 
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made. 
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field. It acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure. 

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is Peer Review? | Types & Examples. Scribbr. Retrieved June 24, 2024, from https://www.scribbr.com/methodology/peer-review/

Is this article helpful?

Tegan George

Other students also liked, what are credible sources & how to spot them | examples, ethical considerations in research | types & examples, applying the craap test & evaluating sources, what is your plagiarism score.

REVIEW article

The use of research methods in psychological research: a systematised review.

• 1 Community Psychosocial Research (COMPRES), School of Psychosocial Health, North-West University, Potchefstroom, South Africa
• 2 WorkWell Research Institute, North-West University, Potchefstroom, South Africa

Research methods play an imperative role in research quality as well as educating young researchers, however, the application thereof is unclear which can be detrimental to the field of psychology. Therefore, this systematised review aimed to determine what research methods are being used, how these methods are being used and for what topics in the field. Our review of 999 articles from five journals over a period of 5 years indicated that psychology research is conducted in 10 topics via predominantly quantitative research methods. Of these 10 topics, social psychology was the most popular. The remainder of the conducted methodology is described. It was also found that articles lacked rigour and transparency in the used methodology which has implications for replicability. In conclusion this article, provides an overview of all reported methodologies used in a sample of psychology journals. It highlights the popularity and application of methods and designs throughout the article sample as well as an unexpected lack of rigour with regard to most aspects of methodology. Possible sample bias should be considered when interpreting the results of this study. It is recommended that future research should utilise the results of this study to determine the possible impact on the field of psychology as a science and to further investigation into the use of research methods. Results should prompt the following future research into: a lack or rigour and its implication on replication, the use of certain methods above others, publication bias and choice of sampling method.

Introduction

Psychology is an ever-growing and popular field ( Gough and Lyons, 2016 ; Clay, 2017 ). Due to this growth and the need for science-based research to base health decisions on ( Perestelo-Pérez, 2013 ), the use of research methods in the broad field of psychology is an essential point of investigation ( Stangor, 2011 ; Aanstoos, 2014 ). Research methods are therefore viewed as important tools used by researchers to collect data ( Nieuwenhuis, 2016 ) and include the following: quantitative, qualitative, mixed method and multi method ( Maree, 2016 ). Additionally, researchers also employ various types of literature reviews to address research questions ( Grant and Booth, 2009 ). According to literature, what research method is used and why a certain research method is used is complex as it depends on various factors that may include paradigm ( O'Neil and Koekemoer, 2016 ), research question ( Grix, 2002 ), or the skill and exposure of the researcher ( Nind et al., 2015 ). How these research methods are employed is also difficult to discern as research methods are often depicted as having fixed boundaries that are continuously crossed in research ( Johnson et al., 2001 ; Sandelowski, 2011 ). Examples of this crossing include adding quantitative aspects to qualitative studies ( Sandelowski et al., 2009 ), or stating that a study used a mixed-method design without the study having any characteristics of this design ( Truscott et al., 2010 ).

The inappropriate use of research methods affects how students and researchers improve and utilise their research skills ( Scott Jones and Goldring, 2015 ), how theories are developed ( Ngulube, 2013 ), and the credibility of research results ( Levitt et al., 2017 ). This, in turn, can be detrimental to the field ( Nind et al., 2015 ), journal publication ( Ketchen et al., 2008 ; Ezeh et al., 2010 ), and attempts to address public social issues through psychological research ( Dweck, 2017 ). This is especially important given the now well-known replication crisis the field is facing ( Earp and Trafimow, 2015 ; Hengartner, 2018 ).

Due to this lack of clarity on method use and the potential impact of inept use of research methods, the aim of this study was to explore the use of research methods in the field of psychology through a review of journal publications. Chaichanasakul et al. (2011) identify reviewing articles as the opportunity to examine the development, growth and progress of a research area and overall quality of a journal. Studies such as Lee et al. (1999) as well as Bluhm et al. (2011) review of qualitative methods has attempted to synthesis the use of research methods and indicated the growth of qualitative research in American and European journals. Research has also focused on the use of research methods in specific sub-disciplines of psychology, for example, in the field of Industrial and Organisational psychology Coetzee and Van Zyl (2014) found that South African publications tend to consist of cross-sectional quantitative research methods with underrepresented longitudinal studies. Qualitative studies were found to make up 21% of the articles published from 1995 to 2015 in a similar study by O'Neil and Koekemoer (2016) . Other methods in health psychology, such as Mixed methods research have also been reportedly growing in popularity ( O'Cathain, 2009 ).

A broad overview of the use of research methods in the field of psychology as a whole is however, not available in the literature. Therefore, our research focused on answering what research methods are being used, how these methods are being used and for what topics in practice (i.e., journal publications) in order to provide a general perspective of method used in psychology publication. We synthesised the collected data into the following format: research topic [areas of scientific discourse in a field or the current needs of a population ( Bittermann and Fischer, 2018 )], method [data-gathering tools ( Nieuwenhuis, 2016 )], sampling [elements chosen from a population to partake in research ( Ritchie et al., 2009 )], data collection [techniques and research strategy ( Maree, 2016 )], and data analysis [discovering information by examining bodies of data ( Ktepi, 2016 )]. A systematised review of recent articles (2013 to 2017) collected from five different journals in the field of psychological research was conducted.

Grant and Booth (2009) describe systematised reviews as the review of choice for post-graduate studies, which is employed using some elements of a systematic review and seldom more than one or two databases to catalogue studies after a comprehensive literature search. The aspects used in this systematised review that are similar to that of a systematic review were a full search within the chosen database and data produced in tabular form ( Grant and Booth, 2009 ).

Sample sizes and timelines vary in systematised reviews (see Lowe and Moore, 2014 ; Pericall and Taylor, 2014 ; Barr-Walker, 2017 ). With no clear parameters identified in the literature (see Grant and Booth, 2009 ), the sample size of this study was determined by the purpose of the sample ( Strydom, 2011 ), and time and cost constraints ( Maree and Pietersen, 2016 ). Thus, a non-probability purposive sample ( Ritchie et al., 2009 ) of the top five psychology journals from 2013 to 2017 was included in this research study. Per Lee (2015) American Psychological Association (APA) recommends the use of the most up-to-date sources for data collection with consideration of the context of the research study. As this research study focused on the most recent trends in research methods used in the broad field of psychology, the identified time frame was deemed appropriate.

Psychology journals were only included if they formed part of the top five English journals in the miscellaneous psychology domain of the Scimago Journal and Country Rank ( Scimago Journal & Country Rank, 2017 ). The Scimago Journal and Country Rank provides a yearly updated list of publicly accessible journal and country-specific indicators derived from the Scopus ® database ( Scopus, 2017b ) by means of the Scimago Journal Rank (SJR) indicator developed by Scimago from the algorithm Google PageRank™ ( Scimago Journal & Country Rank, 2017 ). Scopus is the largest global database of abstracts and citations from peer-reviewed journals ( Scopus, 2017a ). Reasons for the development of the Scimago Journal and Country Rank list was to allow researchers to assess scientific domains, compare country rankings, and compare and analyse journals ( Scimago Journal & Country Rank, 2017 ), which supported the aim of this research study. Additionally, the goals of the journals had to focus on topics in psychology in general with no preference to specific research methods and have full-text access to articles.

The following list of top five journals in 2018 fell within the abovementioned inclusion criteria (1) Australian Journal of Psychology, (2) British Journal of Psychology, (3) Europe's Journal of Psychology, (4) International Journal of Psychology and lastly the (5) Journal of Psychology Applied and Interdisciplinary.

Journals were excluded from this systematised review if no full-text versions of their articles were available, if journals explicitly stated a publication preference for certain research methods, or if the journal only published articles in a specific discipline of psychological research (for example, industrial psychology, clinical psychology etc.).

The researchers followed a procedure (see Figure 1 ) adapted from that of Ferreira et al. (2016) for systematised reviews. Data collection and categorisation commenced on 4 December 2017 and continued until 30 June 2019. All the data was systematically collected and coded manually ( Grant and Booth, 2009 ) with an independent person acting as co-coder. Codes of interest included the research topic, method used, the design used, sampling method, and methodology (the method used for data collection and data analysis). These codes were derived from the wording in each article. Themes were created based on the derived codes and checked by the co-coder. Lastly, these themes were catalogued into a table as per the systematised review design.

Figure 1 . Systematised review procedure.

According to Johnston et al. (2019) , “literature screening, selection, and data extraction/analyses” (p. 7) are specifically tailored to the aim of a review. Therefore, the steps followed in a systematic review must be reported in a comprehensive and transparent manner. The chosen systematised design adhered to the rigour expected from systematic reviews with regard to full search and data produced in tabular form ( Grant and Booth, 2009 ). The rigorous application of the systematic review is, therefore discussed in relation to these two elements.

Firstly, to ensure a comprehensive search, this research study promoted review transparency by following a clear protocol outlined according to each review stage before collecting data ( Johnston et al., 2019 ). This protocol was similar to that of Ferreira et al. (2016) and approved by three research committees/stakeholders and the researchers ( Johnston et al., 2019 ). The eligibility criteria for article inclusion was based on the research question and clearly stated, and the process of inclusion was recorded on an electronic spreadsheet to create an evidence trail ( Bandara et al., 2015 ; Johnston et al., 2019 ). Microsoft Excel spreadsheets are a popular tool for review studies and can increase the rigour of the review process ( Bandara et al., 2015 ). Screening for appropriate articles for inclusion forms an integral part of a systematic review process ( Johnston et al., 2019 ). This step was applied to two aspects of this research study: the choice of eligible journals and articles to be included. Suitable journals were selected by the first author and reviewed by the second and third authors. Initially, all articles from the chosen journals were included. Then, by process of elimination, those irrelevant to the research aim, i.e., interview articles or discussions etc., were excluded.

To ensure rigourous data extraction, data was first extracted by one reviewer, and an independent person verified the results for completeness and accuracy ( Johnston et al., 2019 ). The research question served as a guide for efficient, organised data extraction ( Johnston et al., 2019 ). Data was categorised according to the codes of interest, along with article identifiers for audit trails such as authors, title and aims of articles. The categorised data was based on the aim of the review ( Johnston et al., 2019 ) and synthesised in tabular form under methods used, how these methods were used, and for what topics in the field of psychology.

The initial search produced a total of 1,145 articles from the 5 journals identified. Inclusion and exclusion criteria resulted in a final sample of 999 articles ( Figure 2 ). Articles were co-coded into 84 codes, from which 10 themes were derived ( Table 1 ).

Figure 2 . Journal article frequency.

Table 1 . Codes used to form themes (research topics).

These 10 themes represent the topic section of our research question ( Figure 3 ). All these topics except, for the final one, psychological practice , were found to concur with the research areas in psychology as identified by Weiten (2010) . These research areas were chosen to represent the derived codes as they provided broad definitions that allowed for clear, concise categorisation of the vast amount of data. Article codes were categorised under particular themes/topics if they adhered to the research area definitions created by Weiten (2010) . It is important to note that these areas of research do not refer to specific disciplines in psychology, such as industrial psychology; but to broader fields that may encompass sub-interests of these disciplines.

Figure 3 . Topic frequency (international sample).

In the case of developmental psychology , researchers conduct research into human development from childhood to old age. Social psychology includes research on behaviour governed by social drivers. Researchers in the field of educational psychology study how people learn and the best way to teach them. Health psychology aims to determine the effect of psychological factors on physiological health. Physiological psychology , on the other hand, looks at the influence of physiological aspects on behaviour. Experimental psychology is not the only theme that uses experimental research and focuses on the traditional core topics of psychology (for example, sensation). Cognitive psychology studies the higher mental processes. Psychometrics is concerned with measuring capacity or behaviour. Personality research aims to assess and describe consistency in human behaviour ( Weiten, 2010 ). The final theme of psychological practice refers to the experiences, techniques, and interventions employed by practitioners, researchers, and academia in the field of psychology.

Articles under these themes were further subdivided into methodologies: method, sampling, design, data collection, and data analysis. The categorisation was based on information stated in the articles and not inferred by the researchers. Data were compiled into two sets of results presented in this article. The first set addresses the aim of this study from the perspective of the topics identified. The second set of results represents a broad overview of the results from the perspective of the methodology employed. The second set of results are discussed in this article, while the first set is presented in table format. The discussion thus provides a broad overview of methods use in psychology (across all themes), while the table format provides readers with in-depth insight into methods used in the individual themes identified. We believe that presenting the data from both perspectives allow readers a broad understanding of the results. Due a large amount of information that made up our results, we followed Cichocka and Jost (2014) in simplifying our results. Please note that the numbers indicated in the table in terms of methodology differ from the total number of articles. Some articles employed more than one method/sampling technique/design/data collection method/data analysis in their studies.

What follows is the results for what methods are used, how these methods are used, and which topics in psychology they are applied to . Percentages are reported to the second decimal in order to highlight small differences in the occurrence of methodology.

Firstly, with regard to the research methods used, our results show that researchers are more likely to use quantitative research methods (90.22%) compared to all other research methods. Qualitative research was the second most common research method but only made up about 4.79% of the general method usage. Reviews occurred almost as much as qualitative studies (3.91%), as the third most popular method. Mixed-methods research studies (0.98%) occurred across most themes, whereas multi-method research was indicated in only one study and amounted to 0.10% of the methods identified. The specific use of each method in the topics identified is shown in Table 2 and Figure 4 .

Table 2 . Research methods in psychology.

Figure 4 . Research method frequency in topics.

Secondly, in the case of how these research methods are employed , our study indicated the following.

Sampling −78.34% of the studies in the collected articles did not specify a sampling method. From the remainder of the studies, 13 types of sampling methods were identified. These sampling methods included broad categorisation of a sample as, for example, a probability or non-probability sample. General samples of convenience were the methods most likely to be applied (10.34%), followed by random sampling (3.51%), snowball sampling (2.73%), and purposive (1.37%) and cluster sampling (1.27%). The remainder of the sampling methods occurred to a more limited extent (0–1.0%). See Table 3 and Figure 5 for sampling methods employed in each topic.

Table 3 . Sampling use in the field of psychology.

Figure 5 . Sampling method frequency in topics.

Designs were categorised based on the articles' statement thereof. Therefore, it is important to note that, in the case of quantitative studies, non-experimental designs (25.55%) were often indicated due to a lack of experiments and any other indication of design, which, according to Laher (2016) , is a reasonable categorisation. Non-experimental designs should thus be compared with experimental designs only in the description of data, as it could include the use of correlational/cross-sectional designs, which were not overtly stated by the authors. For the remainder of the research methods, “not stated” (7.12%) was assigned to articles without design types indicated.

From the 36 identified designs the most popular designs were cross-sectional (23.17%) and experimental (25.64%), which concurred with the high number of quantitative studies. Longitudinal studies (3.80%), the third most popular design, was used in both quantitative and qualitative studies. Qualitative designs consisted of ethnography (0.38%), interpretative phenomenological designs/phenomenology (0.28%), as well as narrative designs (0.28%). Studies that employed the review method were mostly categorised as “not stated,” with the most often stated review designs being systematic reviews (0.57%). The few mixed method studies employed exploratory, explanatory (0.09%), and concurrent designs (0.19%), with some studies referring to separate designs for the qualitative and quantitative methods. The one study that identified itself as a multi-method study used a longitudinal design. Please see how these designs were employed in each specific topic in Table 4 , Figure 6 .

Table 4 . Design use in the field of psychology.

Figure 6 . Design frequency in topics.

Data collection and analysis —data collection included 30 methods, with the data collection method most often employed being questionnaires (57.84%). The experimental task (16.56%) was the second most preferred collection method, which included established or unique tasks designed by the researchers. Cognitive ability tests (6.84%) were also regularly used along with various forms of interviewing (7.66%). Table 5 and Figure 7 represent data collection use in the various topics. Data analysis consisted of 3,857 occurrences of data analysis categorised into ±188 various data analysis techniques shown in Table 6 and Figures 1 – 7 . Descriptive statistics were the most commonly used (23.49%) along with correlational analysis (17.19%). When using a qualitative method, researchers generally employed thematic analysis (0.52%) or different forms of analysis that led to coding and the creation of themes. Review studies presented few data analysis methods, with most studies categorising their results. Mixed method and multi-method studies followed the analysis methods identified for the qualitative and quantitative studies included.

Table 5 . Data collection in the field of psychology.

Figure 7 . Data collection frequency in topics.

Table 6 . Data analysis in the field of psychology.

Results of the topics researched in psychology can be seen in the tables, as previously stated in this article. It is noteworthy that, of the 10 topics, social psychology accounted for 43.54% of the studies, with cognitive psychology the second most popular research topic at 16.92%. The remainder of the topics only occurred in 4.0–7.0% of the articles considered. A list of the included 999 articles is available under the section “View Articles” on the following website: https://methodgarden.xtrapolate.io/ . This website was created by Scholtz et al. (2019) to visually present a research framework based on this Article's results.

This systematised review categorised full-length articles from five international journals across the span of 5 years to provide insight into the use of research methods in the field of psychology. Results indicated what methods are used how these methods are being used and for what topics (why) in the included sample of articles. The results should be seen as providing insight into method use and by no means a comprehensive representation of the aforementioned aim due to the limited sample. To our knowledge, this is the first research study to address this topic in this manner. Our discussion attempts to promote a productive way forward in terms of the key results for method use in psychology, especially in the field of academia ( Holloway, 2008 ).

With regard to the methods used, our data stayed true to literature, finding only common research methods ( Grant and Booth, 2009 ; Maree, 2016 ) that varied in the degree to which they were employed. Quantitative research was found to be the most popular method, as indicated by literature ( Breen and Darlaston-Jones, 2010 ; Counsell and Harlow, 2017 ) and previous studies in specific areas of psychology (see Coetzee and Van Zyl, 2014 ). Its long history as the first research method ( Leech et al., 2007 ) in the field of psychology as well as researchers' current application of mathematical approaches in their studies ( Toomela, 2010 ) might contribute to its popularity today. Whatever the case may be, our results show that, despite the growth in qualitative research ( Demuth, 2015 ; Smith and McGannon, 2018 ), quantitative research remains the first choice for article publication in these journals. Despite the included journals indicating openness to articles that apply any research methods. This finding may be due to qualitative research still being seen as a new method ( Burman and Whelan, 2011 ) or reviewers' standards being higher for qualitative studies ( Bluhm et al., 2011 ). Future research is encouraged into the possible biasness in publication of research methods, additionally further investigation with a different sample into the proclaimed growth of qualitative research may also provide different results.

Review studies were found to surpass that of multi-method and mixed method studies. To this effect Grant and Booth (2009) , state that the increased awareness, journal contribution calls as well as its efficiency in procuring research funds all promote the popularity of reviews. The low frequency of mixed method studies contradicts the view in literature that it's the third most utilised research method ( Tashakkori and Teddlie's, 2003 ). Its' low occurrence in this sample could be due to opposing views on mixing methods ( Gunasekare, 2015 ) or that authors prefer publishing in mixed method journals, when using this method, or its relative novelty ( Ivankova et al., 2016 ). Despite its low occurrence, the application of the mixed methods design in articles was methodologically clear in all cases which were not the case for the remainder of research methods.

Additionally, a substantial number of studies used a combination of methodologies that are not mixed or multi-method studies. Perceived fixed boundaries are according to literature often set aside, as confirmed by this result, in order to investigate the aim of a study, which could create a new and helpful way of understanding the world ( Gunasekare, 2015 ). According to Toomela (2010) , this is not unheard of and could be considered a form of “structural systemic science,” as in the case of qualitative methodology (observation) applied in quantitative studies (experimental design) for example. Based on this result, further research into this phenomenon as well as its implications for research methods such as multi and mixed methods is recommended.

Discerning how these research methods were applied, presented some difficulty. In the case of sampling, most studies—regardless of method—did mention some form of inclusion and exclusion criteria, but no definite sampling method. This result, along with the fact that samples often consisted of students from the researchers' own academic institutions, can contribute to literature and debates among academics ( Peterson and Merunka, 2014 ; Laher, 2016 ). Samples of convenience and students as participants especially raise questions about the generalisability and applicability of results ( Peterson and Merunka, 2014 ). This is because attention to sampling is important as inappropriate sampling can debilitate the legitimacy of interpretations ( Onwuegbuzie and Collins, 2017 ). Future investigation into the possible implications of this reported popular use of convenience samples for the field of psychology as well as the reason for this use could provide interesting insight, and is encouraged by this study.

Additionally, and this is indicated in Table 6 , articles seldom report the research designs used, which highlights the pressing aspect of the lack of rigour in the included sample. Rigour with regards to the applied empirical method is imperative in promoting psychology as a science ( American Psychological Association, 2020 ). Omitting parts of the research process in publication when it could have been used to inform others' research skills should be questioned, and the influence on the process of replicating results should be considered. Publications are often rejected due to a lack of rigour in the applied method and designs ( Fonseca, 2013 ; Laher, 2016 ), calling for increased clarity and knowledge of method application. Replication is a critical part of any field of scientific research and requires the “complete articulation” of the study methods used ( Drotar, 2010 , p. 804). The lack of thorough description could be explained by the requirements of certain journals to only report on certain aspects of a research process, especially with regard to the applied design (Laher, 20). However, naming aspects such as sampling and designs, is a requirement according to the APA's Journal Article Reporting Standards (JARS-Quant) ( Appelbaum et al., 2018 ). With very little information on how a study was conducted, authors lose a valuable opportunity to enhance research validity, enrich the knowledge of others, and contribute to the growth of psychology and methodology as a whole. In the case of this research study, it also restricted our results to only reported samples and designs, which indicated a preference for certain designs, such as cross-sectional designs for quantitative studies.

Data collection and analysis were for the most part clearly stated. A key result was the versatile use of questionnaires. Researchers would apply a questionnaire in various ways, for example in questionnaire interviews, online surveys, and written questionnaires across most research methods. This may highlight a trend for future research.

With regard to the topics these methods were employed for, our research study found a new field named “psychological practice.” This result may show the growing consciousness of researchers as part of the research process ( Denzin and Lincoln, 2003 ), psychological practice, and knowledge generation. The most popular of these topics was social psychology, which is generously covered in journals and by learning societies, as testaments of the institutional support and richness social psychology has in the field of psychology ( Chryssochoou, 2015 ). The APA's perspective on 2018 trends in psychology also identifies an increased amount of psychology focus on how social determinants are influencing people's health ( Deangelis, 2017 ).

This study was not without limitations and the following should be taken into account. Firstly, this study used a sample of five specific journals to address the aim of the research study, despite general journal aims (as stated on journal websites), this inclusion signified a bias towards the research methods published in these specific journals only and limited generalisability. A broader sample of journals over a different period of time, or a single journal over a longer period of time might provide different results. A second limitation is the use of Excel spreadsheets and an electronic system to log articles, which was a manual process and therefore left room for error ( Bandara et al., 2015 ). To address this potential issue, co-coding was performed to reduce error. Lastly, this article categorised data based on the information presented in the article sample; there was no interpretation of what methodology could have been applied or whether the methods stated adhered to the criteria for the methods used. Thus, a large number of articles that did not clearly indicate a research method or design could influence the results of this review. However, this in itself was also a noteworthy result. Future research could review research methods of a broader sample of journals with an interpretive review tool that increases rigour. Additionally, the authors also encourage the future use of systematised review designs as a way to promote a concise procedure in applying this design.

Our research study presented the use of research methods for published articles in the field of psychology as well as recommendations for future research based on these results. Insight into the complex questions identified in literature, regarding what methods are used how these methods are being used and for what topics (why) was gained. This sample preferred quantitative methods, used convenience sampling and presented a lack of rigorous accounts for the remaining methodologies. All methodologies that were clearly indicated in the sample were tabulated to allow researchers insight into the general use of methods and not only the most frequently used methods. The lack of rigorous account of research methods in articles was represented in-depth for each step in the research process and can be of vital importance to address the current replication crisis within the field of psychology. Recommendations for future research aimed to motivate research into the practical implications of the results for psychology, for example, publication bias and the use of convenience samples.

Ethics Statement

This study was cleared by the North-West University Health Research Ethics Committee: NWU-00115-17-S1.

Author Contributions

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Aanstoos, C. M. (2014). Psychology . Available online at: http://eds.a.ebscohost.com.nwulib.nwu.ac.za/eds/detail/detail?sid=18de6c5c-2b03-4eac-94890145eb01bc70%40sessionmgr4006&vid$=$1&hid$=$4113&bdata$=$JnNpdGU9ZWRzL~WxpdmU%3d#AN$=$93871882&db$=$ers

Google Scholar

American Psychological Association (2020). Science of Psychology . Available online at: https://www.apa.org/action/science/

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., and Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: the APA Publications and Communications Board task force report. Am. Psychol. 73:3. doi: 10.1037/amp0000191

PubMed Abstract | CrossRef Full Text | Google Scholar

Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., and Beekhuyzen, J. (2015). Achieving rigor in literature reviews: insights from qualitative data analysis and tool-support. Commun. Ass. Inform. Syst. 37, 154–204. doi: 10.17705/1CAIS.03708

CrossRef Full Text | Google Scholar

Barr-Walker, J. (2017). Evidence-based information needs of public health workers: a systematized review. J. Med. Libr. Assoc. 105, 69–79. doi: 10.5195/JMLA.2017.109

Bittermann, A., and Fischer, A. (2018). How to identify hot topics in psychology using topic modeling. Z. Psychol. 226, 3–13. doi: 10.1027/2151-2604/a000318

Bluhm, D. J., Harman, W., Lee, T. W., and Mitchell, T. R. (2011). Qualitative research in management: a decade of progress. J. Manage. Stud. 48, 1866–1891. doi: 10.1111/j.1467-6486.2010.00972.x

Breen, L. J., and Darlaston-Jones, D. (2010). Moving beyond the enduring dominance of positivism in psychological research: implications for psychology in Australia. Aust. Psychol. 45, 67–76. doi: 10.1080/00050060903127481

Burman, E., and Whelan, P. (2011). Problems in / of Qualitative Research . Maidenhead: Open University Press/McGraw Hill.

Chaichanasakul, A., He, Y., Chen, H., Allen, G. E. K., Khairallah, T. S., and Ramos, K. (2011). Journal of Career Development: a 36-year content analysis (1972–2007). J. Career. Dev. 38, 440–455. doi: 10.1177/0894845310380223

Chryssochoou, X. (2015). Social Psychology. Inter. Encycl. Soc. Behav. Sci. 22, 532–537. doi: 10.1016/B978-0-08-097086-8.24095-6

Cichocka, A., and Jost, J. T. (2014). Stripped of illusions? Exploring system justification processes in capitalist and post-Communist societies. Inter. J. Psychol. 49, 6–29. doi: 10.1002/ijop.12011

Clay, R. A. (2017). Psychology is More Popular Than Ever. Monitor on Psychology: Trends Report . Available online at: https://www.apa.org/monitor/2017/11/trends-popular

Coetzee, M., and Van Zyl, L. E. (2014). A review of a decade's scholarly publications (2004–2013) in the South African Journal of Industrial Psychology. SA. J. Psychol . 40, 1–16. doi: 10.4102/sajip.v40i1.1227

Counsell, A., and Harlow, L. (2017). Reporting practices and use of quantitative methods in Canadian journal articles in psychology. Can. Psychol. 58, 140–147. doi: 10.1037/cap0000074

Deangelis, T. (2017). Targeting Social Factors That Undermine Health. Monitor on Psychology: Trends Report . Available online at: https://www.apa.org/monitor/2017/11/trend-social-factors

Demuth, C. (2015). New directions in qualitative research in psychology. Integr. Psychol. Behav. Sci. 49, 125–133. doi: 10.1007/s12124-015-9303-9

Denzin, N. K., and Lincoln, Y. (2003). The Landscape of Qualitative Research: Theories and Issues , 2nd Edn. London: Sage.

Drotar, D. (2010). A call for replications of research in pediatric psychology and guidance for authors. J. Pediatr. Psychol. 35, 801–805. doi: 10.1093/jpepsy/jsq049

Dweck, C. S. (2017). Is psychology headed in the right direction? Yes, no, and maybe. Perspect. Psychol. Sci. 12, 656–659. doi: 10.1177/1745691616687747

Earp, B. D., and Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Front. Psychol. 6:621. doi: 10.3389/fpsyg.2015.00621

Ezeh, A. C., Izugbara, C. O., Kabiru, C. W., Fonn, S., Kahn, K., Manderson, L., et al. (2010). Building capacity for public and population health research in Africa: the consortium for advanced research training in Africa (CARTA) model. Glob. Health Action 3:5693. doi: 10.3402/gha.v3i0.5693

Ferreira, A. L. L., Bessa, M. M. M., Drezett, J., and De Abreu, L. C. (2016). Quality of life of the woman carrier of endometriosis: systematized review. Reprod. Clim. 31, 48–54. doi: 10.1016/j.recli.2015.12.002

Fonseca, M. (2013). Most Common Reasons for Journal Rejections . Available online at: http://www.editage.com/insights/most-common-reasons-for-journal-rejections

Gough, B., and Lyons, A. (2016). The future of qualitative research in psychology: accentuating the positive. Integr. Psychol. Behav. Sci. 50, 234–243. doi: 10.1007/s12124-015-9320-8

Grant, M. J., and Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info. Libr. J. 26, 91–108. doi: 10.1111/j.1471-1842.2009.00848.x

Grix, J. (2002). Introducing students to the generic terminology of social research. Politics 22, 175–186. doi: 10.1111/1467-9256.00173

Gunasekare, U. L. T. P. (2015). Mixed research method as the third research paradigm: a literature review. Int. J. Sci. Res. 4, 361–368. Available online at: https://ssrn.com/abstract=2735996

Hengartner, M. P. (2018). Raising awareness for the replication crisis in clinical psychology by focusing on inconsistencies in psychotherapy Research: how much can we rely on published findings from efficacy trials? Front. Psychol. 9:256. doi: 10.3389/fpsyg.2018.00256

Holloway, W. (2008). Doing intellectual disagreement differently. Psychoanal. Cult. Soc. 13, 385–396. doi: 10.1057/pcs.2008.29

Ivankova, N. V., Creswell, J. W., and Plano Clark, V. L. (2016). “Foundations and Approaches to mixed methods research,” in First Steps in Research , 2nd Edn. K. Maree (Pretoria: Van Schaick Publishers), 306–335.

Johnson, M., Long, T., and White, A. (2001). Arguments for British pluralism in qualitative health research. J. Adv. Nurs. 33, 243–249. doi: 10.1046/j.1365-2648.2001.01659.x

Johnston, A., Kelly, S. E., Hsieh, S. C., Skidmore, B., and Wells, G. A. (2019). Systematic reviews of clinical practice guidelines: a methodological guide. J. Clin. Epidemiol. 108, 64–72. doi: 10.1016/j.jclinepi.2018.11.030

Ketchen, D. J. Jr., Boyd, B. K., and Bergh, D. D. (2008). Research methodology in strategic management: past accomplishments and future challenges. Organ. Res. Methods 11, 643–658. doi: 10.1177/1094428108319843

Ktepi, B. (2016). Data Analytics (DA) . Available online at: https://eds-b-ebscohost-com.nwulib.nwu.ac.za/eds/detail/detail?vid=2&sid=24c978f0-6685-4ed8-ad85-fa5bb04669b9%40sessionmgr101&bdata=JnNpdGU9ZWRzLWxpdmU%3d#AN=113931286&db=ers

Laher, S. (2016). Ostinato rigore: establishing methodological rigour in quantitative research. S. Afr. J. Psychol. 46, 316–327. doi: 10.1177/0081246316649121

Lee, C. (2015). The Myth of the Off-Limits Source . Available online at: http://blog.apastyle.org/apastyle/research/

Lee, T. W., Mitchell, T. R., and Sablynski, C. J. (1999). Qualitative research in organizational and vocational psychology, 1979–1999. J. Vocat. Behav. 55, 161–187. doi: 10.1006/jvbe.1999.1707

Leech, N. L., Anthony, J., and Onwuegbuzie, A. J. (2007). A typology of mixed methods research designs. Sci. Bus. Media B. V Qual. Quant 43, 265–275. doi: 10.1007/s11135-007-9105-3

Levitt, H. M., Motulsky, S. L., Wertz, F. J., Morrow, S. L., and Ponterotto, J. G. (2017). Recommendations for designing and reviewing qualitative research in psychology: promoting methodological integrity. Qual. Psychol. 4, 2–22. doi: 10.1037/qup0000082

Lowe, S. M., and Moore, S. (2014). Social networks and female reproductive choices in the developing world: a systematized review. Rep. Health 11:85. doi: 10.1186/1742-4755-11-85

Maree, K. (2016). “Planning a research proposal,” in First Steps in Research , 2nd Edn, ed K. Maree (Pretoria: Van Schaik Publishers), 49–70.

Maree, K., and Pietersen, J. (2016). “Sampling,” in First Steps in Research, 2nd Edn , ed K. Maree (Pretoria: Van Schaik Publishers), 191–202.

Ngulube, P. (2013). Blending qualitative and quantitative research methods in library and information science in sub-Saharan Africa. ESARBICA J. 32, 10–23. Available online at: http://hdl.handle.net/10500/22397 .

Nieuwenhuis, J. (2016). “Qualitative research designs and data-gathering techniques,” in First Steps in Research , 2nd Edn, ed K. Maree (Pretoria: Van Schaik Publishers), 71–102.

Nind, M., Kilburn, D., and Wiles, R. (2015). Using video and dialogue to generate pedagogic knowledge: teachers, learners and researchers reflecting together on the pedagogy of social research methods. Int. J. Soc. Res. Methodol. 18, 561–576. doi: 10.1080/13645579.2015.1062628

O'Cathain, A. (2009). Editorial: mixed methods research in the health sciences—a quiet revolution. J. Mix. Methods 3, 1–6. doi: 10.1177/1558689808326272

O'Neil, S., and Koekemoer, E. (2016). Two decades of qualitative research in psychology, industrial and organisational psychology and human resource management within South Africa: a critical review. SA J. Indust. Psychol. 42, 1–16. doi: 10.4102/sajip.v42i1.1350

Onwuegbuzie, A. J., and Collins, K. M. (2017). The role of sampling in mixed methods research enhancing inference quality. Köln Z Soziol. 2, 133–156. doi: 10.1007/s11577-017-0455-0

Perestelo-Pérez, L. (2013). Standards on how to develop and report systematic reviews in psychology and health. Int. J. Clin. Health Psychol. 13, 49–57. doi: 10.1016/S1697-2600(13)70007-3

Pericall, L. M. T., and Taylor, E. (2014). Family function and its relationship to injury severity and psychiatric outcome in children with acquired brain injury: a systematized review. Dev. Med. Child Neurol. 56, 19–30. doi: 10.1111/dmcn.12237

Peterson, R. A., and Merunka, D. R. (2014). Convenience samples of college students and research reproducibility. J. Bus. Res. 67, 1035–1041. doi: 10.1016/j.jbusres.2013.08.010

Ritchie, J., Lewis, J., and Elam, G. (2009). “Designing and selecting samples,” in Qualitative Research Practice: A Guide for Social Science Students and Researchers , 2nd Edn, ed J. Ritchie and J. Lewis (London: Sage), 1–23.

Sandelowski, M. (2011). When a cigar is not just a cigar: alternative perspectives on data and data analysis. Res. Nurs. Health 34, 342–352. doi: 10.1002/nur.20437

Sandelowski, M., Voils, C. I., and Knafl, G. (2009). On quantitizing. J. Mix. Methods Res. 3, 208–222. doi: 10.1177/1558689809334210

Scholtz, S. E., De Klerk, W., and De Beer, L. T. (2019). A data generated research framework for conducting research methods in psychological research.

Scimago Journal & Country Rank (2017). Available online at: http://www.scimagojr.com/journalrank.php?category=3201&year=2015

Scopus (2017a). About Scopus . Available online at: https://www.scopus.com/home.uri (accessed February 01, 2017).

Scopus (2017b). Document Search . Available online at: https://www.scopus.com/home.uri (accessed February 01, 2017).

Scott Jones, J., and Goldring, J. E. (2015). ‘I' m not a quants person'; key strategies in building competence and confidence in staff who teach quantitative research methods. Int. J. Soc. Res. Methodol. 18, 479–494. doi: 10.1080/13645579.2015.1062623

Smith, B., and McGannon, K. R. (2018). Developing rigor in quantitative research: problems and opportunities within sport and exercise psychology. Int. Rev. Sport Exerc. Psychol. 11, 101–121. doi: 10.1080/1750984X.2017.1317357

Stangor, C. (2011). Introduction to Psychology . Available online at: http://www.saylor.org/books/

Strydom, H. (2011). “Sampling in the quantitative paradigm,” in Research at Grass Roots; For the Social Sciences and Human Service Professions , 4th Edn, eds A. S. de Vos, H. Strydom, C. B. Fouché, and C. S. L. Delport (Pretoria: Van Schaik Publishers), 221–234.

Tashakkori, A., and Teddlie, C. (2003). Handbook of Mixed Methods in Social & Behavioural Research . Thousand Oaks, CA: SAGE publications.

Toomela, A. (2010). Quantitative methods in psychology: inevitable and useless. Front. Psychol. 1:29. doi: 10.3389/fpsyg.2010.00029

Truscott, D. M., Swars, S., Smith, S., Thornton-Reid, F., Zhao, Y., Dooley, C., et al. (2010). A cross-disciplinary examination of the prevalence of mixed methods in educational research: 1995–2005. Int. J. Soc. Res. Methodol. 13, 317–328. doi: 10.1080/13645570903097950

Weiten, W. (2010). Psychology Themes and Variations . Belmont, CA: Wadsworth.

Keywords: research methods, research approach, research trends, psychological research, systematised review, research designs, research topic

Citation: Scholtz SE, de Klerk W and de Beer LT (2020) The Use of Research Methods in Psychological Research: A Systematised Review. Front. Res. Metr. Anal. 5:1. doi: 10.3389/frma.2020.00001

Received: 30 December 2019; Accepted: 28 February 2020; Published: 20 March 2020.

Reviewed by:

Copyright © 2020 Scholtz, de Klerk and de Beer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Salomé Elizabeth Scholtz, 22308563@nwu.ac.za

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Research Methods: How to Perform an Effective Peer Review

Affiliations.

• 1 Paul C. Gaffney Division of Pediatric Hospital Medicine, UPMC Children's Hospital of Pittsburgh, Pittsburgh, Pennsylvania.
• 2 Department of Pediatrics, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania.
• 3 Weill Department of Medicine, Weill Cornell Medicine, New York, New York.
• 4 Department of Medicine, University of Minnesota Medical School, Minneapolis, Minneapolis.
• 5 Department of Pediatrics, University of Minnesota Medical School, Minneapolis, Minneapolis.
• PMID: 36214067
• DOI: 10.1542/hpeds.2022-006764

Scientific peer review has existed for centuries and is a cornerstone of the scientific publication process. Because the number of scientific publications has rapidly increased over the past decades, so has the number of peer reviews and peer reviewers. In this paper, drawing on the relevant medical literature and our collective experience as peer reviewers, we provide a user guide to the peer review process, including discussion of the purpose and limitations of peer review, the qualities of a good peer reviewer, and a step-by-step process of how to conduct an effective peer review.

Copyright © 2022 by the American Academy of Pediatrics.

PubMed Disclaimer

Similar articles

• How to be a good reviewer: A step-by-step guide for approaching peer review of a scientific manuscript. Sedaghat AR, Bernal-Sprekelsen M, Fokkens WJ, Smith TL, Stewart MG, Johnson RF. Sedaghat AR, et al. Laryngoscope Investig Otolaryngol. 2024 Jun 4;9(3):e1266. doi: 10.1002/lio2.1266. eCollection 2024 Jun. Laryngoscope Investig Otolaryngol. 2024. PMID: 38835335 Free PMC article. Review.
• Tips and tricks for how to become a good reviewer. Hughes JD, Cristiani R, Hirschmann MT, Musahl V, Eriksson K, Karlsson J. Hughes JD, et al. Knee Surg Sports Traumatol Arthrosc. 2023 Nov;31(11):4631-4636. doi: 10.1007/s00167-023-07595-6. Knee Surg Sports Traumatol Arthrosc. 2023. PMID: 37792083
• How to be a good peer reviewer of scientific manuscripts. Dhillon P. Dhillon P. FEBS J. 2021 May;288(9):2750-2756. doi: 10.1111/febs.15705. Epub 2021 Jan 23. FEBS J. 2021. PMID: 33486891
• We need to talk about peer-review-Experienced reviewers are not endangered species, but they need motivation. Ellwanger JH, Chies JAB. Ellwanger JH, et al. J Clin Epidemiol. 2020 Sep;125:201-205. doi: 10.1016/j.jclinepi.2020.02.001. Epub 2020 Feb 14. J Clin Epidemiol. 2020. PMID: 32061827 Review.
• The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise. Kovanis M, Porcher R, Ravaud P, Trinquart L. Kovanis M, et al. PLoS One. 2016 Nov 10;11(11):e0166387. doi: 10.1371/journal.pone.0166387. eCollection 2016. PLoS One. 2016. PMID: 27832157 Free PMC article.
• Search in MeSH

LinkOut - more resources

Full text sources.

• Silverchair Information Systems

Miscellaneous

• NCI CPTAC Assay Portal

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Research Methods: Peer-Reviewed Journal Articles

• Getting Started
• What Type of Source?
• Credible Sources
• Finding Background Information
• Library Databases
• Reference Books
• Electronic Books
• Online Reference Collections
• Databases Presenting Two Sides of an Argument
• Peer-Reviewed Journal Articles
• Writing Style Guides and Citing Sources
• Annotated Bibliographies

What is a Peer-Reviewed (Academic) Journal?

What Is a Peer-Reviewed Journal?

Peer Review is a process that journals use to ensure the articles they publish represent the best scholarship currently available. When an article is submitted to a peer reviewed journal, the editors send it out to other scholars in the same field (the author's peers) to get their opinion on the quality of the scholarship, its relevance to the field, its appropriateness for the journal, etc.

Publications that don't use peer review (Time, Cosmo, Salon) just rely on the judgement of the editors whether an article is up to snuff or not. That's why you can't count on them for solid, scientific scholarship. --University of Texas at Austin

Databases Containing Peer-Reviewed Journal Articles

Each database containing peer-reviewed journals has different content coverage and materials.  The databases listed in this Research Guide are available only to Truckee Meadows Community College students, faculty and staff. You will need your TMCC credentials (Username and Password) to access them off-campus.

When searching a database, a search term frequently will retrieve many articles.  Browse the article abstracts to find one or more relevant to your search.

Some of the databases provide citations for the articles.

Consult a librarian for assistance.

• Databases with peer-reviewed articles and content . This list can also be sorted by subject!

How to Read a Peer-Reviewed Journal Article

Tips for Reading a Research Article

Read the Abstract. It consists of a brief summary of the research questions and methods. It may also state the findings. Because it is short and often written in dense psychological language, you may need to read it a couple of times. Try to restate the abstract in your own nontechnical language.

• Read the Introduction. This is the beginning of the article, appearing first after the Abstract. This contains information about the authors' interest in the research, why they chose the topic, their hypothesis , and methods. This part also sets out the operational definitions of variables.
• Read the Discussion section. Skip over the Methods section for the time being. The Discussion section will explain the main findings in great detail and discuss any methodological problems or flaws that the researchers discovered.
• Read the Methods section. Now that you know the results and what the researchers claim the results mean, you are prepared to read about the Methods. This section explains the type of research and the techniques and assessment instruments used. If the research utilized self-reports and questionnaires, the questions and statements used may be set out either in this section or in an appendix that appears at the end of the report.
• Read the Results section. This is the most technically challenging part of a research report. But you already know the findings (from reading about them in the Discussion section). This section explains the statistical analyses that led the authors to their conclusions.
• Read the Conclusion. The last section of the report (before any appendices) summarizes the findings, but, more important for social research, it sets out what the researchers think is the value of their research for real-life application and for public policy. This section often contains suggestions for future research, including issues that the researchers became aware of in the course of the study.
• Following the conclusions are appendices, usually tables of findings, presentations of questions and statements used in self-reports and questionnaires, and examples of forms used (such as forms for behavioral assessments).

Modified from Net Lab

• << Previous: Databases Presenting Two Sides of an Argument
• Next: Writing Style Guides and Citing Sources >>
• Last Updated: Apr 15, 2024 9:36 AM
• URL: https://libguides.tmcc.edu/researchmethods

• Interlibrary Loan and Scan & Deliver
• Course Reserves
• Purchase Request
• Collection Development & Maintenance
• Current Negotiations
• Ask a Librarian
• Instructor Support
• Library How-To
• Research Guides
• Research Support
• Study Rooms
• Research Rooms
• Partner Spaces
• Loanable Equipment
• Print, Scan, Copy
• 3D Printers
• Poster Printing
• OSULP Leadership
• Strategic Plan

Scholarly Articles: How can I tell?

• Journal Information
• Literature Review
• Author and affiliation
• Introduction
• Specialized Vocabulary

Methodology

• Research sponsors
• Peer-review

The methodology section or methods section tells you how the author(s) went about doing their research. It should let you know a) what method they used to gather data (survey, interviews, experiments, etc.), why they chose this method, and what the limitations are to this method.

The methodology section should be detailed enough that another researcher could replicate the study described. When you read the methodology or methods section:

• What kind of research method did the authors use? Is it an appropriate method for the type of study they are conducting?
• How did the authors get their tests subjects? What criteria did they use?
• What are the contexts of the study that may have affected the results (e.g. environmental conditions, lab conditions, timing questions, etc.)
• Is the sample size representative of the larger population (i.e., was it big enough?)
• Are the data collection instruments and procedures likely to have measured all the important characteristics with reasonable accuracy?
• Does the data analysis appear to have been done with care, and were appropriate analytical techniques used? 

A good researcher will always let you know about the limitations of his or her research.

• << Previous: Specialized Vocabulary
• Next: Results >>
• Last Updated: Apr 15, 2024 3:26 PM
• URL: https://guides.library.oregonstate.edu/ScholarlyArticle

Contact Info

121 The Valley Library Corvallis OR 97331–4501

Phone: 541-737-3331

Services for Persons with Disabilities

In the Valley Library

• Oregon State University Press
• Special Collections and Archives Research Center
• Undergrad Research & Writing Studio
• Graduate Student Commons
• Tutoring Services
• Northwest Art Collection

Digital Projects

• Oregon Explorer
• Oregon Digital
• ScholarsArchive@OSU
• Digital Publishing Initiatives
• Atlas of the Pacific Northwest
• Marilyn Potts Guin Library  
• Cascades Campus Library
• McDowell Library of Vet Medicine

Research Methods In Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

Research methods in psychology are systematic procedures used to observe, describe, predict, and explain behavior and mental processes. They include experiments, surveys, case studies, and naturalistic observations, ensuring data collection is objective and reliable to understand and explain psychological phenomena.

Hypotheses are statements about the prediction of the results, that can be verified or disproved by some investigation.

There are four types of hypotheses :

• Null Hypotheses (H0 ) – these predict that no difference will be found in the results between the conditions. Typically these are written ‘There will be no difference…’
• Alternative Hypotheses (Ha or H1) – these predict that there will be a significant difference in the results between the two conditions. This is also known as the experimental hypothesis.
• One-tailed (directional) hypotheses – these state the specific direction the researcher expects the results to move in, e.g. higher, lower, more, less. In a correlation study, the predicted direction of the correlation can be either positive or negative.
• Two-tailed (non-directional) hypotheses – these state that a difference will be found between the conditions of the independent variable but does not state the direction of a difference or relationship. Typically these are always written ‘There will be a difference ….’

All research has an alternative hypothesis (either a one-tailed or two-tailed) and a corresponding null hypothesis.

Once the research is conducted and results are found, psychologists must accept one hypothesis and reject the other. 

So, if a difference is found, the Psychologist would accept the alternative hypothesis and reject the null.  The opposite applies if no difference is found.

Sampling techniques

Sampling is the process of selecting a representative group from the population under study.

A sample is the participants you select from a target population (the group you are interested in) to make generalizations about.

Representative means the extent to which a sample mirrors a researcher’s target population and reflects its characteristics.

Generalisability means the extent to which their findings can be applied to the larger population of which their sample was a part.

• Volunteer sample : where participants pick themselves through newspaper adverts, noticeboards or online.
• Opportunity sampling : also known as convenience sampling , uses people who are available at the time the study is carried out and willing to take part. It is based on convenience.
• Random sampling : when every person in the target population has an equal chance of being selected. An example of random sampling would be picking names out of a hat.
• Systematic sampling : when a system is used to select participants. Picking every Nth person from all possible participants. N = the number of people in the research population / the number of people needed for the sample.
• Stratified sampling : when you identify the subgroups and select participants in proportion to their occurrences.
• Snowball sampling : when researchers find a few participants, and then ask them to find participants themselves and so on.
• Quota sampling : when researchers will be told to ensure the sample fits certain quotas, for example they might be told to find 90 participants, with 30 of them being unemployed.

Experiments always have an independent and dependent variable .

• The independent variable is the one the experimenter manipulates (the thing that changes between the conditions the participants are placed into). It is assumed to have a direct effect on the dependent variable.
• The dependent variable is the thing being measured, or the results of the experiment.

Operationalization of variables means making them measurable/quantifiable. We must use operationalization to ensure that variables are in a form that can be easily tested.

For instance, we can’t really measure ‘happiness’, but we can measure how many times a person smiles within a two-hour period. 

By operationalizing variables, we make it easy for someone else to replicate our research. Remember, this is important because we can check if our findings are reliable.

Extraneous variables are all variables which are not independent variable but could affect the results of the experiment.

It can be a natural characteristic of the participant, such as intelligence levels, gender, or age for example, or it could be a situational feature of the environment such as lighting or noise.

Demand characteristics are a type of extraneous variable that occurs if the participants work out the aims of the research study, they may begin to behave in a certain way.

For example, in Milgram’s research , critics argued that participants worked out that the shocks were not real and they administered them as they thought this was what was required of them. 

Extraneous variables must be controlled so that they do not affect (confound) the results.

Randomly allocating participants to their conditions or using a matched pairs experimental design can help to reduce participant variables. 

Situational variables are controlled by using standardized procedures, ensuring every participant in a given condition is treated in the same way

Experimental Design

Experimental design refers to how participants are allocated to each condition of the independent variable, such as a control or experimental group.

• Independent design ( between-groups design ): each participant is selected for only one group. With the independent design, the most common way of deciding which participants go into which group is by means of randomization. 
• Matched participants design : each participant is selected for only one group, but the participants in the two groups are matched for some relevant factor or factors (e.g. ability; sex; age).
• Repeated measures design ( within groups) : each participant appears in both groups, so that there are exactly the same participants in each group.
• The main problem with the repeated measures design is that there may well be order effects. Their experiences during the experiment may change the participants in various ways.
• They may perform better when they appear in the second group because they have gained useful information about the experiment or about the task. On the other hand, they may perform less well on the second occasion because of tiredness or boredom.
• Counterbalancing is the best way of preventing order effects from disrupting the findings of an experiment, and involves ensuring that each condition is equally likely to be used first and second by the participants.

If we wish to compare two groups with respect to a given independent variable, it is essential to make sure that the two groups do not differ in any other important way. 

Experimental Methods

All experimental methods involve an iv (independent variable) and dv (dependent variable)..

• Field experiments are conducted in the everyday (natural) environment of the participants. The experimenter still manipulates the IV, but in a real-life setting. It may be possible to control extraneous variables, though such control is more difficult than in a lab experiment.
• Natural experiments are when a naturally occurring IV is investigated that isn’t deliberately manipulated, it exists anyway. Participants are not randomly allocated, and the natural event may only occur rarely.

Case studies are in-depth investigations of a person, group, event, or community. It uses information from a range of sources, such as from the person concerned and also from their family and friends.

Many techniques may be used such as interviews, psychological tests, observations and experiments. Case studies are generally longitudinal: in other words, they follow the individual or group over an extended period of time. 

Case studies are widely used in psychology and among the best-known ones carried out were by Sigmund Freud . He conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

Case studies provide rich qualitative data and have high levels of ecological validity. However, it is difficult to generalize from individual cases as each one has unique characteristics.

Correlational Studies

Correlation means association; it is a measure of the extent to which two variables are related. One of the variables can be regarded as the predictor variable with the other one as the outcome variable.

Correlational studies typically involve obtaining two different measures from a group of participants, and then assessing the degree of association between the measures. 

The predictor variable can be seen as occurring before the outcome variable in some sense. It is called the predictor variable, because it forms the basis for predicting the value of the outcome variable.

Relationships between variables can be displayed on a graph or as a numerical score called a correlation coefficient.

• If an increase in one variable tends to be associated with an increase in the other, then this is known as a positive correlation .
• If an increase in one variable tends to be associated with a decrease in the other, then this is known as a negative correlation .
• A zero correlation occurs when there is no relationship between variables.

After looking at the scattergraph, if we want to be sure that a significant relationship does exist between the two variables, a statistical test of correlation can be conducted, such as Spearman’s rho.

The test will give us a score, called a correlation coefficient . This is a value between 0 and 1, and the closer to 1 the score is, the stronger the relationship between the variables. This value can be both positive e.g. 0.63, or negative -0.63.

A correlation between variables, however, does not automatically mean that the change in one variable is the cause of the change in the values of the other variable. A correlation only shows if there is a relationship between variables.

Correlation does not always prove causation, as a third variable may be involved. 

Interview Methods

Interviews are commonly divided into two types: structured and unstructured.

A fixed, predetermined set of questions is put to every participant in the same order and in the same way. 

Responses are recorded on a questionnaire, and the researcher presets the order and wording of questions, and sometimes the range of alternative answers.

The interviewer stays within their role and maintains social distance from the interviewee.

There are no set questions, and the participant can raise whatever topics he/she feels are relevant and ask them in their own way. Questions are posed about participants’ answers to the subject

Unstructured interviews are most useful in qualitative research to analyze attitudes and values.

Though they rarely provide a valid basis for generalization, their main advantage is that they enable the researcher to probe social actors’ subjective point of view. 

Questionnaire Method

Questionnaires can be thought of as a kind of written interview. They can be carried out face to face, by telephone, or post.

The choice of questions is important because of the need to avoid bias or ambiguity in the questions, ‘leading’ the respondent or causing offense.

• Open questions are designed to encourage a full, meaningful answer using the subject’s own knowledge and feelings. They provide insights into feelings, opinions, and understanding. Example: “How do you feel about that situation?”
• Closed questions can be answered with a simple “yes” or “no” or specific information, limiting the depth of response. They are useful for gathering specific facts or confirming details. Example: “Do you feel anxious in crowds?”

Its other practical advantages are that it is cheaper than face-to-face interviews and can be used to contact many respondents scattered over a wide area relatively quickly.

Observations

There are different types of observation methods :

• Covert observation is where the researcher doesn’t tell the participants they are being observed until after the study is complete. There could be ethical problems or deception and consent with this particular observation method.
• Overt observation is where a researcher tells the participants they are being observed and what they are being observed for.
• Controlled : behavior is observed under controlled laboratory conditions (e.g., Bandura’s Bobo doll study).
• Natural : Here, spontaneous behavior is recorded in a natural setting.
• Participant : Here, the observer has direct contact with the group of people they are observing. The researcher becomes a member of the group they are researching.  
• Non-participant (aka “fly on the wall): The researcher does not have direct contact with the people being observed. The observation of participants’ behavior is from a distance

Pilot Study

A pilot  study is a small scale preliminary study conducted in order to evaluate the feasibility of the key s teps in a future, full-scale project.

A pilot study is an initial run-through of the procedures to be used in an investigation; it involves selecting a few people and trying out the study on them. It is possible to save time, and in some cases, money, by identifying any flaws in the procedures designed by the researcher.

A pilot study can help the researcher spot any ambiguities (i.e. unusual things) or confusion in the information given to participants or problems with the task devised.

Sometimes the task is too hard, and the researcher may get a floor effect, because none of the participants can score at all or can complete the task – all performances are low.

The opposite effect is a ceiling effect, when the task is so easy that all achieve virtually full marks or top performances and are “hitting the ceiling”.

Research Design

In cross-sectional research , a researcher compares multiple segments of the population at the same time

Sometimes, we want to see how people change over time, as in studies of human development and lifespan. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time.

In cohort studies , the participants must share a common factor or characteristic such as age, demographic, or occupation. A cohort study is a type of longitudinal study in which researchers monitor and observe a chosen population over an extended period.

Triangulation means using more than one research method to improve the study’s validity.

Reliability

Reliability is a measure of consistency, if a particular measurement is repeated and the same result is obtained then it is described as being reliable.

• Test-retest reliability :  assessing the same person on two different occasions which shows the extent to which the test produces the same answers.
• Inter-observer reliability : the extent to which there is an agreement between two or more observers.

Meta-Analysis

A meta-analysis is a systematic review that involves identifying an aim and then searching for research studies that have addressed similar aims/hypotheses.

This is done by looking through various databases, and then decisions are made about what studies are to be included/excluded.

Strengths: Increases the conclusions’ validity as they’re based on a wider range.

Weaknesses: Research designs in studies can vary, so they are not truly comparable.

Peer Review

A researcher submits an article to a journal. The choice of the journal may be determined by the journal’s audience or prestige.

The journal selects two or more appropriate experts (psychologists working in a similar field) to peer review the article without payment. The peer reviewers assess: the methods and designs used, originality of the findings, the validity of the original research findings and its content, structure and language.

Feedback from the reviewer determines whether the article is accepted. The article may be: Accepted as it is, accepted with revisions, sent back to the author to revise and re-submit or rejected without the possibility of submission.

The editor makes the final decision whether to accept or reject the research report based on the reviewers comments/ recommendations.

Peer review is important because it prevent faulty data from entering the public domain, it provides a way of checking the validity of findings and the quality of the methodology and is used to assess the research rating of university departments.

Peer reviews may be an ideal, whereas in practice there are lots of problems. For example, it slows publication down and may prevent unusual, new work being published. Some reviewers might use it as an opportunity to prevent competing researchers from publishing work.

Some people doubt whether peer review can really prevent the publication of fraudulent research.

The advent of the internet means that a lot of research and academic comment is being published without official peer reviews than before, though systems are evolving on the internet where everyone really has a chance to offer their opinions and police the quality of research.

Types of Data

• Quantitative data is numerical data e.g. reaction time or number of mistakes. It represents how much or how long, how many there are of something. A tally of behavioral categories and closed questions in a questionnaire collect quantitative data.
• Qualitative data is virtually any type of information that can be observed and recorded that is not numerical in nature and can be in the form of written or verbal communication. Open questions in questionnaires and accounts from observational studies collect qualitative data.
• Primary data is first-hand data collected for the purpose of the investigation.
• Secondary data is information that has been collected by someone other than the person who is conducting the research e.g. taken from journals, books or articles.

Validity means how well a piece of research actually measures what it sets out to, or how well it reflects the reality it claims to represent.

Validity is whether the observed effect is genuine and represents what is actually out there in the world.

• Concurrent validity is the extent to which a psychological measure relates to an existing similar measure and obtains close results. For example, a new intelligence test compared to an established test.
• Face validity : does the test measure what it’s supposed to measure ‘on the face of it’. This is done by ‘eyeballing’ the measuring or by passing it to an expert to check.
• Ecological validit y is the extent to which findings from a research study can be generalized to other settings / real life.
• Temporal validity is the extent to which findings from a research study can be generalized to other historical times.

Features of Science

• Paradigm – A set of shared assumptions and agreed methods within a scientific discipline.
• Paradigm shift – The result of the scientific revolution: a significant change in the dominant unifying theory within a scientific discipline.
• Objectivity – When all sources of personal bias are minimised so not to distort or influence the research process.
• Empirical method – Scientific approaches that are based on the gathering of evidence through direct observation and experience.
• Replicability – The extent to which scientific procedures and findings can be repeated by other researchers.
• Falsifiability – The principle that a theory cannot be considered scientific unless it admits the possibility of being proved untrue.

Statistical Testing

A significant result is one where there is a low probability that chance factors were responsible for any observed difference, correlation, or association in the variables tested.

If our test is significant, we can reject our null hypothesis and accept our alternative hypothesis.

If our test is not significant, we can accept our null hypothesis and reject our alternative hypothesis. A null hypothesis is a statement of no effect.

In Psychology, we use p < 0.05 (as it strikes a balance between making a type I and II error) but p < 0.01 is used in tests that could cause harm like introducing a new drug.

A type I error is when the null hypothesis is rejected when it should have been accepted (happens when a lenient significance level is used, an error of optimism).

A type II error is when the null hypothesis is accepted when it should have been rejected (happens when a stringent significance level is used, an error of pessimism).

Ethical Issues

• Informed consent is when participants are able to make an informed judgment about whether to take part. It causes them to guess the aims of the study and change their behavior.
• To deal with it, we can gain presumptive consent or ask them to formally indicate their agreement to participate but it may invalidate the purpose of the study and it is not guaranteed that the participants would understand.
• Deception should only be used when it is approved by an ethics committee, as it involves deliberately misleading or withholding information. Participants should be fully debriefed after the study but debriefing can’t turn the clock back.
• All participants should be informed at the beginning that they have the right to withdraw if they ever feel distressed or uncomfortable.
• It causes bias as the ones that stayed are obedient and some may not withdraw as they may have been given incentives or feel like they’re spoiling the study. Researchers can offer the right to withdraw data after participation.
• Participants should all have protection from harm . The researcher should avoid risks greater than those experienced in everyday life and they should stop the study if any harm is suspected. However, the harm may not be apparent at the time of the study.
• Confidentiality concerns the communication of personal information. The researchers should not record any names but use numbers or false names though it may not be possible as it is sometimes possible to work out who the researchers were.

Sociological Research Methods 4155 - Dr. Powell

• Finding Books
• Find Articles
• Sociological Research

Peer Reviewed Articles

• Statistics/Data Sets
• Annotated Bibliography
• Academic Integrity

How to recognize peer-reviewed (refereed) journals

In many cases professors will require that students utilize articles from “peer-reviewed” journals. Sometimes the phrases “refereed journals” or “scholarly journals” are used to describe the same type of journals. But what are peer-reviewed (or refereed or scholarly) journal articles, and why do faculty require their use?

Three categories of information resources:

• Newspapers and magazines containing news - Articles are written by reporters who may or may not be experts in the field of the article. Consequently, articles may contain incorrect information.
• Journals containing articles written by academics and/or professionals — Although the articles are written by “experts,” any particular “expert” may have some ideas that are really “out there!”
• Peer-reviewed (refereed or scholarly) journals - Articles are written by experts and are reviewed by several other experts in the field before the article is published in the journal in order to insure the article’s quality. (The article is more likely to be scientifically valid, reach reasonable conclusions, etc.) In most cases the reviewers do not know who the author of the article is, so that the article succeeds or fails on its own merit, not the reputation of the expert.

Helpful hint!

Not all information in a peer-reviewed journal is actually refereed, or reviewed. For example, editorials, letters to the editor, book reviews, and other types of information don’t count as articles, and may not be accepted by your professor.

How do you determine whether an article qualifies as being a peer-reviewed journal article?

First, you need to be able to identify which journals are peer-reviewed. There are generally four methods for doing this

• Limiting a database search to peer-reviewed journals only. Some databases allow you to limit searches for articles to peer reviewed journals only. For example, Academic Search Complete has this feature on the initial search screen - click on the pertinent box to limit the search. In some databases you may have to go to an “advanced” or “expert” search screen to do this. Remember, many databases do not allow you to limit your search in this way.
• Locate the journal in the Library or online, then identify the most current entire year’s issues.
• Locate the masthead of the publication. This oftentimes consists of a box towards either the front or the end of the periodical, and contains publication information such as the editors of the journal, the publisher, the place of publication, the subscription cost and similar information.
• Does the journal say that it is peer-reviewed? If so, you’re done! If not, move on to step d.
• Check in and around the masthead to locate the method for submitting articles to the publication.  If you find information similar to “to submit articles, send three copies…”, the journal is probably peer-reviewed. In this case, you are inferring that the publication is then going to send the multiple copies of the article to the journal’s reviewers. This may not always be the case, so relying upon this criterion alone may prove inaccurate.
• If you do not see this type of statement in the first issue of the journal that you look at, examine the remaining journals to see if this information is included. Sometimes publications will include this information in only a single issue a year.
• Is it scholarly, using technical terminology? Does the article format approximate the following - abstract, literature review, methodology, results, conclusion, and references? Are the articles written by scholarly researchers in the field that the periodical pertains to? Is advertising non-existent, or kept to a minimum? Are there references listed in footnotes or bibliographies? If you answered yes to all these questions , the journal may very well be peer-reviewed. This determination would be strengthened by having met the previous criterion of a multiple-copies submission requirement. If you answered these questions no , the journal is probably not peer-reviewed.
• Find the official web site on the internet, and check to see if it states that the journal is peer-reviewed. Be careful to use the official site (often located at the journal publisher’s web site), and, even then, information could potentially be “inaccurate.”

If you have used the previous four methods in trying to determine if an article is from a peer-reviewed journal and are still unsure, speak to your instructor.

• << Previous: Empirical Research
• Next: Statistics/Data Sets >>
• Last Updated: Jun 17, 2024 12:11 PM
• URL: https://guides.library.charlotte.edu/c.php?g=720757

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Trends in...

Trends in cardiovascular disease incidence among 22 million people in the UK over 20 years: population based study

• Related content
• Peer review
• Geert Molenberghs , professor 4 ,
• Geert Verbeke , professor 4 ,
• Francesco Zaccardi , associate professor 5 ,
• Claire Lawson , associate professor 5 ,
• Jocelyn M Friday , data scientist 1 ,
• Huimin Su , PhD student 2 ,
• Pardeep S Jhund , professor 1 ,
• Naveed Sattar , professor 6 ,
• Kazem Rahimi , professor 3 ,
• John G Cleland , professor 1 ,
• Kamlesh Khunti , professor 5 ,
• Werner Budts , professor 1 7 ,
• John J V McMurray , professor 1
• 1 School of Cardiovascular and Metabolic Health, British Heart Foundation Cardiovascular Research Centre, University of Glasgow, Glasgow, UK
• 2 Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
• 3 Deep Medicine, Nuffield Department of Women’s and Reproductive Health, University of Oxford, Oxford, UK
• 4 Interuniversity Institute for Biostatistics and statistical Bioinformatics (I-BioStat), Hasselt University and KU Leuven, Belgium
• 5 Leicester Real World Evidence Unit, Diabetes Research Centre, University of Leicester, Leicester, UK
• 6 College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK
• 7 Congenital and Structural Cardiology, University Hospitals Leuven, Belgium
• Correspondence to: N Conrad nathalie.conrad{at}kuleuven.be (or @nathalie_conrad on X)
• Accepted 1 May 2024

Objective To investigate the incidence of cardiovascular disease (CVD) overall and by age, sex, and socioeconomic status, and its variation over time, in the UK during 2000-19.

Design Population based study.

Setting UK.

Participants 1 650 052 individuals registered with a general practice contributing to Clinical Practice Research Datalink and newly diagnosed with at least one CVD from 1 January 2000 to 30 June 2019.

Main outcome measures The primary outcome was incident diagnosis of CVD, comprising acute coronary syndrome, aortic aneurysm, aortic stenosis, atrial fibrillation or flutter, chronic ischaemic heart disease, heart failure, peripheral artery disease, second or third degree heart block, stroke (ischaemic, haemorrhagic, and unspecified), and venous thromboembolism (deep vein thrombosis or pulmonary embolism). Disease incidence rates were calculated individually and as a composite outcome of all 10 CVDs combined and were standardised for age and sex using the 2013 European standard population. Negative binomial regression models investigated temporal trends and variation by age, sex, and socioeconomic status.

Results The mean age of the population was 70.5 years and 47.6% (n=784 904) were women. The age and sex standardised incidence of all 10 prespecified CVDs declined by 19% during 2000-19 (incidence rate ratio 2017-19 v 2000-02: 0.80, 95% confidence interval 0.73 to 0.88). The incidence of coronary heart disease and stroke decreased by about 30% (incidence rate ratios for acute coronary syndrome, chronic ischaemic heart disease, and stroke were 0.70 (0.69 to 0.70), 0.67 (0.66 to 0.67), and 0.75 (0.67 to 0.83), respectively). In parallel, an increasing number of diagnoses of cardiac arrhythmias, valve disease, and thromboembolic diseases were observed. As a result, the overall incidence of CVDs across the 10 conditions remained relatively stable from the mid-2000s. Age stratified analyses further showed that the observed decline in coronary heart disease incidence was largely restricted to age groups older than 60 years, with little or no improvement in younger age groups. Trends were generally similar between men and women. A socioeconomic gradient was observed for almost every CVD investigated. The gradient did not decrease over time and was most noticeable for peripheral artery disease (incidence rate ratio most deprived v least deprived: 1.98 (1.87 to 2.09)), acute coronary syndrome (1.55 (1.54 to 1.57)), and heart failure (1.50 (1.41 to 1.59)).

Conclusions Despite substantial improvements in the prevention of atherosclerotic diseases in the UK, the overall burden of CVDs remained high during 2000-19. For CVDs to decrease further, future prevention strategies might need to consider a broader spectrum of conditions, including arrhythmias, valve diseases, and thromboembolism, and examine the specific needs of younger age groups and socioeconomically deprived populations.

Introduction

Since the 1970s, the prevention of coronary disease, both primary and secondary, has improved considerably, largely attributable to public health efforts to control risk factors, such as antismoking legislation, and the widespread use of drugs such as statins. 1 2

Improvements in mortality due to heart disease have, however, stalled in several high income countries, 3 and reports suggest that the incidence of heart disease might even be increasing among younger people. 4 5 6 Conversely, along with coronary heart disease, other cardiovascular conditions are becoming relatively more prominent in older people, altering the profile of cardiovascular disease (CVD) in ageing societies. The importance of non-traditional risk factors for atherosclerotic diseases, such as socioeconomic deprivation, has also been increasingly recognised. Whether socioeconomic deprivation is as strongly associated with other CVDs as with atherosclerosis is uncertain, but it is important to understand as many countries have reported an increase in socioeconomic inequalities. 7

Large scale epidemiological studies are therefore needed to investigate secular trends in CVDs to target future preventive efforts, highlight the focus for future clinical trials, and identify healthcare resources required to manage emerging problems. Existing comprehensive efforts, such as statistics on CVD from leading medical societies or the Global Burden of Diseases studies, have helped toward this goal, but reliable age standardised incidence rates for all CVDs, how these vary by population subgroups, and changes over time are currently not available. 8 9 10

We used a large longitudinal database of linked primary care, secondary care, and death registry records from a representative sample of the UK population 11 12 to assess trends in the incidence of 10 of the most common CVDs in the UK during 2000-19, and how these differed by sex, age, socioeconomic status, and region.

Data source and study population

We used anonymised electronic health records from the GOLD and AURUM datasets of Clinical Practice Research Datalink (CPRD). CPRD contains information on about 20% of the UK population and is broadly representative of age, sex, ethnicity, geographical spread, and socioeconomic deprivation. 11 12 It is also one of the largest databases of longitudinal medical records from primary care in the world and has been validated for epidemiological research for a wide range of conditions. 11 We used the subset of CPRD records that linked information from primary care, secondary care from Hospital Episodes Statistics (HES admitted patient care and HES outpatient) data, and death certificates from the Office for National Statistics (ONS). Linkage was possible for a subset of English practices, covering about 50% of the CPRD records. Data coverage dates were 1 January 1985 to 31 December 2019 for primary care data (including drug prescription data), 1 April 1997 to 30 June 2019 for secondary care data, and 2 January 1998 to 30 May 2019 for death certificates.

Included in the study were men and women registered with a general practice for at least one year during the study period (1 January 2000 to 30 June 2019) whose records were classified by CPRD as acceptable for use in research and approved for HES and ONS linkage.

Study endpoints

The primary endpoint was the first presentation of CVD as recorded in primary or secondary care. We investigated 10 CVDs: acute coronary syndrome, aortic aneurysm, aortic stenosis, atrial fibrillation or flutter, chronic ischaemic heart disease, heart failure, peripheral artery disease, second or third degree heart block, stroke (ischaemic, haemorrhagic, or unspecified), and venous thromboembolism (deep vein thrombosis or pulmonary embolism). We defined incident diagnoses as the first record of that condition in primary care or secondary care regardless of its order in the patient’s record.

Diseases were considered individually and as a composite outcome of all 10 CVDs combined. For the combined analyses, we calculated the primary incidence (considering only the first recorded CVD in each patient, reflecting the number of patients affected by CVDs) and the total incidence (considering all incident CVD diagnoses in each patient, reflecting the cumulative number of CVD diagnoses). We performed sensitivity analyses including diagnoses recorded on death certificates.

To identify diagnoses, we compiled a list of diagnostic codes based on the coding schemes in use in each data source following previously established methods. 13 14 15 We used ICD-10 (international classification of diseases, 10th revision) codes for diagnoses recorded in secondary care, ICD-9 (international classification of diseases, ninth revision) (in use until 31 December 2000) and ICD-10 codes for diagnoses recorded on death certificates (used in sensitivity analyses only), the UK Office of Population Censuses and Surveys classification (OPCS-4) for procedures performed in secondary care settings, and a combination of Read, SNOMED, and local EMIS codes for diagnoses recorded in primary care records (see supplementary table S1). 16 Supplementary texts S1, S2, and S3 describe our approach to the generation of the diagnostic code list as well as considerations and sensitivity analyses into the validity of diagnoses recorded in UK electronic health records.

We selected covariates to represent a range of known cardiovascular risk factors. For clinical data, including systolic and diastolic blood pressure, smoking status, cholesterol (total:high density lipoprotein ratio), and body mass index (BMI), we abstracted data from primary care records as the most recent measurement within two years before the incident CVD diagnosis. BMI was categorised as underweight (<18.5), normal (18.5-24.9), overweight (25-29.9), and obesity (≥30). Information on the prevalence of chronic kidney disease, dyslipidaemia, hypertension, and type 2 diabetes was obtained as the percentage of patients with a diagnosis recorded in their primary care or secondary care record at any time up to and including the date of a first CVD diagnosis. Patients’ socioeconomic status was described using the index of multiple deprivation 2015, 17 a composite measure of seven dimensions (income, employment, education, health, crime, housing, living environment) and provided by CPRD. Measures of deprivation are calculated at small area level, covering an average population of 1500 people, and are presented in fifths, with the first 20% and last 20% representing the least and most deprived areas, respectively. We extracted information on ethnicity from both primary and secondary care records, and we used secondary care data when records differed. Ethnicity was grouped into four categories: African/Caribbean, Asian, white, and mixed/other. Finally, we extracted information on cardiovascular treatments (ie, aspirin and other antiplatelets, alpha adrenoceptor antagonists, aldosterone antagonists/mineralocorticoid receptor antagonists, angiotensin converting enzyme inhibitors, angiotensin II receptor antagonists, beta blockers, calcium channel blockers, diuretics, nitrates, oral anticoagulants, and statins) as the number of patients with at least two prescriptions of each drug class within six months after incident CVD, among patients alive and registered with a general practitioner 30 days after the diagnosis. Supplementary table S2 provides a list of substances included in each drug class. Prescriptions were extracted from primary care records up to 31 December 2019.

Statistical analyses

Categorical data for patient characteristics are presented as frequencies (percentages), and continuous data are presented as means and standard deviations (SDs) for symmetrically distributed data or medians and interquartile ranges (IQRs) for non-symmetrically distributed data, over the whole CVD cohort and stratified by age, sex, socioeconomic status, region, and calendar year of diagnosis. For variables with missing entries, we present numbers and percentages of records with missing data. For categorical variables, frequencies refer to complete cases.

Incidence rates of CVD were calculated by dividing the number of incident diagnoses by the number of patient years in the cohort. Category specific rates were computed separately for subgroups of age, sex, socioeconomic status, region, and calendar year of diagnosis. Age calculations were updated for each calendar year. To ensure calculations referred to incident diagnoses, we excluded individuals, from both the numerator and the denominator populations, with a disease of interest diagnosed before the study start date (1 January 2000), or within the first 12 months of registration with their general practice. Time at risk started at the latest of the patient’s registration date plus 12 months, 30 June of their birth year, or study start date; and stopped at the earliest of death, transfer out of practice, last collection date of the practice, incidence of the disease of interest, or linkage end date (30 June 2019). Disease incidence was standardised for age and sex 18 using the 2013 European standard population 19 in five year age bands up to age 90 years.

Negative binomial regression models were used to calculate overall and category specific incidence rate ratios and corresponding 95% confidence intervals (CIs). 20 Models were adjusted for calendar year of diagnosis, age (categorised into five years age bands), sex, socioeconomic status, and region. We chose negative binomial models over Poisson models to account for potential overdispersion in the data. Sensitivity analyses comparing Poisson and negative binomial models showed similar results.

Study findings are reported according to the RECORD (reporting of studies conducted using observational routinely collected health data) recommendations. 21 We performed statistical analyses in R, version 4.3.3 (R Foundation for Statistical Computing, Vienna, Austria).

Patient and public involvement

No patients or members of the public were directly involved in this study owing to constraints on funding and time.

A total of 22 009 375 individuals contributed data between 1 January 2000 and 30 June 2019, with 146 929 629 patient years of follow-up. Among those we identified 2 906 770 new CVD diagnoses, affecting 1 650 052 patients. Mean age at first CVD diagnosis was 70.5 (SD 15.0) years, 47.6% (n=784 904) of patients were women, and 11.6% (n=191 421), 18.0% (n=296 554), 49.7% (n=820 892), and 14.2% (n=233 833) of patients had a history of chronic kidney disease, dyslipidaemia, hypertension, and type 2 diabetes, respectively, at the time of their first CVD diagnosis ( table 1 ).

Characteristics of patients with a first diagnosis of CVD, 2000-19. Values are number (percentage) unless stated otherwise

• View inline

During 2017-19, the most common CVDs were atrial fibrillation or flutter (age-sex standardised incidence 478 per 100 000 person years), heart failure (367 per 100 000 person years), and chronic ischaemic heart disease (351 per 100 000 person years), followed by acute coronary syndrome (190 per 100 000 person years), venous thromboembolism (183 per 100 000 person years), and stroke (181 per 100 000 patient years) ( fig 1 ).

Incidence of a first diagnosis of cardiovascular disease per 100 000 person years, 2000-19. Incidence rates are age-sex standardised to the 2013 European standard population. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease). See supplementary table S4 for crude incidence rates by age and sex groups. IRR=incidence rate ratio

• Download figure
• Open in new tab
• Download powerpoint

Temporal trends

The primary incidence of CVDs (ie, the number of patients with CVD) decreased by 20% during 2000-19 (age-sex standardised incidence rate ratio 2017-19 v 2000-02: 0.80 (95% CI 0.73 to 0.88)). However, the total incidence of CVD (ie, the total number of new CVD diagnoses) remained relatively stable owing to an increasing number of subsequent diagnoses among patients already affected by a first CVD (incidence rate ratio 2017-19 v 2000-02: 1.00 (0.91 to 1.10)).

The observed decline in CVD incidence was largely due to declining rates of atherosclerotic diseases, in particular acute coronary syndrome, chronic ischaemic heart disease, and stroke, which decreased by about 30% during 2000-19. The incidence of peripheral artery disease also declined, although more modestly (incidence rate ratio 2017-19 v 2000-02: 0.89 (0.80 to 0.98)) ( fig 1 ).

The incidence of non-atherosclerotic heart diseases increased at varying rates, with incidence of aortic stenosis and heart block more than doubling over the study period (2017-19 v 2000-02: 2.42 (2.13 to 2.74) and 2.22 (1.99 to 2.46), respectively) ( fig 1 ). These increasing rates of non-atherosclerotic heart diseases balanced the reductions in ischaemic diseases so that the overall incidence of CVD across the 10 conditions appeared to reach a plateau and to remain relatively stable from 2007-08 (incidence rate ratio 2017-19 v 2005-07: 1.00 (0.91 to 1.10)) ( fig 2 ).

Age standardised incidence of cardiovascular disease by sex, 2000-19. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease). IRR=incidence rate ratio

Age stratified analyses further showed that the observed decrease in incidence of chronic ischaemic heart disease, acute coronary syndrome, and stroke was largely due to a reduced incidence in those aged >60 years, whereas incidence rates in those aged <60 years remained relatively stable ( fig 3 and fig 4 ).

Sex standardised incidence of cardiovascular disease in all age groups. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease)

Sex standardised incidence of cardiovascular diseases by age subgroups <69 years. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease)

Age at diagnosis

CVD incidence was largely concentrated towards the end of the life span, with a median age at diagnosis generally between 65 and 80 years. Only venous thromboembolism was commonly diagnosed before age 45 years ( fig 5 ). Over the study period, age at first CVD diagnosis declined for several conditions, including stroke (on average diagnosed 1.9 years earlier in 2019 than in 2000), heart block (1.3 years earlier in 2019 than in 2000), and peripheral artery disease (1 year earlier in 2019 than in 2000) (see supplementary figure S1). Adults with a diagnosis before age 60 years were more likely to be from lower socioeconomic groups and to have a higher prevalence of several risk factors, including obesity, smoking, and high cholesterol levels (see supplementary table S3).

Incidence rates of cardiovascular diseases calculated by one year age bands and divided into a colour gradient of 20 quantiles to reflect incidence density by age. IQR=interquartile range

Incidence by sex

Age adjusted incidence of all CVDs combined was higher in men (incidence rate ratio for women v men: 1.46 (1.41 to 1.51)), with the notable exception of venous thromboembolism, which was similar between men and women. The incidence of aortic aneurysms was higher in men (3.49 (3.33 to 3.65)) ( fig 2 ). The crude incidence of CVD, however, was similar between men and women (1069 per 100 000 patient years and 1176 per 100 000 patient years, respectively), owing to the higher number of women in older age groups. Temporal trends in disease incidence were generally similar between men and women ( fig 2 ).

Incidence by socioeconomic status

The most deprived socioeconomic groups had a higher incidence of any CVDs (incidence rate ratio most deprived v least deprived: 1.37 (1.30 to 1.44)) ( fig 6 ). A socioeconomic gradient was observed across almost every condition investigated. That gradient did not decrease over time, and it was most noticeable for peripheral artery disease (incidence rate ratio most deprived v least deprived: 1.98 (1.87 to 2.09)), acute coronary syndrome (1.55 (1.54 to 1.57)), and heart failure (1.50 (1.41 to 1.59)). For aortic aneurysms, atrial fibrillation, heart failure, and aortic stenosis, socioeconomic inequalities in disease incidence appeared to increase over time.

Age-sex standardised incidence rates of cardiovascular diseases by socioeconomic status (index of multiple deprivation 2015). Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease). Yearly incidence estimates were smoothed using loess (locally estimated scatterplot smoothing) regression lines

Regional differences

Higher incidence rates were seen in northern regions (north west, north east, Yorkshire and the Humber) of England for all 10 conditions investigated, even after adjusting for socioeconomic status. Aortic aneurysms and aortic stenosis had the strongest regional gradients, with incidence rates about 30% higher in northern regions compared with London. Geographical variations remained modest, however, and did not appear to change considerably over time (see supplementary figure S2).

Sensitivity analyses

In sensitivity analyses that used broader disease definitions, that included diagnoses recorded on death certificates, that relied on longer lookback periods for exclusion of potentially prevalent diagnoses, or that were restricted to diagnoses recorded during hospital admissions, temporal trends in disease incidence appeared similar (see supplementary figures S3-S6).

Secondary prevention treatments

The proportion of patients using statins and antihypertensive drugs after a first CVD diagnosis increased over time, whereas the use of non-dihydropyridines calcium channel blockers, nitrates, and diuretics decreased over time. Non-vitamin K antagonist oral anticoagulants increasingly replaced vitamin K anticoagulants (see supplementary figure S7).

The findings of this study suggest that important changes occurred in the distribution of CVDs during 2000-19 and that several areas are of concern. The incidence of non-atherosclerotic heart diseases was shown to increase, the decline in atherosclerotic disease in younger people was stalling, and socioeconomic inequalities had a substantial association across almost every CVD investigated.

Implications for clinical practice and policy

Although no causal inference can be made from our data, the decline in rates of ischaemic diseases coincided with reductions in the prevalence of risk factors such as smoking, hypertension, and raised cholesterol levels in the general population over the same period, 22 and this finding suggests that efforts in the primary and secondary prevention of atherosclerotic diseases have been successful. The decline in stroke was not as noticeable as that for coronary heart disease, which may reflect the rising incidence of atrial fibrillation. The variation in trends for peripheral artery disease could be due to differences in risk factors (eg, a stronger association with diabetes), the multifaceted presentations and causes, and the introduction of systematic leg examinations for people with diabetes. 23 24

All the non-atherosclerotic diseases, however, appeared to increase during 2000-19. For some conditions, such as heart failure, the observed increase remained modest, whereas for others, such as aortic stenosis and heart block, incidence rates doubled. All analyses in this study were standardised for age and sex, to illustrate changes in disease incidence independently of changes in population demographics. Whether these trends solely reflect increased awareness, access to diagnostic tests, or even screening (eg, for abdominal aortic aneurysm 25 ) and coding practices, is uncertain. Reductions in premature death from coronary heart disease may have contributed to the emergence of these other non-atherosclerotic CVDs. Regardless, the identification of increasing numbers of people with these problems has important implications for health services, especially the provision of more surgical and transcatheter valve replacement, pacemaker implantation, and catheter ablation for atrial fibrillation. Importantly, these findings highlight the fact that for many cardiovascular conditions such as heart block, aortic aneurysms, and non-rheumatic valvular diseases, current medical practice remains essentially focused on the management of symptoms and secondary prevention and that more research into underlying causes and possible primary prevention strategies is needed. 26 27

These varying trends also mean that the contribution of individual CVDs towards the overall burden has changed. For example, atrial fibrillation or flutter are now the most common CVDs in the UK. Atrial fibrillation is also a cause (and consequence) of heart failure, and these two increasingly common problems may amplify the incidence of each other. Venous thromboembolism and heart block also appeared as important contributors to overall CVD burden, with incidence rates similar to those of stroke and acute coronary syndrome, yet both receive less attention in terms of prevention efforts.

The stalling decline in the rate of coronary heart disease in younger age groups is of concern, has also been observed in several other high income countries, and may reflect rising rates of physical inactivity, obesity, and type 2 diabetes in young adults. 4 6 28 The stalled decline suggests prevention approaches may need to be expanded beyond antismoking legislation, blood pressure control, and lipid lowering interventions to include the promotion of physical activity, weight control, and use of new treatments shown to reduce cardiovascular risk in people with type 2 diabetes. 29 Although CVD incidence is generally low in people aged <60 years, identifying those at high risk of developing CVD at a young age and intervening before problems occur could reduce premature morbidity and mortality and have important economic implications.

Our study further found that socioeconomic inequalities may contribute to CVD burden, and that this association is not restricted to selected conditions but is visible across most CVDs. The reasons behind the observed increase in risk in relation to socioeconomic inequalities are likely to be multifactorial and to include environmental, occupational, psychosocial, and behavioural risk factors, including established cardiovascular risk factors such as smoking, obesity, nutrition, air pollution, substance misuse, and access to care. 30 How these findings apply to different countries is likely to be influenced by socioeconomic structures and healthcare systems, although health inequalities have been reported in numerous countries. 30 One important factor in the present study is that access to care is free at the point of care in the UK, 31 and yet socioeconomic inequalities persist despite universal health coverage and they did not appear to improve over time. Independently of the specificities of individual countries, our findings highlight the importance of measuring and considering health inequalities and suggest that dealing with the social determinants of health—the conditions under which people are born, live, work, and age—could potentially bring substantial health improvements across a broad range of chronic conditions.

Finally, our results reflect disease incidence based on diagnostic criteria, screening practices, availability, and accuracy of diagnostic tests in place at a particular time and therefore must be interpreted within this context. 32 Several of the health conditions investigated are likely to being sought and detected with increased intensity over the study period. For example, during the study period the definition of myocardial infarction was revised several times, 33 34 35 and high sensitivity troponins were progressively introduced in the UK from 2010. These more sensitive markers of cardiac injury are thought to have increased the detection rates for less severe disease. 36 37 Similarly, increased availability of computed tomography may have increased detection rates for stroke. 38 These changes could have masked an even greater decline in these conditions than observed in the present study. Conversely, increased use of other biochemical tests (such as natriuretic peptides) and more sensitive imaging techniques might have increased the detection of other conditions. 39 40 41 The implementation of a screening programme for aortic aneurysm and incentive programmes aimed at improving coding practices, including the documentation of CVD, associated risk factors and comorbidities, and treatment of these, are also likely to have contributed to the observed trends. 25 42 43 As a result, the difference in incidence estimates and prevalence of comorbidities over time may not reflect solely changes in the true incidence but also differences in ascertainment of people with CVD. 44 Nonetheless, long term trends in large and unconstrained populations offer valuable insights for healthcare resource planning and for the design of more targeted prevention strategies that could otherwise not be answered by using smaller cohorts, cross sectional surveys, or clinical trials; and precisely because they are based on routinely reported diagnoses they are more likely to capture the burden of disease as experienced by doctors and health services.

Strengths and limitations of this study

A key strength of this study is its statistical power, with >140 million person years of data. The large size of the cohort allowed us to perform incidence calculations for a broad spectrum of conditions, and to examine the influence of age, sex, and socioeconomic status as well as trends over 20 years. One important limitation of our study was the modest ethnic diversity in our cohort and the lack of information on ethnicity for the denominator population, which precluded us from stratifying incidence estimates by ethnic group. Our analyses were also limited by the unavailability or considerable missingness of additional variables potentially relevant to the development of CVD, such as smoking, body mass index, imaging data, women specific cardiovascular risk factors (eg, pregnancy associated hypertension and gestational diabetes), and blood biomarkers. Further research may also need to consider an even wider spectrum of CVDs, including individual types of valve disease, pregnancy related conditions, and infection related heart diseases. Research using databases with electronic health records is also reliant on the accuracy of clinical coding input by doctors in primary care as part of a consultation, or in secondary care as part of a hospital admission. We therefore assessed the validity of diagnoses in UK electronic health records data and considered it to be appropriate in accordance with the >200 independent validation studies reporting an average positive predictive value of about 90% for recorded diagnoses. 45 Observed age distributions were also consistent with previous studies and added to the validity of our approach. Nevertheless, our results must be interpreted within the context and limitations of routinely collected data from health records, diagnostic criteria, screening practices, the availability and accuracy of diagnostic tests in place at that time, and the possibility that some level of miscoding is present or that some bias could have been introduced by restricting the cohort to those patients with at least 12 months of continuous data.

Conclusions

Efforts to challenge the notion of the inevitability of vascular events with ageing, and evidence based recommendations for coronary heart disease prevention, have been successful and can serve as a model for other non-communicable diseases. Our findings show that it is time to expand efforts to improve the prevention of CVDs. Broadening research and implementation efforts in both primary and secondary prevention to non-atherosclerotic diseases, tackling socioeconomic inequalities, and introducing better risk prediction and management among younger people appear to be important opportunities to tackle CVDs.

What is already known on this topic

Recent data show that despite decades of declining rates of cardiovascular mortality, the burden from cardiovascular disease (CVD) appears to have stalled in several high income countries

What this study adds

This observational study of a representative sample of 22 million people from the UK during 2000-19 found reductions in CVD incidence to have been largely restricted to ischaemic heart disease and stroke, and were paralleled by a rising number of diagnoses of cardiac arrhythmias, valve disease, and thromboembolic events

Venous thromboembolism and heart block were important contributors to the overall burden of CVDs, with incidence rates similar to stroke and acute coronary syndromes

Improvements in rates of coronary heart disease almost exclusively appeared to benefit those aged >60 years, and the CVD burden in younger age groups appeared not to improve

Ethics statements

Ethical approval.

This study was approved by the Clinical Practice Research Datalink Independent Scientific Advisory Committee.

Data availability statement

Access to Clinical Practice Research Datalink (CPRD) data is subject to a license agreement and protocol approval process that is overseen by CPRD’s research data governance process. A guide to access is provided on the CPRD website ( https://www.cprd.com/data-access ) To facilitate the subsequent use and replication of the findings from this study, aggregated data tables are provided with number of events and person years at risk by individual condition and by calendar year, age (by five year age band), sex, socioeconomic status, and region (masking field with fewer than five events, as per CPRD data security and privacy regulations) on our GitHub repository ( https://github.com/nathalieconrad/CVD_incidence ).

Acknowledgments

We thank Hilary Shepherd, Sonia Coton, and Eleanor L Axson from the Clinical Practice Research Datalink for their support and expertise in preparing the dataset underlying these analyses.

Contributors: NC and JJVM conceived and designed the study. NC, JJVM, GM, and GV designed the statistical analysis plan and NC performed the statistical analysis. All authors contributed to interpreting the results, drafting the manuscript, and the revisions. NC, GM, and GV had permission to access the raw data and NC and GM verified the raw data. All authors gave final approval of the version to be published and accept responsibility to submit the manuscript for publication. NC and JJVM accept full responsibility for the conduct of the study, had access to aggregated data, and controlled the decision to publish. They are the guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: This study was funded by a personal fellowship from the Research Foundation Flanders (grant No 12ZU922N), a research grant from the European Society of Cardiology (grant No App000037070), and the British Heart Foundation Centre of Research Excellence (grant No RE/18/6/34217). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: NC is funded by a personal fellowship from the Research Foundation Flanders and a research grant from the European Society of Cardiology. JMF, PSJ, JGC, NS, and JJVM are supported by British Heart Foundation Centre of Research Excellence. PSJ and JJVM are further supported by the Vera Melrose Heart Failure Research Fund. JJVM has received funding to his institution from Amgen and Cytokinetics for his participation in the steering sommittee for the ATOMIC-HF, COSMIC-HF, and GALACTIC-HF trials and meetings and other activities related to these trials; has received payments through Glasgow University from work on clinical trials, consulting, and other activities from Alnylam, Amgen, AstraZeneca, Bayer, Boehringer Ingelheim, Bristol Myers Squibb, Cardurion, Dal-Cor, GlaxoSmithKline, Ionis, KBP Biosciences, Novartis, Pfizer, and Theracos; and has received personal lecture fees from the Corpus, Abbott, Hikma, Sun Pharmaceuticals, Medscape/Heart.Org, Radcliffe Cardiology, Alkem Metabolics, Eris Lifesciences, Lupin, ProAdWise Communications, Servier Director, and Global Clinical Trial Partners. NS declares consulting fees or speaker honorariums, or both, from Abbott Laboratories, Afimmune, Amgen, AstraZeneca, Boehringer Ingelheim, Lilly, Hanmi Pharmaceuticals, Janssen, Merck Sharp & Dohme, Novartis, Novo Nordisk, Pfizer, Roche Diagnostics, and Sanofi; and grant support paid to his university from AstraZeneca, Boehringer Ingelheim, Novartis, and Roche Diagnostics. KK has acted as a consultant or speaker or received grants for investigator initiated studies for Astra Zeneca, Bayer, Novartis, Novo Nordisk, Sanofi-Aventis, Lilly, Merck Sharp & Dohme, Boehringer Ingelheim, Oramed Pharmaceuticals, Roche, and Applied Therapeutics. KK is supported by the National Institute for Health and Care Research (NIHR) Applied Research Collaboration East Midlands (ARC EM) and the NIHR Leicester Biomedical Research Centre (BRC). CL is funded by an NIHR Advanced Research Fellowship (NIHR-300111) and supported by the Leicester BRC. PSJ has received speaker fees from AstraZeneca, Novartis, Alkem Metabolics, ProAdWise Communications, Sun Pharmaceuticals, and Intas Pharmaceuticals; has received advisory board fees from AstraZeneca, Boehringer Ingelheim, and Novartis; has received research funding from AstraZeneca, Boehringer Ingelheim, Analog Devices; his employer, the University of Glasgow, has been remunerated for clinical trial work from AstraZeneca, Bayer, Novartis, and Novo Nordisk; and is the Director of Global Clinical Trial Partners. HS is supported by the China Scholarship Council. Other authors report no support from any organisation for the submitted work, no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work.

Transparency: The lead author (NC) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Dissemination to participants and related patient and public communities: Results from this study will be shared with patient associations and foundations dedicated to preventing cardiovascular diseases, such as the European Heart Network and the American Heart Association. To reach the public, findings will also be press released alongside publication of this manuscript. Social media (eg, X) will be used to draw attention to the work and stimulate debate about its findings. Finally, the underlying developed algorithms will be freely available for academic use at https://github.com/nathalieconrad/CVD_incidence .

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/ .

• ↵ Institute for Health Metrics and Evaluation. Gobal Burden of Diseases Viz Hub. 2023. https://vizhub.healthdata.org/gbd-compare/
• Ananth CV ,
• Rutherford C ,
• Rosenfeld EB ,
• O’Flaherty M ,
• Allender S ,
• Scarborough P ,
• Andersson C ,
• Abdalla SM ,
• Mensah GA ,
• Johnson CO ,
• GBD-NHLBI-JACC Global Burden of Cardiovascular Diseases Writing Group
• Almarzooq ZI ,
• American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee
• Townsend N ,
• Atlas Writing Group, European Society of Cardiology
• Herrett E ,
• Gallagher AM ,
• Bhaskaran K ,
• ↵ Wolf A, Dedman D, Campbell J, et al. Data resource profile: Clinical Practice Research Datalink (CPRD) Aurum. International Journal of Epidemiology. 2019 Mar 11 [cited 2019 Mar 22]; https://academic.oup.com/ije/advance-article/doi/10.1093/ije/dyz034/5374844
• Verbeke G ,
• Molenberghs G ,
• Ferguson LD ,
• ↵ Medicines and Healthcare products Regulatory Agency. What coding systems are used in CPRD data? 2023. https://www.cprd.com/defining-your-study-population
• ↵ Department for Communities and Local Government (DCLG). The English Index of Multiple Deprivation 2015: Guidance. 2015; pp1-7. https://www.gov.uk/government/statistics/english-indices-of-deprivation-2015
• ↵ Kirkwood B, Sterne J. Essential medical statistics. 2010.
• ↵ Eurostat’s task force. Revision of the European Standard Population Report. 2013.
• Benchimol EI ,
• Guttmann A ,
• RECORD Working Committee
• ↵ NHS Digital. Health Survey for England, 2021 part 1. https://digital.nhs.uk/data-and-information/publications/statistical/health-survey-for-england/2021
• Criqui MH ,
• ↵ Health and Social Care Information Centre. Quality and Outcomes Framework - Indicators 2011-12. 2011. https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/quality-and-outcomes-framework-2011-12
• Jacomelli J ,
• Summers L ,
• Stevenson A ,
• Earnshaw JJ
• Vahanian A ,
• Beyersdorf F ,
• ESC/EACTS Scientific Document Group
• Glikson M ,
• Nielsen JC ,
• Kronborg MB ,
• ESC Scientific Document Group
• Stevenson C ,
• Peeters A ,
• Federici M ,
• Schultz WM ,
• Verbakel JY ,
• Thygesen K ,
• Alpert JS ,
• Joint ESC/ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction
• Joint ESC/ACCF/AHA/WHF Task Force for the Universal Definition of Myocardial Infarction
• Sandoval Y ,
• High-STEACS investigators
• Camargo ECS ,
• Singhal AB ,
• Wiener RS ,
• Schwartz LM ,
• Sappler N ,
• Roalfe AK ,
• Lay-Flurrie SL ,
• Ordóñez-Mena JM ,
• Herbert A ,
• Wijlaars L ,
• Zylbersztejn A ,
• Cromwell D ,
• ↵ NHS Digital. Quality and Outcomes Framework (QOF), 2019-20. Indicator definitions. 2020. https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/2019-20
• Pronovost PJ
• Thomas SL ,
• Schoonen WM ,

• Open access
• Published: 20 June 2024

The association of social networks and depression in community-dwelling older adults: a systematic review

• Amelie Reiner   ORCID: orcid.org/0009-0002-8789-1303 1 &
• Paula Steinhoff   ORCID: orcid.org/0009-0001-2973-963X 1  

Systematic Reviews volume  13 , Article number:  161 ( 2024 ) Cite this article

193 Accesses

4 Altmetric

Metrics details

Background and objective

Depression is a globally prevalent mental condition, particularly among older adults. Previous research has identified that social networks have a buffering effect on depression. Existing systematic reviews have either limited their research to specific geographic areas or provided evidence from over a decade ago. The vast body of recent literature particularly from the last decade emphasizes the need for a comprehensive review. This systematic review aims to analyze the association of structural aspects of social networks and depression in older adults.

The electronic databases APA PsycINFO, ProQuest, PSYINDEX, PubMed, Scopus, SocINDEX, and Web of Science were searched from date of data base inception until 11 July 2023. Studies were eligible for inclusion if they reported on community-dwelling older adults (defined as a mean age of at least 60 years old), had an acceptable definition for depression, referred to the term social network in the abstract, and were published in English. Quality was appraised using the Newcastle Ottawa Scale for cross-sectional and longitudinal studies. Outcome data were extracted independently from each study and analyzed by direction of the relationship, social network domain and cross-sectional or longitudinal study design.

In total, 127 studies were included. The study categorizes structural network aspects into seven domains and finds that larger and more diverse networks, along with closer social ties, help mitigate depression. The literature on the relationships between depression and network density, homogeneity, and geographical proximity is scarce and inconclusive.

Discussion and implications

Despite inconsistent findings, this review highlights the importance of quantifying complex social relations of older adults. Limitations of this review include publication and language bias as well as the exclusion of qualitative research. Further research should use longitudinal approaches to further investigate the reciprocal relationship between social networks and depression. Following this review, interventions should promote the integration of older adults in larger and more diverse social settings.

Other: This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grant [454899704]. This systematic review was pre-registered. The review-protocol can be accessed at https://doi.org/10.17605/OSF.IO/6QDPK .

Peer Review reports

Depression is a mental condition that is particularly prevalent among older adults [ 1 ]. Scholars have identified a significant association between social networks and depression, with socially integrated older adults showing lower levels of depression than less socially integrated older adults [ 2 , 3 ]. As older adults face a decreasing number of social relationships and a shrinking social network over their life course [ 4 ], this growing population is at risk for depression. Systematizing and quantifying the social networks of older adults is vital to understanding their relationship with depression. The prevalence of depression will increase in the future. Understanding the aspects of social networks that are particularly important for preventing depressive symptomatology in older adults will allow appropriate social gerontological interventions.

Previous systematic reviews have generated important insights into the relationship between social networks and mental health. Across several geographical areas, various social network measures have been found to be significantly associated with mental health in older adults (Middle Eastern countries: [ 5 ]; Iran: [ 6 ]), and specifically depression (Asia: [ 2 ]; Western countries: [ 7 ]). However, only one systematic review has addressed the relationship between social networks and depression among older adults without restricting its evidence to a geographical area [ 3 ]. While Schwarzbach et al.’s [ 3 ] review has been helpful, new evidence about the social relations of older adults and depression outcomes must be reviewed because a significant amount has emerged over the last decade.

Additionally, previous studies and literature reviews have loosely applied the concept of social networks and engaged with different definitions and measures of social networks [ 8 , 9 ]. A social network is traditionally defined as the quantifiable ties binding individuals, families, communities, or businesses (i.e., nodes) together through a shared need, aim, or interest [ 10 , 11 ]. The nature of one’s social network was found to have a significant influence on an individual’s life expectancy, mortality rate, quality of life, and health-related behaviors [ 8 ]. Generally, the literature has distinguished between the quantitative/structural and qualitative/functional aspects of social relationships [ 12 , 13 ]. Qualitative aspects refer to the social network’s function, including the potential of social relationships, such as social support, the perceived quality of support provided, relationship satisfaction, loneliness and social isolation [ 13 , 14 ]. In contrast, quantitative aspects refer to the network’s structure, including its size, composition, and the frequency of contact between network members. Recently, it has become increasingly clear that quantifying social networks, which provides an objective measure of the structure of relationships, is particularly suited for understanding their association with critical health outcomes, such as cognitive decline [ 14 ], dementia [ 15 ], and mortality [ 16 ]. As structural aspects of social networks are causally prior to functional aspects, this review exclusively focuses on their structural aspects while examining their relationship with depression in older adults.

The relationship between social networks and depression can be considered reciprocal. The main effect model [ 17 ] states that social networks positively affect psychological state through mechanisms such as social recognition, a sense of belonging, and normative guidance for health-promoting behavior. Conversely, depression may affect the extent of social networks by causing social withdrawal and decreased social participation. Older adults who experience depression in later life often struggle with maintaining larger and more diverse personal networks and experience disruptions in their contact with social network members [ 18 ]. Existing research has predominantly focused on the effect of social networks on depression. Conversely, the reversed effect of depression on social networks has been largely neglected [ 19 , 20 ].

This systematic review, therefore, aims to synthesize the evidence about the relationship between structural aspects of social networks and depression in community-dwelling older adults. It addresses two research questions: (1) How do structural aspects of social networks impact depression outcomes in community-dwelling older adults? (2) How does depression impact structural aspects of social networks of community-dwelling older adults? It strives to provide a comprehensive picture by gathering cross-sectional as well as longitudinal evidence and by focusing on the reciprocal relationship between social networks and depression in older adults.

This systematic review was pre-registered. The review-protocol can be accessed at https://doi.org/10.17605/OSF.IO/6QDPK . In addition, we followed PRISMA guidelines for the reporting of this systematic review ([ 21 ]; see Additional file 1, Table A1).

Eligibility criteria

We expected to include peer-reviewed articles on the association of structural social network characteristics and depression among community-dwelling older adults. Following the World Health Organization (WHO; [ 22 ]), we define older adults as those, being 60 years and older. To counteract possible regional selection bias induced by language knowledge, we focused on English publications only. We did not exclude studies based on publication year or geographic area.

Related previous systematic reviews informed the inclusion and exclusion criteria [ 2 , 3 , 5 , 6 , 7 , 8 , 13 , 23 , 24 , 25 ]. Articles were included if the population of interest consisted of community-dwelling adults, specifically those older than 40 years, with a study mean age of at least 60 years. We opted for a minimum age in order to include relevant age studies from the age of 40 (e.g., the German DEAS), but focused on older adults by deciding that the mean age of the study participants must be at least 60 years, following the definition of older adults. The exposure or outcome focused on social networks, explicitly mentioned in the abstract of the studies. Further exposure or outcome of interest was depression, with an acceptable definition involving diagnostic criteria or a cut-off point on a depression rating scale. The association between social networks and depression had to be reported using a multivariate analysis adjusting for any confounders (the specifics of the included confounders are evaluated in the quality assessment). Only peer-reviewed journal articles published in English were considered for inclusion. Articles were excluded if they focused on patient groups or included institutionalized individuals, unless the analyses separated community-dwelling and institutionalized participants. Additionally, studies were excluded if they referred to recalled social network characteristics from the past, such as youth and adolescence, to measure present depression outcomes, or if they exclusively focused on online social networks. In terms of study types, editorials, study protocols, conference proceedings, comments, reviews, qualitative studies, grey literature, case studies, and intervention studies were excluded. An overview of the studies that appeared to meet the inclusion criteria but were ultimately excluded and the reasons for this can be found in the Additional file 1, Table A2.

Search strategy

The systematic database search was performed from date of data base inception up to 11 July 2023. The keywords used for the search strategy included related terms for: “depression” AND “social networks” AND “older adults” (see pre-registered review protocol). These were informed by related systematic reviews about the three main terms [ 2 , 3 , 5 , 6 , 7 , 8 , 13 , 23 , 24 , 25 ]. The following seven databases were searched using the same keywords and search designs: APA PsycINFO, ProQuest, PSYINDEX, PubMed, Scopus, SocINDEX, and Web of Science. We also conducted manual searches for potentially eligible studies from reference lists of related systematic reviews [ 2 , 3 , 5 , 6 , 7 , 8 , 13 , 23 , 24 , 25 ].

Study selection

References from the seven databases were imported into Rayyan [ 26 ]. After deduplication, two researchers (AR, PS) independently screened titles and abstracts, forwarding potentially eligible papers for full text review. Two researchers (AR, PS) independently assessed the full text of potentially eligible citations against the eligibility criteria. Disagreements and discrepancies were resolved by consensus between the researchers. The study selection process was piloted twice with a random sample of a hundred studies of the overall sample per pilot. Piloting the study selection process improves the reliability and validity of the review by ensuring all reviewers have a clear and consistent understanding of the selection process [ 27 ].

Data extraction

Using a standardized data collection form informed by related reviews [ 2 , 3 , 5 , 6 , 7 , 8 , 13 , 23 , 24 , 25 ], two reviewers (AR, AL) independently extracted data on the study population including their sample size, average age and age range, gender ratio, and country. Further, we extracted information on the measurement of depression, the social network assessment, type of social ties, potential exclusion of population groups, data source, the statistical methods, and the results. The outcomes of interest were structural aspects of social networks and/or depression scores among community-dwelling older adults. Any disagreements were resolved by discussion. If this failed, a third reviewer (PS) was consulted. The data extraction process was piloted once with a random sample of twenty studies to ensure the completeness of all relevant information in the data collection form [ 28 ].

Quality appraisal

Quality was assessed using the Newcastle Ottawa Scale (NOS; [ 29 ]) for cross-sectional and longitudinal studies by one reviewer (AR) and double-checked by another reviewer (PS). The NOS has been used in systematic reviews before [ 2 , 30 , 31 , 32 ]. The NOS awards each article an amount of stars within three domains, with a greater number of stars indicate a higher‐quality study [ 29 ]. The study quality is evaluated in terms of design, participant selection, comparability and assessment of exposure and outcome. Following the approach of several reviews [ 2 , 31 , 32 ], we adopted a rigorous methodology to assess the quality of studies, adhering to predetermined thresholds for converting the NOS to Agency for Health Research and Quality (AHRQ) standards. For a cross-sectional study to be considered of good quality, it needed to attain between 3 and 5 stars in the selection domain, alongside 1 or 2 stars in the comparability domain, and finally, 2 or 3 stars in the outcome domain. Those studies that achieved 2 stars in the selection domain, coupled with 1 or 2 stars in comparability, and 2 or 3 stars in outcome were classified as fair quality. However, studies falling short of these criteria were deemed poor quality; they either obtained 0 or 1 star in the selection domain, 0 stars in comparability, or 0 or 1 stars in outcome. In contrast, a longitudinal study was considered of good quality if it garnered between 3 and 4 stars in the selection domain, along with 1 or 2 stars in the comparability domain, and finally, 2 or 3 stars in the outcome domain. Those longitudinal studies achieving 2 stars in the selection domain, paired with 1 or 2 stars in comparability, and 2 or 3 stars in outcome were categorized as fair quality. Conversely, studies failing to meet these benchmarks were classified as poor quality; they either received 0 or 1 star in the selection domain, 0 stars in comparability, or 0 or 1 stars in outcome. For the analyses, we included all studies irrespective of the quality assessment results. However, when excluding studies which were considered as poor quality in a sensitivity analysis, the results were found to remain largely stable.

Synthesis method

Citations were firstly sub-grouped by direction of the relationship, then by structural aspect of social networks, and afterwards by the cross-sectional or longitudinal study design. In a further step, we count the significant associations against the insignificant associations. We compare the significant results across study design to identify differences between cross-sectional and longitudinal relationships. Further, we compare the effects of interest across structural aspects of social networks in the discussion. Tables are used to display the sub-grouped evidence. Further comparisons were carried out by geographical location, gender, family versus friends’ social ties and functional versus structural social network aspects. Findings are reported narratively.

Sample description

Starting from an initial result of 47,702 entries, 26,915 unique citations were identified. The two authors (AR, PS) independently screened the titles and abstracts, resulting in 320 potentially eligible articles. Any disagreement over the eligibility of individual studies was resolved through discussion. After adhering to strict inclusion and exclusion criteria, 127 unique publications were identified. Figure 1 Visualizes a PRISMA flowchart of the selection process.

HYPERLINK "sps:id::fig1||locator::gr1||MediaObject::0"Selection flowchart for papers included in the systematic review

The quality appraisal for each NOS-domain and overall evaluation can be found in the Additional file 1, Table A3 for cross-sectional studies and Table A4 for longitudinal studies. Two thirds of the studies ( n  = 86) were classified as good-quality studies, 27 articles with fair quality and 15 articles with poor quality.

The included articles were published between 1985 and 2023, with half published later than 2016. This highlights the vast body of research that has been conducted on this association, particularly in the last decade. The range of sample sizes was 53 to 60,918, with a median sample size of 1349 respondents. The geographic location of most of the studies was North America ( n  = 46), followed by Asian countries ( n  = 42). Thirty-four studies were conducted in European countries (and Israel), and only three were conducted in South American countries. One study has a mixed geographical location by comparing older adults in North America to those in Asia [ 33 ]. One study did not specify its geographic location [ 34 ].

The majority of studies made use of validated instruments to assess particularly depression. They either used various forms of the Center for Epidemiologic Studies Depression Scale (CES-D, n  = 58) or the Geriatric Depression Scale (GDS, n  = 42) to assess depression. Other studies used the EURO-D scale ( n  = 12), the Composite International Diagnostic Interview (CIDI, n  = 3), the nine-item Patient Health Questionnaire (PHQ-9, n  = 3), or other validated instruments ( n  = 9).

Most studies focused on the cross-sectional relationship between the social networks of older adults and depression ( n  = 96), while 30 articles examined the relationship longitudinally. Only one article had both a cross-sectional and longitudinal focus [ 35 ]. In most aspects of social networks, there were no apparent differences between the cross-sectional and longitudinal investigations. Additionally, 90% ( n  = 114) of the studies exclusively used depression as an outcome variable, while 6% ( n  = 8) exclusively used social network variables as outcome variables. Only five studies focused on the existence of a bi-directional relationship [ 19 , 20 , 36 , 37 , 38 ].

All risk factors for depression related to social networks used within the studies were categorized. Seven structural aspects of social networks were identified: network composition, contact frequency, network density, homo-/heterogeneity, network size, geographic proximity, and network scales. Table 1 provides an overview of the social network aspect descriptions. Notably, ties to friends and family were the covered most frequently in social network measures. The results were largely stable across geographic areas.

Depression as outcome variable

In total, 119 articles examined structural network aspects’ effects on depression. Ninety articles did so cross-sectionally, and 28 articles did so longitudinally. One article focused on the relationship both cross-sectionally and longitudinally [ 35 ].

Most publications focused on network scales ( n  = 44), network size ( n  = 44), network composition ( n  = 30), and contact frequency ( n  = 28) as structural network factors determining depression outcomes in older adults. Significantly fewer articles used density ( n  = 4), geographic proximity ( n  = 3), and homogeneity ( n  = 2). The results are presented below according to their frequency.

Network scales

Some articles used standardized network scales to examine various aspects of social networks’ effects on depression among older adults. Most articles used (modifications or translations of) the Lubben Social Network Scale (LSNS) or the Social Network Index (SNI), with higher scores indicating greater social engagement.

Most associations (40 out of 60 = 67%) between network scales and depression among older adults were reported to be significant (Table  2 ). No meaningful difference was identified between cross-sectional and longitudinal studies concerning effect significance or direction. Consistently, scholars found higher scores on social network scales to buffer depression outcomes among older adults. However, different subscales were used to assess family and friends variables. While some studies suggested that family networks were more predictive of depression outcomes in older adults [ 41 , 42 , 43 ], Singh et al. [ 44 ] indicated the opposite, suggesting that the friend network scale was significantly associated with depression. They found no significant associations in the children, relatives, and confidant network scales.

The results appear to be largely stable across gender. Most of the studies considering gender differences did not find the association of network scales and depression to differ in women and men [ 43 , 50 , 60 , 66 ]. The evidence of studies finding gender differences is inconclusive. While two studies found network scales to be only significant associated with depression in men but not women [ 68 , 80 ], another study found a significant association for the friends’ subscale in women but not men [ 47 ]. Conversely, no gender differences were found regarding the family subscale [ 47 ].

Network size

Network size was the most frequently studied variable besides network scales. In total, 66 measured associations were found in 44 articles (see Table  3 ). No meaningful difference was identified between cross-sectional and longitudinal studies concerning effect significance or direction. The results were inconclusive: Half of the studies found no significant association, while the other half provided significant evidence for an effect of social network size on depression in older adults. Of the effects significantly associated with depression, 32 of 33 were negative. This suggests that more extensive social networks are associated with lower levels of depression in older adults.

There seems to be no consensus regarding the association of the size of different social spheres and depression outcomes among older adults. While Palinkas et al. [ 64 ] and Harada et al. [ 96 ] found friend network size to be more important than relative network size, Lee and Chou [ 98 ] found these variables to be equally important. Furthermore, Minicuci et al. [ 103 ] and Oxman et al. [ 114 ] found them equally unimportant for depression outcomes.

There also seems to be no consensus regarding gender differences in the association of network size and depression. While two scholars found a significant association of network size and depression only in women but not men [ 83 , 111 ], three scholars found no evidence for gender differences [ 91 , 104 , 106 ]. Minicuci et al. [ 103 ] found the numbers of relatives with close contacts to only be significantly associated with depression in women but not men, while the number of close contacts was significantly associated with depression in men and women.

Network composition

Network composition was primarily measured by forming network typologies through clustering (see Table  4 ). This method makes it particularly challenging to compare results; however, studies consistently showed that diverse social networks protect against depression compared to more restricted networks [ 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 ]. Concerning network transitions, individuals remaining in and changing to restricted networks showed significantly higher levels of depression than those remaining in non-restricted networks [ 130 , 131 ]. Consistently, Sicotte et al. [ 132 ] found that an increasing diversity of links (measured by diversity of relationship ties) was associated with lower odds of depressive symptoms. Other studies found no significant association [ 105 , 110 ]. When prestige occupation scores were used as a diversity measure, higher diversity was associated with lower levels of depression compared to less diverse networks [ 133 ]. Conversely, Becker et al. [ 83 ] found diverse networks to be less associated with a lack of depressive symptoms compared to those relying solely on their partner as their social network.

Some studies included the share of particular social aspects, such as gender, family, or friends. Consistently, the proportions of females or kin were not identified as significant predictors of depression [ 19 , 100 , 107 , 138 ]. Furthermore, there was no consensus about the composition of family and friends. Social networks primarily consisting of family were found to buffer depression more than networks primarily consisting of friends [ 82 , 87 ]. This was also the case for network transitions [ 140 ]. Conversely, Fiori et al. [ 121 ] found that the absence of family within a friend context was less detrimental than the absence of friends within a family context. Also, Chao [ 109 ] identified that a network proportion of 25–50% family and 50–75% friends was the most advantageous for preventing depression.

While two scholars found no evidence for gender differences in the association of network composition and depression in older adults [ 132 , 136 ], Choi and Jeon [ 120 ] identified gender-specific network types and their association with depression to differ by gender. They found that restricted social network types were associated with increased depressive symptoms in both men and women, whereas a family-centered network was associated with more depressive symptoms only in women.

Contact frequency

Less consistency was found in social interaction frequency’s influence on depression in older adults (see Table  5 ). The cross-sectional studies found 14 significant and 15 insignificant associations. In contrast, among the longitudinal studies, only one significant piece of evidence was found [ 109 ], while six effects were identified as insignificant. Three effects were found to be significant only in certain population groups [ 141 , 142 ]. Furthermore, Blumstein et al. [ 35 ] found a significant negative association between weekly contact with friends and children and depression cross-sectionally; this became insignificant when examined longitudinally. Although cross-sectional results are inconclusive, this could indicate that the frequency of contact has the potential to buffer depression at the time of the event but is not necessarily a sustainable buffer for depression.

There was no consensus among studies about the association of depression with contact frequencies in particular social spheres, such as friends, children, and non-kin [ 35 , 64 , 87 , 97 , 99 , 109 , 141 , 142 , 143 , 144 , 145 , 149 ]. Chi and Chou [ 87 ] found contact frequency with relatives to be more advantageous in buffering depression than the frequency of contact with friends. In contrast, Jeon and Lubben [ 97 ] found contact frequency with non-kin to be negatively associated with depressive symptoms in older Korean immigrants, while contact frequency with kin was not significantly associated.

Only two scholars accounted for gender differences in the association of contact frequency and depression among older adults. Ermer and Proulx [ 91 ] found no significant association of contact frequency and depression in women or men. In their cross-sectional analysis, Blumstein et al. [ 35 ] also found no gender differences in the association between weekly contact with children and depression, but identified weekly contact with friends to only be significantly associated with depression in women but not men. However, these gender differences did not hold longitudinally.

Four articles examined how social network density was associated with depression in older adults (see Table  6 ). The results were inconclusive, cross-sectionally as well as longitudinally. Coleman et al. [ 110 ] and Vicente and Guadalupe [ 107 ] found no significant associations. Furthermore, the significant associations found were contradictory even though the same data and measurements were used. Dorrance Hall et al. [ 90 ] found that confidant network density was negatively associated with levels of depression cross-sectionally. In contrast, Bui [ 19 ] conducted a longitudinal study and found that a higher network density was significantly associated with increased depressive symptoms.

Geographic proximity

Three cross-sectional articles considered geographical proximity as a social network determinant for depression among older adults (see Table  7 ). No study focused on the respective relationship longitudinally. All the articles found significant but inconclusive results. While Litwin et al. [ 102 ] and Vicente and Guadalupe [ 107 ] found that geographically closer social networks buffer depression, Becker et al. [ 83 ] identified that geographically closer social networks increased depression. This may be attributable to the measurement used to assess geographic proximity: Litwin et al. [ 102 ] included individuals living within the respondent’s household, while Becker et al. [ 83 ] did not. This strongly suggests that the direction of effects is dependent on operationalization.

Homogeneity

Furthermore, two cross-sectional studies examined homo-/heterogeneity (see Table  8 ). Their evidence suggested no significant relationship between network homo-/heterogeneity and depression among older adults. Goldberg et al. [ 94 ] determined network homogeneity through questions about the sex, age, and religion of all network members. They found no significant association with depression. Murayama et al. [ 151 ] measured homo-/heterogeneity through respondents’ perceptions of the (dis)similarity of characteristics. They found a significant negative association with depression. This was only found for individuals with a strongly homogenous network and not for those with a weakly homogenous network. No significant relationship was found between network heterogeneity and depression outcomes.

Structural social network variables as outcome variable

Thirteen studies focused on social networks as outcome variables of depression (see Table  9 ). Seven articles examined this association cross-sectionally, while six articles did so longitudinally.

The articles examining the relationship between depression and social networks specifically focused on social network scale outcomes, network size, network composition, density, and contact frequency.

Evidence about the relationship between depression and network scales was mixed. While Merchant et al. [ 154 ] found no evidence cross-sectionally, other scholars found significant evidence that depression was associated with lower scores on network scales [ 37 , 153 , 159 ] and subscales [ 156 ]. However, the longitudinal evidence found was contradictory [ 20 , 36 ].

Depression was primarily identified as a significant predictor for network size. This was found cross-sectionally [ 155 ] and longitudinally [ 19 , 157 , 158 ]. Shouse et al. [ 155 ] found depression to be a predictor for a smaller inner circle network size. Furthermore, Bui [ 19 ] found that depressive symptoms significantly affected an individual’s number of close ties but not total social network size. In contrast, Houtjes et al. [ 157 ] examined differences in network size depending on depression course types. They found decreasing network sizes for all depression course types in older adults.

Cross-sectionally, Ali et al. [ 152 ] found that individuals with more depressive symptoms had smaller and more strained networks. Bui [ 19 ] did not identify depressive symptoms as a significant predictor of the proportion of females in an individual’s network.

No significant evidence suggested that depression affects contact frequency [ 19 , 158 ].

Network density

Bui [ 19 ] did not find depressive symptoms to significantly predict network density.

Reciprocal relationship of structural network aspects and depression

Only five articles examined the relationship between structural network aspects and depression reciprocally [ 19 , 20 , 36 , 37 , 38 ]. However, no reciprocal relationship was found between depression and network size [ 19 , 38 ], composition [ 19 ], contact frequency [ 19 ], and network scales [ 20 , 36 , 37 ]. Bui [ 19 ] only identified greater network density to significantly reduce depressive symptoms 5 years later, but not the other way around. Network size, number of close ties, contact frequency, or network composition did not significantly affect depressive symptoms 5 years later. Furthermore, Domènech-Abella et al. [ 20 ] found that the social network index significantly affects depression longitudinally; however, this relationship was not reciprocal. In contrast, Zhang et al. [ 36 ] found that higher depression scores at baseline predicted lower social network scores at a 6-month follow-up. However, social network scores did not predict depression at a 6-month follow-up. Bui [ 19 ] found more depressive symptoms to be associated with fewer close ties 5 years later. However, all other structural network measures (network size, composition, and contact frequency) were insignificant; therefore, the author concluded that there was no clear reciprocal relationship between structural network measures and depression [ 19 ].

Importance of functional network aspects

Thirty articles included social support in their analysis and examined whether social networks’ structural or functional aspects were more important in predicting depression outcomes in older adults. Singh et al.’s [ 44 ] article was excluded because social support measures’ effect sizes and significance were not presented.

However, no consensus can be reached. Seven studies identified structural aspects as more critical in predicting depression in terms of significant effects [ 35 , 53 , 54 , 74 , 98 , 106 , 117 ], while nine scholars found social support to be more relevant [ 34 , 62 , 82 , 95 , 107 , 108 , 110 , 114 , 129 ]. Sixteen studies found that social support and social network aspects were equally (not) predictive of depressive symptoms [ 19 , 80 , 85 , 86 , 87 , 90 , 92 , 103 , 109 , 118 , 122 , 132 , 133 , 136 , 138 , 142 ].

Social network characteristics and depression among older adults

This study aimed to systematize the evidence about the relationship between social networks and depression in older adults. It focused on the structural aspects of social networks because these are particularly suited for understanding their association with critical health outcomes [ 14 , 15 , 16 ]. It differentiated between the causality of relationships and structural and functional social network characteristics’ impact on depression.

Most articles followed the main-effect model [ 17 ] and considered depression as an outcome variable of social network characteristics in examining the relationship between structural social network aspects and depression among older adults. Only eight articles exclusively accounted for the reversed logic of causality: social network characteristics as an outcome of depression [ 152 , 153 , 154 , 155 , 156 , 157 , 158 , 159 ]. Five out of 127 articles examined the reciprocal relationship between structural social network characteristics and depression [ 19 , 20 , 36 , 37 , 38 ]. However, these articles found no clear reciprocal relationship. Therefore, no theoretical conclusions can be drawn based on these findings.

The majority of articles focused on depression as an outcome of older adults’ social network characteristics. They primarily used cross-sectional evidence. Structural network characteristics were predominantly operationalized through network scales, size, composition, and contact frequency. Conversely, they generally neglected network density, homogeneity, and geographical proximity. Evidence about whether and how the latter three social network aspects affect depression outcomes in older adults was inconsistent [ 19 , 83 , 90 , 94 , 102 , 107 , 110 , 151 ]. Most evidence supported the assumption that higher scores on social network scales buffer depression [ 20 , 37 , 41 , 42 , 43 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 56 , 57 , 58 , 61 , 63 , 64 , 65 , 66 , 68 , 69 , 70 , 71 , 72 , 74 , 75 , 76 , 79 , 80 , 81 ]. Corroborating previous literature reviews [ 2 , 13 ], some evidence suggested that a more extensive network size buffers depression outcomes in older adults compared to a smaller network size [ 33 , 64 , 78 , 82 , 83 , 85 , 86 , 87 , 90 , 92 , 93 , 94 , 96 , 98 , 99 , 100 , 101 , 102 , 106 , 109 , 112 , 114 , 115 , 117 , 119 ]. Three quarters of the studies also identified that network composition was significantly associated with depression outcomes in older adults; diverse social networks were found to be more beneficial than restricted networks [ 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 131 ]. This aligns with Santini et al.’s [ 13 ] findings, who consistently identified diverse types of social networks as associated with favorable depression outcomes. Results on the effect of contact frequency on depression were less consistent: no clear evidence was found cross-sectionally, and no substantial effects of contact frequency were found in longitudinal studies. This confirms Schwarzbach et al.’s [ 3 ] findings, which reported inconsistent results cross-sectionally and longitudinally.

Furthermore, the effects of social network aspects on depression seem to be largely stable for women and men [ 35 , 43 , 47 , 50 , 60 , 66 , 68 , 80 , 83 , 91 , 103 , 104 , 106 , 111 , 120 , 132 , 136 , 151 ]. Notably, no consensus can be reached about whether family or friends are more critical for favorable depression outcomes in older adults [ 41 , 42 , 43 , 44 , 82 , 87 , 109 , 121 , 140 ]. This challenges the previous assumption that family is the most crucial source of good health [ 160 ].

A minority of articles found social network characteristics to be outcomes of depression. While depression did not influence density [ 19 ] and contact frequency [ 19 , 158 ], an unclear effect was found for network scales [ 20 , 36 , 37 , 153 , 154 , 156 , 159 ] and network composition [ 19 , 152 ]. However, depression significantly reduced the size of an individual’s social network and their number of close relationships [ 19 , 155 , 157 , 158 ].

This review does not confirm the previous systematic reviews’ findings [ 3 , 13 ] that social networks’ functional aspects are more important than their structural aspects in predicting depression. The articles that considered functional network characteristics showed no consensus about whether structural or functional network aspects were more important in buffering depression outcomes in older adults [ 19 , 34 , 35 , 53 , 54 , 62 , 74 , 80 , 82 , 85 , 86 , 87 , 90 , 92 , 95 , 98 , 103 , 106 , 107 , 108 , 109 , 110 , 114 , 117 , 118 , 122 , 129 , 132 , 133 , 136 , 138 , 142 ].

Furthermore, very few studies reported effect sizes. However, the studies that reported standardized coefficients almost exclusively identified small effect sizes across all structural social network aspects [ 41 , 43 , 47 , 51 , 52 , 53 , 54 , 55 , 56 , 58 , 59 , 61 , 63 , 64 , 65 , 66 , 85 , 86 , 87 , 93 , 96 , 99 , 101 , 102 , 104 , 112 , 120 , 121 , 123 , 125 , 126 , 128 , 129 , 133 , 137 , 139 , 140 , 147 , 153 , 159 ]. Although the studies covered a wide sample size range, there were no differences in the results. This suggests that structural network aspects have a rather small but stable influence on depression. However, future studies should report effect sizes (e.g., by standardized coefficients) to ensure the comparability of studies and individual effects.

Limitations and future implications

This systematic review is the first to specifically focus on the relationship between structural social network aspects and depression outcomes among older adults. While previous systematic reviews have been helpful, they have loosely applied the constructs of social networks and limited their focus to particular geographic areas. Additionally, the vast body of evidence that has emerged during the last decade highlights the importance of this updated systematic review. However, our review has some limitations. Like other reviews, the articles included in this review may be prone to publication bias. In addition, we did not use controlled vocabulary terms such as MeSH and Psychological Index Terms in our search strategy. As our search strategy and keywords were informed by other reviews [ 2 , 3 , 5 , 6 , 7 , 8 , 13 , 23 , 24 , 25 ], we used a diverse range of keywords relevant to the field. Our comprehensive search strategy is reflected in the high number of initial articles found. Consequently, we anticipate having identified all relevant articles. Furthermore, we only included articles published in English, neglecting the findings reported in different languages. However, we did this to counteract possible regional bias induced by language knowledge of the authors. Additionally, the exclusion of non-English articles was found to have minimal impact on the results and overall conclusions of a review [ 161 , 162 ]. However, future research could employ machine translation to counteract selection bias induced by language restrictions. This should be particularly beneficial in contexts in which limited evidence exists.

Further, it must be emphasized that we focused on community-dwelling older adults, excluding institutionalized individuals from analysis. It should be acknowledged that regional bias may arise, given the different proportions of older adults living in institutions across countries. However, we decided to do this as institutionalized individuals are likely to have predetermined social networks which may affect depression outcomes differently.

Additionally, the use of the term “social network” may exclude studies focusing solely on family networks, which are highly relevant for the mental health of older adults. However, as the individual network should not be limited to family networks alone, we have deliberately opted for the holistic term here, to capture the social network in its entirety. This approach is supported by the ambiguous results on the importance of family and friendship relationships for depression among older adults (see analysis above).

Furthermore, this systematic review included studies from peer-reviewed journals, excluding gray literature. This may limit our findings. However, it ensures that the included articles are high quality. Furthermore, systematic reviews do not allow qualitative studies to be included. While qualitative studies are limited in their potential to establish causal relationships between variables, they provide valuable insights into the understanding and interpretation of psychosocial phenomena that quantitative research often cannot access.

This systematic review aimed to understand the potential of structural social network characteristics holistically by reviewing them all and not limiting the focus on only a few. That is why we did not conduct a meta-analysis. Firstly, evidence is too small to be statistically analyzed, such as in the social network domains network density, homogeneity, and geographical proximity. Secondly, particularly in the social network domain composition, results are not necessarily comparable since cluster analysis results in different numbers of clusters which are consequently characterized differently. However, future research should conduct a meta-analysis with the more comparable domains network scale, size, and contact frequency.

Despite this review’s limitations, its strength lies in its systematic search; multiple keywords and broad terminologies were used to capture as many articles as possible. This is reflected in the significant number of publications included in this review.

Much of the evidence reported here came from cross-sectional studies. Additionally, only eight of the 127 articles exclusively considered social networks as dependent variables, and only five studies examined the reciprocal relationship. This makes it particularly difficult to draw causal conclusions about the relationship between social networks and depression among older adults. Further research is needed to disentangle the reciprocal relationship using longitudinal data. Furthermore, limited literature focused on the relationship between depression and network density, homogeneity, and geographical proximity. Additionally, these results were inconclusive. Therefore, these relationships should be closely examined in future research.

This review gathered evidence and confirmed that having larger and more diverse social networks and closer ties buffers depression among older adults. Evidence about the relationship between contact frequency and depression was inconclusive. Literature on the relationships between depression and network density, homogeneity, and geographical proximity is scarce and inconclusive; therefore, further research is needed. Although this review examined a vast body of research about the relationship between social network aspects and depression among older adults, no conclusions about causality could be drawn. Contrary to other reviews, the evidence suggests that functional and structural networks are equally important in determining depression outcomes in older adults.

This review highlights that quantifying older adults’ social relations is crucial to understanding depression outcomes in older adults. As the population ages and multimorbidity and social isolation increase, appropriate social gerontological interventions are needed. Based on this review, interventions could potentially promote the integration of older adults into larger and more diverse social settings. Following the recommendations of a systematic review about the effectiveness of interventions targeting social isolation in older adults [ 163 ], group interventions like social activities are the most effective in broadening older adults’ social networks and increasing their contacts. These interventions can help to counteract depression in older adults.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. All data generated or analyzed during this study are included in this published article.

World Health Organization. Depression. 2021. Available from: https://www.who.int/news-room/fact-sheets/detail/depression

Mohd T, Yunus R, Hairi F, Hairi N, Choo W. Social support and depression among community dwelling older adults in Asia: a systematic review. BMJ OPEN. 2019;9(7).

Schwarzbach M, Luppa M, Forstmeier S, König H, Riedel-Heller SG. Social relations and depression in late life—A systematic review. Int J Geriatr Psychiatry. 2014;29(1):1–21.

Article   PubMed   Google Scholar  

Wrzus C, Hänel M, Wagner J, Neyer FJ. Social network changes and life events across the life span: A meta-analysis. Psychol Bull. 2013;139(1):53–80.

Tajvar M, Fletcher A, Grundy E, Arab M. Social support and health of older people in Middle Eastern countries: A systematic review: Social support and health of older people. Australas J Ageing. 2013;32(2):71–8.

Fasihi Harandi T, Mohammad Taghinasab M, Dehghan NT. The correlation of social support with mental health: A meta-analysis. Electron Physician. 2017;9(9):5212–22.

Article   Google Scholar  

Gariépy G, Honkaniemi H, Quesnel-Vallée A. Social support and protection from depression: systematic review of current findings in Western countries. Br J Psychiatry. 2016;209(4):284–93.

Ayalon L, Levkovich I. A Systematic Review of Research on Social Networks of Older Adults. Gerontologist. 2019;59(3):e164–76.

Siette J, Gulea C, Priebe S. Assessing Social Networks in Patients with Psychotic Disorders: A Systematic Review of Instruments. Aleksic B editor. PLoS ONE. 2015;10(12):e0145250.

Article   PubMed   PubMed Central   Google Scholar  

Berkman LF, Glass T, Brissette I, Seeman TE. From social integration to health: Durkheim in the new millennium. Soc Sci Med. 2000;51(6):843–57.

Article   CAS   PubMed   Google Scholar  

Cohen S, Underwood LG, Gottlieb BH, Fetzer Institute, editors. Social support measurement and intervention: a guide for health and social scientists. Oxford ; New York: Oxford University Press; 2000. 345 p.

Cohen S. Social relationships and health. Am Psychol. 2004;59(8):676–84.

Santini ZI, Koyanagi A, Tyrovoloas S, Mason C, Haro JM. The association between social relationships and depression: A systematic review. J Affect Disord. 2015;175:53–65.

Kuiper JS, Zuidersma M, Zuidema SU, Burgerhof JGM, Stolk RP, Oude Voshaar RC, et al. Social relationships and cognitive decline: a systematic review and meta-analysis of longitudinal cohort studies. Int J Epidemiol. 2016;45:1169–206.

Kuiper JS, Zuidersma M, Oude Voshaar RC, Zuidema SU, van den Heuvel ER, Stolk RP, et al. Social relationships and risk of dementia: A systematic review and meta-analysis of longitudinal cohort studies. Ageing Res Rev. 2015;22:39–57.

Holt-Lunstad J, Smith TB, Layton JB. Social Relationships and Mortality Risk: A Meta-analytic Review. Brayne C editor PLoS Med. 2010;7(7):e1000316.

Kawachi I, Berkman LF. Social Ties and Mental Health. Journal of Urban Health: Bulletin of the New York Academy of Medicine. 2001;78(3):458–67.

Blazer DG. Depression in Late Life: Review and Commentary. J Gerontol A Biol Sci Med Sci. 2003;58(3):249–65.

Bui BKH. The relationship between social network characteristics and depressive symptoms among older adults in the United States: Differentiating between network structure and network function. Psychogeriatrics. 2020;20(4):458–68.

Domènech-Abella J, Mundó J, Haro JM, Rubio-Valera M. Anxiety, depression, loneliness and social network in the elderly: Longitudinal associations from The Irish Longitudinal Study on Ageing (TILDA). J Affect Disord. 2019;246:82–8.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;29: n71.

World Health Organization. Ageing. n.d. Available from: https://www.who.int/health-topics/ageing#tab=tab_1 . Cited 2024 Jun 4

Kelly ME, Duff H, Kelly S, McHugh Power JE, Brennan S, Lawlor BA, et al. The impact of social activities, social networks, social support and social relationships on the cognitive functioning of healthy older adults: a systematic review. Syst Rev. 2017;6(1):259.

Piolatto M, Bianchi F, Rota M, Marengoni A, Akbaritabar A, Squazzoni F. The effect of social relationships on cognitive decline in older adults: an updated systematic review and meta-analysis of longitudinal cohort studies. BMC Public Health. 2022;22(1):278.

Visentini C, Cassidy M, Bird VJ, Priebe S. Social networks of patients with chronic depression: A systematic review. J Affect Disord. 2018;241:571–8.

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.

Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M, et al. Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. 1st ed. Wiley; 2019. p. 67–107. Available from: https://onlinelibrary.wiley.com/doi/ https://doi.org/10.1002/9781119536604.ch4 . Cited 2024 Jun 3

Li T, Higgins JP, Deeks JJ. Collecting data. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. 1st ed. Wiley; 2019. p. 109–41. Available from: https://onlinelibrary.wiley.com/doi/ https://doi.org/10.1002/9781119536604.ch5 . Cited 2024 Jun 3

Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2014. Available from: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp . Cited 2024 Mar 28.

Hakeem FF, Bernabé E, Sabbah W. Association between oral health and frailty: A systematic review of longitudinal studies. Gerodontol. 2019;36(3):205–15.

Shamsrizi P, Gladstone BP, Carrara E, Luise D, Cona A, Bovo C, et al. Variation of effect estimates in the analysis of mortality and length of hospital stay in patients with infections caused by bacteria-producing extended-spectrum beta-lactamases: a systematic review and meta-analysis. BMJ Open. 2020;10(1): e030266.

Vivekanantham A, Edwin C, Pincus T, Matharu M, Parsons H, Underwood M. The association between headache and low back pain: a systematic review. J Headache Pain. 2019;20(1):82.

Liu J, Guo M, Xu L, Mao W, Chi I. Family Relationships, Social Connections, and Depressive Symptoms Among Chinese Older Adults in International Migrant Families. J Ethn Cult Divers Soc Work. 2016;26(3):167–84.

Miller M, Lago D. The Well-Being of Older Women: The Importance of Pet and Human Relations. Anthrozoös. 1990;3(4):245–52.

Blumstein T, Benyamini Y, Fuchs Z, Shapira Z, Novikov I, Walter-Ginzburg A, et al. The Effect of a Communal Lifestyle on Depressive Symptoms in Late Life. J Aging Health. 2004;16(2):151–74.

Zhang Y, Kuang J, Xin Z, Fang J, Song R, Yang Y, et al. Loneliness, social isolation, depression and anxiety among the elderly in Shanghai: Findings from a longitudinal study. Arch Gerontol Geriatr. 2023;110: 104980.

Sugie M, Harada K, Nara M, Kugimiya Y, Takahashi T, Kitagou M, et al. Prevalence, overlap, and interrelationships of physical, cognitive, psychological, and social frailty among community-dwelling older people in Japan. Arch Gerontol Geriatr. 2022;100:104659.

Reynolds RM, Meng J, Dorrance HE. Multilayered social dynamics and depression among older adults: A 10-year cross-lagged analysis. Psychol Aging. 2020;35(7):948–62.

Keim-Klärner S, Adebahr P, Brandt S, Gamper M, Klärner A, Knabe A, et al. Social inequality, social networks, and health: a scoping review of research on health inequalities from a social network perspective. Int J Equity Health. 2023;22(1):74.

Berkman LF, Syme SL. Social networks, host resistance, and mortality: a nine-year follow-up study of alameda county residents. Am J Epidemiol. 1979;109(2):186–203.

Fernández MB, Rosell J. An Analysis of the Relationship Between Religiosity and Psychological Well-Being in Chilean Older People Using Structural Equation Modeling. J Relig Health. 2022;61(2):1585–604.

Gao J, Hu H, He H. Household indebtedness and depressive symptoms among older adults in China: The moderating role of social network and anticipated support. J Affect Disord. 2022;298(Part A):173–81.

Tang D, Mair CA, Hu Q. Widowhood, social networks, and mental health among Chinese older adults: The moderating effects of gender. Front Psychol. 2023;14:1142036.

Singh L, Singh PK, Arokiasamy P. Social network and mental health among older adults in rural Uttar Pradesh, India: A cross-sectional study. J Cross Cult Gerontol. 2016;31(2):173–92.

Aung MN, Moolphate S, Aung TNN, Kantonyoo C, Khamchai S, Wannakrairot P. The social network index and its relation to later-life depression among the elderly aged >= 80 years in Northern Thailand. CIA. 2016;11:1067–74.

Bae S, Harada K, Chiba I, Makino K, Katayama O, Lee S, et al. A New Social Network Scale for Detecting Depressive Symptoms in Older Japanese Adults. IJERPH. 2020;17(23):8874.

Boey KW, Chiu HFK. Life strain and psychological distress of older women and older men in Hong Kong. Aging Ment Health. 2005;9(6):555–62.

Chan MF, Zeng W. Investigating factors associated with depression of older women in Macau: Depression of older women . J Clin Nurs. 2009;18(21):2969–77.

Chan MF, Zeng W. Exploring risk factors for depression among older men residing in Macau: Depression of older men in Macau. J Clin Nurs. 2011;20(17–18):2645–54.

Chan A, Malhotra C, Malhotra R, Østbye T. Living arrangements, social networks and depressive symptoms among older men and women in Singapore. Int J Geriatr Psychiatry. 2011;26(6):630–9.

Chou KL, Chi I. Stressful life events and depressive symptoms: Social support and sense of control as mediators or moderators? The International Journal of Aging & Human Development. 2001;52(2):155–71.

Article   CAS   Google Scholar  

Gu Y, Ali SH, Guo A. Comparing the role of social connectivity with friends and family in depression among older adults in China: evaluating the moderating effect of urban–rural status. Front Psychiatry. 2023;14:1162982.

Hamid TA, Dzaher A, Ching SM. The role of social network, social support, religiosity and depression among elderly Malaysians who had experienced major life events. Med J Malaysia. 2019;74(3):198–204.

CAS   PubMed   Google Scholar  

Jang Y, Haley WE, Small BJ, Mortimer JA. The role of mastery and social resources in the associations between disability and depression in later life. Gerontologist. 2002;42(6):807–13.

Jang Y, Kim G, Chiriboga DA. Gender Differences in Depressive Symptoms Among Older Korean American Immigrants. Social Work in Public Health. 2011;26(1):96–109.

Jiang F, Kuper H, Zhou C, Qin W, Xu L. Relationship between hearing loss and depression symptoms among older adults in China: The mediating role of social isolation and loneliness. Int J Geriat Psychiatry. 2022;37(6).

Kim K, Lee M. Depressive symptoms of older adults living alone: The role of community characteristics. The International Journal of Aging & Human Development. 2015;80(3):248–63.

Kim BJ, Sangalang CC, Kihl T. Effects of acculturation and social network support on depression among elderly Korean immigrants. Aging Ment Health. 2012;16(6):787–94.

Kim T, Nguyen ET, Yuen EJ, Nguyen T, Sorn R, Nguyen GT. Differential Role of Social Connectedness in Geriatric Depression Among Southeast Asian Ethnic Groups. Prog Community Health Partnersh. 2015;9(4):483–93.

Klug G, Lacruz ME, Emeny RT, Häfner S, Ladwig KH, Huber D. Aging Without Depression: A Cross-Sectional Study. Psychodynamic Psychiatry. 2014;42(1):5–22.

Lee YS, Park SY, Roh S, Koenig HG, Yoo GJ. The Role of Religiousness/Spirituality and Social Networks in Predicting Depressive Symptoms among Older Korean Americans. J Cross Cult Gerontol. 2017;32(2):239–54.

Mehrabi F, Béland F. Frailty as a Moderator of the Relationship between Social Isolation and Health Outcomes in Community-Dwelling Older Adults. IJERPH. 2021;18(4):1675.

Okwumabua JO, Baker FM, Wong SP, Pilgram BO. Characteristics of depressive symptoms in elderly urban and rural African Americans. J Gerontol A Biol Sci Med Sci. 1997;52(4):M241–6.

Palinkas LA, Wingard DL, Barrett-Connor E. The biocultural context of social networks and depression among the elderly. Soc Sci Med. 1990;30(4):441–7.

Park J, Roh S. Daily spiritual experiences, social support, and depression among elderly Korean immigrants. Aging Ment Health. 2013;17(1):102–8.

Park NS, Jang Y, Lee BS, Haley WE, Chiriboga DA. The Mediating Role of Loneliness in the Relation Between Social Engagement and Depressive Symptoms Among Older Korean Americans: Do Men and Women Differ? J Gerontol B Psychol Sci Soc Sci. 2013;68(2):193–201.

Park NS, Lee BS, Chiriboga DA, Chung S. Loneliness as a mediator in the relationship between social engagement and depressive symptoms: Age differences among community-dwelling Korean adults. Health Soc Care Community. 2019;27(3):706–16.

Roh S, Lee YS, Kim Y, Park SY, Chaudhuri A. Gender Differences in the Roles of Religious Support and Social Network Support in Reducing Depressive Symptoms Among Older Korean Americans. J Soc Serv Res. 2015;41(4):484–97.

Santini ZI, Koyanagi A, Tyrovolas S, Haro JM. The association of relationship quality and social networks with depression, anxiety, and suicidal ideation among older married adults: Findings from a cross-sectional analysis of the Irish Longitudinal Study on Ageing (TILDA). J Affect Disord. 2015;179:134–41.

Tang D, Xie L. Whose migration matters? The role of migration in social networks and mental health among rural older adults in China. Ageing Soc. 2021;43:1–20.

Tang D, Lin Z, Chen F. Moving beyond living arrangements: The role of family and friendship ties in promoting mental health for urban and rural older adults in China. Aging Ment Health. 2020;24(9):1523–32.

Tanikaga M, Uemura JI, Hori F, Hamada T, Tanaka M. Changes in Community-Dwelling Elderly’s Activity and Participation Affecting Depression during COVID-19 Pandemic: A Cross-Sectional Study. IJERPH. 2023;20(5):4228.

Taylor H. Social Isolation, Loneliness, and Physical and Mental Health Among Black Older Adults. Annu Rev Gerontol Geriatr. 2021;41(1):123–44.

Google Scholar  

Tsai YF, Yeh SH, Tsai HH. Prevalence and risk factors for depressive symptoms among community-dwelling elders in Taiwan. Int J Geriat Psychiatry. 2005;20(11):1097–102.

Wee LE, Yong YZ, Chng MWX, Chew SH, Cheng L, Chua QHA, et al. Individual and area-level socioeconomic status and their association with depression amongst community-dwelling elderly in Singapore. Aging Ment Health. 2014;18(5):628–41.

Byers AL, Vittinghoff E, Lui LY, Hoang T, Blazer DG, Covinsky KE, et al. Twenty-year depressive trajectories among older women. JAMA Psychiat. 2012;69(10):1073–9.

Förster F, Luppa M, Pabst A, Heser K, Kleineidam L, Fuchs A, et al. The Role of Social Isolation and the Development of Depression. A Comparison of the Widowed and Married Oldest Old in Germany. IJERPH. 2021;18(13):6986.

Kuchibhatla MN, Fillenbaum GG, Hybels CF, Blazer DG. Trajectory classes of depressive symptoms in a community sample of older adults: Trajectory classes of depressive symptoms. Acta Psychiatr Scand. 2012;125(6):492–501.

Ruan H, Shen K, Chen F. Negative Life Events, Social Ties, and Depressive Symptoms for Older Adults in China. Front Public Health. 2022;9: 774434.

Santini ZI, Fiori KL, Feeney J, Tyrovolas S, Haro JM, Koyanagi A. Social relationships, loneliness, and mental health among older men and women in Ireland: A prospective community-based study. J Affect Disord. 2016;204:59–69.

Santini ZI, Koyanagi A, Tyrovolas S, Haro JM, Donovan RJ, Nielsen L, et al. The protective properties of Act-Belong-Commit indicators against incident depression, anxiety, and cognitive impairment among older Irish adults: Findings from a prospective community-based study. Exp Gerontol. 2017;91:79–87.

Antonucci TC, Fuhrer R, Dartigues JF. Social relations and depressive symptomatology in a sample of community-dwelling French older adults. Psychol Aging. 1997;12(1):189–95.

Becker C, Kirchmaier I, Trautmann S. Marriage, parenthood and social network: Subjective well-being and mental health in old age. PLOS ONE. 2019;14(7).

Bisconti TL, Bergeman CS. Perceived Social Control as a Mediator of the Relationships Among Social Support, Psychological Weil-Being, and Perceived Health. Gerontologist. 1999;39(1):94–104.

Braam AW, Beekman ATF, van Tilburg TG, Deeg DJH, van Tilburg W. Religious involvement and depression in older Dutch citizens. Social Psychiatry and Psychiatric Epidemiology: The International Journal for Research in Social and Genetic Epidemiology and Mental Health Services. 1997;32(5):284–91.

Cheng ST, Leung EMF, Chan TWS. Physical and social activities mediate the associations between social network types and ventilatory function in Chinese older adults. Health Psychol. 2014;33(6):524–34.

Chi I, Chou KL. Social support and depression among elderly Chinese people in Hong Kong. The International Journal of Aging & Human Development. 2001;52(3):231–52.

Cho JHJ, Olmstead R, Choi H, Carrillo C, Seeman TE, Irwin MR. Associations of objective versus subjective social isolation with sleep disturbance, depression, and fatigue in community-dwelling older adults. Aging Ment Health. 2019;23(9):1130–8.

Domènech-Abella J, Lara E, Rubio-Valera M, Olaya B, Moneta MV, Rico-Uribe LA, et al. Loneliness and depression in the elderly: the role of social network. Soc Psychiatry Psychiatr Epidemiol. 2017;52(4):381–90.

Dorrance Hall E, Meng J, Reynolds RM. Confidant Network and Interpersonal Communication Associations with Depression in Older Adulthood. Health Commun. 2019;35(7):872–81.

Ermer AE, Proulx CM. The association between relationship strain and emotional well-being among older adult couples: the moderating role of social connectedness. Aging Ment Health. 2022;26(6):1198–206.

Fredriksen-Goldsen KI, Emlet CA, Kim HJ, Muraco A, Erosheva EA, Goldsen J, et al. The physical and mental health of lesbian, gay male, and bisexual (LGB) older adults: The role of key health indicators and risk and protective factors. Gerontologist. 2013;53(4):664–75.

Fuller-Iglesias H, Sellars B, Antonucci TC. Resilience in old age: Social relations as a protective factor. Res Hum Dev. 2008;5(3):181–93.

Goldberg EL, Natta PV, Comstock GW. Depressive symptoms social networks and social support of elderly women. Am J Epidemiol. 1985;121(3):448–56.

Han HR, Kim M, Lee HB, Pistulka G, Kim KB. Correlates of depression in the Korean American elderly: Focusing on personal resources of social support. J Cross Cult Gerontol. 2007;22(1):115–27.

Harada K, Sugisawa H, Sugihara Y, Yanagisawa S, Shimmei M. Big Five Personality Traits, Social Networks, and Depression Among Older Adults in Japan: A Multiple Mediation Analysis. Int J Aging Hum Dev. 2023;97(1):111–28.

Jeon H, Lubben J. The Influence of Social Networks and Supports on Depression Symptoms: Differential Pathways for Older Korean Immigrants and Non-Hispanic White Americans. Care Manag J. 2016;17(1):13–23.

Lee SY, Chou KL. Assessing the relative contribution of social exclusion, income-poverty, and financial strain on depressive symptoms among older people in Hong Kong. Aging Ment Health. 2019;23(11):1487–95.

Lee MS, Crittenden KS, Yu E. Social support and depression among elderly Korean immigrants in the United States. The International Journal of Aging & Human Development. 1996;42(4):313–27.

Li M, Dong X, Kong D. Social networks and depressive symptoms among chinese older immigrants: Does quantity, quality, and composition of social networks matter? Clinical Gerontologist: The Journal of Aging and Mental Health. 2019;44(2):181–91.

Litwin H, Levinsky M. The Interplay of Personality Traits and Social Network Characteristics in the Subjective Well-Being of Older Adults. Res Aging. 2023;45(7–8):538–49.

Litwin H, Stoeckel KJ, Schwartz E. Social networks and mental health among older Europeans: Are there age effects? Eur J Ageing. 2015;12(4):299–309.

Minicuci N, Maggi S, Pavan M, Enzi G, Crepaldi G. Prevalence Rate and Correlates of Depressive Symptoms in Older Individuals: The Veneto Study. The Journals of Gerontology: Series A. 2002;57(3):M155–61.

Pavlidis G, Motel-Klingebiel A, Aartsen M. Exclusion from social relations in later life: on the gendered associations of social networks with mental wellbeing. Aging Ment Health. 2023;27(7):1313–21.

Pilehvari A, You W, Lin X. Retirement’s impact on health: what role does social network play? Eur J Ageing. 2023;20(14).

Sonnenberg CM, Deeg DJH, van Tilburg TG, Vink D, Stek ML, Beekman ATF. Gender differences in the relation between depression and social support in later life. Int Psychogeriatr. 2013;25(1):61–70.

Vicente HT, Guadalupe S. Childlessness, personal social networks and wellbeing at advanced ages: a cross-sectional study in a Southern European familistic welfare state. Ageing Soc. 2022;44:1–25.

Bisschop MI, Kriegsman DMW, Beekman ATF, Deeg DJH. Chronic diseases and depression: the modifying role of psychosocial resources. Soc Sci Med. 2004;59(4):721–33.

Chao SF. Assessing social support and depressive symptoms in older Chinese adults: A longitudinal perspective. Aging Ment Health. 2011;15(6):765–74.

Coleman ME, Manchella MK, Roth AR, Peng S, Perry BL. What kinds of social networks protect older adults’ health during a pandemic? The tradeoff between preventing infection and promoting mental health. Social Networks. 2022;70:393–402.

Hajek A, König HH. Effect of Health Comparisons on Functional Health and Depressive Symptoms - Results of a Population-Based Longitudinal Study of Older Adults in Germany. Tran US, editor. PLoS ONE. 2016;11(5):e0156235.

Harlow SD, Goldberg EL, Comstock GW. A Longitudinal Study of Risk Factors for Depressive Symptomatology in Elderly Widowed and Married Women. Am J Epidemiol. 1991;134(5):526–38.

Holwerda TJ, Jaarsma E, Van Zutphen EM, Beekman ATF, Pan KY, Van Vliet M, et al. The impact of COVID-19 related adversity on the course of mental health during the pandemic and the role of protective factors: a longitudinal study among older adults in The Netherlands. Soc Psychiatry Psychiatr Epidemiol. 2023;58(7):1109–20.

Oxman TE, Berkman LF, Kasl S, Freeman DH, Barrett J. Social Support and Depressive Symptoms in the Elderly. Am J Epidemiol. 1992;135(4):356–68.

Santini ZI, Jose PE, Koyanagi A, Meilstrup C, Nielsen L, Madsen KR, et al. The moderating role of social network size in the temporal association between formal social participation and mental health: a longitudinal analysis using two consecutive waves of the Survey of Health, Ageing and Retirement in Europe (SHARE). Soc Psychiatry Psychiatr Epidemiol. 2021;56(3):417–28.

Schwartz E, Litwin H. Are newly added and lost confidants in later life related to subsequent mental health? Int Psychogeriatr. 2017;29(12):2047–57.

Stringa N, Milaneschi Y, van Schoor NM, Suanet B, van der Lee S, Holstege H, et al. Genetic Liability for Depression, Social Factors and Their Interaction Effect in Depressive Symptoms and Depression Over Time in Older Adults. Am J Geriatr Psychiatry. 2020;28(8):844–55.

Tang F, Jiang Y, Li K, Rosso AL. Residential Segregation and Depressive Symptoms in Older Chinese Immigrants: The Mediating Role of Social Processes. Meeks S, editor. Gerontologist. 2023;63:gnad027.

Werneck AO, Cunha PM, Silva DR. The mediation role of social network size and perception in the association between physical activity and depressive symptoms: a prospective analysis from the SHARE study. Aging Ment Health. 2023;27:1–6.

Choi KW, Jeon GS. Social Network Types and Depressive Symptoms among Older Korean Men and Women. IJERPH. 2021;18(21):11175.

Fiori KL, Antonucci TC, Cortina KS. Social Network Typologies and Mental Health Among Older Adults. J Gerontol B Psychol Sci Soc Sci. 2006;61(1):P25-32.

Harasemiw O, Newall N, Mackenzie CS, Shooshtari S, Menec V. Is the association between social network types, depressive symptoms and life satisfaction mediated by the perceived availability of social support? A cross-sectional analysis using the Canadian Longitudinal Study on Aging. Aging Ment Health. 2019;23(10):1413–22.

Kim YB, Lee SH. Social support network types and depressive symptoms among community-dwelling older adults in South Korea. Asia Pac J Public Health. 2019;31(4):367–75.

Litwin H. The association between social network relationships and depressive symptoms among older Americans: What matters most? Int Psychogeriatr. 2011;23(6):930–40.

Litwin H. Physical activity, social network type, and depressive symptoms in late life: An analysis of data from the National Social Life, Health and Aging Project. Aging Ment Health. 2012;16(5):608–16.

Park NS, Jang Y, Lee BS, Ko JE, Chiriboga DA. The Impact of Social Resources on Depressive Symptoms in Racially and Ethnically Diverse Older Adults: Variations by Groups With Differing Health Risks. Res Aging. 2014;36(3):322.

Park NS, Jang Y, Lee BS, Chiriboga DA, Chang S, Kim SY. Associations of a social network typology with physical and mental health risks among older adults in South Korea. Aging Ment Health. 2018;22(5):631–8.

Sohn SY, Joo W, Kim WJ, Kim SJ, Youm Y, Kim HC, et al. Social network types among older Korean adults: Associations with subjective health. Soc Sci Med. 2017;173:88–95.

Ye L, Zhang X. Social Network Types and Health among Older Adults in Rural China: The Mediating Role of Social Support. IJERPH. 2019;16(3):410.

Kim B, Park S, Antonucci TC. Longitudinal changes in social networks, health and wellbeing among older Koreans. Ageing Soc. 2016;36(9):1915–36.

Förster F, Stein J, Löbner M, Pabst A, Angermeyer MC, König HH, et al. Loss experiences in old age and their impact on the social network and depression– Results of the Leipzig Longitudinal Study of the Aged (LEILA 75+). J Affect Disord. 2018;241:94–102.

Sicotte M, Alvarado BE, León EM, Zunzunegui MV. Social networks and depressive symptoms among elderly women and men in Havana. Cuba Aging & Mental Health. 2008;12(2):193–201.

Cao W, Li L, Zhou X, Zhou C. Social capital and depression: evidence from urban elderly in China. Aging Ment Health. 2015;19(5):418–29.

Golden J, Conroy RM, Bruce I, Denihan A, Greene E, Kirby M, et al. Loneliness, social support networks, mood and wellbeing in community-dwelling elderly. Int J Geriat Psychiatry. 2009;24(7):694–700.

Gumà J, Fernández-Carro C. Life goes on: The influence of the perceived quality of social relations on older women’s mental health after the loss of a partner in Europe. Aging Ment Health. 2021;25(1):53–60.

Mechakra-Tahiri SD, Zunzunegui MV, Preville M, Dube M. Gender, social relationships and depressive disorders in adults aged 65 and over in Quebec. Chronic Diseases in Canada. Chronic Diseases in Canada. 2010;30(2):56–65.

Stoeckel KJ, Litwin H. The impact of social networks on the relationship between functional impairment and depressive symptoms in older adults. Int Psychogeriatr. 2016;28(1):39–47.

Webster NJ, Antonucci TC, Ajrouch KJ, Abdulrahim S. Social networks and health among older adults in Lebanon: The mediating role of support and trust. J Gerontol B Psychol Sci Soc Sci. 2015;70(1):155–66.

Litwin H, Levinsky M. Always alone? Network transitions among detached older Europeans and their effects. Ageing Soc. 2021;41(10):2299–313.

Litwin H, Levinsky M, Schwartz E. Network type, transition patterns and well-being among older Europeans. Eur J Ageing. 2020;17(2):241–50.

Gan DRY, Best JR. Prior Social Contact and Mental Health Trajectories during COVID-19: Neighborhood Friendship Protects Vulnerable Older Adults. IJERPH. 2021;18(19):9999.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Husaini BA. Predictors of depression among the elderly: Racial differences over time. Am J Orthopsychiatry. 1997;67(1):48–58.

Castro-Costa É, Lima-Costa MF, Carvalhais S, Firmo JOA, Uchoa E. Factors associated with depressive symptoms measured by the 12-item General Health Questionnaire in Community-Dwelling Older Adults (The Bambuí Health Aging Study). Rev Bras Psiquiatr. 2008;30(2):104–9.

Forsman AK, Nyqvist F, Schierenbeck I, Gustafson Y, Wahlbeck K. Structural and cognitive social capital and depression among older adults in two nordic regions. Aging Ment Health. 2012;16(6):771–9.

La Gory M, Fitzpatrick K. The Effects of Environmental Context on Elderly Depression. J Aging Health. 1992;4(4):459–79.

Litwin H, Levinsky M. Social networks and mental health change in older adults after the Covid-19 outbreak. Aging Ment Health. 2022;26(5):925–31.

Marshall GL, Rue TC. Perceived Discrimination and Social Networks Among Older African Americans and Caribbean Blacks. Fam Community Health. 2012;35(4):300–11.

Marshall-Fabien GL, Miller DB. Exploring Ethnic Variation in the Relationship Between Stress, Social Networks, and Depressive Symptoms Among Older Black Americans. J Black Psychol. 2016;42(1):54–72.

Taylor HO, Taylor RJ, Nguyen AW, Chatters L. Social isolation, depression, and psychological distress among older adults. J Aging Health. 2018;30(2):229–46.

Wu CS, Yu SH, Lee CY, Tseng HY, Chiu YF, Hsiung CA. Prevalence of and risk factors for minor and major depression among community-dwelling older adults in Taiwan. Int Psychogeriatr. 2017;29(7):1113–21.

Murayama H, Nofuji Y, Matsuo E, Nishi M, Taniguchi Y, Fujiwara Y, et al. Are neighborhood bonding and bridging social capital protective against depressive mood in old age? A multilevel analysis in Japan. Soc Sci Med. 2015;124:171–9.

Ali T, Elliott MR, Antonucci TC, Needham BL, Zelner J, Mendes de Leon CF. Multidimensional Social Network Types and Their Correlates in Older Americans. Savla JT, editor. Innovation in Aging. 2022;6(1):1–16.

CAS   Google Scholar  

Bincy K, Logaraj M, Anantharaman VV. Social network and its effect on selected dimension of health and quality of life among community dwelling urban and rural geriatric population in India. Clinical Epidemiology and Global Health. 2022;16: 101083.

Merchant RA, Liu SG, Lim JY, Fu X, Chan YH. Factors associated with social isolation in community-dwelling older adults: A cross-sectional study. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation. 2020;29(9):2375–81.

Shouse JN, Rowe SV, Mast BT. Depression and cognitive functioning as predictors of social network size. Clinical Gerontologist: The Journal of Aging and Mental Health. 2013;36(2):147–61.

Wendel F, Bauer A, Blotenberg I, Brettschneider C, Buchholz M, Czock D, et al. Social Network and Participation in Elderly Primary Care Patients in Germany and Associations with Depressive Symptoms—A Cross-Sectional Analysis from the Age Well de Study. JCM. 2022;11(19):5940.

Houtjes W, van Meijel B, van de Ven PM, Deeg D, van Tilburg T, Beekman A. The impact of an unfavorable depression course on network size and loneliness in older people: A longitudinal study in the community. Int J Geriatr Psychiatry. 2014;29(10):1010–7.

Voils CI, Allaire JC, Olsen MK, Steffens DC, Hoyle RH, Bosworth HB. Five-year trajectories of social networks and social support in older adults with major depression. Int Psychogeriatr. 2007;19(6):1110–24.

Li W, Wang Q, Yin H, Song Y, Tu W, Wang L, et al. Construction of path analysis model on related factors of social isolation in older people. Psychogeriatrics. 2022;22(5):743–56.

Antonucci TC, Birditt KS, Sherman CW, Trinh S. Stability and change in the intergenerational family: a convoy approach. Ageing Soc. 2011;31(7):1084–106.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Nussbaumer-Streit B, Klerings I, Dobrescu AI, Persad E, Stevens A, Garritty C, et al. Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study. J Clin Epidemiol. 2020;118:42–54.

Dickens AP, Richards SH, Greaves CJ, Campbell JL. Interventions targeting social isolation in older people: a systematic review. BMC Public Health. 2011;11(1):647.

Download references

Acknowledgements

We thank Alexander Trinidad for the useful information in the beginning of the article process. We particularly thank Anna Leuwer (AL) for the extraction of the data as quality check. Special thanks to Lea Ellwardt and Karsten Hank for the valuable feedback on earlier versions of this manuscript.

Open Access funding enabled and organized by Projekt DEAL. This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grant [454899704].

Author information

Authors and affiliations.

Institute of Sociology and Social Psychology, University of Cologne, Albertus-Magnus-Platz, 50923, Cologne, Germany

Amelie Reiner & Paula Steinhoff

You can also search for this author in PubMed   Google Scholar

Contributions

AR participated in the design of the study, screened the articles, led the analysis and interpretation of the data, and drafted the first draft of the manuscript. PS participated in the design of the study, screened the articles, and contributed to the analysis and interpretation of the data. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Amelie Reiner .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Reiner, A., Steinhoff, P. The association of social networks and depression in community-dwelling older adults: a systematic review. Syst Rev 13 , 161 (2024). https://doi.org/10.1186/s13643-024-02581-6

Download citation

Received : 13 March 2024

Accepted : 10 June 2024

Published : 20 June 2024

DOI : https://doi.org/10.1186/s13643-024-02581-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social network
• Mental health
• Older adults
• Systematic review

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

Quantitative and Qualitative Approaches to Generalization and Replication–A Representationalist View

In this paper, we provide a re-interpretation of qualitative and quantitative modeling from a representationalist perspective. In this view, both approaches attempt to construct abstract representations of empirical relational structures. Whereas quantitative research uses variable-based models that abstract from individual cases, qualitative research favors case-based models that abstract from individual characteristics. Variable-based models are usually stated in the form of quantified sentences (scientific laws). This syntactic structure implies that sentences about individual cases are derived using deductive reasoning. In contrast, case-based models are usually stated using context-dependent existential sentences (qualitative statements). This syntactic structure implies that sentences about other cases are justifiable by inductive reasoning. We apply this representationalist perspective to the problems of generalization and replication. Using the analytical framework of modal logic, we argue that the modes of reasoning are often not only applied to the context that has been studied empirically, but also on a between-contexts level. Consequently, quantitative researchers mostly adhere to a top-down strategy of generalization, whereas qualitative researchers usually follow a bottom-up strategy of generalization. Depending on which strategy is employed, the role of replication attempts is very different. In deductive reasoning, replication attempts serve as empirical tests of the underlying theory. Therefore, failed replications imply a faulty theory. From an inductive perspective, however, replication attempts serve to explore the scope of the theory. Consequently, failed replications do not question the theory per se , but help to shape its boundary conditions. We conclude that quantitative research may benefit from a bottom-up generalization strategy as it is employed in most qualitative research programs. Inductive reasoning forces us to think about the boundary conditions of our theories and provides a framework for generalization beyond statistical testing. In this perspective, failed replications are just as informative as successful replications, because they help to explore the scope of our theories.

Introduction

Qualitative and quantitative research strategies have long been treated as opposing paradigms. In recent years, there have been attempts to integrate both strategies. These “mixed methods” approaches treat qualitative and quantitative methodologies as complementary, rather than opposing, strategies (Creswell, 2015 ). However, whilst acknowledging that both strategies have their benefits, this “integration” remains purely pragmatic. Hence, mixed methods methodology does not provide a conceptual unification of the two approaches.

Lacking a common methodological background, qualitative and quantitative research methodologies have developed rather distinct standards with regard to the aims and scope of empirical science (Freeman et al., 2007 ). These different standards affect the way researchers handle contradictory empirical findings. For example, many empirical findings in psychology have failed to replicate in recent years (Klein et al., 2014 ; Open Science, Collaboration, 2015 ). This “replication crisis” has been discussed on statistical, theoretical and social grounds and continues to have a wide impact on quantitative research practices like, for example, open science initiatives, pre-registered studies and a re-evaluation of statistical significance testing (Everett and Earp, 2015 ; Maxwell et al., 2015 ; Shrout and Rodgers, 2018 ; Trafimow, 2018 ; Wiggins and Chrisopherson, 2019 ).

However, qualitative research seems to be hardly affected by this discussion. In this paper, we argue that the latter is a direct consequence of how the concept of generalizability is conceived in the two approaches. Whereas most of quantitative psychology is committed to a top-down strategy of generalization based on the idea of random sampling from an abstract population, qualitative studies usually rely on a bottom-up strategy of generalization that is grounded in the successive exploration of the field by means of theoretically sampled cases.

Here, we show that a common methodological framework for qualitative and quantitative research methodologies is possible. We accomplish this by introducing a formal description of quantitative and qualitative models from a representationalist perspective: both approaches can be reconstructed as special kinds of representations for empirical relational structures. We then use this framework to analyze the generalization strategies used in the two approaches. These turn out to be logically independent of the type of model. This has wide implications for psychological research. First, a top-down generalization strategy is compatible with a qualitative modeling approach. This implies that mainstream psychology may benefit from qualitative methods when a numerical representation turns out to be difficult or impossible, without the need to commit to a “qualitative” philosophy of science. Second, quantitative research may exploit the bottom-up generalization strategy that is inherent to many qualitative approaches. This offers a new perspective on unsuccessful replications by treating them not as scientific failures, but as a valuable source of information about the scope of a theory.

The Quantitative Strategy–Numbers and Functions

Quantitative science is about finding valid mathematical representations for empirical phenomena. In most cases, these mathematical representations have the form of functional relations between a set of variables. One major challenge of quantitative modeling consists in constructing valid measures for these variables. Formally, to measure a variable means to construct a numerical representation of the underlying empirical relational structure (Krantz et al., 1971 ). For example, take the behaviors of a group of students in a classroom: “to listen,” “to take notes,” and “to ask critical questions.” One may now ask whether is possible to assign numbers to the students, such that the relations between the assigned numbers are of the same kind as the relations between the values of an underlying variable, like e.g., “engagement.” The observed behaviors in the classroom constitute an empirical relational structure, in the sense that for every student-behavior tuple, one can observe whether it is true or not. These observations can be represented in a person × behavior matrix 1 (compare Figure 1 ). Given this relational structure satisfies certain conditions (i.e., the axioms of a measurement model), one can assign numbers to the students and the behaviors, such that the relations between the numbers resemble the corresponding numerical relations. For example, if there is a unique ordering in the empirical observations with regard to which person shows which behavior, the assigned numbers have to constitute a corresponding unique ordering, as well. Such an ordering coincides with the person × behavior matrix forming a triangle shaped relation and is formally represented by a Guttman scale (Guttman, 1944 ). There are various measurement models available for different empirical structures (Suppes et al., 1971 ). In the case of probabilistic relations, Item-Response models may be considered as a special kind of measurement model (Borsboom, 2005 ).

Constructing a numerical representation from an empirical relational structure; Due to the unique ordering of persons with regard to behaviors (indicated by the triangular shape of the relation), it is possible to construct a Guttman scale by assigning a number to each of the individuals, representing the number of relevant behaviors shown by the individual. The resulting variable (“engagement”) can then be described by means of statistical analyses, like, e.g., plotting the frequency distribution.

Although essential, measurement is only the first step of quantitative modeling. Consider a slightly richer empirical structure, where we observe three additional behaviors: “to doodle,” “to chat,” and “to play.” Like above, one may ask, whether there is a unique ordering of the students with regard to these behaviors that can be represented by an underlying variable (i.e., whether the matrix forms a Guttman scale). If this is the case, we may assign corresponding numbers to the students and call this variable “distraction.” In our example, such a representation is possible. We can thus assign two numbers to each student, one representing his or her “engagement” and one representing his or her “distraction” (compare Figure 2 ). These measurements can now be used to construct a quantitative model by relating the two variables by a mathematical function. In the simplest case, this may be a linear function. This functional relation constitutes a quantitative model of the empirical relational structure under study (like, e.g., linear regression). Given the model equation and the rules for assigning the numbers (i.e., the instrumentations of the two variables), the set of admissible empirical structures is limited from all possible structures to a rather small subset. This constitutes the empirical content of the model 2 (Popper, 1935 ).

Constructing a numerical model from an empirical relational structure; Since there are two distinct classes of behaviors that each form a Guttman scale, it is possible to assign two numbers to each individual, correspondingly. The resulting variables (“engagement” and “distraction”) can then be related by a mathematical function, which is indicated by the scatterplot and red line on the right hand side.

The Qualitative Strategy–Categories and Typologies

The predominant type of analysis in qualitative research consists in category formation. By constructing descriptive systems for empirical phenomena, it is possible to analyze the underlying empirical structure at a higher level of abstraction. The resulting categories (or types) constitute a conceptual frame for the interpretation of the observations. Qualitative researchers differ considerably in the way they collect and analyze data (Miles et al., 2014 ). However, despite the diverse research strategies followed by different qualitative methodologies, from a formal perspective, most approaches build on some kind of categorization of cases that share some common features. The process of category formation is essential in many qualitative methodologies, like, for example, qualitative content analysis, thematic analysis, grounded theory (see Flick, 2014 for an overview). Sometimes these features are directly observable (like in our classroom example), sometimes they are themselves the result of an interpretative process (e.g., Scheunpflug et al., 2016 ).

In contrast to quantitative methodologies, there have been little attempts to formalize qualitative research strategies (compare, however, Rihoux and Ragin, 2009 ). However, there are several statistical approaches to non-numerical data that deal with constructing abstract categories and establishing relations between these categories (Agresti, 2013 ). Some of these methods are very similar to qualitative category formation on a conceptual level. For example, cluster analysis groups cases into homogenous categories (clusters) based on their similarity on a distance metric.

Although category formation can be formalized in a mathematically rigorous way (Ganter and Wille, 1999 ), qualitative research hardly acknowledges these approaches. 3 However, in order to find a common ground with quantitative science, it is certainly helpful to provide a formal interpretation of category systems.

Let us reconsider the above example of students in a classroom. The quantitative strategy was to assign numbers to the students with regard to variables and to relate these variables via a mathematical function. We can analyze the same empirical structure by grouping the behaviors to form abstract categories. If the aim is to construct an empirically valid category system, this grouping is subject to constraints, analogous to those used to specify a measurement model. The first and most important constraint is that the behaviors must form equivalence classes, i.e., within categories, behaviors need to be equivalent, and across categories, they need to be distinct (formally, the relational structure must obey the axioms of an equivalence relation). When objects are grouped into equivalence classes, it is essential to specify the criterion for empirical equivalence. In qualitative methodology, this is sometimes referred to as the tertium comparationis (Flick, 2014 ). One possible criterion is to group behaviors such that they constitute a set of specific common attributes of a group of people. In our example, we might group the behaviors “to listen,” “to take notes,” and “to doodle,” because these behaviors are common to the cases B, C, and D, and they are also specific for these cases, because no other person shows this particular combination of behaviors. The set of common behaviors then forms an abstract concept (e.g., “moderate distraction”), while the set of persons that show this configuration form a type (e.g., “the silent dreamer”). Formally, this means to identify the maximal rectangles in the underlying empirical relational structure (see Figure 3 ). This procedure is very similar to the way we constructed a Guttman scale, the only difference being that we now use different aspects of the empirical relational structure. 4 In fact, the set of maximal rectangles can be determined by an automated algorithm (Ganter, 2010 ), just like the dimensionality of an empirical structure can be explored by psychometric scaling methods. Consequently, we can identify the empirical content of a category system or a typology as the set of empirical structures that conforms to it. 5 Whereas the quantitative strategy was to search for scalable sub-matrices and then relate the constructed variables by a mathematical function, the qualitative strategy is to construct an empirical typology by grouping cases based on their specific similarities. These types can then be related to one another by a conceptual model that describes their semantic and empirical overlap (see Figure 3 , right hand side).

Constructing a conceptual model from an empirical relational structure; Individual behaviors are grouped to form abstract types based on them being shared among a specific subset of the cases. Each type constitutes a set of specific commonalities of a class of individuals (this is indicated by the rectangles on the left hand side). The resulting types (“active learner,” “silent dreamer,” “distracted listener,” and “troublemaker”) can then be related to one another to explicate their semantic and empirical overlap, as indicated by the Venn-diagram on the right hand side.

Variable-Based Models and Case-Based Models

In the previous section, we have argued that qualitative category formation and quantitative measurement can both be characterized as methods to construct abstract representations of empirical relational structures. Instead of focusing on different philosophical approaches to empirical science, we tried to stress the formal similarities between both approaches. However, it is worth also exploring the dissimilarities from a formal perspective.

Following the above analysis, the quantitative approach can be characterized by the use of variable-based models, whereas the qualitative approach is characterized by case-based models (Ragin, 1987 ). Formally, we can identify the rows of an empirical person × behavior matrix with a person-space, and the columns with a corresponding behavior-space. A variable-based model abstracts from the single individuals in a person-space to describe the structure of behaviors on a population level. A case-based model, on the contrary, abstracts from the single behaviors in a behavior-space to describe individual case configurations on the level of abstract categories (see Table 1 ).

Variable-based models and case-based models.

From a representational perspective, there is no a priori reason to favor one type of model over the other. Both approaches provide different analytical tools to construct an abstract representation of an empirical relational structure. However, since the two modeling approaches make use of different information (person-space vs. behavior-space), this comes with some important implications for the researcher employing one of the two strategies. These are concerned with the role of deductive and inductive reasoning.

In variable-based models, empirical structures are represented by functional relations between variables. These are usually stated as scientific laws (Carnap, 1928 ). Formally, these laws correspond to logical expressions of the form

In plain text, this means that y is a function of x for all objects i in the relational structure under consideration. For example, in the above example, one may formulate the following law: for all students in the classroom it holds that “distraction” is a monotone decreasing function of “engagement.” Such a law can be used to derive predictions for single individuals by means of logical deduction: if the above law applies to all students in the classroom, it is possible to calculate the expected distraction from a student's engagement. An empirical observation can now be evaluated against this prediction. If the prediction turns out to be false, the law can be refuted based on the principle of falsification (Popper, 1935 ). If a scientific law repeatedly withstands such empirical tests, it may be considered to be valid with regard to the relational structure under consideration.

In case-based models, there are no laws about a population, because the model does not abstract from the cases but from the observed behaviors. A case-based model describes the underlying structure in terms of existential sentences. Formally, this corresponds to a logical expression of the form

In plain text, this means that there is at least one case i for which the condition XYZ holds. For example, the above category system implies that there is at least one active learner. This is a statement about a singular observation. It is impossible to deduce a statement about another person from an existential sentence like this. Therefore, the strategy of falsification cannot be applied to test the model's validity in a specific context. If one wishes to generalize to other cases, this is accomplished by inductive reasoning, instead. If we observed one person that fulfills the criteria of calling him or her an active learner, we can hypothesize that there may be other persons that are identical to the observed case in this respect. However, we do not arrive at this conclusion by logical deduction, but by induction.

Despite this important distinction, it would be wrong to conclude that variable-based models are intrinsically deductive and case-based models are intrinsically inductive. 6 Both types of reasoning apply to both types of models, but on different levels. Based on a person-space, in a variable-based model one can use deduction to derive statements about individual persons from abstract population laws. There is an analogous way of reasoning for case-based models: because they are based on a behavior space, it is possible to deduce statements about singular behaviors. For example, if we know that Peter is an active learner, we can deduce that he takes notes in the classroom. This kind of deductive reasoning can also be applied on a higher level of abstraction to deduce thematic categories from theoretical assumptions (Braun and Clarke, 2006 ). Similarly, there is an analog for inductive generalization from the perspective of variable-based modeling: since the laws are only quantified over the person-space, generalizations to other behaviors rely on inductive reasoning. For example, it is plausible to assume that highly engaged students tend to do their homework properly–however, in our example this behavior has never been observed. Hence, in variable-based models we usually generalize to other behaviors by means of induction. This kind of inductive reasoning is very common when empirical results are generalized from the laboratory to other behavioral domains.

Although inductive and deductive reasoning are used in qualitative and quantitative research, it is important to stress the different roles of induction and deduction when models are applied to cases. A variable-based approach implies to draw conclusions about cases by means of logical deduction; a case-based approach implies to draw conclusions about cases by means of inductive reasoning. In the following, we build on this distinction to differentiate between qualitative (bottom-up) and quantitative (top-down) strategies of generalization.

Generalization and the Problem of Replication

We will now extend the formal analysis of quantitative and qualitative approaches to the question of generalization and replicability of empirical findings. For this sake, we have to introduce some concepts of formal logic. Formal logic is concerned with the validity of arguments. It provides conditions to evaluate whether certain sentences (conclusions) can be derived from other sentences (premises). In this context, a theory is nothing but a set of sentences (also called axioms). Formal logic provides tools to derive new sentences that must be true, given the axioms are true (Smith, 2020 ). These derived sentences are called theorems or, in the context of empirical science, predictions or hypotheses . On the syntactic level, the rules of logic only state how to evaluate the truth of a sentence relative to its premises. Whether or not sentences are actually true, is formally specified by logical semantics.

On the semantic level, formal logic is intrinsically linked to set-theory. For example, a logical statement like “all dogs are mammals,” is true if and only if the set of dogs is a subset of the set of mammals. Similarly, the sentence “all chatting students doodle” is true if and only if the set of chatting students is a subset of the set of doodling students (compare Figure 3 ). Whereas, the first sentence is analytically true due to the way we define the words “dog” and “mammal,” the latter can be either true or false, depending on the relational structure we actually observe. We can thus interpret an empirical relational structure as the truth criterion of a scientific theory. From a logical point of view, this corresponds to the semantics of a theory. As shown above, variable-based and case-based models both give a formal representation of the same kinds of empirical structures. Accordingly, both types of models can be stated as formal theories. In the variable-based approach, this corresponds to a set of scientific laws that are quantified over the members of an abstract population (these are the axioms of the theory). In the case-based approach, this corresponds to a set of abstract existential statements about a specific class of individuals.

In contrast to mathematical axiom systems, empirical theories are usually not considered to be necessarily true. This means that even if we find no evidence against a theory, it is still possible that it is actually wrong. We may know that a theory is valid in some contexts, yet it may fail when applied to a new set of behaviors (e.g., if we use a different instrumentation to measure a variable) or a new population (e.g., if we draw a new sample).

From a logical perspective, the possibility that a theory may turn out to be false stems from the problem of contingency . A statement is contingent, if it is both, possibly true and possibly false. Formally, we introduce two modal operators: □ to designate logical necessity, and ◇ to designate logical possibility. Semantically, these operators are very similar to the existential quantifier, ∃, and the universal quantifier, ∀. Whereas ∃ and ∀ refer to the individual objects within one relational structure, the modal operators □ and ◇ range over so-called possible worlds : a statement is possibly true, if and only if it is true in at least one accessible possible world, and a statement is necessarily true if and only if it is true in every accessible possible world (Hughes and Cresswell, 1996 ). Logically, possible worlds are mathematical abstractions, each consisting of a relational structure. Taken together, the relational structures of all accessible possible worlds constitute the formal semantics of necessity, possibility and contingency. 7

In the context of an empirical theory, each possible world may be identified with an empirical relational structure like the above classroom example. Given the set of intended applications of a theory (the scope of the theory, one may say), we can now construct possible world semantics for an empirical theory: each intended application of the theory corresponds to a possible world. For example, a quantified sentence like “all chatting students doodle” may be true in one classroom and false in another one. In terms of possible worlds, this would correspond to a statement of contingency: “it is possible that all chatting students doodle in one classroom, and it is possible that they don't in another classroom.” Note that in the above expression, “all students” refers to the students in only one possible world, whereas “it is possible” refers to the fact that there is at least one possible world for each of the specified cases.

To apply these possible world semantics to quantitative research, let us reconsider how generalization to other cases works in variable-based models. Due to the syntactic structure of quantitative laws, we can deduce predictions for singular observations from an expression of the form ∀ i : y i = f ( x i ). Formally, the logical quantifier ∀ ranges only over the objects of the corresponding empirical relational structure (in our example this would refer to the students in the observed classroom). But what if we want to generalize beyond the empirical structure we actually observed? The standard procedure is to assume an infinitely large, abstract population from which a random sample is drawn. Given the truth of the theory, we can deduce predictions about what we may observe in the sample. Since usually we deal with probabilistic models, we can evaluate our theory by means of the conditional probability of the observations, given the theory holds. This concept of conditional probability is the foundation of statistical significance tests (Hogg et al., 2013 ), as well as Bayesian estimation (Watanabe, 2018 ). In terms of possible world semantics, the random sampling model implies that all possible worlds (i.e., all intended applications) can be conceived as empirical sub-structures from a greater population structure. For example, the empirical relational structure constituted by the observed behaviors in a classroom would be conceived as a sub-matrix of the population person × behavior matrix. It follows that, if a scientific law is true in the population, it will be true in all possible worlds, i.e., it will be necessarily true. Formally, this corresponds to an expression of the form

The statistical generalization model thus constitutes a top-down strategy for dealing with individual contexts that is analogous to the way variable-based models are applied to individual cases (compare Table 1 ). Consequently, if we apply a variable-based model to a new context and find out that it does not fit the data (i.e., there is a statistically significant deviation from the model predictions), we have reason to doubt the validity of the theory. This is what makes the problem of low replicability so important: we observe that the predictions are wrong in a new study; and because we apply a top-down strategy of generalization to contexts beyond the ones we observed, we see our whole theory at stake.

Qualitative research, on the contrary, follows a different strategy of generalization. Since case-based models are formulated by a set of context-specific existential sentences, there is no need for universal truth or necessity. In contrast to statistical generalization to other cases by means of random sampling from an abstract population, the usual strategy in case-based modeling is to employ a bottom-up strategy of generalization that is analogous to the way case-based models are applied to individual cases. Formally, this may be expressed by stating that the observed qualia exist in at least one possible world, i.e., the theory is possibly true:

This statement is analogous to the way we apply case-based models to individual cases (compare Table 1 ). Consequently, the set of intended applications of the theory does not follow from a sampling model, but from theoretical assumptions about which cases may be similar to the observed cases with respect to certain relevant characteristics. For example, if we observe that certain behaviors occur together in one classroom, following a bottom-up strategy of generalization, we will hypothesize why this might be the case. If we do not replicate this finding in another context, this does not question the model itself, since it was a context-specific theory all along. Instead, we will revise our hypothetical assumptions about why the new context is apparently less similar to the first one than we originally thought. Therefore, if an empirical finding does not replicate, we are more concerned about our understanding of the cases than about the validity of our theory.

Whereas statistical generalization provides us with a formal (and thus somehow more objective) apparatus to evaluate the universal validity of our theories, the bottom-up strategy forces us to think about the class of intended applications on theoretical grounds. This means that we have to ask: what are the boundary conditions of our theory? In the above classroom example, following a bottom-up strategy, we would build on our preliminary understanding of the cases in one context (e.g., a public school) to search for similar and contrasting cases in other contexts (e.g., a private school). We would then re-evaluate our theoretical description of the data and explore what makes cases similar or dissimilar with regard to our theory. This enables us to expand the class of intended applications alongside with the theory.

Of course, none of these strategies is superior per se . Nevertheless, they rely on different assumptions and may thus be more or less adequate in different contexts. The statistical strategy relies on the assumption of a universal population and invariant measurements. This means, we assume that (a) all samples are drawn from the same population and (b) all variables refer to the same behavioral classes. If these assumptions are true, statistical generalization is valid and therefore provides a valuable tool for the testing of empirical theories. The bottom-up strategy of generalization relies on the idea that contexts may be classified as being more or less similar based on characteristics that are not part of the model being evaluated. If such a similarity relation across contexts is feasible, the bottom-up strategy is valid, as well. Depending on the strategy of generalization, replication of empirical research serves two very different purposes. Following the (top-down) principle of generalization by deduction from scientific laws, replications are empirical tests of the theory itself, and failed replications question the theory on a fundamental level. Following the (bottom-up) principle of generalization by induction to similar contexts, replications are a means to explore the boundary conditions of a theory. Consequently, failed replications question the scope of the theory and help to shape the set of intended applications.

We have argued that quantitative and qualitative research are best understood by means of the structure of the employed models. Quantitative science mainly relies on variable-based models and usually employs a top-down strategy of generalization from an abstract population to individual cases. Qualitative science prefers case-based models and usually employs a bottom-up strategy of generalization. We further showed that failed replications have very different implications depending on the underlying strategy of generalization. Whereas in the top-down strategy, replications are used to test the universal validity of a model, in the bottom-up strategy, replications are used to explore the scope of a model. We will now address the implications of this analysis for psychological research with regard to the problem of replicability.

Modern day psychology almost exclusively follows a top-down strategy of generalization. Given the quantitative background of most psychological theories, this is hardly surprising. Following the general structure of variable-based models, the individual case is not the focus of the analysis. Instead, scientific laws are stated on the level of an abstract population. Therefore, when applying the theory to a new context, a statistical sampling model seems to be the natural consequence. However, this is not the only possible strategy. From a logical point of view, there is no reason to assume that a quantitative law like ∀ i : y i = f ( x i ) implies that the law is necessarily true, i.e.,: □(∀ i : y i = f ( x i )). Instead, one might just as well define the scope of the theory following an inductive strategy. 8 Formally, this would correspond to the assumption that the observed law is possibly true, i.e.,: ◇(∀ i : y i = f ( x i )). For example, we may discover a functional relation between “engagement” and “distraction” without referring to an abstract universal population of students. Instead, we may hypothesize under which conditions this functional relation may be valid and use these assumptions to inductively generalize to other cases.

If we take this seriously, this would require us to specify the intended applications of the theory: in which contexts do we expect the theory to hold? Or, equivalently, what are the boundary conditions of the theory? These boundary conditions may be specified either intensionally, i.e., by giving external criteria for contexts being similar enough to the ones already studied to expect a successful application of the theory. Or they may be specified extensionally, by enumerating the contexts where the theory has already been shown to be valid. These boundary conditions need not be restricted to the population we refer to, but include all kinds of contextual factors. Therefore, adopting a bottom-up strategy, we are forced to think about these factors and make them an integral part of our theories.

In fact, there is good reason to believe that bottom-up generalization may be more adequate in many psychological studies. Apart from the pitfalls associated with statistical generalization that have been extensively discussed in recent years (e.g., p-hacking, underpowered studies, publication bias), it is worth reflecting on whether the underlying assumptions are met in a particular context. For example, many samples used in experimental psychology are not randomly drawn from a large population, but are convenience samples. If we use statistical models with non-random samples, we have to assume that the observations vary as if drawn from a random sample. This may indeed be the case for randomized experiments, because all variation between the experimental conditions apart from the independent variable will be random due to the randomization procedure. In this case, a classical significance test may be regarded as an approximation to a randomization test (Edgington and Onghena, 2007 ). However, if we interpret a significance test as an approximate randomization test, we test not for generalization but for internal validity. Hence, even if we use statistical significance tests when assumptions about random sampling are violated, we still have to use a different strategy of generalization. This issue has been discussed in the context of small-N studies, where variable-based models are applied to very small samples, sometimes consisting of only one individual (Dugard et al., 2012 ). The bottom-up strategy of generalization that is employed by qualitative researchers, provides such an alternative.

Another important issue in this context is the question of measurement invariance. If we construct a variable-based model in one context, the variables refer to those behaviors that constitute the underlying empirical relational structure. For example, we may construct an abstract measure of “distraction” using the observed behaviors in a certain context. We will then use the term “distraction” as a theoretical term referring to the variable we have just constructed to represent the underlying empirical relational structure. Let us now imagine we apply this theory to a new context. Even if the individuals in our new context are part of the same population, we may still get into trouble if the observed behaviors differ from those used in the original study. How do we know whether these behaviors constitute the same variable? We have to ensure that in any new context, our measures are valid for the variables in our theory. Without a proper measurement model, this will be hard to achieve (Buntins et al., 2017 ). Again, we are faced with the necessity to think of the boundary conditions of our theories. In which contexts (i.e., for which sets of individuals and behaviors) do we expect our theory to work?

If we follow the rationale of inductive generalization, we can explore the boundary conditions of a theory with every new empirical study. We thus widen the scope of our theory by comparing successful applications in different contexts and unsuccessful applications in similar contexts. This may ultimately lead to a more general theory, maybe even one of universal scope. However, unless we have such a general theory, we might be better off, if we treat unsuccessful replications not as a sign of failure, but as a chance to learn.

Author Contributions

MB conceived the original idea and wrote the first draft of the paper. MS helped to further elaborate and scrutinize the arguments. All authors contributed to the final version of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank Annette Scheunpflug for helpful comments on an earlier version of the manuscript.

1 A person × behavior matrix constitutes a very simple relational structure that is common in psychological research. This is why it is chosen here as a minimal example. However, more complex structures are possible, e.g., by relating individuals to behaviors over time, with individuals nested within groups etc. For a systematic overview, compare Coombs ( 1964 ).

2 This notion of empirical content applies only to deterministic models. The empirical content of a probabilistic model consists in the probability distribution over all possible empirical structures.

3 For example, neither the SAGE Handbook of qualitative data analysis edited by Flick ( 2014 ) nor the Oxford Handbook of Qualitative Research edited by Leavy ( 2014 ) mention formal approaches to category formation.

4 Note also that the described structure is empirically richer than a nominal scale. Therefore, a reduction of qualitative category formation to be a special (and somehow trivial) kind of measurement is not adequate.

5 It is possible to extend this notion of empirical content to the probabilistic case (this would correspond to applying a latent class analysis). But, since qualitative research usually does not rely on formal algorithms (neither deterministic nor probabilistic), there is currently little practical use of such a concept.

6 We do not elaborate on abductive reasoning here, since, given an empirical relational structure, the concept can be applied to both types of models in the same way (Schurz, 2008 ). One could argue that the underlying relational structure is not given a priori but has to be constructed by the researcher and will itself be influenced by theoretical expectations. Therefore, abductive reasoning may be necessary to establish an empirical relational structure in the first place.

7 We shall not elaborate on the metaphysical meaning of possible worlds here, since we are only concerned with empirical theories [but see Tooley ( 1999 ), for an overview].

8 Of course, this also means that it would be equally reasonable to employ a top-down strategy of generalization using a case-based model by postulating that □(∃ i : XYZ i ). The implications for case-based models are certainly worth exploring, but lie beyond the scope of this article.

• Agresti A. (2013). Categorical Data Analysis, 3rd Edn. Wiley Series In Probability And Statistics . Hoboken, NJ: Wiley. [ Google Scholar ]
• Borsboom D. (2005). Measuring the Mind: Conceptual Issues in Contemporary Psychometrics . Cambridge: Cambridge University Press; 10.1017/CBO9780511490026 [ CrossRef ] [ Google Scholar ]
• Braun V., Clarke V. (2006). Using thematic analysis in psychology . Qual. Res. Psychol . 3 , 77–101. 10.1191/1478088706qp063oa [ CrossRef ] [ Google Scholar ]
• Buntins M., Buntins K., Eggert F. (2017). Clarifying the concept of validity: from measurement to everyday language . Theory Psychol. 27 , 703–710. 10.1177/0959354317702256 [ CrossRef ] [ Google Scholar ]
• Carnap R. (1928). The Logical Structure of the World . Berkeley, CA: University of California Press. [ Google Scholar ]
• Coombs C. H. (1964). A Theory of Data . New York, NY: Wiley. [ Google Scholar ]
• Creswell J. W. (2015). A Concise Introduction to Mixed Methods Research . Los Angeles, CA: Sage. [ Google Scholar ]
• Dugard P., File P., Todman J. B. (2012). Single-Case and Small-N Experimental Designs: A Practical Guide to Randomization Tests 2nd Edn . New York, NY: Routledge; 10.4324/9780203180938 [ CrossRef ] [ Google Scholar ]
• Edgington E., Onghena P. (2007). Randomization Tests, 4th Edn. Statistics. Hoboken, NJ: CRC Press; 10.1201/9781420011814 [ CrossRef ] [ Google Scholar ]
• Everett J. A. C., Earp B. D. (2015). A tragedy of the (academic) commons: interpreting the replication crisis in psychology as a social dilemma for early-career researchers . Front. Psychol . 6 :1152. 10.3389/fpsyg.2015.01152 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flick U. (Ed.). (2014). The Sage Handbook of Qualitative Data Analysis . London: Sage; 10.4135/9781446282243 [ CrossRef ] [ Google Scholar ]
• Freeman M., Demarrais K., Preissle J., Roulston K., St. Pierre E. A. (2007). Standards of evidence in qualitative research: an incitement to discourse . Educ. Res. 36 , 25–32. 10.3102/0013189X06298009 [ CrossRef ] [ Google Scholar ]
• Ganter B. (2010). Two basic algorithms in concept analysis , in Lecture Notes In Computer Science. Formal Concept Analysis, Vol. 5986 , eds Hutchison D., Kanade T., Kittler J., Kleinberg J. M., Mattern F., Mitchell J. C., et al. (Berlin, Heidelberg: Springer Berlin Heidelberg; ), 312–340. 10.1007/978-3-642-11928-6_22 [ CrossRef ] [ Google Scholar ]
• Ganter B., Wille R. (1999). Formal Concept Analysis . Berlin, Heidelberg: Springer Berlin Heidelberg; 10.1007/978-3-642-59830-2 [ CrossRef ] [ Google Scholar ]
• Guttman L. (1944). A basis for scaling qualitative data . Am. Sociol. Rev . 9 :139 10.2307/2086306 [ CrossRef ] [ Google Scholar ]
• Hogg R. V., Mckean J. W., Craig A. T. (2013). Introduction to Mathematical Statistics, 7th Edn . Boston, MA: Pearson. [ Google Scholar ]
• Hughes G. E., Cresswell M. J. (1996). A New Introduction To Modal Logic . London; New York, NY: Routledge; 10.4324/9780203290644 [ CrossRef ] [ Google Scholar ]
• Klein R. A., Ratliff K. A., Vianello M., Adams R. B., Bahník Š., Bernstein M. J., et al. (2014). Investigating variation in replicability . Soc. Psychol. 45 , 142–152. 10.1027/1864-9335/a000178 [ CrossRef ] [ Google Scholar ]
• Krantz D. H., Luce D., Suppes P., Tversky A. (1971). Foundations of Measurement Volume I: Additive And Polynomial Representations . New York, NY; London: Academic Press; 10.1016/B978-0-12-425401-5.50011-8 [ CrossRef ] [ Google Scholar ]
• Leavy P. (2014). The Oxford Handbook of Qualitative Research . New York, NY: Oxford University Press; 10.1093/oxfordhb/9780199811755.001.0001 [ CrossRef ] [ Google Scholar ]
• Maxwell S. E., Lau M. Y., Howard G. S. (2015). Is psychology suffering from a replication crisis? what does “failure to replicate” really mean? Am. Psychol. 70 , 487–498. 10.1037/a0039400 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Miles M. B., Huberman A. M., Saldaña J. (2014). Qualitative Data Analysis: A Methods Sourcebook, 3rd Edn . Los Angeles, CA; London; New Delhi; Singapore; Washington, DC: Sage. [ Google Scholar ]
• Open Science, Collaboration (2015). Estimating the reproducibility of psychological science . Science 349 :Aac4716. 10.1126/science.aac4716 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Popper K. (1935). Logik Der Forschung . Wien: Springer; 10.1007/978-3-7091-4177-9 [ CrossRef ] [ Google Scholar ]
• Ragin C. (1987). The Comparative Method : Moving Beyond Qualitative and Quantitative Strategies . Berkeley, CA: University Of California Press. [ Google Scholar ]
• Rihoux B., Ragin C. (2009). Configurational Comparative Methods: Qualitative Comparative Analysis (Qca) And Related Techniques . Thousand Oaks, CA: Sage Publications, Inc; 10.4135/9781452226569 [ CrossRef ] [ Google Scholar ]
• Scheunpflug A., Krogull S., Franz J. (2016). Understanding learning in world society: qualitative reconstructive research in global learning and learning for sustainability . Int. Journal Dev. Educ. Glob. Learn. 7 , 6–23. 10.18546/IJDEGL.07.3.02 [ CrossRef ] [ Google Scholar ]
• Schurz G. (2008). Patterns of abduction . Synthese 164 , 201–234. 10.1007/s11229-007-9223-4 [ CrossRef ] [ Google Scholar ]
• Shrout P. E., Rodgers J. L. (2018). Psychology, science, and knowledge construction: broadening perspectives from the replication crisis . Annu. Rev. Psychol . 69 , 487–510. 10.1146/annurev-psych-122216-011845 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Smith P. (2020). An Introduction To Formal Logic . Cambridge: Cambridge University Press. 10.1017/9781108328999 [ CrossRef ] [ Google Scholar ]
• Suppes P., Krantz D. H., Luce D., Tversky A. (1971). Foundations of Measurement Volume II: Geometrical, Threshold, and Probabilistic Representations . New York, NY; London: Academic Press. [ Google Scholar ]
• Tooley M. (Ed.). (1999). Necessity and Possibility. The Metaphysics of Modality . New York, NY; London: Garland Publishing. [ Google Scholar ]
• Trafimow D. (2018). An a priori solution to the replication crisis . Philos. Psychol . 31 , 1188–1214. 10.1080/09515089.2018.1490707 [ CrossRef ] [ Google Scholar ]
• Watanabe S. (2018). Mathematical Foundations of Bayesian Statistics. CRC Monographs On Statistics And Applied Probability . Boca Raton, FL: Chapman And Hall. [ Google Scholar ]
• Wiggins B. J., Chrisopherson C. D. (2019). The replication crisis in psychology: an overview for theoretical and philosophical psychology . J. Theor. Philos. Psychol. 39 , 202–217. 10.1037/teo0000137 [ CrossRef ] [ Google Scholar ]

Communicative Sciences and Disorders

• Online Learners: Quick Links
• ASHA Journals
• Research Tip 1: Define the Research Question
• Reference Resources
• Evidence Summaries & Clinical Guidelines
• Drug Information
• Health Data & Statistics
• Patient/Consumer Facing Materials
• Images/Streaming Video
• Database Tutorials
• Crafting a Search
• Cited Reference Searching
• Research Tip 4: Find Grey Literature
• Research Tip 5: Save Your Work
• Cite and Manage Your Sources
• Critical Appraisal
• What are Literature Reviews?
• Conducting & Reporting Systematic Reviews
• Finding Systematic Reviews
• Tutorials & Tools for Literature Reviews
• Point of Care Tools (Mobile Apps)

Contact Ask a Librarian via email , text us at +1-646-265-1342, or schedule an appointment .

Problem with an e-resource? Fill out this form.

How do you know if a journal is peer reviewed?

• You can look for the "Limits" feature in many databases to narrow search results by publication type.
• An individual journal's website may describe the peer review process and policies - look for sections like "For Authors" or "Editorial Policies"

Selecting a Database for Communicative Sciences

The literature of Communicative Sciences and Disorders is primarily indexed in MEDLINE (via PubMed) , Web of Science , and PsycINFO , linked in the box below .

When choosing a database to start your literature search, consider the topic.

• Is it more  biomedical  (e.g., speech loss related to CNS tumors)?  → MEDLINE via PubMed is a good place to start
• Is it focused on the  psychosocial  aspects of communication disorders?  (e.g., the experience, cognitive, and social aspects of speech loss) → PsycINFO could be worthwhile

Sources of CSD Literature

Biomedical & health databases.

• Medline via PubMed This link opens in a new window PubMed comprises over 30 million citations for biomedical literature from MEDLINE, life science journals, and online books. PubMed citations and abstracts include the fields of biomedicine and health, covering portions of the life sciences, behavioral sciences, chemical sciences, and bioengineering. PubMed also provides access to additional relevant web sites and links to the other NCBI molecular biology resources.
• CINAHL Complete This link opens in a new window CINAHL indexes over 3,800 journals from the fields of nursing and allied health, with indexing for selected journals back to 1937. Topics covered are related to nursing, physical therapy, health education, social service/healthcare, occupational therapy, and related disciplines. CINAHL also provides access to healthcare books, nursing dissertations, selected conference proceedings, standards of professional practice, educational software and audiovisual materials in nursing. Dates of coverage: 1937 to present.
• PsycNET (with PsycINFO) This link opens in a new window APA PsycNET is an integrated collection of databases from the American Psychological Association (APA), including the following: PsycINFO, PsycBOOKS, PsycTESTS, PsycTHERAPY, PsycARTICLES, and PsycEXTRA. PsycINFO, the most popular database in PsycNET, is a bibliographic index covering core literature in the psychological and behavioral sciences and their related disciplines. PsycBOOKS includes online versions of the APA Handbooks in Psychology Series, as well as other ebooks and online encyclopedias published by APA. PsycTESTS is a source of structured information about psychological tests and measures as well as a repository for the full text of select instruments. PsycTHERAPY provides access to streaming psychotherapy demonstration videos. Altogether, PsycNET provides information about and access to journal articles, books and ebooks, dissertations, conference presentations, tests and measures, videos, gray literature, and many other other publication types in the psychological, social, behavioral, and health sciences. More information less... To search only a specific database (e.g., PsycINFO) within the PsycNET interface, use the "Select Databases" features at the top of the PsycNET landing page and check (or uncheck) the database(s) that apply.
• PsycINFO via Ovid This link opens in a new window PsycINFO via Ovid indexes core academic and professional literature in the psychological and behavioral sciences and their related disciplines. Coverage is from 1806 to the present. **Note that the database PsycINFO is also searchable on APA PsycNET platform.
• EMBASE: Excerpta Medica 1974 – present (updated daily) This link opens in a new window EMBASE is a biomedical and pharmaceutical database indexing over 3,500 international journals in the following fields: drug research, pharmacology, pharmaceutics, toxicology, clinical and experimental human medicine, health policy and management, public health, occupational health, environmental health, drug dependence and abuse, psychiatry, forensic medicine, and biomedical engineering/instrumentation. There is selective coverage for nursing, dentistry, veterinary medicine, psychology, and alternative medicine. Dates of coverage: 1974 to present (updated daily)
• speechBITE speechBITE is a database of intervention studies, systematic reviews and clinical guidelines across the scope of speech pathology practice.

Multidisciplinary Sources with CSD Coverage

• Web of Science Core Collection This link opens in a new window Web of Science Core Collection is a multidisciplinary citation index that includes scholarly articles, conference proceedings, and books in the biomedical, psychosocial, and arts and humanities. The collections contain Science Citation, Social Sciences Citation Index, Arts and Humanities Citation Index , Conference Proceedings Citation Index, Book Citation Index, Emerging Sources Citation Index and Current Chemical Reactions index.
• Scopus This link opens in a new window Scopus is a multidisciplinary index of peer-reviewed journals and conference proceedings.
• ProQuest Central This link opens in a new window ProQuest Central is a large, multidisciplinary database with over 11,000 titles, with over 8,000 titles in full-text. It serves as the central resource for researchers at all levels in all markets. Over 160 subjects areas are covered extensively in this product including business and economics, health and medical, news and world affairs, technology, social sciences and more.
• Dissertations & Theses Global This link opens in a new window Dissertations and Theses Global contains indexes, dissertations and some theses. Full-text is available for many dissertations and theses, including those from NYU.

Education, Linguistics, and Social Sciences

• Education Source with ERIC This link opens in a new window Education Source with ERIC provides access to research on all levels of education from early childhood to adult education. The content includes peer reviewed journal articles, ebooks, book reviews, dissertations, conference papers and policy reports on all topics in the realm of education.
• Linguistics and Language Behavior Abstracts (LLBA) This link opens in a new window Abstracts and indexes the international literature in linguistics and related disciplines in the language sciences. The database covers all aspects of the study of language including phonetics, phonology, morphology, syntax and semantics. Complete coverage is given to various fields of linguistics including descriptive, historical, comparative, theoretical and geographical linguistics. Dates of coverage: 1973 to present.
• SocINDEX with Full Text This link opens in a new window SocINDEX covers the sociology literature, encompassing all sub-disciplines and closely related areas of study. It indexes scholarly journals, books, monographs, and conference papers.
• Sociological Abstracts This link opens in a new window Sociological Abstracts represents the core database for sociological scholarship, covering literature in sociology and related disciplines in the social and behavioral sciences. It indexes scholarly journals, dissertations and theses, books, and conference papers and proceedings, and is international in scope.

What is Google Scholar? How should I use it?

Google Scholar is a time-saving, scholarly search interface. With Google Scholar, you can locate scholarly journal articles, books and book sections.  For literature searching, specialized databases have more functionality and access more comprehensive results, but Google Scholar is a good tool to use for a search for a known item .

You can configure Google Scholar to:

• Display a link to NYU options for the full-text of an article
• Export your citations to RefWorks or EndNote

Here's how to display NYU links in Google Scholar:

• Open scholar.google.com
• Go to the left screen, 3 bars, "Settings" 
• Click on menu item, "Library Links"
• Type in: NYU
• From the results, put a check mark next to "New York University Libraries-GetIt at NYU"
• Click on Save to save preferences.

The next time you search scholar.google.com , the GetIt at NYU option will be available next to accessible article citations.

Here's how to change settings for Bibliography manager:

• In the Search Results, go to Bibliography Manager
• Select Show Links  to import citation into: (your choice of bibliography manager, BibTex, Refworks, Endnote, etc.)
• << Previous: Images/Streaming Video
• Next: Database Tutorials >>
• Last Updated: Jun 26, 2024 3:00 PM
• URL: https://guides.nyu.edu/speech

• Open access
• Published: 25 June 2024

Understanding barriers and facilitators to palliative and end-of-life care research: a mixed method study of generalist and specialist health, social care, and research professionals

• Catherine Walshe 1   na1 ,
• Lesley Dunleavy 1   na1 ,
• Nancy Preston 1 ,
• Sheila Payne 1 ,
• John Ellershaw 2 ,
• Vanessa Taylor 3 ,
• Stephen Mason 2 ,
• Amara Callistus Nwosu 4 ,
• Amy Gadoud 4 ,
• Ruth Board 5 ,
• Brooke Swash 6 ,
• Seamus Coyle 7 ,
• Andrew Dickman 8 ,
• Andrea Partridge 5 ,
• Jaime Halvorsen 9 &
• Nick Hulbert-Williams 10  

BMC Palliative Care volume  23 , Article number:  159 ( 2024 ) Cite this article

Metrics details

Palliative care provision should be driven by high quality research evidence. However, there are barriers to conducting research. Most research attention focuses on potential patient barriers; staff and organisational issues that affect research involvement are underexplored. The aim of this research is to understand professional and organisational facilitators and barriers to conducting palliative care research.

A mixed methods study, using an open cross-sectional online survey, followed by working groups using nominal group techniques. Participants were professionals interested in palliative care research, working as generalist/specialist palliative care providers, or palliative care research staff across areas of North West England. Recruitment was via local health organisations, personal networks, and social media in 2022. Data were examined using descriptive statistics and content analysis.

Participants (survey n  = 293, working groups n  = 20) were mainly from clinical settings (71%) with 45% nurses and 45% working more than 10 years in palliative care. 75% were not active in research but 73% indicated a desire to increase research involvement. Key barriers included lack of organisational research culture and capacity (including prioritisation and available time); research knowledge (including skills/expertise and funding opportunities); research infrastructure (including collaborative opportunities across multiple organisations and governance challenges); and patient and public perceptions of research (including vulnerabilities and burdens). Key facilitators included dedicated research staff, and active research groups, collaborations, and networking opportunities.

Conclusions

Professionals working in palliative care are keen to be research active, but lack time, skills, and support to build research capabilities and collaborations. A shift in organisational culture is needed to enhance palliative care research capacity and collaborative opportunities across clinical and research settings.

Peer Review reports

Palliative care provision should be informed by high quality research, so that clinical practice is underpinned by a robust evidence base. Improving the evidence base in palliative care is a ‘moral imperative’, with arguments highlighting that it is ethically important to offer effective treatments supported by an evidence base, and equally that futile treatments are avoided [ 1 ]. A principal focus of much of the research conducted to understand why developing the evidence base is difficult has focused on the specific challenges of recruiting patient and carer participants to palliative care research studies. Gatekeeping can be an issue, with staff concerned about overburdening vulnerable patients and carers, and feeling ill prepared to discuss research with potential participants [ 2 , 3 , 4 ]. This is despite evidence suggesting patients and families are willing to engage in research at the end of life [ 5 , 6 , 7 ]. Despite this readiness, there can be many reasons why patients and carers may not feel able to engage in research such as illness severity, symptom burden, misconceptions about palliative care, lack of cure and perceived therapeutic benefit, and study burden [ 8 , 9 , 10 ]. This can mean that many studies experience recruitment difficulties [ 11 , 12 ]. Facilitators that may address some of these complex structural, cultural and personal barriers include dedicated research staff on site [ 3 , 13 ], training on how to recruit to palliative care studies [ 14 , 15 ], and improving communication with patients and their families to promote research participation, and within staff teams to address gatekeeping.

Researchers outside palliative care have chosen to explore the professional and organisational facilitators and barriers to conducting research [ 16 , 17 ]. Less is known about the personal, professional, organisational, and structural barriers and facilitators to conducting palliative care research. Palliative care requires a multi-professional approach, and patients are cared for in a variety of settings, including hospitals, hospices, nursing homes and primary care. Palliative care research is historically under-funded in comparison to research that focuses on the prevention or cure of cancer and other life-limiting illnesses [ 18 , 19 ]. There may also be challenges with access to staff with the relevant research expertise, and complicated or undeveloped governance arrangements particularly in settings outside statutory provision [ 20 , 21 , 22 , 23 ]. Research may not be a strategic priority, especially for standalone voluntary organisations who largely rely on charitable funding to fund patient care [ 23 ]. Palliative care research can be time consuming and staff may see it is an ‘add on’ to their role and not part of the routine care they provide to patients [ 24 ]. Staff may feel that they lack the necessary knowledge, skills and expertise to be involved in palliative care research [ 4 , 25 ] and may have limited opportunity to participate or learn more, especially when balancing clinical pressures that have increased during the COVID 19 pandemic [ 26 ]. An organisational research culture improves outcomes for all patients, and not just those involved in the research [ 27 ]. The aim of this study therefore is to further understand professional and organisational facilitators and barriers to conducting all types of palliative care research.

Research question

What are the barriers and facilitators to conducting palliative and end-of-life care research across North West Coast England ?

A mixed method study following a convergent design [ 28 ] , incorporating a cross-sectional online survey and working groups using a nominal group technique [ 29 ]. The survey is reported according to the CHERRIES guidelines for e-surveys [ 30 ].

Both the survey and working groups were conducted across the UK NIHR North West Coast region of England (incorporating South Cumbria, Lancashire, Cheshire, and Merseyside). Currently, palliative care research activity within this area is low. In the UK, palliative care is provided by generalists, the patient’s usual care team, in the hospital, community or care home setting. Specialist inpatient, hospital, home and home nursing palliative services are provided by professionals specifically trained in palliative care, and they largely rely on charitable funding [ 31 , 32 ].

All those who had interest in the provision of, or research into, generalist or specialist palliative care across the region including across acute and community NHS Trusts, GP practices, voluntary hospices, other community and private providers of care, clinical research networks, and academic settings including Universities were invited to participate. The survey was accessed via an online link that included a screening question incorporating the inclusion criteria (see Table  1 ).

Survey: The survey used a convenience sampling approach and was designed to collect largely descriptive data and yield rich information across a range of respondents. Without a viable sampling frame of potential participants, no anticipated sample size could be reliably estimated. Working groups : Those who indicated an interest in taking part via their survey response, or who responded to additional calls for participation, were invited to participate, and then purposively selected to maximise variability across professional background, expertise, and geography.

Recruitment

Survey: Potential participants were recruited via several routes that included dissemination via collaborators in local NHS Trusts and Hospices and the North West Coast Clinical Research network to ensure primary care organisations were reached. Information about the survey was openly and widely disseminated through a project website, personal networks, and social media (Twitter, Facebook, and LinkedIn). No incentives for survey completion were offered. Dissemination included a link to the online survey, with screening questions at the start of the survey confirming eligibility, with clicking through to progress to the survey indicating consent. Potential participants were reassured that taking part was voluntary and that survey results would be aggregated and anonymised. It was explained that their data would be inputted into a secure online survey platform, and these data would be then stored in a secure institutional filestore at Lancaster University. (see additional file 1).

Working groups

Individuals who expressed an interest in taking part in further research after completing the survey were sent working group invitation packs. Additionally, collaborators in local NHS Trusts, Hospices and the North West Coast Clinical Research network circulated packs to eligible participants. Social media (Twitter, Facebook, and Instagram) was also used to advertise the working groups. Participants could take part in the working groups even if they had not completed the survey. Participants contacted the research team if they were interested in taking part and electronic consent was obtained prior to the working group.

Data collection

Survey: The open online survey was built using Qualtrics XM [ 33 ], and the full survey is included in additional file 1. Both closed and free-text questions were used, together with skip options dependent on given answers; 19 possible questions (some with multiple components) were asked across 5 blocks. Participants could navigate through the survey using forward and back buttons. The survey identified current and desired levels of palliative care research involvement, current research barriers, suggestions for sustainable solutions and research training needs. The survey was developed from the IPOS survey (a survey of the research barriers and training needs within the International Psycho-Oncology Society) [ 34 ] and literature on barriers and facilitators to palliative care research [ 3 ]. Survey development followed an iterative approach, with members and colleagues of the project steering committee reviewing survey questions to ensure the survey was appropriate. Participants could only complete the survey once. There was not a completeness check for respondents. The survey was open from 02/03/2022 to 08/06/2022.

Four online (via Microsoft Teams) working groups took place. The groups lasted two hours and were facilitated by LD and another member of the research team (from CW, AG, BS, RB). Nominal group technique was used as it is a method that elicits the views and opinions of a group of experts through the ranking of priorities related to a particular topic of interest. It combines both qualitative and quantitative data collection and involves a number of stages that include; introductions, silent generation of ideas, listing of ideas, discussion of ideas, ranking of top ten ideas, voting on top ten ideas, discussion of voting and conclusions [ 29 ]. Mentimeter [ 35 ] was used to facilitate the voting process and the working groups were recorded.

Data analysis

Survey: Data were downloaded from Qualtrics™ as.csv and.sav files for Excel and SPSS, hosted on Lancaster University secure OneDrive, and checked for potential duplicate entries (using IP, email address or organisation name to ensure only one entry per respondent), and to remove incomplete entries. Entries were judged as complete when participants had provided sufficient descriptive personal information alongside survey responses, even if answers to all available questions had not been given. Pseudonymised data were used for analysis. Descriptive analysis included the use of frequency counts, percentages, and rankings, with some collapsing of categories.

For the analysis of free-text comments, data were extracted into Microsoft Excel. Comments tended to be brief, expanding on answers to closed questions [ 36 , 37 ]. After initial familiarisation, a coding framework was inductively developed by LD and CW and applied to the free text data using a conventional content analysis technique [ 38 ]. Coding and theme development were driven by the content of the free-text comments.

Working groups, using nominal group technique

Each working group was initially analysed separately by LD using the group’s Mentimeter rankings as an initial a priori framework [ 39 ]. The working group recordings and transcripts were read and listened to, and the key issues were summarised within the a priori frameworks. The findings were then compared across the working groups by LD, SM, BS, and AP with input from the study’s Patient and Public Involvement group and finally the study steering committee, to identify key themes.

Four overarching groupings were inductively generated after completion of the working groups. Survey free text and working group findings were compared as part of the four theme development. Mentimeter rankings were allocated to the four groups along with the survey statements where there was strongest agreement about the barriers to research across all survey respondents (see Table 5. ).

Approval was granted by the East of England—Cambridge South Research Ethics Committee (Ref: 22/EE/0049) on the 24/02/2022. Organisational approval was obtained via the Health Research Authority and each participating site.

Survey response

The online survey received 495 visitors, of whom 8 declared they did not meet the inclusion criteria, 36 provided no data, and 158 did not proceed beyond the screening questions. Valid responses were received from 293 participants (59% of visitors), with 171 of the 293 (58%) recording 100% survey progress, and a mean progress of 82% (range 100% to 25%).

Characteristics of survey respondents

Full descriptive data from these respondents are found in Table 2 . The highest proportion of respondents worked in hospice settings, were nurses, and had worked in palliative care for over 10 years. Unexpectedly, there was a high number of paramedics who completed the survey ( n  = 17).

Characteristics of working group participants

Twenty palliative care providers/research staff participated in the working groups (see Table 4  for details).

Barriers and facilitators to participating in palliative care research (quantitative data)

Survey respondents were asked to indicate the strength of agreement with statements about facilitators or barriers to engagement and involvement with palliative care research. Working group participants inductively generated statements about barriers which were then ranked. In Table 5.  below we present the survey statements where there was strongest agreement across all survey respondents, together with the ranking of inductively generated statements from each of the working groups. Full survey data are found in additional file 2.

The top research barriers were conceptualised across four main areas: organisational culture and capacity (including prioritisation and time given to research); research knowledge (including research skills, how to obtain funding); research infrastructure and collaborations (including collaborative opportunities and governance arrangements), and patient and public perceptions of palliative care research (including vulnerabilities and burdens). Data on facilitators and training needs were collected in the online survey and are presented in Tables 6 and 7 .

Barriers to participating in palliative care research (qualitative data)

Additional data on the four areas of organisational culture and capacity, research knowledge, research infrastructure and collaborations, and patient and public perceptions of research were generated in both the free text comments from the survey and working group analysis. A narrative exploring each of these is presented in turn, illustrated with verbatim data extracts from the working groups and survey.

Organisational culture and capacity

This was the top barrier identified in the survey and most working groups. The focus was about whether research is prioritised within the organisation, including if people are enabled to conduct research in terms of protected time. Across the working groups and survey, participants explained how staff have no time to be involved in research because of clinical pressures and commitments. Staffing shortages, patient complexity, and the impact of COVID 19 have made the situation even more challenging for clinicians:

‘It's really difficult because everyone is so stretched that everybody's so busy sort of, you know, the AHP's [allied health professionals], the doctors, the nurses, everyone's very busy, sort of fighting fires that nobody's got time to move away from that at the moment’ (Hospice Doctor, working group 2)

‘The main barrier from my experience is not having protected time to spend in research activities. My case load is vast and give me no time to participate in research. This is disheartening to me as we need to constantly develop and not stagnate. Also, with palliative care we get one opportunity to make that difference so we need to be equipped with the best we can do.’ (Survey study ID 163, Hospital Doctor)

Organisational culture and external requirements also mitigate against engagement in palliative care research, where priority is given to meeting key performance indicators, which rarely include research engagement:

‘The clinical demands and their key performance indicators required by our service specifications and our trust, demand that you spend the majority of your time 90% if not more, undertaking clinical aspects of the role and that there isn't necessarily buy in [to research] I don't feel from the senior management within the organisation to support us’ (Palliative care nurse specialist, working group 1)

Research not being part of an organisations culture and ethos and therefore not seen as a strategic priority was an important barrier.

‘Even if someone said here's some funding, what do you want? We reel off a million answers, but research would probably be at the bottom just because there's other things that we need or want that we feel is probably more important than research. Whether that's right or wrong, I think it's just not. Not a priority. It's no one’s first thought.’ (Hospice nurse, Working group 2)

Participants highlighted the need for a ‘research champion’ within an organisation who would be responsible for leading, prioritising and raising the profile of research therefore making research less daunting for staff.

‘I think you're somebody who's motivated to drive a research agenda forward, I think makes a big difference in the organisation that you're in, whether that's hospital based or community Hospice and based because I think if you haven't got anybody who's keen and enthusiastic, you're not going to go anywhere. So you've got to have someone who's willing to take that on.’ (Hospital Doctor, Working group 4)

Research knowledge

Health and social care staff can have a limited understanding of research processes, and therefore may not have the necessary skills to conduct research. Whilst some basic knowledge was covered at pre- and post-registration undergraduate or postgraduate level, continuing to develop skills and knowledge could be challenging:

‘We're encouraging our staff to undertake further education or sort of masters level qualifications, and at that level it does require for the qualification a piece of research and a number of research questions to be undertaken, but it's how do we move beyond that?’ (Hospice manager/admin Working group 1)

You do the research project within the course to get through the course and then you know you like, breathe sigh of relief and then you don't go near research again.’ (Palliative care nurse specialist, Working group 1)

Research can feel distant and overwhelming, academic and jargon filled, without relevant pathways to support professional development:

‘I think from a perspective of peoples understanding and knowledge of research and where to get support and there's a lot of people shy away from it because they don't know where to start. They don't know where to go to. They don't know how to find the literature and they just feel like they're in a minefield of information they don't know which avenue to take.’ (Hospice nurse, Working group 4)

The need for mentorship, support, and guidance from more experienced research staff and how to access this support was clearly identified. Engaging junior staff was seen as important and training sessions/e-learning needed to be accessible, including tailored resources for palliative care, and level of involvement in research.

‘If people haven't done a lot of research and they want to be involved and it's sort of supporting that group of people if they haven't got links to people already or groups within their organization or network that they can link into, and they're really interested in it, it's getting those people involved and how to direct them?’ (Hospice nurse, Working group 4)

‘Need the support of an experienced researcher and also someone to help plan and develop the research, mentor and guide throughout research project and assist with analysis of results-/stats and writing up the project.’ (Survey study ID 39 specialist palliative care clinical manager)

Participants explained how there tended to be a lack of research expertise (e.g. knowledge of research processes) within hospices and how it was important to have someone with the right skill set in the setting/small organisation.

‘Having somebody with the right skill set to take something through ethics committee and everything I suppose, and you need to have that one person in every Hospice or in every setting who can do all that. It's a skill all of its own.’ (Manager/admin, Working group 2)

Research infrastructure and collaborations

Palliative care research was felt to have a weak infrastructure, with few studies in the National Institute for Health Research (NIHR) portfolio, limiting opportunities to be involved in research and access to research nurse support. Hospices had few financial resources to support research activity, and seemed reluctant to divert funds from direct patient care:

‘So, there's huge financial implications in terms of them [charitably funded hospices] providing sort of and delivering research … it was a massive competing pressure on money because you don't want to be impacting on the organisations finances and within the charity sector to the detriment of immediate patient care.’ (Hospice Doctor, working group 1 )

‘Releasing people to take part in research is just impossible for a Hospice with our current funding arrangements. Research feels like a "nice to have" aspect of Hospice work. Even though I know it would be valuable to our sector long-term to be research active, the climate we find ourselves in means research is way down the list of priorities for a charity receiving 30% (and diminishing) funding [from the NHS] to run a 24/7 service.’ (Survey Study ID 85, Hospice CEO)

The lack of or limited research infrastructure outside the hospital setting, particularly within standalone hospices, was raised as a barrier. The necessary structures to support research activity, such as governance arrangements, training, and adequate staffing levels, could often be lacking.

‘I think when you're working with within small groups you could be quite isolated with only having one research nurse who then is on their own, and I think the link I think that's probably an issue in terms of I guess the funding for that person. It can be an issue but also attracting somebody to a post which is going to feel quite isolating.’ (Hospital Doctor, Working group 4)

‘But the thought of actually undertaking some research ourselves. We're a million miles away from that in our hospice you know. We are trying to be involved in other bigger trials, but where to actually put through an ethical approval ourselves. We're nowhere near that here.’ (Hospice Doctor, Working group 2)

The importance of engaging nursing and allied health professionals in research and giving them the opportunity to be involved was raised. The four pillars of professional practice of the clinical nurse specialist and advanced practitioner roles includes research alongside clinical, education and leadership components [ 40 ]. However, research is not always recognised or developed. It was noted that organisations support training in Independent and Supplementary Prescribing, diagnostics, and advanced communication skills, so it was questioned why not research. Some short-term research positions may not provide opportunities for all staff, as posts may be linked to certain roles (e.g. medical, nursing) or require professional registrations, thus limiting opportunities for staff without these qualifications (e.g. healthcare assistants). The importance of recognising the role and expertise of non-clinical staff in research and its potential impact on care and services needs to be promoted.

Currently, there was not a strong sense that people or organisations were working collaboratively locally or regionally to facilitate research:

‘We don't work collaboratively, and we have a really big list of research projects that we'd like to do. We'd like to get started on. We don't have the capacity to do it, but actually other hospices or other professionals in palliative care might be working on it. But we just don't know because we don't talk to each other. Perhaps we just need to talk more?’ (Manager/admin Working group 2)

‘I think we're all busy, aren't we? So, the opportunity to meet, collaborate, share ideas doesn't to me seem like it's there. I could be wrong, but I think lack of existing collaboration, just perhaps due to how busy we all are individually, and rather than what I didn't mean, was competitiveness between hospices, yeah.’ (Hospice nurse, Working group 3)

‘From a researcher perspective, the barriers I face are around making the necessary connections with relevant practitioners interested and available to work on research projects. This is partly to do with few opportunities to meet people in informal environments where research priorities or interests can be discussed….(Survey study ID 43 researcher)

The need for some form of alliance or collaborative infrastructure was highlighted to pool research ideas, share information, collaborate on policies and governance issues. This was felt to need buy in from multiple organisations, potentially with a funded post to lead on research across voluntary hospices:

‘it's almost like we need some sort of alliance, isn't it? And that may well be where all this is headed and in terms of, you know, somewhere in the region somebody's putting a bid in for this research and who wants to jump on board to recruit in their area to get some opportunity for the expertise.’ (Palliative care nurse specialist, Working group 1)

‘And so maybe having some kind of umbrella group or network that… then everything kind of filters through it and information comes back out the other way so that that information is shared and you kind of know where to go. Maybe if you've got an idea to check that no one else is already doing it and to be in touch with the right people at the right time, I don't know if something around the kind of coordination of the whole thing.’ (Hospice manager/admin, Working group 2)

There were concerns raised that the palliative care research community involved a select group of individuals and could be elitist. It could be difficult for those sitting outside the elite to know how to be involved and included in any research activity:

‘I did reflect on initially when I got interested in research it was sort of seen as this area of expertise in which a select group were involved, and it was sort of how do we get into that Network.’ (Hospice nurse, Working group 4)

Patient and public perceptions of palliative care research

Concerns were also raised that patient and public perceptions of palliative care research may be an issue either because there were assumptions that research was not happening, or only in large/cancer settings, that people did not want to take part, or that the end of life is an inappropriate time to request participation.

‘Sometimes staff feel oversensitive. Almost oversensitive to not wanting to upset patients and relatives to recruit them in, or to ask the relevant questions that we need them to ask.’ (Hospice educator, Working group 2)

However, counter arguments were also recognised:

‘Anecdotally, we've had people tell us when they've taken part in studies that we've done, that they've enjoyed taking part that it's been beneficial for them, not because the research will impact them, but because of the process of...I guess the therapeutic aspect that's a side line to them taking part that they've enjoyed taking part and sharing. Their views and being able to put something back and to help other people.’ (Researcher, Working group 3)

The aim of this research is to understand professional and organisational facilitators and barriers to conducting palliative care research. Palliative care research was recognised as important and valuable, with three-quarters of those involved in this study wanting to increase their involvement in research, despite most not being currently research active. Several key barriers to palliative care research were identified including lack of organisational research culture and capacity (including prioritisation and available time); research knowledge (including skills/expertise and funding opportunities); research infrastructure and collaboration (including lack of collaborative opportunities across multiple organisations and governance challenges); and patient and public perceptions of research (including vulnerabilities and burdens). Key facilitators included dedicated research staff, and active research groups, collaborations, and networking opportunities.

What this research adds

A key finding is the apparent lack of progress in facilitating palliative care research over time, and the challenge for the sector is why change has been so slow. Previous palliative care research identifies a suite of remarkably similar barriers [ 23 , 41 , 42 , 43 , 44 ], albeit not necessarily unique to this specialty [ 45 , 46 ]. There needs to be a concerted and sustained focus on collaboration and sharing best practice, developing a research culture and facilitating research within and between palliative care providers, enhancing staff capacity and expertise, and providing guidance on research processes and procedures [ 23 , 41 , 43 , 44 ]. Our research further highlights the importance of organisational barriers, pointing to the need to prioritise organisational solutions.

Organisations have a critical role in building research culture and capacity [ 46 , 47 , 48 ]. It is imperative that organisations recognise and value research and incorporate research into the core business of the organisation. This means that research should be visible throughout, from mission statements to policies, business plans, and job descriptions. They should protect research time and resources, recognise talent, and reward positive research related behaviours [ 48 ]. This may be a particular challenge for those palliative care organisations that are charitably funded due to the uncertainty and volatility of their funding [ 49 , 50 ], and business models that may not account for research activity [ 51 ]. The focus is also set nationally, with the recently launched Hospice UK 2024–29 strategy having no overt mention of research [ 52 ].

A key finding is that for many the organisational lack of support for research translates into research not being seen as a core part of people’s jobs. Again, this is not unique to palliative care, with capacity to be engaged in research limited in time or job plans [ 53 ]. As an example an audit of clinical nurse specialist job descriptions found that 80% had an expectation of research engagement [ 40 ], however, in detailed studies of how such roles are enacted, research is typically absent [ 54 , 55 ]. Where research is mentioned, it was in the context of it being the least important aspect of the role, or that others (such as medical consultants) should be leading research [ 56 ]. However, whilst there is little contemporary data, previously the median time palliative care consultant doctors spent on research was zero hours [ 57 ]. A recent survey of UK palliative medicine consultants found that while 78% ( n  = 140/180) were interested in conducting research, 83% had no allocated time within their job plan [ 58 ]. Given the serious and significant workforce pressures and challenges currently facing many healthcare workers it is unlikely this position will change without both investment in, and prioritisation of, research time and roles. It may be that research time or engagement needs to explicitly form part of key performance indicators or other metrics to enable such prioritisation to occur.

Research should be important to palliative care provider organisations. It is known that a strong research culture and organisational research performance lowers mortality rates, increases patient and staff satisfaction, reduces staff turnover, and improves organisational efficiency [ 59 ]. Our research encompassed a variety of different organisations and settings, demonstrating that these barriers were remarkably similar wherever a person worked. Solutions may differ though depending on the size, funding, and specialism of the organisation. An independent voluntary funded hospice may have different solutions to a palliative care team working as part of a larger general hospital or community care provider.

The opportunity to collaborate between individuals and across organisations may be important, as in other specialities such as General Practice [ 60 ]. Evidence indicates that the creation of research cooperatives, collaborations and partnerships can be fruitful. There are palliative care examples from the UK [ 61 ], US [ 62 , 63 ], Australia [ 64 , 65 , 66 ], and Africa [ 67 ]. Some of these are large collaboratives, across multiple sites, facilitating multiple studies [ 68 ]. It is possible that such collaboratives mitigate the effect of the employing organisation for members, facilitating research in a way that sits above, and possibly either bypasses, negates, or gives the skills to overcome institutional and local organisational barriers. Joint approaches between universities and public and charitable providers of palliative care may help overcome structural issues such as indemnity, sponsorship and gaining research ethics committee approvals. However, funding to sustain some of these collaborations can be fragile or time limited. For example, in the UK, very welcome but time-limited funding to build palliative care research partnerships has been awarded, but it is too early to see the impact of this on the research landscape [ 69 ]. The benefits of such collaborations may also be on the wider research culture of the organisations that participate in such research. The initial impact of participating in a trial may be staff stress and workload, but this has found to be replaced by enthusiasm for the changes and benefits achieved [ 70 ].

Those who completed our survey had wide variability in levels of research experience and involvement. It is important to recognise when considering developing an organisational research culture that not all members of staff need the same level of skill and expertise, and not all organisations will be at the same level of engagement. Previous recommendations for hospices suggested a typology of engagement, through which hospices could progress if they wished, from research aware, to research engaged, to research leading [ 23 , 43 ]. Equally, individuals can have different levels of preparation, with recognition that generating and leading new research likely needs the higher levels of research preparation such as research focused PhDs, and that organisations that aspire to these levels need to invest in educating staff to these levels and supporting their continued research development.

Strengths and limitations of the research

A strength of this research was the breadth of response from across different sectors and professional backgrounds. There was a particularly strong response from nurses, and a reasonable proportion of those providing general palliative care. However, it was harder to recruit respondents who do not provide specialist palliative care (perhaps because they do not identify themselves as palliative care providers despite the high numbers of those with palliative care needs that they provide care for). Care home respondents were particularly poorly represented. We aimed to invite patients, family members and the public to a working group. Whilst we involved Patient and Public Involvement (PPI) study team members in planning this work and attempted to recruit the public to our working groups, challenges both in institutional permissions and recruitment meant that this planned aspect of the study did not go ahead. This work also represents the views of people from across a particular UK geography. Whilst this includes a large, diverse, population it may be that this does not represent wider views, although this is unlikely given the congruence with past and related research. This study also includes participants who were involved or would wish to be involved in palliative care research so the views of those who are not interested are not reflected in the findings.

Engagement in palliative care research appears stagnant, with this study revealing a range of barriers that appear unchanged from a decade or more ago. The challenge for palliative care is not to identify further the barriers and facilitators to research, but to invest time and funding to address the known barriers and enable the facilitators of research. It is likely that such investments will reap dividends in terms of staff satisfaction, organisational performance, and importantly the quality of care provided to patients and families.

Availability of data and materials

Data are stored in Lancaster University’s PURE repository, consent to share data was not given by participants.

Keeley P. Improving the evidence base in palliative medicine: a moral imperative. J Med Ethics. 2008;34(10):757–60.

Article   CAS   PubMed   Google Scholar  

Kars MC, van Thiel GJ, van der Graaf R, Moors M, de Graeff A, van Delden JJ. A systematic review of reasons for gatekeeping in palliative care research. Palliat Med. 2016;30(6):533–48.

Article   PubMed   Google Scholar  

Dunleavy L, Walshe C, Oriani A, Preston N. Using the ‘Social Marketing Mix Framework’ to explore recruitment barriers and facilitators in palliative care randomised controlled trials? Narrative Synth Rev Palliat Med. 2018;32(5):990–1009.

PubMed   Google Scholar  

Dunleavy L. Health care professional recruitment of patients and family carers to palliative care randomised controlled trials: a qualitative multiple case study. Lancaster: Lancaster University; 2021.

Google Scholar  

Aoun S, Slatyer S, Deas K, Nekolaichuk C. Family caregiver participation in palliative care research: challenging the myth. J Pain Symptom Manage. 2017;53(5):851–61.

Coyle S, Scott A, Nwosu AC, Latten R, Wilson J, Mayland CR, et al. Collecting biological material from palliative care patients in the last weeks of life: a feasibility study. BMJ Open. 2016;6(11):e011763.

Article   PubMed   PubMed Central   Google Scholar  

Bloomer MJ, Hutchinson AM, Brooks L, Botti M. Dying persons’ perspectives on, or experiences of, participating in research: an integrative review. Palliat Med. 2018;32(4):851–60.

Brickey J, Flannery M, Cuthel A, Cho J, Grudzen CR, Blaum C, et al. Barriers to recruitment into emergency department-initiated palliative care: a sub-study of a multi-site, randomized controlled trial. BMC Palliat Care. 2022;21(1):22.

Suzuki K, Ikari T, Matsunuma R, Matsuda Y, Matsumoto Y, Miwa S, et al. The possibility of conducting a clinical trial on palliative care: a survey of whether a clinical study on cancer dyspnea Is acceptable to cancer patients and their relatives. J Pain Symptom Manage. 2021;62(6):1262–72.

Escritt K, Mann M, Nelson A, Harrop E. Hope and meaning-making in phase 1 oncology trials: a systematic review and thematic synthesis of qualitative evidence on patient-participant experiences. Trials. 2022;23(1):409.

Bouça-Machado R, Rosário M, Alarcão J, Correia-Guedes L, Abreu D, Ferreira JJ. Clinical trials in palliative care: a systematic review of their methodological characteristics and of the quality of their reporting. BMC Palliat Care. 2017;16(1):10.

Kaiser KS, McGuire DB, Keay TJ, Haisfield-Wolfe ME. Methodological challenges in conducting instrumentation research in non-communicative palliative care patients. Appl Nurs Res. 2020;51:151199.

Preston NJ, Farquhar MC, Walshe CE, Stevinson C, Ewing G, Calman LA, Burden S, Brown Wilson C, Hopkinson JB, Todd C. Strategies designed to help healthcare professionals to recruit participants to research studies. Cochrane Database Syst Rev. 2016;2(2):MR000036. https://doi.org/10.1002/14651858.MR000036.pub2 .

Evans C, Yorganci E, Lewis P, Koffman J, Stone K, Tunnard I, et al. Processes of consent in research for adults with impaired mental capacity nearing the end of life: systematic review and transparent expert consultation (MORECare_Capacity statement). BMC Med. 2020;18(1):1–55.

Article   Google Scholar  

Higginson I, Evans C, Grande G, Preston N, Morgan M, McCrone P, et al. Evaluating complex interventions in End of Life Care: the MORECare Statement on good practice generated by a synthesis of transparent expert consultations and systematic reviews. BMC Medicine. 2014;11(111):  https://doi.org/10.1186/741-7015-11-111 .

Paget SP, Caldwell PH, Murphy J, Lilischkis KJ, Morrow AM. Moving beyond ‘not enough time’: factors influencing paediatric clinicians’ participation in research. Intern Med J. 2017;47(3):299–306.

Morrison L, Johnston B, Cooper M. Mixed methods systematic review: factors influencing research activity among nurses in clinical practice. J Clin Nurs. 2022;31(17–18):2450–64.

Higginson IJ. Research challenges in palliative and end of life care. BMJ Support Palliat Care. 2016;6:2–4.

Vinches M, Neven A, Fenwarth L, Terada M, Rossi G, Kelly S, Peron J, Thomaso M, Grønvold M, De Rojas T. Clinical research in cancer palliative care: a metaresearch analysis. BMJ Support Palliat Care. 2020;10(2):249–58. https://doi.org/10.1136/bmjspcare-2019-002086 . Epub 2020 Mar 24.

Dunleavy L, Griggs A, Wiley G, Hughes M. Overcoming the hurdles: setting up clinical trials in three UK hospices. Int J Palliat Nurs. 2011;17(3):131–4.

Moore DC, Payne S, Van den Block L, ten Koppel M, Szczerbińska K, Froggatt K. Research, recruitment and observational data collection in care homes: lessons from the PACE study. BMC Res Notes. 2019;12(1):1–6.

Froggatt K, Best A, Bunn F, Burnside G, Coast J, Dunleavy L, et al. A group intervention to improve quality of life for people with advanced dementia living in care homes: the Namaste feasibility cluster RCT. Health Technol Assess 2020;24(6).

Payne S, Preston N, Turner M, Rolls L. Research in palliative care: can hospices afford not to be involved. A report for the Commission into the future of hospice care. London: Help the Hospices Commission; https://hospiceuk-files-prod.s3.eu-west-2.amazonaws.com/s3fs-public/2022-11/Research%20in%20palliative%20care.pdf . Accessed 20 June 2024. 

Cardenas V, Rahman A, Giulioni J, Coulourides Kogan A, Enguidanos S. Patient and physician perspectives on engaging in palliative and healthcare trials: a qualitative descriptive study. BMC Palliat Care. 2021;20(1):158.

Shepherd V, Hood K, Wood F. Unpacking the ‘black box of horrendousness’: a qualitative exploration of the barriers and facilitators to conducting trials involving adults lacking capacity to consent. Trials. 2022;23(1):471.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bradshaw A, Dunleavy L, Garner I, Preston N, Bajwah S, Cripps R, et al. Experiences of staff providing specialist palliative care during COVID-19: a multiple qualitative case study. J R Soc Med. 2022;115(6):220–30.

National Institute for Health and Care Research. Embedding a research culture 2024 [Available from: https://www.nihr.ac.uk/health-and-care-professionals/engagement-and-participation-in-research/embedding-a-research-culture.htm#one .

Walshe C. Mixed Method Research in Palliative Care. In: MacLeod RD, Van den Block L, editors. Textbook of Palliative Care. Cham: Springer International Publishing; 2018. p. 1–19.

Jünger S, Payne S. The crossover artist: consensus methods in health research. IN Walshe C. Brearley S. Handbook of Theory and Methods in Applied Health Research. London: Edward Elgar Publishing; 2020.

Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Int Res. 2004;6(3):e34-e.

Bausewein C, Higginson IJ. Challenges in defining “palliative care” for the purposes of clinical trials. Curr Opin Support Palliat Care. 2012;6(4):471–82.

Oluyase AO, Hocaoglu M, Cripps RL, Maddocks M, Walshe C, Fraser LK, et al. The Challenges of Caring for People Dying From COVID-19: A Multinational, Observational Study (CovPall). J Pain Symptom Manage. 2021;62(3):460–70. https://doi.org/10.1016/j.jpainsymman.2021.01.138 .

[Available from: https://www.qualtrics.com/uk/core-xm/survey-software/ .

Lambert SD, Coumoundouros C, Hulbert-Williams NJ, Shaw J, Schaffler J. Building the capacity for psycho-oncology research: a survey of the research barriers and training needs within the International Psycho-Oncology society. J Psych Oncol Res Pract. 2020;2(3):e023.

Mentimeter. Mentimeter 2022 [Available from: https://www.mentimeter.com/ .

Garcia J, Evans J, Reshaw M. “Is there anything else you would like to tell us” – methodological issues in the use of free-text comments from postal surveys. Qual Quant. 2004;38(2):113–25.

O’Cathain A, Thomas KJ. “Any other comments?” Open questions on questionnaires – a bane or a bonus to research? BMC Med Res Methodol. 2004;4(1):25.

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Van Breda Adrian D. Steps to analysing multiple-group NGT data. Soc Work Pract Res. 2005;17(1):1–14.

Cannaby A-M, Carter V, Rolland P, Finn A, Owen J. The scope and variance of clinical nurse specialist job descriptions. British J Nurs. 2020;29(11):606–11.

Preston N, Dunleavy L, Rigby J, Griggs A, Salt S, Parr A, et al. Overcoming barriers to research in palliative care: results from a consensus exercise. Palliat Med. 2014;28(6):745-.

Preston N, Payne S, Todd C. Conducting research in palliative care patients: a burden or an opportunity? Int J Palliat Nurs. 2009;15(11):524–5.

Payne S, Turner M. Methods of building and improving the research capacity of hospices. European J Palliat Care. 2012;19(1):34–7.

Casarett D, Karlawish J, Hirschman K. Are hospices ready to participate in palliative care research? Results of a National survey. J Palliat Med. 2002;5(3):397–406.

Friesen EL, Comino EJ. Research culture and capacity in community health services: results of a structured survey of staff. Aust J Prim Health. 2017;23(2):123–31.

Luckson M, Duncan F, Rajai A, Haigh C. Exploring the research culture of nurses and allied health professionals (AHPs) in a research-focused and a non-research-focused healthcare organisation in the UK. J Clin Nurs. 2018;27(7–8):e1462–76.

Golenko X, Pager S, Holden L. A thematic analysis of the role of the organisation in building allied health research capacity: a senior managers’ perspective. BMC Health Serv Res. 2012;12(1):276.

Cooke J, Gardois P, Booth A. Uncovering the mechanisms of research capacity development in health and social care: a realist synthesis. Health Res Policy Syst. 2018;16(1):93.

Theodosopoulos G. Voluntary hospices in England: a viable business model? Accounting Forum. 2011;35(2):118–25.

Scourfield P. Funding a “good death”: the financial crisis facing hospices. Quality Ageing Older Adults. 2023;24(3):97–102.

Haslam C, Tsitsianis N, Theodosopoulos G, Lee E. Accounting for voluntary hospices in England: a business model perspective. Crit Perspect Account. 2018;54:27–40.

Hospice UK. Hospice UK Strategy. 2024–2029. Hospice care for all, for now and forever. London: Hospice UK; 2024.

Crombie A, Borkowski D, Gardner M, Masman K, Howlett O. Understanding the research capacity and culture of a regional allied health workforce. Aust J Prim Health. 2021;27(5):397–403.

Clark D, Seymour J, Douglas H-R, Bath P, Beech N, Corner J, et al. Clinical nurse specialists in palliative care. Part 2. Explaining diversity in the organization and costs of Macmillan nursing services. Palliat Med. 2002;16(5):375–85.

Kerr H, Donovan M, McSorley O. Evaluation of the role of the clinical nurse specialist in cancer care: an integrative literature review. Eur J Cancer Care (Engl). 2021;30(3):e13415.

Connolly M, Ryder M, Frazer K, Furlong E, Escribano TP, Larkin P, et al. Evaluating the specialist palliative care clinical nurse specialist role in an acute hospital setting: a mixed methods sequential explanatory study. BMC Palliat Care. 2021;20(1):134.

Makin W, Finlay IG, Amesbury B, Naysmith A, Tate T. What do palliative medicine consultants do? Palliat Med. 2000;14:405–9.

Wakefield D, Ta Y, Dewhurst F, Hussain J, Chamberlain C, Etkind S. Qualified and motivated, but limited by specialty-specific barriers: a national survey of UK Palliative Medicine consultants research experience. BMJ Support Palliat Care. 2024;14(1):76–86. https://doi.org/10.1136/spcare-2023-004198 .

Harding K, Lynch L, Porter J, Taylor NF. Organisational benefits of a strong research culture in a health service: a systematic review. Aust Health Rev. 2016;41(1):45–53.

Huas C, Petek D, Diaz E, Muñoz-Perez MA, Torzsa P, Collins C. Strategies to improve research capacity across European general practice: the views of members of EGPRN and Wonca Europe. Eur J Gen Pract. 2019;25(1):25–31.

Payne S, Seymour J, Molassiotis A, Froggatt K, Grande G, Lloyd-Williams M, et al. Benefits and challenges of collaborative research: lessons from supportive and palliative care. BMJ Support Palliat Care. 2011;1(1):5–11.

Abernethy AP, Aziz NM, Basch E, Bull J, Cleeland CS, Currow DC, et al. A strategy to advance the evidence base in palliative medicine: formation of a palliative care research cooperative group. J Palliat Med. 2010;13(12):1407–13.

LeBlanc TW, Kutner JS, Ko D, Wheeler JL, Bull J, Abernethy AP. Developing the evidence base for palliative care: formation of the palliative care research cooperative and its first trial. Hosp Pract. 2010;38(3):137–43.

Philip J, Le B, Pasanen L, Rosens E, Wong A, Mendis R, et al. Palliative care clinical trials: building capability and capacity. J Palliat Med. 2022;25(3):421–7.

Shelby-James TM, Hardy J, Agar M, Yates P, Mitchell G, Sanderson C, et al. Designing and conducting randomized controlled trials in palliative care: a summary of discussions from the 2010 clinical research forum of the Australian palliative care clinical studies collaborative. Palliat Med. 2012;26(8):1042–7.

Eagar K, Watters P, Currow DC, Aoun SM, Yates P. The Australian Palliative Care Outcomes Collaboration (PCOC)–measuring the quality and outcomes of palliative care on a routine basis. Aust Health Rev. 2010;34(2):186–92.

Harding R, Powell RA, Downing J, Connor SR, Mwangi-Powell F, Defilippi K, et al. Generating an African palliative care evidence base: the context, need, challenges, and strategies. J Pain Symptom Manage. 2008;36(3):304–9.

Ritchie CL, Pollak KI, Kehl KA, Miller JL, Kutner JS. Better together: the making and maturation of the Palliative Care Research Cooperative Group. J Palliat Med. 2017;20(6):584–91.

National Institute for Health and Care Research. 21/54 NIHR Palliative and End of Life Care Research Partnerships Cross-programme Funding Committee Public Minutes 28 Sept 2021 2021 [Available from: https://www.nihr.ac.uk/documents/2154-nihr-palliative-and-end-of-life-care-research-partnerships-cross-programme-funding-committee-public-minutes-28-september-2021/29323 .

Grbich C, Abernethy AP, Shelby-James T, Fazekas B, Currow DC. Creating a research culture in a palliative care service environment: a qualitative study of the evolution of staff attitudes to research during a large longitudinal controlled trial (ISRCTN81117481). J Palliat Care. 2008;24(2):100–9.

Download references

Acknowledgements

Not applicable.

This project is funded by the NIHR Palliative and End of Life Care Research Partnerships Funding Committee [NIHR135334]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

Author information

Catherine Walshe and Lesley Dunleavy are joint senior authors.

Authors and Affiliations

International Observatory On End-of-Life Care, Division of Health Research, Faculty of Health and Medicine, Lancaster University, Lancaster, UK

Catherine Walshe, Lesley Dunleavy, Nancy Preston & Sheila Payne

Liverpool University, Liverpool, UK

John Ellershaw & Stephen Mason

University of Huddersfield, Huddersfield, UK

Vanessa Taylor

Lancaster Medical School, Lancaster University, Lancaster, UK

Amara Callistus Nwosu & Amy Gadoud

Lancashire Teaching Hospitals NHS Foundation Trust, Preston, UK

Ruth Board & Andrea Partridge

Chester University, Chester, UK

Brooke Swash

The Clatterbridge Cancer Centre NHS Foundation Trust, Liverpool, UK

Seamus Coyle

Liverpool University Hospitals NHS Foundation Trust, Liverpool, UK

Andrew Dickman

NIHR Clinical Research Network North West Coast, Liverpool, UK

Jaime Halvorsen

Edge Hill University, Ormskirk, UK

Nick Hulbert-Williams

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualisation and funding acquisition: LD, NHW, NP, SP, JE, VT, SM, ACN, AG, RB, BS, SC, AD, AP, JH, CW; Investigation and analysis: LD, NHW, CW, AG, RB, BS; Writing – original draft – CW, LD; Writing – review and editing - LD, NHW, NP, SP, JE, VT, SM, ACN, AG, RB, BS, SC, AD, AP, JH, CW.

Corresponding author

Correspondence to Catherine Walshe .

Ethics declarations

Ethics approval and consent to participate.

Approval was granted by the East of England—Cambridge South Research Ethics Committee (Ref: 22/EE/0049) on the 24/02/2022. Informed consent was obtained from all subjects. Survey instructions clarified that consent to participate was implied when the participant clicked through to the first page of the survey. Electronic consent was obtained prior to the working groups.

All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Walshe, C., Dunleavy, L., Preston, N. et al. Understanding barriers and facilitators to palliative and end-of-life care research: a mixed method study of generalist and specialist health, social care, and research professionals. BMC Palliat Care 23 , 159 (2024). https://doi.org/10.1186/s12904-024-01488-2

Download citation

Received : 16 May 2023

Accepted : 14 June 2024

Published : 25 June 2024

DOI : https://doi.org/10.1186/s12904-024-01488-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Palliative care
• Palliative Medicine
• Surveys and Questionnaires
• Research Priorities
• Health Services Research
• Methodological Studies

BMC Palliative Care

ISSN: 1472-684X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 20 June 2024

Developing survey weights to ensure representativeness in a national, matched cohort study: results from the children and young people with Long Covid (CLoCk) study

• Natalia K Rojas 1 ,
• Bianca L De Stavola 2 ,
• Tom Norris 1 ,
• Mario Cortina-Borja 2 ,
• Manjula D Nugawela 2 ,
• Dougal Hargreaves 3 ,
• Emma Dalrymple 2 ,
• Kelsey McOwat 4 ,
• Ruth Simmons 4 ,
• Terence Stephenson 2 ,
• Roz Shafran 2 ,
• CLoCk Consortium &
• Snehal M Pinto Pereira 1  

BMC Medical Research Methodology volume  24 , Article number:  134 ( 2024 ) Cite this article

137 Accesses

2 Altmetric

Metrics details

Findings from studies assessing Long Covid in children and young people (CYP) need to be assessed in light of their methodological limitations. For example, if non-response and/or attrition over time systematically differ by sub-groups of CYP, findings could be biased and any generalisation limited. The present study aimed to (i) construct survey weights for the Children and young people with Long Covid (CLoCk) study, and (ii) apply them to published CLoCk findings showing the prevalence of shortness of breath and tiredness increased over time from baseline to 12-months post-baseline in both SARS-CoV-2 Positive and Negative CYP.

Logistic regression models were fitted to compute the probability of (i) Responding given envisioned to take part, (ii) Responding timely given responded, and (iii) (Re)infection given timely response. Response, timely response and (re)infection weights were generated as the reciprocal of the corresponding probability, with an overall ‘envisioned population’ survey weight derived as the product of these weights. Survey weights were trimmed, and an interactive tool developed to re-calibrate target population survey weights to the general population using data from the 2021 UK Census.

Flexible survey weights for the CLoCk study were successfully developed. In the illustrative example, re-weighted results (when accounting for selection in response, attrition, and (re)infection) were consistent with published findings.

Conclusions

Flexible survey weights to address potential bias and selection issues were created for and used in the CLoCk study. Previously reported prospective findings from CLoCk are generalisable to the wider population of CYP in England. This study highlights the importance of considering selection into a sample and attrition over time when considering generalisability of findings.

Peer Review reports

By March 2022, most children and young people (CYP) in the United Kingdom (UK) appeared to have been exposed to SARS-CoV-2, with antibodies found in 82% and 99% of primary and secondary school aged pupils, respectively [ 1 ]. Given the scale of infection, a substantial number could develop symptoms of Long Covid (also referred to as Post Covid Condition). Long Covid in CYP can be defined as the presence of one or more impairing, persisting, physical symptom(s) lasting 12 or more weeks after initial SARS-CoV-2 infection that may fluctuate or relapse, either continuing or developing post-infection [ 2 ]. Hence, it is important to study Long Covid, particularly given its potential impact on healthcare systems and need for planning.

Systematic reviews demonstrate that common symptoms of Long Covid in CYP at 3 months post-testing/infection include fatigue, insomnia, loss of smell, and headaches [ 3 ]. The Long Covid (CLoCk) study, is the largest matched cohort study of Long Covid in CYP in the world [ 4 ]. Based in England, CLoCk collected data on over 30,000 CYP testing positive and negative between September 2020 and March 2021 over a two-year period. CLoCk followed 6,804 CYP 3 months after a SARS-CoV-2 PCR-test and found over half of CYP testing negative and 67% of those testing positive reported at least one symptom 3-months post-testing [ 5 ]. The most common symptoms amongst test positives were tiredness (39%), headache (23%) and shortness of breath (23%), with test negatives reporting mainly tiredness (24%) and headache (14%). Results from this, and all other studies, need to be assessed against their methodological limitations, two of which are considered here. First, response rates to study invitation are generally low, for example, the response rate at the 3-months post-testing sweep of the CLoCk study was 13.4% [ 5 ]. Similarly, the UK Office for National Statistics’ [ 6 ] COVID-19 infection survey had a response rate of 12%. Second, all longitudinal studies suffer from attrition over time [ 7 ] which is typically more pronounced in studies with longer follow-up periods [ 8 ].

If non-response and attrition over time systematically differ by sub-groups in the envisioned population, findings could be biased and attempts to generalise findings to the wider population limited [ 9 , 10 , 11 ]. For example, those with particular characteristics (e.g., older, females and from specific ethnicities) are more likely to positively respond to study invitation [ 12 ]. Reasons for attrition over time include study withdrawal, individuals becoming uncontactable [e.g., due to change in contact details; 13 ] or lacking motivation to continue participating. Indeed, both initial non-respondents and those lost to follow-up are often socioeconomically disadvantaged and less healthy [ 14 ]. With studies on Long Covid, particularly those comparing test-positives to test-negatives, an additional source of bias could exist. For example, within the CLoCk study, to isolate the effect of Long Covid from that of living through a pandemic, researchers originally excluded from the analytic sample those (re)infected, that is, test-negatives who subsequently tested positive and test-positive CYP who were subsequently reinfected [ 15 ]. This criterion yields a cohort of CYP who, as per the data available, appear to have either (i) always tested negative, or (ii) tested positive only once. However, these CYP may not be representative of the larger population of CYP in England. One well-established method to assess the impact of potential bias due to non-response, attrition and sample selection is weighting, that is, emphasising the contribution of some individuals over others in an analysis to reconstruct the target population and/or general population [ 9 ]. Such weighting methodology is appropriate when data are missing (due to non-response, attrition, and sample selection) at random [ 16 ], that is, the missingness is dependent on fully observed characteristics such as sex, age, socioeconomic disadvantage and health status. Yet, this powerful statistical technique to address potential selection biases has been underutilised in epidemiological research [ 9 ].

In this manuscript we construct weights for the CLoCk study [ 17 ] and, as an illustrative example, apply them to published findings showing the overall prevalence of shortness of breath and tiredness increases in CYP from baseline (i.e., at the time of their index PCR test) to 12-months post-baseline [ 15 ]. Specifically, to assess the robustness of conclusions drawn from CLoCk data about Long Covid’s symptomatology and trajectory in CYP, the present study aims to (i) create weights for the CLoCk study at its data collection sweeps 3-, 6- and 12-months post-index PCR-test, and (ii) apply developed weights to the analysis of shortness of breath and tiredness over a 12-month period to determine whether accounting for any biases in response, attrition or (re)infection affects published results.

The CLoCk study identified 219,175 CYP (91,014 SARS-CoV-2 Positive and 128,161 SARS-CoV-2 Negative) who had a SARS-CoV-2 PCR-test between September 2020 and March 2021 through the UK Health Security Agency’s (UKHSA) database containing the outcomes of all such tests. At study invitation, test-positives were matched to test-negatives on age, sex, region of residence and month of test. Consenting SARS-CoV-2 Positive and Negative CYP complete a questionnaire about their mental and physical health 3-, 6-, 12- and 24-months post-index PCR-test [ 4 ]. Of note, the sweeps of data collection depend on the CYP’s month of test, with 3-, 6-, 12-, and 24-month data available for some (tested in January-March 2021), while for others only 6-, 12-, and 24-month (tested in October-December 2020), or 12- and 24-month (tested in September 2020 and an additional cohort from December 2020) data were collected. This manuscript is based on all data collected for the 3-, 6-, and 12-month timepoints. The analytic samples for previous CLoCk publications [ 5 , 15 , 18 ] were such that: (i) CYP must have responded within a pre-specified timeframe (i.e., < 24, ≤34, and ≤ 60 weeks post-testing for the 3-, 6-, and 12-month questionnaires, respectively) and (ii) Initial SARS-CoV-2 Negative CYP must have never reported a positive test, with initial SARS-CoV-2 Positive CYP never reporting being reinfected. The latter requirement was determined using a combination of self-report and UKHSA held data. See Figs.  1 and 2 for exclusion criteria at each stage and participant flow.

Logic model for inclusion in the analytic sample at 3-, 6-, and 12-months

a Initially, due to funding constraints, only a portion of those tested in December 2020 were contacted to participate at 6 months. Hence, some children and young people tested in December 2020 provided both 6- and 12- month data, whereas others only 12-month data

b Determined through self-report and UKHSA data. (Re)infected refers to (i) a SARS-CoV-2 Negative subsequently testing positive, or (ii) a SARS-CoV-2 Positive testing positive again

Flow diagram of participants at 3-, 6-, and 12 months

a Determined using the following cut off points: < 24 weeks post-testing for the 3-month questionnaire; ≤ 34 weeks post-testing for the 6-month questionnaire; ≤ 60 weeks post-testing for the 12-month questionnaire

c By definition of a COVID positive episode [ 19 ], a test-positive person cannot be reinfected by 3 months

Research ethics approval was granted by the Yorkshire and The Humber—South Yorkshire Research Ethics Committee (REC reference: 21/YH/0060; IRAS project ID:293,495).

Index COVID status, age, sex and region were determined from data held at UKHSA. Socioeconomic status was proxied using the Index of Multiple Deprivation (IMD), obtained using CYP’s lower super output area (i.e., small local area level-based geographic hierarchy), where higher values are indicative of lower deprivation [ 20 ]. Ethnicity was self-reported and collected at registration. Current (i.e., at time of questionnaire completion) health, current loneliness, and number of symptoms being experienced, including tiredness and shortness of breath, [out of a possible 21, consistent with the ISARIC Paediatric Working Group; 5 ] were self-reported at each data collection sweep. Similarly, standardised measures were collected, including the: Short Warwick and Edinburgh Mental Wellbeing Scale [SWEMWS; 21 ]; EuroQol Visual Analogue Scale [EQ-VAS; 22 ], EQ-5D-Y [ 23 ], Strengths and Difficulties Questionnaire [SDQ; 24 ], UCLA Loneliness Scale [ 25 ], and Chalder Fatigue Scale [CFS; 26 ]. See Additional File 1 : Table  1 for further information.

For each data collection sweep, three indicator variables were created:

Responding given envisioned to take part (Yes/No): If participants completed the whole questionnaire.

Responding timely given responded (Yes/No): If participants who responded, responded to the questionnaire < 24 weeks post-testing (3-month questionnaire); ≤ 34 weeks post-testing (6-month questionnaire) and ≤ 60 weeks post-testing (12-month questionnaire).

(Re)infected given timely response (Yes/No): ‘Yes’ indicates, among those responding timely, SARS-CoV-2 index-test Positives that were reinfected and SARS-CoV-2 index-test Negatives that subsequently tested positive. ‘No’ indicates, among those responding timely, initial SARS-CoV-2 Positives that never report another positive test and initial SARS-CoV-2 Negatives that never report a positive test. A combination of the UKHSA’s testing data and self-reported information on having ever tested positive was used to generate this.

In total nine indicator variables were created: three at each data collection sweep.

Analyses were conducted using Stata v17 [ 27 ].

Weight generation

At each data collection sweep and corresponding to the three indicator variables created (as described above), three ‘mini’ survey weights were generated to account for CYP being lost either due to (i) non-response, (ii) responding after the established cut-off points or (iii) (re)infection with SARS-CoV-2. A fourth, combined ‘envisioned population’ weight was created which accounted for loss in the analytic sample due to all three factors. These four survey weights (three ‘mini’ survey weights and one ‘envisioned population’ weight) were generated for each data collection sweep, (i.e., 3-, 6- and 12-months post-SARS-CoV-2 test), see Fig.  3 for details.

Steps in weight generation

Here, the term ‘envisioned’ population refers to all CYP that could have taken part at the relevant time point (i.e., it is the maximum number of CYP that could provide data at a specific time point and was 50,845, 127,894 and 219,175 at 3-, 6-, and 12-months respectively). The ‘target’ population varies depending on the specific research question. For example, in the illustrative example described below, the target population is all CYP that could have taken part at 6 months (i.e., N  = 127,894; see Fig.  4 ).

Participant flow in the published CLoCk study [ 15 ] to be replicated

a Here, the target population is all children and young people that could have taken part at 6 months

b A late response at 6 months is defined as not responding ≤ 34 weeks post-testing

c Determined through self-report and UKHSA data. (Re)infected refers to (i) a SARS-CoV-2 Negative subsequently testing positive, or (ii) a SARS-CoV-2 Positive testing positive again

d A late response at 12 months is defined as not responding ≤ 60 weeks post-testing

e Of these, 1,826 children and young people registered at 3 months (806 SARS-CoV-2 Negative and 1,020 SARS-CoV-2 Positive)

The three ‘mini’ survey weights were calculated for (i) response given envisioned to take part, (ii) timely response given response, and (iii) (re)infection given timely response. Each ‘mini’ survey weight was calculated as the reciprocal of its corresponding conditional probability (Fig.  3 ). These conditional probabilities were computed using logistic regression (described below).

For the logistic regression of responding given envisioned to take part, all available data (held at UKHSA for study-design matching) and pair-wise interactions were considered as explanatory variables. For the logistic regressions of (i) responding timely given responded and (ii) (re)infected given timely response, questionnaire data was also available for use as predictors. Forward ( p <  0.157) and backward ( p <  0.200) stepwise selection processes were used to refine models used to predict these probabilities with cut-offs selected as per recommendations [ 28 ]. Our weighting approach is appropriate when data are missing at random [ 16 ]. In an attempt to ensure this assumption is valid we included sex, age, region, index COVID Status and IMD in all but one (see below) of the logistic regression models. Of these, age and IMD were continuous variables, while the others were categorical. We determined the appropriate functional form for the relationship between age/IMD and the log odds of the probability of the (three) outcomes by modelling the relationship (i) linearly, (ii) categorically (age: 11–13, 14–15, 16–17 years; IMD deciles, 1–5), (iii) with linear and quadratic terms and (iv) using fractional polynomials with up to two degrees. The functional forms with the lowest Akaike’s information criterion (i.e., the best fitting model) were used in our subsequent models. Importantly, index COVID Status was excluded as a predictor of the probability of being (re)infected given CYP responded timely at 3 months. This is because, by definition of a COVID positive episode [ 19 ], once a person tests positive, they would only be considered to be reinfected should they test positive more than 3 months after the initial positive test. Table  1 summarises the variables included in each model to predict the three conditional probabilities at the three timepoints. When issues with variables perfectly predicting the outcome were encountered, relevant variables were dropped. This only happened at the 3-month time-point. The concordance statistic ( C) was used to assess the predictive performance of the models: values 0.7 and 0.8 denoting good and strong performance, respectively, with a value of ≤ 0.5 indicating poor prediction [Table  1 ; 29 , 30 ].

At each time-point, the envisioned population weight was calculated as the product of the three corresponding ‘mini’ survey weights. Taking the example of 3 months post-testing: to re-weight from the previously used analytic sample to the envisioned CLoCk population, the fourth created survey weight comprised the product of the following three survey weights: Response 3 months , Timely response 3 months , and (Re)infection 3 months (Fig.  3 ). The four survey weights at each time point (twelve in total) are flexible and can be combined as required, to create final survey weights to get to the target population as described in the illustrative example.

Weighting to the general population

Generated survey weights re-weight the analytic sample to the CLoCk envisioned population, that is, CYP invited to participate if they had a PCR-test within the pre-specified timepoints. However, as PCR testing varied by region and stage of the pandemic [ 31 , 32 ], the envisioned population may not be fully representative of the general population of CYP in England. This is because, for example, not all CYP in England will have been able to access/complete a PCR-test. Hence, final survey weights used to get to the required target population were re-calibrated to the general population, using data on sex, age, and region from the 2021 UK Census [ 33 ]. To do this, ratios of the Census data to CLoCk data reweighted to the target population of interest were produced (see Additional File 2 for the interactive tool used to calculate these) with the final target population survey weights then multiplied by these ratios. See Additional File 2 for how this was done for the illustrative example below.

Weight trimming

All survey weights (i.e., each of the response given envisioned to take part, timely response given response, (re)infection given timely response, and the ‘envisioned population’ survey weights) were trimmed to reduce the likelihood of extremely large survey weights increasing variance [ 34 ]. This was done by reducing extreme survey weights to a cut-off defined as the median +  k × interquartile range. k is typically either 3 or 4 [ 35 ]. In the present study we took a conservative approach and set k as 3. All survey weights were multiplied by a factor to re-calibrate back up to the original sum of weights [ 36 ]. When combining survey weights for the illustrative example below, untrimmed survey weights were initially used with the final survey weights trimmed.

Illustrative example: replicating published findings

Findings from CLoCk show the overall prevalence of tiredness and shortness of breath are high in CYP at baseline (i.e., at the time of their index PCR test) and increase over time to 12 months [ 15 ]. Here we compare the prevalence of tiredness and shortness of breath over a 12-month period from a previous publication [ 15 ] to prevalences that were weighted to the (i) target, and (ii) general populations. We demonstrate how uncertainty around generated weights can be accounted for via bootstrapping (with 1000 replications) and supply illustrative code for this (Additional File 1 : Text 1). To be included in the published analytic sample ( n  = 5,085), CYP first registering in January-March 2021 must have completed their 3-month questionnaire (to provide information about their symptoms at the time of their PCR-test, i.e., at baseline), and be in the analytic sample at 6- and 12-months. For those registering in October-December 2020, they must meet the requirements to be included in the analytic samples at both 6- and 12-months (see Fig.  1 for cohort breakdown and Fig.  4 for participant flow for this example). Therefore, longitudinal weights were created by combining the survey weights as detailed in Fig.  5 and further illustrated in the bootstrap example in Text 1 (Additional File 1 ).

Steps taken to combine survey weights to replicate published CLoCk findings [ 15 ]

Note . To be included in the analytic sample, children and young people must have provided information about their symptoms at the time of their PCR test (i.e., 0 months). This information is gathered at study enrolment meaning criteria for inclusion varied depending on month of index PCR-test. Children and young people with an index test in January, February and March 2021 must have responded to the 3-month questionnaire (to gather information about baseline symptoms) as well as meet the criteria for inclusion in the analytic samples at 6- and 12-months post-testing (i.e., responded, done so timely and not (re)infected). Children and young people with an index-test in October, November, and December 2020 only had to meet the criteria for inclusion in the analytic samples at 6- and 12-months

At the 3-month sweep, 7,135 CYP were included in the analytic sample, constituting 14% of the envisioned population at that time-point ( N  = 50,845, Table  2 ; Fig.  2 ). The analytic sample at 6 months ( n  = 12,946) comprised 10% of the envisioned population ( N  = 127,894); at 12-months, 15,624 were included in the analytic sample, forming 7% of the 12-month envisioned population ( N  = 219,175). Overall, 31,012 CYP completed at least one questionnaire, with 42,264 questionnaires completed. CYP in the analytic samples at 3-, 6-, and 12-months completed the questionnaire at a median of 14.9 (IQR: 13.1–18.9), 27.9 (IQR: 26.3–29.7), and 52.7 (IQR: 51.3–54.9) weeks post-testing, respectively. Compared to the envisioned population, CYP in the analytic samples were older, female and from less deprived areas (Table  2 ).

The C statistics for all required conditional probabilities varied between 0.60 (responding timely given responded at 12 months) to 0.77 ((re)infected given timely response at 12-months and 6-months, see Table  1 ). Table  3 displays the survey weights generated for each data collection sweep along with the relevant Ns , medians, and interquartile ranges.

Re-weighting published findings

Consistent with published findings [ 15 ], the overall prevalence of tiredness and shortness of breath increased from baseline to 12-months post-index PCR-test in both test-positive and test-negative CYP even after weighting (and trimming) to the target and general populations (Tables  4 and 5 ; Figs.  6 and 7 ). For example, at time of testing, the unweighted overall prevalence of tiredness in CYP who tested negative for SARS-CoV-2 was 3.63%. When weighted (and trimmed) to the target population the prevalence was 3.51% and when weighted (and trimmed) to the general population the prevalence was 3.69% (Table  4 ). Likewise, prevalences of tiredness and shortness of breath by time of first report remained similar to published findings (Figs.  6 and 7 ). Results using trimmed and untrimmed weights were broadly similar (Additional File 1 : Tables 2 and 3 ; Figs.  1 and 2 ). Table  4 (Additional File 1 ) shows the uncertainty around the generated target population weight (untrimmed); results are broadly consistent.

Weighted (trimmed) and unweighted tiredness prevalences 0-12-months post-index PCR-test by time of first report

Weighted (trimmed) and unweighted shortness of breath prevalences 0-12-months post-index-PCR-test by time of first report

The present study aimed to (i) create weights for the CLoCk study at its data collection sweeps 3-, 6- and 12-months post-index PCR-test, and (ii) apply the developed survey weights to the analysis of shortness of breath and tiredness over the 12-month period to determine whether accounting for any biases in the target population, response, attrition or (re)infection affected published results. Flexible survey weights for the CLoCk study were developed and applied in an illustrative example. When applying the survey weights, results were consistent with published CLoCk findings [ 15 ]. That is, the overall prevalence of tiredness and shortness of breath increased over time from baseline to 12-months post-testing in both test-positive and test-negative CYP.

A major strength of the present study includes the flexibility of the survey weights developed whereby the creation of separate ‘mini’ survey weights (i.e., response, timely response and (re)infection) and the overall ‘envisioned population’ weight ensures researchers are able to combine them to re-create their specific target population, which will vary depending on the specific research question being asked. The interactive tool provided will allow researchers to re-calibrate their target population weights to the general population of CYP in England using the recent Census 2021 data. This re-calibration attempts to address the potential bias in the envisioned CLoCk population due to variation in PCR testing by region and stage of the pandemic [ 31 , 32 ]. Furthermore, by trimming survey weights using a technique that is unaffected by the size of the largest survey weight [ 34 ], we improve the accuracy and precision of final parameter estimates in re-weighted analyses [ 37 ]. Moreover, we used a range of data from both the UKHSA dataset and the CLoCk questionnaire to develop the models that predicted the required conditional probabilities. We acknowledge that the C statistics, particularly for models used to predict the probability of responding given envisioned to take part and the probability of responding timely given responded were somewhat low ranging between 0.60 and 0.73. However, for the probability of responding given envisioned to take part, it should be noted that the C statistic cannot be further improved due to the lack of additional data relating to the envisioned CLoCk population (here, only data held on the UKHSA database for matching was available). Thus, for all survey weight generation, but here in particular, one should note the constraint deriving from the variables used to generate conditional probabilities and the potential for the non-response/attrition/selection mechanisms to be dependent on unmeasured variables. For example, it might be that those with severe tiredness are less likely to respond. Relatedly, our approach is appropriate when missingness is assumed to be dependent on observed characteristics, but as mentioned above this may not be the case. This is an important potential limitation, with the implication being survey weights do not fully adjust for such (non-response, attrition, and sample selection) bias, though we attempt to minimise its impact. In an attempt to avoid potential recall bias, for the latter two ‘mini’ weights, we made the pragmatic decision to only consider questionnaire data asked in relation to health and wellbeing at the time of questionnaire completion.

We acknowledge concerns regarding the use of stepwise selection processes whereby inclusion of too many candidate variables may result in nuisance variables being selected over true variables meaning the best model is not provided [ 38 ]. We were mindful of this when selecting the initial list of potential predictors, determined the best functional forms of continuous variables used in all regressions, and used theoretical arguments to inform our selection, as recommended [ 39 ]. Finally, it should be noted that the survey weights are estimated and if treated as observed there is a risk of overestimating the precision of the estimates. To address this, we provide an example of how variabilities due to generating the weights can be accounted for via bootstrapping.

CLoCk is the largest known prospective study of Long Covid in non-hospitalised CYP, with over 30,000 respondents. Like all longitudinal population-based studies, issues regarding selection into the study and attrition over time need to be considered. The present findings suggest the CLoCk sample is representative of the envisioned and general populations of CYP in England, although the developed weights need to be utilised in multiple and different contexts to assess their impact and identify whether current conclusions are consistent across other CLoCk analyses. The same approach can and should be taken in other research studies to assess sample representativeness. Importantly, application of survey weights more generally is beneficial as a way of addressing the impact of potential bias.

Availability of data and materials

Data are not publicly available. All requests for data will be reviewed by the Children & young people with Long Covid (CLoCk) study team, to verify whether the request is subject to any intellectual property or confidentiality obligations. Requests for access to the participant-level data from this study can be submitted via email to the corresponding author with detailed proposals for approval. A signed data access agreement with the CLoCk team is required before accessing shared data.

Abbreviations

United Kingdom Health Security Agency

United Kingdom

Children and Young People

Children and young people with Long Covid

Strengths and Difficulties Questionnaire

EuroQol Visual Analogue Scale

Index of Multiple Deprivation

Chalder Fatigue Scale

Short Warwick Edinburgh Mental Wellbeing Scale

Office for National Statistics. COVID-19 Schools Infection Survey, England: pupil antibody data and vaccine sentiment, March to April 2022. 2022. https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/covid19schoolsinfectionsurveyengland/pupilantibodiesandvaccinesentimentmarch2022 . Accessed 25 April 2023.

Stephenson T, Allin B, Nugawela MD, Rojas N, Dalrymple E, Pinto Pereira S, et al. Long COVID (post-COVID-19 condition) in children: a modified Delphi process. Arch Dis Child. 2022;107(7):674. https://doi.org/10.1136/archdischild-2021-323624 .

Article   PubMed   Google Scholar  

Behnood SA, Shafran R, Bennett SD, Zhang AXD, O’Mahoney LL, Stephenson TJ, et al. Persistent symptoms following SARS-CoV-2 infection amongst children and young people: a meta-analysis of controlled and uncontrolled studies. J Infect. 2022;84(2):158–70. https://doi.org/10.1016/j.jinf.2021.11.011 .

Article   CAS   PubMed   Google Scholar  

Nugawela MD, Pinto Pereira SM, Rojas NK, McOwat K, Simmons R, Dalrymple E, et al. Data Resource Profile: the children and young people with long COVID (CLoCk) study. Int J Epidemiol. 2023;53(1). https://doi.org/10.1093/ije/dyad158 .

Stephenson T, Pinto Pereira SM, Shafran R, de Stavola BL, Rojas N, McOwat K, et al. Physical and mental health 3 months after SARS-CoV-2 infection (long COVID) among adolescents in England (CLoCk): a national matched cohort study. Lancet Child Adolesc Health. 2022;6(4):230–9. https://doi.org/10.1016/S2352-4642(22)00022-0 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Office for National Statistics. Coronavirus (COVID-19) Infection Survey: technical data. 2023. https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/datasets/covid19infectionsurveytechnicaldata . Accessed 16 Jan 2023.

Plewis I. Non-response in a birth cohort study: the case of the Millennium Cohort Study. Int J Soc Res Methodol. 2007;10(5):325–34. https://doi.org/10.1080/13645570701676955 .

Article   Google Scholar  

Gustavson K, von Soest T, Karevold E, Røysamb E. Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health. 2012;12:918. https://doi.org/10.1186/1471-2458-12-918 .

Article   PubMed   PubMed Central   Google Scholar  

Bu F. Non-response and attrition in longitudinal studies. J Epidemiol Commun Health. 2022;76(12):971. https://doi.org/10.1136/jech-2022-219861 .

Atherton K, Fuller E, Shepherd P, Strachan DP, Power C. Loss and representativeness in a biomedical survey at age 45 years: 1958 British birth cohort. J Epidemiol Commun Health. 2008;62(3):216. https://doi.org/10.1136/jech.2006.058966 .

Article   CAS   Google Scholar  

Drivsholm T, Eplov LF, Davidsen M, Jørgensen T, Ibsen H, Hollnagel H, et al. Representativeness in population-based studies: a detailed description of non-response in a Danish cohort study. Scand J Public Health. 2006;34(6):623–31. https://doi.org/10.1080/14034940600607616 .

Glass DC, Kelsall HL, Slegers C, Forbes AB, Loff B, Zion D, et al. A telephone survey of factors affecting willingness to participate in health research surveys. BMC Public Health. 2015;15:1017. https://doi.org/10.1186/s12889-015-2350-9 .

Young AF, Powers JR, Bell SL. Attrition in longitudinal studies: who do you lose? Aust N. Z J Public Health. 2006;30(4):353–61. https://doi.org/10.1111/j.1467-842x.2006.tb00849.x .

Howe LD, Tilling K, Galobardes B, Lawlor DA. Loss to follow-up in cohort studies: bias in estimates of socioeconomic inequalities. Epidemiology. 2013;24(1):1–9. https://doi.org/10.1097/EDE.0b013e31827623b1 .

Pinto Pereira SM, Shafran R, Nugawela MD, Panagi L, Hargreaves D, Ladhani SN, et al. Natural course of health and well-being in non-hospitalised children and young people after testing for SARS-CoV-2: a prospective follow-up study over 12 months. Lancet Reg Health – Europe. 2022. https://doi.org/10.1016/j.lanepe.2022.100554 .

Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. https://doi.org/10.1136/bmj.b2393 .

Stephenson T, Shafran R, De Stavola B, Rojas N, Aiano F, Amin-Chowdhury Z, et al. Long COVID and the mental and physical health of children and young people: national matched cohort study protocol (the CLoCk study). BMJ Open. 2021;11(8):e052838. https://doi.org/10.1136/bmjopen-2021-052838 .

Pinto Pereira SM, Nugawela MD, Rojas NK, Shafran R, McOwat K, Simmons R, et al. Post-COVID-19 condition at 6 months and COVID-19 vaccination in non-hospitalised children and young people. Arch Dis Child. 2023;archdischild–2022. https://doi.org/10.1136/archdischild-2022-324656 .

Vivancos R, Florence I. Changing the COVID-19 Case Definition. 2022. https://ukhsa.blog.gov.uk/2022/02/04/changing-the-covid-19-case-definition/ . Accessed 25 Jan 2023.

Ministry of Housing Communities & Local Government. English indices of deprivation 2015. 2015. https://www.gov.uk/government/statistics/english-indices-of-deprivation-2015 . Accessed 25 April 2023.

Child Outcomes Research Consortium. Short Warwick-Edinburgh Mental Wellbeing Scale (SWEMWS). 2022. https://www.corc.uk.net/outcome-experience-measures/short-warwick-edinburgh-mental-wellbeing-scale-swemws/#:~:text=The%20SWEMWBS%20is%20a%20short,aim%20to%20improve%20mental%20wellbeing . Accessed 27 Sept 2022.

Feng Y, Parkin D, Devlin NJ. Assessing the performance of the EQ-VAS in the NHS PROMs programme. Qual Life Res. 2014;23(3):977–89. https://doi.org/10.1007/s11136-013-0537-z .

Wille N, Badia X, Bonsel G, Burström K, Cavrini G, Devlin N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010;19(6):875–86. https://doi.org/10.1007/s11136-010-9648-y .

Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337–45. https://doi.org/10.1097/00004583-200111000-00015 .

Office of National Statistics. Children’s and young people’s experiences of loneliness. 2018. https://tinyurl.com/CYPExperiencesOfLoneliness . Accessed 13 April 2021.

Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, et al. Development of a fatigue scale. J Psychosom Res. 1993;37(2):147–53. https://doi.org/10.1016/0022-3999(93)90081-p .

StataCorp. Stata Statistical Software: Release 17. 2021.

Heinze G, Wallisch C, Dunkler D. Variable selection – a review and recommendations for the practicing statistician. Biom J. 2018;60(3):431–49. https://doi.org/10.1002/bimj.201700067 .

Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: Wiley; 2000.

Book   Google Scholar  

Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable. BMC Med Res Methodol. 2012;12:82. https://doi.org/10.1186/1471-2288-12-82 .

UK Health Security Agency. Surge testing for new coronavirus (COVID-19) variants. 2021. https://www.gov.uk/guidance/surge-testing-for-new-coronavirus-covid-19-variants#how-to-get-a-test . Accessed 25 Jan 2023.

UK Health Security Agency. People with a positive lateral flow test no longer required to take confirmatory PCR test. 2022. https://www.gov.uk/government/news/people-with-a-positive-lateral-flow-test-no-longer-required-to-take-confirmatory-pcr-test#:~:text=(COVID%2D19)-,People%20with%20a%20positive%20lateral%20flow%20test%20no,to%20take%20confirmatory%20PCR%20test . Accessed 25 Jan 2023.

Office for National Statistics. Census 2021 results. 2022. https://census.gov.uk/census-2021-results . Accessed 18 Nov 2022.

Potter FJ, Zheng Y, editors. Methods and Issues in Trimming Extreme Weights in Sample Surveys. 2015.

Van de Kerckhove W, Mohadjer L, Krenzke T, editors. A Weight Trimming Approach to Achieve a Comparable Increase to Bias across Countries in the Programme for the International Assessment of Adult Competencies. JSM 2014 - Survey Research Methods Sect. 2014.

Akinbami LJ, Chen TC, Davy O, Ogden CL, Fink S, Clark J, et al. National Health and Nutrition Examination Survey, 2017-March 2020 Prepandemic file: Sample Design, Estimation, and Analytic guidelines. Vital Health Stat. 2022;1(190):1–36.

Google Scholar  

Lee BK, Lessler J, Stuart EA. Weight trimming and propensity score weighting. PLoS ONE. 2011;6(3):e18174. https://doi.org/10.1371/journal.pone.0018174 .

Chowdhury MZI, Turin TC. Variable selection strategies and its importance in clinical prediction modelling. Fam Med Community Health. 2020;8(1):e000262. https://doi.org/10.1136/fmch-2019-000262 .

Smith G. Step away from stepwise. J Big Data. 2018;5(1):32. https://doi.org/10.1186/s40537-018-0143-6 .

Download references

Acknowledgements

Michael Lattimore, UKHSA, as Project Officer for the CLoCk study.

Olivia Swann and Elizabeth Whittaker designed the elements of the ISARIC Paediatric COVID-19 follow-up questionnaire which were incorporated into the online questionnaire used in this study to which all the CLoCk Consortium members contributed.

This work is independent research jointly funded by the National Institute for Health and Care Research (NIHR) and UK Research & Innovation (UKRI) who have awarded funding grant number COVLT0022. SMPP is supported by a UK Medical Research Council Career Development Award (MR/P020372/1). DH is supported by the NIHR through the Applied Research Collaboration (ARC) North-West London and the School of Public Health Research. All research at Great Ormond Street Hospital Charity NHS Foundation Trust and UCL Great Ormond Street Institute of Child Health is made possible by the NIHR Great Ormond Street Hospital Biomedical Research Centre. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR, UKRI or the Department of Health and Social Care.

Additional members of the CLoCk Consortium.

Trudie Chalder, King’s College London, [email protected].

Tamsin Ford, University of Cambridge, [email protected].

Isobel Heyman, University College London, [email protected].

Shamez Ladhani, St. George’s University of London and UK Health Security Agency, [email protected].

Marta Buszewicz, University College London, [email protected].

Esther Crawley, University of Bristol, [email protected].

Bianca De Stavola, University College London, [email protected].

Shruti Garg, University of Manchester, [email protected].

Anthony Harnden, Oxford University, [email protected].

Michael Levin, Imperial College London, [email protected].

Vanessa Poustie, University of Liverpool, [email protected].

Kishan Sharma, Manchester University NHS Foundation Trust (sadly deceased).

Olivia Swann, Edinburgh University, [email protected].

This work is independent research jointly funded by the National Institute for Health and Care Research (NIHR) and UK Research & Innovation (UKRI) who have awarded funding grant number COVLT0022. SMPP is supported by a UK Medical Research Council Career Development Award (MR/P020372/1). DH is supported by the NIHR through the Applied Research Collaboration (ARC) North-West London and the School of Public Health Research. All research at Great Ormond Street Hospital NHS Foundation Trust and UCL Great Ormond Street Institute of Child Health is made possible by the NIHR Great Ormond Street Hospital Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Division of Surgery & Interventional Science, Faculty of Medical Sciences, University College London, London, WC1E 6BT, UK

Natalia K Rojas, Tom Norris & Snehal M Pinto Pereira

UCL Great Ormond Street Institute of Child Health, 30 Guilford Street, London, WC1N 1EH, UK

Bianca L De Stavola, Mario Cortina-Borja, Manjula D Nugawela, Emma Dalrymple, Terence Stephenson & Roz Shafran

Mohn Centre for Children’s Health & Wellbeing, School of Public Health, Imperial College London, London, UK

Dougal Hargreaves

Immunisation Department, Health Security Agency, 61 Colindale Avenue, London, NW9 5EQ, UK

Kelsey McOwat & Ruth Simmons

You can also search for this author in PubMed   Google Scholar

CLoCk Consortium

Contributions.

Natalia K Rojas [email protected] conducted the statistical analysis for the manuscript, accessed and verified the data and drafted the manuscript. Bianca De Stavola [email protected] assisted in the design of the statistical analyses and reviewed the manuscript. Tom Norris [email protected] assisted the statistical analysis including its design and contributed to the drafting of the manuscript. Mario Cortina-Borja [email protected] reviewed the manuscript. Manjula D Nugawela [email protected] assisted the statistical analysis for the manuscript, accessed and verified the data and reviewed the manuscript. Dougal Hargreaves [email protected] reviewed the manuscript. Emma Dalrymple [email protected] contributed to the design of the CLoCk study and reviewed the manuscript. Kelsey McOwat [email protected] adapted the questionnaire for the online SNAP survey platform. Ruth Simmons [email protected] accessed and verified the data, designed the participant sampling and dataflow for the CLoCk study. Terence Stephenson [email protected] conceived the idea for the CLoCk study, submitted the successful grant application and reviewed the manuscript. Roz Shafran [email protected] contributed to the design of the CLoCk study, submitted the ethics and R&D applications and reviewed the manuscript. Snehal M Pinto Pereira [email protected] conceived the idea for the present study, designed and assisted the statistical analyses for the manuscript, accessed and verified the data and drafted the manuscript. All members of the CLoCk Consortium made contributions to the conception or design of the work; were involved in drafting both the funding application and this manuscript; approved the version to be published; and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Natalia K Rojas .

Ethics declarations

Competing interests.

Terence Stephenson is Chair of the Health Research Authority and therefore recused himself from the Research Ethics Application. Dougal Hargreaves had a part-time secondment as Deputy Chief Scientific Adviser from September 2020 to September 2021, whereby his salary for 2 days per week was paid by the Department for Education (England) to Imperial College London. All remaining authors have no conflicts of interest.

Ethics approval and consent to participate

Ethical approval was provided by the Yorkshire & The Humber - South Yorkshire Research Ethics Committee (REC reference: 21/YH/0060; IRAS project ID:293495). Public Health England (now UKHSA) has legal permission, provided by Regulation 3 of The Health Service (Control of Patient Information) Regulations 2002, to process patient confidential information for national surveillance of communicable diseases. Parents/carers were sent an invitation by post sent through PHE/UKHSA on behalf of the research team with a link to the website with the relevant information sheets and consent forms and they had the opportunity to ask any questions about the study. Parents/carers of CYP under 16 years of age were asked to complete an online parent/carer consent form. The young person was also asked to complete an online assent form to indicate their agreement. Consent was asked online from 16–17-year-olds (using the Young Person Consent Form) in line with Health Research Authority recommended processes. Informed consent was obtained from all participants and/or their legal guardian. All experiments were performed in accordance with relevant guidelines and regulations (such as the Declaration of Helsinki).

Consent for publication

Not applicable. The present manuscript does not contain data from any individual person.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

12874_2024_2219_MOESM1_ESM.docx

Additional file 1. Additional Tables, Text and Figures. This file contains additional Tables 1, 2, 3 and 4, Text 1 and Figs. 1 and 2. Table 1. Further information on variables included in stepwise selection processes for weight generation and their handling. Text 1. Illustrative code demonstrating how uncertainty around generated weights can be accounted for via bootstrapping (with 1000 replications). Table 2. Tiredness prevalence 0 to 12-months post-index PCR-test weighted (trimmed and untrimmed) and unweighted. Table 3. Shortness of breath prevalence 0 to 12-months post-index PCR-test weighted (trimmed and untrimmed) and unweighted. Table 4. Illustrative example of tiredness prevalence 0 to 12-months post-index PCR-test weighted to the target population (untrimmed) with bootstrapped confidence intervals (1000 replications). Figure 1. Weighted (trimmed and untrimmed) and unweighted tiredness prevalences by time of first report. Figure 2. Weighted (trimmed and untrimmed) and unweighted shortness of breath prevalences by time of first report.

12874_2024_2219_MOESM2_ESM.xlsx

Additional file 2: Interactive online tool for re-calibration of survey weights to the general population. This can be used to re-calibrate final target population survey weights to the general population using data on sex, age, and region from the 2021 UK Census. The tool allows ratios of the Census data to CLoCk data reweighted to the target population to be produced and provides examples of what to do with these ratios.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rojas, N.K., De Stavola, B.L., Norris, T. et al. Developing survey weights to ensure representativeness in a national, matched cohort study: results from the children and young people with Long Covid (CLoCk) study. BMC Med Res Methodol 24 , 134 (2024). https://doi.org/10.1186/s12874-024-02219-0

Download citation

Received : 09 May 2023

Accepted : 15 April 2024

Published : 20 June 2024

DOI : https://doi.org/10.1186/s12874-024-02219-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Survey weights
• Children and young people
• Representativeness
• Matched cohort study
• Selection bias
• Loss to follow-up
• Post Covid Condition

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• Open access
• Published: 26 June 2024

Cognitive influencing factors of ICU nurses on enteral nutrition interruption: a mixed methods study

• Huiling Pan 1 , 2 ,
• Chuanlai Zhang 1 , 2 ,
• Ruiqi Yang 1 , 2 ,
• Peng Tian 1 , 2 ,
• Jie Song 1 , 2 &
• Zonghong Zhang 1 , 2  

BMC Nursing volume  23 , Article number:  433 ( 2024 ) Cite this article

Metrics details

The incidence of clinically avoidable enteral nutrition interruptions is high. ICU nurses, as the implementers and monitors of enteral nutrition, have a close relationship between their cognitive level of enteral nutrition interruption and the incidence of enteral nutrition interruption. The level of ICU nurses’ cognition of enteral nutrition interruption and the key factors influencing the level of ICU nurses’ cognition of enteral nutrition interruption are not known.

This study aims to explore the cognitive level of ICU nurses on enteral nutrition interruption and delve into the key factors that affect their cognitive level from the perspective of management.

A sequential explanatory mixed methods research design was used.

With the convenience sampling method, an online survey questionnaire was distributed to ICU nurses in Chongqing, and 336 valid questionnaires were collected. After the survey, ICU managers were invited to participate in qualitative interviews, in which 10 participants from five hospitals completed face-to-face individual semi-structured interviews and were analyzed with thematic analysis.

The survey found that ICU nurses had a good level of cognition towards enteral nutrition interruption but poor knowledge about the definition, causes, and consequences of enteral nutrition interruption, as well as negative attitudes toward active learning, assessment, and communication. And the longer work time in the ICU, joining the nutrition team, receiving systematic training, and acquiring relevant knowledge from academic journals more frequently were favorable to improving ICU nurses’ knowledge level of enteral nutrition interruption. Personal interviews further identified the key factors affecting their cognitive level, including (1) lack of knowledge, (2) lack of proactive thinking, (3) lack of enteral nutrition management programs, and (4) lack of quality management tools for enteral nutrition interruption.

Although ICU nurses demonstrate a relatively high level of cognition, there is still room for improvement. ICU administrators must take specific measures to improve the knowledge of ICU nurses, especially in non-tertiary hospitals, in order to prevent nurse-induced enteral nutrition interruption in all ICUs and improve medical quality.

Trial registration

Not applicable.

Peer Review reports

Introduction

Critically ill patients often experience physiological, pathological, and metabolic disorders that limit nutritional intake, and the prevalence of malnutrition is as high as 38–78% [ 1 ]. Malnutrition refers to a state of energy or nutrient deficiency caused by inadequate intake or utilization barriers [ 2 ], and it is a major factor contributing to adverse clinical outcomes for patients. Studies have found [ 3 , 4 , 5 ] that malnutrition in ICU patients increases the incidence of complications such as ICU length of stay, days of mechanical ventilation, infections and organ failure, and mortality. Therefore, nutritional therapy is particularly important in the management of critically ill patients.

Enteral nutrition (EN) has become the preferred nutritional support treatment for ICU patients due to its alignment with normal physiological metabolic processes [ 6 ]. Guidelines recommend [ 6 , 7 ] that ICU patients should receive 80–100% of their target feeding volume within 3–7 days of initiating EN. 60–75% of patients in the ICU, however, as shown in several studies [ 8 , 9 , 10 ], do not reach the target feeding volume. Research [ 10 ] has found that the feeding deficiency rates were 54% and 15% ( p  < 0.001) on trial days with and without enteral nutrition interruption (ENI), respectively, indicating a positive correlation between ENI and insufficient feeding.

Enteral nutrition interruption (ENI) [ 11 ] is defined as an interruption of EN lasting 1 h or more with continuous enteral feeding or if the patient does not receive the expected amount of nutrients within 30 min with intermittent enteral feeding. Studies have found [ 10 ] that the average ENI time for ICU patients is up to 12 (6–24) hours per day. The causes of ENI are underestimated target feeding volumes, feeding intolerance, medical procedures, etc., which can be divided into patient factors and subjective factors [ 12 , 13 ]. Among these, avoidable subjective factors related to medical operations account for approximately 72% of the total time of ENI [ 14 , 15 ]. This is related to multiple factors such as physicians, nurses, frontline administrators, and healthcare institution management. ICU nurses, as the primary role in EN screening, assessment, implementation, monitoring, and complication intervention, are closely related to the occurrence of ENI in patients [ 16 ]. Studies have shown [ 17 ] that nurses not starting EN in a timely manner after medical procedures or outpatient examinations are the primary cause of ENI.

The Theory of Reasoned Action [ 18 ] proposes that individuals make behavioral decisions through rational thinking, and this decision-making process is influenced by various factors such as knowledge, attitude, and social environment. Thus, nurse-induced enteral nutrition interruption may be related to their level of knowledge, beliefs, and consequent practice behaviors related to ENI. To explore the current situation of ENI caused by ICU medical staff, previous studies [ 19 ] have examined the cognition of ENI among ICU medical staff in Wuhan. Little study, however, has been found to explore the key factors that affect their cognitive status. Currently, ICU managers lack a unified and standardized EN management plan. Furthermore, ICU nurses and doctors have different levels of knowledge, and nurses interact with patients more frequently, so a questionnaire is needed to evaluate ICU nurses’ cognition of ENI.

ICU manager [ 20 ] refers to the doctor or nurse who is responsible for the daily operation, management, supervision, and improvement of the ICU. ICU managers, as one of the key personnel in the whole link management and quality control of enteral nutrition, usually view problems from an overall perspective, and their perspectives and observations are more objective, in-depth, and comprehensive, which helps us understand the difficulties and challenges of ICU nurses in practice. We, therefore, use a sequential explanatory mixed methods research design [ 21 ] to investigate the cognitive level and influencing factors of ENI among ICU nurses through a cross-sectional survey. Based on the results, we will develop an interview outline to delve into the key factors influencing ICU nurses’ cognition of ENI from the perspective of ICU managers. This will lay the foundation for developing targeted interventions aimed at improving ICU nurses’ cognition of ENI, and provide the basis for improving the EN management program, so as to avoid nurse-induced ENI and improve medical quality.

Methodology

Research design.

A sequential explanatory mixed methods research design [ 21 ] was used that included both quantitative and qualitative research. The interview guide for the qualitative research was developed based on the findings of the quantitative research and served to complement and explain the quantitative results.

Quantitative research

Participants.

Convenience sampling was used to conduct a cognitive survey on ENI among ICU nurses in Chongqing. The recruited object of this study was ICU registered nurses who had worked in general ICUs for at least one year. The first page of the questionnaire describes the purpose of this study and informed consent. Respondents can only access the survey questions after giving informed consent. After completing and submitting the survey, participants were considered to have given informed consent. In addition, researchers can judge according to the basic information filled in by participants to exclude those who do not meet the inclusion criteria. The sample size of this study was at least 193 according to previous similar studies [ 22 ].

Data collection

The scale used in this study is the “ICU Healthcare Providers’ ENI Knowledge, Attitude, and Practice Scale,” developed by the Yuanyuan Mi team in 2022 [ 22 ], which is used to understand the current level of knowledge, attitude, and practice of ENI among ICU medical staff. This scale comprised three dimensions: knowledge, belief, and practice, with 14, 10, and 17 items, respectively, and total score ranges of 14–70, 10–50, and 17–85. Items were rated using a Likert 5-point scale, with 1 indicating “not at all,” 2 “uncertain,” 3 “slightly,” 4 “fairly,” and 5 “completely.” Scores below 4 indicated poor cognitive levels of ENI among ICU nurses; scores equal to or greater than 4 indicated that ICU nurses have a good level of ENI awareness. Reportedly, the Cronbach’s alpha for the original scale was 0.953, the test-retest reliability was 0.795, and the total content validity coefficient was 0.975, indicating that the scale had good reliability and validity. In addition, the Cronbach’s alpha was 0.965 when the scale was retested using data from this study.

In this study, 10 demographic variables and the “ICU Healthcare Providers’ ENI Knowledge, Attitude, and Practice Scale” developed by the Yuanyuan Mi team [ 22 ] were converted into an online questionnaire. A cross-sectional survey was conducted among ICU nurses in Chongqing in October 2023. 366 questionnaires were distributed through the questionnaire star platform, and 366 were recovered, with a recovery rate of 100%. Two researchers checked the content of the questionnaire and the duration of the questionnaire, deleted 30 invalid questionnaires, and finally found 336 valid questionnaires, for an effective rate of 91.8%.

Data analysis

Data were downloaded from the Questionnaire Star platform and analyzed in SPSS 27.0. Statistical significance was set at p  < 0.05. Means (standard deviations) and frequencies (percentages) were used for descriptive statistics. Differences and associations between ICU nurses’ EN cognition scores and demographic variables were analyzed using t-tests, chi-square tests, and binary logistic regression. Pearson’s correlation was used to assess the relationship between the total cognition score and the scores of each dimension.

Qualitative research

Purposeful sampling was used to select ICU EN managers willing to participate in qualitative interviews from hospitals where the questionnaire was administered. Eligible participants included healthcare providers from general ICUs involved in EN management for at least three years and willing to participate in this semi-structured interview. A total of 10 ICU managers were included in this study for personal interviews. Information saturation [ 23 ] was reached at interview 8, meaning that no new themes emerged at the end of the interview process. Two further interviews were conducted to confirm the results.

Data were collected through semi-structured interviews conducted by the first and second authors with participants in December 2023. The interview guide (see to S1 ) was developed by the lead author, guided by the Theory of Reasoned Action [ 18 ], and based on questionnaire results, a review of domestic and international literature, and expert consultation. Participants were contacted by phone before the interview to explain the purpose and significance of the study, obtain informed consent regarding confidentiality principles, recording, and other issues. Interviews were conducted at mutually agreed-upon times, ensuring privacy and a quiet environment. The interview time should be controlled at about 30 min. During the interviews, non-verbal cues such as body language, facial expressions, and tone of voice were observed and recorded along with audio recordings. A pilot interview was conducted with two ICU managers meeting the inclusion criteria before the qualitative study’s implementation, but their data were not included in the final analysis.

Audio recordings and written notes were transcribed verbatim within 24 h of the interview’s conclusion and stored on a computer for backup. Data analysis was based on the Theory of Reasoned Action [ 18 ] and aimed to identify key factors influencing the improvement of ICU nurses’ cognitive levels regarding ENI. A deductive thematic analysis approach [ 24 ] was employed, involving the following steps: (a) familiarization with the data; (b) initial code generation; (c) theme search based on initial codes; (d) theme review; (e) theme definition and labeling; and (f) report writing.

Quality control

To ensure reliability, the research team met regularly, and team members reviewed the study data and analysis results. For the quantitative study, the online survey was anonymous. To ensure the authenticity and validity of the questionnaire results, each respondent was given only one chance to answer the questionnaire and was required to answer all the questions before submitting the questionnaire. To prevent the inclusion of low-quality questionnaires, it was assumed that each question would take no less than 2 s to answer, and in combination with the number of demographic characteristics entries (10) and scale entries (41), questionnaires with an answer time of less than 2 min were excluded from this study. The researcher observed and collected the filled-in data through the background of the questionnaire and double-checked the extracted information to ensure the completeness of the information. In the qualitative study, interview transcripts were collected by two research members trained in qualitative research, and one researcher organized the audio-recorded interviews into text within 24 h of the end of the interviews, which was then returned to the interviewees for confirmation by two researchers who repeatedly read and proofread the information. Participant recruitment, interviews, and data analysis were conducted simultaneously to help researchers determine information saturation. No repeat interviews were conducted.

Ethical considerations

Ethical approval was obtained from the ethics committee of the Second Affiliated Hospital of Chongqing Medical University (Ke Lunshen No. (139) in 2023). The front page of the questionnaire sent to potential participants during the quantitative phase had an “informed consent” option, which was clicked on to allow participants to access the electronic questionnaire. Participants who submitted the questionnaire were considered to have obtained their informed consent. Participants in the quantitative phase volunteered their participation, and the questionnaire’s demographic data did not include names. Each participant was assigned a numerical code to ensure the confidentiality of survey responses. In the qualitative phase, participants provided written informed consent, and their interview recordings were analyzed anonymously and reported solely for research purposes by the study team.

Quantitative phase

Demographic characteristics of icu nurses.

Among the 366 participants who completed the questionnaire, 336 (91.8%) were considered to have provided valid questionnaires. The mean age of the 336 study subjects was 31.24 ± 5.68 years, ranging from 22 to 59 years old. Among them, 192 (57.1%) nurse had junior professional title, a total of 285 (84.8%) held a bachelor’s degree or higher, and the average ICU working time was 6.88 ± 5.05 years. Most of the nurses worked in tertiary care hospitals [ N  = 212 (63.1%)], but a few were members of the nutrition team [ N  = 83 (24.7%)]. This survey showed that only 54 (16.1%) nurses had received systematic training on knowledge related to enteral nutrition, and only 25 (7.4%) nurses reported that they regularly obtained knowledge related to enteral nutrition from academic journals. (See Table  1 )

Cognitive level of ICU nurses regarding enteral nutrition interruption

As shown in Table  2 , the mean score of ICU nurses’ knowledge of enteral nutrition interruption was 165.04 (22.86), which was higher than 164 (41 × 4), i.e., the cognitive level of ICU nurses regarding ENI was better. On the knowledge dimension, the mean score of ICU nurses’ knowledge of the definition, causes, and consequences of ENI was lower than 4, which was poor in this area; while " Unless contraindicated, the head of the bed should be elevated by 30–45° during EN administration to critically ill patients " and “When the medical and nursing-related examination, diagnosis, and treatment procedures are completed, enteral nutrition feeding should be resumed in a timely manner” had the highest scores, which were both higher than 4, indicating better knowledge in this area. The mean scores of ICU nurses in the belief dimension of ENI were all higher than 4, indicating better beliefs. On the behavioral dimension, ICU nurses scored higher than 4 on all behaviors except for lower scores on active learning about ENI, active patient assessment, and communication with physicians.

Pearson’s correlation analysis among knowledge, belief, and behavior dimensions

As shown in Table  3 , there was a strong positive correlation between the total cognitive score and the scores for the knowledge, belief, and behavior dimensions ( r  = 0.830, 0.766, and 0.850, respectively, P  < 0.01). There was also a positive correlation between the knowledge dimension score and the scores for the belief and behavior dimensions ( r  = 0.487 and 0.549, respectively, P  < 0.01). Furthermore, there was a positive correlation between the belief dimension score and the behavior dimension score ( r  = 0.535, P  < 0.01).

Univariate analysis of knowledge, belief and behavior against demographic characteristics

ICU nurses were deemed to have a low cognitive capacity about ENI if they received a single-item score of less than 4. Therefore, a cutoff value of ≥ 4 was used to categorize the participants’ total cognitive scores, knowledge dimension scores, belief dimension scores, and behavior dimension scores into two categories: low (= 0) and high (= 1). These were used as dependent variables. Univariate analysis of ICU nurses’ demographics and cognitive scores showed that age, nutrition team membership, and frequency of acquiring relevant knowledge from academic journals were associated with ICU nurses’ level of cognition about ENI; professional title, nutrition team membership, systematic training, and frequency of acquiring relevant knowledge from academic journals were associated with ICU nurses’ knowledge scores about ENI; and frequency of acquiring relevant knowledge was associated with ICU nurses’ ENI belief dimension and behavioral dimension scores. A P-value of < 0.05 was considered statistically significant. (See Table  4 )

Factors associated with improving ICU nurses’ cognitive level

Variables with a P-value of < 0.10 from the univariate analysis were included as independent variables in a logistic regression model. The results showed that a high frequency of reading academic journals was a facilitating factor for improving ICU nurses’ cognitive level regarding ENI. Additionally, longer work time in the ICU, participation in nutritional groups, receipt of systematic training, and a high frequency of acquiring related knowledge about EN from academic journals were promoting factors for enhancing ICU nurses’ knowledge dimension scores regarding ENI (see Table  5 ).

Qualitative phase

Ten ICU managers with bachelor’s degrees or above, ages ranging from 40 to 53, took part in individual semi-structured interviews from five hospitals. The duration of the interviews was roughly 12–36 min (see to S2 ). Four key factors were identified from qualitative data analysis that influence ICU nurses’ cognitive level regarding ENI: (1) Lack of knowledge; (2) Lack of active thinking; (3) Lack of EN management plans; and (4) Lack of quality management tools for ENI.

Lack of knowledge

According to participants, ENI is common in the ICU and is related to ICU nurses’ lack of knowledge about it. Many nurses are unclear about the definition, causes, and consequences of ENI. As Participant 5 described, ‘Many nurses are not yet aware of the concept of ENI and do not know how long a sustained pumping pause is an interruption of enteral nutrition, so much so that they are not particularly concerned about the time of restarting EN after a pause in EN, which leads to an increase in the duration and frequency of ENI in patients’. Furthermore, many participants stated that many nurses believe that pausing EN for a few hours during continuous enteral feeding does not constitute an interruption because the gastrointestinal tract remains active, which can damage a patient’s gastrointestinal function. Therefore, pausing for a few hours is similar to intermittent enteral feeding, allowing the patient’s intestine to rest. ICU nurses have a vague understanding of the definition and causes of ENI. What’s more, Participant 9 added, ‘Many nurses directly suspend EN when the gastric residual volume (GRV) exceeds 200 mL! Sometimes, when the GRV is assessed to be below 200 mL, the returned nutrient solution is discarded without realizing the relationship between ENI and adverse outcomes related to inadequate feeding’.

Lack of active thinking

Participants believed that the limitations in ICU nurses’ cognitive level regarding ENI were related to their mechanical work and lack of active thinking. Various reasons for ICU nurses’ lack of active thinking were described. Notably, due to limited human resources, ICU nurses, apart from handling doctor’s orders and basic care, also need to deal with emergencies and adverse reactions among critically ill patients, such as resuscitation, vomiting, and diarrhea. At the same time, they need to dynamically assess patients and fill out numerous assessment forms, making their workload heavy. As Participant 5 explained, ‘For example, when ICU nurses administer a doctor’s order of 1000 mL of nutrient solution to a patient, they routinely adjust the feeding speed, mechanically fill out various forms, and habitually assess the patient’s enteral feeding intolerance. If the patient tolerates it, they simply finish the feeding and move on, rarely thinking about whether the patient’s EN feeding has reached their nutritional goals……If the patient is intolerant, they habitually discard the syringe return fluid when the GRV is greater than 200 mL or even 50 mL and directly suspend the patient’s EN!’ Participants felt that ICU nurses, as implementers and monitors of EN, had a diminished sense of active learning as their sense of active thinking weakened. Participant 6 stated, ‘ICU nurses lack knowledge of biochemical indicators related to EN (such as phosphorus), hemodynamics, patients’ total enteral nutrition target, calories, and protein, and believe that nurses do not need to master these, lacking active learning consciousness’. Although many hospitals have EN management teams, most participants stated that team members are not very motivated, often forced to accept tasks, and lack active learning consciousness, which may be related to their lack of demand, competition, and conflict of interest.

Lack of EN management plans

It was evident from the interviews that the management level varies among different medical units, and there is inconsistency in the quality of care provided by doctors and nurses. The absence of standardized EN management plans that can be referred to has limited the improvement of ICU nurses’ cognitive level regarding ENI. For example, there is a lack of solutions to address inconsistencies between theory and practice. Participant 4 described, ‘Nurses are confused about the different gastric residual volume thresholds recommended by multiple guidelines, resulting in behaviors such as suspending EN when the volume exceeds 200mL. There is a lack of regulations regarding GRV thresholds and guidance on how to adjust or reduce the feeding rate in our department’. Participant 1 stated, ‘Nurses are unclear about whether it is necessary to routinely aspirate gastric residuals every 4–6 hours’. Participant 6 added, ‘The department lacks an active feeding strategy for restarting enteral nutrition to promote early active venting of patients’. Furthermore, participants felt that the management of EN in ICU patients requires multidisciplinary collaborative management, but the triad of physicians, nurses, and nutritionists each had their own role and lacked a closely linked management process. Participant 7 described, ‘ICU doctors have better knowledge of nutrition, less consultation with the Nutrition Department is requested, and nutritionists are unable to dynamically assess the EN status of patients in a timely manner, to the extent that it is mostly left to the ICU doctors themselves to determine the problem of patients’ EN compliance’. And participant 3 said, ‘Currently, ICU nurses put a lot of effort into screening, assessment, implementation, monitoring, and complication intervention of EN, and their awareness is gradually increasing (smiled), while physicians are less involved in the management of the EN process!’ What’s more, participants described that the initial nutritional screening assessor varies from ICU to ICU, that some are nurses whereas others are physicians, that it is not yet known who leads the management of EN in ICU patients, and that there is a lack of a collaborative management process between the medical and nursing professions.

Lack of quality management tools for enteral nutrition interruptions

Participants noted that current clinical EN management primarily consists of EN guidelines, implementation procedures, nutritional screening tools, enteral nutrition tolerance assessment forms, and aspiration risk assessment forms. However, there is still a lack of quality management tools specifically designed for ENI. This makes it difficult for ICU nurses to identify avoidable causes of ENIs, which in turn hinders their ability to reduce the occurrence of such interruptions. Participants described some avoidable issues related to ENIs. Participant 6 described, ‘ICU nurses often pause EN when the amount of GRV exceeds 200 mL, lacking a standardized deceleration or reduction in volume’. Participant 2 described, ‘Clinical situations often arise where infusions are not completed within 24 hours……This is attributed to unreasonable infusion speed settings, excessive preoperative fasting durations, forgetting to report to doctors after suspensions, forgetting to restart infusions, and equipment malfunctions.” Although the EN management team has identified issues related to ENIs during the management process, they lack plans for implementation and problem-solving. They expressed a desire to use quality management tools to manage ENIs and reduce those caused by human factors.

Understanding the cognitive level and influencing factors of ICU nurses regarding ENIs is crucial, as their cognition has a direct relationship with achieving the nutritional targets for ICU patients’ EN [ 16 ]. This study helps ICU managers understand the key factors affecting the cognitive level of ICU nurses’ ENI in order to lay the foundation for ICU managers to develop targeted interventions aimed at improving the cognitive level of ICU nurses’ ENI. Analysis of the questionnaire revealed that ICU nurses generally have a good level of cognition regarding ENIs, with a poorer understanding of their definitions, causes, and consequences. Additionally, they exhibited a negative attitude towards actively seeking knowledge, assessing, and communicating. However, there is still room for improvement, such as by joining nutrition groups, receiving systematic training on EN, participating in related academic conferences, and regularly acquiring EN knowledge from academic journals. Based on this, ICU managers further explained the key factors influencing nurses’ cognitive levels: a lack of knowledge regarding ENIs, inactive thinking about achieving EN feeding targets, a lack of management processes for addressing inconsistencies between theory and practice, and a lack of quality management tools for ENIs. These findings provide a basis for ICU managers to improve EN management plans. Therefore, it is recommended that ICU managers accordingly develop targeted interventions aimed at improving ICU nurses’ cognition of enteral nutrition interruptions in order to avoid nurse-induced ENI and improve medical quality.

This study is consistent with the findings of Mi Yuanyuan [ 19 ] et al. that ICU nurses have a better level of ENI cognition. However, this study also found that the number of years working in the ICU and nutrition team members were the influencing factors for the ICU nurses’ ENI knowledge dimension scores. This may be related to the fact that only ICU healthcare workers in tertiary hospitals were included in the study by Mi Yuanyuan [ 19 ] et al. or to the fact that nutrition team members accounted for as much as one-third of the ICU nurses in the study by Mi Yuanyuan [19] et al. This is also a side effect of the unequal levels of ENI awareness among ICU nurses in different levels of hospitals. In the future, more ICU nurses in secondary hospitals can be included to explore the current status of ENI cognitive level of ICU nurses in different grades of hospitals. Furthermore, unlike previous studies [19] , this study conducted qualitative interviews with ICU managers on the basis of a questionnaire survey of ICU nurses, which explored the key factors affecting the cognitive level of ICU nurses’ ENI in more depth and laid the foundation for ICU managers to formulate targeted interventions aiming to enhance the cognitive level of ICU nurses’ enteral nutrition interruption.

In this study, we found that high years of working experience in ICU, joining the nutrition team, receiving systematic training, and a high frequency of acquiring knowledge related to enteral nutrition from academic journals were the contributing factors to increasing the level of ICU nurses’ knowledge of enteral nutrition interruption. The longer the working years, the richer the clinical experience and related knowledge of ICU nurses. However, as shown in this study, nearly half [ N  = 154 (45.8%)] of the ICU nurses had less than 5 years of working experience; therefore, there is an urgent need to improve the level of ICU nurses’ cognition of ENI in other ways in order to balance the level of cognition of ICU nurses with different years of working experience. For example, by joining a nutrition team and receiving relevant systematic training, ICU nurses can be helped to gain a systematic, comprehensive, and in-depth understanding of knowledge related to enteral nutrition and to increase nurses’ awareness of and interest in the interruption of enteral nutrition [ 25 ]. This is to promote proactive thinking by ICU nurses and to improve their scores in proactive learning about interruption of enteral nutrition, proactive assessment of patients, and communication with physicians [ 26 ]. Further, ICU nurses can also compensate for knowledge blindness by frequently acquiring knowledge related to enteral nutrition from academic journals. Academic journals, as authoritative repositories of academic knowledge, have the most cutting-edge knowledge in the field, such as clinical guidelines and original research with practical guidance, and ICU nurses’ frequent acquisition of enteral nutrition-related knowledge from academic journals is conducive to a systematic and in-depth understanding of the guidelines, consensus, original research, and the frontiers of enteral nutrition in order to enhance nurses’ knowledge of enteral nutrition interruption. Therefore, ICU administrators can encourage nurses to join nutrition teams and conduct multi-pathway training to promote nurses’ acquisition of knowledge from academic journals in order to improve ICU nurses’ level of knowledge about enteral nutrition interruptions, as well as to promote nurses’ proactive thinking in order to avoid unnecessary enteral nutrition interruptions.

Nurses are susceptible to the influence of external factors, and procedures and systems are fundamental to regulating nurses’ behavior. The development of enteral nutrition management protocols is beneficial to standardizing ICU nurses’ management of patients with enteral nutritional feedings in order to improve the level of ICU nurses’ perception of enteral nutritional interruption. A national survey [ 27 ] found that enteral nutrition is usually prioritized lower than other urgent care needs for ICU patients. Furthermore, there is a lack of uniform and standardized clinical protocols for enteral nutrition management in critically ill patients [ 28 , 29 ]. This has hindered the improvement of the level of ENI awareness among ICU nurses in different levels of hospitals to a certain extent and is not conducive to the homogenization of ICU healthcare personnel in various healthcare institutions. Enteral nutrition is critical to the recovery of ICU patients [ 4 ]. It is necessary to enhance ICU nurses’ knowledge of enteral nutrition management to facilitate the development of standardized enteral nutrition protocols [ 30 , 31 ]. Currently, the threshold for GRV is not uniform in clinical settings, with 200–500 mL being the most common [ 32 , 33 ]. This is not conducive to ICU nurses’ judgment of GRV thresholds, which may lead to some degree to nurse-induced ENI. Furthermore, guidelines have recommended that routine monitoring of GRV [ 7 ] during the EN may not be necessary, but most clinical nurses still habitually aspirate gastric residual to monitor patients’ gastrointestinal intolerance, which may be related to the ICU nurses’ fear of the risk of patients’ vomiting or aspiration [ 34 ] or to their insufficiently in-depth view of the problem. At the same time, there is currently a clinical controversy over whether the gastric residual aspirates should be returned or discarded [ 35 ]. This may explain, in part, why some ICU nurses currently choose to discard the gastric residual aspirates directly to avoid contamination, and some ICU nurses choose to tie back the gastric residual aspirates to minimize the risk of fluid and electrolyte imbalance in the patient. Therefore, there is an urgent need for the development of standard enteral nutrition management protocols to address the currently controversial issues and to standardize ICU nurses’ behavior regarding enteral nutrition management.

The formulation of the scheme is conducive to standardizing the behavior of nurses, but the optimization of the implementation effect of the scheme requires the application of quality management tools. Currently, there is a lack of quality management tools in clinical practice to monitor the rate of implementation of EN measures [ 5 , 6 ]. Previous studies have shown [ 12 , 13 ] that the reasons for ENI in ICU patients include hemodynamic instability, high GRV, and medical procedures. It is difficult to avoid ENI, but as shown by Kagan et al. [ 36 ], the use of nutritional management feeding platforms (such as the smART + platform) can monitor ICU patients’ ENI in real-time, calculate the amount of compensation needed when restarting, and ultimately help patients reach their EN goal. In other words, most ENIs caused by ICU nurses can be avoided through the use of management tools28. As a fine and process management method, the Plan-Do-Check-Act (PDCA) cycle method is a continuous quality management tool that targets clinical weaknesses, proposes countermeasures, and improves the implementation rate of measures. It has been widely used in ICU quality management [ 37 ]. Therefore, in the future, ICU managers can use quality management tools to dig deeper into the reasons for enteral nutrition interruption, promote the development and implementation of related plans, and solve the problem at the source in order to reduce avoidable enteral nutrition interruption, standardize nurses’ behaviors, and maximize the application of enteral nutrition management programs.

Strengths and limitations

This study boasts both strengths and limitations. Leveraging the advantages of mixed methods research, we delved into the key factors influencing ICU nurses’ cognition of ENI from both the nurses’ and management’s perspectives. This lays the foundation for targeted interventions aimed at enhancing ICU nurses’ understanding of ENI, ultimately aiming to prevent such interruptions caused by the nurses themselves. Rather, we must acknowledge its limitations. Our use of sequential explanatory mixed methods means our ability to explore the critical factors influencing ICU nurses’ cognition of ENI is somewhat limited, but this could be addressed through alternative mixed methods designs. Furthermore, our study sample was limited to a geographical region, potentially limiting the generalizability of our findings. Future research could expand the scope of the investigation. Nevertheless, this study provides novel insights and valuable perspectives for ICU managers to improve their department’s EN management strategies.

Overall, the level of ICU nurses’ cognition of enteral nutrition interruption is good, but there is still room for improvement. ICU nurses can improve the level of knowledge related to ENI and increase their proactive thinking about the management of enteral nutrition target feeding compliance by joining the nutrition team, participating in the systematic training of knowledge related to enteral nutrition, and frequently acquiring knowledge from academic journals. Furthermore, ICU managers should apply a quality management tool for enteral nutrition interruptions and develop targeted interventions aimed at improving ICU nurses’ cognition of enteral nutrition interruptions in order to provide a basis for improving the department’s enteral nutrition management program, so as to avoid nurse-induced ENI and improve medical quality.

Data availability

All data generated or analyzed during the study are available from the corresponding author [Chuanlai Zhang] on request.

Abbreviations

• Intensive care units
• Enteral nutrition
• Enteral nutrition interruption

Gastric residual volume

Díaz Chavarro BC, Molina-Recio G, Assis Reveiz JK, Romero-Saldaña M. Factors Associated with Nutritional Risk Assessment in critically ill patients using the Malnutrition Universal Screening Tool (MUST). J Clin Med. 2024;13(5):1236.

Article   PubMed   PubMed Central   Google Scholar  

Cortés-Aguilar R, Malih N, Abbate M, Fresneda S, Yañez A, Bennasar-Veny M. Validity of nutrition screening tools for risk of malnutrition among hospitalized adult patients: a systematic review and meta-analysis. Clin Nutr. 2024;43(5):1094–116.

Article   PubMed   Google Scholar  

Nigatu YD, Gebreyesus SH, Allard JP, Endris BS. The effect of malnutrition at admission on length of hospital stay among adult patients in developing country: a prospective cohort study. Clin Nutr ESPEN. 2021;41:217–24.

Karpasiti T, Bear DE. The importance of nutrition to morbidity and mortality in critically ill patients. Intensive Crit Care Nurs. 2023;76:103365.

Pohlenz-Saw J, Merriweather JL, Wandrag L. (Mal)nutrition in critical illness and beyond: a narrative review. Anaesthesia. 2023;78(6):770–8.

Article   CAS   PubMed   Google Scholar  

Singer P, Blaser AR, Berger MM, et al. ESPEN practical and partially revised guideline: clinical nutrition in the intensive care unit. Clin Nutr. 2023;42(9):1671–89.

Yang H, Zhu M, et al. Guideline for clinical application of parenteral and enteral nutrition in adults patients in China (2023 edition). Zhonghua Yi Xue Za Zhi. 2023;103(13):946–74.

Google Scholar  

Tatucu-Babet OA, Ridley EJ. How much underfeeding can the critically ill adult patient tolerate? J Intensive Med. 2022;2(2):69–77.

Zaher S, Sumairi FA, Ajabnoor SM. Understanding nursing perspective towards barriers to the optimal delivery of enteral nutrition in intensive care settings. BMC Nurs. 2024;23(1):42. Published 2024 Jan 15.

Salciute-Simene E, Stasiunaitis R, Ambrasas E, et al. Impact of enteral nutrition interruptions on underfeeding in intensive care unit. Clin Nutr. 2021;40(3):1310–7.

Singer P, Blaser AR, Berger MM, et al. ESPEN guideline on clinical nutrition in the intensive care unit. Clin Nutr. 2019;38(1):48–79.

Kasti AN, Theodorakopoulou M, Katsas K et al. Factors Associated with interruptions of Enteral Nutrition and the impact on macro- and Micronutrient deficits in ICU patients. Nutrients. 2023;15(4).

Onuk S, Ozer NT, Savas N, et al. Enteral nutrition interruptions in critically ill patients: a prospective study on reasons, frequency and duration of interruptions of nutritional support during ICU stay. Clin Nutr ESPEN. 2022;52:178–83.

Kasti AN, Theodorakopoulou M, Katsas K, et al. Factors Associated with interruptions of Enteral Nutrition and the impact on macro- and Micronutrient deficits in ICU patients. Nutrients. 2023;15(4):917.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Solana MJ, Slocker M, Martínez DCZ et al. Prevalence, risk factors and Impact of Nutrition Interruptions in critically Ill Children. Nutrients. 2023;15(4).

Jordan EA, Moore SC. Enteral nutrition in critically ill adults: literature review of protocols. Nurs Crit Care. 2020;25(1):24–30.

Xuemei Gong X, Ye J, Xing W, Li. The causes of early enteral nutrition feeding interruption in critically ill patients: a systematic review. Parenter Enter Nutr. 2018;25(5):285–90.

Yzer M. Theories of Reasoned Action and Planned Behavior. 2022:1–7.

Yuanyuan Mi F, Tian L, Bao, et al. Development, reliability, and validity of a scale for knowledge, attitude, and practice of intensive care unit staff towards enteral nutrition feeding interruption. J Nurs. 2022;37(19):82–6.

Cuyvers K, Van Oostveen C, Endedijk MD, Struben V. Nurses’ self-regulated learning in clinical wards: important insights for nurse educators from a multi-method research study. Nurse Educ Today. 2024;137:106179.

Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the mixed methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018;24(3):459–67.

Yuanyuan Mi. Development and application of knowledge, belief, and practice scale for ICU medical staff on interruption of Enteral Nutrition. Zhengzhou University; 2022.

Braun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Res Sport Exerc Health. 2021;13(2):201–16.

Article   Google Scholar  

Kiger ME, Varpio L. Thematic analysis of qualitative data: AMEE Guide 131. Med Teach. 2020;42(8):846–54.

Mancin S, Sguanci M, Cattani D, et al. Nutritional knowledge of nursing students: a systematic literature review. Nurse Educ Today. 2023;126:105826.

Moghadam KN, Chehrzad MM, Masouleh SR, et al. Nursing workload in intensive care units and the influence of patient and nurse characteristics. Nurs Crit Care. 2021;26(6):425–31.

Bloomer MJ, Clarke AB, Morphet J. Nurses’ prioritization of enteral nutrition in intensive care units: a national survey. Nurs Crit Care. 2018;23(3):152–8.

Ke L, Lin J, Doig GS, et al. Actively implementing an evidence-based feeding guideline for critically ill patients (NEED): a multicenter, cluster-randomized, controlled trial. Crit Care. 2022;26(1):46.

Bendavid I, Singer P, Theilla M, et al. Nutrition Day ICU: a 7 year worldwide prevalence study of nutrition practice in intensive care. Clin Nutr. 2017;36(4):1122–9.

Doménech BV, Gea-Caballero V, Chover-Sierra E, et al. Knowledge level of ICU nurses regarding Nutritional Assessment of critically ill patients: a systematic review. Nurs Rep. 2024;14(1):586–602.

Orinovsky I, Raizman E. Improvement of Nutritional Intake in Intensive Care Unit patients via a nurse-led Enteral Nutrition Feeding Protocol. Crit Care Nurse. 2018;38(3):38–44.

Li J, Wang L, Zhang H, et al. Different definitions of feeding intolerance and their associations with outcomes of critically ill adults receiving enteral nutrition: a systematic review and meta-analysis. J Intensive Care. 2023;11(1):29.

Yasuda H, Kondo N, Yamamoto R, Asami S, Abe T, Tsujimoto H, Tsujimoto Y, Kataoka Y. Monitoring of gastric residual volume during enteral nutrition. Cochrane Database Syst Reviews 2021, Issue 9. Art. No.: CD013335.

Tume LN, Lynes AA, Waugh V et al. Nurses’ decision-making around gastric residual volume measurement in UK adult intensive care: A four-centre survey. Nurs Crit Care Published Online March 7, 2024.

Wen Z, Xie A, Peng M, Bian L, Wei L, Li M. Is discard better than return gastric residual aspirates: a systematic review and meta-analysis. BMC Gastroenterol. 2019;19(1):113.

Kagan I, Hellerman-Itzhaki M, Bendavid I, et al. Controlled enteral nutrition in critical care patients - a randomized clinical trial of a novel management system. Clin Nutr. 2023;42(9):1602–9.

Zhong X, Wu X, Xie X, et al. A descriptive study on clinical department managers’ cognition of the Plan-Do-Check-act cycle and factors influencing their cognition. BMC Med Educ. 2023;23(1):294.

Download references

Acknowledgements

We would like to thank the nurses who participated in this study.

This work was supported by the Medical Quality (Evidence-Based) Management Research Program (Award No.: YLZLXZ23G107) in 2023 of National Institute of Hospital Administration, National Health and Health Commission of the People’s Republic of China, Kuanren Talents Program of The Second Affiliated Hospital of Chongqing Medical University and Chongqing Municipal Education Commission’s 14th Five-Year Key Discipline Support Project.

Author information

Authors and affiliations.

Intensive Care Unit, The Second Affiliated Hospital of Chongqing Medical University, Yuzhong, Chongqing, People’s Republic of China

Huiling Pan, Chuanlai Zhang, Ruiqi Yang, Peng Tian, Jie Song & Zonghong Zhang

School of Nursing, Chongqing Medical University, Yuzhong, Chongqing, People’s Republic of China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, ZCL; Methodology, PHL, ZCL, YRQ, TP, SJ and ZZH; Data curation, PHL, YRQ, TP, SJ and ZZH; Investigation, PHL, ZCL, YRQ, TP, SJ and ZZH; Formal analysis, PHL and YRQ; Writing- Original draft preparation, PHL; Funding acquisition, ZCL; Supervision, ZCL; Resources, TP, SJ and ZZH; Validation, TP, SJ and ZZH; Writing –review & editing, ZCL and YRQ.

Corresponding author

Correspondence to Chuanlai Zhang .

Ethics declarations

Ethical approval.

The study was conducted in accordance with the Declaration of Helsinki. And ethical approval was obtained from the ethics committee of the Second Affiliated Hospital of Chongqing Medical University (Ke Lunshen No. (139) in 2023).

Consent to participate

Informed consent was obtained from all subjects involved in this study.

Consent for publication

Conflict of interest.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Pan, H., Zhang, C., Yang, R. et al. Cognitive influencing factors of ICU nurses on enteral nutrition interruption: a mixed methods study. BMC Nurs 23 , 433 (2024). https://doi.org/10.1186/s12912-024-02098-2

Download citation

Received : 04 April 2024

Accepted : 17 June 2024

Published : 26 June 2024

DOI : https://doi.org/10.1186/s12912-024-02098-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mixed method

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Open access
• Published: 24 June 2024

Evaluation of a knowledge-attitude-practice model based narrative life education program for community-dwelling older adults: a mixed-methods feasibility study

• Xifeng Xie 1   na1 ,
• Li Zhou 2   na1 ,
• Xiaoling Zhang 1 ,
• Huina Zou 1 ,
• Yuanfeng Lu 1 &
• Huimin Xiao 1 , 3  

BMC Geriatrics volume  24 , Article number:  547 ( 2024 ) Cite this article

17 Accesses

Metrics details

The global aging population presents challenges that are particularly acute in China. Older Chinese adults’ attitudes towards death significantly impact their quality of life. Death education is crucial for promoting positive perspectives on life and death. Narrative education offers a promising approach to facilitating death education. Integrating the Knowledge-Attitude-Practice (KAP) model into death education will enhance the feasibility and acceptability of death education programs.

This mixed-methods feasibility study included a quasi-experimental trial and semi-structured interviews. Older adults in the intervention group ( N  = 27) received a 6-week KAP-based narrative life education program in addition to standard community health education; participants in the control group ( N  = 20) received only the normal community health education. In both groups, attitudes toward death and the meaning of life were assessed at baseline and immediately after the intervention. A post-intervention semi-structured interview and satisfaction survey were also conducted for the intervention group.

Forty out of 47 older adults completed the program for an 85.1% retention rate. All of the older adults in the experiment were very satisfied and satisfied with the life education program, and no adverse events were reported. Compared to the control group, participants in the intervention group had a significant decrease in the fear of death ( P  =  0 .028), and substantial improvement in their value of life ( P  =  0 .031), goal of life ( P  =  0 .035), freedom of life ( P  =  0 .003), and the total score for purpose in life ( P  =  0 .017). The qualitative results yielded four themes: profound recognition of life and death, contradiction between thoughts and action, conflict between one’s acceptance and others’ avoidance, and evaluation of the life education program.

Conclusions

The KAP-based narrative life education program is feasible and acceptable for older Chinese community-dwelling adults. It is also potentially effective in improving attitudes toward death attitudes and the meaning of life in this cohort.

Trial registration

This study was retrospectively registered at China Clinical Trial Registry as ChiCTR2300069551 on 2023-03-20. URL of registration: https://www.chictr.org.cn/showproj.html?proj=183176 .

Peer Review reports

The elderly population is poised to significantly increase around the world. By 2050, adults over 65 are projected to account for 16% of the global population, with the proportion of individuals aged 60 and above in China likely reaching 35% [ 1 , 2 ]. Death is inevitable, but it is a sensitive topic, especially in China. Older adults are prone to being increasingly aware of death due to their decline in physical function, the threat of chronic disease, and an increased witnessing of their peers’ deaths [ 3 ].

Attitudes towards death are not only related to the physical and mental health of older adults, but also affect their preparation for death and quality of life beforehand [ 4 ]. Ignorance of death preparation increases fear and anxiety about death [ 5 ]. However, a deep-rooted traditional culture that, emphasizes life and neglects death has made death a taboo subject in China. It is not easy for most Chinese people to communicate about death-related issues [ 6 ]. Although older adults can accept death as a part of life, most of them still feel fear and avoid talking about death-related topics [ 7 ]. Compared to nursing home residents, community-dwelling older adults are more afraid of facing death and feel it is more difficult to deal with life-and death-related issues [ 8 ]. Thus, further exploration is needed regarding how to help Chinese older adults establish a reasonable understanding of and attitudes toward life and death.

The essence of life education for older adults is orientation regarding the subjects of life and death, with death education comprising the core content [ 9 ]. The program teaches individuals how to recognize and face death [ 10 ]. The goal is to facilitate acceptance of end of life, process of death, and experience of bereavement in terms of the individual’s knowledge, attitudes, and skills [ 11 , 12 ]. Previous studies have shown that death education promotes positive changes in death-related attitudes, enhances the sense of meaning in life, and improves the quality of life [ 13 , 14 , 15 ]. However, previous programs have mainly focused on the stages of life and meaning of death and failed to address cultural conflicts in the process of death education, which may result in participants’ psychological discomfort. Thus, developing death education programs have been proposed that operate from the perspective of life’s course in order to reduce negative emotions and the fear of death [ 16 ]. Given the sensitivity of the topic, the method of delivering such education must be carefully considered.

Narrative education is an approach to achieving educational and research purposes by narrating, explaining, and reconstructing the experiences of educators and participants [ 17 ]. When addressing a sensitive topic, narratives generate less resistance because of the storytelling model [ 18 ]. Narratives may also facilitate older adults establishing reasonable cognition, knowledge, and behaviors related to death through introspection regarding their experiences and creation of meaning in their lives [ 19 ]. Therefore, the method can be used in death education in older adults [ 20 , 21 ].

A theoretical model is critical for framing the program, guiding data collection, and interpreting findings [ 22 ]. Various death education models have been developed such as lecture teaching and experience-sharing models [ 16 , 23 ]. In fact, compared to non-narrative messages, messages in narrative education have a stronger persuasive impact on one’s attitudes, intentions, and behaviors, both immediately and over time [ 24 ]. Therefore, a model that comprehensively attaches information acceptance, attitude modification, and behavior transition should be employed. The theory of Knowledge-Attitude-Practice (KAP) was first proposed by Cust and Mayo to explain the progressive relationship of moving from knowledge acquisition to behavior modification in individuals [ 25 ]. With the goal of helping individuals establish positive attitudes and beliefs and shifting towards correct behavior based on the reception and mastery of relevant knowledge, the theory has been widely applied in predicting health-related behaviors and implementing practice-improvement programs [ 26 ]. However, few studies have integrated the KAP model into death education for older adults, though it has the potential to communicate essential information, achieve reasonable life and death cognition, facilitate the maintenance of a positive attitude, and encourage the development of death-coping strategies [ 27 ]. Therefore, this study aimed to develop a KAP-based narrative life education program and explore its feasibility and effects on attitudes toward death and sense of meaning of life in older community-dwelling adults.

Study design

This mixed-methods feasibility study involved a quasi-experimental trial and semi-structured interviews. The goal was to determine the feasibility, acceptability, and primary efficacy of a narrative death education program for community-dwelling older adults. This study was reported following the Mixed Methods Reporting in Rehabilitation & Health Sciences (MMR-RHS) and was performed in accordance with the Declarations of Helsinki [ 28 ].

Setting and sample

From September to November 2022, older adults were recruited from a community located in Fuzhou City, China, from September to November 2022. It home to approximately 4,500 individuals aged 60 and above, constituting more than 19.90% of the total residential population. The inclusion criteria were: (a) aged 60 years and above and (b) able to understand and communicate in Chinese. The exclusion criteria were: (a) with cognitive impairment or (b) with severe visual, auditory, or mental disorders. A sample size between 24 and 50 participants is recommended for feasibility studies [ 29 ]. A total of 47 community-dwelling older adults were recruited for this study. Details of recruitment are in Additional file 1 .

Recruitment

Participants were recruited at the community health service center via two approaches. For on-site recruitment, a recruitment poster was posted at the center. Potential participants who were interested in the study could directly contact the research assistant (RA). The RA then introduced this study to them through a face-to-face interview. For tele-recruitment, the RA interviewed potential participants via telephone, based on a list of older adults provided by the center. Written informed consent was obtained from each participant. After baseline data collection, individuals were invited to voluntarily join either the intervention or control group according to their preferences. Given the sensitivity of the topic of death among Chinese older adults, we did not employ randomization.

Intervention program

Both groups received the usual health education provided by the community health center. The intervention group also received the KAP-based Narrative Life Education (KAPNLE) program.

Intervention group

The KAPNLE program was initially drafted after reviewing KAP theory and multimedia material on life education developed by the research team and engaging in internal brainstorming. Details of the program were revised according to two rounds of comments from a six-expert panel whose research areas involved geriatric nursing, life education, community nursing, psychology, and social work. We also conducted interviews with five community-dwelling older adults and further refined the program based on their feedback.

The final version was composed of four modules: Understanding Life and Death (Knowledge), Viewing Life and Death (Attitude), Preparing for Death (Practice), and Transcending Life and Death (Practice). These four modules covered a total of six sessions, including Life Course , Growing Old Peacefully , Passing Away in Pain , Saying “Goodbye” Well , Expressing “Love” , and Living a Wonderful Life . Each session was conducted according to a four-step narrative process by a researcher who served as the facilitator. Initially, the facilitator presented material on life and death issues and created a context within which participants could easily discuss topics of life and death. Participants were then invited to redescribe the topics in their own way and share their impressions. Next, they were guided to further reflect on their own experiences and discuss views on issues related to life and death. In the final step, activities on related themes were conducted in a relaxed environment to deepen participants’ knowledge and experience of life and death. The program was conducted once a week over six weeks and lasted 30 to 60 min per session. It was held offline in the visiting room of the community healthcare center and attended by groups of six or seven older adults. The details of the program are shown in Additional file 2 .

Control group

The control group met twice a month and only received the usual community health education, which includes topics related to chronic diseases, medication safety, and lifestyle management, their course did not involve life education.

Measurements

Basic information questionnaire.

Demographics and information regarding life and death issues were collected by a self-reported questionnaire designed by our research team (Additional file 3 ). The demographic information included age, gender, religion, education level, marital status, living status, and number of children. The issues related to life and death included life-threatening illness experiences, self-perceived physical health, most profound encounters with death, and communication about death topics.

Death Attitude Profile-Revised (DAP-R)

Death attitudes were assessed using the Chinese version of Death Attitude Profile Revised (DAP-R) [ 30 ]. It consists of 32 items and five dimensions: fear of death, death avoidance, neutral acceptance, approach acceptance, and escape acceptance. Each item is rated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Death attitudes are judged by the scores for each dimension. The higher the score, the more inclined the respondent is to this dimension’s attitude. In Chinese older adults, the Cronbach’s α values for the five dimensions were 0.796, 0.670, 0.621, 0.842 and 0.771.

Purpose in Life Test (PIL)

Meaning of life was measured using the Chinese version of Purpose in Life Test (PIL) [ 31 ]. It contains 20 items across four dimensions: quality of life, value of life, goal of life, and freedom of life. Scores are assigned using a five-point Likert scale, with each dimension including positive and negative questions. Scores selected for negative questions are reversed. Higher total scores indicate a greater sense of meaning of life. The Chinese version of the PIL has been validated, with a Cronbach’s α of 0.878.

Satisfaction of the program questionnaire

Respondents’ satisfaction was assessed using a self-designed questionnaire with six items: education theme, education content, education form, education schedule, benefits and practicability, and overall satisfaction. Each item is rated from “strongly satisfied” to “strongly dissatisfied” (Additional file 4 ).

Semi-structured interview

To assess the feasibility of the program for Chinese older adults, we conducted semi-structured interviews with the intervention participants. The interview outline was developed by the research team and began with a primary open-ended question: “What are your perceptions of the KAPNLE?” This question allowed participants to freely express their feelings and feedback about the program. Probing questions were then asked to facilitate in-depth exploration. The interview guideline is shown in Additional file 5 .

Data collection

The quantitative data were collected by another trained RA, who was blind to the group assignments. All participants’ death attitudes and ideas regarding the meaning of life were assessed at baseline and immediately after the program. In addition, the experimental participants were invited to describe their satisfaction with the program.

Participants in the intervention group were interviewed about their perceptions and experiences immediately after the program. A one-on-one semi-structured interview of each was conducted by the RA in the visiting room of the community healthcare center. Each interview lasted about 30 to 45 min, and the content was recorded with the participants’ informed consent.

Data analysis

Quantitative data.

The quantitative data were analyzed using IBM SPSS 27.0. The data were normality tested before being analyzed. Mean and standard deviation, median (P 25 , P 75 ), number, and percentage were determined to describe the older adults’ characteristics. A Chi-square test, t-test, Fisher’s Exact Test, Wilcoxon rank sum test, and multiple regression were used in this study. The Wilcoxon rank sum test was employed to test differences in the attitudes toward death and meaning of life between the groups for data with abnormal distributions. Multiple regression analysis was used to adjust for baseline imbalances in attitudes toward death and meaning of life between the two groups. Line graphs were used to describe any changes.

Qualitative data

The interviews were transcribed verbatim by a researcher within 24 h of their being conducted. Qualitative content analysis was used to analyze the qualitative data [ 32 ]. The steps applied were as follows: (a) identify and segment meaningful sentences within each interview text to generate “meaning units”, (b) condense semantic units into “condensed meaning units”, (c) abstract condensed semantic units to generate “codes”, (d) compare codes for commonalities, categorize codes into “categories”, (e) discuss and consensus on categories and formulate “themes” (Table  1 ). The interview data were coded by two researchers working independently. To ensure the credibility of the results, we used peer debriefing, member checks and held regular meetings to discuss the data analysis process and inconsistent opinions.

Feasibility of the program

A total of 47 older adults were recruited, including 27 in the intervention group and 20 in the control group. Seven in the intervention group withdrew from the study. Therefore, a total of 40 individuals completed the follow-up measurement and were included in analysis (Additional file 3 ). The total retention rate was 85.1%. Sixteen participants in the intervention group finished four to six sessions of the program, while four participants missed three out of six sessions. Their reasons included physical illness, family members’ illness or death, schedule conflicts, and self-isolation due to COVID-19 infection. The completion rate was 80.0% (16/20). All 20 intervention participants were very satisfied or satisfied with the education program, including its modules, sessions, implementation theme, and overall participation experience. No adverse events were reported during the study.

Participant characteristics

The mean age of the participants was 73.33 (6.16) years, with ages ranging from 63 to 88 years. Most of them were women (62.5%), not religious (75.0%), married (75.0%), had a high school education or above (75.0%), had one child (62.5%), lived with their children or spouse (85.0%), perceived themselves as in general or poor physical health (65.0%),were moved by their parents’ death (72.5%), and never communicated about death (62.5%). A small percentage had suffered (22.5%) or had family member who suffered (15.0%) from a life-threatening disease. There were no significant differences in demographic characteristics between the two groups (Additional file 6 ).

Preliminary efficacy of the program

Death attitudes.

Compared to the control group, a significant decrease was observed in fear of death in the intervention group ( P  = 0.028), no significant differences were detected in the other dimensions or total score of the DAP-R ( P  > 0.05). However, upward trends were observed in the DAP-R’s natural acceptance and approach acceptance. For the dimensions of death avoidance and escape acceptance, slight changes could also be found. After the intervention, there was no significant difference in total score of the DAP-R between the two groups, but the score for the control group fluctuated greatly during the follow-up.

Meaning of life

Compared to the baseline measurement, there were greater increases in value of life, goal of life, freedom of life, and total score of the PIL for the intervention group ( P <  0.05), and a slight upward trend was observed for freedom of life. However, no significant difference in the PIL’s quality of life was found between the two groups ( P  = 0.141). As shown in the line chart, differences were observed in the trends in the total scores for the PIL between the two groups, after the intervention, the scores for the intervention group increased markedly compared to the control group after the intervention (Additional file 7 and Additional file 8 ).

Perceptions of the program

According to the post-intervention interview, four themes and ten sub-themes were identified: (a) profound recognition of life, (b) contradiction between thoughts and action, (c) conflict between self-acceptance and others’ avoidance, and (d) evaluation of the life education program.

Profound recognition of life

This theme relates to older adults’ cognition of life and death, and contains three sub-themes:

Vague concept of life and death at the early stage

Some respondents stated that they had a vague understanding of life and death at the beginning of the program and had difficulties in describing or explaining them. They also expressed that they paid no attention to life and death-related issues in their daily lives over in years past.

“When you asked me about life and death, I really didn’t know how to answer. I never thought about it before. Life means I’m still alive; and death means I am away from the world. Is it right?” (Participant 2) .

Gradually clarifying life and death issues

After the first two sessions of the program. participants expressed that they had a figurative understanding of life and death. They realized the logical relationship between the two, and further accepted their unique lives.

“I didn’t know what life was like before, but now I do. My life is like a sunflower gone to seed. If I pass away like a flower withers, I still have something left in this world.” (Participant 1) .

“I feel my that life is a line with ups and downs, starting from zero, maybe ending at 100. Each number represents a stage of my life, and contains many important things.” (Participant 10) .

Discussing life and death with an open mind at the final stage

At the end of the program, participants expressed that they could easily discuss topics related to life and death and felt comfortable in the process. They also said that the program reduced their negative feelings about death and encouraged them to pursue meaning in their lives.

“Now I have a new perspective on life, and the fears about death seem to have vanished. Dying at the age of 20 or 100 are both lifetimes and being dead or alive cannot be decided by oneself. So, I will cherish my life when I am alive and enjoy life every day.” (Participant 11) .

“I realized that talking about death is not as difficult as I imagined. It actually could be very easy, just like this program.” (Participant 6) .

Contradictions between thoughts and actions

This theme is related to inconsistencies between positive thoughts about life and death and passive behaviors regarding death preparation, it includes the following two sub-themes:

Hold positive thoughts on to embrace life and deaths

Some participants expressed that they held rational and open attitudes about life and death after the program. They realized the inevitability of death, and calmly accept it as a normal phenomenon.

“I think that everyone will die in the end, and nobody can avoid death. I must go on my last journey well. As long as I have done all things, I can go without any regret.” (Participant 12) .

“I never thought about it (death) until I participate in this program. It reminded me that I would pass away one day. Then, I started to think about death issues in advance. If I had not participated in it, I wouldn’t have come to this step.” (Participant 5) .

Hesitating to make life and death plan

Some of the older adults emphasized living in the present, and were unwilling to make death preparation in advance.

“I am not thinking about what I should do about death at present. I just want live in the moment, do what I need to do at present, and stay happy.” (Participant 7) .

“I will think about these things, such as the cemetery or family arrangements, when I am more than 70 or 80 years old. But now, I live in the present and enjoy life.” (Participant 8) .

Conflict between one’s acceptance and others’ avoidance

This theme is related to the acceptability of life education, two sub-themes comprise this category.

Self-acceptance and openly discussion about life and death

Some participants noted that the program changed their attitudes about life and death. They not only felt comfortable talking about death-related topics, but also recommended the program to others.

“I think this program should be recommended to more older adults, especially to those who are sensitive and concerned about death. It can teach them how to deal with life and death, and overcome fear of death.” (Participant 10) .

Others’ opposition to life education due to stereotypes

Some participants mentioned that their family members or friends opposed their participation in the life education program due to the sensitivity of death-related topics.

“Most older adults around me resist talking about death. They tried to persuade me not to participate in life education because death is a taboo.” (Participant 4) .

“My family didn’t want me to attend such activity, for it will bring bad fortune. Therefore, I attended this class without telling my family.” (Participant 13) .

Evaluation of the life education program

This theme is related to the participants’ perspectives on the program and includes two sub-themes:

Affirmation of the program

Some of the older adults reflected that conversations about life and death were sensitive but acceptable. They further expressed that they could benefit from life education.

“It is not easy to talk about death-related topics, but I think life education is very important. Because it can help older adults do enough preparation and pass away without any regrets. I support this program.” (Participant 3) .

Some participants believed that the program was feasible because narrative life education enabled them to pick up death topic more easily. Moreover, the program was conducted in groups, which established a supportive environment.

“The most impressive thing about this program is telling stories. I have received various stories from others. Then, I felt pleased to share my thoughts and discuss with others about these stories.” (Participant 15) .

“It was easier for me to talk about life and death in groups. When someone started to talk about that, then we thought we could talk about that as well. You know, such an environment is important.” (Participant 17) .

Comments and suggestions

Some of the respondents mentioned that the group-based education might ignore individuals’ specific needs and suggested combining group education with individual counseling in the future.

“There was often someone absent in the group. Therefore, I think the program could add some individual education content, which would help the absentee to catch up with the progress.” (Participant 10) .

Regarding the resistance of their family or close friends to this program, some of the older adults hoped that the program could expand to include outside participants, such as by allowing them to invite people around them to participate.

“In fact, I still hope to get support and understanding from my family or friends, so maybe you can try to invite them to participate in this life education together.” (Participant 4) .

To the best of our knowledge, this is the first study to develop and evaluate the feasibility and preliminary effects of the KAPNLE. Our quantitative findings demonstrate that the program is effective at promoting a positive transition in death attitude and improving the meaning of life for community-dwelling older adults. The qualitative results indicate that the program is both acceptable and feasible. It also supports the potential of using the KAPNLE to change people’s attitudes toward life and death.

Our study indicates an acceptable feasibility among older adults. Seven participants withdrew from the study (a dropout rate of 14.8%), which was higher than a previous study [ 33 ]. One possible reason for the dropout rate may be that the sensitivity of the topic may have negatively affected these Chinese older adults’ willingness to participate [ 34 ]. Additionally, the study was conducted during the period of the COVID-19 pandemic, when older adults were more concerned about their physical health and less likely to engage in social activities [ 35 ]. There were no reported adverse events during the study for the intervention group, and all participants were either very satisfied or satisfied with the KAPNLE, indicating that the program is acceptable and safe. The KAP theory focuses on shared goals, transparency, accountability, and respect, and these factors are essential for effective collaboration in patient engagement [ 36 ]. Based on group-formatted discussions, the contents of the KAPNLE programs matched knowledge to attitudes, and then to action, in sequence rather than in a fragmented fashion. Participants’ feeling of freedom was encouraged at every step to promote their acceptance of death education. The older adults participating in our study stated that narration helped them address topics of life and death more easily and discuss related issues with less emotional resistance. In fact, the narrative approach emphasized finding psychosocial strengths and highlighting their own personal meaning of life [ 37 ]. Compared to didactic messages, those in narrative form tend to be more acceptable due to their natural format and the emotional engagement, and positive thoughts they inspire [ 38 ]. The narrative method can provide rich insights into the meanings associated with phenomena due to its deep subjectivity and inherent explanations of information [ 39 ]. Based on the multimedia cases we provided, the KAPNLE program adopted a four-step group-based narrative process that created a relaxed and supportive environment by focusing on storytelling, thus promoting acceptance of issues related to life and death [ 40 ]. This approach is useful in helping older adults understand and reconsider events in their lives, encouraging them to share information and transform their attitudes about death in an acceptable way [ 41 ].

Our quantitative results demonstrate that it is feasible to use the KAPNLE to promote a positive transition in attitudes toward death, especially in terms of reducing the fear of death, and these results are consistent with previous studies [ 33 , 42 ]. The post-program qualitative interviews indicated similar findings. The program provided relevant information about the process of life, hospice care, living wills, and death preparations. Correlating with KAP theory, the provision of comprehensive information about death can promote a more positive attitude, which can then help bridge the knowledge-intention gap in discussions about and preparations for death [ 43 ]. Moreover, participants also disclosed that the program had a promising effect on their understanding of the meaning of life and improved their acceptance of death. The KAPNLE guided these older adults to realize the inevitability of death through free discussion in a non-didactic group format and relaxing circumstance. Once their cognition of death was modified, they may experience greater openness to the possibilities available to them throughout the rest of their lives [ 44 ]. In line with previous studies, our results suggest that the KAPNLE improves the sense of meaning of life in older adults [ 45 , 46 ]. Participants were guided to be more aware of their achievements and self-value by reviewing their lives, and resist the consciousness of death by perceiving and maintaining a positive sense of self-meaning [ 47 ]. In line with concepts of KAP theory, modification of the cognition of and attitudes about death plays a crucial role in clarifying the meaning of life, finding purpose in life, and identifying suitable coping strategies [ 48 ].

Although the program encouraged participants to set goals and make plans for the rest of their lives, no direct changes in participants’ behavior were observed. The behaviors related to death preparation among older adults are often affected by family circumstances and subjective norms, and it is challenging to establish advanced death preparation due to the Chinese culture [ 49 , 50 ]. In addition, health status is associated with death preparation in older adults. Most participants in our study were not facing life-threatening illnesses, which may have reduced their initiative to engage with this program [ 51 ].

This study has some limitations, because of the cultural influences and stereotypes of death education from older adults’ family members, the dropout rate was relatively high. Moreover, although most participants had accepted the life process and found the meaning of life, they remained negative about making pre-death plans. This indicates that some of them may not have been operationally prepared for death and thus could not achieve the behavioral transition envisioned by the KAP model. Some participants also suggested that their family members or acquaintances should be involved in this program. In fact, due to the sensitivity of discussing about death within the traditional views of Chinese older adults, we allowed participants to choose group assignment by themselves to prevent ethical conflicts. This probably introduced selection bias for participants who chose to receive the intervention may have already been more open-minded about discussing death. In addition, this study did not employ a randomized controlled trial because our purpose was to explore the feasibility and acceptability of the program. Further research with rigorous design is needed to enhance participants’ adherence to narrative death education and to optimize intervention strategies.

This study constructed a KAP-based narrative life education program for community-dwelling older adults. We found that the life education program is acceptable and feasible among this cohort and could potentially improve attitudes toward death and the meaning of life. Future research with a rigorous design is necessary to test the effectiveness of narrative death education in older adults.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request. The data are not publicly available due to privacy and ethical restrictions.

Abbreviations

Knowledge-Attitude-Practice

KAP-based Narrative Life Education

Research assistant

Death Attitude Profile-Revised

Purpose in Life Test

The Challenges of. Population Aging in the People’s Republic of China; 2021.

Gerland P, Hertog S, Wheldon M, Kantorova V, Gu D, Gonnella G, Williams I, Zeifman L, Bay G, Castanheira H et al. World Population Prospects : 2022: Summary of results; 2022.

Nakagi S, Tada T. Relationship between identity and attitude toward death in Japanese senior citizens. J Med Invest. 2014;61(1–2):103–17. https://doi.org/10.2152/jmi.61.103 .

Article   PubMed   Google Scholar  

Mohammadpour A, Sadeghmoghadam L, Shareinia H, Jahani S, Amiri F. Investigating the role of perception of aging and associated factors in death anxiety among the elderly. Clin Interv Aging. 2018;13:405–10. https://doi.org/10.2147/CIA.S150697 .

Article   PubMed   PubMed Central   Google Scholar  

Berlin P, von Blanckenburg P. Death anxiety as general factor to fear of cancer recurrence. Psycho-oncology. 2022;31(9):1527–35. https://doi.org/10.1002/pon.5974 .

Hsu CY, O’Connor M, Lee S. Understandings of death and dying for people of Chinese origin. Death Stud. 2009;33(2):153–74. https://doi.org/10.1080/07481180802440431 .

Wysokiński M, Fidecki W, Jarosz M. Elderly People’s Acceptance of Death: A Study of a Polish Cohort. Int J Environ Res Public Health. 2019;16(18). https://doi.org/10.3390/ijerph16183374 .

Daaleman TP, Dobbs D. Religiosity, spirituality, and death attitudes in chronically ill older adults. Res Aging. 2010;32(2):224–43. https://doi.org/10.1177/0164027509351476 .

Article   Google Scholar  

Matsui M. Effectiveness of end-of-life education among community-dwelling older adults. Nurs Ethics. 2010;17(3):363–72. https://doi.org/10.1177/0969733009355372 .

Park S, Kim H, Jang MK, Kim H, Raszewski R, Doorenbos AZ. Community-based death preparation and education: a scoping review. Death Stud. 2023;47(2):221–30. https://doi.org/10.1080/07481187.2022.2045524 .

Martínez-Heredia N, Soriano Díaz A, Amaro Agudo A, González-Gijón G. Health Education as a Means of addressing death in the Elderly. Int J Environ Res Public Health. 2021;18(12):6652. https://doi.org/10.3390/ijerph18126652 .

Liu M, Chi I. Development and formative evaluation of a Death Education Program for Community-Dwelling Chinese older adults. Innov Aging. 2021;5(Suppl 1):20. https://doi.org/10.1093/geroni/igab046.072 .

Article   PubMed Central   Google Scholar  

Testoni I, Ronconi L, Cupit IN, Nodari E, Bormolini G, Ghinassi A, Messeri D, Cordioli C, Zamperini A. The effect of death education on fear of death amongst Italian adolescents: a nonrandomized controlled study. Death Stud. 2020;44(3):179–88. https://doi.org/10.1080/07481187.2018.1528056 .

Hwang H-L, Chen W-T, Lin H-S. Evaluation of Life and Death studies Course on attitudes toward life and death among nursing students. Kaohsiung J Med Sci. 2005;21(12):552–60. https://doi.org/10.1016/S1607-551X(09)70207-4 .

Chen W, Ma H, Wang X, Chen J. Effects of a death education intervention for older people with chronic disease and family caregivers: a quasi-experimental study. Asian Nurs Res (Korean Soc Nurs Sci). 2020;14(4):257–66. https://doi.org/10.1016/j.anr.2020.08.002 .

Kim BR, Cho OH, Yoo YS. The effects of Dying Well Education Program on Korean women with breast cancer. Appl Nurs Res. 2016;30:61–6. https://doi.org/10.1016/j.apnr.2015.11.007 .

Diekelmann N. Reawakening thinking: is traditional pedagogy nearing completion? J Nurs Educ. 1995;34(5):195–6. https://doi.org/10.3928/0148-4834-19950501-03 .

Article   CAS   PubMed   Google Scholar  

Ratcliff CL, Sun Y. Overcoming resistance through narratives: findings from a Meta-Analytic Review. Hum Commun Res. 2020;46(4):412–43. https://doi.org/10.1093/hcr/hqz017 .

Randall W, Baldwin C, McKenzie-Mohr S, McKim E, Furlong D. Narrative and resilience: a comparative analysis of how older adults story their lives. J Aging Stud. 2015;34:155–61. https://doi.org/10.1016/j.jaging.2015.02.010 .

Ratcliffe M, Byrne EA. Grief, self and narrative. Philosophical Explorations. 2022;25(3):319–37. https://doi.org/10.1080/13869795.2022.2070241 .

Wass H. A perspective on the current state of death education. Death Stud. 2004;28:289–308. https://doi.org/10.1080/07481180490432315 .

Keller Celeste R, Colvara B, Rech R, Reichenheim M, Bastos J. Challenges in operationalizing conceptual models in aetiological research. Commun Dent Oral Epidemiol. 2023;51(1):58–61. https://doi.org/10.1111/cdoe.12786 .

Testoni I, Palazzo L, Ronconi L, Donna S, Cottone PF, Wieser MA. The hospice as a learning space: a death education intervention with a group of adolescents. BMC Palliat Care. 2021;20(1):54. https://doi.org/10.1186/s12904-021-00747-w .

Oschatz C, Marker C. Long-term Persuasive effects in Narrative Communication Research: a Meta-analysis. J Communication. 2020;70(4):473–96. https://doi.org/10.1093/joc/jqaa017 .

Wicker A. Attitudes Versus actions: the relationship of verbal and overt behavioral responses to attitude objects. J Soc Issues. 2010;25:41–78. https://doi.org/10.1111/j.1540-4560.1969.tb00619.x .

Luo Y-F, Chen L-C, Yang S-C, Hong S. Knowledge, attitude, and practice (KAP) toward COVID-19 pandemic among the Public in Taiwan: a cross-sectional study. Int J Environ Res Public Health. 2022;19(5):2784. https://doi.org/10.3390/ijerph19052784 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Valente TW, Paredes P, Poppe PR. Matching the message to the process: the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res. 1998;24(3):366–85. https://doi.org/10.1111/j.1468-2958.1998.tb00421.x .

Tovin MM, Wormley ME. Systematic development of standards for Mixed Methods Reporting in Rehabilitation Health Sciences Research. Phys Ther. 2023;103(11). https://doi.org/10.1093/ptj/pzad084 .

Julious S. Sample size of 12 per group rue of thumb for a pilot study. Pharm Stat. 2005;4:287–91. https://doi.org/10.1002/pst.185 .

Lu T, Ling Z, Yuxiang L, Lingjun Z, Jing C, Xianli M, Jijun Z, Anesthesiology DO. Validation and reliability of a Chinese version death attitude Profile-revised (DAP-R) for nurses. J Nurs Sci. 2014;29:64–6. https://doi.org/10.3760/CMA.J.ISSN.1672-7088.2014.22.012 .

Shek DT. Reliability and factorial structure of the Chinese version of the purpose in Life Questionnaire. J Clin Psychol. 1988;44(3):384–92. https://doi.org/10.1002/1097-4679(198805)44:3%3C384::aid-jclp2270440312%3E3.0.co;2-1 .

Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12. https://doi.org/10.1016/j.nedt.2003.10.001 .

Chen W, Ma H, Wang X, Chen J. Effects of a death education intervention for older people with chronic disease and family caregivers: a quasi-experimental study. Asian Nurs Res. 2020;14:257–66. https://doi.org/10.1016/j.anr.2020.08.002 .

Cheng HW, Li CW, Chan K, Ho R, Sham MK. Bringing Palliative Care into Geriatrics in a Chinese Culture Society—results of a collaborative model between Palliative Medicine and Geriatrics Unit in Hong Kong. J Am Geriatr Soc. 2014;62(4):779–81. https://doi.org/10.1111/jgs.12760 .

Jiang W, Sun F, Prieto L, Fang Y, Gao Y, Yue L, Lin X, Zhao L, Dang J, Qiu J, et al. Worries, strategies, and confidence of older Chinese adults during the 2019 novel coronavirus outbreak. Int J Geriatr Psychiatry. 2020;35(12):1458–65. https://doi.org/10.1002/gps.5430 .

Pushparajah DS. Making patient Engagement a reality. Patient. 2018;11(1):1–8. https://doi.org/10.1007/s40271-017-0264-6 .

Lind M, Bluck S, McAdams D. More vulnerable? The Life Story Approach highlights older people’s’ potential for Strength during the pandemic. Journals Gerontol Ser B Psychol Sci Social Sci. 2020;76(2):e45–8. https://doi.org/10.1093/geronb/gbaa105 .

Scherr C, Nam K, Augusto B, Kasting M, Caldwell M, Lee M, Meade C, Pal T, Quinn G, Vadaparampil S. A Framework for Pilot Testing Health Risk Video narratives. Health Commun. 2019;35:1–10. https://doi.org/10.1080/10410236.2019.1598612 .

Toledano N, Anderson A. Theoretical reflections on narrative in action research. Action Res. 2017;18:302–18. https://doi.org/10.1177/1476750317748439 .

Weiss CR, Johnson-Koenke R. Narrative Inquiry as a Caring and Relational Research Approach: adopting an evolving paradigm. Qual Health Res. 2023;33(5):388–99. https://doi.org/10.1177/10497323231158619 .

Taylor L, Blain J, Kingston P, Eost-Telling C. Personal narratives of aging. Innov Aging. 2019;3:S757–757. https://doi.org/10.1093/geroni/igz038.2780 .

Chow EOW, Fung SF. Narrative Group Intervention to Rediscover Life Wisdom among Hong Kong Chinese older adults: a single-blind Randomized Waitlist-Controlled Trial. Innov Aging. 2021;5(3):igab027. https://doi.org/10.1093/geroni/igab027 .

Schlueter K, Vamos S, Wacker C, Welter V. A conceptual model map on Health and Nutrition Behavior (CMM HB/NB). Int J Env Res Pub He. 2020;17(21):7829. https://doi.org/10.3390/ijerph17217829 .

van Wijngaarden E, Merzel M, Berg V, Zomers M, Hartog I, Leget C. Still ready to give up on life? A longitudinal phenomenological study into wishes to die among older adults. Soc Sci Med. 2021;284:19. https://doi.org/10.1016/j.socscimed.2021.114180 .

Chow E, Fok D. Recipe of life: a Relational Narrative Approach in Therapy with persons Living with Chronic Pain. Res Social Work Pract. 2020;30(3):320–9. https://doi.org/10.1177/1049731519870867 .

Li B. Navigating through the narrative montages: including voices of older adults with Dementia through Collaborative Narrative Inquiry. Int J Qualitative Methods. 2022;21. https://doi.org/10.1177/16094069221083368 .

Heinz M, Benton N, Gleissner L. Older adults documenting purpose and meaning through photovoice and narratives. Gerontologist. 2023;63(8):1289–99. https://doi.org/10.1093/geront/gnad008 .

Can Oz Y, Duran S, Dogan K. The meaning and role of spirituality for older adults: a qualitative study. J Relig Health. 2022;61(2):1490–504. https://doi.org/10.1007/s10943-021-01258-x .

Ke L-S, Cheng H-C, Ku Y-C, Lee M-J, Chang S-Y, Huang H-Y, Lin Y-L. Older adults’ behavioral intentions toward Advance Care Planning based on theory of reasoned action. J Hospice Palliat Nurs. 2022;24(6):e294–300. https://doi.org/10.1097/NJH.0000000000000907 .

Miyashita J, Kohno A, Yamamoto Y, Shimizu S, Azuma T, Takada T, Hayashi M, Fukuhara S. How psychosocial factors contribute to Japanese older adults’ initiation of Advance Care Planning discussions: a qualitative study. J Appl Gerontol. 2021;40(10):1180–8. https://doi.org/10.1177/0733464820911537 .

Siconolfi D, Bandini J, Chen E. Individual, interpersonal, and health care factors associated with informal and formal advance care planning in a nationally-representative sample of midlife and older adults. Patient Educ Couns. 2021;104(7):1806–13. https://doi.org/10.1016/j.pec.2020.12.023 .

Download references

Acknowledgements

The authors gratefully thank all participants for their support and cooperation.

No funding or sources of support are declared.

Author information

Xifeng Xie and Li Zhou contributed equally to this work.

Authors and Affiliations

School of Nursing, Fujian Medical University, Fuzhou City, Fujian Province, China

Xifeng Xie, Xiaoling Zhang, Huina Zou, Yuanfeng Lu & Huimin Xiao

Nanjie Community Health Service Center, Fuzhou City, Fujian Province, China

Research Center for Nursing Humanity, Fujian Medical University, Fuzhou City, Fujian Province, China

Huimin Xiao

You can also search for this author in PubMed   Google Scholar

Contributions

X.X.: Methodology, Investigation, Data Curation, Data Analysis, Visualization, Writing - Original Draft; L.Z.: Methodology, Investigation, Data Curation, Data Analysis; X.Z.: Data Curation, Data Analysis; H.Z.: Data Curation, Data Analysis; Y.L.: Writing-Review & Editing, Revision; H.X.: Conceptualization, Methodology, Resources, Supervision, Writing-Review & Editing. All authors reviewed the manuscript.

Corresponding authors

Correspondence to Yuanfeng Lu or Huimin Xiao .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the Biomedical Ethics Review Committee of Fujian Medical University (IRB Ref No:2022-71). Written informed consent was obtained from each participant prior to the study. Participants were free to choose their group according to their preferences, and also to withdraw.

Consent for publication

Informed verbal consent for publication was obtained from all participants included in the study.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Xie, X., Zhou, L., Zhang, X. et al. Evaluation of a knowledge-attitude-practice model based narrative life education program for community-dwelling older adults: a mixed-methods feasibility study. BMC Geriatr 24 , 547 (2024). https://doi.org/10.1186/s12877-024-05153-4

Download citation

Received : 26 March 2024

Accepted : 17 June 2024

Published : 24 June 2024

DOI : https://doi.org/10.1186/s12877-024-05153-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Older adults
• Knowledge-Attitude/Belief-Practice theory
• Death education
• Feasibility study

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals

You are here

• Volume 14, Issue 6
• Perceptions of families and healthcare providers about feeding preterm infants in the neonatal intensive care unit: protocol for a qualitative systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0009-0008-5313-9272 Willow R Schanz 1 ,
• Aunum Akhter 2 ,
• Georgette Richardson 3 ,
• http://orcid.org/0000-0003-0229-966X William T Story 4 ,
• Riley Samuelson 5 ,
• http://orcid.org/0000-0002-7026-0006 Aamer Imdad 6
• 1 The University of Iowa Roy J and Lucille A Carver College of Medicine , Iowa City , Iowa , USA
• 2 Division of Neonatology , The University of Iowa Health Care, Stead Family Department of Pediatrics, Roy J and Lucille A Carver College of Medicine , Iowa City , Iowa , USA
• 3 Division of Pediatric Psychology , The University of Iowa Health Care, Stead Family Department of Pediatrics , Iowa City , Iowa , USA
• 4 Department of Community and Behavioral Health , The University of Iowa College of Public Health , Iowa City , Iowa , USA
• 5 University of Iowa Hardin Library for the Health Sciences , Iowa City , Iowa , USA
• 6 Division of Gastroenterology, Hepatology, Pancreatology and Nutrition , University of Iowa Health Care, Stead Family Department of Pediatrics, Roy J and Lucille A Carver College of Medicine , Iowa City , Iowa , USA
• Correspondence to Dr Aamer Imdad; aamer-imdad{at}uiowa.edu

Introduction The underdevelopment of preterm infants can lead to delayed progression through key early milestones. Demonstration of safe oral feeding skills, constituting proper suck-swallow reflex are requirements for discharge from the neonatal intensive care unit (NICU) to ensure adequate nutrition acquisition. Helping an infant develop these skills can be draining and emotional for both families and healthcare staff involved in the care of preterm infants with feeding difficulties. Currently, there are no systematic reviews evaluating both family and healthcare team perspectives on aspects of oral feeding. Thus, we first aim to evaluate the current knowledge surrounding the perceptions, experiences and needs of families with preterm babies in the context of oral feeding in the NICU. Second, we aim to evaluate the current knowledge surrounding the perceptions, experiences and needs of healthcare providers (physicians, advanced practice providers, nurses, dietitians, speech-language pathologists and occupational therapists) in the context of oral feeding in the NICU.

Methods and analysis A literature search will be conducted in multiple electronic databases from their inception, including PubMed, CINHAL, Embase, the Cochrane Central Register for Controlled Trials and PsycINFO. No restrictions will be applied based on language or data of publication. Two authors will screen the titles and abstracts and then review the full text for the studies’ inclusion in the review. The data will be extracted into a pilot-tested data collection sheet by three independent authors. To evaluate the quality, reliability and relevance of the included studies, the Critical Appraisal Skills Programme checklist will be used. The overall evidence will be assessed using the Grading of Recommendation Assessment, Development and Evaluation criteria. We will report the results of the systematic review by following the Enhancing Transparency in Reporting the synthesis of Qualitative research checklist.

Ethics and dissemination Ethical approval of this project is not required as this is a systematic review using published and publicly available data and will not involve contact with human subjects. Findings will be published in a peer-reviewed journal.

PROSPERO registration number CRD42023479288.

• Paediatric gastroenterology
• Systematic Review
• Percieved Social Support
• NEONATOLOGY

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2024-084884

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

This will be a systematic review evaluating both the perspectives of families and neonatal healthcare professionals on feeding practices of preterm infants in the neonatal intensive care unit (NICU).

Evaluating the perspectives of both family members and neonatal healthcare professionals involved in the care of preterm babies with feeding difficulties may uncover shared grievances and mutually beneficial opportunities for quality improvement in the NICU.

Included studies might be conducted in diverse settings, so generalisability to clinical practice may be affected by cultural, language and healthcare systems context.

Introduction

An estimated 13.4 million babies were born preterm (<37 weeks gestation) in 2020, which represented about 10% of all live births worldwide. 1 Preterm birth is a serious health event that contributes to significant morbidity, mortality and increased healthcare cost in neonates. Over 40% of premature infants will experience feeding difficulties, such as struggling to develop typical feeding reflexes (sucking, swallowing, appropriate breathing) and coordinated oesophageal bolus transport. 2 Consequently, feeding difficulties are associated with elevated healthcare costs due to increased length of stay in the neonatal intensive care unit (NICU) and invasive measures, such as a central line or other parenteral support, to supply the infant with adequate nutrients. 3 4 Poor feeding skills are associated with increased morbidity through malnutrition and growth restriction as well as increased mortality through oropharyngeal aspiration. 5 6

Despite the global prevalence, expense and severity of feeding difficulties, no universal guidelines function as the gold standard of care for feeding preterm infants. 7 The resulting high variability in approach may lead to dissatisfaction among NICU families and healthcare professionals. Families of preterm infants have been shown to express concerns about the technicality of feeding interventions, communication with providers regarding their child and feeling isolated from the feeding approaches in the NICU. 8 Tube feeding, a common feeding intervention for preterm infants, has been associated with increased cost, rehospitalisation, stress and anxiety for families. Due to the emotional nature of feeding a newborn, family members may struggle with learning to feed their infant in this manner. 8 Additionally, nurse perceptions of oral feeding in the NICU have emphasised the impactful role they hold in teaching feeding techniques and relieving emotional distress for the family, which has highlighted a need for greater collaboration between the family and care providers. 9 Family integrated care has been perceived to be helpful in the reduction of maternal stress by parents of preterm infants as well as a necessary and feasible care model by neonatologists and NICU nurses that has the potential to lower length of hospitalisation, decrease healthcare costs and improve breastfeeding rates in preterm infants. 10–12 The approach to feeding preterm infants requires a multidisciplinary effort, including the family, nurses, dietitians, occupational therapists, speech-language pathologists, social workers, advanced practice providers and physicians. Despite these experiences being reported, there is still limited understanding regarding the perceptions of families and caregivers on feeding preterm infants in the NICU. 13 14 This qualitative systematic review aims to analyse the current global knowledge of the perceptions, experiences and needs of families and healthcare staff (nurses, physicians, advanced practice providers, dietitians, occupational therapists, social workers and speech-language pathologists) involved in the feeding process of preterm infants in the NICU, as well as possible improvements to decrease barriers to high-quality care.

Methods and analysis

This systematic review will be conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and checklist. 15

Literature search

Systematic electronic queries, available in online supplemental appendix A , will be conducted in major databases, including PubMed, CINHAL, Embase, the Cochrane Central Register for Controlled Trials and PsycINFO from their inception to date of inquiry. Key terms used in the search are those related to population, context and phenomena of interest (perspectives, views, needs, experiences, perceptions, barriers, challenges). Studies will not be excluded based on the publication year, publication status, geographical location or language. Thus, this analysis will include studies from all countries. Studies evaluating specific racial, gender, geographic, age (of family or provider) differences will be included in this review as long as they evaluate qualitative aspects of our phenomena of interest. Bibliographic software (EndNote) will be used to combine database search results, and duplicates will be removed.

Supplemental material

Eligibility criteria.

The populations of interest include families of preterm infants (parents, mother, father, grandmother, grandfather and guardians) and neonatal healthcare professionals (nurses, physicians, advanced practice providers, caregivers, dietitians, speech-language pathologists, social workers and occupational therapists) involved in the feeding care of preterm infants. For this review, preterm birth will be defined as gestational age <37 weeks at birth. If relevant, additional definitions such as late preterm: 34–36 weeks, moderately preterm: 32–34 weeks, very preterm: 28–32 weeks, extremely preterm: <28 weeks gestational age at birth, will be used and clearly reported.

We are interested in the global state of enteral and oral feeding in preterm infants while in NICUs from the perspective of both families and healthcare providers.

Phenomena of interest

The main phenomena of interest are the experiences and perceptions of families with preterm infants and healthcare providers of preterm infants as outlined below:

Family experiences regarding NICU feeding practices.

Family perceptions of NICU feeding practices.

Family needs regarding care of infants with feeding difficulties.

Family barriers regarding care of infants with feeding difficulties.

Healthcare staff perceptions of the NICU feeding practices.

Healthcare staff needs regarding care of infants with feeding difficulties.

Healthcare staff barriers regarding care of infants with feeding difficulties.

Screening and selection of studies

Screening of studies will be conducted through systematic review software Covidence by three authors (WRS, GR and AI). The initial review will consist of title and abstract filtering for relevance to systematic review objective by three authors (WRS, GR and AI). For studies to progress to future screening, they must evaluate the perceptions regarding feeding practices of preterm infants in the NICU in one of our two populations of interest: (1) families and (2) healthcare providers. Studies deemed irrelevant or out of context will be excluded, such as those evaluating children in the paediatric intensive care unit and those evaluating NICU graduates following up in outpatient clinics. The second stage of study selection will include a complete text review of each potential article by three authors (WRS, GR and AI). Conflicts at all stages will be resolved by discussion and contacting a senior author. Additionally, the references of relevant reviews will be evaluated for inclusion in the review. In the case that only an abstract is available for a given study, authors will be contacted to obtain information on and evaluate methods and results. If we are unable to obtain additional information, the abstract will be evaluated exclusively by inclusion criteria. If a paper is published in a language other than English, we will attempt to translate the article for use in this review. If we are unable to translate the article, we will exclude it from this review.

Data extraction

Data extraction will occur independently by three authors (WRS, GR and AI) and subsequent comparison will occur. Conflicts will be resolved through discussion. To standardise data acquisition, a custom data extraction template will be piloted and used in Covidence. Information to be collected from each study will include:

Study design, study duration, study setting, setting country/region, study year and interventions.

Participants

Recruitment methods, including inclusion and exclusion criteria; group differences; sample size; sample size calculation; relevant baseline characteristics (family participants: maternal age, infant gestational age at birth, infant weight at birth, race/ethnicity, etc.; healthcare professional participants: role, experience, race/ethnicity, etc.); intervention groups.

Qualitative: Phenomena of interest (perceptions, experiences, change in satisfaction, change in feeding rate, etc); definitions of phenomena of interest.

Quantitative (if regarding phenomena of interest): variable type (continuous, dichotomous, qualitative); reporting measure (continuous variable: CIs, SD, SE, etc; dichotomous variable: the number of participants, percentage of participants, OR, etc; qualitative); statistical significance of outcome (p value).

Major themes addressed

Stress, anxiety, fear, needs, barriers, satisfaction, etc.

Other relevant constructs

First-order constructs (participant quotes); second-order constructs (author interpretations).

This data extraction protocol is modelled from thematic analysis principles of qualitative evidence synthesis and recommendations by the Cochrane Qualitative and Implementation Methods Group guidance for data extraction and data synthesis. 16 17 After data extraction, these data will be exported to Excel for synthesis and organised by relevant population.

Data synthesis

Data will be synthesised for each relevant population and outcome combination by three authors (WRS, GR and AI). Major themes will be described in a narrative fashion and simple descriptive statistics may be utilised for clarity. In the case of studies having quantitative measures of our qualitative interests, we will report the data as follows: If relevant, dichotomous data will be reported with OR, 95% CIs, and risk ratios, and continuous data will be reported as confidence intervals. Significant construct findings will be reported as quotes, percentages or other descriptive reports. Any inconsistencies or discrepancies between studies will be considered and reported. Data will be reported in narratives and tables for presentation.

Reporting results

Once the study analysis is complete, we will provide a narrative synthesis of all included studies and analysis between comparable studies. We will compare knowledge, beliefs, attitudes and perceptions of families with infants in the NICU within this population as well as compare these findings to the knowledge, beliefs, attitudes and perceptions knowledge of neonatal healthcare professionals. We will include all findings listed in the ‘Phenomena of interest’ section. Reporting of results will be in accordance with PRISMA and Enhancing Transparency in Reporting the synthesis of Qualitative research guidelines. 15 18

Critical appraisal of the studies

To evaluate the quality, reliability and relevance of the included studies, we plan to follow the Critical Appraisal Skills Programme checklist. 19 This tool is often used to appraise qualitative research and is adaptable to emphasise particular areas of interest within our research question. It is recommended by Cochrane and complements the use of the Grading of Recommendations Assessment, Development and Evaluation—Confidence in Evidence from Reviews of Qualitative Research (GRADE-CERQ) approach through evaluating the strengths and weaknesses of each study rather than on the basis of exclusion. This tool will be used by three members of the review team (WRS, GR and AI), and disagreements will be mediated through conversation.

Certainty of review findings

The GRADE-CERQ approach will be used to evaluate the overall certainty of evidence. 20 This approach is a comprehensive framework used to assess the overall certainty of the evidence for an outcome using study characteristics such as study design, inconsistency, indirectness of evidence, risk of bias, publication bias and imprecision estimates. We will include the GRADE-CERQ assessment results in an evidence profile that contains certainty ratings, including very low, low, moderate or high, based on the evidence across studies for primary outcomes. We will follow the GRADE-CERQ guidelines for assessing confidence in our qualitative evidence findings, which are based on four components: methodological limitations, relevance, adequacy and coherence. Based on analysis in each of these categories, the study will be given a score of either strong or weak. Concerns with any of the components may reduce our confidence in a review finding.

Patient and public involvement

Ethics and dissemination.

This is a qualitative systematic review that evaluates data present in the public domain through published studies and does not involve contact with human subjects. As a study of published literature, this study was not subject to formal IRB (Institutional Reviw Board) approval. We anticipate that the systematic review will be complete by fall of 2024 and will be submitted for publication in a peer-reviewed journal.

Ethics statements

Patient consent for publication.

Not applicable.

Acknowledgments

The authors would like to acknowledge Paul Casella for his help in editing the manuscript

• Moller A-B ,
• Bradley E , et al
• Yamasaki JT , et al
• Rolnitsky A ,
• Urbach D , et al
• Victora CG ,
• Walker SP , et al
• Jackson B ,
• Mörelius E ,
• Sahlén Helmer C ,
• Hellgren M , et al
• Zhu X , et al
• Li Y , et al
• Wang S , et al
• Osborn EK ,
• Alshaikh E ,
• Nelin LD , et al
• Gulati IK ,
• Jadcherla S
• McKenzie JE ,
• Bossuyt PM , et al
• Flemming K , et al
• Flemming K ,
• McInnes E , et al
• Glenton C , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Contributors Conceptualisation: WRS, AA, GR and AI; Methodology: WRS, AA, GR, WTS, RS and AI; Writing–original draft preparation: WRS and AI; Writing–review and editing: WRS, AA, GR, WTS, RS and AI. All authors have read and agreed to the published version of the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Javascript is currently disabled in your browser. Several features of this site will not function whilst javascript is disabled.

• Why Publish With Us?
• Editorial Policies
• Author Guidelines
• Peer Review Guidelines
• Open Outlook
• Submit New Manuscript

• Sustainability
• Press Center
• Testimonials
• Favored Author Program
• Permissions
• Pre-Submission

open access to scientific and medical research

Back to Journals » Patient Preference and Adherence » Volume 18

Comparing Methods for Identifying Post-Market Patient Preferences at the Point of Decision-Making: Insights from Patients with Chronic Pain Considering a Spinal Cord Stimulator Device

• Get Permission
• Cite this article

Authors Golembiewski EH   , Leon-Garcia M , Gravholt DL , Brito JP , Spatz ES , Bendel MA   , Montori VM   , Maraboto AP , Hartasanchez SA , Hargraves IG

Received 19 July 2023

Accepted for publication 10 May 2024

Published 25 June 2024 Volume 2024:18 Pages 1325—1344

DOI https://doi.org/10.2147/PPA.S431378

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Jongwha Chang

Elizabeth H Golembiewski, 1 Montserrat Leon-Garcia, 1– 3 Derek Loy Gravholt, 1 Juan P Brito, 1 Erica S Spatz, 4 Markus A Bendel, 5 Victor M Montori, 1 Andrea P Maraboto, 1 Sandra A Hartasanchez, 1 Ian G Hargraves 1 1 Knowledge and Evaluation (KER) Unit, Mayo Clinic, Rochester, MN, USA; 2 Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain; 3 Department of Pediatrics, Obstetrics, Gynaecology and Preventive Medicine, Universidad Autónoma de Barcelona, Barcelona, Spain; 4 Division of Cardiovascular Medicine, School of Medicine, Yale University, New Haven, CT, USA; 5 Division of Pain Medicine, Mayo Clinic, Rochester, MN, USA Correspondence: Ian G Hargraves, Knowledge and Evaluation Research (KER) Unit, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA, Email [email protected] Purpose: To compare three methods for identifying patient preferences (MIPPs) at the point of decision-making: analysis of video-recorded patient-clinician encounters, post-encounter interviews, and post-encounter surveys. Patients and Methods: For the decision of whether to use a spinal cord stimulator device (SCS), a video coding scheme, interview guide, and patient survey were iteratively developed with 30 SCS decision-making encounters in a tertiary academic medical center pain clinic. Burke’s grammar of motives was used to classify the attributed source or justification for a potential preference for each preference block. To compare the MIPPs, 13 patients’ encounters with their clinician were video recorded and subsequently analyzed by 4 coders using the final video coding scheme. Six of these patients were interviewed, and 7 surveyed, immediately following their encounters. Results: For videos, an average of 66 (range 33– 106) sets of utterances potentially indicating a patient preference (a preference block), surveys 33 (range 32– 34), and interviews 25 (range 18– 30) were identified. Thirty-eight unique themes (75 subthemes), each a preference topic, were identified from videos, surveys 19 themes (12 subthemes), and interviews 39 themes (54 subthemes). The proportion of preference blocks that were judged as expressing a preference that was clearly important to the patient or affected their decision was highest for interviews (72.8%), surveys (68.0%), and videos (27.0%). Videos mostly attributed preferences to the patient’s situation (scene) (65%); interviews, the act of receiving or living with SCS (43%); surveys, the purpose of SCS (40%). Conclusion: MIPPs vary in the type of preferences identified and the clarity of expressed preferences in their data sets. The choice of which MIPP to use depends on projects’ goals and resources, recognizing that the choice of MIPP may affect which preferences are found. Keywords: patient preferences, decision making, regulatory, preference identification, preference elicitation

Introduction

Patient preferences, broadly defined as the relative desirability or acceptability of different attributes of health interventions, 1 strongly influence the use of medical products. In the specific context of chronic pain management, patient preferences are particularly influential given the large array of available treatment alternatives. For example, patient preferences, driven by direct-to-consumer advertising, may have contributed to the use, overuse, and abuse of opioids. 2 In addition, treatment preferences may be further influenced by gender differences in pain experiences and the quality of health care received when seeking pain management. 3 Where relevant, empirical assessments of patient preferences can provide valuable insights during the regulatory process by which medical treatments and devices are evaluated for safety and effectiveness and ultimately approved to be sold on the market. Although the Food and Drug Administration (FDA), the government agency tasked with regulating medical drugs and devices in the United States, is increasingly encouraging the appropriate use of patient preference studies in the regulatory process, efforts have largely focused on pre-market evaluations during medical product development, clinical trial design, and benefit-risk assessments. 4 , 5 However, these preferences may not reflect how patient preferences affect the use of a drug or device after it becomes available for use (ie, during the post-market phase).

Post-market assessment of patient preferences can shed light on changes in patient perceptions or benefit-harm determinations once a drug or device has been used more widely. Some aspects of post-market preferences can be derived from the experience of those who have adopted and lived with a medical product. Other insights may come from the preferences that patients hold when they are making decisions with their clinicians about product use. These preferences are particularly significant as they likely determine if the patient will adopt the drug or device. Understanding the post-market, pre-adoption preferences that are at play during clinical decision-making can facilitate patient-centered communication of benefit-harm information on drug or device labeling, inform the production of shared decision-making tools, contribute to formulating clinical practice guidelines, and support clinicians in making decisions with their patients. 1 , 5–7 There is a need to characterize post-market patient preferences as they manifest when making decisions and which then affect drug or device use.

However, the best methods for understanding the nature and role of post-market preferences are underdeveloped, 8 , 9 particularly methods for understanding the preferences that affect patient’s decision-making. Patient preferences are commonly elicited with methods that rely on hypothetical choices or scenarios, such as discrete choice experiments or best-worst scaling. 9 In addition, most patient preference studies are conducted outside of real-time clinical decision-making, asking patients to consider their values and preferences in the abstract, either retrospectively for past decisions or in a speculative fashion for future decisions. Methods to evaluate real—not hypothetical— patient preferences at the point of decision-making are underdeveloped.

The objective of this study was to compare three distinct methods for identifying patient preferences (MIPPs) as they manifest in decision-making encounters between patients and clinicians: (a) video analysis of clinical encounters, (b) post-encounter interviews, and (c) post-encounter surveys. In this study, we used the decision of whether to use a spinal cord stimulator (SCS) for the management of chronic pain to compare these methods. In the context of the many alternative treatment options available for chronic pain management, the decision to adopt SCS can be described as highly sensitive to patient preferences. Notably, the FDA has identified benefit-harm tradeoffs related to attributes of chronic pain therapy as a patient preference-sensitive priority area. 10

Clinical Context

SCS is a neuromodulation treatment for chronic pain that involves implantation of electrical leads in the epidural space connected to an implanted pulse generator. SCS is indicated for patients with chronic pain for whom oral or topical medications, physical therapy, and clinic-administered injections have been ineffective. 11 The first fully implantable SCS device became available commercially in 1981, shortly after the FDA began regulating medical devices under the Medical Device Amendments of 1976. 12 Today, there are nine FDA-approved SCS devices on the market, with variability in features such as battery life and recharging frequency, stimulation settings, magnetic resonance imaging (MRI) compatibility, and need to deactivate while driving. 13 SCS, as a treatment modality, also has different use characteristics than medical approaches.

In general, placement of the stimulator follows a two-stage process consisting of placing temporary leads for one week in the epidural space and connecting to an external trial SCS, followed by permanent implantation of the leads and device. An SCS trial is considered successful when the patient experiences ≥50% reduction in pain level. 14 However, many other considerations influence the suitability of SCS for an individual patient, including ability to attend multiple appointments before, during, and after implantation, history of comorbid conditions that increase risk of infection and postoperative complications, and capacity to follow instructions for postoperative care and ongoing use of the device. 11 , 14–16

Development and Comparison of the MIPPs

Study Setting and Population

All data collection activities, for both the MIPP development and comparison phases, were conducted in the Pain Clinic at Mayo Clinic, Rochester, MN. Eligible patients were 18 or older, had no indication of impaired cognitive function, and were scheduled to have an outpatient consultation or education visit related to SCS with a pain medicine physician, nurse educator, or clinical fellow. Participants were enrolled between November 2019 and November 2021. All study procedures were conducted in accordance with the Declaration of Helsinki and approved by the Mayo Clinic Institutional Review Board (study ID: 19–006857).

Recruitment and Consent

Project investigators identified clinicians at the Pain Clinic involved in SCS consults or patient education. After meeting with each clinician to explain the purpose of the study and obtain their written informed consent to participate, a study coordinator screened their schedules weekly for eligible patients. Clinicians could raise any concerns about the inclusion of any patient prior to their invitation. A total of 28 clinicians (including staff physicians, nurse educators, and clinical fellows) were approached for consent throughout the course of both study phases, with 2 declining to participate (93% enrollment rate).

Shortly before an eligible patient’s scheduled appointment time, a study team member entered the exam room to obtain written informed consent from the patient (and oral permission from any patient guests) to participate in the study. During the MIPP development phase, 64 eligible patients were approached in clinic, 28 declined, and 36 consented to have their encounter filmed and complete a post-encounter survey or interview (56% enrollment rate). For the comparison phase, 20 patients were approached, 7 declined, and 13 were enrolled (65% enrollment rate).

For patients who consented to participate, the study team member then placed a small video recorder in a central position in the exam room such that the patient and clinician would both be captured in the frame. The study team member started the recording, left the room, and returned to the nursing workroom. At the conclusion of the appointment, the study team member retrieved the video recorder from the exam room and stopped the recording.

Any draft surveys or interviews were administered immediately following the encounter. Due to the COVID-19 pandemic, some study encounters were virtual telemedicine visits. In these cases, recruitment, consent, and data collection procedures were adapted to the virtual environment but otherwise remained similar to in-person processes.

Comparators

Comparator 1: coded video observations of patient-clinician encounters.

Video-recorded patient-clinician encounters were viewed by the research team, transcribed, and imported to an Excel file. Consecutive transcribed speech turns of the patient, the clinician, or the guest that discussed a single primary topic or issue that was conceivably desirable or undesirable in some way related to SCS were grouped into “preference blocks”. Preference blocks ranged in length from a single patient statement or question to multiple contributions from the participants involving a series of back-and-forth verbal exchanges. We set a low threshold to identify a preference block given that preferences are often expressed implicitly, indirectly, or ambiguously in patient-clinician exchanges. 17

In addition, we used a framework based on Kenneth Burke’s Pentad of Motives 18 , 19 to classify each preference block according to patient motivations for taking action (in this case, proceeding with SCS; see Supplementary File 1 ). The five Burkean categories were used in this study to distinguish different aspects of SCS that participants indicate may be desirable or undesirable. “Act” refers to qualities of the act of acquiring, utilizing, and living with SCS treatment or other acts involved in pain management (eg, “Is [the SCS implantation] like a laparoscopic kinda thing up there, or is it more of like the opening, like with my lower back?”; “[The SCS device] is waterproof, so I’d be able to shower for that, too?”). “Agent” refers to aspects of a person or group of people, such as the opinion of a clinician or word-of-mouth knowledge of SCS from friends, family members, or acquaintances (eg, Clinician: “One of the things they sent you to talk to us about was something called a spinal cord stimulator. Did they talk to you about that at all?” Patient: “No, but I know quite a few people that’s had it done”. Clinician: “How have they done with it?” Patient: “They swear by it”). “Agency” refers to attributes of the SCS device itself, including its size, frequency of charging, likelihood of malfunction, and other characteristics (eg, “I worry about when you’re laying down on [the SCS device], if there’s a wire sticking out getting snagged…”). “Scene” refers to contextual factors that make SCS more or less desirable, including the patients’ medical history, current level of pain, and experience with other pain management approaches (eg, “[I’ve] tried Tramadol. I didn’t really like it. It just made me go to sleep”). Finally, “purpose” is used when a participant attributes desirability to the results that SCS may achieve or a favorable alignment with patient goals or values (eg, “My goal would be to get off some of these pain meds. I just do not like that”). Expressions of preference may be attributed to be arising from, or be because of, one of these categories.

Comparator 2: Coded Post-Encounter Interviews

For post-encounter patient interviews, we iteratively developed a semi-structured interview guide that was informed by observing five encounters and debriefing subsequently with their participants ( Supplementary File 2 ). The guide included topics observed in the encounters that were judged by the researchers to be relevant to decision-making. Using the finalized interview guide, interviews were conducted, audio or video recorded, transcribed, and grouped into preference blocks using the same process as for videos. Coding of preference blocks in interviews was performed using the same categories as for the video transcripts, with the exception of the domain “expression format” which was not pertinent to interviews ( Table 2 ).

Comparator 3: Coded Post-Encounter Survey Results

We produced the survey instrument using the same iterative method used to develop the interview guide ( Supplementary File 3 ). We used Likert-type scales for the responder to characterize the importance of each topic and the degree of effect it may have on their decision-making. Each survey question was intended to capture a distinct aspect of SCS preferences, a design choice that resulted in each survey question and its answer counting as a preference block. While this structure resulted in a narrow, pre-determined number of preference blocks obtained from a given survey (unlike the coded videos or interviews, in which the absolute number of preference block could be highly variable), and our goal was to leverage the advantages of a survey by capturing a wide array of preferences efficiently. These preference blocks were analyzed using the same domains as for the videos and interviews, with exception that “interaction with decision partner” and “expression format” were omitted as not pertinent to the survey format ( Table 2 ).

Comparison of DFE vs IPE MIPPs

In a 1:1 alternating order, patients were either video recorded and then surveyed or video recorded and then interviewed. The survey was fielded in Qualtrics on a research touchscreen tablet and completed independently by the patient immediately after their clinical encounter, with a study team member available in the room to address technical issues or answer clarifying questions. The interview took place in person immediately after the clinical encounter. D.G. conducted all interviews for consistency. Video and interview speech turns were blocked and coded by one of the four coders (EG, D.G., I.H., M.L) and then reviewed by the whole group in meetings. A consensus process was used to resolve disputes. A member of the coding team coded individual participants’ preference blocks and the whole coding team reviewed these codes in virtual meetings. As part of this process, themes and subthemes were inductively generated and assigned.

The MIPPs were compared on the basis of two domains: 1) distribution of preference blocks and 2) the context of these preference blocks. Specifically, for distribution of preference blocks, we evaluated the number of unique preferences identified (each theme and sub theme was treated as a preference); the proportion of preference blocks related to each theme or subtheme within each instrument; and the proportion of preference blocks indicating a preference that was important to the patient or affected their decision. In addition, because we judged that preferences that are important or affected the decision are of particular interest for users of MIPPs for regulatory purposes, we combined these two categories and characterized patient preference blocks as “clearly important or clearly affects the decision” if one or both were indicated for a given preference block. To assess the context of preferences, we examined Burkean categories and the clarity of preference expression.

All analyses were conducted at the preference block level. Descriptive statistics were computed for each variable and compared across instruments. Comparisons are reported both in aggregate for each instrument as well as for individual participants (ie, comparing results from the video and interview for the same participant).

All analyses were performed in SPSS statistical software, Version 28 (Armonk, NY: IBM Corp).

In terms of length of time needed to acquire data for each MIPP, videos had an average encounter duration of 01:03:51, followed by interviews (00:13:34) and surveys (00:04:42).

Distribution of Preferences

Overall, we identified more absolute preference blocks per encounter on average from videos (mean: 66 preference blocks; range: 33–106) than from interviews (mean: 25; range: 18–35). Since, by design, there was a direct correlation between the number of survey items and the number of preference blocks, the number of preference blocks per encounter was almost identical among survey participants (mean: 33; range: 32–34).

Nearly three-quarters of preference blocks identified in interviews (72.8%) and surveys (68.0%) were coded as expressing clearly important preferences compared to 21.6% of preference blocks in videos. Videos contained a higher proportion of preference blocks of unclear importance (77.6%) than surveys (24.9%) or interviews (23.2%).

Surveys (63.6%) and interviews (50.3%) contained a higher proportion of preference blocks coded as representing preferences that clearly affect the decision, compared to videos (16.2%). A higher proportion of preference blocks identified in videos were unclear as to whether they affected the decision (83.4%) compared to interviews (45.0%) and surveys (1.3%).

For the combined variable indicating whether a preference block was clearly important and/or clearly affected the decision, interviews (72.8%) and surveys (68.0%) contained higher proportions of preference blocks described as clearly important or affecting the decision compared to videos (27.0%). Conversely, preference blocks described as clearly not important and/or not affecting the decision comprised 26.0% of preference blocks in surveys, followed by 5.3% of preference blocks in interviews and 0.8% of those in videos.

Preference Themes

Context of Preferences

In this study, we compared three methods for identifying patient preferences (MIPPs) at the point of decision-making in the context of SCS as a treatment for chronic pain. We found that each of the MIPPs has strengths and weaknesses, making the appropriateness of their use in understanding post-market preferences for purposes dependent on what is sought to be achieved and the resources available to deploy them. Specifically, tensions exist in multiple directions between the MIPPs, including clarity of preference expression, comprehensiveness and nuance of preferences, certainty of preference, and effort required to employ the method.

Surveys were by far the least time demanding for the researcher both to administer and interpret and yielded clear expression of the importance of a preference to the patient (68% of preference blocks were clearly important and/or clearly affected the decision), but the scope of preference information was constrained to topics explicitly included in the survey. This limitation is reflected in the relatively small range of preference blocks identified among participants receiving surveys (32 to 34) and the low number of themes identified (19) compared to the other methods. Therefore, use of surveys may be inappropriate as an exploratory method for preference assessment in cases where little is known about patient-important attributes or risks of the drug or device. However, these features make surveys helpful in confirming the presence and importance of a limited hypothesized set of patient preferences.

Interviews, while relatively brief to administer, were significantly more time-consuming than surveys to interpret through development and application of the coding scheme. From an average interview duration of 13.5 minutes, they yielded the lowest average number of preference blocks per interview (25) of any instrument but with a similar proportion of preference blocks (73%) identified as clearly important and/or clearly affected the decision as surveys (68%). However, across all participants, interviews were able to garner a broader set of preference themes (39) than surveys (19).

Video-recorded clinical encounters were the most time-consuming for the researcher, given the time required to watch the videos and code the transcripts, as well as the high number of preference blocks per video (mean = 66 blocks) compared to interviews (25) or surveys (33). However, collection of video data did not require the patient to set aside time after a lengthy encounter to complete a survey or participate in an interview. Although videos yielded the highest mean number of preference blocks, the proportion of those blocks expressing preferences that were clearly important and/or clearly affected the decision was by far the lowest among the three MIPPs (27% compared to 73% of interviews and 68% of surveys). In terms of specific preference topics discussed, videos produced a similar number of distinct themes (38) to interviews (39), but differences were apparent at the more nuanced level of sub-themes (75 in videos vs 54 in interviews).

Clarity of preference expression complicated interpretation of the video data. The existence and importance of a preference is easy to establish when a patient makes a clear expression of will (for example, “I don’t like this” or “I’d prefer that”). However, such expressions were rare within the encounter interactions, arising only 48 times within 858 total preference blocks identified across all videos. This is in stark contrast with the survey, and to a lesser extent, the interview, in which patients were asked directly about their preferences regarding particular topics and their importance. Clinicians seldom asked patients to express preferences in this way within consultations.

A further challenge in deriving preferences directly from encounters is the presence of potentially important preferences in conversations that were not clearly expressed because they could quickly be dispensed with as of no concern. For example, if a patient asked if they could have post-implant imaging performed close to home at their local clinic and the clinician responded affirmatively, then the issue may be quickly dropped without any indication of if that fact was important to the patient—which it may have been. Because these preferences are of no concern, they may not emerge spontaneously in interviews. The broad latitude given in identifying preference blocks means that instances such as the imaging example above were identified as suggesting a potential preference, even if the significance of this preference could not be ascertained.

A further sign of the relative ambiguity of preference in video encounters is that for each participant in general, videos yielded many more preference blocks than surveys, while surveys yielded more preference blocks (equivalent to survey item responses) that were marked as clearly important or clearly affecting the decision than video. However, of the three MIPPs, videos yielded greater nuance at the sub-theme level, especially when compared with surveys. In addition, the minimal overlap of themes between MIPPs may reflect an interaction between the method of preference identification (video, interview, survey) and the Burkean category a potential preference originated from or was oriented towards. For example, scene-related preferences (and their corresponding themes) were more prevalent in videos than in interviews or surveys, likely reflecting a focus on medical history in encounters.

Differences in Burkean attribution of preference towards aspects of the SCS therapy were also seen between MIPPs. For example, 65% of video preference blocks indicated desirability in terms of qualities of the “scene”, in contrast to interviews (25%) and surveys (6%). Conversely, attribution to “purpose” was lowest in videos (4%) with interviews (25%) and surveys (40%). This may reflect the fact that in discussions between patients and clinicians, the purpose of an intervention is often implied in descriptions of scene. For example, when a patient expresses difficulty in walking for extended periods of time, a purpose of SCS to enhance mobility is implied. Notably, “act” was the first or second most prevalent Burkean category across all MIPPs, while “agency” was consistently at or next to the bottom. This suggests that the practicalities involved (“act”) of pursuing, undergoing, and living with SCS were of greater concern than direct attributes of the device (“agency”).

The most appropriate MIPP for identifying patient preferences is dependent on the purpose for which it is to be used. If researchers or other stakeholders wish to understand with greater nuance the breadth or universe of potential preferences, collecting data directly from the encounter (in our case, through video-recording) might be helpful. On the other hand, for confirming or testing the relative importance of known or assumed topics, surveys provide an efficient means of doing so. Finally, a middle-ground approach is through interviewing those wishing to explore a known set of preference topics while leaving room for some expansion. Of course, MIPPs may be combined.

In the instrument development phase, we found it necessary to draw on insights garnered through reviewing videos of encounters to create a survey and interview guide that was appropriate for the particularities of SCS therapy and decision-making. In the absence of generally applicable instruments, we recommend a similar process of empirically informed instrument development for other post-market interventions. While working with video data is very time-intensive, the video data drawn upon in designing surveys or interviews are itself useful for triangulating results when surveys or interviews are later used.

Finally, our approach could be used in future research to explore the impact of gender on preference expression in medical decision-making contexts. Observations from our data indicated differences in some areas between men and women in articulating preferences, though our small sample prevents us from drawing firm conclusions. For example, men were more likely than women to situate preferences in the context of information about past therapies and the characteristics of their current pain (eg, duration, intensity), while women were more likely to discuss the influence of family, friends, and health care providers on their preferences and treatment decision-making. However, the prevalence of specific themes and topics identified within each instrument were similar for men and women. While there is a large research literature detailing the influence of gender on pain expression, communication styles, and medical decision-making, respectively, little is known specifically concerning differences in medical treatment preference expression or measurement in the context of chronic pain management. Widely used frameworks suggest that socialized differences in communication styles lead men to rely on “report talk” (ie, conveying objective or instrumental information) and assertive language, while women tend to emphasize relationships and rapport-building. 20 , 21 Similarly, in health care settings, women have been found to place a stronger focus than men on social and contextual factors when reporting symptoms and describing past treatment experiences. 22 Given well-documented treatment disparities that result in inadequate pain control among women, 23 , 24 future research should explore how differences in preference expression by gender impact treatment decisions.

The key strength of this study lies in its comprehensive exploration of methods to identify patient preferences at the point of decision-making. However, there are important limitations. The small sample size (n = 13) reduces the reliability, validity, and ability to draw definitive conclusions from these data. Nevertheless, we hope that future studies will build on and extend this exploratory work with larger samples. Next, the instruments used to assess patient preferences were developed for this study by the research team and have not been formally evaluated for their measurement properties or soundness. However, in the absence of any gold standard methods for identifying patient preference, steps were taken to ensure a high level of quality for all methods used. For example, the survey was reviewed at multiple points by an expert in survey research methods and pilot tested extensively with patients before arriving at the final version.

Another limitation of this study lies in the nature of patient decision-making, which is a complex, non-linear process. Although we designed data collection procedures to capture patient preferences as close as possible to the point of actual decision-making (ie, during or immediately following the encounter), we recognize that our approach may exclude important aspects of patient thought processes. Future work could follow patients longitudinally and explore how preferences evolve throughout the process of choosing to adopt or not adopt a medical intervention, it could also illuminate if and how preferences change between the consultation and when patients call back to confirm their decision or schedule SCS procedures.

In addition, the generalizability of our findings beyond the specific context and patient population of this study, which was limited to patients seeking a pain medicine consultation at Mayo Clinic in Minnesota, is unclear. The application of these methods for post-market preference exploration or elicitation in other clinical contexts will likely uncover additional insights into the relative advantages or limitations of each method. For example, although we found all three methods of data collection (ie, video recording the encounter, post-encounter interviews, and post-encounter surveys) to be both feasible and acceptable to most clinicians and patients, studies employing these methods in other settings might encounter challenges to their use (for example, when the health condition of interest is particularly sensitive or stigmatized). In addition, the clinicians involved in our study practice in an academic medical setting and may have the benefit of more exposure to or training in shared decision-making and preference elicitation than clinicians in other settings, which may further limit the generalizability of the MIPPs described in this study.

Patient preferences regarding regulated medical products are an important consideration in the regulatory decision and ongoing assessment of interventions post-market. Relevant preferences may arise when patients gather information to determine what and if options may be available, at the time that preferences play an active role in decision-making, and after patients have experience living with an intervention. Relatively little attention has been given to the expression, function, and use of preferences at the point of decision-making even though these preferences greatly influence intervention adoption and indicate important patient-centered criteria for judging the usefulness, desirability, and ongoing sustainability of interventions. This study has both opened a view into the complexities of preference and the strengths and limitations of methods for identifying preferences at the point of decision-making. Future work may build on this to illuminate how patients and clinicians use or operationalize preferences to make decisions with important implications for shared decision-making and patient-centered care.

Acknowledgments

Montserrat León-García is a doctoral candidate for the Ph.D. in Methodology of Biomedical Research and Public Health, Universitat Autònoma de Barcelona, Barcelona, Spain; and this study will be part of her thesis dissertation.

This publication is supported by the Office of Women’s Health (OWH), Food and Drug Administration (FDA) of the US Department of Health and Human Services (HHS) as part of a financial assistance award (U01FD005938) totaling $459,092 with 100% funded by OWH/FDA/HHS. The contents are those of the authors and do not necessarily represent the official views of, nor an endorsement, by OWH/FDA/HHS, or the US Government.

Victor Montori works at Mayo Clinic’s Knowledge and Encounter Research Unit where we design, test, implement and disseminate shared decision-making interventions. We release free to use and derive no income from their use. The authors report no other conflicts of interest in this work.

1. U.S. Department of Health and Human Services Food and Drug Administration. Patient preference information—voluntary submission, review in premarket approval applications, humanitarian device exemption applications, and de novo requests, and inclusion in decision summaries and device labeling. U.S. Department of Health and Human Services Food and Drug Administration; 2016.

2. National Academies of Sciences, Engineering, and Medicine. Pain Management and the Opioid Epidemic: Balancing Societal and Individual Benefits and Risks of Prescription Opioid Use . National Academies of Sciences, Engineering, and Medicine; 2017.

3. Hampton SB, Cavalier J, Langford R. The influence of race and gender on pain management: a systematic literature review. Pain Manag Nurs . 2015;16(6):968–977. doi:10.1016/j.pmn.2015.06.009

4. Hunter NL, O’Callaghan KM, Califf RM. Engaging patients across the spectrum of medical product development: view from the US Food and Drug Administration. JAMA . 2015;314(23):2499–2500. doi:10.1001/jama.2015.15818

5. Benz HL, Lee T-HJ, Tsai J-H, et al. Advancing the use of patient preference information as scientific evidence in medical product evaluation: a summary report of the patient preference workshop. Patient . 2019;12(6):553–557. doi:10.1007/s40271-019-00396-5

6. Ho M, Saha A, McCleary KK, et al. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health . 2016;19(6):746–750. doi:10.1016/j.jval.2016.02.019

7. Montori VM, Brito JP, Murad MH. The optimal practice of evidence-based medicine: incorporating patient preferences in practice guidelines. JAMA . 2013;310(23):2503–2504. doi:10.1001/jama.2013.281422

8. van Overbeeke E, Whichello C, Janssens R, et al. Factors and situations influencing the value of patient preference studies along the medical product lifecycle: a literature review. Drug Discov Today . 2019;24(1):57–68. doi:10.1016/j.drudis.2018.09.015

9. Soekhai V, Whichello C, Levitan B, et al. Methods for exploring and eliciting patient preferences in the medical product lifecycle: a literature review. Drug Discov Today . 2019;24(7):1324–1331. doi:10.1016/j.drudis.2019.05.001

10. List of Patient Preference-Sensitive Priority Areas. U.S. Food and Drug Administration (FDA), Center for Devices and Radiological Health (CDRH) Patient Science and Engagement Program; 2020. Available from: https://www.fda.gov/about-fda/cdrh-patient-science-and-engagement-program/list-patient-preference-sensitive-priority-areas . Accessed September 11 , 2022 .

11. Sdrulla AD, Guan Y, Raja SN. Spinal cord stimulation: clinical efficacy and potential mechanisms. Pain Pract . 2018;18(8):1048–1067. doi:10.1111/papr.12692

12. Gildenberg PL. History of Electrical Neuromodulation for Chronic Pain . Malden, USA: Blackwell Publishing Inc; 2006:S7–S13.

13. Clingan JA, Patel A, Maher DP. Survey of spinal cord stimulation hardware currently available for the treatment of chronic pain in the United States. Front Pain Res . 2020;1:572907. doi:10.3389/fpain.2020.572907

14. Deer TR, Mekhail N, Provenzano D, et al. The appropriate use of neurostimulation of the spinal cord and peripheral nervous system for the treatment of chronic pain and ischemic diseases: the neuromodulation appropriateness consensus committee. Neuromodulation . 2014;17(6):515–550. doi:10.1111/ner.12208

15. Bendel MA, O’Brien T, Hoelzer BC, et al. Spinal cord stimulator related infections: findings from a multicenter retrospective analysis of 2737 implants. Neuromodulation . 2017;20(6):553–557. doi:10.1111/ner.12636

16. Celestin J, Edwards RR, Jamison RN. Pretreatment psychosocial variables as predictors of outcomes following lumbar surgery and spinal cord stimulation: a systematic review and literature synthesis. Pain Med . 2009;10(4):639–653. doi:10.1111/j.1526-4637.2009.00632.x

17. Stevenson FA, Barry CA, Britten N, Barber N, Bradley CP. Doctor–patient communication about drugs: the evidence for shared decision making. Soc Sci Med . 2000;50(6):829–840. doi:10.1016/s0277-9536(99)00376-7

18. Burke K. A Grammar of Motives . Prentice Hall; 1945.

19. Kunneman M, Hargraves IG, Sivly AL, et al. Co-creating sensible care plans using shared decision making: patients’ reflections and observations of encounters. Patient Educ Couns . 2022;105(6):1539–1544. doi:10.1016/j.pec.2021.10.003

20. Tannen D. You Just Don’t Understand: Women and Men in Conversation . Virago London: William Morrow; 1991.

21. Leaper C, Ayres MM. A meta-analytic review of gender variations in adults’ language use: talkativeness, affiliative speech, and assertive speech. Pers Soc Psychol Rev . 2007;11(4):328–363. doi:10.1177/1088868307302221

22. Meeuwesen L, Schaap C, van der Staak C. Verbal analysis of doctor-patient communication. Soc Sci Med . 1991;32(10):1143–1150. doi:10.1016/0277-9536(91)90091-p

23. Tait RC, Chibnall JT, Kalauokalani D. Provider judgments of patients in pain: seeking symptom certainty. Pain Med . 2009;10(1):11–34. doi:10.1111/j.1526-4637.2008.00527.x

24. Hoffmann DE, Tarzian AJ. The girl who cried pain: a bias against women in the treatment of pain. J Law Med Ethics . 2001;29(1):13–27. doi:10.1111/j.1748-720x.2001.tb00037.x

Contact Us   •   Privacy Policy   •   Associations & Partners   •   Testimonials   •   Terms & Conditions   •   Recommend this site •   Cookies •   Top

Contact Us   •   Privacy Policy