research study about child and adolescent development

Child and Adolescent Development

• First Online: 28 January 2017

Cite this chapter

• Rosalyn H. Shute 3 &
• John D. Hogan 4  

11k Accesses

For school psychologists, understanding how children and adolescents develop and learn forms a backdrop to their everyday work, but the many new ‘facts’ shown by empirical studies can be difficult to absorb; nor do they make sense unless brought together within theoretical frameworks that help to guide practice. In this chapter, we explore the idea that child and adolescent development is a moveable feast, across both time and place. This is aimed at providing a helpful perspective for considering the many texts and papers that do focus on ‘facts’. We outline how our understanding of children’s development has evolved as various schools of thought have emerged. While many of the traditional theories continue to provide useful educational, remedial and therapeutic frameworks, there is also a need to take a more critical approach that supports multiple interpretations of human activity and development. With this in mind, we re-visit the idea of norms and milestones, consider the importance of context, reflect on some implications of psychology’s current biological zeitgeist and note a growing movement promoting the idea that we should be listening more seriously to children’s own voices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Achenbach, T. M. (2009). The Achenbach system of empirically based assessment (ASEBA): Development, findings, theory, and applications . Burlington, VT: Research Center for Children, Youth and Families, University of Vermont.

Google Scholar  

Ainsworth, M. D., & Bell, S. M. (1970). Attachment, exploration, and separation: Illustrated by the behavior of one-year-olds in a strange situation. Child Development, 41 , 49–67.

Article   PubMed   Google Scholar  

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, (DSM-5) (5th ed.). Arlington, VA: American Psychiatric Association.

Book   Google Scholar  

Ames, L. B. (1996). Louise bates Ames. In D. N. Thompson & J. D. Hogan (Eds.), A history of developmental psychology in autobiography (pp. 1–23). Boulder, CO: Westview Press.

Anderson, V., Spencer-Smith, M., & Wood, A. (2011). Do children really recover better? Neurobehavioural plasticity after early brain insult. Brain, 134 , 2197–2221. doi: 10.1093/brain/awr103 .

Annan, J., & Priestley, A. (2011). A contemporary story of school psychology. School Psychology International, 33 (3), 325–344. doi: 10.1177/0143034311412845 .

Article   Google Scholar  

Aries, P. (1962). Centuries of childhood: A social history of family life (R. Baldick, Trans.). New York: Random House.

Arnett, J. J. (2007). Emerging adulthood: What is it, and what is it good for? Child Development Perspectives, 1 (2), 68–73. doi: 10.1111/j.1750-8606.2007.00016.x .

Arnett, J. J. (2013). The evidence for Generation We and against Generation Me. Emerging adulthood, 1 , 5–10. doi: 10.1177/2167696812466842 .

Bandura, A., & Walters, R. H. (1963). Social learning and personality development . New York: Holt Rinehart and Winston.

Bjorklund, D. E., & Pellegrini, A. D. (2002). The origins of human nature: Evolutionary developmental psychology . Washington, DC: APA.

Bowlby, J. (1969). Attachment and loss (Vol. 1). New York: Basic Books.

Boyce, W. T., & Ellis, B. J. (2005). Biological sensitivity to context: I. An evolutionary developmental theory of the origins and function of stress reactivity. Development and Psychopathology, 17 , 271–301. doi: 10.1017/S0954579405050145 .

British Psychological Society. (2011). Response to the American Psychiatric Association: DSM-5 development . Retrieved January 16, 2015, from http://www.bps.org.uk/news/british-psychological-still-has-concerns-over-dsm-v

Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design . Cambridge, MA: Harvard University Press.

Bronfenbrenner, U., & Morris, P. A. (1998). Chapter 17: The ecology of developmental processes. In R. M. Lerner (Ed.), Handbook of child psychology: Theoretical models of human development (5th ed., Vol. 1, pp. 535–584). San Francisco: Jossey-Bass.

Burman, E. (2008). Deconstructing developmental psychology . London: Routledge. (Original work published 1994)

Burns, M. K. (2011). School psychology research: Combining ecological theory and prevention science. School Psychology Review, 40 (1), 132–139.

Byers, L., Kulitja, S., Lowell, A., & Kruske, S. (2012). ‘Hear our stories’: Child-rearing practices of a remote Australian Aboriginal community. Australian Journal of Rural Health, 20 , 293–297. doi: 10.1111/j.1440-1584.2012.01317.x .

Cefai, C. (2011). Chapter 2: A framework for the promotion of social and emotional wellbeing in primary schools. In R. H. Shute, P. T. Slee, R. Murray-Harvey, & K. L. Dix (Eds.), Mental health and wellbeing: Educational perspectives (pp. 17–28). Adelaide: Shannon Research press.

Chomsky, N. (1957). Syntactic structures . The Hague/Paris: Mouton.

Copple, C., & Bredekamp, S. (Eds.). (2009). Developmentally appropriate practice in early childhood programs . Washington, DC: National Association for the Education of Young Children.

Crain, W. (2010). Theories of development: Concepts and applications (6th ed.). Upper Saddle River, NJ: Pearson.

Darwin, C. (1877). A biographical sketch of an infant. Mind, 2 , 285–294.

Dekker, S., Lee, N. C., Howard-Jones, P., & Jolles, J. (2012). Neuromyths in education: Prevalence and predictors of misconceptions among teachers. Frontiers in Psychology, 3 , Article 429. doi: 10.3389/fpsyg.2012.00429

Department of Education and Training, Victoria. (2005). Research on learning: Implications for teaching. Edited and abridged extracts . Melbourne: DET. Retrieved October 8, 2014, from www.det.vic.gov.au

Elkind, D. (2011). Developmentally appropriate practice: Curriculum and development in early education (C. Gestwicki, Guest editorial, 4th ed.). Belmont, CA: Wadsworth.

Erikson, E. (1950). Childhood and society . New York: Norton.

Freud, S. (2005). The unconscious . London, UK: Penguin.

Goodman, R., & Scott, S. (1999). Comparing the strengths and difficulties questionnaire and the child behavior checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27 , 17–24.

Gredler, M. E. (2012). Understanding Vygotsky for the classroom: Is it too late? Educational Psychology Review, 24 , 113–131. doi: 10.1007/s10648-011-9183-6 .

Greene, S. (2006). Child psychology: Taking account of children at last? The Irish Journal of Psychology, 27 (1/2), 8–15.

Guenther, J. (2015). Analysis of national test scores in very remote Australian schools: Understanding the results through a different lens. In H. Askell-Williams (Ed.), Transforming the future of learning with educational research . Hershey, PA: IGI Global.

Hall, G. S. (1883). The contents of children’s minds. Princeton Review, 11 , 249–272.

Hall, G. S. (1904). Adolescence: Its psychology and its relation to physiology, anthropology, sociology, sex, crime, religion, and education (Vol. 1–2). New York: D. Appleton.

Heary, C., & Guerin, S. (2006). Research with children in psychology: The value of a child-centred approach. The Irish Journal of Psychology, 27 (1/2), 6–7.

Hindman, H. D. (2002). Child labor: An American history . Armonk, NY: M. E. Sharpe.

Hogan, J. D. (2003). G. Stanley Hall: Educator, innovator, pioneer of developmental psychology. In G. Kimble & M. Wertheimer (Eds.), Portraits of pioneers in psychology (Vol. V, pp. 19–36). Washington, DC: American Psychological Association.

Iossifova, R., & Marmolejo-Ramos, F. (2013). When the body is time: Spatial and temporal deixis in children with visual impairments and sighted children. Research in Developmental Disabilities, 34 , 2173–2184.

Jantz, P. B., & Plotts, C. A. (2014). Integrating neuropsychology and school psychology: Potential and pitfalls. Contemporary School Psychology, 18 , 69–80. doi: 10.1007/s40688-013-0006-2 .

Jarvis, J. (2011). Chapter 20: Promoting mental health through inclusive pedagogy. In R. H. Shute, P. T. Slee, R. Murray-Harvey, & K. L. Dix (Eds.), Mental health and wellbeing: Educational perspectives (pp. 237–248). Adelaide: Shannon Research press.

Jiang, X., Kosher, H., Ben-Arieh, A., & Huebner, E. S. (2014). Children’s rights, school psychology, and well-being assessments. Social Indicators Research, 117 , 179–193. doi: 10.1007/s11205-013-0343-6 .

Jones, M. C. (1924). A laboratory study of fear: The case of Peter. Pedagogical Seminary, 31 , 308–315.

Karmiloff-Smith, A. (2012). Perspectives on the dynamic development of cognitive capacities: Insights from Williams syndrome. Current Opinion in Neurology, 25 , 106–111. doi: 10.1097/WCO.0b013e3283518130 .

Kennedy, H. (2006). An analysis of assessment and intervention frameworks in educational psychology services, in Scotland. Past, present and possible worlds. School Psychology International, 27 (5), 515–534. doi: 10.1177/0143034306073396 .

Ketelaar, T., & Ellis, B. J. (2000). Are evolutionary explanations unfalsifiable? Evolutionary psychology and the Lakatosian philosophy of science. Psychological Inquiry, 11 (1), 1–21.

Lansdown, G., Jimerson, S. R., & Shahrooz, R. (2014). Children’s rights and school psychology: Children’s right to participation. Journal of School Psychology, 35 , 3–12. doi: 10.1016/j.jsp.2013.12.006 .

Lorenz, K. (1961). King Solomon’s ring (M. K. Wilson, Trans.). London: Methuen.

Magai, C., & McFadden, S. H. (1995). The role of emotions in social and personality development: History, theory and research . New York: Plenum.

Matthews, S. H. (2007). A window on the ‘new’ sociology of childhood. Sociology Compass, 1 , 322–334. doi: 10.1111/j.1751-9020.2007.00001.x .

Moon, C., Lagerkrantz, H., & Kuhl, P. K. (2013). Language experience in utero affects vowel perception after birth: A two-country study. Acta Paediatrica, 102 , 156–160. doi: 10.1111/apa.12098 .

Article   PubMed   PubMed Central   Google Scholar  

Moriceau, S., & Sullivan, R. M. (2005). Neurobiology of infant attachment. Developmental Neurobiology, 47 (3), 230–242. doi: 10.1002/dev.20093 .

Nsamenang, A. B. (2006). Cultures in early childhood care and education . Paper commissioned for the EFA Global Monitoring Report 2007, Strong Foundations: Early Childhood Care and Education . Retrieved May 6, 2014, from http://unesdoc.unesco.org/images/0014/001474/147442e.pdf

O’Donnell, J. (1985). The origins of behaviorism. American psychology, 1870–1920 . New York: New York University Press.

Oberklaid, F. (2011). Is my child normal? Retrieved October 2, 2014, from http://www.racgp.org.au/afp/2011/september/is-my-child-normal/

Overton, W. F., & Ennis, M. D. (2006). Developmental and behavior-analytic theories: Evolving into complementarity. Human Development, 49 , 143–172. doi: 10.1159/000091893 .

Packenham, M., Shute, R. H., & Reid, R. (2004). A truncated functional behavioral assessment procedure for children with disruptive classroom behaviors. Education and Treatment of Children, 27 (1), 9–25.

Perry, B. D. (1997). Incubated in terror: Neurodevelopmental factors in the “cycle of violence”. In J. D. Osofky (Ed.), Children in a violent society . New York: Guilford.

Ritchie, J., & Ritchie, J. (1979). Growing up in Polynesia . Sydney: Allen and Unwin.

Romer, D. (2010). Adolescent risk taking, impulsivity, and brain development: Implications for prevention. Developmental Psychobiology, 52 (3), 263–276. doi: 10.1002/dev.20442 .

PubMed   PubMed Central   Google Scholar  

Rousseau, J. J. (1914). Emile on education (Foxley, Trans.). New York: Basic Books. (Original work published 1762)

Rutherford, A. (2009). Beyond the box: B. F. Skinner’s technology of behaviour from laboratory to life, 1950s to 1970s . Toronto: University of Toronto Press.

Sattler, J. M. (2008). Assessment of children: Cognitive foundations (5th ed.). La Mesa, CA: Sattler.

Seligman, M. (2002). Authentic happiness . New York: Free Press.

Shier, H. (2001). Pathways to participation: Openings, opportunities and obligations. Children in Society, 15 (2), 107–117.

Shute, R. H. (2015). Transforming the future of learning: People, positivity and pluralism (and even the planet). In H. Askell-Williams (Ed.), Transforming the future of learning with educational research . Hershey, PA: IGI Global.

Shute, R. H. (2016). ‘Promotion with parents is challenging’. The role of teacher communication skills and parent-teacher partnerships in school-based mental health initiatives. In R. H. Shute & P. T. Slee (Eds.), Mental health through schools: The way forward . Hove: Routledge.

Shute, R. H., & Slee, P. T. (2015). Child development: Theories and critical perspectives . Hove: Routledge.

Siegel, A. W., & White, S. H. (1982). The child study movement: Early growth and development of the symbolized child. In H. W. Reese (Ed.), Advances in child development and behavior (Vol. 17). New York: Academic.

Skinner, B. F. (1957). Verbal behavior . Acton, MA: Copley Publishing.

Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland adaptive behaviour scales (Vineland II) (2nd ed.). Melbourne, VIC, Australia: Pearson Australia.

Spears, B., Slee, P., Campbell, M., & Cross, D. (2011). Educational change and youth voice: Informing school action on cyberbullying. Centre for Strategic Education, Seminar Series Paper 208 . Melbourne, VIC, Australia: CSE.

Strauss, E., Sherman, E. S., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms and commentary (3rd ed., pp. 46–52). Retrieved October 8, 2014, from http://scholar.google.com.au/scholar_url?hl=en&q=http://www.iapsych.com/articles/strauss2006.pdf&sa=X&scisig=AAGBfm0m5ZgjHj9p9Ew9NuSmlUnv9CWiUA&oi=scholarr&ei=ZQUuVKLUGOKsjAKiuoDoBw&ved=0CCQQgAMoADAA

Teo, T. (1997). Developmental psychology and the relevance of a critical metatheoretical reflection. Human Development, 40 , 195–210.

Thelen, E., & Adolph, K. E. (1992). Arnold L. Gesell: The paradox of nature and nurture. Developmental Psychology, 28 , 368–380.

Thelen, E., & Smith, L. B. (1994). A dynamic systems approach to the development of cognition and action . Cambridge, MA: MIT Press.

Timimi, S., Moncrieff, J., Jureidini, J., Leo, J., Cohen, D., Whitfield, C., … White, R. (2004). A critique of the international consensus statement on ADHD. Clinical Child and Family Psychology Review, 7 , 59–63.

United Nations Convention on the Rights of the Child. (1989). Geneva, Switzerland: United Nations. Accessed October 31, 2014, from http://www.ohchr.org/EN/ProfessionalInterest/Pages/CRC.aspx

Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20 , 158–177.

Watson, J. B., & Rayner, R. R. (1920). Conditioned emotional reactions. Journal of Experimental Psychology, 3 , 1–14.

Weaver, I. C., et al. (2004). Epigenetic programming by maternal behavior. Nature Neuroscience, 7 , 847–854.

Wood, D. (1998). How children think and learn . Oxford: Blackwell. (Original work published 1988)

Yell, M. (1998). The law and special education . Upper Saddle River, NJ: Prentice-Hall.

Additional Sources

Fancher, R. E., & Rutherford, A. (2012). Pioneers of psychology (4th ed.). New York: Norton.

Kessen, W. (1979). The American child and other cultural inventions. American Psychologist, 34 , 815–820.

Parke, R. D., Ornstein, P. A., Rieser, J. J., & Zahn-Waxler, C. (Eds.). (1994). A century of developmental psychology . Washington, DC: American Psychological Association.

Thompson, D. N., Hogan, J. D., & Clark, P. M. (2012). Developmental psychology in historical perspective . Chichester: Wiley.

Download references

Author information

Authors and affiliations.

School of Psychology, Flinders University, GPO Box 2100, Adelaide, SA, 5001, Australia

Rosalyn H. Shute

St John’s University, Jamaica, NY, USA

John D. Hogan

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rosalyn H. Shute .

Editor information

Editors and affiliations.

Department of Psychological Sciences, Swinburne University of Technology, Hawthorn, Victoria, Australia

Monica Thielking

St. John’s University, Jamaica, New York, USA

Mark D. Terjesen

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Shute, R.H., Hogan, J.D. (2017). Child and Adolescent Development. In: Thielking, M., Terjesen, M. (eds) Handbook of Australian School Psychology. Springer, Cham. https://doi.org/10.1007/978-3-319-45166-4_4

Download citation

DOI : https://doi.org/10.1007/978-3-319-45166-4_4

Published : 28 January 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-45164-0

Online ISBN : 978-3-319-45166-4

eBook Packages : Behavioral Science and Psychology Behavioral Science and Psychology (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

•   [email protected]

Psychology Research Guide

Child & adolescent development.

“Child development”, or “child and adolescent development” refers to the process of growth and maturation of the human individual from conception to adulthood. The term “adolescence” has particular connotations in particular cultural and social contexts. Child & Adolescent Psychology focuses on understanding the physical, social, psychological, and cognitive needs of young human beings. You can read more about the focus of Child & Adolescent Development on the American Psychological Association's Society of Clinical Child and Adolescent Psychology website This link opens in a new window . To find ideas for paper/research topics within child & adolescent development, visit these sites:

APA Psychology Topics This link opens in a new window (Try Bullying; Children; Education; Kids & the Media; Learning & Memory; Parenting; Teens)

Child & Adolescent Development Databases

Research in child & adolescent psychology utilizes core psychology resources, as well as resources in child & family development and sociology. You may find it helpful to search the following databases for your child & adolescent development topics or research questions, in addition to the core resources listed on the home page.

Child & Adolescent Development Subject Headings

You may find it helpful to take advantage of predefined subjects or subject headings in Shapiro Databases. These subjects are applied to articles and books by expert catalogers to help you find materials on your topic.

• Learn more about Subject Searching

Consider using databases to perform subject searches, or incorporating words from applicable subjects into your keyword searches. Here are some social psychology subjects to consider:

• adopted children
• Attachment Theory
• child abuse
• child behavior
• children of alcoholics
• cognitive development
• developmental stages
• early childhood development
• emotional development
• family relations
• middle school/junior high school/high school students
• parent child relations
• peer pressure
• personality

Child & Adolescent Development Organization Websites

• American Academy of Child and Adolescent Psychiatry (AACAP) This link opens in a new window A national professional medical association dedicated to treating and improving the quality of life for children, adolescents, and families affected by mental, behavioral, or developmental disorders.
• Child & Adolescent Development course module (UNHCR) This link opens in a new window This Resource Pack published by the United Nations High Commissioner on Refugees' Action for the Rights of Children (ARC) is a training module for those working with children and teen refugees. It covers major areas of child development acknowledging that " the concept of childhood is understood differently in different cultural and social contexts."
• Eunice Kennedy Shriver National Institute of Child Health & Human Development (NICHD) This link opens in a new window NICHD’s mission is to lead research and training to understand human development, improve reproductive health, enhance the lives of children and adolescents, and optimize abilities for all.
• Society of Clinical Child & Adolescent Psychology (APA Division) This link opens in a new window The Society of Clinical Child and Adolescent Psychology is Division 53 of the American Psychological Association. Its purpose is to encourage the development and advancement of clinical child and adolescent psychology through integration of its scientific and professional aspects.
• Child Welfare Information Gateway - Understanding Adolescent Development This link opens in a new window United States Health & Human Services Children's Bureau Child Welfare Information Gateway has extensive resources on child & adolescent development. This link leads to their "Understanding Adolescent Development" resources page.
• << Previous: Applied Psychology
• Next: Forensic Psychology >>

• Search Menu
• Sign in through your institution
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Evolution
• Language Reference
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Religion
• Music and Media
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Strategy
• Business Ethics
• Business History
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Systems
• Economic History
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Political Theory
• Politics and Law
• Politics of Development
• Public Administration
• Public Policy
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

8 Research Methodology in Clinical Child and Adolescent Psychology

Jonathan S. Comer, Florida International University

Laura J. Bry Department of Psychology Florida International University Miami, FL, USA

• Published: 07 November 2018
• Cite Icon Cite
• Permissions Icon Permissions

To continue to move the field of clinical child and adolescent psychology forward, researchers must systematically rely on research strategies that achieve favorable balances between scientific rigor and clinical relevance. This chapter presents an overview of modern methods and considerations that maximize both rigor and relevance in the evaluation of child and adolescent treatments. This research methodology chapter is organized around the four stages of a clinical trial: (a) planning a clinical trial; (b) conducting a clinical trial; (c) analyzing trial outcomes, and (d) reporting results. Sample selection, random assignment, control condition selection, treatment integrity, missing data, clinical significance, treatment mechanisms, and consolidated standards for communicating study findings to the scientific community are addressed. Collectively, the methods and design considerations detail modern research strategies for the continually evolving science of clinical child and adolescent psychology.

Children’s mental health problems impose a staggering public health burden. For example, roughly 40% of adolescents in the United States have reportedly suffered from a mental disorder in the past year (Kessler, Avenevoli, Costello, et al., 2012), and these disorders are associated with enormous individual, family, and societal costs. Youth mental disorders are associated with complex comorbid presentations (Kessler, Avenevoli, McLaughlin, et al., 2012); elevated substance use (Kendall & Kessler, 2002; Wu, Goodwin, Comer, Hoven, & Cohen, 2010 ); and medical comorbidities ( Merikangas et al., 2015 ). When left untreated, they persist into adulthood, during which time they are associated with family dysfunction; disability in major life roles ( Merikangas et al., 2007 ); poorer educational attainment ( Breslau, Lane, Sampson, & Kessler, 2008 ); criminality; suicide ( Nock & Kessler, 2006 ); and overall reduced health-related quality of life ( Comer et al., 2011 ).

Despite these daunting statistics, recent years have witnessed very promising advances in the development and evaluation of evidence-based interventions for the broad range of children’s mental health problems ( Kendall, 2012 ; Ollendick & King, 2004 ). Evaluations of therapeutic efficacy and effectiveness have evolved from a historical reliance on simply professional introspection and retrospective case histories to modern reliance on complex multimethod experimental investigations and well-controlled randomized trials across well-defined and increasingly generalizable samples.

However, much still remains to be learned about the treatment of child and adolescent mental health problems, and this should not be surprising. After all, whereas many sciences have been progressing for centuries (e.g., physics, chemistry, biology), it has been only relatively recently that empiricism and the scientific method have been applied systematically to clinical child and adolescent psychology ( Comer & Kendall, 2013b ). At this relatively early stage in the science of clinical child and adolescent psychology, most of the research is still ahead of us. As we face the challenge of optimally informing best practices in youth mental health care with data, the prepared investigator must be familiar with the portfolio of modern research strategies for conducting clinical evaluations of treatment methods—a set of “directions” so to speak for getting from “here” to “there” (see Comer & Kendall, 2013b ). Just as with any travel directions, where there may be many acceptable ways to get to the same destination (e.g., the scenic way, the quick way, the cheap way), for each testable question in clinical child and adolescent psychology, there are many methods that can be used to reveal meaningful information, each with limitations and strengths.

To continue to move the field of clinical child and adolescent psychology forward, investigators must systematically rely on research strategy “routes” that achieve favorable balances between scientific rigor and clinical relevance ( Comer & Kendall, 2013b ). This necessitates careful considerations regarding the trade-offs between internal validity (which is typically linked with rigor ) and external validity (which is typically linked with relevance ). Internal validity pertains to the extent to which the independent variable, rather than an extraneous influence, accounts for variance in the dependent variable. The more rigorous and tightly controlled a study design, the more persuasively the study is able to rule out the possibility that variables beyond the independent variable might be accounting for variance in the dependent variable. External validity, on the other hand, pertains to the extent to which study results generalize to people, settings, times, measures, and characteristics other than those included in a particular study. Accordingly, design decisions focusing on internal validity and aiming to improve interpretative conclusions typically have the consequence of reducing the external validity and generalizability of findings to broadly relevant settings and vice versa. With this in mind, we present an overview of modern methods and considerations that maximize both rigor and relevance in the evaluation of child and adolescent treatments.

Planning a Clinical Trial

When planning a clinical evaluation to examine the efficacy or effectiveness of treatment for child and adolescent mental health problems, six sets of considerations are essential: (a) design considerations; (b) control condition considerations; (c) independent variable considerations; (d) dependent variable considerations; (e) assessment point considerations; and (f) sample and setting considerations.

Design Considerations in Clinical Child and Adolescent Psychology

Broadly speaking, the development and evaluation of novel therapeutic interventions occur through a sequence of three progressive stages ( Rounsaville, Carroll, & Onken, 2001 ). Stage 1 encapsulates an iterative development process, combining previous research, clinical expertise, and consultation with experts. An experimental intervention is tested, preliminarily, on a small number of subjects of the population for which the treatment is intended. This idiographic approach provides initial evidence to show the relationship between treatment and symptoms on an individual basis. Stage 1 designs also address issues related to intervention feasibility and acceptability and provide opportunities for intervention refinement or tweaking before progressing to Stage 2, large-scale evaluation. In Stage 2, tightly controlled, systematic, and rigorous evaluations with high internal validity establish broad efficacy of the intervention by looking at nomothetic patterns. Stage 3 designs evaluate intervention effectiveness, prioritizing external validity, generalizability to a wider range of patients, and transportability across clinical settings and practitioners.

Depending on research goals and the stage of the intervention, a range of study designs is available to evaluate an experimental treatment. Selecting a study design involves finding a balance between one’s research question and goals and the limitations associated with each design option. We now turn our attention to leading design options available to investigators, including single-case and multiple-baseline experimental designs, the randomized controlled trial (RCT), and sequenced treatment designs.

Single-case and multiple-baseline designs.

Systematic research designs encompassing a single individual or a small sample of subjects are useful for informing our understanding of individual behavior change and revealing a signal of “how,” “why,” and “when” treatment-related changes may occur. This idiographic portrait of the relationship between an intervention and symptoms makes such designs particularly useful during Stages 1 and 3 of treatment evaluation ( Barlow & Nock, 2009 ). Indeed, these designs have played a prominent role in developing clinical guidelines and best practices, underscoring their importance in evidence-based practice ( American Psychological Association, 2002 ). Understanding treatment-related change at the individual level provides an opportunity for intervention refinement prior to initiating costly large-scale evaluations. After large-scale clinical evaluations have been conducted, single-case designs can again be useful to evaluate the intervention’s applicability to individuals in new settings or with different symptom profiles or when implemented by clinicians of different training backgrounds. Single-case and multiple-baseline trials are also relatively cost-efficient, making them valued designs in the context of limited funding.

Generally, single-case experimental designs employ a systematic, repeated-measures approach wherein data related to a specific dependent variable (i.e., clinical target) are collected across a baseline and treatment phase. Researchers must balance a data collection schedule that is frequent enough to provide clues about when treatment-related changes occur, while avoiding potential subject response fatigue ( Barlow, Nock, & Hersen, 2009 ; Gallo, Comer, & Barlow, 2013 ; Kazdin, 2001 ). Typically, the baseline phase is referred to as the “A” phase. Dependent variable data are collected across a baseline phase (rather than at a single time point) to document the stability of the target behavior as it occurs naturally. These data are then compared to observations collected during the treatment phase (or “B” phase) of the design. Capturing stability of the behavior during the baseline period is critical for attributing changes to the treatment, rather than attributing changes to a natural cycle of fluctuations of that behavior ( Gallo et al., 2013 ).

The most traditional representation of the single-case experimental design is the A–B design , in which a target behavior is measured repeatedly across both a baseline A phase and a treatment B phase. This design allows the researcher to capture data on a naturally occurring, preintervention behavior, which is then directly compared to observations of the behavior after the intervention has been introduced. A more rigorous permutation adds an additional A or baseline phase of data collection. In such an A–B–A design , the A and B phases are followed by an additional phase of data collection during which the intervention has been withdrawn (A). Such introduction and removal of treatment allow for stronger conclusions. Importantly, within clinical psychology, withdrawal designs are often hard to attain and perhaps even less desirable. For example, unlike psychopharmacology evaluations in which a patient can simply stop taking a medication, “unlearning” a specific coping skill or behavioral strategy can be difficult, if not impossible, and in some cases may even be unethical.

Some investigators will add a second B phase to an A–B–A design to address some of the shortcomings of the A–B–A design. Such an A–B–A–B design adds rigor by offering an inherent replication of findings and the ethical shortcomings of withdrawing an effective intervention. Another permutation, the B–A–B design allows the investigator to begin the evaluation with the intervention. This is useful when assessing a clinical behavior that requires immediate attention and for which waiting throughout a baseline period may be contraindicated (e.g., suicidal ideation, self-injury), but it does not provide observations of target behaviors as they occur naturally in the absence of intervention. Moreover, to control for possible placebo effects, the A–B–C–B design introduces a third “C” phase that corresponds to a placebo condition (e.g., education, support, and attention). Addition of a C phase allows investigators to more readily attribute improvements seen in the treatment B phase to the specific intervention, rather than broadly to any intervention that could have been applied.

For researchers evaluating psychosocial interventions that do not provide opportunity for withdrawal phases, a multiple-baseline design can serve as a valuable alternative. Multiple-baseline designs employ an A–B design that differentially extends the length of the baseline (A) phase across behaviors, subjects, or settings. To establish intervention efficacy, improvements in the target behavior must be seen only after the treatment phase (B) is initiated. The baseline phase length may be determined prior to beginning a study or a researcher may wait until the target behavior stabilizes in participants and then initiate treatment (B phase) only after stabilization is achieved.

Multiple-baseline designs may occur across behaviors , across subjects , and across settings . A multiple-baseline design across behaviors evaluates the effects of an intervention on different behaviors, but within the same individual. Improvements seen in the clinical targets are attributed to intervention only if they occur after initiation of the phase of the intervention in which they were specifically targeted. Multiple-baseline designs across subjects evaluate the effects of a single intervention on multiple individuals who share a similar clinical presentation (e.g., Comer et al., 2012 ; Jarrett & Ollendick, 2012 ; Ollendick, 1995 ; Suveg, Kendall, Comer, & Robin, 2006). Each subject is assigned to a baseline period of varying and randomly determined lengths, and efficacy is demonstrated when improvements in target behaviors occur after the treatment phase is initiated, regardless of the duration of the baseline period assigned. In multiple-baseline designs across settings, intervention is applied sequentially in different settings for the same individual (e.g., at home, at school). To demonstrate treatment efficacy, improvements in the target behavior should occur in a specific setting only after intervention has been implemented in that setting.

Strengths of the multiple-baseline design include its ability to circumvent challenges of withdrawal designs when applied to psychosocial interventions. Moreover, multiple-baseline designs allow researchers to examine an intervention across multiple behaviors, settings, or individuals, which yields more generalizable findings. Some have argued that the strength of the multiple-baseline design decreases when fewer than three or four behaviors, individuals, or settings are measured ( Barlow & Nock, 2009 ; Gallo et al., 2013 ), although there is some debate about this.

Randomized controlled trials.

Whereas single-case experimental designs offer idiographic data and inferences regarding treatment effects on individual children and adolescents, to examine causal impacts of therapeutic interventions in ways that can inform clinical and policy decision-making, a treatment must be tested with tightly controlled procedures derived from experimental science in a nomothetic manner. By maximizing internal validity and systematically manipulating the intervention as the independent variable in a randomized controlled trial, researchers can more confidently and robustly conclude whether observed changes in clinical targets resulted from the intervention itself or from other extraneous factors ( Kendall, Comer, & Chow, 2013 ).

The RCT can take the form of a small pilot RCT or a larger scale clinical trial. The small pilot RCT represents a randomized, controlled study design with a restricted sample size and is useful at the end of Stage 1 research following refinement of the intervention but before entering into a larger, more costly RCT (for an example, see Comer et al., 2017 ). Small pilot RCTs ensure the intervention is suitable for a randomized study design and identify issues related to feasibility to be addressed before reaching Stage 2 research. Large-scale RCTs represent Stage 2 evaluations of therapeutic intervention and use adequately powered sample sizes to examine nomothetic effects across groups of children and adolescents with similar clinical portraits.

Regardless of sample size, the defining characteristic of the RCT is random assignment between groups . Youth are randomly assigned to either an active treatment condition where the independent variable (e.g., a given therapeutic intervention) is applied or a control condition where the experimental intervention is absent. Assignment to treatment conditions must be determined randomly and independent of baseline symptom levels, family preferences, or therapist/investigator sense of which condition would be best for a given child. At trial outset, each child has an equal chance of being assigned to various conditions (although for variations, see Kendall et al., 2013 ). Including both a treatment and control condition allows researchers to directly compare observations of a target behavior across youth who have been similarly matched on key clinical characteristics. Because of the controlled study design, changes seen uniquely or more prominently in the treatment group can confidently be attributed to the therapeutic intervention.

Importantly, randomly assigning youth across treatment conditions does not guarantee ultimate comparability across conditions, although the likelihood of such is high. Simply due to chance, participants in one group may be older, more impaired, or different on any number of meaningful variables. After data collection is complete, researchers can evaluate the comparability of youth across groups, and if baseline group differences are found, such differences are attended to as covariates at the data analysis phase. Alternately, to ensure children and adolescents across groups are matched on key characteristics, researchers can use randomized block assignments . Participants are arranged into small, equal numbered subgroups based on comparability on key characteristics (e.g., subgroups of one boy and one girl to ensure gender comparability across groups). Randomization then occurs at the subgroup level, rather than at the individual child level, retaining the randomized element while also ensuring comparability across groups.

Sequenced treatment designs.

In clinical care settings, treatments result in a range of outcomes, including improvements on target symptoms (treatment response), worsening of target symptoms (deterioration), no change in target symptoms (nonresponse), or some, but not sufficient, improvement of target symptoms (partial response). Throughout the course of treatment, therapists make clinical decisions based on response to that point to determine what, if any, changes should be made to the child’s treatment plan (e.g., continuing with the treatment course vs. switching to another treatment). The rigor and structure of the traditional RCT does not allow for flexibility during treatment implementation and therefore cannot inform clinical decision-making in cases of nonresponse, partial response, or clinical deterioration during the course of treatment.

Sequenced treatment designs and adaptive treatment regimens retain randomization procedures while also systematically evaluating shifting treatment strategies across time for children and adolescents who are not sufficiently improving. The most common and increasingly popular adaptive treatment design is the sequential multiple-assignment randomized trial (i.e., the SMART design ) ( Dawson & Lavori, 2012 ; Murphy, 2005 ), which yields quality data with which to develop evidence-based adaptive treatment algorithms that differentially incorporate the benefits of intervention forms across critical points in treatment. A SMART includes multiple intervention stages, but as each child moves through intervention stages, randomization options at key decision points are determined by the child’s treatment response at that point (see Barlow & Comer, 2013 ). Indeed, the design of a SMART improves on traditional factorial RCT designs focused on broad main effects of treatment conditions across a single treatment phase and instead recognizes the true multiphase nature of the treatment process for the majority of children and adolescents in clinical practice. The sample SMART design illustrated in Figure 8.1 examines sequences of treatment in the context of behavioral parent training (BPT) and individual child therapy (ICT) and yields data to meaningfully inform eight distinct adaptive treatment regimens. This single design requires a very large sample size but can efficiently inform sequenced treatment decisions for children and adolescents showing a range of clinical responses to different forms of initial intervention. Despite the adaptive nature of children’s individual intervention courses, the randomization element of a SMART at critical decision points still affords causal conclusions ( Barlow & Comer, 2013 ; Lei, Nahum-Shani, Lynch, Oslin, & Murphy, 2012 ). Accordingly, the SMART offers a hybrid of the nomothetic groups-based (factorial) design strategy that typically informs policy decisions and the more idiographic single-case experimental designs that clarify individualized changes.

A sample sequential multiple-assignment randomized trial (SMART) design.

A recent SMART in clinical child and adolescent psychology ( Pelham et al., 2016 ), for example, found that central nervous system stimulant medication for attention deficit hyperactivity disorder (ADHD) is most effective when it is used as a supplemental second-line treatment following an adequate course of quality low-dose behavior therapy, rather than as a first-line treatment. Pelham and colleagues were also able to document that the behavioral-first treatment strategy was far less expensive for the healthcare system than starting treatment with medication. This SMART has the potential to meaningfully influence treatment sequencing for children with ADHD in primary care, where medication alone has traditionally been the most often used treatment, with poor long-term outcomes and high associated costs.

Control Condition Considerations in Clinical Child and Adolescent Psychology

Once the investigator has decided on an appropriate study design, the investigator must select an appropriate control condition. In a “controlled” evaluation, comparable children and adolescents are randomly assigned to either the treatment condition and receive the experimental intervention or a control condition and do not receive the intervention. By contrasting changes between youth across conditions, the efficacy of the intervention beyond outcomes produced by extraneous factors (e.g., passage of time, family expectations) can be assessed. Control conditions take many forms, each carrying a unique set of strengths and limitations that affect the inferences that can be made.

No-treatment control condition.

Youth assigned to groups in which they receive no treatment are considered to be in a no-treatment control condition . This straightforward design allows researchers to draw comparisons between treatment and no treatment and consider the effect of intervention above and beyond the passage of time. Comparing intervention outcomes to outcomes in a no-treatment control condition allows the investigator to rule out the possibility that intervention effects are simply due to the regression of extreme scores to the mean across the study time period. Importantly, however, a no-treatment control condition does not rule out other explanatory factors beyond the possibility that changes represent what might naturally unfold with the passage of time. Sometimes when participants simply know they are going to get treatment, it affects their expectancies, and they show symptom improvements. Accordingly, a no-treatment control condition cannot rule out the possibility that superior changes in the treatment condition are accounted for by differences in participant expectancies associated with being assigned to (any) treatment. Accordingly, no-treatment control conditions are best suited for early stages of treatment development and evaluation and are not appropriate to meaningfully address conceptual questions about treatment efficacy and active treatment components. That said, pragmatic considerations make no-treatment controlled designs hard to implement, given difficulties of recruiting and retaining participants in a no-treatment condition.

Wait-list control condition.

An improvement over the no-treatment control condition that accounts for patient expectancies is the wait-list control condition. In a wait-list controlled design, children are assigned to receive the treatment either immediately or after a predetermined waiting period. At outset, all participants know they will receive treatment at some point in the study and likely hold similar expectations that their symptoms will improve, regardless of condition. Target clinical behaviors are assessed at uniform intervals throughout both conditions. For example, if an experimental intervention is 12 weeks, then the wait-list interval would ideally be 12 weeks as well.

Although wait-list control conditions effectively account for the passage of time as well as patient expectancies of ultimate symptom improvement, wait-list control conditions do not account for inherent benefits associated with receiving care and attention from clinical staff that have nothing to do with the specific therapeutic components hypothesized to be responsible for treatment-related change. Further, participants in a wait-list control condition are prohibited from accessing other care services during the interim wait period. Accordingly, attrition from wait-list control conditions can be high. Moreover, it can be unethical to implement a wait-list control design when alternative treatments for the clinical target have been supported in previous work. In such cases, a multiple-treatment comparison design (discussed further in this chapter) is more appropriate.

With these limitations in mind, similar to the no-treatment control, wait-list control conditions are best suited for early stages of treatment development and evaluation. Importantly, wait-list and no-treatment control conditions can carry with them ethical dilemmas. Children and adolescents in these control conditions must be regularly monitored throughout study participation to ensure they do not show serious clinical deterioration that would suggest they should be withdrawn from their assigned condition. Indeed, these control condition designs are not suitable for clinical populations that cannot tolerate a wait-list or no-treatment phase (e.g., adolescents showing suicidal behaviors).

Attention-placebo control condition.

Attention-placebo control conditions are valuable for investigators looking to additionally rule out “common factors” associated with all therapeutic interventions (e.g., receiving care and attention from warm clinical staff, having an outlet through which problems can be discussed). These designs contrive a control condition that mimics elements of treatment by inviting participants to receive face-to-face interactions with attentive clinical staff. Importantly, the attention-placebo control condition is explicitly devoid of elements that are believed to be specifically effective in the experimental intervention. Attention-placebo control conditions typically consist of general psychoeducation, clinical monitoring, and broad patient support.

Despite the advantage that attention-placebo control conditions have for accounting for common, nonspecific therapeutic factors, it can be difficult to establish credibility (for patients and for therapists) when implementing these control conditions. It is useful for therapists to hold equally positive expectations of improvement for participants across conditions ( Kazdin, 2003 ), and establishing positive expectations for a condition oriented around nonspecific treatment factors can be difficult to achieve. Thus, researchers utilizing attention-placebo control conditions should measure participant expectations across conditions so that participant expectancy effects can be accounted for in analyses.

Standard-treatment comparison condition.

A standard-treatment comparison or treatment-as-usual control condition consists of an invention that is routinely given and allows the investigator to evaluate the incremental benefits of an experimental intervention over and above the existing standard of care. Ethical concerns that arise in no-treatment, wait-list, and attention-placebo conditions are minimized because children in this condition are receiving exactly what they would have received for their problems had the study never taken place. Moreover, attrition is minimized as all children receive active care, and patient and therapist expectations for change are likely to be more comparable. Despite these benefits, however, what exactly constitutes “treatment as usual” has been difficult to operationalize as it varies widely across settings, making it difficult to integrate findings across studies incorporating these control conditions. Further, differences between an experimental intervention and a treatment-as-usual condition might be attributed to differences in therapist quality, training, supervision, or organization, rather than to differences specific to the hypothesized active ingredients of the experimental intervention. Moreover, it can be difficult to match the intensity, dosing, or duration of treatments when comparing an experimental treatment condition to a treatment-as-usual condition. For example, suppose an experimental treatment protocol calls for weekly 60-minute sessions with a therapist for 12 weeks, whereas the standard care that is currently offered in a setting entails 20-minute sessions every other week for up to 8 weeks. If the investigator changes the treatment-as-usual condition to have control participants meet weekly and for longer periods of time with therapists, the control group is no longer a “standard care” condition; it is a new condition contrived by the investigator. Alternatively, if the investigator in this scenario compares the experimental condition to the true treatment as usual, it is possible that differences between the conditions could simply be due to differences in the frequency and intensity of care and not to the putative active ingredients of the experimental treatment.

Multiple-treatment comparisons.

Some more rigorous and revealing studies include multiple active treatment conditions and are thus able to address issues of relative or comparative efficacy. These studies offer direct comparisons of alternative active treatments. For multiple treatment comparisons, it is important that each treatment is comparable on a number of characteristics, including duration, session length, and frequency; setting; and level of credibility. For example, if children who received Treatment A were found to show superior outcomes to children in Treatment B, but Treatment A was 8 weeks and Treatment B was 4 weeks, the investigator would not be able to determine whether Treatment A had stronger effects than Treatment B or whether the study just found that 8 weeks of treatment was better than 4 weeks of treatment. Further, multiple-treatment comparison studies must ensure comparability of therapists across conditions. Therapists should be matched on their levels of training and experience, expertise in administration of study treatment protocols, and attitudes toward the treatments, including their allegiance to specific therapeutic approaches and their intervention expectancies. For example, it would be problematic if in a multiple-treatment comparison design a group of psychodynamic therapists conducted both a behavioral intervention (in which their expertise is low) and a psychodynamic therapy (in which their expertise is high). If outcomes differed across conditions, it would not be clear whether this was the result of true differences between behavioral and psychodynamic approaches or whether this was simply due to differences in therapist expectancies across the conditions.

Researchers using a multiple-treatment comparison design must also consider issues of sample size and outcome measurement. Whereas comparisons of active conditions against inactive control conditions typically yield large effect sizes, comparisons of multiple active conditions typically yield smaller effect sizes and accordingly require larger samples for adequate power. Moreover, to avoid potential biases, measures should cover a range of target clinical symptoms, and assessments should be equally sensitive to expected changes associated with each treatment type. For example, a measure that primarily evaluates children’s self-talk may be a well-suited measure for examining the impact of cognitive behavioral therapy but may not evaluate meaningful changes associated with antidepressant medication, for which the direct targeting of children’s self-talk is not a proposed mechanism of change ( Comer & Kendall, 2013a ).

Independent Variable Considerations in Clinical Child and Adolescent Psychology

In the context of a clinical trial, the independent variable that is manipulated is treatment assignment, that is, whether a child does or does not receive treatment or which treatment condition a child will receive. As in any experimental study, this independent variable must be carefully operationalized and implemented with integrity. Specifically, when evaluating an experimental intervention, the treatment must be adequately detailed and described in order to replicate the evaluation or to be able to communicate to others how to conduct the treatment ( Comer & Kendall, 2013a ). A treatment protocol that clearly defines the intervention and dictates how it is to be administered is critical for internal validity and ensuring the integrity of the independent variable. However, manualized intervention protocols can limit external validity, especially when attempting to generalize findings to settings and practitioners who do not typically use treatment manuals to guide their services. Some critics argue that manualized treatment protocols are overly rigid and do not afford clinicians needed flexibility to adapt to the complex and individualized patient needs encountered in routine practice settings ( Addis & Krasnow, 2000 ). Although most supported treatment manuals have always afforded a great deal of flexibility to individual patient needs, more modern treatment protocols are increasingly taking a modular approach, in which supported practices for specific identified problems are structured as free-standing modules, and decision flowcharts guide treatment component sequencing and module selection ( Chorpita, 2007 ; Comer, Elkins, Chan, & Jones, 2014 ). Modularized treatment protocols address complex comorbidities and shifting clinical needs by accommodating personalized tailoring of care for specific problems presenting in each child.

Dependent Variable Considerations in Clinical Child and Adolescent Psychology

The investigator must decide which dependent variables will be assessed and how they will be measured. Indeed, it is critical to measure outcomes using a variety of methods in order to minimize bias. Given research documenting poor cross-informant agreement in the assessment of child psychopathology (e.g., Comer & Kendall, 2004 ; De Los Reyes & Kazdin, 2005 ; Grills & Ollendick, 2003 ), investigators are wise to collect reports from multiple informants (e.g., parents, teachers, therapists, children). Such a multi-informant strategy allows researchers to evaluate symptoms that may differentially present across various contexts and life domains or that may be perceived differently across key people in children’s lives ( Silverman & Ollendick, 2005 ). Features of cognitive development can interfere with the accuracy of young children’s reports, and demand characteristics may cause children to offer what they believe to be desired responses. Accordingly, it is important to collect data simultaneously from important adults in children’s lives who observe their behavior across different settings. On the other hand, parents and teachers may not be privy to more internal and unobservable symptoms (e.g., anxiety).

Multimodal assessment strategies draw on multiple modes of assessment (e.g., observations, questionnaires) to evaluate the same dependent variable. For example, positive parenting practices might be measured via behavioral codings of structured parent–child interactions, as well as parent self-reports. For other dependent variables, objective records (e.g., medical or school records) might be collected. Data on peer relations might draw on sociometric data and peer nominations.

Finally, multiple targets should be assessed ( De Los Reyes & Kazdin, 2006 ). Improvement can take many forms, including decreased symptoms, loss of clinical diagnosis, improved quality of life, higher academic functioning, and improved interpersonal functioning. No single dependent variable independently and sufficiently captures treatment response. Inherent in a multiple-domain assessment strategy, however, is the fact that treatments rarely produce uniform effects across assessed domains. For example, one treatment might improve child anxiety but not peer relationships, whereas another treatment might improve children’s peer relationships but not anxiety. If a clinical trial were to compare these two treatments, it is not readily apparent which treatment should be deemed more efficacious ( Comer & Kendall, 2013a ). Typically, the investigator selects a primary outcome, as well as secondary and exploratory outcomes that provide more nuanced information about treatment responses. Importantly, selection of a primary outcome variable must occur prior to collection and review of the findings, so that decisions about which variables are most important are made a priori and are not biased by the significance of results. De Los Reyes and Kazdin (2006) have argued for a multidimensional conceptualization of intervention change, and similarly we caution consumers of the treatment literature against simplistic dichotomous appraisals of treatments as effective or not.

Assessment Point Considerations in Clinical Child and Adolescent Psychology

Evaluating a novel intervention through an experimental design requires a clinical researcher to take careful observations of the dependent variables across the duration of the study at key time points. Target clinical behaviors are selected for measurement and should be assessed at the outset of the study to provide baseline data . Baseline data serve as benchmarks against which subsequent observations of dependent variables are assessed. Post-treatment assessments are another critical time point for assessment, as those observations speak to acute treatment outcomes or the impact of an experimental intervention on clinical symptoms immediately after treatment is complete.

Although post-treatment data are critical, post-treatment data do not allow researchers to examine enduring treatment effects. To demonstrate lasting treatment gains or maintenance , researchers must also measure clinical outcomes at predetermined intervals after treatment has been completed (e.g., 3 months post-treatment, 6 months post-treatment). Such follow-up evaluations add methodological rigor to a study. For example, in a study comparing multiple active treatments, acute post-treatment outcomes may be comparable, but follow-up assessments may reveal that children in one experimental treatment condition showed higher maintenance of treatment gains with continued time. Importantly, for follow-up assessments to capture true lasting effects, participants should not have contact with other clinical services during the follow-up assessment period. Because follow-up intervals can be lengthy, it is not always feasible or ethical to prevent participants from receiving outside services during a follow-up interval. Many investigators, accordingly, include a naturalistic follow-up component that allows participants to seek outside services during the interval between post-treatment and follow-up evaluation. Additional service use after treatment completion may actually be a variable of interest, and when it is not, outside service use during the follow-up interval should be controlled for statistically.

Investigators are also increasingly incorporating assessments at different points during treatment, or midtreatment, to establish growth curves, consider the rate and shape of change during the treatment phase, and better understand potential mediators of treatment response. Midtreatment assessments provide revealing data on when symptom changes occur during treatment, at what pace, and how changes across different domains of response may unfold and interact with one another across time ( Chu, Skriner, & Zandberg, 2013 ; Gallo, Cooper-Vince, Hardway, Pincus, & Comer, 2014 ; Kendall et al., 2009 ; Marker, Comer, Abramova, & Kendall 2013 ).

Sample and Setting Considerations in Clinical Child and Adolescent Psychology

Careful consideration is needed when selecting a sample to best represent the clinical population of interest. Those youth chosen to participate in the trial will strongly influence the extent to which findings can be generalized to the larger population of youth who may benefit from the treatment. A genuine clinical sample made up of youth shown to have a disorder and who are seeking treatment will afford greatest external validity and generalizability. However, genuine clinical samples can be difficult for researchers to recruit into studies; moreover, they frequently carry more complex clinical portraits, which can threaten the internal validity of the study. Alternatively, analogue or selected samples can afford a higher degree of control and internal validity in study design, but youth in such samples are not necessarily comparable to the majority of patients typically seen in clinical practice.

Broadly speaking, it is important that the sample in a study evaluating an experimental intervention reflect the population for which that intervention is intended to ultimately benefit. Thus, in addition to considering a sample’s clinical characteristics, researchers in clinical child and adolescent psychology must consider sociodemographic diversity. Race, ethnicity, gender, socioeconomic status, education level, and other related demographic characteristics must all be considered when recruiting an appropriate study sample that can generalize to the general population.

The setting in which a study takes place will also have important implications for the generalizability of results. Early stage evaluations of therapeutic interventions are often conducted in clinical research laboratory settings and require investigators to recruit subjects to participate. Therapists in these trials are typically part of the investigator’s research team, and as such their outcomes may not generalize to the practices of front-line clinicians who differ from research staff clinicians with regard to experience, caseload size, supervision, and oversight. It is ultimately critical to demonstrate the transportability of an intervention to front-line service settings. Therefore, later stage evaluations of therapeutic interventions must evaluate outcomes beyond tightly controlled research settings.

Conducting a Clinical Trial

Once a clinical trial has been carefully designed, it does not simply run itself. The investigator must play a highly active role in organizing and implementing each aspect of the study in order to ensure a successful trial.

Training and organizing study staff merits special attention. Independent evaluators ( IEs ) refer to staff members who participate in assessment procedures and who are masked to each participant’s treatment assignment. IEs must be trained to a prespecified criterion (e.g., must match the diagnostic profile generated by the principal investigator on at least three consecutive diagnostic interviews) prior to their active participation on the trial, and throughout the course of the study periodic reliability checks are further necessary to ensure interrater reliability across study IEs. Systematic safeguards must be put in place to guarantee that IEs are kept unaware of each participant’s treatment assignment. IEs should try to avoid patient waiting rooms in which they might run into families assigned to treatment. IEs should not attend clinical supervision meetings that would reveal participant assignment information. For smaller teams in which the same staff members serve as both IEs and as therapists on different cases, multiple supervision teams are required, and staff members can only serve as IEs on cases carried on opposite supervision teams. Prior to post-treatment assessments, participating families should be reminded that their post-treatment assessor does not know which treatment they received (or even whether they received treatment if it is a wait-list controlled trial), and families should be cautioned against speaking about any treatment experiences during the interview.

Study therapists must be adequately trained. This typically involves initial didactic training on the study protocol and knowledge quizzes, followed by role plays. Ideally, there is opportunity for trainee therapists to shadow and then cotreat several cases using the study protocol prior to their active participation as a therapist on the study. Investigators should set a criterion that must be met by study therapists prior to their carrying study cases independently (e.g., complete didactic training, achieve a score of 80% or greater on a knowledge quiz, shadow one case, and cotreat one case). Regular supervision is critical to avoid therapist drift and to ensure treatment integrity.

Just because study therapists have been trained to criterion does not guarantee that they will deliver the independent variable (treatment) as intended. In the course of a study, the treatment that was assigned may not in fact be the treatment that is provided (see also Perepletchikova & Kazdin, 2005 ). To ensure that study treatments are implemented as intended, treatment integrity checks should be conducted. Therapy sessions should be regularly recorded such that independent raters can view them and provide quantifiable judgments on the implementation of key treatment components. McLeod, Islam, and Wheat (2013) provided more detailed descriptions of procedural issues in the conduct of quality assurance and treatment integrity checks.

Throughout the course of a clinical trial, someone must be responsible for monitoring the sample and the data and for ensuring that all data are collected as designed and intended. This person must be omniscient: The person must know the condition of every participant, must know who is assigned to which cases and who is responsible for collecting each piece of data, must know who can know about each participant’s condition, and must be aware of where each family is in the flow of study phases. Even with such a person dedicated to this role, data will inadvertently be missed, blinds will unintentionally be broken, and families will mistakenly not be contacted at their appropriate follow-up points. For a large clinical trial, this is a full-time job, but for smaller studies, a principal investigator can often perform in this role. Most important, the individual in charge of tracking cannot serve as an IE or as an IE supervisor because their role inherently unmasks them to all study-related information that could bias responses.

Throughout treatment, study staff must regularly monitor adverse events, and an individual or panel of individuals must be responsible for deciding whether a particular child suffering adverse events should be withdrawn from the study. This is particularly important when treatment conditions include medications that can introduce unfavorable side effects, but psychological treatments can also be stressful and associated with adverse events.

Retaining the sample throughout the study can be challenging. It is recommended that study staff phone, email, or text families weekly to “welcome” them to their next session or appointment. During the treatment phase, there is an attrition risk when study treatment cannot address complex and shifting patient needs that may present. For example, a family crisis, an emergent academic issue, or a serious peer conflict may present, and such unforeseen events may become a clinical priority for the family that is not explicitly addressed by the treatment protocol. Adjunctive services and attrition prevention ( ASAP ) procedures (e.g., Abikoff et al., 2002 ) are often implemented to maintain the sample, in which each case in a trial is allowed a prespecified number of additional sessions during the treatment phase to address exigencies or clinical crises that fall outside of the scope of the treatment protocol. Even children who do not complete treatment should be invited to participate in as many post-treatment and follow-up evaluations as possible. Monthly calls, birthday wishes, and holiday cards are recommended during follow-up intervals in order to maintain contact with families between the end of treatment and follow-up evaluation, thus maximizing sample participation at follow-up.

Analyzing Trial Outcomes

After conducting a clinical trial, the data analysis phase entails the active process through which the investigative team extracts relevant information from the collected data in ways that permit statistical inferences about the larger population of youth the sample was recruited to represent. A comprehensive outline of clinical trial data analysis is beyond the scope of the present chapter (the interested reader is referred to Jaccard & Guilamo-Ramos, 2002a , 2002b ; Read, Kendall, Carper, & Rausch, 2013 ); here, we briefly address (a) missing data and attrition; (b) evaluation of clinical significance; and (c) evaluation of change mechanisms.

Missing data and attrition.

Even in the most diligently organized and carefully implemented clinical trial, not every child randomized will actually complete participation. Mason (1999) estimated that on average roughly 20% of participants withdraw prematurely from their participation in a clinical trial. Attrition can be problematic for data analysis, particularly when large numbers of youth do not complete treatment or when rates of attrition vary across study conditions ( Leon et al., 2006 ).

When there is a meaningful discrepancy between the number of children randomized to the various treatment conditions and the number of children who completed their participation, the investigator can conduct and report two sets of data analyses: (a) treatment completer analyses that evaluate only those youth who completed the full course of their treatment and (b) intent-to-treat analyses that include all those initially randomized. Treatment completer analyses evaluate intervention effects when someone receives a full “dose” of treatment. Those who drop out of treatment, those who refuse treatment, and those who do not adhere to treatment are not included in such analyses ( Kendall et al., 2013 ). Treatment completer outcomes may be somewhat inflated because they only capture the results of children who fully adhered to and completed treatment. At the same time, treatment completer analyses directly examine outcomes associated with true exposure to the experimental manipulation and therefore provide very valuable information.

On the other hand, intent-to-treat analyses are more conservative and evaluate outcomes for all children involved at the point of randomization. Such analyses speak more directly to issues of generalizability of findings as they incorporate information about treatment tolerability. A simplistic method for handling missing data for intent-to-treat analyses is the last observation carried forward ( LOCF ) method, which assumes that the scores for children who withdraw from treatment remain constant from their last assessment point throughout the conclusion of the study. For example, if a family withdraws participation at Week 8, then the data values from that child’s Week 7 assessment (or most recently completed assessment) would be substituted for all subsequent assessment points. However, LOCF introduces systematic bias and fails to take into account the uncertainty of postdropout functioning ( Leon et al., 2006 ). Accordingly, LOCF methods have fallen out of fashion, in favor of (a) multiple imputation methods , which impute a range of values for missing data by incorporating the uncertainty of the true values of missing data ( Little & Rubin, 2002 ); and (b) mixed-effects modeling , which relies on regression modeling to address missing data in the context of random (e.g., child) and fixed (e.g., treatment condition, gender) effects. Mixed-effects modeling is a particularly strong approach to handling missing data when numerous assessments are collected across a treatment trial (e.g., weekly data are collected).

Evaluation of statistical and clinical significance.

Statistical significance is identified when the mean difference between treatment conditions is beyond that which could have resulted by chance alone (most commonly defined as p < .05). Tests of statistical significance are critical as they indicate how likely it is that observed differences between conditions were not due solely to chance. However, tests of statistical significance alone do not provide compelling information on the clinical significance of group differences. Relying solely on statistical significance can lead an investigator to interpret treatment gains as meaningful when in fact they may be clinically insignificant ( Kendall et al., 2013 ). For example, suppose that a treatment for disruptive behavior problems results in significantly lower scores on the Externalizing Scale of the Child Behavior Checklist (CBCL). An examination of CBCL means however reveals only a small but reliable shift from a mean of 81 to a mean of 78. With a large enough sample size, this change can achieve statistical significance at the conventional p < .05 level, but a 3-point change from 81 to 78 on the CBCL Externalizing Scale is of limited practical significance. At both baseline and still at post-treatment, the scores are within the clinically elevated range, and such a small magnitude of change may have little effect on a child’s functioning.

Clinical significance refers to the meaningfulness or persuasiveness of the magnitude of change ( Jacobson & Truax, 1991 ; Kendall, 1999 ). Whereas tests of statistical significance ask the question, “Were there intervention-related changes?” tests of clinical significance ask the question, “Were intervention-related changes convincing and meaningful?” Clinical significance can be evaluated by (a) considering the extent to which treated youth are returned within normal limits (i.e., they are indistinguishable from a normative sample of youth; Kendall, Marrs-Garcia, Nath, & Sheldrick, 1999 ); (b) evaluating the magnitude of effect sizes of change, regardless of statistical significance; or (c) computing the Reliable Change Index (RCI; Jacobson & Truax, 1991 ) across participants. To calculate the RCI, the investigator assesses the extent to which each individual participant’s change pre- to post-treatment was reliable, versus the possible result of simple measurement error. For each participant, the investigator calculates a difference score (e.g., post score minus baseline score) and compares it to the standard error of measurement (i.e., ±1.96 SE ). As such, the RCI is determined by two factors: (a) the magnitude of change and (b) the reliability of measurement. Each of these approaches to assessing clinical significance (e.g., normative comparisons, effect size interpretations, RCI) provides an important, but unique, perspective on the meaningfulness of treatment outcomes; thus, they are often used in conjunction with one another. For example, an investigator might use published norms of a measure to evaluate which participants crossed over from the clinical range to the nonclinical range and also calculate an RCI for each participant. The investigator can use these data to group participants into the following categories: recovered (i.e., passed both normative cutoff and RCI criteria), unchanged (i.e., passed neither criteria), improved (passed RCI but not normative cutoff criteria), or deteriorated (i.e., passed RCI criteria in the negative direction) ( Comer & Kendall, 2013a ; Jacobson & Truax, 1991 ; McGlinchey, Atkins, & Jacobson, 2002 ).

Evaluation of change mechanisms.

Researchers and funding agencies are increasingly interested in identifying the conditions that determine when an intervention is more or less potent (moderation) and the processes through which an intervention produces change (mediation). A moderator is a variable that delineates the conditions under which a given intervention is related to an outcome. Conceptually, moderators identify on whom and under which circumstances treatments have different effects, and they are usually measured prior to treatment ( Kendall et al., 2013 ; Kraemer, Wilson, Fairburn, & Agras, 2002 ). Functionally, a moderator is a variable that influences either the direction or magnitude of an association between the independent variable (treatment condition) and a dependent variable (outcome). Treatment moderators help identify which youth might be most responsive to which interventions and for which youth alternative interventions might be appropriate. Of note, when a variable broadly predicts treatment response across all treatment conditions in a clinical trial, conceptually that variable is simply a predictor , not a moderator (see Kraemer et al., 2002 ).

A mediator, on the other hand, is a variable that is measured during treatment and clarifies the process by which an intervention influences an outcome. Conceptually, mediators identify how and why treatments have the effect they do ( Kraemer et al., 2002 ). The mediator effect reveals the mechanism through which treatment is associated with outcomes. Significant meditation affords causal conclusions. If a supported treatment for child anxiety was found to influence negative self-talk, which in turn was found to have a significant influence on child anxiety and avoidance, then negative self-talk might be considered to mediate the treat-to-outcome association. Specific statistical methods used to evaluate the presence of treatment moderation and mediation can be found elsewhere ( MacKinnon, Lockhart, Baraldi, & Gelfand, 2013 ).

Funding agencies are increasingly prioritizing interventions research that explicitly examines mechanisms that can explain treatment effects. The experimental therapeutics paradigm has the researcher first hypothesize a “target” or mechanism of action. Rather than focusing on clinical effects and treatment response, the experimental therapeutics researcher studies an intervention first as a manipulation to verify whether the intervention has a predicted effect on the target mechanism (i.e., target engagement). Once target engagement has been documented, the experimental therapeutics researcher then examines whether clinical outcomes are indeed related to successful target engagement.

Reporting Results

Presenting the written study findings to the scientific community in a peer-reviewed outlet is the final step of a clinical trial. A well-constructed data report must present all relevant methodological and study-related information with enough context to afford meaningful interpretation of results and to allow for replication. Study aims and results must be placed in the context of related research to illustrate how the current findings compare to previous results, and the investigator must discuss how the results build on, support, or diverge from other findings in the field. A candid and nondefensive articulation of study limitations and shortcomings is also critical in order to direct future research.

To avoid potential bias in the reporting of clinical trial results, a multidisciplinary panel of experts established a checklist of guidelines for maximizing transparency in reporting (i.e., CONSORT guidelines; see Begg et al., 1996 ). CONSORT (i.e., Consolodated Standards of Reporting Trials) guidelines offer a minimum set of recommendations for preparing reports of clinical trial findings that ensure transparent, comprehensive reporting to facilitate critical evaluation and interpretation. Chief among the CONSORT items is inclusion of a graphical representation of the flow of study participation from baseline to study completion. Such a “CONSORT chart” provides important information on recruitment, randomization, retainment, and participant attrition across treatment conditions and assessment time points.

We remain at a relatively nascent stage in the science of clinical child and adolescent psychology, with the majority of work ahead of us. Having reviewed key considerations for planning, conducting, analyzing, and reporting clinical evaluations of child and adolescent treatments, it is clear that no individual investigation, even with optimal design and procedures, is able to adequately answer all relevant questions. Rather, a collection and series of investigations, drawing on a broad range of methodological strategies, is needed to progress our understanding of best practices for the widely diverse range of mental health problems that present in childhood and adolescence.

Those looking for the “correct” research methodology with which to address all questions in clinical child and adolescent psychology are misguided. Throughout this chapter, we have outlined how for each testable question there are many research strategies that can be used to reveal meaningful information, each with strengths and limitations. Collectively, the methods and design considerations outlined in this chapter detail a portfolio of modern research strategies for the continually evolving science of clinical child and adolescent psychology: a set of alternative and complementary “directions” so to speak for advancing our field from where we are now to where we need to be.

Abikoff, H. , Arnold, L. E. , Newcorn, J. H. , Elliott, G. R. , Hechtman, L. , Severe, J. B. , . . . Wells, K. C. ( 2002 ). Emergency/adjunct services and attrition prevention for randomized clinical trials in children: The MTA manual-based solution.   Journal of the American Academy of Child and Adolescent Psychiatry , 41 , 498–504.

Google Scholar

Addis, M. , & Krasnow, A. ( 2000 ). A national survey or practicing psychologists’ attitudes toward psychotherapy treatment manuals.   Journal of Consulting and Clinical Psychology , 68 , 331–339.

American Psychological Association. ( 2002 ). Criteria for practice guideline development and evaluation.   American Psychologist , 57 (12), 1048–1051.

Barlow, D. H. , & Comer, J. S. ( 2013 ). What are the optimal treatment courses for geriatric anxiety, and how do we find out?   American Journal of Psychiatry , 170 , 707–711.

Barlow, D. H. , & Nock, M. ( 2009 ). Why can’t we be more idiographic in our research?   Perspectives on Psychological Science , 4 (1), 19–21.

Barlow, D. H. , Nock, M. , & Hersen, M. ( 2009 ). Single case experimental designs: Strategies for studying behavior change (3rd ed.). Boston: Pearson/Allyn and Bacon.

Google Preview

Begg, C. B. , Cho, M. K. , Eastwood, S. , Horton, R. , Moher, D. , Olkin, I. , . . . Stroup, D. F. ( 1996 ). Improving the quality of reporting of randomized clinical trials: The CONSORT statement.   Journal of the American Medical Association , 276 , 637–639.

Breslau, J. , Lane, M. , Sampson, N. , & Kessler, R. C. ( 2008 ). Mental disorders and subsequent educational attainment in a US national sample.   Journal of Psychiatric Research , 42 (9), 708–716.

Chorpita, B. F. ( 2007 ). Modular cognitive-behavioral therapy for childhood anxiety disorders. New York: Guilford Press.

Chu, B. C. , Skriner, L. C. , & Zandberg, L. J. ( 2013 ). Shape of change in cognitive behavioral therapy for youth anxiety: Symptom trajectory and predictors of change.   Journal of Consulting and Clinical Psychology , 81 , 573–587.

Comer, J. S. , Blanco, C. , Hasin, D. S. , Liu, S. M. , Grant, B. F. , Turner, J. B. , & Olfson, M. ( 2011 ). Health-related quality of life across the anxiety disorders: Results from the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC).   Journal of Clinical Psychiatry , 72 (1), 43–50.

Comer, J. S. , Elkins, R. M. , Chan, P. T. , & Jones, D. J. ( 2014 ). New methods of service delivery for children’s mental health care. In C. A. Alfano & D. Beidel (Eds.), Comprehensive evidence-based interventions for school-aged children and adolescents (pp. 55–72). New York: Wiley.

Comer, J. S. , Furr, J. M. , Kerns, C. E. , Miguel, E. , Coxe, S. , Elkins, R. M. , . . . Freeman, J. B. ( 2017 ). Internet-delivered, family-based treatment for early-onset OCD: A pilot randomized trial.   Journal of Consulting and Clinical Psychology , 85 (2), 178–186.

Comer, J. S. , & Kendall, P. C. ( 2004 ). A symptom-level examination of parent-child agreement in the diagnosis of anxious youths.   Journal of the American Academy of Child and Adolescent Psychiatry , 43 , 878–886.

Comer, J. S. , & Kendall, P. C. ( 2013 a). Methodology, design, and evaluation in psychotherapy research. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed.). Hoboken, NJ: Wiley.

Comer, J. S. , & Kendall, P. C. ( 2013 b). A place for research strategies in clinical psychology. In J. S. Comer & P. C. Kendall (Eds.), The Oxford handbook of research strategies for clinical psychology. New York: Oxford University Press.

Comer, J. S. , Puliafico, A. C. , Aschenbrand, S. G , McKnight, K. , Robin, J. A. , Goldfine, M. , & Albano, A. M. ( 2012 ). A pilot feasibility evaluation of the CALM Program for anxiety disorders in early childhood.   Journal of Anxiety Disorders , 26 , 40–49.

Dawson R. , & Lavori, P. W. ( 2012 ). Efficient design and inference for multistage randomized trials of individualized treatment policies.   Biostatistics , 13 , 142–152.

De Los Reyes, A. , & Kazdin, A. E. ( 2005 ). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical foundation, and recommendations for further study.   Psychological Bulletin , 131 , 483–509.

De Los Reyes, A. , & Kazdin, A. E. ( 2006 ). Conceptualizing changes in behavior in intervention research: The range of possible changes model.   Psychological Review , 113 , 554–583.

Gallo, K. P. , Comer, J. S. , & Barlow, D. H. ( 2013 ). Single-case experimental designs and small pilot trial designs. In J. S. Comer & P. C. Kendall (Eds.), The Oxford handbook of research strategies for clinical psychology (pp. 24–39). New York: Oxford University Press.

Gallo, K. P. , Cooper-Vince, C. E. , Hardway, C. , Pincus, D. B. , & Comer, J. S. ( 2014 ). Trajectories of change across outcomes in intensive treatment for adolescent panic disorder and agoraphobia.   Journal of Clinical Child and Adolescent Psychology , 43 , 742–750.

Grills, A. E. , & Ollendick, T. H. ( 2003 ). Multiple informant agreement and the Anxiety Disorders Interview Schedule for Parents and Children.   Journal of the American Academy of Child and Adolescent Psychiatry , 42 , 30–40.

Jaccard, J. , & Guilamo-Ramos, V. ( 2002 a). Analysis of variance frameworks in clinical child and adolescent psychology: Advanced issues and recommendations.   Journal of Clinical Child and Adolescent Psychology , 31 , 278–294.

Jaccard, J. , & Guilamo-Ramos, V. ( 2002 b). Analysis of variance frameworks in clinical child and adolescent psychology: Issues and recommendations.   Journal of Clinical Child and Adolescent Psychology , 31 , 130–146.

Jacobson, N. S. , & Truax, P. ( 1991 ). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research.   Journal of Consulting and Clinical Psychology , 59 (1), 12.

Jarrett, M. A. , & Ollendick, T. H. ( 2012 ). Treatment of comorbid attention-deficit/hyperactivity disorder and anxiety in children: A multiple baseline design analysis.   Journal of Consulting and Clinical Psychology , 80 , 239–244.

Kazdin, A. E. ( 2001 ). Behavior modification in applied settings (6th ed.). Belmont, CA: Wadsworth/Thompson Learning.

Kazdin, A. E. ( 2003 ). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.

Kendall, P. C. ( 1999 ). Introduction to the special section: Clinical significance.   Journal of Consulting and Clinical Psychology , 67 , 283–284.

Kendall, P. C. ( 2012 ). Child and adolescent therapy: Cognitive-behavioral procedures (4th ed.). New York: Guilford.

Kendall, P. C. , Comer, J. S. , & Chow, C. ( 2013 ). The randomized controlled trial: Basics and beyond. In J. S. Comer & P. C. Kendall (Eds.), The Oxford handbook of research strategies for clinical psychology (pp. 40–61). New York: Oxford University Press.

Kendall, P. C. , Comer, J. S. , Marker, C. D. , Creed, T. A. , Puliafico, A. C. , Hughes, A. A. , . . . Hudson, J. L. ( 2009 ). In-session exposure tasks and therapeutic alliance across the treatment of childhood anxiety disorders.   Journal of Consulting and Clinical Psychology , 77 , 517–525.

Kendall, P. C. , & Kessler, R. C. (2002). The impact of childhood psychopathology interventions on subsequent substance abuse: Policy implications, comments, and recommendations.   Journal of Consulting and Clinical Psychology , 70 (6), 1303 .

Kendall, P. C. , Marrs-Garcia, A. , Nath, S. R. , & Sheldrick, R. C. ( 1999 ). Normative comparisons for the evaluation of clinical significance.   Journal of Consulting and Clinical Psychology , 67 , 285–299.

Kessler, R. C. , Avenevoli, S. , Costello, E. J. , Georgiades, K. , Green, J. G. , Gruber, M. J. , . . . Merikangas, K. R. ( 2012 ). Prevalence, persistence, and sociodemographic correlates of DSM-IV disorders in the National Comorbidity Survey Replication Adolescent Supplement. Archives of General Psychiatry , 69 , 372–380.

Kessler, R. C. , Avenevoli, S. , McLaughlin, K. A. , Green, J. G. , Lakoma, M. D. , Petukhova, M. , . . . Merikangas, K. R. ( 2012 ). Lifetime co-morbidity of DSM-IV disorders in the U. S. National Comorbidity Survey Replication Adolescent Supplement (NCS-A). Psychological Medicine , 42 , 1997–2010.

Kraemer, H. C. , Wilson, G. T. , Fairburn, C. G. , & Agras, W. S. ( 2002 ). Mediators and moderators of treatment effects in randomized clinical trials.   Archives of General Psychiatry , 59 , 877–883.

Lei, H. , Nahum-Shani, I. , Lynch, K. , Oslin, D. , & Murphy, S. A. ( 2012 ). A “SMART” design for building individualized treatment sequences.   Annual Review of Clinical Psychology , 8 , 21–48.

Leon, A. C. , Mallinckrodt, C. H. , Chuang-Stein, C. , Archibald, D. G. , Archer, G. E. , & Chartier, K. ( 2006 ). Attrition in randomized controlled clinical trials: Methodological issues in psychopharmacology.   Biological Psychiatry , 59 , 1001–1005.

Little, R. J. A. , & Rubin, D. ( 2002 ). Statistical analysis with missing data (2nd ed.). New York: Wiley.

MacKinnon, D. P. , Lockhart, G. , Baraldi, A. N. , & Gelfand, L. A. ( 2013 ). Evaluating treatment mediators and moderators. In J. S. Comer & P. C. Kendall (Eds.), Oxford handbook of research strategies for clinical psychology . New York: Oxford.

Marker, C. D. , Comer, J. S. , Abramova, V. , & Kendall, P. C. ( 2013 ). The reciprocal relationship between alliance and symptom improvement across the treatment of childhood anxiety.   Journal of Clinical Child and Adolescent Psychology , 42 , 22–33.

Mason, M. J. ( 1999 ). A review of procedural and statistical methods for handling attrition and missing data.   Measurement and Evaluation in Counseling and Development , 32 , 111–118.

McGlinchey, J. B. , Atkins, D. C. , & Jacobson, N. S. ( 2002 ). Clinical significance methods: Which one to use and how useful are they?   Behavior Therapy , 33 , 529–550.

McLeod, C. , Islam, N. , & Wheat, E. ( 2013 ). Designing, conducting, and evaluating therapy process research. In J. S. Comer & P. C. Kendall (Eds.), The Oxford handbook of research strategies for clinical psychology . New York: Oxford University Press.

Merikangas, K. R. , Ames, M. , Cui, L. , Stang, P. E. , Ustun, T. , Von Korff, M. , & Kessler, R. C. ( 2007 ). The impact of comorbidity of mental and physical conditions on role disability in the US adult household population.   Archives of General Psychiatry , 64 (10), 1180–1188.

Merikangas, K. R. , Calkins, M. E. , Burstein, M. , He, J. P. , Chiavacci, R. ., Lateef, T. , . . . Gur, R. E. ( 2015 ). Comorbidity of physical and mental disorders in the neurodevelopmental genomics cohort study.   Pediatrics , 135 , e927–e938.

Murphy, S. A. ( 2005 ). An experimental design for the development of adaptive treatment strategies.   Statistics in Medicine , 24 , 1455–1481.

Nock, M. K. , & Kessler, R. C. ( 2006 ). Prevalence of and risk factors for suicide attempts versus suicide gestures: Analysis of the National Comorbidity Survey.   Journal of Abnormal Psychology , 115 (3), 616–623.

Ollendick, T. H. ( 1995 ). Cognitive behavioral treatment of panic disorder with agoraphobia in adolescents: A multiple baseline design analysis.   Behavior Therapy , 26 , 517–531.

Ollendick, T. H. , & King, N. J. ( 2004 ). Empirically supported treatments for children and adolescents: Advances toward evidence-based practice. In P. M. Barrett & T. H. Ollendick (Eds.), Handbook of interventions that work with children and adolescents. New York: Wiley.

Pelham, W. E. , Fabiano, G. A. , Waxmonsky, J. G. , Greiner, A. R. , Gnagy, E. M. , Pelham, W. E., 3rd , . . . Murphy, S. A. ( 2016 ). Treatment sequencing for childhood ADHD: A multiple-randomization study of adaptive medication and behavioral interventions.   Journal of Clinical Child and Adolescent Psychology , 45 , 396–415.

Perepletchikova, F. , & Kazdin, A. E. ( 2005 ). Treatment integrity and therapeutic change: Issues and research recommendations.   Clinical Psychology: Science and Practice , 12 , 365–383.

Read, K. , Kendall, P. C. , Carper, M. , & Rausch, J. R. ( 2013 ). Statistical methods for use in the analysis of randomized clinical trials using a pretest, posttest, follow-up (PPF) design. In J. S. Comer & P. C. Kendall (Eds.), Oxford handbook of research strategies for clinical psychology . Oxford University Press: New York.

Rounsaville, B. J. , Carroll, K. M. , & Onken, L. A. ( 2001 ). A stage model of behavioral therapies research: Getting started and moving on from stage I.   Clinical Psychology: Science and Practice , 8 (2), 133–142.

Silverman, W. K. , & Ollendick, T. H. ( 2005 ). Evidence-based assessment of anxiety and its disorders in children and adolescents.   Journal of Clinical Child and Adolescent Psychology , 34 , 380–411.

Suveg, C. , Kendall, P. C. , Comer, J. S. , & Robin, J. A. ( 2006 ). Emotion-focused cognitive-behavioral therapy for anxious youth: A multiple-baseline evaluation.   Journal of Contemporary Psychotherapy , 36 , 77–85.

Wu, P. , Goodwin, R.   Comer, J. S. , Hoven, C. , & Cohen, P. ( 2010 ). The relationship between anxiety disorders and substance use among adolescents in the community: Specificity and gender differences.   Journal of Youth and Adolescence , 39 , 177–188.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 2: Psychological Research

Experimental Research

As you’ve learned, the only way to establish that there is a cause-and-effect relationship between two variables is to conduct a scientific experiment. Experiment has a different meaning in the scientific context than in everyday life. In everyday conversation, we often use it to describe trying something for the first time, such as experimenting with a new hairstyle or new food. However, in the scientific context, an experiment has precise requirements for design and implementation.

Video 2.10 Experimental Research Design  provides explanations and examples for correlational research.

The Experimental Hypothesis

In order to conduct an experiment, a researcher must have a specific hypothesis to be tested. As you’ve learned, hypotheses can be formulated either through direct observation of the real world or after careful review of previous research. For example, if you think that children should not be allowed to watch violent programming on television because doing so would cause them to behave more violently, then you have basically formulated a hypothesis—namely, that watching violent television programs causes children to behave more violently. How might you have arrived at this particular hypothesis? You may have younger relatives who watch cartoons featuring characters using martial arts to save the world from evildoers, with an impressive array of punching, kicking, and defensive postures. You notice that after watching these programs for a while, your young relatives mimic the fighting behavior of the characters portrayed in the cartoon. Seeing behavior like this right after a child watches violent television programming might lead you to hypothesize that viewing violent television programming leads to an increase in the display of violent behaviors. These sorts of personal observations are what often lead us to formulate a specific hypothesis, but we cannot use limited personal observations and anecdotal evidence to test our hypothesis rigorously. Instead, to find out if real-world data supports our hypothesis, we have to conduct an experiment.

Designing an Experiment

The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The  experimental group  gets the experimental manipulation—that is, the treatment or variable being tested (in this case, violent TV images)—and the  control group does not. Because experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to experimental manipulation rather than chance.

In our example of how violent television programming might affect violent behavior in children, we have the experimental group view violent television programming for a specified time and then measure their violent behavior. We measure the violent behavior in our control group after they watch nonviolent television programming for the same amount of time. It is important for the control group to be treated similarly to the experimental group, with the exception that the control group does not receive the experimental manipulation. Therefore, we have the control group watch non-violent television programming for the same amount of time as the experimental group.

We also need to define precisely, or operationalize, what is considered violent and nonviolent. An  operational definition  is a description of how we will measure our variables, and it is important in allowing others to understand exactly how and what a researcher measures in a particular experiment. In operationalizing violent behavior, we might choose to count only physical acts like kicking or punching as instances of this behavior, or we also may choose to include angry verbal exchanges. Whatever we determine, it is important that we operationalize violent behavior in such a way that anyone who hears about our study for the first time knows exactly what we mean by violence. This aids peoples’ ability to interpret our data as well as their capacity to repeat our experiment should they choose to do so.

Once we have operationalized what is considered violent television programming and what is considered violent behavior from our experiment participants, we need to establish how we will run our experiment. In this case, we might have participants watch a 30-minute television program (either violent or nonviolent, depending on their group membership) before sending them out to a playground for an hour where their behavior is observed and the number and type of violent acts are recorded.

Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias.  Experimenter bias  refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which child was in which group, it might influence how much attention they paid to each child’s behavior as well as how they interpreted that behavior. By being blind to which child is in which group, we protect against those biases. This situation is a  single-blind study , meaning that the participants are unaware as to which group they are in (experiment or control group) while the researcher knows which participants are in each group.

In a  double-blind study , both the researchers and the participants are blind to group assignments. Why would a researcher want to run a study where no one knows who is in which group? Because by doing so, we can control for both experimenter and participant expectations. If you are familiar with the phrase  placebo effect , you already have some idea as to why this is an important consideration. The placebo effect occurs when people’s expectations or beliefs influence or determine their experience in a given situation. In other words, simply expecting something to happen can actually make it happen.

Why is that? Imagine that you are a participant in this study, and you have just taken a pill that you think will improve your mood. Because you expect the pill to have an effect, you might feel better simply because you took the pill and not because of any drug actually contained in the pill—this is the placebo effect.

To make sure that any effects on mood are due to the drug and not due to expectations, the control group receives a placebo (in this case, a sugar pill). Now everyone gets a pill, and once again, neither the researcher nor the experimental participants know who got the drug and who got the sugar pill. Any differences in mood between the experimental and control groups can now be attributed to the drug itself rather than to experimenter bias or participant expectations.

Video 2.11 Introduction to Experimental Design introduces fundamental elements for experimental research design.

Independent and Dependent Variables

In a research experiment, we strive to study whether changes in one thing cause changes in another. To achieve this, we must pay attention to two important variables, or things that can be changed, in any experimental study: the independent variable and the dependent variable. An independent variable is manipulated or controlled by the experimenter. In a well-designed experimental study, the independent variable is the only important difference between the experimental and control groups. In our example of how violent television programs affect children’s display of violent behavior, the independent variable is the type of program—violent or nonviolent—viewed by participants in the study (Figure 2.11). A dependent variable is what the researcher measures to see how much effect the independent variable had. In our example, the dependent variable is the number of violent acts displayed by the experimental participants.

Figure 2.1 1  In an experiment, manipulations of the independent variable are expected to result in changes in the dependent variable.

We expect that the dependent variable will change as a function of the independent variable. In other words, the dependent variable  depends  on the independent variable. A good way to think about the relationship between the independent and dependent variables is with this question: What effect does the independent variable have on the dependent variable? Returning to our example, what effect does watching a half-hour of violent television programming or nonviolent television programming have on the number of incidents of physical aggression displayed on the playground?

Selecting and Assigning Experimental Participants

Now that our study is designed, we need to obtain a sample of individuals to include in our experiment. Our study involves human participants, so we need to determine who to include.  Participants are the individuals of psychological research, and as the name implies, individuals who are involved in psychological research actively participate in the process. Often, psychological research projects rely on college students to serve as participants. In fact, the vast majority of research in psychology subfields has historically involved students as research participants (Arnett, 2008; Sears, 1986). But are college students truly representative of the general population? College students tend to be younger, more educated, more liberal, and less diverse than the general population. Although using students as test subjects is an accepted practice, relying on such a limited pool of research participants can be problematic because it is difficult to generalize findings to the larger population.

Our hypothetical experiment involves children, and we must first generate a sample of child participants. Samples are used because populations are usually too large to reasonably involve every member in our particular experiment. If possible, we should use a random sample (there are other types of samples, but for the purposes of this chapter, we will focus on random samples). A random sample is a subset of a larger population in which every member of the population has an equal chance of being selected. Random samples are preferred because if the sample is large enough we can be reasonably sure that the participating individuals are representative of the larger population. This means that the percentages of characteristics in the sample—sex, ethnicity, socioeconomic level, and any other characteristics that might affect the results—are close to those percentages in the larger population.

In our example, let’s say we decide our population of interest is fourth graders. But all fourth graders is a very large population, so we need to be more specific; instead, we might say our population of interest is all fourth graders in a particular city. We should include students from various income brackets, family situations, races, ethnicities, religions, and geographic areas of town. With this more manageable population, we can work with the local schools in selecting a random sample of around 200 fourth-graders that we want to participate in our experiment.

In summary, because we cannot test all of the fourth graders in a city, we want to find a group of about 200 that reflects the composition of that city. With a representative group, we can generalize our findings to the larger population without fear of our sample being biased in some way.

Figure  2.12  Researchers may work with (a) a large population or (b) a sample group that is a subset of the larger population.

Now that we have a sample, the next step of the experimental process is to split the participants into experimental and control groups through random assignment. With random assignment, all participants have an equal chance of being assigned to either group. There is statistical software that will randomly assign each of the fourth graders in the sample to either the experimental or the control group.

Random assignment is critical for sound experimental design. With sufficiently large samples, random assignment makes it unlikely that there are systematic differences between the groups. So, for instance, it would be improbable that we would get one group composed entirely of males, a given ethnic identity, or a given religious ideology. This is important because if the groups were systematically different before the experiment began, we would not know the origin of any differences we find between the groups: Were the differences preexisting, or were they caused by manipulation of the independent variable? Random assignment allows us to assume that any differences observed between experimental and control groups result from the manipulation of the independent variable.

Use this  online tool to generate randomized numbers instantly and to learn more about random sampling and assignments.

Issues to Consider

While experiments allow scientists to make cause-and-effect claims, they are not without problems. True experiments require the experimenter to manipulate an independent variable, and that can complicate many questions that psychologists might want to address. For instance, imagine that you want to know what effect sex (the independent variable) has on spatial memory (the dependent variable). Although you can certainly look for differences between males and females on a task that taps into spatial memory, you cannot control a person’s sex. We categorize this type of research approach as quasi-experimental and recognize that we cannot make cause-and-effect claims in these circumstances.

Experimenters are also limited by ethical constraints. For instance, you would not be able to conduct an experiment designed to determine if experiencing abuse as a child leads to lower levels of self-esteem among adults. To conduct such an experiment, you would need to randomly assign some experimental participants to a group that receives abuse, and that experiment would be unethical.

Interpreting Experimental Findings

Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. The statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this experiment 100 times, we would expect to find the same results at least 95 times out of 100.

The greatest strength of experiments is the ability to assert that any significant differences in the findings are caused by the independent variable. This occurs because random selection, random assignment, and a design that limits the effects of both experimenter bias and participant expectancy should create groups that are similar in composition and treatment. Therefore, any difference between the groups is attributable to the independent variable, and now we can finally make a causal statement. If we find that watching a violent television program results in more violent behavior than watching a nonviolent program, we can safely say that watching violent television programs causes an increase in the display of violent behavior.

the group of participants in an experiment who receive the independent variable

a comparison group that is equivalent to the experimental group, but is not given the independent variable

the possibility that a researcher’s expectations might skew the results of the study

the participants are unaware as to which group they are in (experiment or control group) while the researcher knows which participants are in each group

both the researchers and the participants are blind to group assignments

occurs when people's expectations or beliefs influence or determine their experience in a given situation

variable that is manipulated or controlled by the experimenter

what the researcher measures to see how much effect the independent variable had

individuals who are involved in psychological research actively participate in the process

every member of the population an equal chance of being selected for the sample

Child and Adolescent Development Copyright © 2023 by Krisztina Jakobsen and Paige Fischer is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Librarians/Admins

• EBSCOhost Collection Manager
• EBSCO Experience Manager
• EBSCO Connect
• Start your research
• EBSCO Mobile App

Clinical Decisions Users

• DynaMed Decisions
• Dynamic Health
• Waiting Rooms
• NoveList Blog

Child Development & Adolescent Studies

This bibliographic database is today's source for references to the current and historical literature related to growth and development of children through the age of 21.

Indexed Source Types Include:

• Academic journals
• Conference papers
• Working papers

Customers also bought:

The Swedish Twin study of CHild and Adolescent Development: the TCHAD-study

Affiliation.

• 1 Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Sweden. [email protected]
• PMID: 17539366
• DOI: 10.1375/twin.10.1.67

The Swedish Twin study of CHild and Adolescent Development (TCHAD) is a longitudinal study of how genes and environments contribute to development of health and behavioral problems from childhood to adulthood. The study includes 1480 twin pairs followed since 1994, when the twins were 8 to 9 years old. The last data collection was in 2005 when the twins were 19 to 20 years old. Both parents and twins have provided data. In this article we describe the sample, data collections, and measures used. In addition, we provide some key findings from the study, focusing on antisocial behavior, criminality, and psychopathic personality.

Publication types

• Research Support, Non-U.S. Gov't
• Antisocial Personality Disorder* / genetics
• Antisocial Personality Disorder* / psychology
• Follow-Up Studies
• Social Behavior Disorders* / genetics
• Social Behavior Disorders* / psychology
• Surveys and Questionnaires*
• Twins* / genetics
• Twins* / psychology

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

College of Liberal Arts and Sciences

Human Development and Family Sciences

Child and adolescent development.

Strengths of UConn HDFS research in this area include:

• An overarching lifespan perspective: Faculty study individuals during infancy, early childhood, middle childhood, across adolescence, and into the transition to adulthood. Faculty also consider precursors of development during childhood and adolescence, and adult outcomes of development during this period
• The study of development in context: Faculty examine child and adolescent development in a range of contexts, including families, parent work environment, friends and peers, childcare and schools, housing, culture, juvenile justice and child welfare systems, and economic status
• The consideration of multiple domains: Faculty study peer relationships; bullying, aggression, and peer victimization; intergroup stereotypes and attitudes; multiracial child development; math, spatial, and verbal learning during early childhood; moral reasoning; emotion regulation; mental health; social and emotional development; prosocial development; romantic relationships; sexuality; communication about sexuality and other topics; risk behaviors; substance use; child maltreatment and foster care; youth mentoring and positive youth development
• The study of diverse populations: Faculty study Latino American and immigrant children; multiracial youth; children in the United States, Bangladesh, Botswana, Germany, Guatemala, Israel, Kenya, Netherlands, and Philippines; children in low-income families; children involved with juvenile justice and child welfare; deaf children; and sexual/gender minority youth.
• Theory-driven research:  Faculty test existing theories and develop new theories to address unanswered questions
• A focus on improving children’s and adolescents’ lives: Faculty develop real world interventions for children, adolescents, families, and schools. Specific interventions include mindfulness and compassion-based social emotional programs to reduce stereotyping in Israeli-Jewish school children; peer mentorship in the child welfare system; interventions for behaviorally challenging students in kindergarten through high school; family-centered interventions for parents of adolescents with chronic pain; home visiting programs for improving maternal, infant, and early child wellbeing; supportive housing for families with children in child welfare; higher education preparation and support for youth in foster care

HDFS students also have access to a number of other health and prevention related resources both in and outside of the department, including the Center for Applied Research in Human Development , the Center for the Study of Interpersonal Acceptance & Rejection ; the Center for the Study of Culture, Health and Human Development ; the  Institute for Collaboration on Health, Intervention, and Policy , the  Rudd Center for Food Policy and Obesity ; the Collaboratory on School and Child Health , and the UConn Center for Excellence in Developmental Disabilities . Students can also concurrently pursue a range of relevant certificate options, including, Culture, Health, and Human Development or School-wide Positive Behavior Support . The Graduate School website has a full list of all available Certificate Programs for Master's and Doctoral students.

• Open access
• Published: 24 October 2013

Research priorities for child and adolescent physical activity and sedentary behaviours: an international perspective using a twin-panel Delphi procedure

• Lauren Gillis 1 ,
• Grant Tomkinson 1 ,
• Timothy Olds 1 ,
• Carla Moreira 2 ,
• Candice Christie 3 ,
• Claudio Nigg 4 ,
• Ester Cerin 5 ,
• Esther Van Sluijs 6 ,
• Gareth Stratton 7 ,
• Ian Janssen 8 ,
• Jeremy Dorovolomo 9 ,
• John J Reilly 10 ,
• Jorge Mota 2 ,
• Kashef Zayed 11 ,
• Kent Kawalski 12 ,
• Lars Bo Andersen 13 ,
• Manuel Carrizosa 14 ,
• Mark Tremblay 15 ,
• Michael Chia 16 ,
• Mike Hamlin 17 ,
• Non Eleri Thomas 18 ,
• Ralph Maddison 19 ,
• Stuart Biddle 20 ,
• Trish Gorely 21 ,
• Vincent Onywera 22 &
• Willem Van Mechelen 23  

International Journal of Behavioral Nutrition and Physical Activity volume  10 , Article number:  112 ( 2013 ) Cite this article

98k Accesses

44 Citations

43 Altmetric

Metrics details

The quantity and quality of studies in child and adolescent physical activity and sedentary behaviour have rapidly increased, but research directions are often pursued in a reactive and uncoordinated manner.

To arrive at an international consensus on research priorities in the area of child and adolescent physical activity and sedentary behaviour.

Two independent panels, each consisting of 12 experts, undertook three rounds of a Delphi methodology. The Delphi methodology required experts to anonymously answer questions put forward by the researchers with feedback provided between each round.

The primary outcome of the study was a ranked set of 29 research priorities that aimed to be applicable for the next 10 years. The top three ranked priorities were: developing effective and sustainable interventions to increase children’s physical activity long-term; policy and/or environmental change and their influence on children’s physical activity and sedentary behaviour; and prospective, longitudinal studies of the independent effects of physical activity and sedentary behaviour on health.

Conclusions

These research priorities can help to guide decisions on future research directions.

Recent research has shown that both physical activity and sedentary behaviour are associated with a wide range of current and future health outcomes [ 1 – 3 ]. In fact, physical activity and sedentary behaviour are two independent and not mutually exclusive behaviours with different effects on health outcomes [ 4 ]. In the short term, physical activity has been shown to be moderately and positively associated with bone health, aerobic fitness, blood lipid levels, self-esteem, mental activity and fundamental movement skills in children and adolescents [ 1 – 3 , 5 ]. In the long term, both physical activity and sedentary behaviour have been identified as major, independent, modifiable risk factors for mortality and morbidity from many chronic, non-communicable and potentially preventable diseases [ 6 – 9 ]. New evidence also suggests that the relation between sedentary behaviour and all-cause end cardiovascular disease mortality is independent of physical activity levels [ 7 ].

Chronic diseases place a large economic burden on health services and impose significant costs on society (e.g. premature death, underappreciated economic effects and greater reliance on treatment) [ 8 ]. Although the ill effects of chronic disease largely manifest in adulthood, it is increasingly understood that the development typically begins in childhood or adolescence [ 9 ]. Therefore, physical activity levels and sedentary behaviour performed in the early years could potentially influence the development of disease later on in life.

At present, a large quantity of research is being conducted into the physical activity and sedentary behaviour of children, yet the research community remains challenged to provide a solid evidence base [ 10 ]. This is in part due to a lack of international research collaboration and a high degree of study repetition. The aim of this study therefore was to arrive at a set of international research priorities for physical activity and sedentary behaviour to guide more meaningful and focussed research. Specifically, this study aimed to answer the following research question: “What are the most important international research issues for the next 10 years in child and adolescent physical activity and sedentary behaviour?” Agreement on research priorities may help to inform evidence-based policy, guide funding allocation, and direct research options for postgraduate students [ 11 , 12 ].

Existing literature

To identify existing evidence in this area, a systematic review of the English and non-English literature was performed using the following search terms: physical activit* OR motor activity (MeSH) OR sedentary behavio* AND child* OR adolescen* OR youth* AND research priorit* OR research agenda* OR research issue*. The databases PsychINFO (1887–), SPORTDiscus (1949–), Cochrane (1992–), CINAHL (1937–), ERIC (1966–) and PubMed (1950–) were searched in May 2012. Additional studies were also identified by contacting experts, Google searching and identifying potential studies in the reference lists of identified studies. Only four previously published papers that arrived at research priorities in child physical activity and/or sedentary behaviour were identified [ 11 , 13 – 15 ]. A working paper by Bull et al. [ 11 ] identified research priorities in physical activity with a focus on low to middle income countries. Evenson and Mota [ 13 ] highlighted research on the determinants and outcomes of physical activity and made recommendations for future study designs. Mountjoy et al. [ 15 ] identified existing gaps in physical activity research for children, with a focus on the need for greater collaboration between sport and existing programmes. The final study by Fulton et al. [ 14 ] had two aims. Firstly, the study aimed to review the current knowledge of existing methods for assessing physical activity and sedentary behaviour. Secondly, on the basis of this, the study aimed to set research priorities on the use of reliable and valid measurement tools to assess physical activity and sedentary behaviour in children aged 2–5 years.

While these studies were valuable contributions, they also had many limitations, including unsystematic participant selection, unstructured data collection procedures, and limited reporting on the process followed to arrive at the research priorities. Furthermore, the participants involved in the decision-making processes did not always represent the broader community of researchers, either from a geographical or institutional point of view. In addition, the anonymity of participants was not maintained during the consensus process. These limitations warranted a further study with an aim to arrive at a set of research priorities by employing a structured and rigorous methodology and improving reporting quality.

Methodology

Ethical approval for all aspects of the methodology was granted by the University of South Australia Human Research Ethics Committee in September 2011.

This study employed a Delphi procedure. This procedure is appropriate for research questions which cannot be answered with complete certainty, but rather by the subjective opinion of a collective group of informed experts [ 16 ]. It allowed systematic refinement of the experts’ opinions over the course of several rounds while minimising confounding factors present in other group response methods [ 17 – 20 ].

The experts who participated in the Delphi procedure were identified by a 3–step procedure. Firstly, the lead study investigators independently recommended known researchers for the study. Secondly, a lengthy and extensive search was carried out to identify potential researchers from every world region and sub-region. Identifying potential experts from these regions involved searching for staff of relevant international bodies, government departments, non-government organisations, professional organisations and educational institutions. Thirdly, following email communication with the experts who have previously been identified, new experts were referred to the study investigators.

Once participants had been identified, it was important to determine their eligibility for inclusion in the study. Thus they were assessed using pre-determined inclusion and exclusion criteria. To be eligible, a researcher had to be an author of at least one peer-reviewed scientific publication on the physical activity or sedentary behaviour of children or adolescents, and must hold (at the time of selection) a senior position in their organisation. In addition, the experts were deliberately chosen to give geographical coverage of every world region and sub-region. Relevant information was gathered from staff homepages, Scopus author searches, the Journal and Author Name Estimator ( http://www.biosemantics.org/jane/ ) and other relevant Internet searches to ascertain whether a researcher met these criteria.

Forty-six eligible experts were invited to participate, with each sent information and consent forms via email. As a whole, these participants were representative of every region and sub-region. Of those invited, 20 did not respond to the invitation, two declined to participate, and 24 returned signed consent forms. An outline of this process is illustrated in Figure  1 .

Purposive sampling process undertaken.

The 24 participating experts (17 male and 7 female) were randomly allocated to either Panel A or Panel B and assigned identification code names accordingly. Furthermore the following major institution types were represented by the selected experts; educational institutions, government organisations, non-government organisations, professional organisations and community organisations.

The Delphi procedure used three rounds [ 21 ], each consisting of data collection, data analysis and controlled feedback. The survey was administered entirely online using a Survey Gizmo questionnaire. A novel feature of this study was the use of two parallel panels of experts. The existence of an alternate panel was only made known to the participants in Round 3, when each panel was asked to rank the priorities of the other panel. This allowed quantitative comparisons to be made between each panel’s rankings of each research issue and cross-validated the rankings of research priorities developed by each panel.

To commence each round, experts were sent an email containing a direct link to the online questionnaire. Briefly, Round 1 required each expert to answer the question “What are the five most important research issues for the next 10 years in the area of child and adolescent physical activity and sedentary behaviour?” Each expert put forward five research issues which they believed were priorities in the area. They also provided a brief description of each issue and reasons why they believed the issue to be a priority. The three study investigators reviewed all issues that were provided by each panel, with common issues combined into a single issue. The experts were then fed back their panel’s list of research issues and asked to ensure that the five research issues they provided were accurately represented.

Round 2 then asked experts to “review the research issues put forward in Round 1 and rate how important they believe each issue is for global research in child and adolescent physical activity and sedentary behaviour”. Experts rated each research issue independently using a 5-point Likert scale (5 = very important, 4 = important, 3 = moderately important, 2 = of little importance and 1 = unimportant). The three study investigators then short-listed each panel’s research issues to 20 according to those with highest mean Likert scale ratings. Following this, the top 20 research issues from each panel were fed back to the experts of the relevant panels.

In Round 3, experts were first asked to “rank their panel’s top 20 research issues in order of perceived international importance in child and adolescent physical activity and sedentary behaviour over the next 10 years”. The experts were then similarly asked to rank the alternate panel’s top 20 research priorities. The data analysis procedure was as follows. Firstly, the overall sum of each panel’s rankings was calculated for Panel A and Panel B’s top 20 research issues. Secondly, the two lists of research issues were combined with common issues provided by both panels merged. This resulted in 29 unique issues. Thirdly, the experts’ individual rankings for each research issue were summed. This allowed the issues to be ranked according to the sum of Panel A and Panel B’s overall rankings for each issue. Intra-panel agreement was quantified using Spearman’s rho by creating a matrix to compare individuals’ rankings to one another within the same panel. Inter-panel agreement was also quantified using Spearman’s rho to compare the overall sum and rank for each issue between panels.

Expert demographics

All 24 experts completed the three Delphi rounds. Data was collected on the 24 experts’ geographical distributions, institutional affiliations and years worked in the study area.

As a group, the 24 experts represented every geographical region and 12 sub-regions. This geographical distribution is illustrated in Figure  2 .

Geographical distributions of participating experts. The numbers indicate the number of participating experts from that region.

In terms of institutional affiliation, twenty-three experts acknowledged they were affiliated with an educational institution, eleven were affiliated with a professional organisation, six with an international organisation, six with a non-government organisation and four with a government organisation. It was noted that due to the nature of their work, experts were often affiliated with more than one institution type.

In regards to years worked in the study area, twelve experts had worked in for greater than 16 years, five had worked for 11 to 15 years, four had worked for 6 to10 years and three had worked for less than five years.

Results from Delphi rounds

In Round 1, each expert put forward five research issues. Collectively this provided a total of 120 issues across all 24 experts, with 60 for each panel. Following qualitative reduction of overlapping issues, 26 issues from Panel A and 34 issues from Panel B, were carried forward to Round 2. On reviewing the amended list, all exerts agreed that the issues they had raised were adequately represented.

From Round 2, the mean Likert-scale ratings were used to determine the top 20 issues for each panel. For Panel A, the mean Likert-scale ratings of the top 20 issues ranged from 3.5 to 5.0, with 18 of 20 issues having a median rating of >4.0 (“important”). For Panel B, the mean Likert-scale ratings of the top 20 issues ranged from 4.0 to 4.8, with all 20 research issues having a median rating of >4.0.

In Round 3, the 20 issues from Panel A and 20 issues from Panel B were qualitatively analysed to form one list. Eleven of each panel’s top 20 research issues were common to both panels and were therefore combined, with the remaining 18 issues (nine from each panel) unique. The resultant was a set of 29 unique research issues that were then ranked in order of importance by summing Panel A and Panel B’s rankings for each issue Table  1 .

There was only weak intra-panel agreement. The mean inter-individual rho ( ± 95% CI) was 0.20 ±0.05 for Panel A and 0.13 ±0.04 for Panel B. The average standard deviation of the rankings for individual issues was 5.1 (Panel A) and 5.3 (Panel B). When Panel B ranked Panel A’s issues, the correlation was very strong ( rho ± 95% CI: 0.79 ±0.17), and when Panel A ranked Panel B’s issues, the correlation was strong ( rho ± 95% CI: 0.52 ±0.31). Figures  3 and 4 clearly illustrate the correlations for each research issue.

Agreement between Panel A’s rankings and Panel B’s rankings of Panel A’s identified issues. The line shown is the identity line.

Agreement between Panel B’s rankings and Panel A’s rankings of Panel B’s identified issues. The line shown is the identity line.

Study outcomes

The primary outcome of this study was the development of 29 international research priorities in child and adolescent physical activity and sedentary behaviour. In order for the research priorities to be useful, it is important that they be neither too general nor too specific. The research priorities in this study appear broad enough to enable them to be transferable to researchers’ specific regions and contexts.

The final set of research priorities address a broad range of areas from epidemiology, determinants and correlates, through to intervention effectiveness and translational research. Of the 29 identified research priorities, ten related directly to translational research centred on intervention design and effectiveness. These focussed on specific behaviours (active transport, screen time, sport, physical education), settings (schools, communities, whole of population), or vehicles (mass advertising, policy). Translational research, centred on intervention design and effectiveness, can potentially guide governments and stakeholders to fund interventions that are the most effective, sustainable and transferable for changing behaviours [ 7 ]. This is important because to date, the research community has not been very successful at developing interventions for children and adolescents that bring about long-term and sustained change in health behaviours [ 10 ]. In addition, little attention has been given to the importance of the intervention setting and establishing what works in what situation and with whom [ 22 ].

Nine of the research priorities had a focus on capturing and quantifying the health benefits of engaging in physical activity and limiting sedentary behaviour, These research priorities were concerned with the impact of physical activity and sedentary behaviour on obesity, cognition, and general health and well being, and on describing behavioural patterns (across the day or the life-course or in specific populations such as pre-school children). Epidemiological research was considered important to address the cause, distribution and patterns of childhood physical activity and sedentary behaviour on current and future health [ 2 , 6 , 9 , 23 ].

Six research issues related to determinants and correlates research such as psychosocial and cultural/parental factors, the impact of technology, and the importance of enjoyment and lifestyle in general. Research that focuses on the determinants and correlates of behaviours is important. This is because while many correlates appear to be intuitively obvious, at present they have mixed support from high quality research [ 3 ].

Four issues did not fit into the aforementioned categories. They were related to the theory of behaviour change, injury prevention, measurement of behaviours and the physical education in culture of movement. Objective measurement of behaviours was ranked highly and is thought to be a “necessary first step for conducting meaningful epidemiological surveillance, public health research and intervention research” [ 14 ] p.124.

Strengths and limitations

Unlike previously identified priority reports [ 11 , 13 – 15 ] this study employed a Delphi method to arrive at a more valid set of research priorities. Strengths related to the Delphi method include participant blinding, iterative data collection and controlled feedback between rounds. For example, the identities and responses of the experts were anonymised so that the identified research priorities could not be dominated by certain individuals [ 24 ]. Furthermore, the provision of controlled feedback allowed experts to individually consider their views in light of their panel’s collective opinion.

Other strengths related to the methodology were the use of criterion and purposive sampling methods. This procedure meant that all participants held a senior position in their respective organisations and had published in the study area. In addition, experts collectively represented every major world region and a wide range of discipline areas, affiliations and interests. This approach meant that the identified research issues were more likely to reflect the most important physical activity and sedentary behaviour issues facing the children and adolescents worldwide.

A novel component of this study was split-panel approach, which allowed comparisons to be made between the rankings given by the two expert panels. The experts from each panel were taken from the same population, given the same study information, answered identical online questionnaires and participated simultaneously and independently. One can therefore be confident that comparing the Round 3 rankings of Panel A and Panel B experts would provide valid measures of inter-panel agreement.

The weak intra-panel agreement was weak, which is likely a reflection of the natural variation of individual’s opinions and areas of interest within the broad study area. This weak agreement could also highlight the advantages of the methodology which retained anonymity and used an online mode of data collection. There were fewer pressures to conform to others opinions due to decreased likelihood of peer dominance and status. Evidence to reinforce confidence in the results is the strong to very strong (rho = 0.52–0.79) inter-panel agreement. While experts were invited from every United Nations sub-region (United Nations 2011), no experts from the following sub-regions took part: Southern Africa, Middle Africa, Caribbean, Eastern Europe, Australia, Central Asia and Western Asia. This was significant because many of these sub-regions are heavily involved in physical activity and sedentary behaviour research. Consequently, caution should be applied when recommending that the identified research priorities truly provide a global perspective. Nonetheless, these research priorities provide an international context from which priorities at the regional, national and local levels can be developed.

In addition the priorities were set for the broad area of child and adolescent physical activity and sedentary behaviour. Due to the generality of this topic, it may be that the research priorities are not relevant when conducting research into minority populations. For example, children and adolescents with disabilities may warrant different research issues not identified in this study.

Implications for research

We hope that the identification of a set of ranked research priorities may contribute to more co-ordinated international research. For example, research priorities can help inform post-graduate students regarding where the current evidence gaps exist. This may be especially helpful for researchers who reside in less developed or marginalised research regions. In addition, encouraging more guided research can help to conceptualise how findings can be used as a basis for policy decisions. Lastly, research priorities can help to direct valuable funding into priority areas and away from studies on over-researched or lower priority topics.

This study engaged two panels of study experts in a three-round Delphi communication procedure. The outcome of this procedure was the identification of a ranked set of 29 research priorities in child and adolescent physical activity and sedentary behaviour. For example, the top three ranked priorities were: developing effective and sustainable interventions to increase children’s physical activity long-term; policy and/or environmental change and their influence on children’s physical activity and sedentary behaviour; and prospective, longitudinal studies of the independent effects of physical activity and sedentary behaviour on health. We hope these research priorities will help inform the spectrum of future studies undertaken, guide post-graduate study choices, guide allocation of funding to priority areas and assist with policy decisions.

Ekelund E, Heian F, Hagen KB, Abbott J, Nordheim L: Exercise to improve self-esteem in children and young people. Cochrane Database Syst Rev. 2004, Art. No. CD: 003683, doi:10.1002/14651858.CD003683.pub2, Iss. 1

Book   Google Scholar  

Lubans D, Morgan P, Cliff D, Barnett L, Okely A: Fundamental movement skills in children and adolescents: review of associated health benefits. Sports Med. 2010, 40 (12): 1019-1035. 10.2165/11536850-000000000-00000.

Article   Google Scholar  

Sallis J, Prochaska J, Taylor W: A review of correlates of physical activity of children and adolescents. Med Sci Sports Exerc. 2000, 32 (5): 963-975.

Article   CAS   Google Scholar  

Chinapaw M, Proper K, Brug J, et al: Relationship between young people’s sedentary behaviour and biomedical health indicators; a systematic review of prospective studies. Obes Rev. 2011, 12: 621-625. 10.1111/j.1467-789X.2011.00865.x.

Sibley B, Etnier J: The relationship between physical activity and cognition in children: a meta-analysis. Paediatr Exerc Sci. 2003, 15 (2): 243-256.

Google Scholar  

Bauman A, Owen N, Leslie E: Physical activity and health outcomes: epidemiological evidence, national guidelines and public health initiatives. Aust J Nutr Diet. 2000, 57: 229-232.

Katzmarzyk P: Physical activity, sedentary behaviour and health; paradigm paralysis or paradigm shift?. Diabetes. 2010, 59: 2717-2725. 10.2337/db10-0822.

World Health Organization: World Health Organization global strategy on diet, physical activity and health. 2004, Geneva: World Health Organization

Parsons T, Power C, Logan S, Summerbell C: Childhood predictors of adult obesity: a systematic review. Int J Obes Relat Disord. 1999, 23 (8): 1-107.

Van Sluijs E, McMinn A, Griffin S: Effectiveness of interventions to promote physical activity in children and adolescents: systematic review of controlled trials. Br J Med. 2007, 42 (8): 653-657.

Bull F, Bauman A, Hallal P, Kohl H, Tremblay M: A Prioritised Agenda for the Prevention and Control of Noncommunicable Disease. Paper 3. Research priorities. Physical activity with a focus on low and middle income countries. 2010, [ http://webcache.googleusercontent.com/search?q=cache:pptwZodFXI8J:www.world-heart-federation.org/fileadmin/user_upload/documents/Advocacy/Resources/Meetings_-_Activities _and_Partnerships/Research%2520priorities%2520-Physical%2520activity_01.pdf]

Nchinda T: Research capacity development for CVD prevention: the role of partnerships. Ethn Dis. 2003, 3 (2): 40-44.

Evenson K, Mota J: Progress and future directions on physical activity research among youth. J Phys Act Heal. 2011, 8 (2): 149-151.

Fulton J, Burgeson C, Perry G, Sherry B, Galuska D, Alexander M, Wechsler H, Caspersen C: Assessment of physical activity and sedentary behaviour in preschool-age children: priorities for research. Paediatr Exerc Sci. 2001, 13: 113-126.

Mountjoy M, Engebretsen L, Ekelund U, Brandl Bedenbeck HP, Boreham C, Biddle S, Armstrong N, Bo Andersen L, Mountjoy M, Hardman K, Hills A, Kahlmeier S, Kriemler S, Lambert E, Ljungqvist A, Matsudo V, McKary H, Micheli L, Pate R, Riddoch C, Schamasch P, Sundberg CJ, Tomkinson G, van Sluijs E, van Mechelen W: International Olympic Committee consensus statement on the health and fitness of young people through physical activity and sport. Br J Sports Med. 2011, 45: 839-848. 10.1136/bjsports-2011-090228.

Yousuf M: Using experts’ opinions through Delphi technique. Pract Assess Res Eval. 2007, 12 (4): Available online: http://pareonline.net/getvn.asp?v=12%26;n=4

Dalkey N, Helmer O: An experimental application of the Delphi method to the use of experts. Manag Sci. 1963, 9 (3): 458-467. 10.1287/mnsc.9.3.458.

Linstone H, Turoff M: The Delphi Method: Techniques and Applications. 2002, New Jersey: New Jersey’s Department of Information Systems

Sharma D, Nair S, Balasubramanian R: Analytical search of problems and prospects of power sector through Delphi study: case study of Kerala State, India. Energy State. 2003, 31: 1245-1255.

Sinha I, Smyth R, Williamson P: Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. Public Libr Sci Med. 2011, 8 (1): doi:10.1371/journal.pmed.1000393

Custer R, Scarcella J, Stewart B: The modified Delphi technique: a rotational modification. J Vocat Tech Educ. 1999, 15 (2): 1-10.

Strong W, Malina R, Blimkie C, Daniels S, Dishman R, Gutin B, Hergenroeder A, Must A, Nixon P, Pivarnik J, Rowland T, Trost S, Trudeau F: Evidence based physical activity for school-age youth. J Pediatr. 2005, 146 (6): 732-737. 10.1016/j.jpeds.2005.01.055.

Must A, Tybor D: Physical activity and sedentary behavior: a review of longitudinal studies of weight and adiposity in youth. Int J Obes. 2005, 29: 84-96.

Delbecq A, Van den Ven A, Gustafson D: Group techniques for program planning: A guide to nominal group and Delphi processes. 1975, Glenview: IL Scott Foresman

Download references

Acknowledgements

The authors would like to acknowledge the Health and Use of Time Group at the University of South Australia.

Author information

Authors and affiliations.

Health and Use of Time (HUT) Group, University of South Australia, Adelaide, South Australia, Australia

Lauren Gillis, Grant Tomkinson & Timothy Olds

Research Centre in Physical Activity, Health and Leisure, Faculty of Sport, University of Porto, Porto, Portugal

Carla Moreira & Jorge Mota

Ergonomics Unit, Rhodes University, Grahamstown, South Africa

Candice Christie

Department of Public Health Sciences, University of Hawaii at Manoa, Honolulu, HI, USA

Claudio Nigg

Institute of Human Performance, The University of Hong Kong, Hong Kong, China

Ester Cerin

MRC Epidemiology Unit, Institute of Metabolic Science, Addenbrooke’s Hospital, Cambridge, UK

Esther Van Sluijs

The Research Institute for Sport and Exercise Sciences, Liverpool John Moores University, Liverpool, UK

Gareth Stratton

School of Physical and Health Education, Queen's University, Kingston, Ontario, Canada

Ian Janssen

University of the South Pacific, Laucala Campus, Suva, Fiji, Islands

Jeremy Dorovolomo

Physical Activity for Health Research Group, School of Psychological Sciences and Health, University of Strathclyde, Jordanhill, Glasgow, UK

John J Reilly

Department of Physical Education, Sultan Qaboos University, Muscat, Sultanate of Oman

Kashef Zayed

Physical and Health Education and Department of Psychology, School of Exercise Science, University of Victoria, Victoria, BC, Canada

Kent Kawalski

Center for Research in Childhood Health, Institute of Sport Sciences and Clinical Biomechanics, University of Southern Denmark, Odense, 5230, Denmark

Lars Bo Andersen

Education Faculty, University of Extremadura, Avda de Elvas s/n, Badajoz, Spain

Manuel Carrizosa

Healthy Active Living and Obesity Research Group, Children's Hospital of Eastern Ontario Research Institute, Ottawa, Canada

Mark Tremblay

Physical Education & Sports Science, National Institute of Education, Nanyang Technological University, Singapore, Singapore

Michael Chia

Department of Social Science, Parks, Recreation, Tourism and Sport, Lincoln University, Christchurch, New Zealand

Mike Hamlin

Centre for Children and Young People's Health and Well-Being, School of Human and Health Sciences, Swansea University, Swansea, UK

Non Eleri Thomas

Clinical Trials Research Unit, School of Population Health, University of Auckland, Auckland, 1142, New Zealand

Ralph Maddison

School of Sport, Exercise & Health Sciences, Loughborough University, Loughborough, Leicestershire, UK

Stuart Biddle

Institute of Youth Sport, School of Sport and Exercise Sciences, Loughborough University, Loughborough, LE, 11 3TU, UK

Trish Gorely

Department of Exercise, Kenyatta University, Recreation and Sport Science, Nairobi, Kenya

Vincent Onywera

Department of Public & Occupational Health, EMGO Institute, VU University Medical Center, Amsterdam, The Netherlands

Willem Van Mechelen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Grant Tomkinson .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

The study was conceived by GT and TO. LG was primarily responsible for conducting the participant selection process and the three rounds of data collection. LG, GT and TO were each involved in data analysis. LG produced the first draft of the paper with all other authors providing sections and critically reviewing the paper. All authors approved submission.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, authors’ original file for figure 4, rights and permissions.

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Gillis, L., Tomkinson, G., Olds, T. et al. Research priorities for child and adolescent physical activity and sedentary behaviours: an international perspective using a twin-panel Delphi procedure. Int J Behav Nutr Phys Act 10 , 112 (2013). https://doi.org/10.1186/1479-5868-10-112

Download citation

Received : 26 March 2013

Accepted : 09 September 2013

Published : 24 October 2013

DOI : https://doi.org/10.1186/1479-5868-10-112

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Physical activity
• Sedentary behaviour
• Research priorities
• Adolescents

International Journal of Behavioral Nutrition and Physical Activity

ISSN: 1479-5868

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Future Students
• Current Students
• Faculty/Staff

News and Media

• News & Media Home
• Research Stories
• School's In
• In the Media

You are here

Stanford-led study links school environment to brain development.

For decades, researchers have linked differences in school-age children’s brain development to their out-of-school environment, using indirect socioeconomic factors such as parental income and neighborhood characteristics. 

In a new paper , researchers from Stanford Graduate School of Education (GSE) demonstrate for the first time that, even when controlling for those other factors, there is a direct link between a child’s school environment and the development of their white matter, or the network of nerve fibers that allows different parts of the brain to communicate. 

In other words, schools that do better than average at promoting learning are showing greater year-by-year advances in brain development, even for students coming from a wide range of socioeconomic environments. 

For their study, the authors, including GSE doctoral candidate Ethan Roy , Professor Bruce McCandliss , and Associate Professor Jason Yeatman , leveraged data from the Adolescent Brain Cognitive Development (ABCD) Study, the largest long-term study of brain development and child health in the United States, and the Stanford Education Data Archive (SEDA), a national database of academic performance developed by the Educational Opportunity Project at Stanford University. 

Their findings show that children who attend higher-performing schools have accelerated white matter development, including in an area of the brain closely associated with reading skills.

Roy said the results, published in Developmental Cognitive Neuroscience on April 26, were “striking.”

“What jumped off the page for us is that, even when controlling for things like parental income, parental education, neighborhood context, and household conflict levels, we were still able to observe a significant relationship between the school environment of an individual and growth properties of their brain,” he said.

Filling a gap in learning science research

Yeatman, who along with McCandliss serves as an advisor to Roy, said the study is the first to show how variation in the educational opportunities afforded to children is related to brain development.

“Essentially, two children from similar families who are born on two sides of a school boundary have measurable differences in how their brains wire together,” said Yeatman, who holds a joint faculty appointment at the GSE and Stanford Medicine, is a faculty affiliate of the Stanford Accelerator for Learning , and directs the Brain Development & Education Lab and Rapid Online Assessment of Reading . 

The study looked at fractional anisotropy, a measure of how water moves through brain tissue and an indication of how insulated, or myelinated, a neuron’s axons are (higher myelination increases the speed of transmission between neurons and is associated with improved learning). The observational results show that fractional anisotropy is directly linked to a school’s national grade equivalence score, or a measure of how third graders from that school perform compared with the national average.

The paper fills a gap in learning science research. Although past studies have linked socioeconomic status to white matter development, they have not been able to focus in on specific attributes of a child’s development, such as the school they attend. Other research — including from Yeatman’s lab — has shown that educational interventions can lead to changes in white matter, but those have been relatively small-scale studies with participants who are not representative of the broader population. 

“This is one of the first cases where we can measure the thing we actually care about at the population level,” Yeatman said.

The authors also trained a deep learning model to conduct a global analysis of white matter, finding that children who attend schools with higher SEDA scores had brains that appeared developmentally “more mature” than their chronological age.

A measurable impact

The implications are “potentially game-changing,” said McCandliss, who directs the Stanford Educational Neuroscience Initiative (SENSI) and is a faculty affiliate of the Stanford Accelerator for Learning. 

“National discussions of the importance of elementary school quality have never before been framed in terms of having a measurable impact on physical brain development of our young children,” he said. “I think this changes the frame of the discussion and decision-making around the impact of inequity.”

The study was only possible because of the comprehensive data included in the ABCD Study and SEDA, the researchers said. McCandliss, an investigator in the ABCD Study, first approached the ABCD team leaders about linking the SEDA data with the ABCD data in 2018, and his SENSI team spent about two years creating the resulting “crosswalk.” 

McCandliss called the ABCD study a “dream come true,” and the linked data a way to “finally” answer “elusive questions about how inequities in educational opportunities may actually be changing the course of physical and functional brain development during the vulnerable elementary school years across the nation.”

To analyze the brain white matter from the MRI data included in the ABCD study, the authors used pyAFQ , an open-source software developed by Yeatman’s lab. “It was a really fruitful collaboration across both labs,” Roy said.

The authors hope their methods and the newly linked ABCD and SEDA data, which is now freely available to a community of registered researchers around the world, will allow other scholars to pursue their own ideas and hypotheses at the intersection of education and neuroscience.

Yeatman said the methods and data used in the study will allow researchers to be more precise about environmental factors linked to brain development and the mechanisms behind those connections.

“The environment influences brain development,” he said. “That’s obvious. But what about the environment influences brain development? This is the first step in actually unraveling that specificity.”

More Stories

⟵ Go to all Research Stories

Get the Educator

Subscribe to our monthly newsletter.

Stanford Graduate School of Education

482 Galvez Mall Stanford, CA 94305-3096 Tel: (650) 723-2109

• Contact Admissions
• GSE Leadership
• Site Feedback
• Web Accessibility
• Career Resources
• Faculty Open Positions
• Explore Courses
• Academic Calendar
• Office of the Registrar
• Cubberley Library
• StanfordWho
• StanfordYou

Improving lives through learning

• Stanford Home
• Maps & Directions
• Search Stanford
• Emergency Info
• Terms of Use
• Non-Discrimination
• Accessibility

© Stanford University , Stanford , California 94305 .

• Open access
• Published: 21 May 2024

MyHospitalVoice – a digital tool co-created with children and adolescents that captures patient-reported experience measures: a study protocol

• Jane Hybschmann 1 ,
• Jette Led Sørensen 1 , 2 ,
• Jakob Thestrup 1 ,
• Helle Pappot 3 ,
• Kirsten Arntz Boisen 4 ,
• Thomas Leth Frandsen 1 &
• Line Klingen Gjærde 1  

Research Involvement and Engagement volume  10 , Article number:  49 ( 2024 ) Cite this article

Metrics details

Children and adolescents have the right to participate in decisions concerning their health and express their views, also regarding hospital experiences. Patient-reported experience measures (PREMs) are valuable tools for systematically incorporating patient voices into healthcare systems. New developments have focused on PREMs for children and adolescents, though they are more commonly used in adults. A recent systematic review mapping their use for children and adolescents indicates a growing interest in this area. However, most PREMs are completed by proxy, in this case parents, so they do not necessarily reflect children’s experiences or align with their rights. Innovation is required to support and engage children and adolescents in responding to these types of questionnaires.

Collaborating with children and adolescents (4–17 years), the primary aim of this study is to develop and validate the tool MyHospitalVoice containing digital and developmentally appropriate PREMs. The secondary aim is to document and evaluate the approaches used to involve children and adolescents and to assess the impact of their involvement. Based on the European Organisation for Research and Treatment of Cancer framework, we will divide its development and validation into four phases. First, we will discuss PREM items with children and adolescents, who will select and prioritise what they perceive as most important. Second, we will create items targeting different age groups (4–7, 8–12, and 13–17 years) and design a responsive digital interface with child and youth friendly ways of responding to the questionnaires. Third, we will explore how children and adolescents perceive MyHospitalVoice using cognitive interviewing techniques and other age-appropriate methods. Last, we will pilot test MyHospitalVoice to explore patient experiences and response rates. In each phase, children and adolescents will play an active role. We will involve young adults as peer researchers in the project group to ensure that their perspectives are part of the decision-making process.

This project will contribute to research on co-creating with children and adolescents and enhance our understanding of their patient experiences. A validated tool like MyHospitalVoice can help improve quality of care by translating the needs and preferences of children and adolescents into clinical practice.

Plain English summary

We will collaborate with children and adolescents (4–17 years) to develop a digital tool called MyHospitalVoice. We believe that we can improve the quality of children and adolescents’ healthcare encounters by using MyHospitalVoice to incorporate their needs and preferences into clinical practice. Children and adolescents have the right to be involved in decisions about their health, also when in hospital. Being in hospital can be tough for children and adolescents and adults may not always pay enough attention to what children and adolescents experience. Giving children and adolescents an easy way to share their needs and preferences may protect their rights and ensure that healthcare professionals consider them more. One way to let patients share their hospital experience is by using questionnaires called patient-reported experience measures (PREMs). While similar tools are used for adults, there is a growing recognition of the need for them specifically tailored to the perspectives of children and adolescents. This project will develop a digital tool called MyHospitalVoice, which consists of PREMs designed for and with children and adolescents. We will follow a four-phase process based on established guidelines. We will gather existing PREMs and let children and adolescents decide which questions are most important to them. Next, we will create questions that are suitable for different age groups (4–7, 8–12, and 13–17 years) and design a user-friendly digital interface with playful features to make the questionnaire more engaging. Finally, we will assess how children and adolescents perceive MyHospitalVoice and end by doing a pilot test.

Peer Review reports

Children and adolescents have a right to participate in matters regarding themselves and their health, just as they have a right to express their own views [ 1 , 2 ]. These rights also apply to children and adolescents in hospital. Moreover, healthcare systems have a responsibility to facilitate and support these rights. Hospitalisation places children, adolescents, and their families in a particularly vulnerable life situation, where there may be little awareness of the child’s rights or the capacity to prioritise them. Thus, providing them with a tool to make their voices heard represents a way to secure these rights and put them more in focus. Patient-reported experience measures (PREMs) are a valuable method for systematically incorporating the voices and participation of children and adolescents into the healthcare system [ 3 ]. PREMs are questionnaires that systematically measure the patient experience, providing patients with the opportunity to express their views on their encounter with the hospital and healthcare professionals [ 4 ].

PREMs, which make it possible for hospital administrators and healthcare professionals to gain valuable insights into what is important from the patient perspective, are useful in identifying areas of improvement and in translating patient needs and preferences into actual qualitive improvement projects to promote patient-centred care [ 5 ].

Although PREMs are still more commonly used in adult patient populations, the development, validation, and implementation of PREMs for children and adolescents have increased rapidly over the last few years [ 6 , 7 ]. In 2021, a systematic review on their use for children and adolescents in high-income countries included 39 different PREMs reported in 83 peer-reviewed articles [ 4 ]. The review showed that only six of the 39 PREMs had been developed explicitly for completion by children and adolescents, with the remainder completed by parents (as proxies) or not reported. Asking the parents about their child’s experience does not correspond to the rights of the child, with research showing that parental evaluation of their child’s experience may not accurately represent the views and experiences of the child [ 8 , 9 ].

Since the review’s publication, more action has been taken to represent the voice of children and adolescents. The overall goal of a European project initiated in 2021 entitled, V alue o f i ncluding the C hildren’s E xperience for improving their right S during hospitalization’ (VoiCES) is to develop and implement a joint PREM observatory across European children’s hospitals [ 10 ]. Involving several phases and qualitative and quantitative methods, researchers, clinicians, children, adolescents, and families from Finland, Italy, Latvia, and the Netherlands participated in developing and validating the VoiCES PREMs. Consensus has been reached on a final version of (age grouped) PREMs but implementation is in its early stages. No results have been published yet and the questionnaires are not publicly accessible.

The implementation process plays a particularly key role in the feasibility and applicability of PREMs, just as the mode of administration has a prominent role. The VoiCES PREMs will be administered electronically, even though the authors of the aforementioned systematic review found that paper and pencil are still most commonly used [ 4 ], for example Wray et al.’s Children and Young People’s PREM (CYP-PREM), which was developed and validated for and with children aged 8–11 and 12–15 years of age at Great Ormond Street Hospital in London [ 11 ]. The CYP-PREMs have been translated and culturally adapted to at least four other languages [ 12 , 13 ].

The response rate on these paper-and-pencil PREMs in the UK is around 24% [ 14 ], which is considered acceptable but there is still room to engage even more children and adolescents in actively participating in the evaluation of their hospital experiences. In a study assessing the implementation of PREMs for children in Canada, McCabe et al. found that researchers, PREM administrators, and healthcare professionals express concerns about paper-and-pencil PREMs because they perceive this mode of administration as a hindering factor in their implementation and utilisation [ 15 ]. The authors set out a list of recommendations for integrating PREMs into the paediatric health system. Along with endorsing electronic platforms as preferable, the authors also list short yet flexible measures with multiple forms to reflect various age groups as a priority. They also stress the importance of clinicians knowing that the measures are relevant, reliable, and valid for their intended use. In addition, engaging patients, families, and staff in planning their implementation might also ease the implementation [ 15 ].

To accommodate these recommendations and address some of the challenges and gaps in research on PREMs for children and adolescents, we will build on the existing literature, co-create, and validate MyHospitalVoice in collaboration with children and adolescents with hospital experiences. The MyHospitalVoice tool will contain developmentally appropriate questions and response options (questionnaires) for children aged 4 to 7 year, 8 to 12 years, and adolescents aged 13 to 17 years about their hospital experiences and also feature an interactive digital interface that adapts to the age of the user (when they respond to the questionnaires). We intend to engage children and adolescents with hospital experiences not only as informants but also as active collaborators in shaping the project and during methodological considerations when appropriate and meaningful. Previous research shows that children and adolescents can contribute considerably as informants, testers, and to some extent as partners [ 16 , 17 ]. In accordance with the National Institute for Health and Care Research’s definition of the involvement of children as “research being carried out ‘with’ or ‘by’ rather than ‘to’, ‘about’ or ‘for’ them’” [ 18 ], we also consider the engagement of children and adolescents as co-creation, which is defined as “to create jointly” or “to create (something) by working with one or more others” [ 19 ].

Together with children and adolescents, our primary aim is to develop and validate MyHospitalVoice, while our secondary aim is to document and evaluate the approaches used to involve children and adolescents and to assess the impact of their involvement on the process and MyHospitalVoice.

The development of MyHospitalVoice will involve creating a set of developmentally appropriate items (questions and response options in the questionnaires) and also an interactive digital interface that adapts to the age of the users (this interface is what the child/adolescent sees when filling out the MyHospitalVoice questionnaires).

Based on the European Organisation for Research and Treatment of Cancer’s (EORTC) framework for developing and validating questionnaire modules [ 20 ], we will build on existing PREMs for children and adolescents [ 10 , 11 , 12 , 13 ] and collaborate with children and adolescents to co-create, digitalise, and validate our MyHospitalVoice PREMs.

Project structure, participants, and recruitment

Involvement of children, adolescents, and young people is central to this project, and the involvement appears on two separate levels: project level and content level (illustrated in Additional file 1 ).

At the project level, we will involve three adolescents and/or young adults as peer researchers in the project group which means they are involved in planning and conducting of workshops, interpreting and dissemination of results from the workshops, and discuss next steps throughout the study and thus share decision-making power with the researchers. They will be recruited through an established network called the Youth Panel at Copenhagen University Hospital – Rigshospitalet, Denmark [ 21 ]. The Youth Panel, which comprises 15 adolescents and young adult patients 15–24 years of age, meets eight times a year to discuss new initiatives to make the hospital more youth friendly. The Youth Panel also engages in training staff and quality improvement projects, in addition to awarding an annual prize to a healthcare professional who puts in a special effort for adolescents and young people. Although the young adults are not part of paediatrics in Denmark (0 to 17 years), we have chosen to involve young adults at the project level both for practical reasons (Youth Panel is already established) and to recognise the potential work burden and complexities of this project. To acknowledge the work burden, the Youth Panel members will be asked to engage for shorter time periods (e.g. 3 months) and then reconsider and potentially switch place with someone else at the end of the period. Those who engage will be paid for the time they invest in the project based on a student assistant salary rate. They decide how much time they invest in the project, but as a starting point they will be invited to: (1) An introductory meeting (about co-conducting research and PREMs), (2) Planning of one workshop (two meetings), (3) Co-facilitate a workshop, and (4) Discuss findings and next steps.

At the content level, we involve children and adolescents from the paediatric population (4 to 17 years, thus not the youngest age group of 0 to 3 years) though workshops. The workshops will be age grouped (4 to 7 years, 8 to 12 years, and 13 to 17 years) and be about either the age-specific questionnaire or age-specific digital interface. For these workshops, we will invite children and adolescents from various hospital departments at Rigshospitalet as well as children and adolescents who are part of the MARYS’ user panel [ 22 ], which is being used in the planning, development, and conceptualisation of Rigshospitalet’s new children’s hospital called Mary Elizabeth’s Hospital (MARY). The user panel is used to recruit participants, e.g., for user tests, focus groups, and research purposes. When the panel was established, information about it and how to register was published on social media forums and in posters and flyers at departments for children and adolescents at Rigshospitalet. To date, the user panel contains 127 children and adolescents 0–17 years of age and their or their parent(s)’ contact information. To recruit participants for the MyHospitalVoice project, we will contact them by e-mail using Research Electronic Data Capture (REDCap) [ 23 ].

For the workshops, the participant inclusion criteria are 4–17 years of age, having experienced being in a hospital, and being able to speak and understand Danish. In addition, the parents must understand written Danish, English, or another Scandinavian language to be able to provide written consent for their child’s participation. We have not predefined any other criteria for patient characteristics, e.g. gender or ethnicity, thus the recruitment is based on convenience sampling and might not represent marginalised voices. Workshop participants will have their travel costs reimbursed and food and drinks will be provided at the workshops.

Approaches to document and assess involvement

Standards for patient and public involvement in research and models for participatory design will guide the involvement of children, adolescents, and young adults [ 24 , 25 , 26 , 27 ]. We will report the involvement according to the Guidance for Reporting Involvement of Patients and the Public Checklist (GRIPP2) [ 28 ] (can be found in Additional file 2 ).

Preston et al. [ 29 ], who tested the Patient Engagement Quality Guidance Tool [ 30 ] in cases that involved children and adolescents in research, found that it was helpful and informative in terms of systematising reflections on the practicalities and experieces of involving them in research and to identify gaps in practice. Based on their suggestion, we used the tool in the planning stage but in an adapted version that is available in Additional file 1 .

Inspired by Dawson et al. [ 31 ] we state here our (a priori) planned involvement activities and will then compare them with the actual involvement activities at the end of the project. Our comparisons will be made based on notes and minutes from meetings. For each activity, we will reflect on whether our expectations were met and how the involvement impacted and shaped the project (Fig. 1 ).

Planned activities that are related to the involvement of children, adolescents, and young people. Preston et al.’s [ 32 ] matrix guided the use of wording

If we find that the degree and frequency of involvement is not feasible and/or meaningful, we will reconsider our initial plans and seek alternative ways to involve children, adolescents, and young adults. We expect our a priori and explicitly documented plan will help guide our evaluation of involvement and aid in defining and describing the facilitators and/or barriers to involvement, regardless of whether our approach is successful or will require an alternative plan. In Additional file 1 , we have included a matrix adapted by Preston et at [ 32 ] to illustrate our planned degree of involvement throughout the research phases.

The adolescent/young adult peer researchers will be asked to fill out the Patient Engagement in Research Scale (PEIRS-22) to measure the engagement at ‘project level’. The scale has been developed to measure meaningful patient engagement [ 33 ] and was recently translated and culturally adapted to Danish [ 34 ].

Phases of the project

Based on the EORTC framework, the development and validation process comprises four phases (Fig.  2 ) [ 20 ].

In phase 1 , the first author (JH) will extract data from existing PREMs for children and adolescents, translate the items into Danish, and categorise ones that relate to the same dimension. Without sharing these dimensions with the adolescent/young adult peer researchers, we will ask them to indicate which dimensions they perceive as important and to suggest new items. After this, researchers and peer researchers will discuss and compare the categories and dimensions based on existing PREMs and the ones the peer researchers provide. We will clarify the translations, rephrase, and choose between redundant items, if necessary. Based on the identified categories and dimensions, the researchers and peer researchers will plan co-creative workshops to be held in phase 2.

In phase 2 , we will conduct three age grouped workshops to co-create the questionnaire content, using participatory design methods to guide them [ 24 , 35 , 36 ]. The format and content will depend on the age group: 4 to 7 years, 8 to 12 years, and 13 to 17 years with eight to 16 participants at each workshop. For children and adolescents aged 8 to 17 years, parents are allowed to participate, if their child wants them to. For children 4 to 7 years, parents will participate but be encouraged to let their child be in control.

We will start with running workshops for the oldest age group and then progress to the younger ones building on knowledge gained in workshops for older children/adolescents.

JH will be responsible for organising the workshops and act as the primary facilitator, collaborating with the peer researchers. At the workshops and in developmentally appropriate, playful, and creative ways, the participants will be invited to explore hospital experiences and discuss, select, and prioritise which dimensions they perceive as most important in addition to adding new ones. To provide an example, the workshop for adolescents aged 13 to 17 years will consist of: welcome, ice-breaker activities, process exercises about hospital encounters and experiences, and evaluation and sum-up. A table of agenda and draft activities are provided in Additional file 1. The specific activities and exercises have not been decided yet, as they are to be designed in collaboration with young adult peer researchers, but they will be based on the Danish “Handbook for Child Involvement” ( Håndbog for børneinddragelse ) and Participation Works’ “The Toolkit” [ 26 , 27 ]. During the activities paper, cardboard, scissors, sticky tape, and stickers will be available and participants will be encouraged to draw, write, discuss, and reflect.

Development and validation of MyHospitalVoice. PREMs: patient-reported experience measures

Depending on the age group, we will discuss general considerations regarding questionnaire development, for instance introducing texts and type of response options (e.g. yes/no and Likert scales).

During the workshops researchers will take notes and each activity/exercise and its process and outcome will be reflected upon. After the workshops, the resulting materials will be collected and stored according to Danish regulations on data management. Based on workshop exercises, summing up, and the materials produced, JH will make a first draft of the dimensions, preliminary items, and results to present them to the researchers and peer researchers at a work meeting. JH will facilitate the meeting to ensure that the perspectives of both groups are included in the interpretation of data. We will also compare our items to original items from existing PREMs, and related questions will undergo forward-backward translations [ 37 ] to ensure comparability and research collaboration with other countries. The process will be iterative but will result in the first set of questionnaires for MyHospitalVoice.

We will then initiate an iterative creation of the digital interface for MyHospitalVoice (Fig.  3 ). Again, we will conduct at least three age grouped workshops: 4 to 7 years, 8 to 12 years, and 13 to 17 years with eight to 16 participants at each workshop, starting with the oldest age group. At these workshops, parents are welcome too, but with the child’s view and preferences being the central point.

Development of the digital interface for MyHospitalVoice

The children and adolescent for these workshops will be recruited from the same population as the first round of workshops (user panel and paediatric departments), but it is not a requirement to have participated in the first-round workshops about the questionnaire content.

The second-round workshops will engage the children and adolescents in the co-creation of the design of the digital interface. Their ideas will help to incorporate playfulness, storytelling, and perhaps gamification in the development of age-appropriate ways of responding to a questionnaire. Researchers have found that children and adolescents find it easier and more fun to answer questionnaires in an animated application than in a paper questionnaire or orally [ 38 ]. Thus, the digital interface for MyHospitalVoice will be age appropriate and incorporate playful aspects. The digital interface will undergo multiple testing and the same or new children/adolescents might be invited to several test sessions.

The questionnaire will be stored in REDCap, and the digital interface will be developed as a responsive website with an application programming interface that pushes data to REDCap when the questionnaires are filled out. Once the digital interface has been developed, we will establish this connection, resulting a beta version of MyHospitalVoice.

In phase 3 , we will conduct two types of pre-testing. First, the questionnaires will be administered to children and adolescents to identify and solve potential problems in phrasing and/or the sequencing of questions and to identify any missing or redundant items. In the second pre-test, we will conduct cognitive interviews with participants from the target population, children and adolescents with hospital experience, to investigate their understanding of the items in more detail. We will explore how they perceive the items (introductory text, questions, and response options), and the word choice. Cognitive interviewing methods will use the think-aloud method, where subjects are explicitly instructed to think aloud while answering the questionnaire [ 20 ]. Studies show that children as young as eight years of age can engage in this process and give meaningful feedback on their understanding of items [ 39 ]. After completing the two pre-tests, MyHospitalVoice will undergo refinement and, depending on the extent of the changes required, the pre-testing will be repeated.

In the last phase, phase 4 , we will pilot test MyHospitalVoice with approximately 200 children and adolescents from in- and outpatient departments at Copenhagen University Hospital – Rigshospitalet. They will also receive a short age-appropriate debriefing questionnaire. There are no formal requirements on sample size in developing and testing a questionnaire, but others have included 10 patients per item, though more with a heterogeneous target population [ 40 ]. In the later stages of the project, hospital department leaders will be consulted to help determine which patient populations will take part in pilot testing MyHospitalVoice.

We will evaluate the pilot test(s) by calculating response rates and exploring the factors that either hindered or facilitated implementation. Based on the evaluation, we may incorporate minor modifications. We will explore the psychometric properties of MyHospitalVoice using exploratory factor analysis, confirmatory factor analysis, Cronbach’s alpha, and differential item functioning analysis. We will also perform an exploratory analysis of correlations between experience factors and patient characteristics, e.g. age, disease severity, and prior hospitalisation, in addition to determining the required sample sizes to find associations between experience factors and, e.g. readmission.

Implementation of MyHospitalVoice

After conducting and evaluating our pilot study, we will implement MyHospitalVoice at Copenhagen University Hospital – Rigshospitalet in paediatric departments, adult departments, and outpatient clinics that admit children and adolescents. The implementation process will start in early phases of the project, so that the MyHospitalVoice tool is designed to fit into existing structures (e.g. the electronic health record system) and clinical practices. To do this, we will have ad hoc involvement of relevant stakeholders (e.g. clinicians, leaders, digitalisation and implementation managers). Implementation will adhere to existing management structures and be coordinated with other relevant bodies.

The Implementation Research Logic Model (IRLM), which will guide the implementation (Fig.  4 ), is a useful tool for planning, executing, and reporting the implementation of research projects [ 41 ]. IRLM incorporates aspects from other well-known implementation frameworks, including the Consolidated Framework for Implementation Research (CFIR) [ 42 ]. Factors from CFIR will be included in the first step in the IRLM to help identify the determinants of implementation that can act as either facilitators or barriers to successful implementation. The determinant domains, which include innovation, inner setting, outer settings, individuals, and process, will be identified partly through our pilot study and partly through discussions with stakeholders and staff. Based on the determinants, we will identify and select relevant implementation strategies to facilitate and ease implementation. The mechanisms that relate to how the implementation strategies affect the determinants and their implications for outcomes will be discussed and hypothesised in the research group and with relevant collaborators at Copenhagen University Hospital – Rigshospitalet.

Smith et al.’s implementation research logic model [ 41 ]

Collecting information on the hospital experiences of children and adolescents can help facilitate translate their needs and preferences into clinical practice and represent a possible way of operationalising patient-centred care. Thus, validated PREMs for children and adolescents can serve as a powerful tool when seeking to improve the quality of care provided to children and adolescents. In contrast to most PREMs for children and adolescents, MyHospitalVoice will provide an interactive digital interface to the questionnaire. A high response rate to the digital questionnaire that we develop will indicate that it is successful and likely increase the impact of MyHospitalVoice. Broad implementation will make rapid, real-time data analysis possible for the benefit of patients, healthcare professionals, and hospital administrators. Moreover, electronic storage of the questionnaire and responses will reduce the work burden for staff and administrators and promote the seamless spread of MyHospitalVoice.

The processes of co-creation during the development of MyHospitalVoice will generate valuable insights into conducting research that is done ‘with’ rather than ‘on’ children and adolescents. We hope to inspire and foster collaboration with other researchers in this area by sharing our methods, knowledge, and experiences of co-creating a digital evaluation tool with children and adolescents.

The results of co-creation will always depend on the people who participate, which is why being aware of and describing the background and prior experiences of researchers, peer researchers, and other participants is essential. Moreover, applying recognised frameworks and guidelines will strengthen our project and help increase the transparency and utilisation of our research.

Data availability

No datasets were generated or analysed during the current study.

Abbreviations

Consolidated Framework for Implementation Research

Young People’s Patient-Reported Experience Measures

European Organisation for Research and Treatment of Cancer

Guidance for Reporting Involvement of Patients and the Public Checklist 2

Implementation Research Logic Model

Patient Engagement in Research Scale

Patient-reported experience measures

Research Electronic Data Capture

V alue o f i ncluding the C hildren’s E xperience for improving their right S during hospitalization

Unicef. Convention on the Rights of the Child. 1989.

NOBAB. Nordisk standard för barn och ungdomar inom hälso-och sjukvård. Tillgänglig på internet Senast tillgänglig. 2011;80330.

Nordlind A, Sundqvist AS, Anderzén-Carlsson A, Almblad AC, Ängeby K. How paediatric departments in Sweden facilitate giving children a voice on their experiences of healthcare: a cross‐sectional study. Health Expect. 2022;25(1):384–93.

Article   PubMed   Google Scholar  

Bele S, Teela L, Zhang M, Rabi S, Ahmed S, van Oers HA et al. Use of patient-reported experience measures in Pediatric Care: a systematic review. Front Pead. 2021;9.

Beckers E, Webers C, Boonen A, ten Klooster PM, Vonkeman HE, van Tubergen A. Validation and implementation of a patient-reported experience measure for patients with rheumatoid arthritis and spondyloarthritis in the Netherlands. Clin Rheumatol. 2020;39(10):2889–97.

Article   PubMed   PubMed Central   Google Scholar  

Toomey SL, Zaslavsky AM, Elliott MN, Gallagher PM, Fowler FJ, Klein DJ, et al. The development of a pediatric inpatient experience of care measure: child HCAHPS®. Pediatrics. 2015;136(2):360–9.

Kemp KA, Ahmed S, Quan H, Johnson D, Santana MJ. Family experiences of Pediatric Inpatient Care in Alberta, Canada: results from the child HCAHPS Survey. Hosp Pediatr. 2018;8(6):338–44.

Ali H, Fatemi Y, Cole A, Tahat S, Ali D. Listening to the Voice of the hospitalized child: comparing children’s experiences to their parents. Children. 2022;9(12):1820.

Toomey SL, Elliott MN, Zaslavsky AM, Klein DJ, Ndon S, Hardy S et al. Variation in Family Experience of Pediatric Inpatient Care as measured by child HCAHPS. Pediatrics. 2017;139(4).

De Rosis S, Bonciani M, Spataro V, Corazza I, Conti E, Sibbles B, et al. Value of including the children’s experience for improving their rights during hospitalization: protocol for the VoiCEs Project. JMIR Res Protoc. 2023;12:e42804.

Wray J, Hobden S, Knibbs S, Oldham G. Hearing the voices of children and young people to develop and test a patient-reported experience measure in a specialist paediatric setting. Arch Dis Child. 2018;103(3):272.

Ryberg AM, Nielsen PB, Graarup KS, Ingeman K, Thellefsen MR, Jensen CS. Danish translation and cultural adaptation of the ‘What do you think of hospital’ patient reported experience measure for children and adolescents in outpatient settings. J Pediatr Nurs. 2023;68:e36–42.

Nordlind A, Anderzén-Carlsson A, Sundqvist AS, Ängeby K, Wray J, Oldham G et al. Translation, cultural adaptation and validation of a patient-reported experience measure for children. Health Expect. 2023:e13924.

Survey Coordination Centre for Existing Methods (SCCEM). Picker Institute Europe. Survey development report: Children and young people’s patient experience survey 2020. 2020.

McCabe E, Rabi S, Bele S, Zwicker JD, Santana MJ. Factors affecting implementation of patient-reported outcome and experience measures in a pediatric health system. J Patient-Reported Outcomes. 2023;7(1):1–12.

Article   Google Scholar  

Ruland CM, Starren J, Vatne TM. Participatory design with children in the development of a support system for patient-centered care in pediatric oncology. J Biomed Inform. 2008;41(4):624–35.

Stålberg A, Sandberg A, Söderbäck M, Larsson T. The child’s perspective as a guiding principle: young children as co-designers in the design of an interactive application meant to facilitate participation in healthcare situations. J Biomed Inf. 2016;61:149–58.

National Institute for Health Care Research. Briefing notes for researchers–public involvement in NHS, health and social care research2021. Available from: https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371 .

2024. Cocreate Merriam-Webster Dictionary.

Sprangers M, Cull A, Bjordal K, Groenvold M, Aaronson N. The European Organization for Research and Treatment of Cancer approach to quality of life assessment: guidelines for developing questionnaire modules. Qual Life Res. 1993;2(4):287–95.

Article   CAS   PubMed   Google Scholar  

Rehling E, Blix C, Hertz P, Larsen J, Noergaard V, Teilmann G. The magic of a youth advisory board. Turk Arch Ped. 2013:122.

BørneRiget søger prøvekaniner. 2021 [cited 2023. Available from: https://www.rigshospitalet.dk/presse-og-nyt/nyheder/nyheder/Sider/2021/december/boerneriget-soeger-proevekaniner.aspx .

Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

Jessen S, Mirkovic J, Ruland CM. Creating Gameful Design in mHealth: a participatory Co-design Approach. JMIR Mhealth Uhealth. 2018;6(12):e11579.

Hickey G, Brearley S, Coldham T, Denegri S, Green G, Staniszewska S, et al. Guidance on co-producing a research project. Southampton: INVOLVE; 2018.

Google Scholar  

Børnerådet. Håndbog i Børneinddragelse: Børnerådet København; 2016.

Participation Works. THE TOOLKIT - EVALUATING PARTICIPATION WORK. 2008.

Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ (Clinical Res ed). 2017;358.

Preston J, Nafria B, Ohmer A, Gaillard S, Dicks P, West L, et al. Developing a more tailored Approach to patient and public involvement with Children and Families in Pediatric Clinical Research: lessons learned. Ther Innov Regul Sci. 2022;56(6):948–63.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Patient Focused Medicines Development. Patient Engagement Quality Guidance Tool [Planning a PE project]. 2018.

Dawson S, Ruddock A, Parmar V, Morris R, Cheraghi-Sohi S, Giles S, et al. Patient and public involvement in doctoral research: reflections and experiences of the PPI contributors and researcher. Res Involv Engagem. 2020;6:1–13.

Preston J, Biglino G, Harbottle V, Dalrymple E, Stalford H, Beresford MW. Reporting involvement activities with children and young people in paediatric research: a framework analysis. Res Involv Engagem. 2023;9(1):1–16.

Hamilton CB, Hoens AM, McKinnon AM, McQuitty S, English K, Hawke LD, et al. Shortening and validation of the Patient Engagement in Research Scale (PEIRS) for measuring meaningful patient and family caregiver engagement. Health Expect. 2021;24(3):863–79.

Christiansen MG, Pappot H, Hamilton C, Lund-Jacobsen T, Kristensen AW, Nordentoft S, et al. Patient Engagement in Research Scale (PEIRS-22): Danish translation, applicability, and user experiences. Res Involv Engagem. 2023;9(1):115.

Vandekerckhove P, de Mul M, Bramer WM, de Bont AA. Generative participatory design methodology to develop electronic health interventions: systematic literature review. J Med Internet Res. 2020;22(4):e13780.

Haijes HA, van Thiel GJ. Participatory methods in pediatric participatory research: a systematic review. Pediatr Res. 2016;79(5):676–83.

Eremenco SL, Cella D, Arnold BJ. A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof. 2005;28(2):212–32.

Arvidsson S, Gilljam B-M, Nygren J, Ruland CM, Nordby-Bøe T, Svedberg P. Redesign and validation of Sisom, an interactive Assessment and Communication Tool for Children with Cancer. JMIR Mhealth Uhealth. 2016;4(2):e76.

Irwin DE, Varni JW, Yeatts K, DeWalt DA. Cognitive interviewing methodology in the development of a pediatric item bank: a patient reported outcomes measurement information system (PROMIS) study. Health Qual Life Outcomes. 2009;7(1):1–10.

Fayers PM, Machin D. Quality of life: the assessment, analysis and interpretation of patient-reported outcomes. Wiley; 2013.

Smith JD, Li DH, Rafferty MR. The implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84.

Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):75.

Download references

Acknowledgements

Not applicable.

Funding for this study was received from TrygFonden (grant no.153464) and Helsefonden (grant no. 21-A-0071).

Open access funding provided by Copenhagen University

Author information

Authors and affiliations.

Mary Elizabeth’s Hospital and Juliane Marie Centre, Copenhagen University Hospital – Rigshospitalet, Blegdamsvej 9, Copenhagen, 2100, Denmark

Jane Hybschmann, Jette Led Sørensen, Jakob Thestrup, Thomas Leth Frandsen & Line Klingen Gjærde

Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Blegdamsvej 3B, Copenhagen, 2200, Denmark

Jette Led Sørensen

Department of Oncology, Copenhagen University Hospital – Rigshospitalet, Blegdamsvej 9, Copenhagen, 2100, Denmark

Helle Pappot

Department of Paediatrics and Adolescent Medicine, Center of Adolescent Medicine, Copenhagen University Hospital – Rigshospitalet, Blegdamsvej 9, Copenhagen, 2100, Denmark

Kirsten Arntz Boisen

You can also search for this author in PubMed   Google Scholar

Contributions

JH, JLS, and LKG contributed to the conception and design of this study. JH wrote the initial draft manuscript and prepared the figures and tables. JLS, LKG, TLF, HP, JT, and KB revised the manuscript critically for important intellectual content. All authors approved the final version of the manuscript and agreed to be accountable for all aspects of it.

Corresponding author

Correspondence to Jane Hybschmann .

Ethics declarations

Ethics approval and consent to participate.

This project adheres to the Danish Code of Conduct for Research Integrity and the Declaration of Helsinki. We will provide children, adolescents, and their parents with age-appropriate information about the study and inform of their right to withdraw from the study at any time. The information will include details on the scope and aims of the project, and we will ask children < 15 years for their assent to participate, while adolescents and young adults ≥ 15 year (and potentially parents) will be asked to provide written consent. In accordance with the General Data Protection Regulation, the project is registered with the Capital Region of Denmark’s Knowledge Centre for Data Review ( Videnscenter for dataanmeldelse ) (file no.: P-2022-458). The institutional review board of the Danish Research Ethics Committee (file no.: VEK22028868) evaluated this study even though ethics approval is not required.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hybschmann, J., Sørensen, J.L., Thestrup, J. et al. MyHospitalVoice – a digital tool co-created with children and adolescents that captures patient-reported experience measures: a study protocol. Res Involv Engagem 10 , 49 (2024). https://doi.org/10.1186/s40900-024-00582-2

Download citation

Received : 26 January 2024

Accepted : 07 May 2024

Published : 21 May 2024

DOI : https://doi.org/10.1186/s40900-024-00582-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Patient-reported experience measure
• Child participation
• Adolescent participation
• Research involvement
• Patient and public involvement
• Co-creation
• Hospital experience
• Quality of healthcare

Research Involvement and Engagement

ISSN: 2056-7529

• General enquiries: [email protected]

• Copy/Paste Link Link Copied

Understanding How Digital Media Affects Child Development

Technology and digital media have become ubiquitous parts of our daily lives. Screen time among children and adolescents was high before COVID-19 emerged, and it has further risen during the pandemic, thanks in part to the lack of in-person interactions.  

In this increasingly digital world, we must strive to better understand how technology and media affect development, health outcomes, and interpersonal relationships. In fact, the fiscal year 2023 federal budget sets aside no less than $15 million within NICHD’s appropriation to investigate the effects of technology use and media consumption on infant, child, and adolescent development.

Parents may not closely oversee their children’s media use, especially as children gain independence. However, many scientific studies of child and adolescent media use have relied on parents’ recollections of how much time the children spent in front of a screen. By using software embedded within mobile devices to calculate children’s actual use, NICHD-supported researchers found that parent reports were inaccurate more often than they were on target. A little more than one-third of parents in the study underestimated their children’s usage, and nearly the same proportion overestimated it. With a recent grant award from NICHD, researchers at Baylor College of Medicine plan to overcome the limitation of relying on parental reports by using a novel technology to objectively monitor preschool-age children’s digital media use. They ultimately aim to identify the short- and long-term influences of technology and digital media use on children’s executive functioning, sleep patterns, and weight. This is one of three multi-project program grants awarded in response to NICHD’s recent funding opportunity announcement inviting proposals to examine how digital media exposure and use impact developmental trajectories and health outcomes in early childhood or adolescence. Another grant supports research to characterize the context, content, and use of digital media among children ages 1 to 8 years and to examine associations with the development of emotional regulation and social competence. A third research program seeks to better characterize the complex relationships between social media content, behaviors, brain activity, health, and well-being during adolescence.

I look forward to the findings from these ongoing projects and other studies that promise to inform guidance for technology and media use among children and adolescents. Additionally, the set-aside funding for the current fiscal year will allow us to further expand research in this area. These efforts will help us advance toward our aspirational goal to discover how technology exposure and media use affect developmental trajectories, health outcomes, and parent-child interactions.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Research Council (US) and Institute of Medicine (US) Forum on Adolescence; Kipke MD, editor. Adolescent Development and the Biology of Puberty: Summary of a Workshop on New Research. Washington (DC): National Academies Press (US); 1999.

Adolescent Development and the Biology of Puberty: Summary of a Workshop on New Research.

• Hardcopy Version at National Academies Press

Key Findings of Recent Studies

The workshop included a series of panel discussions that focused on adolescence as experienced by both human and nonhuman primates, including neuroendocrine physiology at puberty, the interplay between pubertal development and behavior, and implications for research, policy, and practice. Here we briefly summarize key findings from some of the studies that were discussed at the workshop (also see Crockett and Petersen, 1993; Grumbach and Styne, 1998; Pusey, 1990; Suomi, 1997;1991). As previously noted, this summary is not intended to provide a comprehensive review of the new research in this field; rather, it highlights important new findings that emerged during the workshop presentations and discussions.

• In the United States, the Onset of Puberty Occurs Earlier than was Previously Recognized.

Over the last 150 years, girls' sexual maturation, as measured by the age of menarche, is occurring at younger ages in all developed countries by at least two to three years. In the mid-nineteenth century, the average age at which girls reached menarche was approximately 15. The trend toward earlier menarche is now being documented in developing countries as well. Improved diets and more effective public health measures are the reasons often cited for this trend (Garn, 1992).

Research conducted during the 1990s greatly enhanced researchers' understanding of the age of puberty among girls. For example, although the onset of menarche is still considered to be a significant indicator of the tempo of maturation, researchers now view menarche as a late event in the pubertal process. At the workshop, Frank Biro presented data from the Growth and Health Study funded by the National Heart, Lung, and Blood Institute. This longitudinal study enrolled a cohort of over 2,000 girls, ages 9 to 10 years in 1987–1988; approximately half of the sample was white and half was black; the sample was recruited from clinics at three clinical centers located in Richmond, California, Cincinnati, Ohio, and metropolitan Washington, D.C. According to the study design, girls' maturation stage and body mass index were assessed annually; data for other variables, such as household income, nutrition, physical activity, cardiovascular risk factors, self-esteem and self-perception, and other psychosocial measures, were collected biennially (Brown et al., 1998). Almost half of the participants had begun puberty before the onset of the study. According to Biro, indicators of pubertal growth have been observed as early as age 7. These findings suggest that as children experience puberty and other developmental changes at earlier ages, there may be the need to consider how to design and deliver age-appropriate interventions during the middle childhood and preteen years, to help them avoid harmful or risky behaviors and develop a health-promoting lifestyle.

• There is Significant Variation Among Individuals in the Timing of Puberty.

There is variation in both the onset and the tempo of puberty. Research shows that the timing of puberty can affect other aspects of development, especially for girls. Jeanne Brooks-Gunn discussed the findings from a recent study, which recruited a community sample of nearly 2,000 high school students from urban and rural areas of western Oregon. The study found that early-maturing girls and late-maturing boys showed more evidence of adjustment problems than other adolescents (Graber et al., 1997).

• Multiple Factors Affect the Age of Puberty.

Research now suggests that the timing of puberty can be affected by a wide range of factors, including genetic and biological influences, stress and stressful life events, socioeconomic status, environmental toxins, nutrition and diet, exercise, amount of fat and body weight, and the presence of a chronic illness. Research also shows that the family, the peer group, the neighborhood, the school, the workplace, and the broader society have all been shown to influence adolescent developmental outcomes, although it is less clear if these factors influence pubertal development. With respect to school settings, research suggests that the transition from small elementary schools to larger, more anonymous middle schools can be a stressful event in the lives of children (National Research Council, 1993). Some of the stressful influences or events factors mentioned above have been correlated with pubertal timing, but a causal relationship cannot be assumed.

• Stress does not Trigger Puberty, But it does Modulate the Timing of Puberty.

In her remarks at the workshop, Elizabeth Susman took note of research correlating stress and the timing of puberty. 1 A review of this literature shows that researchers observe different effects of stress at different stages of puberty (Susman et al., 1989). For example, stress appears to delay maturation for young adolescents but to precipitate puberty for older adolescents. According to Susman, it makes sense that stress would delay maturation because stress hormones tend to suppress reproductive hormones (Susman, 1997; Graber and Warren, 1992). She added that her research has not yet resolved the question of directionality: Do environmental stressors affect the reproductive hormones, or does the rate of maturation affect the level of circulating stress hormones? Other participants at the meeting noted that social factors influence this process as well. For example, family conflict appears to be associated with earlier menarche in girls (Graber et al., 1995).

• There is some Evidence that, on Average, Girls experience more distress during adolescence than boys.

Some researchers have speculated that, for girls, the transition during puberty brings about greater vulnerability to other environmental stressors (Ge et al., 1995). In particular, a growing literature suggests that the early onset of puberty can have an adverse effect on girls' development (Caspi et al., 1993; Ge et al., 1996). It can affect their physical development (they tend to be shorter and heavier), their behavior (they have higher rates of conduct disorders); and emotional development (they tend to have lower self-esteem and higher rates of depression, eating disorders, and suicide). The youngest, most mature children are those at greatest risk for delinquency.

Early-maturing boys also appear to have higher rates of delinquency (Graber et al., 1997; Rutter and Smith, 1995). Generally speaking, however, boys who mature early fare better than late bloomers. Because they are taller and more muscular than their age-mates, they may be more confident, more popular, and more successful both in the classroom and on the playing field. In contrast, late-maturing boys have a poorer self-image, poorer school performance, and lower educational aspirations and expectations (Dorn et al., 1988; Litt, 1995).

• Girls from Ethnic Minority Groups may be Reaching Puberty Earlier than White Girls.

Data presented at the workshop show that for black girls, the average age of menarche is 12.1 years, compared with 12.9 years for white girls (see Brown et al., 1998). Black girls also begin pubertal development earlier than their white peers do—by 15 months. Interestingly, even though they reach menarche earlier, tempo of the pubertal development is slower. Researchers have also found that self-esteem does not follow the same developmental pattern in black and white girls. It appears that black girls' higher self-esteem may be rooted in cultural differences in attitudes toward physical appearance and obesity (Brown et al., 1998). In general, however, the factors that protect some girls and place others at risk are not well understood. It is important to note that these findings are preliminary in nature, and more research is need to further validate them, as well as determine if these differences apply to girls from other ethnic, and racial groups, such as Hispanics, American Indians, Asians, and Pacific Islanders.

• Puberty may be a Better Predictor of Aggression and Problem Behaviors than Age.

There is growing evidence to suggest that puberty rather than chronological age may signal the onset of delinquency and problem behaviors among some teenagers (Keenan and Shaw, 1997; Rutter et al., 1998). For example, early maturers—both mate and female—are more likely than other adolescents to report delinquency. Early-maturing females also appear to be at increased risk for victimization, especially sexual assault, and this may partially explain their greater likelihood of problem behaviors (Flannery et al., 1993; Raine et al., 1997). These findings suggest the need for interventions that are targeted to early-maturing adolescents who may be at increased risk for a wide range of behavior problems and associated poor developmental outcomes.

• Physical Maturation Appears to have Little Correlation with Cognitive Development.

Many developmental psychologists, most notably Jean Piaget, have documented an expanded capacity for abstract reasoning during adolescence. Today's adolescents are often capable of complex reasoning and moral judgment; their capacities frequently astonish parents and teachers. Indeed, IQ tests show an overall gain in cognitive capacities since the 1940s, when military personnel were tested in large numbers and achieved a median score of about 100. However, there appears to be little relationship between physical and cognitive maturation.

Researchers have tested the hypothesis that growth across the developmental spectrum—physical, cognitive, social, and emotional—proceeds on a similar timetable, and they have found little evidence to support this hypothesis. However, the research in this area is relatively weak, in part due to a lack of reliable, valid, easily administered instruments for assessing cognitive development (Litt, 1995). When cognitive development and capacities are not in sync with physical and sexual maturation, young people are more vulnerable; this also creates special challenges for designing and delivering age appropriate clinical interventions and services. Adults will often assume that adolescents who look older have a better grasp of the consequences of their actions.

• Brain Development Appears to Continue During Adolescence.

One of most remarkable findings in neurobiology over the last decade is the extent of change that can occur in the brain, even in the adult brain, as a function of the physical, social, and intellectual environment.

Starting in infancy and continuing into later childhood, there is a period of exuberant synapse growth followed by a period of synaptic ''pruning" which is largely completed by puberty. Although, neuroscientists have documented the time line of this synaptic waxing and waning, they are less sure about what it means for changes in childrens' and adolescents' cognitive development, behavior, intelligence, and capacity to learn. Generally, they point to correlations between changes in synaptic density or numbers and observed changes in behavior based on developmental and cognitive psychology. In coming decades, research tools such as positron emission tomography (PET) scans and functional magnetic resonance imaging (MRI) scans should greatly expand researchers' knowledge about adolescent brain development. In particular, functional imaging, if repeated over time, carries the potential for providing a better understanding of the functional connections between brain development and psychological performance (including cognitive development). New insights into brain development may also shed light on some psychopathologies and learning disabilities that affect preteens and adolescents, such as attention deficit/ hyperactivity disorder (ADHD), depressive disorders, and schizophrenia.

• Researchers are Also Providing New Insights into the Relationship Between Gender, Hormones, Brain Development, and Behavior.

In terms of the onset of puberty, boys generally follow girls by two years. For example, boys typically reach their maximum height velocity two years later than girls. In the realm of neuroscience, there is new evidence of divergent patterns of male and female brain development; these patterns have been observed between the ages of 5 and 7. Case in point: during this period, the amygdala (a part of the limbic system concerned with the expression and regulation of emotion and motivation) increases robustly in males, but not in females; the hippocampus (a part of the limbic system that plays an important role in organizing memories) increases robustly in females, but not in males. The basal ganglia are larger in females; this appears to be significant, since boys are more likely to have disorders, such as ADHD, that are associated with smaller basal ganglia. Girls may have extra protection against this type of disorder. Although there are clear differences in the path of brain development for girls and boys, it is not yet possible to look at a brain scan and determine whether the subject is male or female.

• Pregnancy During Adolescence may Alter the Physiological Development of Girls.

During pregnancy, young women at different points in pubertal development show comparable hormone profiles. Pregnancy in very young women may compromise their skeletal growth, preventing them from reaching maximum bone mass. Frank Biro noted that his research team, which followed several hundred adolescent pregnancies, found that, after giving birth, adolescent mothers were on average significantly heavier (by approximately 10 pounds) and fatter (having thicker skin folds) than their counterparts who had not given birth.

For the purposes of this discussion, stress is defined as a physical, mental, or emotional strain or tension. Stress is a normal part of everyone's life and need not be either good or bad; reactions to stress however, can vary considerably, with some reactions being unpleasant and/or undesirable.

• Cite this Page National Research Council (US) and Institute of Medicine (US) Forum on Adolescence; Kipke MD, editor. Adolescent Development and the Biology of Puberty: Summary of a Workshop on New Research. Washington (DC): National Academies Press (US); 1999. Key Findings of Recent Studies.
• PDF version of this title (248K)

In this Page

Recent activity.

• Key Findings of Recent Studies - Adolescent Development and the Biology of Puber... Key Findings of Recent Studies - Adolescent Development and the Biology of Puberty

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers