Recent Internal Resources from the D&I Program:
This study is innovative in that it examines which factors and in what combination are
necessary for successful implementation of a registry. The findings have implications for primary care
Documenting Adaptations in The Triple AIM QUERI Projects: Tracking Changes
Recent talk by Russell Glasgow, Marina McCreight, and Borsika Rabin at the VAHSRD/QUERI National Conference in Alexandria, VA, July 2017.
What Makes for a Successful Registry Implementation: A Qualitative Comparitive Analysis
Article citation: Holtrop, J. S., Hall, M. T., Rubinson, P. C., & And, L. M. (September 2017). What Makes for a Successful Registry Implementation: A Qualitative Comparitive Analysis. JABFM. vol. 30no. 5 657-665.
How can clinical practices pragmatically increase physical activity for patients with type 2 diabetes? A systematic review.
Article citation: Luoma, K. A., Leavitt, I. M., Marrs, J. C., Nederveld, A. L., Regensteiner, J. G., Dunn, A. L., . . . Huebschmann, A. G. (2017, June 06). How can clinical practices pragmatically increase physical activity for patients with type 2 diabetes? A systematic review. PMID: 28589531 DOI: 10.1007/s13142-017-0502-4
Click here for the abstract
Understanding the Context of Health for Persons With Multiple Chronic Conditions: Moving From What Is the Matter to What Matters
Article citation: Bayliss, E. A., Bonds, D. E., Boyd, C. M., Davis, M. M., Finke, B., Fox, M. H., . . . Stange, K. C. (2014, May). Understanding the Context of Health for Persons With Multiple Chronic Conditions: Moving From What Is the Matter to What Matters. Annals of Family Medicine. May/June 2014. Vol.12, No.3. 260-269.
Download the PDF
Writing D&I grants: Getting started in a fast moving field.
Enola Proctor's Keynote speech from the CU ACCORDS Getting D&I Ideas Funded workshop June 21, 2017.
Practical use of dissemination and implementation outcomes, thoeries, and research designs.
Paul Estabrooks' Keynote speech from the CU ACCORDS Getting D&I Ideas Funded workshop June 21, 2017.
Download the Slide Deck
Implementation Science: How It Can Complement, Extend, and Challenge How You Do Science (and increase your success).
Dan Matlock and Russ Glasgow presented this talk on T3-T4 translational research at the monthly Grand Rounds of the Eastern Colorado VA and University of Colorado Medical School. The talk covered both implementaton science in general and RE-AIM applications in particular.
Translating Research Into Practice - Pragmatic Research Approaches
Russell Glasgow's recent talk at the National Network of Depression Centers (NNDC) in August.
Implementation Science: Opportunities and Resources.
This Presentation reviews, provides links to and discusses future research needs for measures relevant for D&I research. It comes from a 2013 meeting of invited D&I experts to assess the satus of the field and make recommendations to close existing gaps.
Bringing it home: expanding the local reach of dissemination and implementation training via a university-based workshop.
Implementation Science. 2015
Jul 4;10:94. doi: 10.1186/s13012-015-0281-6.
Morrato EH1,2, Rabin B3,4,5,6, Proctor J7, Cicutto LC8,9, Battaglia CT3,5,6,8, Lambert-Kerzner A3,5,6, Leeman-Castillo B10, Prahl-Wretling M3, Nuechterlein B7, Glasgow RE4, Kempe A4.
A stepped-wedge evaluation of an initiative to spread the collaborative care model for depression in primary care.
Solberg LI, Crain AL, Maciosek MV, Unützer J, Ohnsorg KA, Beck A, Rubenstein L, Whitebird RR, Rossom RC, Pietruszewski PB, Crabtree BF, Joslyn K, Van de Ven A, Glasgow RE
Key General References in D&I:
JPHMP 2017 v00, nbr00
Getting the Word Out: New Approaches for Disseminating Public Health Science
Ross C. Brownson, PhD; Amy A. Eyler, PhD; Jenine K. Harris, PhD; Justin B. Moore, PhD, MS;
Rachel G. Tabak, PhD, RD
Science 2017, 12:110 | Published on: 2 September 2017
Developing a comprehensive definition of sustainability
Julia E. Moore, Alekhya Mascarenhas, Julie Bain, and Sharon E. Straus
Read the Article Here
© Health Research and Educational Trust DOI: 10.1111/1475-6773.12751
Development of the Learning Health System Researcher Core Competencies
Christopher B. Forrest , Francis D. Chesley Jr., Michelle L. Tregear, and Kamila B. Mistry
Download the article
Medical Care Volume 00, Number 00, ’’ 2016
Accelerating Research Impact in a Learning Health Care System: VA’s Quality Enhancement Research Initiative in the Choice Act Era
Kilbourne, A, Elwy R, Sales A, Atkins D.
A scoping review of classification schemes of interventions to promote and integrate evidence into practice in healthcare
Cynthia Lokker, K Ann McKibbon, Heather Colquhoun and Susanne Hempel
Implement Sci. 2014 May 1;9:51. doi: 10.1186/1748-5908-9-51.
Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies.
Colquhoun H1, Leeman J, Michie S, Lokker C, Bragge P, Hempel S, McKibbon KA, Peters GJ, Stevens KR, Wilson MG, Grimshaw J.
Am J Prev Med. 2015 May;48(5):543-51. doi: 10.1016/j.amepre.2014.11.015.
Understanding mis-implementation in public health practice.
Brownson RC1, Allen P2, Jacob RR2, Harris JK2, Duggan K2, Hipp PR3, Erwin PC4.
Resources for Practical Measures for D&I Research
The ‘Ultimate Use’ question
for D&I measurement (a goal; not achievable in any one project but to guide
interventions (programs or policies) and what components of these
interventions, conducted under what conditions and in what settings, conducted
by which agents, is effective in producing which outcomes, for which
populations (and subgroups), how much does it cost, and how does it comes
Click Title to view Article source
- Rabin B, Purcell P, and Glasgow RE (2013). Harmonizing Measures for Implementation Science using Crowd-Sourcing. Clinical Medicine and Research, September;11(3):158.
- Rabin B, Purcell P, Naveed S, Moser R, Henton M, Proctor EK, Brownson RC, Glasgow RE (2012). Advancing the Application, Quality, and Harmonization of Implementation Science Measures. Implementation Science, Dec 11;7:119
- Glasgow RE (2009). Critical Measure Issues in Translational Research. Research on Social Work Practice 19:560-568.
- Goetzel RZ, Staley P, Ogden L, Stange P, Fox J, Spangler J, Tabrizi M, Beckowski M, Kowlessar N, Glasgow RE, Taylor MV (2011). A framework for patient-centered health risk assessments – providing health promotion and disease prevention services to Medicare beneficiaries. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention, Available at: http://www.cdc.gov/policy/opth/hra/.
- Glasgow RE, Kaplan R, Ockene J, Fisher EB, Emmons KM, Society of Behavioral Medicine Health Policy Committee (2012). Patient-Reported Measures of Psychosocial Issues And Health Behavior Should be Added to Electronic Health Records. Health Affairs 31:3497-504. Estabrooks PA, Boyle M, Emmons KM, Glasgow RE, Hesse BW, Kaplan RM, Krist AH, Moser RP,
- Taylor MV (2012). Harmonized Patient-Reported Data Elements in the EHR: Supporting Meaningful Use by Primary Care Action on Health Behaviors and Key Psychosocial Factors. Journal of American Medical Informatics Association July-August;19(4):575-82.
- Fernald, D.H., Froshaug, D., Dickinson, M., Balasubramanian, B.A., Dodoo, M., Holtrop, J.S., Hung, D., Glasgow, R.E., et al. (2008). A field test of brief health behavior measures in primary care: Results from the COMBO study. American Journal of Preventive Medicine 35(5S):S414-S422.
- Fisher, L., Glasgow, R.E., Mullan, J., Skaff, M. (2008). Development of a Brief Diabetes Distress Screening Instrument. Annals of Family Medicine 6(3):246-252.
- McCormack, L.A., Bann, C., Williams-Piehota, P.A., Burton, J., Kamerow, D., Square, C., Fisher, E., Glasgow, R.E.. (2008). Development and Validation of an Instrument to Measure Resources and Support for Illness Self-Management: A Model Using Diabetes. The Diabetes Educator, 34(4):707-718.
- Schmittdiel, J., Mosen, D.M., Glasgow, R.E., Hibbard, J., Sobel, D., Remmers, C., Bellows, J. (2008) Patient Assessment of Chronic Illness Care (PACIC) and Improve Processes and Outcomes for Chronic Conditions. Journal of General Internal Medicine, 23(1)77-80.
- Glasgow, R. E., Toobert, D.J., and Barrera, M., Jr. (2005). The Chronic Illness Resources Survey: Cross-Validation and Sensitivity to Intervention. Health Education Research, 29(4):402-409.1.
- Glasgow, R. E., Ory, M.G., Klesges, L.M., Cifuentes, M., Fernald, D.H., and Green, L.A. (2005). Practical and Relevant Self-Report Measures of Patient Health Behaviors for Primary Care Research. Annals of Family Medicine, 3:73-81.
- Glasgow, R. E., Wagner, E., Schaefer, J., Mahoney, L., Reid, R.J., and Greene, S. (2005). Development and Validation of the Patient Assessment of Chronic Illness Care (PACIC). Medical Care, 43(5):436-444.
- Bonomi, A.E., Glasgow, R. E., Wagner, E.H., et al. (2002). Assessment of Chronic Illness Care: A Practical Tool for Quality Improvement. Health Services Research 37(3):791-820.
- Glasgow, R. E., Gillette, C., Toobert, D.J. (2001) Psychosocial Barriers to Diabetes Self-Management and Quality of Life. Diabetes Spectrum, 14(1):
- Glasgow, R. E., Strycker, L.A., Toobert, D.J., and Eakin, E.G. (2000). The Chronic Illness Resources Survey: A social-ecologic support for disease self-management. Journal of Behavioral Medicine, 2000;23:559-583.
- Toobert, D. J., Glasgow, R. E., and Hampson, S. (2000). The Summary of Diabetes Self-Care Activities Measure: Results from Seven Studies and a Revised Scale. Diabetes Care, 2000;23:943-950.
- Glasgow RE and Riley T (2013). Pragmatic Measures: What They Are and Why We Need Them. American Journal of Preventive Medicine, August;45(2):237-43.
- Weiner, B. J., Lewis, C. C., Stanick, C., Powell, B. J., Dorsey, C. N., Clary, A. S., . . . Halko, H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science. 2017, 12:108
- Sauro, J., Johnson, K., Meenan, C. (2017) “From Snake-Oil to Science: Measuring UX Maturity.” Proceedings of the Conference in Human Factors in Computing Systems (CHI 2017) Denver, CO, USA.
Resources for Intervention Design
Making Communication More Patient-Centered
Patient-centered communication has been linked
to greater patient satisfaction, treatment adherence, and quality of life, and has the potential to help patients:
• Handle the emotional impact of a diagnosis
• Make sense of complex medical information
• Manage side effects and symptoms
• Make informed decisions about their care
• Deal with uncertainties
• Adopt health-promoting behaviors
The user guide
aims to help cancer researchers and practitioners identify gaps in patient-centered communication, which considers the patient’s needs and values, involves the patient in care decisions, and builds stronger patient-clinician relationships. The ultimate goal of this tool is to improve communication with patients who have cancer.
is available as both a 36- and 6-item questionnaire and can be self-administered using a paper- or Web-based survey.
D&I Evaluation Resources
- In Development-
Other References for D&I
L.K., Parcel, G.S., Kok, G., Gottlieb, N.H., and Fernández, M.E., Planning
Health Promotion Programs: An Intervention Mapping Approach. 3rd ed ed. 2011,
San Francisco: Jossey-Bass.
R.C., Colditz, G.A., and Proctor, E.K., Dissemination and implementation
research in health: Translating science
to practice. 1st ed. 2012, New York: Oxford University Press.
L.M., Estabrooks, P.A., Dzewaltowski, D.A., Bull, S.S., and Glasgow, R.E.
(2005) Beginning with the application in mind: designing and planning health
behavior change interventions to enhance dissemination. Annals of behavioral
medicine : a publication of the Society of Behavioral Medicine. 29 Suppl:
K.M. (2013) Considering context in quality improvement interventions and
implementation: concepts, frameworks, and application. Academic pediatrics.
13(6 Suppl): S45-53.
E., Powell, B.J., and McMillen, J.C. (2013) Implementation strategies:
recommendations for specifying and reporting. Implementation science : IS. 8:
E.M., Diffusion of innovations. 5th ed. 2003, New York: Free Press. xxi, 551 p.
A., Scammon, D.L., Waitzman, N.J., Cronholm, P.F., Halladay, J.R., Driscoll,
D.L., Solberg, L.I., Hsu, C., Tai-Seale, M., Hiratsuka, V., Shih, S.C.,
Fetters, M.D., Wise, C.G., Alexander, J.A., Hauser, D., McMullen, C.K.,
Scholle, S.H., Tirodkar, M.A., Schmidt, L., Donahue, K.E., Parchman, M.L., and
Stange, K.C. (2013) Context matters: the experience of 14 research teams in
systematically reporting contextual factors important for practice change.
Annals of family medicine. 11 Suppl 1: S115-23.
Chambers, D.A., Glasgow, R.E., and
Stange, K.C. (2013) The dynamic sustainability framework: addressing the
paradox of sustainment amid ongoing change. Implementation
science : IS. 8: 117.
Davidoff, F., Batalden, P., Stevens, D., Ogrinc, G., Mooney,
S., and Group, S.D. (2008) Publication guidelines for improvement studies
in health care: evolution of the SQUIRE Project. Annals of
internal medicine. 149(9): 670-6.
Gaglio, B., Shoup, J.A., and Glasgow, R.E. (2013) The RE-AIM
framework: a systematic review of use over time. American
journal of public health. 103(6): e38-46.
Glasgow, R.E.and Emmons, K.M. (2007) How can we increase
translation of research into practice? Types of evidence needed. Annual
review of public health. 28: 413-33.
Glasgow, R.E., Vinson, C., Chambers, D., Khoury, M.J.,
Kaplan, R.M., and Hunter, C. (2012) National Institutes of Health approaches to
dissemination and implementation science: current and future directions. American
journal of public health. 102(7): 1274-81.
Glasgow, R.E.and Chambers, D. (2012) Developing robust,
sustainable, implementation systems using rigorous, rapid and relevant science. Clinical
and translational science. 5(1): 48-55.
Gold, M.R., Siegel, J.E., Russell, L.B., and Weinstein,
M.C., Cost-effectiveness in health and medicine. 2003, New York:
Oxford University Press.
Green, L.W.and Glasgow, R.E. (2006) Evaluating the
relevance, generalization, and applicability of research: issues in external
validation and translation methodology. Evaluation & the
health professions. 29(1): 126-53.
Husereau, D., Drummond, M., Petrou,
S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A.H.,
Mauskopf, J., Loder, E., and Force, C.T. (2013) Consolidated Health Economic
Evaluation Reporting Standards (CHEERS) statement. Bmj. 346:
Kessler, R.S., Purcell, E.P.,
Glasgow, R.E., Klesges, L.M., Benkeser, R.M., and Peek, C.J. (2013) What does
it mean to "employ" the RE-AIM model? Evaluation & the
health professions. 36(1): 44-66.
Klassen, A.C., Creswell, J., Plano Clark, V.L., Smith, K.C.,
and Meissner, H.I. (2012) Best practices in mixed methods for quality of life
research. Quality of life research : an international journal of
quality of life aspects of treatment, care and rehabilitation. 21(3):
Leviton, L.C., Khan, L.K., Rog, D., Dawkins, N., and Cotton,
D. (2010) Evaluability assessment to improve public health policies, programs,
and practices. Annual review of public health. 31:
Peters, D.H., Adam, T., Alonge, O., Agyepong, I.A., and Tran,
N. (2013) Implementation research: what it is and how to do it. Bmj. 347:
Riley, W.T., Glasgow, R.E., Etheredge, L., and Abernethy,
A.P. (2013) Rapid, responsive, relevant (R3) research: a call for a rapid
learning health research enterprise. Clinical and translational
medicine. 2(1): 10.
Ritzwoller, D.P., Sukhanova, A., Gaglio, B., and Glasgow,
R.E. (2009) Costing behavioral interventions: a practical guide to enhance
translation. Annals of behavioral medicine : a publication of
the Society of Behavioral Medicine. 37(2): 218-27.
Stokols, D. (2000) Social ecology and behavioral medicine:
implications for training, practice, and policy. Behavioral
medicine. 26(3): 129-38.
Stirman, S.W., Miller, C.J., Toder, K., and Calloway, A.
(2013) Development of a framework and coding system for modifications and
adaptations of evidence-based interventions. Implementation
science : IS. 8: 65.
Tabak, R.G., Khoong, E.C., Chambers, D.A., and Brownson,
R.C. (2012) Bridging research and practice: models for dissemination and
implementation research. American journal of preventive medicine. 43(3):
Thorpe, K.E., Zwarenstein, M., Oxman, A.D., Treweek, S.,
Furberg, C.D., Altman, D.G., Tunis, S., Bergel, E., Harvey, I., Magid, D.J.,
and Chalkidou, K. (2009) A pragmatic-explanatory continuum indicator summary
(PRECIS): a tool to help trial designers. CMAJ : Canadian
Medical Association journal = journal de l'Association medicale canadienne. 180(10):
Zwarenstein, M., Treweek, S., Gagnier, J.J., Altman, D.G.,
Tunis, S., Haynes, B., Oxman, A.D., Moher, D., group, C., and Pragmatic Trials
in Healthcare, g. (2008) Improving the reporting of pragmatic trials: an
extension of the CONSORT statement. Bmj. 337:
- More References and Annotations In Development-
The Living Textbook reflects a collection of expert consensus regarding special considerations, standard approaches, and best practices in the design, conduct, and reporting of pragmatic clinical trials. Given the rapid pace of change in this field, this electronic textbook will continue to be added to and updated.
The RE-AIM Online Training is a web-based module that provides instruction and case examples to illustrate the five dimensions of the RE-AIM framework: Reach, Effectiveness or efficacy, Adoption, Implementation, and Maintenance. The RE-AIM framework is useful for planning new interventions, adapting existing interventions, and designing evaluations that assess the potential public health impact of interventions. The module provides users with examples of real-life application to policy/environmental change interventions.
The MRM website consists of four main tools:
> Planning Tool-an interactive survey which provides a tailored report that aids researchers with their dissemination plan;
> Resource Library-a searchable database consisting of a compilation of D&I related articles from multiple sources which is updated monthly;
> Narrative Library-a freely accessible online library containing video vignettes and transcripts with junior and senior D&I experts of "how-to" knowledge to D&I problems;
> Glossary-containing over 100 definitions of terminology used in D&I health research. Additionally, users of the MRM website can learn more about current publications and presentations, and current news related to D&I.
This interactive website was designed to help researchers and practitioners to select the D&I Model that best fits their research question or practice problem, adapt the model to the study or practice context, fully integrate the model into the research or practice process, and find existing measurement instruments for the model constructs.
- The term ‘Models’ is used to refer to both theories and frameworks that enhance dissemination and implementation of evidence-based interventions more likely.
- Pragmatic trials are randomized controlled studies whose purpose is to inform decisions about practice. They address questions of major clinical and public health importance and produce results that can be generalized and applied in usual care settings. Because they produce real-world evidence, they help to close the translation gap between discovery and practice.
- This workbook is designed to be a navigation guide providing key references and online resources to aid you in further study and application.
- Closing the gap between research discovery and clinical and community practice is essential if we are to succeed in improving our nation’s health. Understanding how best to disseminate and implement evidence-based healthcare and prevention, while not implementing inappropriate or unnecessary services, is challenging.
- This Guide was developed by the Center for Research in Implementation Science and Prevention (CRISP) in partnership with colleagues from the Colorado Clinical and Translational Sciences Institute (CCTSI), the Kaiser Permanente Center for Health Education Dissemination and Implementation Research, and the Department of Veterans Affairs.
RE-AIM Tools and Calculators
- This site provides an explanation of and resources for those wanting to apply the RE-AIM framework. The RE-AIM framework is designed to enhance the quality, speed, and public health impact of efforts to translate research into practice.
- Within Tools and Resources, you will find measures, checklists, visual displays, figures and tables, helpful links, online modules and even a self-rating quiz to determine confidence in acheiving each element of the RE-AIM dimensions.
- The Grid-Enabled Measures Database (GEM) is a web-based collaborative tool containing behavioral, social science, and other relevant scientific measures. The goal of GEM is "to support and encourage a community of users to drive concensus on best measures and share the resulting data from those measures."
- Because users can search for and download measures by subject area or keyword, this web resource is a great place for dissemination and implementation experts to access, contribute toward, rate and even add to a database of measures for D & I and other areas.
- The Evaluation Hub is a resource available to all members of the Department of Family Medicine (DFM) - that brings together evaluation expertise and resources via one easy access point. The DFM has made available a limited amount of funding to facilitate access to evaluation consultation services at no cost to you.
- If you are working on a departmental program or project that needs evaluation help, the Hub is here to support you!
Measures for Person Centred Coordinated Care:
The website is an online database of Patient Reported Outcomes Measures (PROMs) and Patient Reported Experience Measures (PREMs) that can be utilized within programs that aim to deliver or evaluate person centred coordinated care (P3C).
- This project is designed to be of value to a range of stakeholders (such as healthcare professionals, commissioners and managers), but the resource is also a useful research tool, with features such as links to PubMed literature.
- An international collaboration to improve evaluation practice and theory by sharing and generating information about options (methods or processes) and approaches.
- Having an agreed set of Key Evaluation Questions (KEQs) makes it easier to decide what data to collect, how to analyze it, and how to report it. The link above will take you directly to the entry on "Key Evaluation Questions."
- SIRC is a society that brings together researchers and stakeholders committed to the rigorous evaluation of implementation of evidence based psychosocial interventions. SIRC conferences occur biennially. The next conference will be held in Seattle on September 25 & 26, 2015 centered around the theme of “Advancing Efficient Methodologies through Community Partnerships and Team Science.”
- Resources on the site include event registration, information on past conferences, news, updates, description of initatives and access to the SIRC "Network of Expertise" (NoE). While the core group of the NoE includes experienced investigators, practicioners, and champions of EBP, the network is welcome to members at all levels.
1. Chambers DA, Proctor
EK, Brownson RC, Straus SE. Mapping training needs for dissemination and
implementation research: lessons from a synthesis of existing D&I research
training programs. Transl Behav Med. 2016 Mar 30
2. Luke DA, Baumann AA,
Carothers BJ, Landsverk J, Proctor EK. Forging a link between mentoring and
collaboration: a new training model for implementation science. Implement
3. Meissner HI, Glasgow RE, Vinson CA, et al. The U.S. training institute for dissemination and implementation research in health. Implement Sci. 2013;8:12.
4. Padek M, Colditz G, Dobbins M, et al. Developing educational competencies for dissemination and implementation research training programs: an exploratory analysis using card sorts. Implement Sci. 2015;10:114.
5. Proctor EK, Landsverk J, Baumann AA, et al. The implementation research institute: training mental health implementation researchers in the United States. Implement Sci. 2013;8:105.