Glossary

The following definitions have been drawn from numerous sources and are tailored specifically for content on the NREPP Web site. The terms defined here may have slightly different meanings in other settings. 

A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z

Addiction 
The overpowering physical or emotional urge to continue alcohol or other drug (AOD) use in spite of adverse consequences. In the context of AOD, addiction is a cluster of chronic disorders that spring from multiple, interacting etiological influences and that vary considerably in their onset, course, and outcome. 

Source: William White Papers

Adjusted effect size 
NREPP uses the term adjusted effect size to refer to effect sizes that take into account or control for pretest differences when calculated. Adjusted effect sizes are, on balance, more reliable estimates of a treatment effect than unadjusted effect sizes. 

Analysis method 
A criterion on the NREPP outcome rating instrument that assesses whether an analysis method is appropriate, given the nature of the data. 

Attention deficit and hyperactivity disorder (ADHD) 
A mental disorder characterized by excessive hyperactivity, impulsivity, or inattention. While ADHD is typically diagnosed in childhood, symptoms may persist into adulthood. ADHD is most effectively treated with a combination of medication and counseling. 

Source: SAMHSA

Attrition 
The loss of study participants during the course of the study due to voluntary dropout or other reasons. Higher rates of attrition can potentially threaten the validity of studies. Attrition is one of the six NREPP criteria used to rate Quality of Research. 

Author Query 
A request sent to study authors for additional or clarifying information needed to rate a study. Studies are rated based solely on the information presented in the study in the absence of additional information in response to an author query. 

Source: What Works Clearinghouse 

back to top 

Behavioral Health 
A state of mental/emotional being and/or choices and actions that affect wellness. Substance use and misuse are one set of behavioral health problems. Others include (but are not limited to) serious psychological distress, suicide, and mental illness. 

Source: SAMHSA 

back to top 

Cognitive behavioral therapy 
A blend of two therapies: cognitive therapy and behavioral therapy. Cognitive therapy focuses on a person's thoughts and beliefs, and how they influence a person's mood and actions, and aims to change a person's thinking to be more adaptive and healthy. Behavioral therapy focuses on a person's actions and aims to change unhealthy behavior patterns. 

Source: NIMH

Cohen’s d 
A standardized mean difference estimate, or effect size, which reflects the size or magnitude of the difference between two groups on a continuous dependent variable, with larger values representing a greater differentiation between the two groups. A sample-size corrected version of the Cohen’s d is a Hedges’ g. 

Cohort 
A group of people with some characteristic in common, such as age, sex, race, educational background, geographic location, employment, and the like. For example, the Women’s Health Initiative was a large study that collected information from a group of older women (a cohort) who were followed over several years. 

Source: AHRQ

Comorbidity 
When two disorders or illnesses occur in the same person, simultaneously or sequentially. Comorbidity also implies interactions between the disorder/illness that affect the course and prognosis of both. 

Source: NIDA

Comparative effectiveness research 
The conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in “real world” settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances. 

Source: Sage Encyclopedia of Social Science Research Methods

Comparison group 
A group of units (e.g., persons, classrooms) that receive either no treatment or an alternative treatment. The purpose of a comparison group is to serve as a source for an alternative causal inference. To make the comparison valid, the composition and characteristics of the comparison group should resemble that of the treatment group as closely as possible. However, a comparison group is not as closely matched to the treatment group as is a control group; a control group might be constructed using random assignment of study participants to the treatment group and to the control group. 

Source: Sage Encyclopedia of Social Science Research Methods

Conceptual framework 
A graphic or written description intended to articulate a program’s goals, components, and theory of change. 

Confidence Interval 
A numeric interval that indicates the likely range of an intervention’s effect. It is primarily influenced by sample size (smaller samples will have larger confidence intervals) and the standard deviation observed in the intervention and comparison groups (smaller the standard deviations will result in smaller confidence intervals). A confidence interval is defined by two numbers, one lower than the result found in the study and the other higher than the study's result. 

Confounding variables 
A numeric interval that indicates the likely range of an intervention’s effect. It is primarily influenced by sample size (smaller samples will have larger confidence intervals) and the standard deviation observed in the intervention and comparison groups (smaller the standard deviations will result in smaller confidence intervals). A confidence interval is defined by two numbers, one lower than the result found in the study and the other higher than the study's result. 

Continuous outcome 
A quantitative measure for which the number represents a quantity that can be described in terms of order, spread between the numbers, and/or relative amounts. Age, for example, is a continuous variable. 

Control group 
A group of units (e.g., persons, classrooms) that are randomly assigned to receive either no treatment or an alternative treatment (in contrast to the units randomly assigned to the intervention group). The purpose of a control group is to serve as a source for an alternative causal inference. 

Co-occurring disorders 
The co-existence of a substance use disorder and a mental health disorder at the same time (for example, alcohol dependence and depression). The combination of disorders can include any two or more of those identified in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). This condition is sometimes referred to as comorbidity. 

Core components 
The most essential and indispensable components of an intervention (core intervention components) or the most essential and indispensable components of an implementation program (core implementation components). 

Cronbach’s alpha coefficient 
The consistency of responses on a measure that contains more than one component (i.e., items, raters).The estimate captures the extent to which all the separate items on the measure move in the same direction (e.g.., if one item of anxiety is rated high, then other items related to anxiety on a measure are also rated high). The greater the similarity among items, the higher the reliability (and thus the higher the value of Cronbach’s alpha coefficient). Values of the alpha can range from -1.0 to 1.0, with greater values indicating stronger internal consistency. 

Cultural appropriateness 
In the context of public health, sensitivity to the differences among ethnic, racial, and/or linguistic groups and awareness of how people's cultural background, beliefs, traditions, socioeconomic status, history, and other factors affect their needs and how they respond to services. Generally used to describe interventions or practices. 

Source: Journal of Community Psychology

Cultural competence 
The process by which individuals and systems respond respectfully and effectively to people of all cultures, languages, classes, races, ethnic backgrounds, religions, and other diversity factors in a manner that recognizes, affirms, and values the worth of individuals, families, and communities and protects and preserves the dignity of each. Because of the changing nature of people and cultures, cultural competence is seen as a continual and evolving process of adaptation and refinement. 

Source: NASW

back to top 

Dichotomous outcome 
A measure for which there are only two values (e.g., yes or no). 

Source: The Practice of Research in Social Work

Differential attrition 
The difference in attrition rates for the intervention and comparison groups. 

Source: What Works Clearinghouse

Dissemination 
The targeted distribution of program information and materials to a specific audience. The intent is to spread knowledge about the program and encourage its implementation. 

DSM (Diagnostic and Statistical Manual of Mental Disorders) 
The Diagnostic and Statistical Manual of Mental Disorders, or DSM, is the standard reference handbook used by mental health professionals in the United States to classify mental disorders. There have been multiple revisions of the DSM since it was first published by the American Psychiatric Association in 1952. The most recent version is the DSM-5, or Fifth Edition, which was published in 2013. Other versions include the DSM-IV, or Fourth Edition, which was published in 1994, and a text revision of the DSM-IV (DSM-IV-TR), which was published in 2000. Earlier editions that may be referenced in NREPP include the DSM-III (1980) and the DSM-III-R (1987). 

back to top 

Effect class 
NREPP assigns an effect class to each evaluated measure, based on the confidence interval of the effect size. For more information, see our web page, ‘Review Process’. There are four different effect classes:

  • Favorable: Confidence interval lies completely within the favorable range

  • Probably favorable: Confidence interval spans both the favorable and negligible range

  • Trivial: Confidence interval lies completely within the negligible range or spans the negligible and favorable range

  • Probably harmful: Confidence interval spans both the harmful and negligible range

For more information, see Review Process

Effect size 
A measure of the impact of an intervention; or the measure of the amount of change in a condition or symptom that results from receiving an intervention, as compared with not receiving the intervention. Effect size may be measured, calculated, and reported in many different ways, for example, as an odds ratio, a Hedges’ g, or a Cohen’s d. Effect size (ES) is a criterion on the NREPP outcome rating instrument. NREPP reports Hedges’ g effect sizes. 

Effective 
NREPP uses the term Effective to refer to outcomes with an evidence base that produced strong evidence of a favorable effect. For more information, see Review Process. 

Efficacy 
Refers to whether an intervention or treatment works under the best possible conditions; for example, under the controlled conditions of a rigorous research study. 

Emotional Disturbance 
Formerly referred to as serious emotional disturbance (SED), is a category of educational disability that is most often applied to academically struggling students whose emotional and behavioral problems are the most prominent aspect of their educational disability. 

Evidence class 
NREPP assigns an evidence class to each evaluated measure (classes a through g); based on a combination of the evidence score and effect class. For more information, see our web page, ‘Review Process’ For more information, see Review Process. 

Evidence score 
NREPP assigns an evidence score to each evaluated measure, based on the rigor and fidelity dimensions and is rated highest quality, sufficient, or inconclusive. For more information, see Review Process. 

Evidence-based practice 
A practice that is based on rigorous research that has demonstrated effectiveness in achieving the outcomes that it is designed to achieve. 

Exposure 
Also called dosage, refers to the amount of an intervention received by participants in a program. It may include any of the following: the number of sessions implemented; the length of each session; or the frequency with which program techniques were implemented. 

Source: Emotional & Behavioral Disorders in Youth

External validity 
Also referred to as generalizability, refers to the degree to which a program effect can be generalized to other populations, settings, or contexts beyond those represented in the study. 

Externalizing behaviors 
Refers to aggressive, defiant, reactive, impulsive, and other acting-out or antisocial behaviors. Examples include spontaneous weeping, "acting out" and uncharacteristic aggression. Reduction of externalizing behaviors is a frequently used measure of the success of treatment or intervention for mental or emotional disorders. 

back to top 

Face validity 
A measurement approach is said to have face validity when that approach appears to measure what it is intended to measure. Face validity is a very subjective form of validity and it is the weakest form of validity. 

Fidelity 
Fidelity of implementation occurs when implementers of a research-based program or intervention (e.g., teachers, clinicians, counselors) closely follow or adhere to the protocols and techniques that are defined as part of the intervention. Program fidelity is a criterion on the NREPP outcome rating instrument that assesses the integrity of service utilization (whether an intervention reaches the target population) and service delivery (the degree to which core program services or components are implemented as designed).

back to top 

Generalizability 
Also referred to as external validity, refers to the degree to which a program effect can be generalized to other populations, settings, or contexts beyond those represented in the study. Generally speaking, the external validity of research in behavioral health is not a key factor in assessing methodological rigor and therefore it is not assessed as part of the NREPP Outcome Rating Instrument. 

back to top 

Hedges’ g 
A sample-size corrected, standardized mean difference estimate, or effect size, which reflects the size or magnitude of the difference between two groups on a continuous dependent variable, with larger values representing a greater differentiation between the two groups. It is similar to a Cohen’s d, except that a Cohen’s d is not corrected for sample size. 

back to top 

Implementation 
A planned, coordinated group of activities, processes, and procedures designed to achieve a specific purpose. An intervention should have specified goals, objectives, and structured components (e.g., a defined curriculum, an explicit number of treatment or service hours, and an optimal length of treatment) to ensure the intervention is implemented with fidelity to its model. 

Implementation team 
A core set of individuals charged with providing guidance through full implementation of the intervention. This team helps ensure engagement of the stakeholders, increases readiness for implementation, ensures fidelity to the intervention, monitors outcomes, and addresses barriers to implementation. 

Inconclusive 
NREPP uses the term Inconclusive to refer to outcomes with weak overall rigor and to outcomes that lack effect size information; both situations preclude us from providing an accurate assessment of the effect. For more information, see our web page, Review Process. 

Indicated prevention strategies 
Focus on preventing the onset or development of problems in individuals who may be showing early signs but are not yet meeting diagnostic levels of a particular disorder. This is one of the three categories (universal, selective, indicated) developed by the Institute of Medicine to classify preventive interventions. 

Source: Delaware Department of Health and Social Services

Ineffective 
NREPP uses the term Ineffective to refer to outcomes based on an evidence base which would produce sufficient evidence of a negligible, possibly harmful, or wide-ranging effect. For more information, see our web page, Review Process 

Intent-to-treat analysis 
A type of analysis that ensures that all participants’ results are assessed based on their initial group of assignment, not on the treatment eventually administered. For example, if the study has a high attrition rate, or outcomes are compared only for those who completed the treatment, the study results may be biased. An intent-to-treat design ensures that all study participants are followed until the conclusion of the study, irrespective of whether the participant is still receiving or complying with the treatment. 

Source: Crime Solutions.gov

Intent-to-treat Original-Group Assignment 
A criterion of the NREPP Outcome Rating Instrument that assesses the degree to which the analysis preserves assignment of individuals/clusters to their original groups, regardless of the intervention they actually received or of how much of the intervention was received. 

Internal consistency 
An estimate of the consistency (or internal reliability) of the responses toward the items or a subset of the items on a measure. The more highly correlated items are with each other, the higher the internal consistency and Cronbach’s alpha coefficient will be. 

Source: Sage Encyclopedia of Social Science Research Methods

Internal validity 
A research study is said to have internal validity when it accurately identifies a causal relationship.

Source: Sage Encyclopedia of Social Science Research Methods 

Internalizing behaviors 
Behavior or symptoms that reflect an individual's transfer of external social or situational stresses to emotional, psychological, or physical symptoms. One well-known internalizing behavior is a child's development of stomach cramps when the parents argue; another is insomnia during a high-stress situation at work. Reduction of internalizing behaviors is a frequently used measure of the success of treatment or intervention for mental or emotional disorders. 

Intervention 
A strategy or approach intended to prevent an undesirable outcome (preventive intervention), promote a desirable outcome (promotion intervention) or alter the course of an existing condition (treatment intervention). 

back to top 

Legacy Programs 
Refers to those programs that were reviewed under earlier NREPP criteria and standards in effect from 2008 through September 2015. NREPP is re-reviewing all programs posted prior to September 2015 under the new criteria, whereupon the programs will be removed from the legacy programs and be posted with the newly reviewed NREPP programs. For more information on the new review criteria, see Review Criteria page.

Likert scale 
A very popular device for measuring psychological constructs. It refers to a type of survey question in which a person is asked to rate their reaction to a statement along a continuum or scale. The scale typically runs from a positive rating to a negative rating with a neutral score in between. 

Source: Sage Encyclopedia of Social Science Research Methods

Logic model 
Depicts the relationships among the inputs, activities, outputs, and outcomes of a program or programs. A logic model can be used to develop a strategic plan for designing, implementing, and evaluating the outcomes of a program. 

back to top 

Measure 
In behavioral health, a measure is the method by which knowledge, perceptions, emotions, symptoms, behaviors, and interactions are quantified. They may express whether a condition is present (incidence), how often a condition is present (frequency) or how severe a condition is (severity). Examples of measures include observation protocols, scales, indices, diagnostic interviews, and physiological tests. 

Measurement quality 
A criterion of the NREPP Outcome Rating Instrument that is based on the face validity and reliability of the measurement(s) used to assess the outcome. 

Mental health disorder 
A behavioral or psychological syndrome or pattern that occurs in an individual that reflects an underlying psychobiological dysfunction and results in clinically significant distress (e.g., a painful symptom) or disability (i.e., impairment in one or more important areas of functioning). Individuals with similar syndromes/patterns should not be classified as having a mental health disorder if they are exhibiting an expectable response to common stressors and losses (for example, the loss of a loved one) or a culturally sanctioned response to a particular event (for example, trance states in religious rituals). 

Source: American Psychiatric Association. 

Mental health promotion 
Refers to attempts to optimize positive mental health by increasing protective factors and healthy behaviors that can help prevent the onset of a diagnosable mental disorder, as well as reducing risk factors that can lead to the development of a mental health disorder. It also involves creating living conditions and environments that support mental health and allow people to adopt and maintain healthy lifestyles. 

Source:  Youth.gov 

Mental health treatment 
Services provided to individuals for existing mental health conditions or disorders by trained professionals (e.g., psychiatrists, psychologists, counselors). 

Meta-analysis 
Refers to the statistical analysis of results from individual studies for purposes of integrating the findings to arrive at a more general, and more generalizable, conclusion about the impact of an intervention on specified outcomes. 

Source: Sage Encyclopedia of Social Science Research Methods

Missing data 
Information that researchers intended to collect during a study that was not actually collected or was collected incompletely. Missing data may occur, for example, when survey respondents do not answer all questions in a survey, or when the researchers "throw out" or exclude survey questions because the responses do not meet validation checks. Missing data can threaten the validity and reliability of a study if steps are not taken to compensate for or "impute" (replace with calculated data) the missing information. 

back to top 

Odds ratio 
An effect size appropriate for dichotomous or binary (yes/no) outcomes. An odds ratio is calculated by dividing a) the odds of an event occurring by b) the odds of an event not occurring. Hence, it is a ratio of odds. The odds of an event occurring is generally calculated by dividing the number of occurring events by the total number of events measured. 

Outcome 
The state of the target population or the social conditions that a program is expected to change. The change can be quantified using standardized scales or assessment tools. In the context of NREPP, outcomes refer to measurable changes in the behavioral health of an individual or group of people that are attributable to the intervention. In contrast, an output of an intervention is usually an activity or process that is produced by the intervention. For example, the number of therapy sessions is an output of a psychotherapy intervention, whereas a change in attitudes or behaviors is an outcome of that intervention. 

Source: Evaluation: A systematic approach (7th ed). 

Outcome evaluation 
Assesses impact of an intervention on one or more specified outcomes. For example, an outcome evaluation of psychotherapy could measure the impacts on the attitudes or behaviors of the patients. An outcome evaluation is different from a process evaluation: a process evaluation examines the impact of the intervention on outputs, such as activities or processes that are intended to contribute to changes in outcomes. 

Source: Evaluation: A systematic approach (7th ed). 

Outcome rating 
The rating NREPP designates for each program outcome. NREPP has four outcome ratings: Effective, Promising, Ineffective, and Inconclusive. 

OR 
The evidence base produced sufficient evidence of a possibly harmful effect. 

Inconclusive: Limitations in the study design or a lack of effect size information preclude from reporting further on the effect. 

For more information, see Review Process. 

Overall attrition 
The percent of the pretest sample not represented in the analytic sample. 

Source: What Works Clearinghouse

back to top 

Posttraumatic stress disorder (PTSD) 
A condition that may develop after a terrifying ordeal that involved extreme emotional and/or physical harm or the threat of extreme emotional and/or physical harm. The person who develops PTSD may have been the one who was harmed, or who observed harm being done to others. Individuals with PTSD may experience such symptoms as recurring flashbacks, bad dreams, and terrifying thoughts, which cause problems in a person’s daily routine. 

Source: NIMH 

Practice 
A general category of strategies or procedures that share similar characteristics in order to achieve a similar purpose. 

Source: CrimeSolutions.gov

Pre-experimental 
An experimental design that follows some basic steps used in experiments, but either fail to include a pretest, a comparison group, or both. In addition, randomization procedures are not used to control for extraneous variables. Pre-experimental designs represent the simplest form of research designs and are not included in NREPP evidence reviews.

Source: Sage Encyclopedia of Social Science Research Methods

Pretest adjustment 
A criterion on the NREPP outcome rating instrument that assesses whether observed pretest differences are adjusted for in statistical analyses.

Pretest equivalence
A criterion on the NREPP outcome rating instrument that assesses the similarities between study groups on observed variables at pretest.

Prevention strategies
Strategies that seek to prevent the onset of various physical and behavioral health disorders. The Institute of Medicine has defined three types of preventions strategies -- universal prevention strategies, selective prevention strategies, and indicated prevention strategies -- defined as follows:

  • Universal prevention strategies address the entire population (such as national, local community, school, or neighborhood), with messages and programs to prevent or delay the development of behavioral health disorders.

  • Indicated prevention strategies focus on preventing the onset or development of problems in individuals who may be showing early signs but are not yet meeting diagnostic levels of a particular disorder.

  • Selective prevention strategies focus on specific groups viewed as being at higher risk for mental health disorders or substance use disorders because of highly correlated risk factors (for example, the children of parents with substance use problems).

Source: National Academies of Science 

Process evaluation 
Measures the impact of an intervention on processes and other outputs that are intended to contribute to changes in outcomes. A process evaluation can be used to determine whether an intervention has been implemented as designed and to the target population. 

Source: Evaluation: A systematic approach (7th ed)

Program 
A program is a planned, coordinated group of activities, processes, and procedures designed to achieve a specific purpose. A program should have specified goals, objectives, and structured components (e.g., a defined curriculum, an explicit number of treatment or service hours, and an optimal length of treatment) to ensure the program is implemented with fidelity to its model. 

Source: CrimeSolutions.gov

Program drift 
The gradual deviation of a program from its original design, as the program is implemented and, perhaps inadvertently, altered in response to conditions in new communities and new populations. 

Program fit 
How well a program matches, or is appropriate for, the community, organization, stakeholders, and potential participants (i.e., youth). Appropriate program selection increases the likelihood that the program will be implemented with fidelity, thereby increasing the likelihood of achieving the desired health outcomes. Appropriate questions for assessing program fit include: Is the program appropriate for the population, culture, and setting? Is it feasible culturally, politically, administratively, technically, and financially? 

Source: ACF  

Promising 
Outcomes based on an evidence base which produced sufficient evidence of a favorable effect. For more information, see Outcome Ratings.

Protective factor 
Any factor, internal or external to an individual, that contributes to the prevention of negative health or behavioral health outcomes or that promotes positive outcomes.

Source: British Journal of Psychiatry

Protocol 
A carefully designed instrument or set of directions used consistently in research and implementation to safeguard the participants’ health, answer specific research questions, or maintain fidelity.

Source: NIH

p-value 
Expresses the probability that a given statistical result is due to chance. Typically, if the p-value is less than or equal to 1 in 20 (i.e., ≤ 5 in 100, or p ≤ .05), researchers can conclude that the outcome of the study is not due to chance alone. Due to the fact that p values are influenced by sample size, NREPP uses information from effect sizes and their confidence intervals, which are less affected by sample size, to determine final outcome ratings. 

Source: Sage Encyclopedia of Social Science Research Methods

back to top 

Quality assurance 
The activities and processes used to check fidelity and the quality of implementation. 

Quasi-experimental 
A study design in which (1) the intervention is compared with one or more control or comparison conditions, (2) subjects are not randomly assigned to study conditions, and (3) data are collected at pretest and posttest. The quasi-experimental study design provides strong but more limited scientific rigor relative to an experimental design. 

back to top 

Randomized controlled trial 
A study design in which individuals or clusters are randomly assigned to two or more groups. 

Source: AHRQ

Readiness for Dissemination 
One of the two main categories of the NREPP rating system used from 2008 through September 2015. Readiness for Dissemination (RFD) was how NREPP quantified and described the quality and availability of an intervention's training and implementation materials. More generally, it described how easily the intervention can be implemented with fidelity in a real-world application using the materials and services that are currently available to the public. Legacy programs contain this information. For more information, see Legacy Programs. 

Recovery 
A process of change through which individuals improve their health and wellness, live a self-directed life, and strive to reach their full potential. Through the Recovery Support Strategic Initiative, SAMHSA has delineated four major dimensions that support a life in recovery: Health, Home, Purpose, and Community. 

Source: SAMHSA 

Relapse
The return of a disease or the signs and symptoms of a disease after a period of improvement. Relapse also refers to returning to the use of an addictive substance or behavior, such as cigarette smoking.

Source: NCI Dictionary of Cancer Terms

Reliability 
A measurement approach is said to be reliable, or to have reliability, when that approach measures the same characteristic or outcome in a consistent fashion, even over multiple measurements over a period of time. In short, reliability refers to the ability of a measurement approach to be consistent. Measurement reliability is a criterion on the NREPP outcome rating instrument that assesses the stability and consistency of the measurement. 

Replication
A follow-on study that uses the same study protocol on the same or a similar study population, or that uses a slightly modified protocol on a slightly different population, and that comes up with results that are largely consistent with the results of the original study. 

Reported effect 
The dependent variable being rated in the NREPP outcome rating instrument, referred to on NREPP program profiles as a ‘measure’. Findings for each reported effect are aggregated to the outcome level and combined with information about program fidelity and conceptual framework to obtain an outcome rating. 

Resources for Dissemination and Implementation (RFDI) 
Materials and assets that have been developed to facilitate the effective implementation or dissemination of practices or programs. These may include materials to educate the public or potential participants about the program and its goals (dissemination) or materials such as training manuals to guide implementation activities. For more information, see NREPP 

Resources for Dissemination and Implementation (RFDI) checklist 
Inventory that provides the opportunity for the practice or program developer(s) to identify materials and assets that have been developed to facilitate the effective implementation or dissemination of practices or programs. For more information, see NREPP 

Rigor 
A category of the NREPP outcome rating instrument that assesses aspects of study design, statistical analysis, attrition, measurement and other factors related to the internal validity of an effectiveness study. 

Risk assessment 
An instrument that assesses the potential for negative health or behavioral health outcomes for a study population, in light of the various risk factors and protective factors that are affecting or could be affecting that study population. 

Risk factor
Any factor, internal or external to an individual, which may contribute to negative health conditions and behavioral health outcomes for an individual or a population of individuals. 

back to top 

Sample size 
The number of units (e.g., people, schools) included in a study. In research reports, overall sample size is usually expressed as "N,” whereas “n” refers to the sample size in subgroups such as intervention groups. In general, studies with larger sample sizes have a broader range of participants. This increases the chance that the study's findings apply to the general population. Larger sample sizes also increase the chance that rare events (such as adverse effects) will be detected. 

Source: AHRQ

Schizophrenia 
A brain disorder that impacts the way a person thinks (often described as a “thought disorder”), and is characterized by a range of cognitive, behavioral, and emotional experiences that can include: delusions, hallucinations, disorganized thinking, and grossly disorganized or abnormal motor behavior. Although these symptoms are chronic and severe, significantly impairing occupational and social functioning, recovery is possible. 

Source: SAMHSA

Selective prevention strategies 
Selective prevention strategies focus on specific groups viewed as being at higher risk for mental health or substance use disorders because of highly correlated risk factors (for example, the children of parents with substance use problems). This is one of the three categories (universal, selective, indicated) developed by the Institute of Medicine to classify preventive interventions.

Sources: Institute of Medicine 

Serious mental illness (SMI) 
SMI among people ages 18 and older is defined at the federal level as having, at any time during the past year, a diagnosable mental, behavioral, or emotional disorder that causes serious functional impairment, which substantially interferes with or limits one or more major life activities. Serious mental illnesses include major depression, schizophrenia, and bipolar disorder, and other mental disorders that cause serious impairment.

Source: SAMHSA  

Service delivery 
A measure of whether the service delivery and support functions are consistent with the program design specifications or other appropriate standards. Service delivery is likely to have integrity if (1) standardized materials are used to implement the program, (2) training is provided to program staff, (3) the key features of the program are implemented as designed, and (4) participants are provided sufficient exposure to the program. 

Source: Evaluation: A systematic approach (7th ed)

Service utilization 
A measure of whether the program is reaching the appropriate target population. 

Source: Evaluation: A systematic approach (7th ed)

Statistical significance 
Statistical significance is the term used to describe whether the results are not likely due to a chance occurrence. Authors often express the results of a hypothesis test (e.g., testing whether an intervention effect is statistically significant) in the form of a p-value. A program effect is considered statistically significant if the p-value is less than a certain threshold (traditionally .05). 

Source: AHRQ

Study design 
The method by which intervention and comparison groups are determined. Design/assignment is a criterion on the NREPP outcome rating instrument that assesses how well the study minimizes selection bias at the design phase. The two types of study designs reviewed by NREPP are randomized controlled trials (RCTs) and quasi-experimental designs (QEDs).

Source: What Works Clearinghouse 

Study population 
The group of people participating in a research study. The study population often includes people with a particular problem, condition, or common set of particular characteristics.

Source: AHRQ 

Substance abuse 
The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), no longer uses the terms substance abuse and substance dependence, rather it refers to substance use disorders (see substance use disorder below). 

Source: SAMHSA 

Substance use prevention 
Substance use prevention refers to activities, practices, and strategies that promote healthy behavior, either by increasing protective factors or by limiting risk factors. 

Substance use treatment 
Substance use treatment refers to the provision of assistance to individuals with existing substance use disorders. 

Substance misuse 
The use of a legal substance, such as prescription or over-the-counter drugs, in a way that is either not legal or not intended, and for which the consequences can be hazardous to health and safety. 

Substance use
The consumption of legal or illegal, or both, psychoactive substances.

Source: SAMHSA

Substance use disorder 
Occurs when the recurrent use of alcohol and/or drugs causes clinically and functionally significant impairment, such as health problems, disability, and failure to meet major responsibilities at work, school, or home. According to the DSM-5, a diagnosis of substance use disorder is based on evidence of impaired control, social impairment, risky use, and pharmacological criteria.

Source: SAMHSA

Sustainability 
The long-term survival and continued effectiveness of an intervention. 

Symptomatology 
The combined symptoms or signs of a disorder or disease. 

Systematic review 
An evaluation of all research studies that address a particular issue. The researchers use an organized method of locating, assembling, and evaluating a body of literature on a particular topic using a set of specific criteria. A systematic review typically includes a description of the findings of the collection of research studies. The systematic review may also include a quantitative pooling of data, called a meta-analysis.

Source: AHRQ

back to top 

Threat to internal validity 
When a source of systematic error has the potential to minimize the accuracy of causal claims, or the internal validity of a study. Other threats to internal validity is a criterion on the NREPP outcome rating instrument that assesses threats that are not otherwise captured by the other rating criteria. 

back to top 

Unadjusted effect size 
NREPP uses this term to describe effect sizes that are not adjusted for baseline differences. Unadjusted effect sizes are, on balance, less reliable estimates of a treatment effect than adjusted effect sizes. 

Universal prevention strategies 
Prevention strategies that address entire population (such as national, local community, school, or neighborhood), with messages and programs to prevent or delay the use/abuse of alcohol, tobacco, and other drugs. This is one of the three categories (universal, selective, indicated) developed by the Institute of Medicine to classify preventive interventions. 

Sources: Institute of Medicine

back to top 

Variable 
Any characteristic that can be measured in different individuals. A variable is any factor that can affect the outcome of an experiment or study (an independent variable) or it can be an outcome that is predicted in an experiment or study (a dependent variable).

Source: AHRQ

back to top 

Wellness 
Concerns maintaining an overall quality of life and the pursuit of optimal emotional, mental, and physical health.

Source: SAMHSA

back to top 

Last Updated: 08/21/2017