Abstinence‐only programs for HIV infection prevention in high‐income countries

Criteria for considering studies for this review

Search methods for identification of studies

We refined our search strategy with recommendations from the Cochrane HIV/AIDS Review Group. No language restrictions were imposed and translations were sought where necessary. No restrictions on journal of publication were imposed, and no country names or other geographical terms were used in the search. Most databases were searched from 1980 onward. Databases were initially searched in May 2005, with all searches updated in February 2007. Full search strategies for each database are included in Table 1.

Electronic databases 
 We searched the following electronic databases, ending February 15, 2007: 
 1. ADOLEC (Inception‐2007) 
 2. AIDSLINE (1980‐2007) 
 3. AMED (1985‐2007) 
 4. ASSIA (1987‐2007) 
 5. BiblioMap (1887‐2007) 
 6. BIOSIS (1969‐2007) 
 7. BNI (1985‐2007) 
 8. Catalog of US Government Publications (1976‐2007) 
 9. CENTRAL (Cochrane Central Register of Controlled Trials) (1980‐2007) 
 10. CHID (1985‐2005; went offline in 2005) 
 11. CINAHL (1982‐2007) 
 12. DARE (1991‐2007) 
 13. Dissertation Abstracts International (1997‐2007) 
 14. EMBASE (1974‐2007) 
 15. ERIC (1991‐2007) 
 16. EurasiaHealth Knowledge Multilingual Library (Inception‐2007) 
 17. Global Health Abstracts (1973‐2007) 
 18. HealthPromis (1997‐2005; went offline in 2005) 
 19. HMIC (1983‐2007) 
 20. PAIS (1972‐2007) 
 21. Political Science Abstracts (1975‐2007) 
 22. PsycINFO (1887‐2007) 
 23. PubMed (1980‐2007) 
 24. RCN (1985‐1996; updating ended in 1996) 
 25. SCISEARCH (Web of Knowledge) (1974‐2007) 
 26. SERFILE (Inception ‐ 2005; inaccessible in 2007) 
 27. SIGLE (1980‐2005; went offline in 2005) 
 28. Social Services Abstracts (1979‐2007) 
 29. Sociological Abstracts (1963‐2007) 
 30. TRoPHI (Inception‐2007)

Other relevant libraries of international agencies, especially those concerned with the prevention of HIV/AIDS (UNAIDS, USAID, WHO, UNFPA, World Bank, and Centers for Disease Control and Prevention) were searched. We made additional efforts to identify and acquire unpublished literature.

Handsearching 
 Handsearching of various conference proceedings from 2000 onwards was conducted to identify unpublished reports. These included proceedings from the International AIDS Conference, the Conferences on Retroviruses and Opportunistic Infections, the US National HIV Prevention Conferences, the Abstinence Education Evaluation Conference, and the International Society of STD Research (ISSTDR).

Personal communication 
 We contacted leading experts in the field of abstinence‐based programs to solicit potentially relevant unpublished papers, ongoing research, and suggestions for other contacts.

Cross‐references 
 The reference lists of related reviews and primary studies were examined for additional citations.

Search terms 
 We searched with combinations of the following terms, truncating where possible. MeSH terms (e.g., "Sexual Abstinence") were also identified and included. We did not include country names or program names, as they may not be specified in searchable headings.

Intervention terms: abstinence, abstain, chastity, chaste, virgin, celibacy, celibate, sex education, marriage, delay, postpone.

Study terms: randomized controlled trial, controlled clinical trial, random allocation, double‐blind, single‐blind, clinical trial, mask, comparative study, control, pre‐post controlled designs, comparison group, cohort study, comparative study, evaluation study, feasibility study, follow up studies.

HIV terms: HIV Infections, HIV, HIV‐1, HIV‐2, human immunodeficiency virus, human immunedeficiency virus, human immuno‐deficiency virus, human immune‐deficiency virus, AIDS, acquired immunodeficiency syndrome, acquired immunedeficiency syndrome, acquired immuno‐deficiency syndrome, acquired immune‐deficiency syndrome, sexually transmitted diseases.

PUBMED SAMPLE SEARCH STRATEGY 
 #1 Search HIV Infections[MeSH] OR HIV[MeSH] OR hiv[tw] OR hiv‐1*[tw] OR hiv‐2*[tw] OR hiv1[tw] OR hiv2[tw] OR hiv infect*[tw] OR human immunodeficiency virus[tw] OR human immunedeficiency virus[tw] OR human immuno‐deficiency virus[tw] OR human immune‐deficiency virus[tw] OR ((human immun*) AND (deficiency virus[tw])) OR acquired immunodeficiency syndrome[tw] OR acquired immunedeficiency syndrome[tw] OR acquired immuno‐deficiency syndrome[tw] OR acquired immune‐deficiency syndrome[tw] OR ((acquired immun*) AND (deficiency syndrome[tw])) OR "Sexually Transmitted Diseases, Viral"[MeSH:NoExp] Limits: Publication Date from 1980 to 2007 
 #2 Search SEXUAL ABSTINENCE [MESH] Limits: Publication Date from 1980 to 2007 
 #3 Search ABSTINENCE OR ABSTAIN* OR CHASTITY OR CHASTE OR VIRGIN* OR CELIBA* OR (SEX* AND EDUCAT*) OR (SEX* AND (MARRIAGE* OR MARRIED)) Limits: Publication Date from 1980 to 2007 
 #4 Search #2 OR #3 Limits: Publication Date from 1980 to 2007 
 #5 Search randomized controlled trial [pt] OR controlled clinical trial [pt] OR randomized controlled trials [mh] OR random allocation [mh] OR double‐blind method [mh] OR single‐blind method [mh] OR clinical trial [pt] OR clinical trials [mh] OR ("clinical trial" [tw]) OR ((singl* [tw] OR doubl* [tw] OR trebl* [tw] OR tripl* [tw]) AND (mask* [tw] OR blind* [tw])) OR ( placebos [mh] OR placebo* [tw] OR random* [tw] OR research design [mh:noexp] OR comparative study [mh] OR evaluation studies [mh] OR follow‐up studies [mh] OR prospective studies [mh] OR control* [tw] OR prospectiv* [tw] OR volunteer* [tw]) NOT (animals [mh] NOT human [mh]) Limits: Publication Date from 1980 to 2007 
 #6 Search #1 AND #4 AND #5 Limits: Publication Date from 1980 to 2007

Data collection and analysis

Selection of studies 
 All reviewers independently examined citations reviewed by the search strategy. KU obtained full articles of potentially relevant papers. All reviewers independently assessed their eligibility using an eligibility form based on pre‐specified inclusion criteria. Reasons for excluding potentially relevant trials are specified in the "Characteristics of Excluded Studies." We contacted the authors for clarification when necessary. Disagreements were resolved by discussion and, if necessary, referral to the Cochrane HIV/AIDS Review Group.

Data extraction and management 
 All reviewers independently extracted data on study design, participants, interventions, outcomes and methodological quality, using pre‐designed data collection forms. Any disagreements were resolved by discussion.

The extracted data included the following measures: citation, study design, methodological criteria, inclusion and exclusion criteria, comparison group intervention, participant characteristics, trial setting, theoretical basis for intervention, elements of intervention, all relevant outcome measures, and results. We expected the programs and trial settings to vary widely (e.g., schools, clinics, prisons, community centers, and churches), and examined this carefully when considering the heterogeneity of our data set. We also examined any process or cost‐effectiveness data reported, including training, monitoring, acceptability, costs, and sustainability.

Where reports were uncertain or included summary measures, authors were contacted for clarification. We noted any data that were consistently underreported, and highlighted this deficit along with future research needs in the discussion.

Assessment of methodological quality of included studies 
 Depending on study design, we evaluated the following methodological components, since there is evidence that these may be associated with biased estimates of effects: 
 1. similarity of the intervention and control groups (whether groups were treated equally except for the intervention); 
 2. method of generation of the randomization sequence; 
 3. generation of allocation concealment; 
 4. blinding (participants, intervention staff, and outcome assessors); 
 5. intention‐to‐treat analysis; 
 6. loss to follow‐up. (In our results and discussion sections, we highlight attrition as a particular limitation of any studies with a total dropout exceeding one‐third of baseline enrollment.)

Measures of treatment effect 
 To investigate the feasibility of a meta‐analysis, all eligible studies were summarized in RevMan to the fullest extent possible. All reviewers assisted in abstracting the data, with KU entering all data into RevMan and DO and PM rechecking all entries. Disagreements were resolved by discussion. A narrative synthesis was provided for all results, but we determined that a statistical meta‐analysis was not appropriate for this review due to differences in program and evaluation design, underreporting of key data, and inability to retrieve unpublished or missing data from authors. Had the studies been comparable and more completely reported, our meta‐analysis would have measured the weighted average effect size for each outcome measure, weighted mean differences (for continuous outcomes), odds ratios of effects (for dichotomous outcomes), and 95% confidence intervals. Odds ratios less than one express a protective intervention effect.

Unit of analysis issues 
 Studies with multiple treatment groups 
 A meta‐analysis was not conducted for this version of the review. However, had meta‐analysis been possible, we would have dealt with multiple treatment groups in the following way: when studies had more than two treatment groups, we planned to accept only two groups for the meta‐analysis. In keeping with procedures used by the US HIV/AIDS Prevention Research Synthesis project (Johnson 2002), we planned to take the arm with the greatest or most intense exposure (as identified by the trialist), compared to the arm with the least or least intense exposure, thereby giving each study the most favorable chance to yield significant program effects. When different trial arms emphasized sexual abstinence to a differing extent, we would have accepted the arm that emphasized abstinence the most, compared to the arm that emphasized abstinence the least.

Crossover trials 
 We did not anticipate dealing with these trials in the review, and no crossover trials were recovered by the search. If we had identified any studies using this design, we would have analyzed only the follow‐up data from the period before the crossover.

Cluster‐randomized trials 
 When studies use cluster randomization, analyzing the data at the level of the individual participant has two implications: first, p‐values may be artificially small, suggesting significant effects where none occurred (Higgins 2005). Second, the confidence intervals for these results may appear artificially narrow, which would cause the studies to receive disproportionately more weight in the context of a meta‐analysis (Donner 2002, Johnson 2002). The second concern was unimportant for this version of the review because no meta‐analysis was conducted. However, the first remained relevant because wherever possible, we reported the individual study results as calculated in RevMan.

Before entering the results of cluster‐randomized studies into RevMan, we transformed these data according to the procedure in the Cochrane Handbook (Higgins 2005, supported in Adams 2004), dividing the number of events and number of participants by the design effect [1 + (1 ‐ m) * r]. We used the details provided by each study (total n and number of clusters) to calculate the average cluster size (m), and we decided upon consultation with the review group and a number of statistical experts to use the intra‐class correlation coefficients (r) recommended in Johnson 2002. These were ICC=0.015 for school‐based studies and ICC=0.005 for community‐based studies (regardless of cluster size); these values were calculated respectively from school‐based studies of youth smoking behavior and community‐based studies of HIV prevention for men who have sex with men.

We acknowledge that we are uncertain about the applicability of these values to the context of this review. However, insufficient data were available for us to estimate a reliable ICC from the studies in this review. Since no meta‐analysis was conducted, we report the results of cluster‐randomized trials analyzed both before and after controlling for clustering. Insufficient data were available for us to calculate average cluster size for Anderson 1999, so these results are reported on an individual level.

Dealing with missing data 
 Missing data arose from two sources: participant attrition and missing statistics.

Attrition 
 To deal with participant dropout, we accepted studies that used any type of analysis, including complete case analyses (i.e., participants were analyzed according to original assignments, and no data were imputed for drop‐outs). If a meta‐analysis had been possible, we would have conducted sensitivity analyses to investigate attrition as a source of heterogeneity and possible bias.

Missing statistics 
 Where statistics were missing (e.g., numbers of participants per group, attrition rates, means and standard deviations, or percentages), we contacted primary study authors on two separate occasions to supply the information. Where the information was unavailable due to data loss or non‐response, we reported the results as stated in the trial report.

Assessment of heterogeneity 
 If a meta‐analysis had been possible for this version of the review, we would have assessed heterogeneity using the chi square test of heterogeneity, visual inspection of the data, and the I2 statistic (Higgins 2002). The I2 statistic determines the percentage of variability that is due to heterogeneity rather than sampling error, where a value greater than 50% suggests moderate heterogeneity. If any of these methods indicated heterogeneity, we would have investigated possible explanations, including clinical and methodological characteristics. Even when tests for heterogeneity were non‐significant, we planned to conduct sub‐group analyses to explore other potential moderators.

Assessment of reporting biases 
 If a meta‐analysis had been possible for this version of the review, we would have used funnel plots (plotting effect size against standard error) to detect potential bias. Additional analyses may have included the planned Egger regression approach with a weight‐function model. Asymmetry can be due to publication bias, but it can also be due to clinical and methodological heterogeneity. In the event that a relationship had been found, these sources of heterogeneity would also have been examined as possible explanations (Egger 1997).

Data synthesis 
 If a meta‐analysis had been possible for this version of the review, we would have considered conducting analyses according to both fixed‐effects and random‐effects models. The random‐effects model would have been used where there was indication of heterogeneity and the source of such heterogeneity could not be explained. The random‐effects model would also have been used for analyses incorporating small numbers of studies, for which tests of heterogeneity may be underpowered. If there were no sources of heterogeneity beyond differences in the observed covariates, we would have conducted both fixed‐effects and random‐effects analyses and investigated differences between the two procedures.

The narrative synthesis was conducted according to the following methods. First, we entered each individual study in RevMan. Next, we prepared the supplemental Charts of Effects appended to this review (see Figures 1‐10, Figure 1). These charts are organized by outcome, and each chart shows the effects of individual studies according to RevMan analyses (where possible) or as reported in the primary study. The charts also integrate key methodological features of each study (e.g., control group, attrition), along with each study's definition of the outcome of interest (e.g., condom use was variously defined as condom use at last sex among sexually experienced participants, or condom use in the last month among sexually active participants). The charts were then explained in text format for the description of studies and results.

Sub‐group analyses 
 If we had conducted a meta‐analysis, we would have stratified our analysis by intervention setting (e.g., school, clinic) and by participants' age, country, ethnicity, and socioeconomic status. Sub‐group analyses would only have been conducted if data had allowed, with the knowledge that using a number of sub‐groups can lead to statistically misleading conclusions.

Sensitivity analysis 
 If we had conducted a meta‐analysis, we would have performed separate analyses for studies using complete case, per‐protocol, and intention‐to‐treat methods, with a sensitivity analysis to investigate disparities between the three groups.

Description of studies

Results of the search 
 See Figure 2.

A total of 20070 citations were assessed for inclusion (of these, 19,892 were recovered from electronic database searches ending February 2007, 99 were from handsearching, 17 were from personal contacts, and 62 were from cross‐referencing). After the screening process, 330 papers were deemed potentially relevant evaluation studies. Full‐text copies (or sufficient information) for 326 of these citations were obtained.

Of the four papers we could not obtain, two were conference abstracts of unclear relevance whose authors could not be located. One was an in‐press paper of unclear relevance that could not be obtained from the authors. One was no longer available in print and the authors could not be located; this appeared to be a non‐random evaluation of an abstinence‐only program.

Of the 326 papers for which sufficient study information was obtained, 318 were excluded for the following reasons: 16 were not program evaluations; 69 duplicated other papers; 130 did not evaluate an abstinence‐only program; 91 evaluated an abstinence‐only program but did not meet the criteria for study design; two evaluated abstinence‐only programs using randomized controlled designs, but the programs did not list HIV prevention as a program goal (Jorgensen 1993, Mitchell‐DiCenso 97); five evaluated an abstinence‐only HIV prevention program and possibly used a randomized controlled design, but did not present outcomes of interest (Lage 2005, O'Donnell 2005, Pallone 2007, Roberts‐Gray 2007, Tobin 2005), and five were ongoing trials (Borawski 2007, Jemmott 2006, Miller‐Heyl 2007, Nagel 2007, Markham 2007).

After these studies were excluded, analyses were limited to eight separate papers reporting 13 trials (Anderson 1999, Blake 2001, Clark 2005, Goldfarb 1999, Hernandez 1990, Kirby 1997a, Kirby 1997b, Kirby 1997c, Miller 1993, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). One paper described three evaluations (Kirby 1997a, Kirby 1997b, Kirby 1997c), and two supplementary papers duplicated these reports (Kirby 1995, Cagampang 1997). Another paper described four evaluations (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), and was also supplemented by two reports (Devaney 2002, Maynard 2005). One paper was included on the basis of unpublished information about the method of allocation (Goldfarb 1999).

Included studies 
 Detailed information about individual studies may be found in the Table of Included Studies.

Design 
 All of the abstinence‐only evaluations were randomized controlled trials; no quasi‐randomized controlled trials evaluating abstinence‐only programs were retrieved by the search. Of the 13 evaluations included, three were part of the same report (Kirby 1997a, Kirby 1997b, Kirby 1997c), which used separate evaluation designs to assess Postponing Sexual Involvement (PSI) program. These differed by units of randomization (which were classroom, school, and individual, respectively). Four evaluations were included in another report (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d); these differed by site and experimental program.

Six of the evaluations involved cluster randomization. The unit of randomization was the school for two evaluations (Goldfarb 1999, Kirby 1997b), the classroom for three evaluations (Blake 2001, Clark 2005, Kirby 1997a), the individual for six evaluations (Hernandez 1990, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), the family for one evaluation (Miller 1993 ‐ it appeared that only one "target teen" per family was included), and "naturally occurring groups" in the final evaluation (Anderson 1999; this designation included the school unit).

Control groups varied. One study compared an abstinence‐only program to a non‐intervention control (Anderson 1999); the control group also received the intervention after the 12‐month follow‐up assessment.

Blake 2001 compared an enhanced version of a program against a non‐enhanced version of the same program. The control group was exposed to the Managing the Pressures before Marriage (MPM) curriculum only, while the other received the curriculum plus extra homework assignments to facilitate parental involvement.

Hernandez 1990 involved four arms: abstinence‐only (i.e., "Abstinence," which presented abstinence as the best and only strategy for avoiding HIV), abstinence‐plus (i.e., "Protection," which presented abstinence as the safest alternative and safer sex as another alternative, focusing on condoms), safer‐sex (i.e., "Decision," which focused on safer sex and personal decision‐making), and a non‐intervention control. University students were randomized to each of the four arms. All programs took place in the same setting and used the same one‐session delivery format.

Miller 1993 used a three‐arm design to assess the effect of a video‐based intervention intended to enhance parent‐child communication about sex and abstinence. Families were randomized to receive one of three programs: videos plus mailed newsletters suggesting further family discussions, videos alone, and no intervention.

Nine effectiveness studies compared an abstinence‐only program against usual care (Clark 2005, Goldfarb 1999, Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), with "usual care" defined as the HIV‐prevention intervention normally delivered by the school. Since few details were provided to describe what "usual care" consisted of, it was impossible to clarify whether these control groups received safer‐sex programs, abstinence‐plus programs, other abstinence‐only interventions, or any combination of the above. The available details were as follows:

An unpublished report for Kirby 1997a, Kirby 1997b, and Kirby 1997c states that "the vast majority of students in the control groups did not receive another sexuality education curriculum instead of PSI. Instead, they received instruction in some other topic area ... many students in both the PSI groups and the control groups did receive some type of instruction on sexuality either before or during the time period of this evaluation." Although the published report of Kirby 1997c states that the study used a no treatment control, the unpublished evaluation states that control group participants received the "standard curriculum" delivered by the community agencies.

The report for Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, and Trenholm 2007d provides some detail about "usual care" in each trial. For Trenholm 2007a, controls received a nine‐week physical education class that did not discuss abstinence or HIV, as well as a health class emphasizing abstinence but providing no details about HIV. Controls in Trenholm 2007b received a week‐long unit on human growth and development, as well as a curriculum covering HIV and abstinence. Controls in Trenholm 2007c received "units on abstinence and contraceptive use" beginning in fifth grade. Finally, controls in Trenholm 2007d received "heath, family‐life, and sex education curricula," but it was unclear what these classes entailed.

Sample sizes 
 At baseline, the studies together enrolled approximately 15940 participants. Sample sizes for the abstinence‐only evaluations at the time of baseline assessment ranged from 248 (Clark 2005) to 5244 (Kirby 1997b); baseline enrolment was not specifically reported and could not be computed for Goldfarb 1999. The median baseline enrolment was approximately 551. Five trials reported using a power calculation (Clark 2005, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d).

After attrition at each study's longest follow‐up, the analyses reported in this review encompassed approximately 11879 participants. Again, this value is somewhat uncertain due to unclear reporting in several of the primary studies, and it is an overall estimate (i.e., it does not account for non‐response on any specific outcome measure).

Setting 
 All 13 trials enrolled adolescents in the United States. Program sites included California (Kirby 1997a, Kirby 1997b, Kirby 1997c, Anderson 1999), upstate New York (Blake 2001), "a southeastern metropolitan area" (Clark 2005), New Jersey (Goldfarb 1999), North Carolina (Hernandez 1990), Utah (Miller 1993), Virginia (Trenholm 2007a), Florida (Trenholm 2007b), Wisconsin (Trenholm 2007c), and Mississippi (Trenholm 2007d).

The interventions were delivered in elementary and/or middle schools (Blake 2001, Clark 2005, Goldfarb 1999, Kirby 1997a, Kirby 1997b, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d); a university (Hernandez 1990); community centers (including churches), youth recreation programs, and social service agencies (Kirby 1997c); a mixture of schools and community centers (Anderson 1999); and family homes (Miller 1993).

Participants 
 All participants were adolescents or young adults in the US. All but one study (Hernandez 1990) targeted youth between ages 9‐13 years; the remaining study targeted young people aged 18 to 21 years. The mean participant ages in the 13 studies ranged from 10.6 years (Anderson 1999) to 19.25 years (Hernandez 1990), the median of these average ages was 12.8 years. Participant groups in all but one study included both males and females; one trial included females only (Trenholm 2007b).

Participants were primarily African‐American in four studies (Clark 2005, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), largely of minority ethnicity in three studies (Anderson 1999, Kirby 1997a, Kirby 1997c), "high minority" in another study (Goldfarb 1999), equally representative of white and minority ethnicity in one study (Kirby 1997b), and primarily white in four studies (Blake 2001, Hernandez 1990, Miller 1993, Trenholm 2007a). One study took place in a community with a high proportion of Mormon families, who were described as having a "highly abstinence‐valued culture" (Miller 1993).

Four studies reported little or no information about the socio‐economic status of participants (Hernandez 1990, Kirby 1997a, Kirby 1997b, Kirby 1997c), although the Kirby studies took place in areas with elevated teen pregnancy rates, potentially indicating lower socioeconomic status. The communities of two studies were described as "middle‐class" (Blake 2001, Trenholm 2007a), and five were of lower socioeconomic status (Clark 2005, Goldfarb 1999, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). One evaluation mentioned that participants' mothers' education varied between high school graduation and some college coursework (Anderson 1999). In the family‐based study, 38% of mothers and 56% of fathers were college‐educated (Miller 1993).

Parental consent procedures were relevant ethical concerns for this review, since 12 of the studies involved legal minors as participants. Eight studies used active parental consent procedures (Anderson 1999, Blake 2001, Clark 2005, Miller 1993, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d); three studies required parental notification, with permission forms used by 54% of the contractors hired to deliver the intervention (Kirby 1997a, Kirby 1997b, Kirby 1997c); and one study used passive parental consent procedures (Goldfarb 1999). Rates of parental opposition in every study were low. Incentives for participation were offered in one study that granted its university‐level participants course credits (Hernandez 1990), one study that gave participants prizes for homework completion (Blake 2001), one study that gave participants $5 for each survey completed (Clark 2005), and three studies for which contractors often had "reunions" or pizza parties to facilitate the administration of post‐test surveys (Kirby 1997a, Kirby 1997b, Kirby 1997c).

Interventions 
 The 13 abstinence‐only programs evaluated in this review had many similarities. All of the programs stressed sexual abstinence as the best and only means of HIV prevention, and no program promoted condom or contraceptive use as a viable alternative to abstinence. Eleven of the programs were at least partially school‐based (Kirby 1997c and Miller 1993 were the exceptions), and most of the programs emphasized communication and decision‐making. Five of the interventions were based on the same program model: Kirby 1997a, Kirby 1997b, Kirby 1997c, and Trenholm 2007d assessed the effects of Postponing Sexual Involvement (PSI), which was modified slightly to form the curriculum Managing the Pressures before Marriage (MPM) as evaluated in Blake 2001. All but one of the programs were designed for adolescents and delivered to middle‐school‐aged youth. Trenholm 2007d also involved components of the Sex Can Wait program, which was evaluated in the study by Goldfarb, et al. (Goldfarb 1999).

Although the interventions did appear similar enough to permit a systematic review, the information reported about intervention design and implementation was variable and generally limited. Although most reports specified basic information about intended delivery format, subject material, activities, and exposure, very few collected or reported data on adherence, implementation fidelity, actual exposure, context, or modifications to the original design. Additionally, it was often difficult or impossible to quantify the extent to which the subject matter of a program focused on abstinence (and excluded information regarding contraception and condom use), which made the recognition of abstinence‐only programs somewhat difficult. There is a need for more detailed information on intervention design and actual implementation in primary trial reports.

Importantly, 12 of the interventions emphasized pregnancy prevention to an equal or greater extent than HIV prevention. One experimental program was limited to HIV prevention (Hernandez 1990). It is unclear how this evidence generalizes to programs with an exclusive HIV prevention focus.

All but one of the evaluations (Hernandez 1990) involved multiple sessions, and every program aimed to stop, decrease, or postpone sexual activity within the participant population. Six programs appeared to be delivered exclusively in a large‐group classroom format (Blake 2001, Goldfarb 1999, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). These programs may also have utilized small‐group activities, but this was not stated explicitly. One of these interventions (Blake 2001) supplemented classroom activities with parent‐child homework for completion in the family home. Trial reports for five programs mentioned both classroom and small‐group activities (Anderson 1999, Clark 2005, Kirby 1997a, Kirby 1997b, Kirby 1997c). Two programs were media‐based: one included videos and pamphlets delivered in a university classroom (Hernandez 1990), while the other included videos viewed in the family home (Miller 1993).

Trial reports indicated that at least eleven of the interventions aimed to improve communication between parents and adolescents (Anderson 1999, Blake 2001, Goldfarb 1999, Miller 1993, Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). Strategies for parent involvement included homework assignments (Anderson 1999, Blake 2001, Goldfarb 1999), parent meetings or "parent nights" (Anderson 1999, Kirby 1997a, Kirby 1997b, Kirby 1997c), home visits by social workers (Trenholm 2007b), a parent curriculum (Trenholm 2007a), and videos for parental viewing (Miller 1993). However, reports acknowledged that parental uptake of these activities was often low. Parent‐child activities were not mentioned in two studies (Clark 2005, Hernandez 1990).

Program exposure (i.e. dosage) varied from one session (Hernandez 1990) to 720 sessions (Trenholm 2007c), with a median exposure of 8 sessions across trials.

Staff members for most interventions (Anderson 1999, Clark 2005, Goldfarb 1999, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d) were adults, including teachers and public health personnel. One design randomized classrooms to peer‐led PSI, adult‐led PSI, or a control group (Kirby 1997a), while another intervention was delivered entirely by peer leaders (Blake 2001). The two media‐based interventions had minimal staff involvement by adults (Hernandez 1990, Miller 1993).

Four of the evaluation reports stated theoretical bases for the interventions; these included cognitive behavioral theory (Anderson 1999), social learning theory (Anderson 1999, Blake 2001, Goldfarb 1999), social cognitive theory (Blake 2001), and the theory of possible selves (Clark 2005). Four additional evaluations used a program based on social inoculation theory (Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007d, original PSI program design reported in Howard 1990 and Howard 1992).

Specific intervention details are as follows:

Anderson 1999 evaluated Reaching Adolescents and Parents (RAP), which targeted students in grades 5‐7 and their parents. The eight‐session curriculum focused on puberty, self‐esteem, communication, relationships, and decision‐making. Six sessions involved the youth alone; one involved joint parent‐child activities focusing on communication (e.g. discussions, video viewing); and the final session was a discussion forum for parents.

Blake 2001 assessed the effects of Managing the Pressures before Marriage (MPM), a five‐session curriculum for middle school students modeled on PSI, but with an additional emphasis on delaying sex until marriage. The five sessions were delivered by slightly older peer leaders (i.e., high‐school aged) in a classroom setting. The independent variable in this evaluation was the addition of five parent‐child homework assignments to the curriculum; these assignments aimed to improve parent‐child communication and focused on concepts such as personal values, HIV‐related myths, reasons to postpone sex, sex in the media, self‐esteem, and peer pressure.

Clark 2005 evaluated the Adult Identity Monitoring program, a 10‐session program delivered by a male‐female pair of African‐American adult facilitators. The program targeted African‐American adolescents, and it attempted to change behavior by helping participants to envision their future selves and understand how risk‐taking might affect their goals.

Goldfarb 1999 evaluated Project C.A.R.E. (Creating Awareness through Relationship Education), a classroom initiative with the primary focus of delaying sexual activity. The curriculum focused on communication, factual information about sexual behavior, the consequences of sex, goal‐setting, social support, and decision‐making. Participants in this study received slightly different programs depending on their grade levels: sixth grade students received 23 sessions based on the "upper elementary" Sex Can Wait curriculum, seventh grade students received 24 sessions from the "middle school" Sex Can Wait curriculum, and eighth grade students received 24 of the 45 sessions in the Abstinence Pick and Choose Activities curriculum. All participants were grouped together for analysis, which made it impossible to isolate the effects of each program. It was unclear how many students were in each grade level.

Hernandez 1990, the only intervention to take place in a university setting, compared three active intervention groups to a non‐intervention control. In a large group setting, participants in each arm received a separate seven‐page health education curriculum with information about the biology, symptoms, and transmission of HIV. Participants then viewed a 20‐minute AIDS prevention film that included commentary and interviews; each intervention arm watched a separate video. Lastly, participants in each arm received a separate brochure highlighting transmission modes, symptoms, and prevention of HIV. The content of the three programs varied, with the intent of following three "prototypes of programs advocated by boards of education, health organizations, or health educators." One program focused on abstinence only ("Abstinence"), one prioritized abstinence but also promoted safer sex behaviors ("Protection"), and one focused on safer sex ("Decision"). This was the only study to explicitly compare an abstinence‐only program against an abstinence‐plus or a safer‐sex design.

Kirby 1997a, Kirby 1997b, and Kirby 1997c assessed the effects of Postponing Sexual Involvement (PSI), a five‐session curriculum for middle school students focusing on the risks of early sex, reasons to delay sex, resisting social pressures, limit‐setting, and communication. This program was implemented as part of the California initiative ENABL (Education Now and Babies Later). Kirby 1997a systematically varied the staff delivering the intervention between adult health educators and peer leaders, comparing each to a usual care control group. Kirby 1997b and Kirby 1997c each used adult staff to deliver the intervention. The PSI curriculum has been evaluated elsewhere as an abstinence‐plus program with the addition of a curriculum including contraception instruction (Aarons 2000, Howard 1990, Howard 1992), but it functioned as an abstinence‐only intervention in this study (Kirby 2002).

The one intervention not delivered in a group setting was Facts and Feelings, a video‐based curriculum designed for weekly viewing by parents and adolescents in the home (Miller 1993). Six 15‐ to 20‐minute videos were distributed to parents for the duration of three months, with biweekly follow‐up calls by intervention staff to collect usage data. Videos focused on puberty, sexual values, facts about sexual anatomy and reproduction, the media's portrayal of sex, the consequences of sex, and refusal skills. The study compared two active treatment groups to non‐intervention controls; one treatment group received the videos along with supplementary newsletters designed to suggest further family discussions and discussions, while the other group received the videos only.

Trenholm 2007a compared a mandatory school‐based abstinence‐only program to usual care. The program, entitled My Choice, My Future! was a composite of three curricula: Reasonable Reasons to Wait (30 classroom sessions over the eighth‐grade year focusing on character development, relationships, and abstinence), The Art of Loving Well (eight classroom sessions over the ninth‐grade year focusing on healthy relationships), and WAIT Training (14 sessions over the tenth‐grade year focusing on relationships skills, risks, and a slide show about STDs and abstinence).

Trenholm 2007b assessed an elective program for 8th‐grade girls entitled "ReCapturing the Vision," comparing the program to usual care. The program met daily throughout one school year (180 sessions), and was a composite of two programs. "ReCapturing the Vision" focused on personal and career goals and included home visits by social workers, tutoring, community service, a retreat, a Teen Abstinence Rally, and a symposium with celebrity speakers. "Vessels of Honor" was added to this curriculum; this program focuses on "honorable behavior," communication, relationships, physical development, sexual abuse, and "strategies for choosing a mate and the benefits of a committed marital relationship." Once girls elected to join the program, attendance was mandatory.

Trenholm 2007c assessed an elective program entitled "A Life Options Model Curriculum for Youth," which met for 2.5 hours daily during the school year, for four consecutive years (approximately 720 program sessions). The program focused on self‐esteem, values, decision‐making, risks, communication, relationships, sexuality, development, sexually transmitted infections, and social skills. The program also included parent workshops and mentoring programs. Attendance was voluntary.

Trenholm 2007d compared the Teens in Control program to usual care. Teens in Control was a composite of Postponing Sexual Involvement (also evaluated in Kirby 1997a, Kirby 1997b, and Kirby 1997c) and Sex Can Wait (also evaluated in Goldfarb 1999). The program met for two hours weekly throughout one school year (approximately 36 program sessions), and focused on risks, pressures, puberty, self‐esteem, values, communication, career goals, and skills to facilitate abstinence.

Outcomes 
 All outcomes were assessed via written self‐report questionnaires, which were bilingual (English/Spanish) in four of the evaluations (Anderson 1999, Kirby 1997a, Kirby 1997b, Kirby 1997c). Questionnaires were generally completed in the intervention setting and administered to groups (or, in Miller 1993, to parents and children separately) by project staff or intervention facilitators. Questionnaires in four trials were sometimes administered by phone (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). One report mentioned modifying questionnaires in response to opposition from concerned parents by removing questions from the survey (Anderson 1999). Nine of the evaluations described piloting the surveys (Anderson 1999, Hernandez 1990, Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d); three used or slightly modified surveys developed by others, but did not explicitly describe piloting procedures (Clark 2005, Goldfarb 1999, Miller 1993), and one did not explicitly report how the survey was developed (Blake 2001). No evaluations confirmed the results of questionnaires with medical records or biological assessments, making all data vulnerable to recall bias and the limitations of self‐report.

The studies used a variety of techniques to improve the validity of self‐reported data. These included having bilingual (English/Spanish) staff assist students during the interviews (Anderson 1999), reading questionnaires aloud (Anderson 1999, Hernandez 1990), allowing students to fill in the questionnaires anonymously (Anderson 1999, Goldfarb 1999, Hernandez 1990), emphasizing the confidential and voluntary nature of the exam before questionnaire administration (Blake 2001), seating participants far apart during survey administration (Hernandez 1990, Kirby 1997a, Kirby 1997b, Kirby 1997c), asking participants to sign consent forms that highlight confidentiality (Hernandez 1990), monitoring the survey room to prevent talking or participant wandering (Hernandez 1990, Kirby 1997a, Kirby 1997b, Kirby 1997c), using ID numbers and cover sheets to protect participants' identities (Kirby 1997a, Kirby 1997b, Kirby 1997c), not placing any personal identifying information on any survey form or data file (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), giving students papers to cover their answers (Kirby 1997a, Kirby 1997b, Kirby 1997c), and placing surveys in unmarked envelopes (Miller 1993).

Data were generally reported as statistical tests and/or group percentages (with or without the numbers of participants per trial arm); no study reported effect sizes. One study provided statements of significance without group means or other specific data (Miller 1993).

Timing of follow‐up assessments ranged from six weeks (Hernandez 1990) to approximately 65 months (Trenholm 2007b) after baseline. For the purposes of this review, follow‐up data were blocked into short‐term (0‐5 months after baseline), medium‐term (6 through 11 months), and long‐term (12 months or more). Ten of the 13 studies reported long‐term follow‐up data (Anderson 1999, Clark 2005, Kirby 1997a, Kirby 1997b, Kirby 1997c, Miller 1993, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d); the other three were limited to short‐term follow‐ups (Blake 2001, Goldfarb 1999, Hernandez 1990). The median final follow‐up assessment occurred 17 months after baseline.

The recall periods for self‐reported outcomes varied. Outcomes such as STI, pregnancy, number of partners, and sexual initiation tended to be reported for participants' lifetimes, while recall periods for outcomes such as recent vaginal sex or condom use included 1 month (Goldfarb 1999), 1.5 months (Hernandez 1990), 3 months (Blake 2001) 12 months (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), time since baseline (Clark 2005, Kirby 1997a, Kirby 1997b, Kirby 1997c), and ever (Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d).

The biological outcomes of interest for this review were underutilized. No study evaluated HIV incidence, and seven evaluations, conducted by two groups of trialists, assessed the incidence of STI. Eight studies assessed pregnancy incidence.

The behavioral outcomes of most interest (i.e., unprotected oral, anal, and vaginal sex) were also underutilized. Five studies assessed the incidence of unprotected vaginal intercourse among all participants. No evaluation separated sexual behavior outcomes into vaginal, oral, and anal behaviors; we entered the non‐specific outcomes "intercourse" and "sex" as vaginal sex. Seven studies reported incidence of vaginal sex. Four studies evaluated the frequency of recent vaginal sex, and eight evaluated number of partners . Condom use and sexual initiation were reported in eight and ten trials respectively; these were the most commonly reported behavioral outcomes.

One study (Miller 1993) created a sexual behavior index for outcome observation, asking participants to report lifetime behaviors ranging from "holding hands" to "the sexual act by which pregnancy can occur." Data on vaginal intercourse could not be disaggregated from this index, making the result less relevant for this review.

Excluded studies 
 Excluded studies consist of all evaluations produced by the search that evaluated abstinence‐only programs using non‐randomized controlled, quantitative designs among any population group in high‐income countries. Cross‐sectional studies and comparisons of program participants against national survey samples were included. These abstinence‐only programs could focus on preventing pregnancy or preventing HIV, and the studies could report any outcomes. Details of the excluded studies, including results for behavioral and biological outcomes only, are provided in the Table of Excluded Studies. The results mentioned in this table are those reported by the trialists, with no further analysis by the reviewers.

Ongoing studies 
 Information regarding ongoing studies (Jemmott 2006, Markham 2007, Miller‐Heyl 2007, Nagel 2007, Borawski 2007) is provided in the Table of Ongoing Studies.

Risk of bias in included studies

Allocation 
 Four trials reported the method of randomization (via a random number generator, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). One study did not report in the published version that it was a randomized controlled trial, but this was discovered upon communication with the trialist (Goldfarb 1999). Overall, the potential for inadequate randomization and allocation concealment is a severe limitation of this data set.

Randomization was made particularly difficult by the need to contain the potential for contamination; this led Anderson 1999, Blake 2001, Clark 2005, Goldfarb 1999, Kirby 1997a, and Kirby 1997b to use cluster randomization, which heightened the potential for baseline differences. Seven trials reported at least one significant baseline difference (Blake 2001, Clark 2005, Kirby 1997b, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), and each of these studies statistically controlled for these differences in analyses. One study (Miller 1993) did not provide an explicit statement of group equivalence at baseline.

Contamination and threats to program differentiation were openly acknowledged in the three studies that evaluated the same program (PSI) in California (Kirby 1997a, Kirby 1997b, Kirby 1997c). This three‐study design may have allowed control‐group participants for one evaluation to receive the intervention in another setting (e.g., a control participant in a school‐based study could have received the intervention as an experimental participant in the community‐based study). Threats to program differentiation could also have occurred in Blake 2001, for which the same personnel delivered both the experimental and control interventions. This study monitored program leaders to assess program differentiation and the potential for performance bias. The potential for contamination was unclear in four studies (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d), as it was not stated who delivered the experimental and control programs.

Allocation concealment was not specifically addressed by any study, suggesting that every study may be prone to selection bias. We gave every study the "B" designation for quality of allocation concealment, indicating that procedures for allocation concealment were not reported.

Blinding 
 Given the time and human contact required for these interventions, the blinding of participants and intervention staff was generally not feasible. The impossibility of blinding intervention staff may have led to performance bias.

The blinding of assessors and data collectors was not entirely crucial for this group of studies, because every study utilized written self‐reported questionnaires for data collection. As mentioned, questionnaires were administered in group settings by trained data collectors; blinding of these data collectors was not reported in any study. Notably, Goldfarb 1999 used students' own teachers as monitors for data collection, which may have affected self‐report if students had confidentiality concerns.

Follow‐up and exclusions 
 Attrition was not known for one study, which did not report the number of participants randomized at baseline (Goldfarb 1999). Final attrition rates in the other nine studies ranged from 5% (Hernandez 1990) to 45% (Kirby 1997c), with a median final attrition rate of 20%. Four studies had a final attrition above 33%, which must be considered when judging both internal and external validity; these included the following: Anderson 1999 (38.0%), Clark 2005 (37%), Kirby 1997a (33.7%), and Kirby 1997c (45.0%).

Five studies reported attrition analyses (Anderson 1999, Clark 2005, Kirby 1997a, Kirby 1997b, Kirby 1997c). In Anderson 1999, participants who dropped out of the program were older by one year, more likely to be male, and less likely to report skipping school than the participants retained. Kirby 1997a, Kirby 1997b, and Kirby 1997c reported attrition analyses for the entire three‐study design; approximately 1% more participants were lost from the intervention than the control group. Using two‐way ANOVAs, the study found no significant interaction effects between treatment group and remaining in the study (i.e., survey completion at 17‐month follow‐up) in relation to age, gender, family background, risk factors such as alcohol use or low grades, mediating variables, or outcome variables. In Clark 2005, there was no difference in attrition by group; this report further specified that there were no differences by age, sexual initiation, sexual activity, birth control, or condom use.

Attrition analyses were not available for Blake 2001, but the study reported that non‐completion of homework assignments in the MPM‐enhanced group was significantly higher among males, students reporting recent sexual intercourse, students who did not receive "mostly A's in school," and black or Hispanic students (as opposed to non‐Hispanic whites). Goldfarb 1999 also did not provide attrition analyses; however, although groups were reportedly comparable on all variables at baseline, follow‐up comparisons on demographic variables showed several differences. Fewer control group participants were in the oldest age group or highest grade in school, and there were significant differences in living arrangements (the majority of control group participants lived in an apartment, while the majority of intervention participants lived in a house).

Hernandez 1990 and Miller 1993 did not report attrition analyses. The studies by Trenholm et al (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d) also did not report attrition analyses; attrition was slightly higher in the control group in three trials (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c), and slightly higher in the intervention group in one trial (Trenholm 2007d).

All 13 evaluations employed complete case analyses (Hollis 1999), although one study had 0% attrition at short‐term follow‐up, allowing analyses for that follow‐up to follow intention‐to‐treat procedures (Anderson 1999). Since several studies reported that dropouts may have been at higher risk than those who were assessed at follow‐up; the use of a complete case analysis may obscure intervention effects for the most at‐risk participants.

Selective reporting 
 The quality of outcome reporting varied across studies, as did the statistical methods of analysis and the sub‐groups for which data was reported. The variable reporting among the 13 trials suggests that no standard set of outcomes exists for the evaluation of sexual behavior for this population.

One trial collected data on number of sex partners and regularity of condom use, but did not report these data due to the small size of the subset of participants reporting sex (Blake 2001).

One trial originally collected data on frequency of sex, but the item was removed from the survey at follow‐up due to parental objections, resulting in de facto selective reporting of a pre‐defined outcome (Anderson 1999).

One study aggregated data into a sexual behavior index (Miller 1993), but did not disaggregate individual risk behaviors; this resulted in the data being less useful for the evaluation of HIV‐related risk.

A common source of potential confusion was the use of non‐specific self‐report outcome measures, such as "sexual intercourse," "sex," or "virginity." Given that these terms may be difficult to understand, or may have many different definitions among adolescents, adults, medical practitioners, program coordinators, and the general public (Haglund 2002, Horan 1998, Pitts 2001, Remez 2000, Sanders 1999, Sonfield 2001), these constructs may require more specific definitions. No study separated oral, anal, and vaginal behaviors.

Some results, particularly those for outcomes such as condom use or number of partners, were analyzed only among participants with certain levels of sexual experience. Eight studies made these distinctions (Goldfarb 1999, Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). Goldfarb 1999 limited condom use to participants reporting sex (either in the past month or the past year, although this was unclear), while Kirby 1997a, Kirby 1997b, and Kirby 1997c limited analyses for frequency of vaginal sex, number of partners and condom use to participants reporting sexual experience at baseline or follow‐up. Clark 2005, Kirby 1997a, Kirby 1997b, and Kirby 1997c appropriately assessed sexual initiation among participants who reported never having had sex at baseline. The trials by Trenholm, et al (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d) only assessed STI, pregnancy, and condom use among participants reporting sex ever. All other outcomes were assessed among all participants.

The data were reported in a variety of formats, many of which were unsuitable for meta‐analysis in RevMan due to missing data. We made repeated attempts to contact trialists for missing statistical data; while many authors were helpful in providing additional data, further information was often unavailable due to data loss or non‐response.

Blake 2001, Clark 2005, Goldfarb 1999, Anderson 1999, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, and Trenholm 2007d provided sufficient data for us to analyze all relevant outcomes in RevMan. We had insufficient data to control for cluster randomization for Anderson 1999. This study evaluated pregnancy and observed only one event in each trial arm at 12‐month follow‐up (a non‐significant effect), so controlling for cluster randomization was unlikely to change the results. Kirby 1997a, Kirby 1997b, and Kirby 1997c provided sufficient information to enter dichotomous outcome data, but we could not enter continuous outcomes because we lacked standard deviations for group means. We had sufficient information to dichotomize outcomes for number of partners for four studies (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d); however, we are unaware of any consensus on a clinically meaningful difference (e.g., did or did not have more than three lifetime sex partners), so we have accepted the trialists' analyses according to frequency distribution.

Although controlling for baseline values was a strength of these reports, some of the statistical analyses reported did not control for the unit of randomization or correct for multiple tests. Of the six studies that used cluster randomization (Anderson 1999, Blake 2001, Clark 2005, Goldfarb 1999, Kirby 1997a, Kirby 1997b), two performed analyses on an individual level without reporting statistical procedures to accommodate the unit of assignment (Anderson 1999 and Goldfarb 1999). The other four studies incorporated multi‐level analyses to control for cluster randomization (Blake 2001, Clark 2005, Kirby 1997a, Kirby 1997b). Only three studies used a correction to accommodate multiple statistical tests, accepting a significance level of p<0.01 instead of p<0.05 (Kirby 1997a, Kirby 1997b, Kirby 1997c). The lack of control for multiple statistical tests makes the remaining papers more vulnerable to Type I error (i.e., believing that programs significantly affected behavior, when significance occurred due to chance).

Other potential sources of bias 
 The reliance on self‐reported data is a salient but inevitable source of bias for all studies evaluating sexual behavior. The limitations of self‐report for sexual behavior have been analyzed elsewhere (Alexander 1993, Binson 1998, Brener 2004, Catania 1996, Lauritsen 1997, Meston 1998, Newcomer 1988, Rosenbaum 2006, Zabin 1984), particularly with respect to adolescents, and these data are also vulnerable to misreport due to the variety of definitions for terms such as "sex" or "partner." Self‐report biases may be heightened in evaluations of abstinence‐only programs due to the nature of the interventions; recent survey data suggest that youth who take virginity pledges may be more likely to retract their sexual histories at a later date (Rosenbaum 2006). Furthermore, there are demonstrable limits to the use of sexual behavior as a proxy for HIV/STI risk (Peterman 2000). The most robust data for abstinence‐only program evaluations would be medical evaluations of HIV or STI incidence; however, due perhaps to the young age of the participants, resource limitations, or logistical complications, no study was able to evaluate outcomes in this way.

Due to the limited space allocated to program design and implementation in published journals, the majority of studies reported little data on these aspects of the evaluation. A lack of specific definitions of "abstinence" and "sex" leaves programs vulnerable to varying interpretations by participants and staff. Different programs may emphasize abstinence to differing extents or provide different amounts of information regarding the limitations of condom use; however, we could not uncover these differences from the available data. Some of our difficulty identifying abstinence‐only programs and program similarities may be partly due to the controversial political climate surrounding such programs, along with confusion regarding the difference between abstinence‐only and abstinence‐plus designs.

All 13 of the evaluations attempted to some extent to monitor program dosage, promote treatment adherence, or assess the quality of implementation, but few details regarding actual implementation were provided. Efforts to promote implementation fidelity included monitoring attendance, homework completion, or video exposure (Blake 2001, Kirby 1997a, Kirby 1997b, Kirby 1997cMiller 1993); making trial staff available to assist in the conduct of program sessions (Blake 2001); having independent evaluators observe program delivery (Anderson 1999); requiring quarterly implementation reports from program facilitators (Kirby 1997a, Kirby 1997b, Kirby 1997c); interviewing program facilitators, program contractors, and participants (Kirby 1997a, Kirby 1997b, Kirby 1997c); conducting site visits and observing program sessions (Kirby 1997a, Kirby 1997b, Kirby 1997c); using standard intervention materials; and using treatment manuals. Seven studies reported to any extent on training for program facilitators (Anderson 1999, Blake 2001, Clark 2005, Goldfarb 1999, Kirby 1997a, Kirby 1997b, Kirby 1997c). Two programs were media‐based and used standard materials (Hernandez 1990, Miller 1993). It is unclear whether strategies for monitoring and enhancing adherence were successful; few studies provided details on the actual extent of implementation fidelity (e.g., proportion of core components actually delivered, participant attendance rates).

In the trials by Trenholm, et al, attendance data suggested that the two elective programs (Trenholm 2007b, Trenholm 2007c) were poorly attended. In Trenholm 2007b, 35% of participants assigned to the experimental group chose not to participate; however, they were appropriately analyzed according to original assignment. In Trenholm 2007c, 43% of these participants chose not to participate in any of the program activities. Of the 57% experimental‐group participants who did participate in Trenholm 2007c, 55% participated in less than half of the daily program sessions available in the first year (a total of approximately 720 daily program sessions were offered over the course of four years).

Effects of interventions

The protocol for this review specified that a meta‐analysis would be conducted if appropriate. After a thorough investigation of the data set, we determined that a statistical synthesis of the given data would be uninformative for several reasons.

Incomparability of interventions: One compelling reason to refrain from meta‐analyzing the results was clinical heterogeneity; there were differences among the 13 programs in terms of intervention design, delivery, exposure, staff, and extent of emphasis on abstinence. These differences made the programs unsuitable for aggregation, since a meta‐analysis aggregating disparate interventions may not provide useful insight regarding which types of programs are effective. Furthermore, the evaluations utilized a number of different control groups, recall periods, and follow‐up times. Grouping such different comparisons together could have been misleading, but separating the evaluations into sub‐comparisons rendered the groupings too small for an informative meta‐analysis.

Data availability: Another major obstacle to meta‐analyzing these results was the underutilization and underreporting of primary outcomes and difficulty entering some results into RevMan. Data unavailability also made it difficult to assess publication bias.

Statistical: There were several statistical reasons why a meta‐analysis may have been misleading. Meta‐analyzing primary studies with analyses that did not account for dropouts may have compounded this error, particularly since dropouts sometimes differed significantly from trial completers. Other statistical obstacles included accommodating cluster randomization, adjusting results for baseline differences, and handling trials with more than two arms, although these could have been overcome had we decided to proceed with the meta‐analysis.

For these reasons, we present results from individual trials instead of providing a quantitative synthesis. RevMan analyses are provided where possible. Where possible, we transformed the results of cluster‐randomized trials (Blake 2001, Clark 2005, Goldfarb 1999, Kirby 1997a, Kirby 1997b) before entering them into RevMan. Since Goldfarb 1999 did not report numbers of participants enrolled at baseline, average cluster size was calculated using the number of participants analyzed at follow‐up; we calculated average cluster size for Kirby 1997a and Kirby 1997b using the available baseline enrollment numbers.

Analyses were impossible in RevMan for the following comparisons: Hernandez 1990; Miller 1993; continuous outcomes for Kirby 1997a, Kirby 1997b, and Kirby 1997c; and number of partners for Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, and Trenholm 2007d. These results are reported as available in the primary studies. All results are summarized in Figures 1‐9 (see Figure 1).

Throughout these results, odds ratios less than one indicate a protective effect of the abstinence‐only programs. Odds ratios are displayed with 95% confidence intervals. Results were significant at p<0.05.

Biological outcomes 
 Sexually transmitted infection 
 See Figure 3. Seven trials, with analyses representing 9779 participants at any time point, assessed participants' reports of ever having been diagnosed with a sexually transmitted infection by a doctor or nurse (Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). All compared an abstinence‐only curriculum against usual care. No trial found a protective effect at any time point, and one trial found significant adverse effects compared to usual care.

Three trials (Kirby 1997a, Kirby 1997b, and Kirby 1997c) assessed STI diagnosis among participants who reported never having been diagnosed with an STI at baseline. Kirby 1997b and Kirby 1997c observed no significant findings at long‐term follow‐up (n=4133). Kirby 1997a systematically varied the staff delivering the intervention between adult and peer leaders; participants who received the intervention from adult leaders were significantly more likely than controls to report having been diagnosed with an STI at three‐month follow‐up (n=2711, OR=4.16 [1.16, 14.94], p=0.03) and 17‐month follow‐up (n=2313, OR=2.73 [1.05, 7.14], p=0.04). It was unclear, however, whether this was due to changes in reporting, frequency of testing, or actual risk behavior. The youth‐led arm of Kirby 1997a did not report significantly different rates of STI infection at three‐month (n=1895, OR=3.04 [0.59, 15.73], p=0.18) or 17‐month (n= 1545, OR=2.06 [0.67, 6.32], p=0.21) follow‐up.

Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, and Trenholm 2007d assessed STI diagnosis among participants who reported ever having had sex. No trial found a significant effect at long‐term follow‐up compared to usual care (n=1040).

For the six studies with non‐significant findings, odds ratios ranged from 0.31 (Kirby 1997c) to 1.73 (Trenholm 2007d). Long‐term findings for Kirby 1997a and Kirby 1997c are limited by high attrition. Trial‐specific results can be found in Figure 3.

Pregnancy 
 See Figure 4. Eight trials evaluated self‐reported pregnancy incidence among 9417 participants (Anderson 1999, Kirby 1997a, Kirby 1997b, Kirby 1997c, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). No study found a protective at any time point, and one found a significant adverse effect compared to usual care.

Anderson 1999 assessed pregnancy since baseline among all participants, finding no significant effect compared to no treatment at short‐term follow‐up (n=405) or long‐term follow‐up (n=251).

Three trials assessed pregnancy incidence ever, among participants who had reported never having been pregnant or caused a pregnancy at baseline (Kirby 1997a, Kirby 1997b, Kirby 1997c). Kirby 1997b and Kirby 1997c found no significant effects at long‐term follow‐up compared to usual care (n=4129). In Kirby 1997a, classrooms who received the intervention from peer leaders were significantly more likely than usual care controls to report having been pregnant at 17‐month follow‐up (n=1548, OR=2.35 [1.12, 4.95], p=0.02). However, this effect was largely influenced by the affirmative reports of six males from the seventh‐grade class of a single school, and it was not mirrored by significant changes in any measured sexual behavior. There was no significant program effect for the adult‐led sub‐group of Kirby 1997a (n=2295, OR=1.35 [0.73, 2.52], p=0.34).

Four trials assessed pregnancy incidence ever among participants who reported ever having had sex (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). No trial found a significant effect at long‐term follow‐up compared to usual care (n=1040).

Odds ratios in studies reporting non‐significant findings ranged from 0.35 (Anderson 1999 without controlling for clustering) to 1.36 (Kirby 1997b and Trenholm 2007c). Long‐term findings for Anderson 1999, Kirby 1997a, and Kirby 1997c are limited by high attrition. Trial‐specific findings appear in Figure 4.

Behavioral outcomes 
 Incidence of unprotected vaginal sex 
 See Figure 5. Five trials provided sufficient data to extract incidence of unprotected vaginal sex among all participants (Goldfarb 1999, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). Every trial compared an abstinence‐only program to usual care. This outcome was assessed among all participants, including those who did not report recent sex; analyses represented 2892 participants at any time point. In analyses that controlled for clustering, no trial found a significant program effect at short‐term or long‐term follow‐up.

Goldfarb assessed unprotected sex among all participants in the past month, and results were limited to two‐month follow‐up. When we conducted analyses on the level of the individual, results significantly favored the program group over usual care (n=839, OR=0.54 [0.34, 0.84], p=0.007). However, when we re‐analyzed these data using ICC=0.015 to control for cluster randomization, the effect was no longer significant (OR=0.56 [0.25, 1.23], p=0.15).

We extracted outcome data from the remaining four studies by examining the percentage of participants who had sex in the last year and reported inconsistent condom use over the same period (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). No trial found a significant effect at long‐term follow‐up (n=2053). Odds ratios ranged from 1.01 (Trenholm 2007a) to 1.16 (Trenholm 2007d).

Trial‐specific findings appear in Figure 5.

Incidence of vaginal sex 
 See Figure 6. Seven evaluations reported incidence of recent vaginal sex at immediate post‐intervention follow‐up, with analyses representing 3454 participants (Blake 2001, Clark 2005, Goldfarb 1999, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). Analyses represent all participants who were followed up in each trial. In analyses that controlled for clustering, one trial observed protective effects compared to usual care at short‐term follow‐up.

Goldfarb 1999 evaluated sex in the past month among all participants; in analyses that controlled for clustering, findings significantly favored the abstinence‐only program over usual care at two‐month follow‐up (n=839, OR=0.53 [0.29, 0.97], p=0.04). Measurement error was possible for this finding, because a higher percentage of control participants reported having sex in the past month than reported ever having sex.

Blake 2001 assessed sex in the past three months among all participants, and found no significant effect at 1.5‐month follow‐up compared to a non‐enhanced version of the experimental program (i.e., without parent‐child homework, n=351). Findings may not accurately evaluate program effect because the recall period encompassed both pre‐intervention and post‐intervention behaviors.

Clark 2005 assessed sex since baseline at 4.4‐month follow‐up (n=211), and sex since the 4.4‐month follow‐up at 12 months (n=156); findings were not significant in RevMan at either follow‐up compared to usual care, regardless of whether we adjusted for clustering. However, using hierarchical level modeling, the paper reported a significant effect favoring the intervention group at short‐term follow‐up (p<0.015).

Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, and Trenholm 2007d assessed sex in the past 12 months among all participants at long‐term follow‐up, and none found a significant effect compared to usual care (n=2053).

Odds ratios in trials with non‐significant findings ranged from 0.69 (Clark 2005) to 1.10 (Trenholm 2007c). Long‐term results for Clark 2005 were limited by high attrition. Trial‐specific results appear in Figure 6.

Frequency of vaginal sex 
 See Figure 7. Four studies assessed participants' frequency of vaginal sex, with analyses representing 2376 participants at any time point; no study observed protective effects at any time point, and one found an iatrogenic effect compared to usual care at short‐term follow‐up.

Hernandez 1990 used a four‐arm study to compare participants in abstinence‐only, abstinence‐plus, safer‐sex, and non‐intervention conditions. At six weeks after baseline (n=388), results indicated no significant program effects on the frequency of vaginal sex in the past six weeks according to a 4 x 2 x 2 x 2 factorial design (program assignment x gender x sexual activity at baseline x time).

Kirby 1997a, Kirby 1997b, and Kirby 1997c examined the mean change in frequency of vaginal sex using a recall period of three months. Analyses were limited to participants who reported ever having had sex, and results are from t‐tests based on change scores. Kirby 1997b and Kirby 1997c found no significant effects compared to usual care (n=1064). At three‐month follow‐up, the youth‐led arm of Kirby 1997a reported a significantly larger increase in frequency of sex than usual care controls (n=338, means 0.9 vs. 0.3, p=0.02 in the primary study). The paper did not report this as a significant result due to a Bonferroni correction for multiple statistical tests. The effect was not significant at the 17‐month assessment (n=338, means 1.8 vs. 1.6, p=0.62). No significant effects overall were found in the adult‐led arm of Kirby 1997a at three‐month follow‐up (n=464, means 0.8 vs. 0.4, p=0.11) or 17‐month follow‐up (n=586, means 2.0 vs. 1.6, p=0.16). 
 
 Long‐term findings for Kirby 1997a and Kirby 1997c are limited by high attrition. Results could not be analyzed in RevMan due to missing data. Trial‐specific findings appear in Figure 7.

Number of sex partners 
 See Figure 8. Eight trials reported participants' number of sex partners, with analyses representing 4483 participants at any time point. No trial found a significant effect at any time point.

Hernandez 1990 compared an abstinence‐only program to an abstinence‐plus program, a safer‐sex program, and no treatment; the trial assessed participants' number of partners since baseline at six‐week follow‐up (n=388). The trial found no significant effect according to a 4 x 2 x 2 x 2 factorial design (program assignment x gender x sexual activity at baseline x time).

Three trials assessed participants' mean change in lifetime number of sex partners since baseline, with analyses limited to sexually experienced participants. These trials found no significant effect compared to usual care at short‐term follow‐up (n=812, Kirby 1997a), or at long‐term follow‐up (n=2042, Kirby 1997a, Kirby 1997b, Kirby 1997c).

Four trials assessed lifetime number of sex partners among all participants; according to F‐tests of distributional differences, none found a significant effect at long‐term follow‐up compared to usual care (n=2053, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d).

Long‐term findings for Kirby 1997a and Kirby 1997c are limited by high attrition. Results could not be analyzed in RevMan due to missing data. Trial‐specific findings appear in Figure 8.

Lack of condom use 
 See Figure 9. We transformed condom use data to reflect lack of condom use, so that all odds ratios less than 1 would continue to indicate a protective effect. Eight trials assessed lack of condom use, with analyses representing 3254 participants at any time point; none found a significant program effect at short‐term or long‐term follow‐up.

Goldfarb 1999 compared an abstinence‐only program to usual care and reported the following at two‐month follow‐up: "For those reporting intercourse, 47% of controls and 50% of the intervention group reported using a condom." It was unclear whether this referred to condom use at last sex, condom use in the past month, or condom use ever. It was also unclear whether this referred to participants who had sex in the past month or participants who had ever had sex. We transformed this analysis to reflect "lack of condom use," and we analyzed this data first among participants who reported sex in the past month (n=171, OR=0.98 [0.33, 2.91], p=0.97), and then among participants who reported sex ever (n=167, OR=0.75 [0.26, 2.23], p=0.61). Neither finding was significant. 
 Three trials assessed lack of condom use at last sex among sexually experienced participants, and found no significant effect compared to usual care at short‐term follow‐up (n=810, Kirby 1997a) or long‐term follow‐up (n= 2043, Kirby 1997a, Kirby 1997b, Kirby 1997c). Results at three‐month follow‐up for the adult‐led arm of Kirby 1997a initially favored the intervention group when we conducted analyses on an individual level (n=471, OR=0.67 [0.46, 0.97], p=0.03), but were not significant once we adjusted for clustering (OR=0.64 [0.44, 1.01], p=0.06). The paper does not report this finding as significant due to a Bonferroni correction.

Four trials found no significant effect on lack of condom use at first sex at long‐term follow‐up, among participants who reported ever having had sex (n=1040, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d).

Specific odds ratios for lack of condom use ranged from 0.60 (Trenholm 2007b) to 1.22 (Kirby 1997c). Long‐term findings for Kirby 1997a and Kirby 1997c are less robust due to high attrition. Trial‐specific findings appear in Figure 9.

Additionally, one trial assessed the absolute number of times participants used condoms in the past six weeks, and found no significant difference between participants in abstinence‐only, abstinence‐plus, safer‐sex, and no‐treatment conditions (n=388, Hernandez 1990). However, this outcome does not indicate the proportion of sex acts for which condoms were used; it simply reports the number of times participants used condoms at all.

Sexual initiation 
 See Figure 10. Ten studies reported the incidence of sexual initiation (i.e., whether participants had ever had sex), with analyses representing 11298 participants at any time point. No study found a significant effect at short‐term or long‐term follow‐up.

Six trials assessed all participants' reports of ever having sex, and none found a significant effect at short‐term follow‐up compared to a non‐enhanced program version (without parent‐child homework, n=351, Blake 2001), at short‐term follow‐up compared to usual care (n=839, Goldfarb 1999), or at long‐term follow‐up compared to usual care (n=2053, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). Results for Goldfarb 1999 were initially significant without controls for clustering (n=839, OR=0.64 [0.45, 0.90], p=0.01), but findings did not remain significant with adjustments for clustering (OR=0.66 [0.36, 1.21], p=0.18).

Four trials assessed sexual initiation among participants who reported never having had sex at baseline; all four compared an abstinence‐only program to usual care. None found a significant effect at short‐term follow‐up (n=4247, Clark 2005, Kirby 1997a), or at long‐term follow‐up (n=7474, Clark 2005, Kirby 1997a, Kirby 1997b, Kirby 1997c). When we analyzed results for Clark 2005 on an individual level, findings significantly favored the intervention group (n=134, OR=0.25 [0.07, 0.97], p=0.04), but were no longer significant once we adjusted for clustering (OR=0.28 [0.07, 1.10], p=0.07).

Odds ratios across trials ranged from 0.28 (Clark 2005) to 1.17 (Trenholm 2007d). Long‐term findings for Clark 2005, Kirby 1997a, Kirby 1997c are limited by high attrition. Trial‐specific findings appear in Figure 10.

Sexual risk behavior index 
 See Figure 11. Miller 1993 was the only study to use a risk behavior index, which included behaviors ranging from hand‐holding to sexual intercourse or "the sexual act by which pregnancy can occur." The study was a three‐arm trial: the three arms received (1) a video‐based intervention complete with periodic newsletters, (2) the video‐based intervention alone, or (3) no intervention. At 12‐month follow‐up, no significant group‐by‐time interaction was observed (p=0.662) according to a 3 (group) x 3 (time) repeated measures ANOVA. Reports of vaginal sex could not be disaggregated from this outcome, although the incidence of vaginal sex reportedly ranged from 3% to 5% in the three intervention arms, with a total of 503 participants at follow‐up.

Knowledge 
 See Figure 12. Seven trials reported any measure of knowledge, but none of these specifically assessed knowledge related to HIV/AIDS (Blake 2001, Goldfarb 1999, Miller 1993, Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d).

Blake, et al assessed whether participants knew "the effectiveness of abstinence" and the risk of pregnancy at first intercourse (Blake 2001), and found no significant difference compared to a non‐enhanced program version (without parent‐child homework) at immediate post‐intervention (n=351).

Goldfarb, et al assessed "sex knowledge" with a usual care control group at two‐month follow‐up (n=839), but did not report a measure of between‐group significance (Goldfarb 1999). As the study reports, "the intervention group showed significant increases in the number of correct answers to all four knowledge questions ... the control group showed no significant change."

Miller assessed knowledge of sexual facts, including "physical development, myths about teen sex, risks of sex, [and] interpersonal respect" (Miller 1993). The trial compared an abstinence‐only program with newsletters, a program without newsletters, and a no‐treatment control, with assessments at immediate post‐intervention and 12 months. Again, between‐groups tests of significance were not provided. As the trial reports, intervention groups had "a significant increase in knowledge from pre‐test to post‐test, with a plateau thereafter," while controls "showed a gradual increase in knowledge, with their delayed post‐means having caught up with the treatment post‐means."

The four trials by Trenholm, et al compared different abstinence‐only programs to usual care. These trials assessed three outcomes: identification of "true and false STDs," knowledge of the risks of unprotected sex, and knowledge of the consequences of STDs (Trenholm 2007a, Trenholm 2007b, Trenholm 2007c, Trenholm 2007d). One trial found significantly protective effects for all three outcomes at 62.5‐month follow‐up (n=447; Trenholm 2001); the remaining three trials found no protective effects at long‐term follow‐up (n=1606; Trenholm 2007b, Trenholm 2007c, Trenholm 2007d) Additionally, an analysis of all four trials together suggested that the programs may have led to a significant adverse effect on knowledge of condoms. At long‐term follow‐up, the overall analysis showed that a larger proportion of intervention youth reported that condoms "never prevent HIV" compared to usual care controls (21% vs 17%); intervention youth were also significantly less likely than controls to report that condoms "usually" prevent HIV (34% vs. 38%).

Cost data and participant satisfaction 
 None of the evaluations reported cost‐effectiveness, although PSI/ENABL, as evaluated in Kirby 1997a, Kirby 1997b, and Kirby 1997c was reportedly "expensive" (Saunders 1996, cited in Cagampang 1997). Overall, the PSI/ENABL program received $15 million for the first three‐year funding cycle, including the cost of evaluation (Cagampang 1997). The three‐part evaluation was presented to state administrators, including the governor, before the removal of state funding from the PSI program (Cagampang 1997).

Four studies collected data on participant satisfaction; Anderson 1999 reported that participant surveys, ethnographic observations, interviews, and focus groups indicated high levels of satisfaction and youth participant engagement. Kirby 1997a, Kirby 1997b, and Kirby 1997c collected participant satisfaction data from focus groups, interviews, and surveys, and similarly reported high levels of satisfaction (along with political popularity). Of the participants surveyed, 82% reported that the program was "good or excellent," while 88% stated that the program information was helpful. However, participants who had a "serious boyfriend or girlfriend" reported significantly lower levels of satisfaction than youth who had not had a serious relationship; additionally, 82% of participants who suggested additional program topics recommended that the program contain more information about "how to keep from getting STDs and HIV."

Characteristics of included studies [ordered by study ID]

Characteristics of excluded studies [ordered by study ID]

Characteristics of ongoing studies [ordered by study ID]

Internal sources

• Centre for Evidence‐Based Intervention, University of Oxford, UK.

• Department of Social Policy and Social Work, University of Oxford, UK.

External sources

• No sources of support supplied