UWGC BellXcel Summer Program

The BellXcel Summer Program is an evidence-based summer learning program that targets rising sixth graders at three middle schools in Greenville County, South Carolina. The program’s aim is to combat summer learning loss and support the transition between elementary and middle school among low-income students who were identified as academically and/or behaviorally at-risk. The primary intended outcome of the BellXcel Summer Program was to improve student course performance in math and English/language arts (ELA).

Study Goals:

The evaluation includes an implementation and impact study. The aim was to measure model fidelity as well as the impact of the BellXcel Summer Program on students’ academic outcomes.

Research Questions:

The evaluation included both implementation and impact questions. The research questions were:

• Impact Research Questions
  • Did students who participated in the BellXcel Summer Program demonstrate improved math and English/language arts course performance, as measured by the MAP assessment, when compared to matched comparison students?
  • Did students who participated in the BellXcel Summer Program have:
    • Fewer behavioral incidences than matched comparison students?
    • Higher school attendance rates than matched comparison students?
  • Did students who participated in the BellXcel Summer Program demonstrate:
    • Higher self-confidence than matched comparison students?
    • A more positive attitude towards learning than matched comparison students?
  • At the conclusion of the BellXcel Summer Program:
    • Did students who participated in the program demonstrate an increase in reading and math skills as measured by the STAR assessment scores?
    • Did the parents of students who participated demonstrate an increase in engagement in their children’s education?
  • Did students who participated in the BellXcel Summer Program report improved:
    • Relationships with their teachers?
    • Relationships with caring adults in their school?
    • Rchool engagement?
    • Participation in educational activities over the summer at a higher frequency than comparison students?

• Implementation Research Questions
  • Were program components (such as staffing, curriculum, data systems, provision of transportation, and community engagement events) implemented as intended at each school?
  • Did the quality of the program reflect the intended design?
  • What aspects of the design were modified at each school, if any, and why were they modified? What were the barriers to implementation, if any?
  • Did 100 percent of teachers, teaching assistants, and enrichment assistants receive the proper dosage of training?
  • To what extent did the training provide teachers and assistants with the necessary skills and knowledge they needed to implement the program? What additional knowledge or skills did teachers and assistants require, if any? To what extent were teachers able to find support to fill in knowledge or skill gaps?
  • To what extent did teachers implement the curriculum as intended? What challenges emerged as teachers implemented the curriculum?
  • What suggestions do teachers have for the improvement of the various program components as implemented?
  • How were scholars recruited at each site and to what extent was the EWRS used to identify scholars? What percentage of scholars who participated in BellXcel at each site met the original target characteristics specified in the model (at or below the 60th percentile in English Language Arts or mathematics)? Which, if any, additional target characteristics were added during the recruitment phase? If additional criteria were added, what percentage of scholars met each of the target characteristics? What recommendations do district and BellXcel staff have for improving recruitment next year?
  • Were scholars provided the proper dosage of the program (i.e., six weeks operating six and a half hours per day, four days per week, with three hours of academic instruction and two hours of enrichment)?
  • What percentage of scholars completed the program at each program site? Of those who completed the program, was their average daily attendance 80 percent or higher?
  • What barriers, if any, prevented scholars from attending the program and/or using transportation at the desired rates? What strategies encouraged attendance and transportation utilization?
  • What aspects of the program did scholars prefer over others?
  • What suggestions do scholars have for improvement of the various program components as implemented?
  • Did at least 70 percent of parents attend parent engagement events?
  • How effective were the parent engagement activities?
  • How might parent engagement events be improved?

Findings:

The evaluation found the following:

• The actual implementation of the BellXcel Summer Program at each of the three OnTrack Greenville middle schools was very close to the intended implementation.
• Researchers were not able to confirm the hypothesis that BellXcel scholars would have improved course performance in ELA or math as measured by the MAP assessment when compared to matched students.
• The evaluation did yield promising findings for exploratory outcomes, indicating BellXcel scholars had significantly higher attendance rates and fewer behavioral incidences than comparison students.

For more information, download the full report and report brief.