PS430: Program Design and Evaluation
Use qualitative and quantitative data to evaluate the effectiveness of behavior analytic programs.
You have another Assignment in your role as an intern at Acme Behavior Consultation.
Pathways is an alternative high school. Pathways allows students with behavior challenges to complete their high school experience with a more flexible and positive approach. Their mission is to create an environment that allows students to consistently exceed expectations, maintain high levels of student and family satisfaction, and meet cost effectiveness standards. They strive to provide the highest education possible to students from a variety of communities and cultural backgrounds.
Acme, Inc., assisted Pathways with the creation of individual behavior intervention plans (BIP) for students displaying high levels of inappropriate behaviors. In addition, ACME used a behaviorally based training procedure to train all staff on the basics of ABA and how to implement each BIP.
A behavior management division headed by a full time behavior specialist at the school is responsible for implementing the plans that were originally created in collaboration with Pathways’ staff. In addition, Acme is responsible for ongoing staff training and monitoring the effectiveness of each plan.
You are asked to do a follow up to determine the success of the procedures that were put in place. In addition, you were asked to conduct some Inter-observer Agreement(IOA) probes and determine whether the staff has maintained the skills taught during training.
You are provided with a variety of data to analyze and will need to use the data to determine the effectiveness of the current interventions.
The behavior specialist has been collecting and compiling data and presents them in the figures that are located in the Doc Sharing area of the course titled Unit 7 Assignment Figures. Below is a list of the figures contained in the Document:
Figure 1: Summary of comments from staff questionnaires related to the success of the intervention and satisfaction with training.
Figure 2: Line graph of the frequency of aggressive behaviors incidents for the entire school reported before(Baseline) and after (Treatment) the program began
Figure 3: Frequency data from an observation session with two different teachers and one student’s behavior. Figure 3 reflects the frequency of physical aggression during the sessions.
Figure 4: Data from Staff Integrity Checklist.
You must now review the data and present a summary of your findings to your supervisor at ACME, Inc., as well as the school. You should include the following in your summary:
1. For all Figures:
a. Determine the purpose and importance of the data above by discussing how it can be used in the program evaluation process. Discuss the importance of data collection and analysis in program design and evaluation.
b. Identify the data collected as either “quantitative” or “qualitative.” Discuss whether the type of data is appropriate for the context of the evaluation.
2. For Figure 1:
a. Discuss the overall purpose of this data and how it should be used in analyzing the effectiveness of the program.
b. Using the data contained in this figure, discuss the implications with regard to the effectiveness of the program.
3. For Figure 2:
a. Use visual analysis and discuss the level, trend, and variability of the challenging behavior incidents reported during baseline (before) and treatment (after the intervention).
b. Using the data, describe the effect that the intervention had on the frequency of behavior incidents within Pathways.
c. Considering the level of data, as well as any trends in the data, discuss the effectiveness of the program.
4. For Figure 3:
a. Calculate the total frequency of physical aggression for each observer.
b. Using the total count method of IOA, calculate the percentage of Inter-observer Agreement.
c. Discuss the result of IOA.
1. Is the IOA percentage within the acceptable range?
2. Discuss what changes should be considered in relation to increase the IOA percentage. If changes are not necessary, discuss what factors contribute to high percentage of Inter-observer agreement.
d. Discuss how IOA measures can be used to assist with determining the effectiveness of the program.
5. For Figure 4:
a. Calculate the percent correct results of the treatment integrity checklist before intervention, one day after intervention, and at the 3-month follow up.
· This is the total factors with a “Yes” out of the total number of factors, multiplied by 100.
b. Relate these treatment integrity measures to the effectiveness of the program.
c. Discuss the importance of treatment integrity in relation to the success of behavior management programs.
Your paper should be a minimum of 4–5 pages in length, include proper APA formatting (including properly formatted headings for each section of your plan), and include at least two academic references (including your course readings).