Appendix A.1: Critical Assignment Guidelines
Use this guideline to assist you in the development of your program evaluation of an academic or behavioral program implemented in your school, district, classroom, community or agency.
Program
Component and Description
Comments/
Notes:
1.
Program
narrative. Include program name, program’s history and why the program was
developed, who is served by the program, who developed the program, the
primary goals, secondary or other goals, strategies and activities (3-7).
(1-2 written pages)
2.
Logic
Model – Headings for strategies, early and intermediate objectives, and
long-term goals. Strategies identified using rounded rectangles (3-7
strategies should be noted. Early and intermediate objectives are represented
in rectangles and worded as outcomes. Long term goals identified using ovals
and worded as outcomes. All strategies need to link to long term goal. Arrows
representing theory are accurate.
3.
Program Theory – Theory linking each
strategy to long-term goals clearly identified with bullet points or are
numbered. Long term goals worded as outcomes, clearly identified and
distinguished (bold or underlined). Strategies are clearly identified with
bolding or underling. All strategies lead to a long-term goal. The first
objective for each strategy addresses the opportunity for the strategy to be
successful. Theory has at least two early/intermediate objectives between
activity and long-term goal. Theory is well thought out and logical.
Contextual conditions address infrastructure and training necessary for
program. Contextual conditions address both program and evaluation funding
4.
Evaluation Design & Matrix
–Create an evaluation design for your program, as well as identify and
explain the evaluation approaches that will be used. Clearly explains
evaluation approach(es), including who developed and how evaluation
approach(es) will be implemented within the evaluation. Explains strengths
and weaknesses of design, including threats to validity and feasibility of
the program. Clearly indicates the number of groups, the number of subjects,
and demographics of subjects. Clearly indicates the length of the study. At
least one additional method is provided to enrich the evaluation design.
Matrix includes
name of program at top, includes proper column headings, rows are grouped
correctly by logic model component type (strategies, objectives, goals). Each
model component has a separate row in the matrix and every model component is
included in the matrix (once). Every model component has at least one
question, strategy questions are open-ended and measurable, strategy
questions address implementation, early and intermediate objectives questions
are open-ended and measurable, Early and intermediate objectives questions
address corresponding model component, Long-term goal questions are
open-ended and measurable and address corresponding model component
5.
Video
record presentation sharing all components. Upload presentation
6.
View
assigned peer presentation and provide feedback using assignment rubric and
guidelines determine improvement of program. Video record feedback and share
with peer.
Appendix A: Critical Assignment Rubric
Program Design & Evaluation Project (Critical Assignment) ____/60 pts
Through the course students will complete components of an evaluation of a project. Due to the nature of the project it is important for students to identify an educational or behavioral program (reading, mathematics, social-emotional learning, social skills etc.,) implemented in your school, classroom, district, community, or agency designed to support diverse students with exceptionalities in various learning environments within the first two weeks of the course. Components of the plan will include an overview of the program with strategies and activities, draft of a logic model, program theory, one evaluation question per logic model component, proposed evaluation design, and evaluation matrix. A recorded presentation of your project will be uploaded to Canvas and you will be assigned one or two peers to provide feedback based on specific criteria for improvement. (Objectives 1, 2, 3, 4, 5, 6 & 7)
EEX 6320 Program Design & Evaluation
Project
Exemplary
(3 pts)
Satisfactory
(2 pts)
Emerging
(1 pt)
Unsatisfactory
(0 pt)
No Submission
(0 pt)
Program Narrative
(CEC 3.2; ALC Content and DEI)
10% of total points
6 pts
Submission clearly and succinctly
includes program name, history, and explains why the program was developed, individuals
served by the program, who developed the program and how the program aligns
with the needs of diverse individuals and promotes equity, the primary and
secondary goals of the program, and a detailed explanation of the activities
or strategies of the program.
Submission clearly and includes
program name, history, and explains why the program was developed, individuals
served by the program, who developed the program and how the program aligns
with the needs of diverse individuals and promotes equity, the primary goals
of the program, and an explanation of the activities or strategies of the
program.
Submission identifies the program
name, provides a brief history and explanation of why the program was
developed, some of the goals of the program, as well as activities or
strategies
Submission is: missing significant
elements as described in the Satisfactory column; not in the required format;
and/or, not clearly written.
Did not attempt/include this section;
and, or failed to submit/submit on time.
Logic Model
(CEC 3.1; ALC Communication)
20%
of total points
12
pts
(Use
Lammert et al as a guide)
Submission includes the creation of
a complete Logic model description and visual. The model includes between
five to seven strategies, connecting to long term, short and intermediate
term goals. Links from all strategies to goals are clearly and accurately
delineated.
Submission includes the creation of
a complete Logic model description and visual. The model includes between
three to four strategies, connecting to long term, short and intermediate
term goals. Links from all strategies to goals are delineated.
Submission includes the creation of
a partially complete Logic model description and/or visual. The model
includes two or fewer strategies,
connecting to long term or short or
intermediate term goals. Links from all strategies to goals are
somewhat delineated.
Submission is: missing significant
elements of the Logic model as described in the Satisfactory column; not in
the required format; and/or, not clearly written.
Did not attempt/include this section;
and, or failed to submit/submit on time.
Program Theory
(CEC 3.1, 3.); (ALC DEI)
20%
of total points
12
pts
Submission clearly and
comprehensively identifies and explains the theory linking to each strategy
supported with connections to the course content. Explanations of theory will
include two or more considerations for diverse student populations and
providing equitable outcomes for all. Contextual conditions clearly address
infrastructure and training needs for the program as well as funding.
Submission clearly identifies and
explains the theory linking to each strategy supported with connections to
the course content. Explanations of theory will include at least one
consideration for diverse student populations and providing equitable
outcomes for all. Contextual conditions address infrastructure and training needs
for the program as well as funding., then and now.
Submission overall identifies and
explains the theory linking to each strategy supported with some connections
to the course content. Explanations of theory will may lack consideration for diverse student
populations and providing equitable outcomes for all. Minimal contextual
conditions address infrastructure and training needs for the program as well
as funding
Submission is: missing significant
elements as described in the Satisfactory column; not in the required format;
and/or, not clearly written.
Did not attempt/include this section;
and, or failed to submit/submit on time.
Evaluation Design & Matrix
(CEC 3.1,3.2, 3.3, 3.4,3.5) (ALC DEI)
30%
of total points
18
pts
(Use
Lammert et al as a guide)
Submission concisely identifies and
explains the evaluation approaches including who developed the approach and
how it will be implemented. Evaluation design will be explained to include
strengths, weaknesses, and threats to validity. The plan will indicate number
groups, subjects, and demographics expanding on how a diverse demographic
will be recruited. Two or more enrichments are to be provided to improve the
program.
Matrix clearly and accurately aligns
with logic model. Every strategy in the logic model will have a minimum one
question with some having more than one. Each long, short and intermediate
goal has an open-ended and measurable question to support improvement.
Submission identifies and explains
the evaluation approaches including who developed the approach and how it
will be implemented. Evaluation design will be explained to include
strengths, weaknesses, and threats to validity. The plan will indicate number
groups, subjects, and demographics expanding on how a diverse demographic will
be recruited. At least one enrichment is to be provided to enhance the design
are provided.
Matrix aligns with logic model.
Every strategy in the logic model will have a minimum one question. Each
long, short and intermediate goal will have an open-ended and measurable
question to support improvement.
Submission identifies and explains
the evaluation approaches generally. Evaluation design will include
strengths, weaknesses, and threats to validity. The plan will indicate number
groups, subjects, and demographics. An enrichment is to be provided to
enhance the design are provided.
Matrix aligns with logic model. Most
strategies in the logic model have one question. Most long, short and
intermediate goal will have an open-ended and measurable question to support
improvement.
Submission is: missing significant
elements as described in the Satisfactory column; not in the required format;
and/or, not clearly written.
Did not attempt/include this section;
and, or failed to submit/submit on time.
Evaluation Presentation & Peer
Feedback
(CEC 6.4, 6.5); ALC Content and DEI)
15% total points
9
Submission includes video recorded
presentation of program overview, theory, logic model, evaluation design and
matrix in a cohesive and rehearsed video recorded presentation with closed
captioning or included text of presentation. Peer feedback is comprised of
reflections and actionable comments for each section of the matrix that
support improvement of program evaluated supported by evidence for course
content. .
Submission includes video recorded
presentation of program overview, theory, logic model, evaluation design and
matrix in a rehearsed video recorded presentation. Peer feedback is comprised
of reflections and actionable comments for most sections of the matrix that
support improvement of program evaluated supported by evidence for course
content.
Submission includes video recorded
presentation of program overview, theory, logic model, evaluation design and
matrix however presentation indicates lack of rehearsal and
cohesiveness. Peer feedback is
comprised of reflections and comments for some sections of the matrix that
lack actionable items for improvement.
Submission is: missing significant
elements as described in the Satisfactory column; not in the required format;
and/or, not clearly written.
Did not attempt/include this section;
and, or failed to submit/submit on time.
Mechanics
5%
of total points
3
pts
Paper is typed, double-spaced,
utilize Times New Roman font, and 1” margins; title page, in-text citations, reference (page), and
headings strictly adhere to the
most recent APA guidelines; and/or contains no more than 1 writing error per
page.
Paper is typed, double-spaced,
utilize Times New Roman font, and 1” margins; title page, in-text citations, reference (page), and
headings mostly adhere to the most
recent APA guidelines; and/or no more than 2 writing errors per page.
Paper is typed, double-spaced,
utilize Times New Roman font, and 1” margins; title page, in-text citations, reference (page), and
headings loosely adhere to the most
recent APA guidelines; and/or no more than 3 writing errors per page.
Paper is NOT typed, double-spaced, utilize Times New Roman font, or 1”
margins; title page, in-text
citations, reference (page), and headings do
not adhere to/reflect the most recent APA guidelines; and/or contains 4
or more writing errors per page.
Did not attempt/include this section;
and, or failed to submit/submit on time.
Note:
