Summative evaluations -- which link us back to the needs assessment-- are conducted to document the strengths and weaknesses of instructional materials, to verify their effectiveness with target learners, and to make decisions about whether to maintain currently used materials or adopt materials that have the potential for meeting an organization's defined instructional needs. (pp.340-341)

Summative evaluations answer the questions:

• "Did the intervention, including the instruction, solve the problem that led to the need for the instruction in the first place?" (p.340)
• "Did it fill the identified gap?" (Dr. Watkins)

1. Expert Judgment Phase: to determine the potential of candidate instruction for meeting the needs of an organization.
   • Overall decisions: do the materials have the potential for meeting this organization's needs?
   • Specific decisions: (p.341)
     1. Congruence analysis: are the needs and goals of the organization congruent with those in the instruction? > information summary form
     2. Content analysis: are the materials complete, accurate, and current?  > product checklist or rating scale
     3. Design analysis: are the principles of learning, instruction, and motivation clearly evident in the materials?  > product checklist or rating scale
     4. Utility & Feasibility analysis: are the materials convenient, durable, cost-effective, and satisfactory for current users?  > information summary form, product checklist, or rating scale
2. Field Trial Phase: to document the effectiveness of instruction with target learners in the intended setting.
   • Overall decisions: are the materials effective with target learners in the prescribed setting?
   • Specific decisions: (p.341)
     1. Outcomes analysis: > criterion-reference tests, attitude questionnaires, interviews, observations, company records
        • Impact on learners: are their achievement and motivation satisfactory after instruction?
        • Impact on job: are learners able to transfer information/skills/attitudes to the job setting or to units of related instruction?
        • Impact on organization: are learners' changed behaviors (performance, attitudes) making positive differences in the achievement of the organization's mission and goals?
     2. Management analysis: > questionnaires, interviews, observations, company records
        • Are instructor and manager attitudes satisfactory?
        • Are recommended implementation procedures feasible?
        • Are costs related to time, personnel, equipment, and resources reasonable?

Formative Evaluation Vs. Summative Evaluation:

• Formative Evaluation: to collect data in order to revise instruction
• Summative Evaluation: to collect data in order to document the strengths and weaknesses of instruction

Posted by Nada at 12:01 AM EDT

Updated: 05/05/09 12:02 AM EDT

Share This Post

Post Comment | Permalink