In the past, too often were instructors blamed for poor teaching and learners for poor learning when, in fact, the materials were not sufficient to support the instructional effort. This is why formative evaluation to newly developed materials, to selected and adapted materials, to instructor delivered instruction, and to combinations of these 3 presentation modes, is nowadays a must in order to ensure that instruction is properly implemented and managed! 

Formative evaluation is the collection of data and information during the development of instruction that can be used to improve the effectiveness of the instruction (Dick and Carey, p. 277; based on Cronbach, 1975 and Scriven 1967).

=> Formative evaluation is the process designers use to obtain data that can be used to revise their instruction to make it more efficient and effective (gathering information from learners in order to revise the materials before proceeding with the design process).

The emphasis in formative evaluation is on the

1. collection and analysis of data
2. revision of the instruction

[Two related activities that share many of the same principles as formative evaluation:  usability testing and rapid prototyping]

Three basic phases of formative evaluation: (p.279)

1. One-to-one or clinical evaluation: the designer works with individual learners to obtain data to revise the materials
2. Small-group evaluation: a group of 8 to 20 learners (who represent the target population) study the materials on their own and are tested to collect the required data
3. Field trial: often 30 learners are sufficient. The emphasis here is on the testing of the procedures

The three phases of formative evaluation are typically preceded by the review of instruction by interested specialists who are not directly involved in the instructional development project, but have relevant expertise.

Designing Formative Evaluations (FEs): (p.279)

The purpose for the formative evaluation is to pinpoint specific errors in the materials in order to correct them.

The best anchor or framework for the design of the FE is the instructional strategy: create a matrix that lists the components of the instructional strategy on one side and the major areas of questions about the instruction on the other.

The 5 areas of questions that would be appropriate for all materials:

1. Type of learning: are the materials appropriate for the type of learning outcome? (ask expert)
2. Content: do the materials include adequate instruction on the subordinate skills, and are these skills sequenced and clustered logically? (ask expert)
3. Clarity: are the materials clear and readily understood by representative members of the target group? (ask target learners)
4. Motivation: do learners find the materials relevant to their needs and interests? (ask target learners)
5. Management: can the materials be managed effectively in the manner they are mediated? (ask expert & target learners)

In designing instrumentation for gathering information from learners, you must consider:

1. the phase: one-to-one, small-group, field trial
2. the setting: learning or performance context
3. the nature of the information being gathered

The types of data the we need to collect:

1. data collected on entry behavior tests, pretests, posttests, and performance context
2. learners' comments
3. data collected on attitude questionnaires and/or debriefing comments (reveals learners' overall reactions to instruction)
4. time required
5. SME's reactions
6. manager or supervisor's reactions

Relevant Link:

Example of a Formative Evaluation

Posted by Nada at 12:01 AM EDT

Updated: 05/05/09 1:17 AM EDT

Share This Post

Post Comment | Permalink