Validating Instructional Design
The last step is to validate the material by using representative samples of the target population trialing the program and then revising the program as needed. The heart of the systems approach to training is revising and validating the instructional material until the learners meet the planned learning objectives. Also, it should not be thought of as a single shot affair. Success or failure is not measured at a single point.
This is quite similar to "prototyping," except trialing is normally on a larger scale. Bill Moggridge (2007) wrote that iterative prototyping, understanding people, and synthesis are the core skills of design and that these skills are central to design:
- Iterative Prototyping - successive small-scale tests on variations of a limited function prototype in order to permit continual design refinements.
- Understanding People - having a basic foundation of the cognitive and behavioral sciences.
- Synthesis - applying prior knowledge and skills to produce a new, innovative, or original whole.
Prototyping allows designers to look at their concept in real world usage before final design decisions are committed to, which makes it quite useful in highly complex areas. Understanding people has always been a big part of designing for performance, however it now extends out to the real world and the concepts and products we create. And of course this is all brought together in a unified whole.
The initial validation will depend upon the complexity of the training material and your resources. Listed below is a procedure that provides an effective validation of a large training program. Adjust it as needed to fit the size and complexity of your program, but keep in mind that the closer your validation follows this one, the less problems you will encounter during your training.
Step One
Select the participants that will be in the trials. The participants should be randomly selected, but they must represent all strata of the target population, bright, average, and slow learners. They should be clearly told what their roles are in the validation process are. Let them know that they are helping to develop and improve the lessons and that they should feel free to tell you what they think about it. The participants should be pretested to ensure that the students learn from the instructional material and not from past experience.
Step Two
Conduct individual trials. This trial is performed on one learner at a time. The instruction is presented to the learner. The separate pieces of instructions, tests, practice periods, etc., should be timed to ensure they match the estimated times. Do not tutor unless the learner cannot understand the directions. Whenever you help or observe the learner having difficulty with the material, document it.
Step Three
Revise instruction. Using the documents from the individual trials, revise the material as needed. Closely go over any evaluations that were administered. A large number of wrong answers for an item indicates a trouble area. Conversely, a large number of correct answers for an item could indicate the learners already knew the material, the test items were too easy, or the lessons over taught the material.
Step Four
Repeat individual trials until the lesson does what it is supposed to do. There is no magical number for individual trials. From three to five times seems to be the usual number. Also, if you are trialing a large course, you might only need to trial specific troublesome areas of the course, rather than the whole course itself.
Step five
Conduct group trial. After you are satisfied with the results of the individual tryouts, move on to the group tryouts. These can be of any size. It may consist of several small groups, one large group, or a combination of both. The procedure is the same as the individual tryouts except for one difference. At some point in the trials you must determine if the program needs to be accepted or if it needs major revision. Usually a minimum of two successful tryouts are conducted to ensure the program does what it is supposed to do. Minor problems should not hold up implementing the program. As was stated earlier in this section, revisions do not stop upon the first implementation of the program, but are performed throughout the life of the program.
Reference
Moggridge, B. (2007). Designing Interactions. Cambridge, Massachusetts: The MIT Press.
Next Steps
Go to the next chapter: Implementation (Delivery) Phase
Return to the Table of Contents