Instructional System Design - Design Phase

This model or blueprint will contain five key outputs:

• Entry behaviors
• Learning objectives
• Learning steps (performance steps)
• Performance test
• Structure and sequence program outline

The learning objectives tell what tasks the learners will be able to perform after the training, while the learning steps tell how to perform the tasks, while the performance test tells how well the tasks must be met.

Finally, the learning objectives are sequenced in an orderly fashion that provides the best opportunity for learning, such as arranging the learning objective from easy to hard or in the order they are performed on the job.

A learning objective is a statement of what the learners will be expected to do once they have completed a specified course of instruction. It prescribes the conditions, behavior (action), and standard of task performance for the training setting. The objective is sometimes referred to as performance or behavioral objectives. For example, knowledge is a state of mind which cannot be directly measured. This requires an indirect method of evaluation, that of observing behavior or performance.

Also, there is a difference between goals and objectives. Goals describe a learning outcome in general, for example, "the learner will successfully complete the supervisor course, before moving on to the leadership course." Notice that this is so general that it provides no guidance about how to achieve it. On the other hand, an objective is a specific statement of instructional intent which attempts to change knowledge, skills or attitudes as a result of a learning experience, for example, "the learner will use Maslow's Hierarchy of Needs when deciding upon motivators."

Specific terminal learning objectives (Mager, 1975) must be developed for each of the tasks selected in the learning program. A terminal learning objective is at the highest level of learning (KSA) appropriate to the human performance requirements a student will accomplish. 

Each terminal learning objective is analyzed to determine if it needs one or more enabling learning objectives, that is, if it needs to be broken down into smaller, more manageable objectives. An enabling learning objective measures an element of the terminal learning objective.

A learning objective has three main parts:

• Observable Action (task) - This describes the observable performance or behavior. An action means a verb must be in the statement, for example "type a letter" or "lift a load." Each objective covers one behavior, hence, only one verb should be present. If the are many behaviors or the behaviors are complicated, then the objective should be broken down into one or more enabling learning objectives that supports the main terminal learning objective. 

• At Least One Measurable Criterion (standard) - This states the level of acceptable performance of the task in terms of quantity, quality, time limitations, etc. This will answer any question such as "How many?" "How fast?" or "How well?". For example "At least 5 will be produced", "Within 10 minutes", "Without error". There can be more than one measurable criterion. Do not fall into the trap of putting in a time constraint because you think there should be a time limit or you cannot easily find another measurable criterion — use a time limit only if required under normal working standards.

• Conditions of performance (usually) (condition) - Describes the actual conditions under which the task will occur or be observed. Also, it identifies the tools, procedures, materials, aids, or facilities to be used in performing the task. This is best expressed with a prepositional phase such as "without reference to a manual" or "by checking a chart".
  

Example 1: Write a customer reply letter with no spelling mistakes by using a word processor.

Observable Action: Write a customer reply letter

Measurable Criteria: with no spelling mistakes

Conditions of Performance: using a word processor

NOTE: If more that one type of word processor or computer is used in the organization, then it should be more specific. For example: Given a personal computer, Word for Windows, and printer, create a printed customer reply letter with no spelling mistakes. The conditions of performance are "Given a personal computer, Word for Windows, and printer". Generally speaking, the larger the organization or the more technical the task, the more specific the conditions of performance must be spelled out.

Example 2: Copy a table from a spreadsheet into a word processor document within 3 minutes without reference to the manual.

Observable Action: Copy a table from a spreadsheet into a word processor document

Measurable Criteria: within 3 minutes

Conditions of Performance: without referencing the manual.

Note: The Conditions of performance may also include a variable as shown in the next example.

Example 3: Smile at all customers, even when exhausted, unless the customer is irate.

Observable action: Smile

Measurable Criteria: at all customers

Conditions: even when exhausted

Variable: unless the customer is irate

Note: Sometimes its helpful to start with the phase "After training, the worker will be able to..."

Example 4: After training, the worker will be able to load a dumptruck within 3 loads with a scooploader, in the hours of darkness, unless the work area is muddy.

Observable Action: load a dumptruck

Measurable Criteria: within 3 loads

Conditions: with a scooploader in the hours of darkness

Variable: unless the work area is muddy

A clearly formulated objective has two dimensions, a behavioral aspect and a content aspect. The behavioral aspect is the action the learner must perform, while the content is the product or service that is produced by the learner's actions. For example, "the student will learn forklift operations by studying the operator's manual" refers not to an outcome of training but to an activity of learning. If you observed the student reading, you could make no judgment if he or she was actually learning (behavioral aspect) and there is no service produced by the learner's action (content aspect).

A better example would be "Given a forklift, load a pallet onto a trailer without any safety errors." In this example, the behavioral aspect is loading a trailer, while the content aspect is a pallet placed on the trailer.

Notice that learning objectives look a lot like tasks. A task analysis itemizes each discrete skill found in a job, but it provides only end goal statements. While learning objectives spell out the prerequisite skills and makes them the course objectives.

Using the Correct Verb

For example, the learning steps for the objective "Given a cash register and at least ten products, calculate the exact total for the purchase transaction," might read like this:

1. Enter the sales price and the department key for each product.
2. Repeat step one until all products have been entered.
3. After all items have been entered, press the subtotal key.
4. Press the Tax key.
5. Press the Total key.
   

The learning objective should be a good simulation of the conditions, behaviors and standards of the performance needed in the real world, hence the evaluation at the end of the instruction should match the objective. The methodology and contents of the learning program should directly support the learning objectives. The instructional media should explain, demonstrate, and provide practice. Then, when students learn, they can perform on the test, meet the objective, and perform as they must in the real world. The diagram below, shows how it all flows together:

To focus the training program on the required task performance measurements, the ISD model is based on the following development order:

1. Analyze the task to determine the objective.
2. Develop the learning objective fully and determine if it has any enabling objectives. If it does, then spell them out.
3. List the steps required to perform the objective to standards.
4. Build a test instrument to determine if the learner can perform the steps that are required to reach the objective.
5. Construct courseware that will train the learners to perform the objective. You know the learners can perform the objective if they can meet the evaluation standards.
   

Testing Terms

• Test or Test Instrument: (Brown, 1971) A systematic procedure for measuring a sample of an individual's behavior, such as multiple-choice, performance test, etc.
• Evaluation: (Wolansky, 1985) A systematic process for the collection and use of information from many sources to be applied in interpreting the results and in making value judgments and decisions. This collection of results or scores is normally used in the final analysis of whether a learner passes or fails. In a short course the evaluation could consist of one test, while in a larger course the evaluation could consist of dozens of tests. The process of determining the value and effectiveness of a learning program, module, and course
• Measurement: (Wolansky, 1985) The process employed to obtain a quantified representation of the degree to which a learner reflects a trait or behavior. This is one of the many scores that an individual may achieve on a test. An evaluator is most interested in the gap between a learner's score and the maximum score possible. If the testing instrument is true, then this is the area that the learner did not master

• Criterion Referenced Test: Evaluates the cognitive domain which includes the recall or recognition of specific facts, procedural patterns, and concepts that serve in the development of intellectual abilities and skills. The testing of these abilities and skills are often measured with a written test or a performance test. Note: A criterion referenced evaluation focuses on how well a learner is performing in terms of a known standard or criterion. This differs from a norm referenced evaluation which focuses on how well a learner performs in comparison with other learners or peers.

• Performance Test: Evaluates the psychomotor domain which involves physical movement, coordination, and use of the motor-skill areas. Measured in terms of speed, precision, distance, procedures, or techniques in execution. Can also be used to evaluate the cognitive domain. A performance test is also a criterion referenced test if it measures against a set standard or criterion. A performance test that evaluates to see who can perform a task the quickest would be a norm referenced performance test.

• Attitude Survey: Evaluates the affective domain which addresses the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes. Attitudes are not observable; therefore a representative behavior must be measured. For example, we cannot tell if a worker is well motivated by looking at her or testing her. But we can observe some representative behaviors, such as being on time, working well with others, performing tasks in an excellent manner, etc.

If a performance test is not possible, then a criterion referenced written test should be used to measure the learners' achievements against the objectives. The test items should determine the learner's acquisition of the KSAs required to perform the task. Since a written measuring device samples only a portion of the population of behaviors, the sample must be representative of the behaviors associated with the task. Since it must be representative, it must also be comprehensive.

Written Tests

1. Open-ended question: This is a question with an unlimited answer. The question is followed by an ample blank space for the response.
2. Checklist: This question lists items and directs the learner to check those that apply to the situation.
3. Two-Way question: This type of question has alternate responses, such as yes/no or true/false.
4. Multiple-Choice question: this gives several choices, and the learner is asked to select the most correct one.
5. Ranking Scales: This type of question requires the learner to rank a list of items.
6. Essay: Requires an answer in a sentence, paragraph, or short composition. The chief criticism leveled at essay questions is of the wide variance in which instructors grade these. A chief criticism of the other types of questions (multiple choice, true/false, etc.) is that they emphasize isolated bits of information and thus measure a learner's ability to recognize the right answer, but not the ability to recall or reproduce the right answer. In spite of this criticism, learners who score high on these types of questions also do well on essay examinations. Thus the two kinds of tests appear to measure the same type of competencies.

Multiple Choice

When writing multiple-choice questions follow these points to build a well constructed test instrument:

• The stem should present the problem clearly
• Only one correct answer should be included.
• Distracters should be plausible
• 'All the above' should be used sparingly. If used, an equal number of 'All the above' should be correct and incorrect (distracters). Do not use 'None of the above'.
• Each item should test one central idea or principle. This enables the learner to fully concentrate on answering the question instead of dissecting the question. It also allows the instructor to determine exactly which principles were not comprehended by the learner
• The distracters and answer for a question should be listed in series. That is, high to low, low to high, alphabetical, longest to shortest, like vs. unlike, function, etc
• Often, test items can be improved by modifying the stem. In the two examples below, the stem has been modified to eliminate duplicate words in the distracters. This makes the question easier to read.
  

Better example:

The distracters should be believable and in sequence:
Poor example:

In the above example, all the distracters were simply chosen at random. A better example with believable distracters and numbers in sequence would be:

If an item analysis (see chapter 6) is performed on the above example, we might discover that none of the learners choose the first distracter, (a). In our search for a better distracter the trainer informs us that some of the learners are entering the class with the myth that the mean is found by using the incorrect formula shown on the left below, instead of the correct formula shown to its right:

That is, they are adding a 1 to the total number of scores. We could change the first distracter (a) as follows:

Although a new item analysis might show that the learners are not choosing the new distracter, because the myth is adequately being dispelled by the trainer, it could still be left in as a distracter to let the instructor know if the myth is properly being dispelled. If a plausible distracter cannot be found, then go with a fewer number of distracters. Although four choices are considered the standard for multiple-choice questions as they only allow a 25% chance of the learner guessing the correct answer, go with three if another believable distracter cannot be constructed. A distracter should never be used just to provide four choices as it wastes the learner's time reading through the possible choices.

Also, notice that the layout of the above example question makes an excellent score sheet for the instructor as it gives all the required information for a full review of the evaluation.

True and False

True and false questions are constructed as follows:

Question 1 is false as there should be approximately an equal number of true and false items. Question 2 is true for any type of question. Other pointers when using True and False tests are:

1. Use definite and precise meanings in the statements.
2. Do not lift statements directly from books or notes.
3. Distribute the true and false statements randomly in the test instrument.
   

Open Ended Questions

The blank should be placed near the end of the sentence:
Poor example:

Better example:

Placing the blank at or near the end of a statement allows the learner to concentrate on the intent of the statement. Also, the overuse of blanks tends to create ambiguity. For example:
Poor example:

Better example:

Performance Tests

There are three critical factors in a well conceived performance test:

• The learner must know what behaviors (actions) are required in order to pass the test. This is accomplished by providing adequate practice and coaching sessions throughout the learning sessions. Prior to the performance evaluation, the steps required for a successful completion of the test must be understood by the learner.

• The necessary equipment and scenario must be ready and in good working condition prior to the test. This is accomplished by prior planning and a commitment by the leaders of the organization to provide the necessary resources.

• The evaluator must know what behaviors are to be looked for and how they are rated. The evaluator must know each step of the task to look for and the parameters for the successful completion of each step

A survey example can be see at Job Survey.

For example, an instructional program might instruct several advanced uses of a piece of diagnostic equipment. The instructional plan is initially based on the assumption that the learners have mastered the basic diagnostic procedures for the equipment from prior experience or training. These basic diagnostic procedures should be tested on the proposed learners to validate the instructional plan assumption. If they have not mastered one or more of the basic diagnostic procedures, then these unmastered procedures will have to be accounted for in the training plan.

Once you have tested their present KSAs, then the tasks to be taught should be tested on a small sample of personnel who have mastered the tasks previously to ensure the proposed test is correct. Finally, a sample of the proposed learners are tested to see if they can pass any portions of the test without any instruction.

Photo by catbagan Good trainers, like good producers, lay out their work in a logical sequence so that the audience (learners) can easily follow it.

Proper sequencing also helps to avoid inconsistencies in the content of the instruction. When material is carefully sequenced, duplication is far less likely. Indeed, the presence of duplication often indicates that the program has not been properly sequenced.

Some of the techniques and considerations used in sequencing are:

• Job Performance Order: The learning sequence is the same as the job sequence
• From Simple to Complex: Objectives may be sequenced in terms of increasing complexity
• Critical Sequence: Objects are ordered in terms of their relative importance
• Known to Unknown: Familiar topics are considered before unfamiliar ones
• Dependent Relationship: Mastery of one objective requires prior mastery of another
• Supportive relationship: Transfer of learning takes place from one objective to another, usually because common elements are included in each objective. These should be placed as close together as possible so that the maximum transfer of learning can take place
• Cause to Effect: Objectives are sequenced from cause to effect

If the training program is long, then reinforcement also has to be accounted for. One of the behavioral characteristics of learners indicates that not only the rate of which people learn must be accounted for, but also the rate of decay that takes place after an objective is mastered must also be accounted for. To account for this decay factor, reinforcement loops must be built into the instructional process. The decay factor also has to be considered once the learner graduates from the program. If a task is taught in the instructional program and then is not used for some time after the learners return to their duties, then some decay is likely to take place. The remedy for this is to coordinate with the learner's supervisor to ensure the learners perform their newly acquired skills as soon as possible upon returning to the job.

In any instructional program, there is usually a wide variety of abilities among the learners. Some will have extensive experience, while others are somewhat limited. The educational background may extend from high school dropout to college graduate. Many other variables will affect the progression and productivity of the learners. Provisions must be made to compensate for these differences. In a self-paced course, extra modules can help the learners that are having difficulties. In a lock-step course, additional instruction, reading assignments, or study halls may be required to keep the slower learners on pace with the other learners.

The product of the sequencing step should be a learning map which shows the proposed layout of the objectives. An example is shown below.

Click chart to enlarge

References

Krathwohl, David R., Bengamin S. Bloom, and Bertram B. Mesia (1964). Taxonomy of Educational Objectives (two vols: The Affective Domain & The Cognitive Domain). New York: David McKay. Return

Mager, Robert, F. (1975). Preparing Instructional Objectives (2nd ed.). Belmont, CA: Fearon. Return

Wolansky, William, D. (1985). Evaluating Student Performance in Vocational Education. Ames, Iowa: Iowa State University Press. Return