Testing Process

Test development is a cyclical process. Although students in your classes change from semester to semester, assessment is ongoing.

At the end of this activity, you will be able use a cyclical test development process to improve your assessments.

Let's begin with the first step - creating a test blueprint.

Test Blueprint

The test blueprint identifies 1) the objectives and skills which are to be tested and 2) the relative weight on the test given to each.

A blueprint provides clear direction for you as you build/write your test. Tests constructed without this type of plan often result in certain objectives being over-represented and others under-represented.

An example of a basic nursing blueprint:

Patient Safety 50%
Documentation 25%
Critical Thinking 15%
Communication 10%

With your blueprint in hand, the next step is to write the test items.

Item Development

The term "item" is used as a shorthand for questions on a test. Item development can proceed only when clearly defined objectives are available.

Ideally, each item should measure only a single objective. Each objective, however, could be measured by one or several items, depending on the test blue print.

For Example:

Objective: Distinguish between reliability and validity of measurement
Two items that measure this learning objective:

  1. T F The validity of an instrument is the extent to which it measures the target phenomenon.
  2. When a researcher performs a test-retest by administering the same instrument to the same subjects under similar conditions on two or more occasions, they are evaluating the instrument’s
    1. validity
    2. reliability
    3. semantic differential
    4. obtrusivene

Next we'll give you some helpful tips that you can use during item development.

Dos and don'ts for developing items and assessments

Do:

Don't:

Of course, once you've written the test, you will administer and score it.

Administration and Scoring

Administering your tests isn't complicated. However, the atmosphere you create in the test room and the attitude you display at test time is extremely important. In fact, your manner, bearing, and attitude may well inspire confidence in students and put them at ease while participating in the testing process.

In general, scoring fixed-response items, such as multiple-choice items, can be done faster and less expensively than scoring free-response items such as fill-in-the-blanks, short answer or essay items.

To maintain consistency and fairness when scoring short answer and essay questions, it is important to use a detailed scoring rubric that identifies the basis for awarding or subtracting points. This is especially important if you have multiple graders.

The data you receive from administering and scoring should be used to evaluate your items.

Item Evaluation

After administering a test, you will have a wealth of information about how students performed on each item. The data will inform you about the appropriateness of the exam content and the effectiveness of the individual items in future exams.

After analyzing how well your test worked you may discover that you need to revise your test blueprint, some of the items, or both.

As you can see, it is important to think of the testing process as cyclical rather than linear.

Test blueprints and items need not remain static. Pedagogy is not static and the blueprints for each test need to be reviewed and modified to reflect the current state of knowledge.

Implementing this strategy benefits both you and your students because you will be developing higher quality assessments.

References:

Case, S. M. & Swanson, D. B. (2003). Constructing written test questions for the basic and clinical sciences, 3rd edition (revised). Philadelphia: National Board of Medical Examiners. http://www.nbme.org/about/itemwriting.asp

Cohen, A. S. & Wollack, J. A. Handbook on test development: Helpful tips for creating reliable and valid classroom tests. Testing & Evaluation Services, University of Wisconsin-Madison. http://wiscinfo.doit.wisc.edu/exams/instructional_support.htm

Park University, Faculty Development web site, Writing Test Items. http://captain.park.edu/facultydevelopment/writing_test_items.htm

United States Office of Personnel Management (2003). Delegated Examining Operations Handbook. http://www.opm.gov/deu/Handbook_2003/DEOH-Administ.asp

 

Research: Brenda Kupsch

Design: Eileen Horn

Development funded through a grant from the Health Resources and Service Administration (1D09HP03288-01-00).