Steps in Constructing a Fidelity Scale

1. Identify the principle.
2. Develop items.
3. Create response categories.
4. Create protocols.
5. Validate the scale
   

Transcript

There are four standard steps used in the development of the Supported Employment Fidelity Scale. These steps were used to develop fidelity scales for all 6 of the “evidence-based practices” that were identified by a national panel of experts in 1998 to evaluate services for people with severe mental illness. Developing a fidelity scale is complicated! Let’s look briefly at these steps

Step 1: Identify the Principles of the Evidence-based Practices.

These principles should be found in major review articles, such as a series on evidence-based practices published in Psychiatric Services in 2001-2002. The principles for supported employment have been summarized in one of these Psychiatric Services articles (Bond and others, 2001) and have been reviewed earlier in this lecture.

Step 2: Develop Items for Each Principle.

The task here is to devise an operational definition for the principle. How would we know the principle if we saw it? This step include activities found in the development of any scale. Items should be worded clearly with objective criteria. There should be at least one fidelity item for each principle. We recommend that a fidelity scale consist of 15 –30 items.

Step 3: Create Response Categories for Each Item.

The Supported Employment Fidelity Scale uses a 5-point response scale, with 5 = full implementation and 1 = not implemented at all. There are intermediate points reflecting gradations between full implementation and an absence of implementation. In some cases, 1 represents usual practice. So, for example, many traditional mental health and rehabilitation services involve large caseloads, so that scale point 1 might be a caseload of 80 or more clients. Each of the response alternatives need to be behaviorally anchored with objective criteria if the scale is to be reliably rated.

Step 4: Create Protocol to Use Fidelity Scale and Validate the Scale.

A protocol refers to a set of instructions for using the scale, including a description of the scale For instance, this may include what programs it can be used for, who can use it, how much time it takes, how to administer the scale, who to interview, what you should observe, what questions to ask, and how to score the scale. Before a scale is ready for dissemination, it should be tested for its usability, reliability, and practicality.