Gathering Fidelity Information and Making Fidelity Ratings

Independent Ratings
Two or More Assessors
Site Visits
Multiple Data Sources
Multiple Observation Points

Transcript

To obtain the data necessary to rate a fidelity scale, the most rigorous approach involves independent assessors who are knowledgeable about the evidence-based practice and trained in interviewing techniques. Two fidelity assessors are recommended. The data is best collected during a day-long site visit. Prior to the visit, the fidelity assessors should:
Identify a contact person at site.
Provide a rationale for the activity.
Explain who will receive the assessment report.
Schedule a date that suits the host’s schedule and deadlines.
Finalize the fidelity assessors’ schedule before the visit.

During the visit, the assessors should gather data from multiple data sources, using strategies known to improve the accuracy of data. The assessors should listen carefully, use respondent’s terminology, avoid giving away the “right” answer, and ask for specific data, not for general opinions.

The accuracy of information improves when fidelity assessors have opportunities to observe a site more than once. Although the most objective fidelity assessment is one conducted by independent assessors, internal fidelity assessments conducted by program staff are also recommended as part of quality improvement efforts. The one caveat with internal fidelity assessments is the reminder that program staff need to be objective in making ratings, even if the ratings make them “look bad.”

During the site visit, the fidelity assessors collect all the data necessary to make the ratings, but usually only some of the ratings are made on the spot. Sometimes the ratings require calculations that are completed after the data collection. An example would be the frequency of contacts as abstracted from the charts.

It is very important that the fidelity scale ratings be made soon after the fidelity visit, ideally within one day. This is to enhance the recall of information while it is fresh in the assessors’ mind. In addition, the ratings should be made independently by each assessor. Soon after the ratings are made, the assessors should meet to discuss their ratings. They should reach consensus on any discrepancies, citing data to resolve discrepancies. If assessors realize that they need more information to make a rating, then they should follow up with a call or email to the program site.