Techniques available for data collection
Survey
You may use a survey if the practice involves large numbers of practitioners and the process is complex. Surveys are more suited for situations where anonymity is preferred. They can be used to explore intentions, attitudes, knowledge and practice behaviour, as well as barriers to change and strategies to overcome barriers.
Interview
Interviews allow a topic to be approached from the point of view of participants and where anonymity is not a priority. The advantage of interviews is the possibility of going deeper into the topic without the need for structured questions in advance. Suitable for exploratory projects where the questions can be adjusted according to how the interview flows. They can be used to explore intentions, attitudes, knowledge and practice behaviours, as well as barriers to change as strategies to overcome those barriers.
Focus group
Focus groups can be helpful if the group is homogenous and participants are willing to talk openly about the topic. Figures of authority can however influence the results, so the choice of the participants in the focus group needs careful consideration.
Observation
Observation can be used to count how often practice activities are being delivered, and quantify the way things are done.
Chart or file audit
A clinical audit is a detailed review of selected clinical records. An audit is usually completed by health professionals in order to help improve the quality of patient care and outcomes. Originally referred to as medical audit, the concept of a clinical audit is regarded as either uni- or multi professional according to how many professions are involved. An audit seeks to determine how much of a procedure or treatment is being delivered as recommended by a clinical guideline. Information sought includes the percentage of clients screened/receiving an intervention, the number of sessions provided per client, and the content of sessions.
Types of audit
There have been three approaches to audit:
- Generic audit, measuring overall quality on a unit or ward
- Problem specific audit, measuring quality related to a clinical topic
- Activity specific audit, measuring quality of care provided by a person or group of people
Problem specific and activity specific audits may be more suitable for the purpose of describing and quantifying evidence-practice gaps.
Advantage and disadvantages of audits
The following advantages of clinical audits were described by Holmboe and Hawkins, 2009.
1. Availability: Getting to the records is usually not a major problem but pulling out specific aspects of care may be a challenge.
2. Feedback: Allow for corrective feedback centred on actual clinical care in a timely manner
3. Changing clinical behaviour: Once a practice gap is discovered this would encourage introducing changes in the practice
4. Practicality: Audits allow for a targeted or random selection of patients to be surveyed
5. Evaluation of clinical reasoning: Depending on the quality of the documentation, evaluation of skills in analysis, interpretation and management is possible. Evaluation of particular groups or treatments, conditions is also possible.
6. Reliability and validity: Since audits use explicit criteria, a high degree of reliability is possible
7. Learning and evaluating by doing: The audit is done with the active participation of the practitioners involved or practice; this encourages the constant improvement of the services
8. Self-assessment and reflection: Comparing the practice to benchmarks allows practitioners to be their own assessors and to reflect in their own practice. The practitioner must be prepared to assess his/her own performance.
Potential disadvantages of clinical audits
The following potential disadvantages need to be taken into consideration in order to achieve a balance (Holmboe and Hawkins, 2009).
1. Quality of the documentation: The quality of the audit can be only as good as the quality of the documentation. Was all the pertinent information collected?
2. Process versus outcomes: The utility to using the clinical record audit to determine causation for patient outcome is limited
3. Implicit review: Reviewing a clinical record without a minimal framework, structure or especially well defined criteria results in low reliability and reduced validity.
4. Assessment of clinical judgment: Was the judgement of the clinician properly recorded? Did the judgement translate into the appropriate management plan?
5. Time: Can be very time consuming, in addition to the normal tasks of running the practice.
6. Cost: Consider the costs, (audit may not be billable time).
Audit resources are available on the Resources page
References
Addis, M.E. & Krasnow, A.D. (2000). A national survey of practicing psychologists' attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology, 68(2),331-339.
Cabana, M.D., Rand, S., Powe, N.R., Wu, A.W., Wilson, M.H., Abboud, P.A., Rubin, H.R. (1999). Why don't physicians follow clinical practice guidelines? A framework for improvement. Journal of the American Medical Association, 282(15),1458-1465.
Holmboe, E. S. Practice Audit, Medical Record Review, and Chart-Stimulated Recall. In Holmboe E. S. and Hawkins R. E. (eds): Practical Guide to the Evaluation of Clinical Competence. Philadelphia, USA: Mosby Elsevier, 2008, pp 60-74.
Kramer, T.L. & Burns, B.J. (2008). Implementing Cognitive Behavioural Therapy in the real world: A case study of two mental health centers. Implementation Science, 3:14.