Morag Harris outlines some basic premises from the literature that can improve supervision evaluation practices, and thus reduce the ambiguity and uncertainty about evaluation in supervision.
Professional competence evaluation is made in a series of formal and informal measurements that result in a judgment that an "individual is fit to practice a profession autonomously" (McGaghie, 1991). Summative evaluation describes "how effective or ineffective, how adequate or inadequate, how good or bad, how valuable or invaluable, and how appropriate or inappropriate" the supervisee is "in terms of the perceptions of the individual who makes use of the information provided by the evaluator" (Isaac 8 Mitchell, 1981, p. 2). Counsellor supervisors are responsible for summative evaluations and assessments of supervisee competence to university departments, state licensing boards, and agency administrators. Summative evaluation is described by Bernard and Goodyear (1992) as "the moment of truth when the supervisor steps back, takes stock, and decides how the trainee measures up" (p. 105).
Effective summative evaluation requires clearly stated performance objectives that can be assessed in both quantitative and qualitative terms and that have been made explicit to the supervisee or trainee during initial supervision contacts. The heart of counsellor evaluation, however, is an on-going formative process which uses feedback and leads to trainee skills improvement and positive client outcome. In this case the trainee is the person using the information. Bernard and Goodyear (1992) refer to this kind of evaluation as "a constant variable in supervision." As a result, every supervision session will contain either an overt or covert formative evaluation component.
Evaluation practices and procedures
When supervisors measure behavioural therapeutic skills they find several difficult areas. First, they find that measurement and subsequent evaluation of therapeutic skill is a complex process in a field where many skills inventories and behavioural checklists abound, and research findings suggest that these may lack adequate reliability and validity. Second, in teaching settings supervisors recognize the tension between providing a supportive facilitative environment within which counsellors-in-training can feel free to stretch and learn counselling skills and the anxiety that results from academic grades. Third, lacking a theory of supervision, supervisors are unable to articulate desired outcomes for their supervisees and may revert to the evaluation of administrative detail and case management.
As a result of these difficulties, numerous areas of competency may be neglected, anxiety may persist, and supervisors may resort to summative evaluation practices in global and poorly measured terms.
There are resources which outline required skills and knowledge for effective evaluation practices. The Curriculum Guide for Training Counsellor Supervisors (Borders et al., 1991) provides specific learning objectives for supervisors in training. Other current publications (Bernard & Goodyear, 1992; Borders & Leddick, 1987; McGaghie, 1991; Stoltenberg & Delworth, 1987) further develop the Guide’s "three curriculum threads" (p.60) of self awareness, theoretical and conceptual knowledge, and skills and techniques.
The guidelines and suggestions from these resources are summarized in the following list of effective evaluation practices:
1. Clearly communicate evaluation criteria to supervisees and develop a mutually agreed upon written contract reflecting these criteria.
2. Identify and communicate supervisee strengths and weaknesses. The Ethical Guidelines for Counsellor Supervisors (ACES, 1993) recommend that supervisors "provide supervisees with ongoing feedback on their performance." This performance feedback establishes for supervisees a clear sense of what they do well and which skills need to be developed. Supervisee strengths and weaknesses can be evaluated in terms of process, conceptual, personal, and professional skills (Bernard R Goodyear, 1992, p. 42).
3. Use constructive feedback techniques during evaluations. Supervisees are more likely to "hear" corrective feedback messages when these are preceded by positive feedback, focussed on observable behaviours, and are delayed until a positive relationship has been established.
4. Use specific, behavioural, observable feedback dealing with counselling skills and techniques; avoid terms such as "understanding," "knowing" and "appreciating," and "being aware of." Successful evaluation practices should include behaviourally-based learning objectives (Gage & Berliner, 1984).
5. Use Interpersonal Process Recall (IPR) to raise supervisees” awareness about their personal developmental issues. The unobtrusive and nonthreatening nature of IPR is particularly helpful as supervisees retrospectively explore their thoughts, feelings, and a variety of client stimuli during counselling sessions. This process can assist supervisees in contributing to, and benefiting from, formative evaluation.
6. Employ multiple measures of supervisee counselling skills. These can include a variety of standardized rating scales including measures completed by both supervisor and supervisee, client ratings, and behavioural scales (Stoltenberg & Delworth, 1987). Additional measures such as work samples from audio/videos, critiques of counselling sessions, and conceptual case studies (both brief and detailed) can provide a comprehensive picture of a supervisee’s competency, expectations, needs and professional development, as well as an understanding of the context within which both the counselling and the supervision take place.
7. Maintain a series of work samples in a portfolio for summative evaluation. Since the evaluation of only one session provides an inadequate assessment of supervisee competency, and the selective nature of work samples may prove to he an overly negative reflection of current competency level, the portfolio provides both the supervisor and the supervisee with a more comprehensive and useful basis for a summative evaluation.
8. Use a developmental approach which emphasizes both progressive growth toward desired goals and the learning readiness of the trainee (Nance, 1990). The Nance model emphasizes a learning readiness based on the supervisee’s ability, confidence, and willingness “the assessment of which directs the roles and practices of the supervisor. As a result, supervisors can "match" their supervisee’s level and "move" them toward independent functioning one step at a time. Although Nance does not specify evaluation practices, he clearly describes effective supervisory styles, interventions, role, contracts, and agendas for each developmental stage. These variables can guide the evaluation process indirectly by enabling the supervisor to understand the characteristics and appropriate expectations for supervisees at each developmental level.
A structured approach to supervisee assessment and evaluation produces several beneficial outcomes.
First, supervisors can reduce their own, as well as their supervisee–s, anxiety about the process. The meanings associated with assessment can be altered to suggest a positive experience from which both partners can grow and learn.
Second, supervisors who articulate their adopted "supervision theory" to their supervisees will also clarify their evaluation criteria as well as their supervision practices.
Third, when evaluation is viewed as a process of formative and summative "assessment" of the skills, techniques, and developmental stage of the supervisee, both supervisees and their clients benefit.
Fourth, as supervisors deal successfully with the process of supervisee evaluation, they also bring similar skills to the evaluation of their training programs, an area in search of an appropriate evaluation paradigm.
Finally, just as training is most successful when multiple methods (didactic, modelling, and experiential) of skills acquisition are employed, so too the use of multiple methods for evaluation contributes to the supervisee’s sense of self-worth and success.
Association for Counsellor Education and Supervision. Ethical Guidelines for Counselling Supervisors (1993). ACES Spectrum, 53(4), 5-8.
Bernard, J. M., 8c Goodyear, R. K. (1992). Fundamentals of clinical supervision. Boston: Allyn & Bacon.
Borders, L. D., Bernard, J. M., Dye, H. A., Fong, M. L., Henderson, P., 8 Nance, D. (1991). Curriculum guide for training counsellor supervisors: Rationale, development, and implementation. Counsellor Education and Supervision, 31, 58-77.
Borders, L. D., & Leddick, G. R. (1987). Handbook of counselling supervision. Alexandria, VA: Association for Counsellor Education and Supervision.
Gage, N. L., & Berliner, D. C. (1984). Educational psychology (3rd ed.). Boston: Houghton Mifflin.
Isaacs, S., & Michael, W. B. (1981). Handbook in research and evaluation (2nd ed.). San Diego: EdITS.
McGaghie, W. C. (1991). Professional competence evaluation. Educational Researcher, 20, 3-9.
Nance, D. W. (1990). ACES Workshop on Counsellor Supervision. Workshop presented at the annual convention of American Association for Counselling and Development, Cincinnati, Okl.
Stoltenberg, C. D., & Delworth, U. (1987). Supervising counsellors and therapists: A developmental approach. San Francisco: Jossey-Bass.
Acknowledgements: ERIC Digest