Safer Surgery part 21 docx

This page has been left blank intentionally
Chapter 11
Integrating Non-Technical Skills into
Anaesthetists’ Workplace-based Assessment
Tools
Ronnie Glavin and Rona Patey
Introduction
In this chapter, the authors will provide a brief account of the development of the
Anaesthetists’ Non-Technical Skills (ANTS) system and then consider how this
system may nd a suitable home in the UK anaesthetic curriculum (Anaesthetists’
Non-Technical Skills System Handbook).
But rst, what are non-technical skills (NTS)? Within the context of anaesthesia
they have been dened as behaviours in the operating theatre environment, which
are not directly related to the use of medical expertise, drugs or equipment (Fletcher
et al. 2003). They encompass both interpersonal skills (e.g., communication, team
working and leadership) and also cognitive skills (e.g., situation awareness and
decision making) (Anaesthetists’ Non-Technical Skills System Handbook). NTS
are important because we need them to function effectively in our professional
lives. As with many important aspects of living, we really only notice them when
they are absent or poorly performed. Then, our practice is not so effective and
may become dangerous. In anaesthesia, and many other professions, studies of
adverse incidents consistently suggest that the underlying factors are more likely
to be due to difculties with the application of NTS rather than a lack of basic
knowledge or practical skill (Flin et al. 2008). Is it then reasonable to ask: ‘If NTS
are so important why have they not featured prominently in anaesthetic training?’
One might reply that they might have done had we known what they were exactly
in the context of anaesthesia. Until now NTS have been elusive and difcult to
describe or pin down. This inability to accurately dene or describe NTS has
resulted in them generally only being considered under a broad umbrella category
of ‘soft skills’. However, there is nothing soft about these skills. The platform they
provide, along with the knowledge and skills from more conventional areas of the

curriculum, is the solid base required for the development of anaesthetic practice.
So far, we have referred to NTS as if they were a generic set of skills found
across many circumstances and professions. Bob Helmreich, emeritus psychologist
at University of Texas, Austin, played a key role in the early NASA work in this
eld of key human factors and then oversaw the development of the concept from
Safer Surgery
176
infancy to maturity in other industries. He reminds us that there is no generic set
of NTS and has warned that different organizations/professions and cultures have
their own variations of NTS (Helmreich 2000). Therefore, whilst NTS identied
in other professions can give us a avour of those used in anaesthesia, we cannot
just assume that they cross directly into our practice. With this in mind, in 1999,
having conrmed that such work was not already under way, a research team
comprising clinical anaesthetists (including both authors) and psychologists set
about the business of identifying the non-technical skills important for anaesthetic
practice (funded by NHS Education for Scotland).
Development of the Anaesthetists’ Non-Technical Skills System
How did we do this? We believed that anaesthetists had always learned and used
NTS but that this was largely an unconscious process. Therefore, we could not
simply ask anaesthetists ‘What non-technical skills are important in your practice?’
Nor would the quantitative or positivistic approach, which was the research tradition
that the clinical anaesthetists in the team were familiar with, be appropriate. This
was not an investigation where quantitative measurement, such as where a new
technique is developed to impact on blood pressure or cardiac output, would help.
A different approach was required. A set of task analysis techniques was used with
questionnaires, observations and interviews. For the last, a form of cognitive task
analysis was employed (Crandall et al. 2006, Fletcher et al. 2004). This is a qualitative
research method whereby the researcher uses a set of probes, usually in the form of
questions, to help the interviewee (in this case consultant anaesthetists) describe one
or more episodes of clinical practice. The interviews contained the raw data and

were analysed by extracting all the behaviours and skills that were mentioned. These
approaches allowed access to behaviours which may not otherwise have impinged
upon the consciousness of the participants. These elicited behaviours do not come
neatly ordered, and so for clarity and ease of use, a system has to be assembled
from the data. For this, a series of iterative steps took place where the research
team developed a prototype categorization of the principal behaviours and took
rudimentary versions of the system into the workplace and studied their apparent
effectiveness. After several iterations the current Anaesthetists’ Non-Technical Skills
(ANTS) system was arrived at (see Table 11.1).
The next step was to evaluate some key properties of this system. Was it valid?
Could it be used in a reliable manner – would different assessors give similar
scores to the same performance? At an even more fundamental level, could clinical
anaesthetists actually use the system? To address these questions, 50 consultant
anaesthetists across Scotland took part in a study. They attended a one-day session
comprising a morning of teaching and familiarization with the concepts behind
the system and the system itself followed by an afternoon of applying the system
to eight video-recorded anaesthetic scenarios that had been conceived, performed
and produced by the research team. The ndings have been published elsewhere
Non-Technical Skills and Anaesthetists’ Workplace-based Assessment Tools
177
(Fletcher et al. 2003), but the main conclusions were that the clinical anaesthetists
could use the system and could provide broad consensus on the scores awarded,
and that the system captured the NTS observed. However, these anaesthetists
were scoring performances while observing them on videotape, without any
other distractions. It was important to assess if the system could be used in the
workplace. Here, if trainers were to apply the system to assess the performance of
a trainee in order to provide feedback, they would also be required to continue to
monitor the patient and the operating theatre environment.
Of the original volunteers, 17 participated in the usability study. The consultant
anaesthetists and the trainees being supervised were asked to complete questionnaires

at the end of the theatre session. The results of this usability study, which are
detailed elsewhere, were positive (Patey et al. 2005). It seemed that the system
could be used in the workplace and furthermore participating trainers and trainees
responded in a very positive manner to use of the system as a way of providing
formative assessment and teaching of NTS. Was the anaesthetic community in the
UK now ready to adopt the ANTS system into routine training?
Early Experiences of Promoting the ANTS System
The body responsible for standards of anaesthetic practice and training of
anaesthetists in the UK is the Royal College of Anaesthetists. Presentations on the
ANTS system to the relevant committees and bodies of stakeholders produced a
very positive response. Following this, the next hurdle was how to best assimilate
Category Element
Task management
Planning and preparing
Prioritizing
Providing and maintaining standards
Identifying and utilizing resources
Team working
Coordinating activities with team members
Exchanging information
U
sing authority and assertiveness
Assessing capabilities
Supporting others
Situation awareness
G
athering information
Recognizing and understanding
A
nticipating

Decision-making
Identifying options
Balancing risks and selecting options
Re-evaluating
Table 11.1 The ANTS system: categories and elements
Safer Surgery
178
the ANTS system into the teaching and assessment of anaesthetists. The usability
study had shown that individual consultant anaesthetists could use the system,
but could the ANTS system be incorporated into the teaching practice of a whole
hospital department of anaesthesia?
The proposed strategy for introduction to a department was to use a cascade
system for training the trainers. One of the authors of this chapter would visit
several UK anaesthetic departments that had volunteered to trial the use of the
ANTS system and deliver a one-day teaching session for local enthusiasts. The
cascade approach was based on the notion that those who had attended the one-
day course would use the system, become more familiar with its application and
then reach a stage where they would be comfortable to instruct their consultant
colleagues. Similar to the initial system evaluation study, the training day would
consist of a morning going over the background to NTS, followed by familiarization
with the ANTS system. In the afternoon, the performance of anaesthetists in
video-recorded scripted scenarios would be assessed allowing practice using the
system. The afternoon differed from the research work in that the participants
were encouraged to discuss their scores to allow calibration to take place. When
the participants reached consensus and awarded very similar scores then that part
of the session ended. Would this cascade approach work?
Although NTS could be identied and discussed by consultants during ANTS
training sessions, on the whole the cascade approach was not successful. It seems
from discussion with consultants that the reasons for this were multifactorial
(Dodd 2005). Firstly, most consultants only had limited familiarity with NTS

and experience with the ANTS taxonomy (only the single day of training and
some recommended reading) and wanted further opportunities and support to
become more familiar with the tool before using it in the workplace. Accurately
observing behaviours is a skill, like any other, which requires deliberate practice,
and although consultants regularly participated in clinical training they had
not received training in behavioural observation. Where the consultants had no
formal training or experience in providing structured feedback (which was often
the case), they were particularly uncomfortable. In all probability, too many new
changes were introduced at one time but it is worth exploring the intended training
approach and barriers to its success in greater detail.
A conventional approach to instructing anaesthetic trainees in the use of the NTS
highlighted in the ANTS system would be to incorporate the underlying theoretical
concepts into classroom-style teaching and then build on that base during clinical
training by demonstrating application of those principles. Knowledge and practice
would then be formally assessed in some kind of examination. We were adopting
an entirely different approach, where the system was being used to score the
performance of a trainee anaesthetist in the workplace. The scores awarded for the
relevant categories and elements were intended to form the basis for a discussion
between consultant and trainee as a way of drawing attention to the trainees’ existing
strengths and limitations in the use of NTS and then plan for further development. We
had made the assumption that consultant anaesthetists were used to assessing trainees
Non-Technical Skills and Anaesthetists’ Workplace-based Assessment Tools
179
on accompanied theatre lists and then providing structured feedback. However,
although consultants had considerable experience assessing the performance of
trainees this was done in a very informal manner. Many consultants in the UK had
little experience in the use of formal workplace-based assessment tools. This was not
surprising because no such tools were in widespread use in the UK.
Workplace-Based Assessment and the Curriculum in Anaesthesia
One result of this limited use of workplace-based assessment tools is that little

attention has previously been paid to the requirements to support consultants in
providing structured feedback in the workplace. Consultant training, to highlight
the essential role of feedback in both optimal learning and in the use of effective
feedback techniques, has been limited. In addition, it can be difcult during a busy
working day for the consultant and trainee to identify time for feedback. If time
can be identied, perhaps at the end of the day, it may not be close to the event
under consideration, and at this time both parties may be concerned with other
duties such as post-operative or pre-operative visiting. The conditions during a
training session in operating theatres are also not ideal. The presence of the whole
theatre team can be inhibiting, particularly where there is discussion to be had on
an area where there are difculties. The authors have noted that this seems to cause
particular concern where NTS are concerned and consultants perceive that they
are commenting on personality-related matters. These various factors all conspire
to make it easy to delay a feedback discussion.
When time can be identied for feedback, use of the ANTS system should actually
reduce the anxiety of commenting on a trainee’s personality traits. The system allows
discussion and emphasis on specic NTS behaviours either observed or thought
to be decient or absent, rather than comments which are difcult to interpret or
may be interpreted as personality assessments. Discussion can then follow on how
these specic behaviours impact on the ANTS system elements and categories. The
consultant can then encourage the trainee to consider why these behaviours were
observed or not. However, as we have previously noted, providing feedback is not
the only skill in which consultant anaesthetists lack training. Accurately observing
behaviour is a skill in itself and requires deliberate practice for learning.
This situation, where there is limited use of formal workplace-based assessment,
is changing in the UK in response to the processes brought about under the umbrella
of Modernising Medical Careers (The Foundation Programme). The body with
regulatory powers for postgraduate medical training (the Postgraduate Medical and
Training Board, referred to as PMETB) with the right to approve curricula, including
assessment tools, is promoting workplace-based assessment tools as one method for

providing evidence that trainees are achieving the appropriate competencies for their
respective curricula (The Postgraduate Medical Education and Training Board).
The proposed workplace-based tools are being modied from tools introduced in
the UK in 2005 for use during the rst two years of postgraduate training (the UK
Safer Surgery
180
Foundation Training programme). These tools are the Mini Clinical Evaluation
Exercise (MiniCEX), the Directly Observed Procedural Skills (DOPS), the Case
Based Discussion (CBD) and a form of 360° feedback. The MiniCEX can deal
with many aspects of interaction with a patient including history taking, conducting
an examination or explaining benets and risks of procedures. The DOPS focuses
on procedural skills including the decision-making, preparation and set up of the
procedure. The CBD is intended to look at decision-making processes by referring
to information recorded by trainees in the patient’s case notes, while the 360° allows
peers and other colleagues from a range of professions to comment on professional
attitudes and teamworking skills. Specially modied versions of these tools are now
described for anaesthetic speciality training (The Royal College of Anaesthetists,
Workplace-based Assessment).
At the time of writing, there is a sense of concern in the anaesthetic community
with regard to these tools. Consultant anaesthetists have embraced the principles
behind workplace-based assessment. However, as we found when considering use
of the ANTS system, consultants have expressed concern about the level of their
training, support and preparation for using these tools. Moreover, for many who have
not yet considered NTS, these reactions have been focused on areas of anaesthetic
practice which have been traditionally assessed (technical skills). It is therefore
hardly surprising that asking consultants to conduct workplace-based assessments
of NTS, an area in which they had no previous experience, was a step too far. It is
also likely that the cascade approach was overambitious in that too much was asked
of too few people without ongoing support (Grant and Stanton 1999). Nevertheless,
despite the cascade approach having had limited success, the Royal College of

Anaesthetists has continued to support the concept of using the ANTS system and it
featured as one of the assessment tools submitted to PMETB for approval.
Using the ANTS System to Frame Teaching
However, the difculty with the cascade approach to incorporating NTS into
anaesthetic training was not just a problem of becoming familiar with observing
behaviours, using the ANTS system and considering how best to give feedback.
There were barriers because of the very nature of clinical training in the UK. In this
training system the trainee spends regular time providing anaesthetic care with a
trainer, gradually taking a more central role in the clinical work in a wider range of
clinical settings. This would seem to be ideal in that there are regular opportunities
for the trainer to observe the trainee’s practice and provide feedback. The method
of use of the ANTS system which was favoured by the 17 anaesthetists during the
usability study was to allocate the perioperative management of a particular patient
to the trainee accompanying them on the theatre list. They would then assess the
performance of the trainee and, using the observed behaviours, provide feedback
after the case. The consultant, of course, retained overall responsibility and could
step in and make changes if necessary but, in the absence of concerns about patient
Non-Technical Skills and Anaesthetists’ Workplace-based Assessment Tools
181
or trainee safety, would let the trainee handle the management of the patient. This
approach had some implications. The trainer does not only have the trainee to
observe but also the clinical work. As a senior trainee has more experience than
a junior trainee, the consultant is more likely to have condence that the trainee
will identify the key issues involved with the management of a wider range of
patients for either the whole anaesthetic management or specic parts of a case.
There are therefore more opportunities to delegate responsibility to a senior
trainee than a junior trainee. Consultant participants in the study reported that they
were better able to use the system with senior rather than junior trainees. There is
some apparent logic in this nding. If we want to look at ANTS categories such
as decision-making then it is necessary to let trainees make decisions, therefore

you have to delegate that responsibility. This message was also conveyed when
evaluating the cascade training.
One might therefore imagine that the system could be used satisfactorily with
senior trainees at least. However, frequently senior trainees are given responsibility
not only for individual patients but for a complete surgical list, either intentionally
to provide them with required clinical experience, or to cover for leave or sickness.
Although a consultant anaesthetist will have supervisory responsibility for that
trainee, they may be occupied elsewhere and unable to give continuous personal
supervision. These factors limit the operating lists where a consultant and a senior
trainee work together, and participants in the cascade training initiative listed this
as another factor contributing to their difculties in using the ANTS system. In
addition, often, no sooner had a trained consultant explained the system than the
trainee was re-allocated to another theatre list.
Given the various difculties which had been encountered with the cascade
approach, it was important to consider whether there were further approaches
which could be used in addition. As the clinical anaesthetists in the research team
continued to use the system in their own practice, some other uses began to become
apparent. It was possible to use the system as a framework on which to base both the
NTS elements and also the more conventional technical tasks during anaesthesia.
The reluctance of consultants to use the ANTS system with junior trainees was
partly based on a concern that anaesthetists in the early stages of training have
large areas of knowledge and practical skills to assimilate and struggle to cope
with an additional load of considering NTS. However, if the teacher or instructor
was using the ANTS system as a framework or mental model then he or she need
not communicate that framework to the learner. The consultant as instructor could
use the system to provide scaffolding for the trainee without this being visible to
the trainee. Some examples will help to clarify this process.
Let us begin with the category of ‘Situation Awareness’. One of the elements
in the ANTS system category Situation Awareness is gathering information. A
key technical skill to be mastered by novice anaesthetists is taking an anaesthetic

history from a patient. This skill resembles the history taking skills that medical
undergraduates have to master but there is sufcient difference to require specic
additional instruction. When not familiar with the ANTS system, the trainee will
Safer Surgery
182
view this skill as history taking, not as gathering information. Of course there is
more to gathering information for situation awareness than history taking from
the patient. Information about the intended surgical procedure is required to
formulate a suitable anaesthetic plan. Trainees are traditionally taught to ask for the
information about the proposed surgical procedure that they require, such as how
long will it take; what position will the patient be in; does the surgeon need access
to the abdominal, chest or pelvic cavities; what is the anticipated blood loss? This
information may be gathered from a variety of sources such as the surgeons and
the operating department nursing staff. The trainee is now exchanging information
and coordinating activities with team members, which are elements from the
category Team Working.
As a further example, during the course of anaesthesia there will be episodes
where the anaesthetic trainee needs to focus on surgical events, such as when the
surgeon makes the skin incision. At this time the anaesthetist should be observing
the response of the patient to conrm that the depth of anaesthesia is appropriate
for that patient. The relevant elements are gathering information and anticipation
from Situation Awareness, prioritizing from Task Management and exchanging
information from Team Working. Prioritization is included because the anaesthetist
should not be involved in other activities that are less essential (such as drawing up
antibiotic drugs). The anaesthetist should anticipate the skin incision by observing
the progress of the prepping of the surgical eld and the placing of sterile drapes
around the surgical eld. They should be alert for an exchange of information (a
surgeon with appropriate NTS will ask if it is all right to perform the incision and
the anaesthetist should be in a position to reply).
Taking this method further, the relevant NTS behaviours can be discussed during

procedural skills training and the key role of NTS for safety can be highlighted.
To illustrate this technique, consider the insertion of a central venous line. The
trainee is taught to attach cardiac monitoring so that they can be aware of arrhythmia
generation should the guide wire be passed too far. Of course this allows information
gathering to provide more accurate Situation Awareness of how the technical
procedure is proceeding. There can be discussion of where the monitor is best placed
for information gathering. As this is sometimes found to be out of clear sight for the
person inserting the cannula, the trainee may suggest that they would ask an assistant
(exchanging information) to watch the monitor during the insertion (coordinating
activities). It is not uncommon for trainee anaesthetists to assume that their assistants
in theatre are familiar with all their activities. In this situation the trainee may
assume that the assistant will understand both what to look for and when to look
at the monitor. In fact this is often not so, and facilitating discussion with ANTS
elements and categories claries the importance of the exchange of information and
coordination of activities so that the trainee and his/her assistant have shared situation
awareness and the safety of the procedure for the patient will be enhanced.
At this time, in the earliest stages of postgraduate training, it is likely that
most UK medical graduates will not be aware of the concept of NTS. The novice
trainee may not see the behaviours being highlighted in the examples given above
Non-Technical Skills and Anaesthetists’ Workplace-based Assessment Tools
183
as NTS. They may just be seen as behaviours to be copied and applied as part of
the initiation into the profession of anaesthesia. However, where the instructor
carries the framework and uses it to structure the components of practice, this can
help direct the behaviours of the novice into good habits of practice. Then, when
the trainee is introduced to the theoretical aspects of NTS, there are examples of
clinical practice readily available to be used for illustrative purposes. Furthermore,
when trainees have familiarity with the ANTS system, they can be asked to
incorporate NTS when they articulate discussion of cases or events.
To summarize, we can say that a trainer who is familiar with the ANTS system

can use that system as a framework to help trainees acquire non-technical skills
from the earliest stages of training. It will also have become obvious that there can
be considerable overlap between anaesthetists’ technical and non-technical skills.
A trainee anaesthetist can only anticipate what will happen during an operation if
they have acquired some experience of that procedure. In a similar vein a trainee
can only anticipate what impact a particular drug will have on a patient if that
trainee has the necessary understanding of physiology and pharmacology. It may
now become a little clearer why in the early days of applying the ANTS system the
emphasis was on use with more experienced trainees.
Future Directions
So, where do we go from here? Let us review the position briey.
The ANTS system can be used in the clinical workplace.
The cascade model introduced by the Royal College of Anaesthetists to
introduce the ANTS system as a tool to be used at departmental level has
not been successful.
Workplace-based assessment tools are now being introduced into
postgraduate medical training in the UK.
The proposed assessment tools have been adapted from existing tools and
are being modied for purpose.
There appears to be an overlap between technical and non-technical skills
in anaesthesia.
One possible strategy is to incorporate appropriate aspects of the ANTS
system into the miniCEX, DOPS and CBD workplace-based assessment tools. As
anaesthetists learn to use these tools in the workplace they can begin to include
NTS in the manner outlined previously. In this way, we may assist consultant
anaesthetists in becoming more familiar with the ANTS system. An experimental
MiniCEX form (Figure 11.1 and 11.2) was designed in a collaborative venture
between UK and New Zealand departments. Results from the UK limb are in the
process of being collected and analysis is awaited.
1.

2.
3.
4.
5.