Do we need a unit standard for Verifiers?
by Erik Hallendorff - a director of TLN Human Performance Systems (Pty) Ltd and a member of the Assessor SGB.
The title of this paper asks whether we need a unit standard for verifiers. The central question is not: do we need verifiers? Or, are verifiers important? This paper is presented in the writers personal capacity to provide a counter-perspective to calls for the resurrection of the expired verifier unit standard.
The position taken in this paper is that verifiers are critical to the success of the NQF system, but that calls for a verifier unit standard reflect
a possible misunderstanding of the role of unit standards and/or
a possible failure to consider the role of verifiers in relation to existing unit standards.
What unit standards are and what they are not
Unit standards are one part of the whole performance system, and the purpose of a unit standard is simply to describe the standards associated with competent performance. Thus unit standards identify required outcomes of learning that are worthy of national recognition, together with associated standards of performance.
Unfortunately, unit standards are sometimes seen in ways that distort their fundamental purpose. A common distortion is found in the confusion between unit standards and standard procedures. Thus we find those who believe that we need a unit standard on this and that to make sure people do this and that properly. To state it differently, it is not uncommon to find situations where people believe they cannot function unless there is a unit standard that gives them permission to do so. Such thinking leads to a proliferation of meaningless unit standards that fail to serve their purpose, while detracting from the intended purpose of unit standards.
If we need to make sure that people know what to do in any given situation, then we should write a procedure for it, not a unit standard. We do not need unit standards to tell us what to do, when to do it or how to do it. Where such a need exists, then the appropriate instruments are policies and procedures, not unit standards. Unit standards are not work instructions, process descriptions, task details or standard operating procedures. Unit standards are not intended to take the place of any of these ways of communicating the requirements of a job or task.
Applying this thinking to the topic at hand, we do not need a verifier unit standard in order for there to be verifiers . We do not need a unit standard to define the work role of a verifier, nor to outline the procedures for verifiers to follow. Such instructions are more usefully and appropriately to be found in a variety of other documents, including Terms of Reference, job descriptions and work procedures. What we do need are unit standards that define the competencies that verifiers need to have. If our analysis of the required competencies leads us to a particular unit standard, then we should develop such a unit standard, but if the competencies are defined via a collection of unit standards, then we will be covered. The writers position is that the role of the verifier is adequately covered by a number of unit standards, and that there is no need for a separate verifier unit standard.
Components of a quality ETD system
Before exploring the particular subject in more detail, let us consider the components of a quality education, training and development (ETD) system.
Quality assurance is critical for the ongoing success of ETD. The decade-long NQF system provides many inter-related mechanisms for quality assurance, including:
The generation and registration of unit standards and qualifications, which are intended to communicate agreed standards;
The accreditation of providers, which is intended to ensure providers have internal quality assurance systems and the capacity to deliver (teach and assess) the outcomes of learning they claim they can deliver;
The registration of assessors, which is intended to ensure that assessors have the technical expertise and the assessment expertise required to make fair, valid and reliable decisions concerning a learners competence in relation to an agreed standard;
Internal and external moderation of assessment, which is intended to confirm that assessment decisions are fair, valid and reliable.
Clearly, the success of the NQF system depends on all of the above, and will be greatly compromised if any of the above mechanisms are compromised. In particular, the SAQA structures tasked with external quality assurance, i.e. ETQAs, play a critical role, particularly in relation to accreditation of providers and endorsement of certificates of learner competence. It is at this point that we may consider the role of verifiers more closely.
Broad role of verifiers
In a perfect world, we would not need verifiers. We could simply trust that learners would be assessed by competent and honest assessors, and that we could trust any declaration of competence so made. However, in the real world, we need to check whether the declarations of competence are valid.
The first level of confirmation is at provider level, where providers are required to engage in processes to check whether assessments were carried out to standard, and that the assessment findings are fair, valid and reliable. This process should also seek to support and improve upon assessment practices and processes for the future. We call this process of confirmation, moderation, although there are those who call it internal verification.
The second level of confirmation is no different. The second level of confirmation necessarily takes place externally to the provider, but has an identical function i.e. to engage in processes to check whether assessments were carried out to standard, and that the assessment findings are fair, valid and reliable. Clearly, this level of confirmation is much broader than at provider level, and can therefore be considered to be more at systems level; such confirmation also has the benefit of engaging with internal moderation findings, and will therefore need to confirm those findings. Just as is the case at provider level, this process should also seek to support and improve upon assessment practices and processes for the future, but such support will generally be at systems level. We have come to call this process of confirmation, verification, although some may wish to call it external moderation.
The above relationship between internal and external confirmation of assessments is supported by the ETDP SETAs definition of verification as follows:
"External verification is a process of checking moderation processes and confirming or overturning moderation findings. External verification is conducted through a team of verifiers who are competent assessors and moderators.'
So it is that ETQAs (and other Quality Assurance Bodies) employ the services of verifiers, and in some cases, ETQAs need to train verifiers. However, it is tempting to assume that because we need an assessor unit standard for assessors, and a moderator unit standard for moderators, we necessarily need a verifier unit standard for verifiers. But an examination of the role of verifiers yields an interesting picture in terms of the skills or competencies needed by verifiers, and thus raises interesting questions about what the training of verifiers should consist of.
What then is the express role of verifiers? In some situations, verifiers simply carry out document checks to determine whether the provider submitted all the required documentation and that everything was complete. Clearly we would not spend time debating the subject to meet what is purely a low-level clerical role. But what is clear is that verifiers do fulfil varying roles from situation to situation, and thus the precise cluster of competencies needed by verifiers will largely be determined by the particular role that is assigned. For the purposes of this paper, let us consider the broadest role spectrum of a verifier, and then identify the competencies associated with this broad role, and ultimately, the unit standards that express those competencies in outcome terms.
What do verifiers do?
As stated earlier, the precise role of a verifier will depend on the particular circumstances of the task at hand, but we can identify the following general role. According to the ETDP SETA (2005), verifiers will generally be required to:
confirm the results of an assessment against unit standards and qualifications
look for evidence from the moderators reports on the validity of the assessment process
sample moderated assessments
ensure that a fair judgement is passed on the verification of all assessments under review, based on ETQA criteria
evaluate the moderated process focusing on learner achievements
verify that providers use registered assessors and moderators
record and report on verification findings and make recommendations to the ETQA
advise and support moderators
The above role of the verifier pertains to the external verification of a particular set of internally moderated assessments at provider level. In the process, the verifier will make more general checks on the moderation and assessment system of the provider, and we may thus add the following further aspects to the verifiers role:
evaluate the internal moderation and assessment system of the provider
decide what impact the findings have on the set of assessments being verified
provide support and feedback to the provider for the purposes of improvement.
To carry out the above role, verifiers will need to have knowledge of and be able to follow the particular policies and procedures of the Quality Assurance Body at hand. It is also clear that verifiers will assume responsibility for checking on many of the requirements that pertain to accreditation of providers, but with the advantage of being able to check the actual practice of the provider.
Relationship between internal moderators and external verifiers
There are clear indications that the terms "moderator' and "verifier' are used interchangeably within the South African context. It seems however that the following trend has emerged: when referring to internal moderation/verification, we use the term "moderation', and when referring to external moderation/verification, we use the term "verification'. However, the main distinction has to do with the "internal' versus the "external' rather than any difference between the terms moderation and verification. It has been suggested that a key difference between an internal moderator/verifier and an external moderator/verifier has to do with subjectivity. While subjectivity may in fact be an issue, this does not point to a difference of skill i.e. if we were to use an internal moderator for the purposes of external moderation in another setting, the subjectivity issue would not present itself.
What is clear is that one of the main roles of the verifier, as defined by ETQAs, is moderation. The Assessor SGB has indicated that the moderator unit standard is intended for internal and external moderation i.e. whether you carry out moderation of assessments internally or externally, the same skills are required and therefore the same unit standard applies. Although the processes for doing internal versus external moderation may differ (from an organisational point of view), the skills are essentially the same i.e. moderation. Therefore the moderator standard should be an essential part of any verifier training programme.
Skills required by verifiers
Based on an analysis of the role/s of a verifier, we can identify the following critical skills:
understanding of the outcomes-based system and the NQF
working knowledge and understanding of outcomes-based assessment
the ability to check on the quality and validity of assessments through the application of a variety of moderation techniques and to verify the validity of internal moderations
the ability to evaluate provider quality assurance systems, in particular related to assessment and moderation
working knowledge of quality systems for ETD
the ability to compile reports on findings and recommendations related to moderation of assessment, checks on internal moderation, and checks on provider quality systems
general clerical, administration and record keeping ability
the ability to communicate with relevant parties in writing and orally
working knowledge of the Quality Assurance Bodys policies and procedures for verification.
In most cases, verifiers will need to have broad subject expertise in the area in which verification takes place, although this can be compensated for, where required, by having ready access to relevant experts. In some cases, it may be advantageous for verifiers to also have a working knowledge of how to design outcomes-based assessments, but that is not a fundamental requirement.
Unit standards required by verifiers
Given the skills required by verifiers, it is clear that such skills are addressed by a number of unit standards as below.
|understanding of the outcomes-based system and the NQF|| This can effectively be addressed via 115753, although the following unit standard can be considered:
Optional: NLRD 114924: Demonstrate understanding of the outcomes-based education and training approach within the context of a National Qualifications Framework (L5)
|working knowledge and understanding of outcomes-based assessment|| NLRD 115753:
Conduct outcomes-based assessments (L5)
|the ability to check on the quality and validity of assessments through the application of a variety of moderation techniques, and to verify the validity of internal moderations|| NLRD 115759:
Conduct moderation of outcomes-based assessments (L6)
the ability to evaluate provider quality assurance systems, in particular related to assessment and moderation
working knowledge of quality systems for ETD
| NLRD 15191:
Evaluate education and training providers and systems (L7)
(Added option: NLRD 15228:
|the ability to compile reports on findings and recommendations related to moderation of assessment, checks on internal moderation, and checks on provider quality systems||Covered by above|
|general clerical, administration and record keeping ability||Covered by above|
|the ability to communicate with relevant parties in writing and orally||Covered by above as well as generic communication standards|
|knowledge of the Quality Assurance Bodys policies and procedures for verification.||This is particular to each QA Body, and should not be addressed by a unit standard, but rather via the QA Bodys particular procedures and codes of conduct|
Having identified the above cluster of unit standards that are suitable for verifiers, and observing that the listed unit standards cover all the skill requirements, without leaving any gaps, the notion of a separate verifier standard is rendered redundant.
What do others think?
It seems that there are significant others who essentially support the position taken by this paper, whether such support is intentional or not. As a case in point, when the ETDP SETA advertises for verifiers, its requirement is for verifiers to have the assessor and moderator unit standards.
It seems there is support given by Marietta van Rooyen in a recent paper on this site concerning the need for verifiers, by stating:
(Internal verifiers/moderators) "fulfill the same roles and functions as external verifiers, and will often be the contact person for quality assurance bodies. ETQAs could delegate the verification of an organisations moderation systems to a good internal verifier, but will never abdicate the responsibility. This means that they will still be responsible for the quality assurance of the internal verification processes'.
The writer of this paper suggests that since the roles and functions fulfilled by internal and external verifiers are essentially the same, and having addressed the majority of verification needs via the existing moderator unit standard, there can surely be no need for a further verification unit standard. This paper however suggests that we give very careful consideration to developing verifiers as far as possible in the "evaluate providers' unit standard (15191), even though this has primarily to do with accreditation needs. The reason is because verifiers may often be called upon to verify that the accreditation requirements are in fact practiced by the provider.
Apart from those who seem to agree with the position taken by this paper, it seems there are some ETQAs and other Quality Assurance Bodies that continue to express a need for training of verifiers against the expired verifier unit standard. This could be for a variety of reasons, including the possibility that they do not know that the unit standard expired and that the Assessor SGB and SAQA decided not to re-register the unit standard. It could also be due to the possibility that such bodies are unaware that the function of the verifier is more than adequately addressed by the unit standards identified in this paper. Should this in fact be the case, it calls for SAQA and the ETDP SETA to take the lead by communicating to other SETAs and ETQAs this position.
This paper agrees with those who assert that verifiers perform a very important role within the quality assurance of ETD. This paper also supports the need to ensure verifiers are properly trained and assessed, and thus seeks to identify those unit standards that should be used for the training and assessment of verifiers. In essence, this paper shows that the identified unit standards address the skills requirements of verifiers adequately, thus leaving no gap that could conceivably be filled by a so-called verifier standard. Those who assert that there ought to be a separate verifier unit standard will first need to show what gap exists that is not met by the identified unit standards, and if such a gap exists, that a unit standard is justified to fill this gap.
This paper contends that those people who have achieved the identified unit standards, and who have also been trained in the particular procedures and requirements of the Quality Assurance Body at hand (for which no unit standard should exist or be awarded), will more than adequately fulfil the emerging needs of verification.