In February 2001 the Assessment SGB registered four unit standards through NSB 05 at SAQA. ASSMT 03 was the standard called Verify Moderation of Assessment. This standard was meant for those who do quality assurance of the assessment and moderation systems in organisations.
Examples of such persons are ETQA officials from SETAs and professional bodies, provincial school examination officials, and coordinators of moderation in large organisations such as government departments and corporations. An extensive article on the role of the verifier was first published in the Cutting Edge Newsletter of January 2002.
The purpose was to assist ETD Practitioners to understand the function of the verifier, as defined by the Assessment SGB and confirmed by NSB 05 and SAQA, after public consultation. What surprised me, on reading this article recently, four years later, is that there was so little uptake on this unit standard, even though it has a crucial role in establishing quality structures.
I edited down the article extensively and added new comments on the previous article in bold. Above all, the decision of the Assessment SGB to decline to review or renew the verifier unit standard needs to be debated and reconsidered. Many ETQAs are still training verifiers and there seems to be a need for their appointment.
An Introduction to Verification
The stated purpose of the NQF Unit Standard for Verifiers, SAQA ID no. 7975 is: This unit standard is for people who are involved in verifying, at a systems level, the results of moderation of assessments against unit standards and/or qualifications.
People who have achieved this unit standard will be able to plan, design and implement verification of moderation and assessment in a fair, valid, reliable and practicable manner that is free of all bias and discrimination, paying particular attention to the three groups targeted for redress: race, gender and disability.
In particular, people credited with this unit standard can:
- Plan and prepare for verification
- Conduct verification
- Evaluate verification plans and processes
- Record and report verification findings and recommendations
- Advise and support moderators and providers.
Verification is thus a process for gathering information and obtaining findings relative to moderation systems and processes. These findings are used to support moderation and validate or overturn moderation findings. We focus on verification on a systems level, rather than direct moderation of assessment.
One of the challenges faced by ETQAs is that the verifier needs to be both a certificated assessor and moderator. However, many of the officials in quality assurance bodies, who are supposed to be verifiers, have no experience in assessment or moderation.
The Objectives of Verification
There are two main objectives of verification: accountability and improvement.
Accountability may be internal (to managers, committees etc.) or external (ETQA, SAQA, government etc.). Verification for accountability is summative, and verification for improvement is formative.
Accountability
An organisation needs to clarify the issues underlying accountability. Trow (1996) has written extensively on accountability. He defines it as "the obligation to report to others, to explain, to justify, to answer questions about how resources have been used, and to what effect'.
The extent to which accountability exists within an organisation will depend on the structure, culture and procedures, and where decision-making powers reside. It has recently become clear from court finding that ETQAs are accountable for assuring that assessments and moderations are done properly and according to the principles prescribed.
This means that the ETQA is not able to shift the responsibility of quality assuring the assessment and moderation systems onto providers. After all, the certification responsibility resides squarely with the ETQA.
Improvement
Improvements are usually achieved by identifying weaknesses. Improvement involves some form of change. Change is often mistakenly associated with the need for additional resources; this is not always the case.
Improvement-focused verification can result in cost savings by identifying duplicated or wasted effort, and proposing ways in which costs can be eliminated or reduced. By working on the improvement of internal moderation systems ETQAs can reduce their workloads considerably.
It they can put more trust in the internal moderation system they will be able to verify the moderation system only. Then external moderation only comes into play where the internal moderation system fails to meet the standards.
Verification for Continuous Improvement If we strive towards a quality service or process, we strive towards continuous improvement. If an organisation is dedicated to continuous improvement, it will constantly improve its internal performance, systems and quality.
Continuous improvement begins with an understanding of where you are and where you want to be. Everyone wants to be the best in their field, however, our ambitions should be realistic and in line with our resources. Goals should be achievable while challenging employees to extent their current abilities.
Where do we begin to move along a path of continuous improvement? Like any journey, we have to determine where we are and where we want to go.
Through analysis of collected data, opportunities for improvement will become evident. Verifiers can assist the providers and workplaces to develop and grow their moderation systems from the present state to a future quality system by setting them milestones and deadlines.
The idea is not to close down providers that do not have moderation systems, but rather to assist them to set up these systems.
Roles of the Verifier
External verifier
External verifiers are often associated with monitoring and audits. It is done under the auspices of a statutory, non-statutory or independent body, e.g. SAQA, ETQAs and quality assurance agencies. The external verifier has an auditing function, often linked to the verification for accountability.
Monitoring and auditing the moderation system is a time consuming activity and should rather not be combined with accreditation monitoring and auditing. Combining the two activities will lead to neglecting of the quality assurance of learner abilities and the qualification itself (QALA).
The role of the external verifier While the internal moderator is responsible for the maintenance of the quality of the internal assessment system, the external verifier is responsible for checking the moderation and moderation system on behalf of the ETQA.
Competencies of the external verifier
The external verifier need not be a technical expert in a specific area, but should have a good knowledge and understanding of the particular learning field in which the standards exist. Furthermore, the external verifier should have an in-depth understanding of assessment and moderation practices, and should be able to ensure the smooth and efficient running of a verification system.
External verification process
The external verifier represents the ETQA in assuring the ongoing quality and integrity of internal assessment and moderation systems. The external verifier will perform the verification when the internal moderator requests it via the ETQA if there is a discrepancy in the assessment results the ETQA requests it for approval of an internal assessment system or for regular monitoring and evaluation of the internal assessment system.
Guidelines for external verification
The external verifier is a representative of the ETQA and is an agent for the quality of the education and training system. As such, the external verifier's role is vital to maintaining the integrity of qualifications being awarded. The external verifier will rely on guidelines laid down by the ETQA for accrediting providers and verifying moderation decisions.
Verification procedures should be transparent and should involve all necessary stakeholders.
The external verifier needs to verify the following in terms of the moderation system:
1. Is there a moderation system with policies and procedures in place?
2. Is this moderation system suitable for the type of assessments and the field within which it is used?
3. Is the moderation system actually used effectively and how are finding on non-conformances in assessment followed up?
If any of the above questions lead to negative answers, the verifier needs to advise the moderators and provider on remedial steps coupled with deadlines.
Internal verifier
One of the functions of the internal moderator is to set up, implement and oversee the internal assessment and moderation system of the enterprise. This includes the establishment of policies and procedures, schedules, documentation and information management system.
This information should all be made available to the ETQA. An internal verifier operates within the organisation and it is their duty to ensure continuing standards and ongoing improvements in the organisation. Their inputs are often subjective, but informed, accurate and contextual.
They are usually part of the organisation and fulfill a monitoring function. They fulfill the same roles and functions as external verifiers, and will often be the contact person for quality assurance bodies. ETQAs could delegate the verification of an organisation's moderation systems to a good internal verifier, but will never abdicate the responsibility.
This means that they will still be responsible for the quality assurance of the internal verification processes.
Verification Techniques
Verification should not be a paperbound reiteration of information; a more holistic approach should be followed. The verification areas should be adopted according to the specific circumstances and should be seen as part of the ongoing business.
A quality culture should be sustained and created by adopting practical, appropriate and effective procedures. Futuristic quality planning is necessary to assure successful applications of new systems, processes, and products. The core members of this team should consist of people who have the necessary authority to make decisions that support the programme.
During the course of planning activities, it will often be necessary to recruit employees who have special insight or knowledge of the process being evaluated. It is best to use a structured approach for this process.
This will assure consistency of purpose and allow easy inclusion of participants who have had experience on other quality teams. The following steps should be followed to assure efficiency of services offered:
- Flow chart the process chain for the activity
- Assess the current method and effectiveness of quality control
- Do a failure mode and effects analysis of high-risk process steps
- Develop a control plan to assure quality
Examples of such techniques are the SWOT Analysis and Situation-Analysis approach.
The importance of the role played by the verifier cannot be over-estimated. This person will need to have the respect and trust of the constituent providers and moderators.
It is therefore necessary to take care when appointing verifiers in the system to ensure that they have the required maturity, knowledge and skills to be able to fulfill their objectives.
I repeat the decision of the Assessment SGB to decline to review or renew the verifier unit standard needs to be debated and reconsidered. Many ETQAs are still training verifiers and there seems to be a need for their appointment.
