By Marietta van Rooyen - Chair: Assessment College
While Basil and I were travelling in the USA we came across a remarkable book called Animals in Translation by Temple Grandin and Catherine Johnson. Temple Grandin uses her own autism as the key to unlock a riveting world of animal consciousness and animal behaviour. I found some fascinating parallels and many insights between the way animals behave and learn and the way we train and educate human beings.
Temple Grandin is an associate professor at the Colorado State University, where she holds a Ph.D. in Animal Science from the University of Illinois. She is the author of four books. Through her company, Grandin Livestock Systems, she works with fast food purveyors, including McDonald?s and Burger King, to monitor the conditions at animal facilities worldwide.
One of the lessons we can learn from Temple is found in her approach to the animal welfare audits she does for the US Department of Agriculture. She uses the HACCP (pronounced hassip) or Hazard Analysis Critical Control Point analysis. Her approach is to analyse the critical control points in a farm animal?s well being. She defines a critical control point as a single measurable element that covers a multitude of sins.
I believe that the auditors in our NQF system can learn much from Temple regarding the simplification and practicality of quality audits of ETQAs and providers.
An example from Animals in Translation
When Temple audits animals on a farm, she needs to know whether the animal?s legs are sound. There are multiple reasons for animals not being able to walk properly. Some regulators will list all these reasons for lameness, because they think a good audit is a thorough audit. But that is not Temple?s approach. She measures one thing only: How many cattle are limping?
That one measurement covers a multitude of sins. If too many animals are limping, the farm will fail its audit. In order to pass the next audit, the farmer will need to fix whatever it is that makes these animal lame. Management now needs to find out what the problem is and fix it.
Temple came up with five key measurements inspectors need to take to ensure animals receive humane treatment at a meatpacking plant:
? Percentage of animals stunned or killed correctly on the first attempt (Must be 95%)
? Percentage of animals that remain unconscious after stunning (must be 100%)
? Percentage of animals who vocalise during handling and stunning (no more than 3%)
? Percentage of animals that fall down (no more than 1 %)
? Electric prod usage (no more than 25%).
She also has a list of five acts of abuse that are automatic failure. I will not dwell on these out of respect for those who are sensitive about animal welfare.
A case of not seeing the wood for all the trees
According to Temple, some auditors are concentrating so much on the conditions of the floors and equipment that they forget to focus on the humane treatment of the animals. The above five points are objective and directly measurable. In addition, the people working on these plants can remember two sets of five items.
Temple remarks that many auditors think that a simple system cannot work.
"They tend to make five critical mistakes:
? They write verbal auditing standards that are too subjective and vague, with requirements like "minimal use of electric prod' and "non-slip flooring.' Individual inspectors have to figure out for themselves what "minimal use' means. A good audit checklist has objective standards that have or have not been met.
? For some reason, highly verbal people have a tendency to measure inputs, such as maintenance schedules, employee training records, and equipment design problems, instead of outputs, which is how animals are actually doing. A good animal welfare audit has to measure the animals, not the plant.
? Highly verbal people often want to make the audit way too complicated. A 100-item checklist doesn?t work nearly as well as a 10-item checklist, and I can prove it
? Verbal people drift into paper audits, in which they audit a plant?s records instead of its animals. A good animal welfare audit has to audit the animals, not the paper and not the plant.
? Verbal people tend to loose sight of what?s important and end up treating small problems the same way they treat big problems.'
All five of these mistakes actually cause the animals to be harmed. In stead of looking at outcomes to the animals, auditors want to focus on the inputs by telling the plants how to build its floors, and then they want to inspect the construction of the floors. The animals get lost in the confusion.
Apply these principles to ETD Quality Assurance in South Africa
So what has this got to do with learning and outcomes-based education and training? In my capacity as a ISO listed private provider as and the Chair of the ETQA Sub-committee at SAQA, I get to know much about audits. In addition our associates and staff carry out many audits of other providers on behalf of several ETQAs.
I have found that these audits are too bulky and could very well be erring in exactly the way Temple warns about. The very verbal lists of hundreds of items that both ETQAs and providers need to comply with is simply too confusing! We need to find the HACCP way to do these audits. That means that we need to find a as a single measurable element that covers a multitude of sins in terms of learning systems.
More lessons from Temple
If you give an auditor a 100-item checklist, he will tend to treat 50 items as important, while perhaps only 10 items are critical enough to warrant an adverse audit. When a plant fails one of ten critical items it is clear that the plant could get an adverse audit. But when the item is one of a hundred, it does not seem serious enough to close down the plant, yet it may be a critical shortcoming. By complicating the list with hundreds of detailed items, the auditor stands a good chance of not seeing the wood for all the trees.
Temple points out that the dangerous thing about paper audits with 100-item checklists is that they can set you up for a situation where things slowly get worse without anyone realising it. She says: "When you drift away from the animals themselves and start auditing the paperwork, the bad can become normal pretty quickly.'
Even though Temple?s five-item list is short it is rigorous enough for the plants to fear that they may not comply. But the simple five-point checklist has proven itself and works beautifully. Nevertheless, we are warned that "The whole principle of HACCP is that you have to keep measuring standards and compliance or everything goes bad on you.' Temple also warns that paper audits end up masking small, incremental declines in standards that result in very large drops in animal welfare.
Accreditation criteria are too detailed
A good example of provider accreditation being too detailed is the way some ETQAs evaluate the learning material. Even if the materials are not outcomes based, the entire curriculum may well be outcomes based and this is easy to assess when evaluating the assessment instruments and activities. Thus the single measurable element here is the assessment, which needs to be outcomes-based. This assessment covers a multitude of sins, such as not being standards- or qualifications-based, lack of content knowledge, pitched on the wrong level and lack of use of critical outcomes.
We need to stop focussing on the inputs and start looking at the outcomes. For example, we will not require a provider to have a library and librarian, but rather ensure that HET learners are actually encouraged to do research during their studies. This skill can be assessed through appropriate assignments and assessment instruments, which are more objective. It seems to me that the specific assessment instrument and activity used is definitely one of the single measurable element that covers a multitude of sins that covers a multitude of sins.
Our trusted internal Auditor, Rudi Britz, tells me that the ISO standards are at present being adapted to be more outcomes-based. He promised me some more information on these changes, which we will publish in our November newsletter. I do hope that we can start being more developmental in our attitude towards providers and stop being focussed on lots of petty details based on the whims of academics and what Temple calls highly verbal people.
Ref: Animals in Translation; Temple Grandin and Catherine Johnson, Scribner, New York: 2005 ISBN-13: 978-0-15-603144-2