Testing the testers

The recent debacle with the SAT marking in the UK has now reached its climax with the termination of the ETS contract. As the Guardian reported earlier this week:

"The Qualifications and Curriculum Authority and ETS Europe agreed to dissolve the five-year contract to handle the national curriculum tests with immediate effect, after test results were delayed this summer. Under the deal, ETS will make a payment of £19.5m to QCA, and the watchdog will save approximately £4.6m in cancelled invoices and charges.

Full results for tests sat by 14-year-olds this year were still not available when national figures were released earlier this week.

The agreement to end the contract early has been made by both parties and there will be no payments made by QCA to ETS Europe for any future years of the contract.

QCA signed the £156m five-year contract with ETS in February last year. However, serious problems with the administration and marking of the tests this year meant the publication of results for key stage 2 and 3, which should have been released on July 8, were delayed by almost a month."

This type of story has no obvious winners. There is already an inquiry into why this situation was reached but it looks like the competence of both the QCA (in its procurement selection process) and ETS (in its operational processes) will be called into question. Hopefully the outcome will lead both procurement divisions and organisations running new processes to view testing more seriously. Testing is often viewed as an IT activity, but business process testing and regulatory requirements testing are wider in scope than just testing the technology, computers and software. It is uncertainty that testing helps eliminate not just defects. And in this case more certainty would have saved a lot of trouble.