Software testing FAQ - No. 5

Is the number of testers in the UK a perfect number?

Another question from Oliver Letwin.

What is it with you and numbers Oliver? You may think this is an odd question but you'd be wrong. Nobody has ever found an odd perfect number. As all software testers know, a perfect number is one that is equal to the sum of its divisors. 6 is the first perfect number. Its divisors are 1,2 and 3 and 1+2+3 is 6. Perfect? Well Pythagoras thought so. There are loads of them but nobody has found one that is an odd number.

In theory there is no reason why there isn't an odd perfect number so if you have a few spare moments and are good at adding and dividing you might want to see if you can find one. This will make you famous. To be exact you will be more famous than the bloke who solved Fermat's last theorem (Andrew Wiles) but not quiet as famous as Bernard Yoeuns (who pretended to be Stan Ogden in a documentary series about life in England).

Anyway back to the question - the answer is no.




load performance testing

Testing business continuity

Disaster Recovery is not Business Continuity. Many companies do not have full business continuity plans. They say they do have business continuity plans but they really mean that they have a disaster recovery plan, usually meaning that they have alternative premises and possibly equipment that can be used in the case of a full scale disaster. Business continuity covers far more than just the IT systems. Think of all the paper records an organisation needs to continue working. Think of the most important asset of all to most organisations: its staff.

Without its staff these organisation ceases to exist. A business continuity plan contains information for all staff and their activities in the case of problems affecting the organisation.

A preliminary to the testing of any plan is to establish some form of Business Continuity Group consisting of representatives from each of the main business areas, together with those responsible for finance, facilities and IT.

Once a business continuity plan exists it needs to be maintained and tested regularly. Once again, many organisations say their plan is tested but what happens is that they show that the major IT systems can be seen to be working on equipment at a disaster recovery site. Often there is no involvement other than from the IT Group.

It is essential that business continuity testing follows a risk based approach. This provides 2 main advantages. Firstly any business continuity must be aligned to the business and that the plan should be designed to cope with risks to the business. Secondly, by following a risk based testing approach to business continuity, this highlights the areas not to test, by prioritising the main risks to business and therefore identifying areas of negligible or zero risk.

A business continuity test need not be onerous or expensive. There are a number of ways in which testing can take place; each is mentioned below. Business continuity testing can be broken down into 2 main areas, desktop testing and physical testing.

Desktop testing can be a paper walkthrough where a group of people work through the plan looking for areas which require further work. It can also be scenario testing where a group sit and work through a scenario given to them, such as electrical failure, fire, bomb threat etc. The scenario is defined by a different group of people who then monitor the accuracy of the business continuity plan.

Physical testing means a form of business continuity testing that happens outside the conference room. This is broken down into a number of different tests. Firstly a communications test. Can everyone who needs to be notified during a problem actually be contacted? Second in physical testing is a disaster recovery test, where the IT systems are established on a secondary set of computers, and thirdly, a full relocation test, where the business areas relocate to another site. All of these tests are carried out in order to hone the business continuity plan and to provide assurance that it will be effective when required.

In summary, all business continuity plans need to be tested. Some companies believe that the testing would be too complex, time consuming or expensive. It is therefore essential to use a 3rd party group of experts to advise, help carry out and monitor the tests that are carried out. The 3rd party would also make suggestions regarding any changes believed necessary to the existing plan.

© Acutest Ltd 2005

About Acutest

Acutest is a consultancy specialising in testing software, IT and technology-enabled change. Acutest provides a full portfolio of outsourced software testing services. The principal aim of all these services is to increase the value businesses derive from testing by reducing the elapsed time spent testing, reducing the cost of testing and reducing the risks to launching new products and services. Popular services include performance testing, load testing and business continuity testing.

Software Testing FAQ - No. 4

Is it possible to get pirated copies of free defect management tools like Bugzilla or Tracker ?

That question came from former Brotherhood of Man singer, Sandra.Well Sandra, if you follow the links you will find out how to get copies of these tools. There are loads of free defect management tools but I can't remember what they are all called. Perhaps informed readers could send me a list. I have not seen any pirated copies though. Maybe you could mention it to pirates next time you meet some. There is a definite gap in the market there.

Did you know that Johnny Kidd and the Pirates were claiming to be pirates long before software piracy became popular! They were probably "Shakin' all over" because they'd heard that FAST (Federation Against Software Theft) was going to be created a mere 23 years after they released this song and would probably catch them and kick their butts.


Software testing UK

Scalability testing: 7 steps towards success

Systems that work well during development, deployed on a small scale, can fail to meet performance goals when the deployment is scaled up to support real levels of use.

An apposite example of this comes from a major blue chip company that recently outsourced the development of an innovative high technology platform. Though development was behind schedule this was deemed acceptable. The system gradually passed through functional elements of the user acceptance testing and eventually it looked like a deployment date could be set. But then the supplier started load testing and scalability testing. There followed a prolonged and costly period of architectural changes and changes to the system requirements. The supplier battled heroically to provide an acceptable system, until finally the project was mothballed.

This is not an isolated case. IT folklore abounds with similar tales. From ambulance dispatch systems to web-sites for the electronic submission of tax returns, systems fail as they scale and experience peak demands. All of these projects appear not to have identified and ordered the major risks they faced. This is a fundamental stage of risk based testing, and applies equally to scalability testing or load testing as it does to functionality testing or business continuity testing. With no risk assessment they did not recognise that scaling was amongst the biggest risks, far more so that delivering all the functionality

Recent trends towards Service Oriented Architecture (SOA) attempt to address the issue of scalability but also introduce new issues. Incorporating externally provided services into your overall solution means that your ability to scale now depends upon these external system operate under load. Assuring this is a demanding task and sadly the load testing and stress testing here is often overlooked.

Better practice is to start the development of a large scale software system with its performance clearly in mind, particularly scalability testing, volume testing and load testing. To create this performance test focus:

1. Research and quantify the data volumes and transaction volumes the target market implies. Some of these figures can be eye openers and help the business users realise the full scale of the system. This alone can lead to reassessment of the priority of many features.



2. Determine the way features could be presented to users and the system structured in order to make scaling of the system easier. Do not try and have the same functionality you would have for a single user desktop solution provide an appropriate scalable alternative.



3. Recognise that an intrinsic part of the development process is load testing at representative scale on each incremental software release. This is continual testing, focusing on the biggest risk to the project: the ability to operate at full scale.



4. Ensure load testing is adequate both in scope and rigour. Load testing is not just about measuring response times with a performance test. The load testing programme needs to include other types of load testing including stress testing, spike testing and volume testing.



5. Don’t forget that failures will occur. Large scale systems generally include server clusters with fail-over behaviour. Failure testing, fail-over testing and recovery testing carried out on representative scale systems operating under load should be included.



6. Don’t forget catastrophic failure could occur. For large scale problems, disaster testing and disaster recovery testing should be carried out at representative scale and loads. These activities can be considered the technical layers of business continuity testing.



7. Recognise external services if you use them. Where you are adopting an SOA approach and are dependent on external services you need to be certain that the throughput and turnaround time on these services will remain acceptable as your system scales and its demands increase. A smart system architecture will include a graceful response and fall-back operation should the external service behaviour deteriorate or fail.

© Acutest 2005

Software Testing FAQ - No. 3

Did the Romans invent software testing?

That one is from Roman Abramovich.

Interesting question, Roman. Not sure why you think this.
The answer is no.


Software testing UK