Number of testers needed

One of the most common Classic Testing Mistake is about "Planning the Testing Effort". That means how should the whole testing team's work be assigned?

Test leads (or managers) often need to make an initial estimate of the number of people that will be required to test a particular product, before the information or the time to do a detailed task breakdown is available. One thing that is almost always available is the number of developers that are or will be working on that project. Now the question is how to estimate the total number of people that will test the application.

From the last one and the half year of my testing experience as a Technical Lead (Testing), I also encountered this type of problem. Management always wants to know that how many testers I am going to assign in a particular project for smooth and ontime testing. Initially, I was not sure how to estimate this number but now I am pointing out some facts that needs to be taken care before doing this estimation tasks. In my opinion, number of testers needed for a particular project depends on following points :

1. Tester Efficiency
2. Tester time spent on other matters
3. Values and speed of finding defects.
4. Number of developers working on the project.
5. Amount of functionality
6. Developer efficiency at removing defects before tests.
7. Developers efficiency at inserting defects.

1.Tester Efficiency :

If your testing team members are efficient and sharp, you need less number of testers to test the application. That depends on the experience and the way of working of the tester. In one of my previous company, I was leading a team of three testing engineers. Out of those three, one resource was too good in testing. I always kept that resource with me in each application testing (if possible) for smooth testing. Infact. in my initial years of testing experience, my manager always trust me for testing of any of the application. And I always done my best to keep his trust alive. As a tester, what I believe is that, you should be methodical, systematic, tactful and diplomatic (but firm when necessary).You should be skeptical, especially about assumptions, and wants to see concrete evidence.

2. Tester time spent on other matters

If your testers are busy with different projects (maintenance/deployment phase), then as a test lead, you should consider this point for estimation. This is because if you consider that tester a full resource and he might be busy in some other project, your current application will suffer a lot. And also switching testing tasks from one project to another project frequently reduces the quality of testing. In one of my previous company, as a member of testing team, I was engaged in four different projects simultaneously as a main resource of test team. I tried to convince this point to my management that I am not able to perform well in testing for each application as switching from one project to other project frequently is diverting my mind. Surprisingly, the management agreed on that and done the needful to prevent the testing for all the applications.

3. Values and speed of finding defects

Again, a crucial point for estimation. Although this point is more or less similar to the first point. But there is a difference. If one of your test team member is very quick to find out the bugs in the application, then, definitely that is one of the major resource in your team. That doesn't depend on the efficiency, but on the experience, attitude (Test to Break attitude) and way of working. Now as an experience of around 6 yrs. in testing, I can predict, just after viewing the application, the approximate number of good issues in it.

4. Number of developers working on the project

This is directly proportional to the number of testers in any project. More the number of developers in the project, more the testers you require. Ideally the number of testers should be one-third of the number of developers in any project. i.e. if your application is being developed by 15 engineers, there should be atleast 5 testing engineers for proper testing. Again this equation can be changed if your team have some extraordinary testers. That means if your team have some good and intelligent testers that you can assume as a 1.5 resource (instead of normal 1 resource), you can change the number of testers accordingly. In one of my previous company, I was the only tester for a project where a team of 5 engineers developed the application. I am not saying that was done intentionally but it was due to the management issues. What I have done at that time is to work with developers and tried to get the bugs from them itself. I have done some diplomatic moves at that time and that project was successfully tested and delivered to the customer but not in the required time frame. A smooth working relationship between developers and testers is essential to efficient testing

5. Amount of Functionality

Amount of functionality also decides the number of testers that you have to assign in a particular project. Here amount of functionality means number of functionality that have been implemented for a particular project. This "Amount of Functionality" is somehow related to "Number of developers". More number of functionality that have to be implemented in the project, more number of developers required to develop it and consequently, more number of testers required. As a Testing Lead/Manager, this point should be taken care seriously because the test team member assignment depends a lot on this factor.

6. Developer efficiency at removing defects before test

Ideally, this is the point which should not taken care by the Test Lead/Manager. This is because whatever the efficiency of the developer, as a Test Team, you should work according to your efficiency. This means that you should not take this point as granted and should test the things thoroughly and with your full strength.

7. Developers efficiency at inserting defects

As a Test Lead/Manager, if you are aware of the developers work, then you can estimate the number of testers required to test that specific application.In one of my previous jobs, I worked with a very talented team of developers. Some of the developers (more precisely 3 ) are really brilliant in that team. After completion of that project, I again worked with only those 3 developers for some other project. So at that time, I realized that the development team have worked quite good and they are very less efficient at inserting defects in the application.

Products are better and cheaper to produce when everyone, at every stage in development, is responsible for the quality of their work
If you are not going to identify the defects, the customer would and that will be a gift from the customer that tells you how to test better in the future.

-- Sanat Sharma

Offshore Resources : + or -

Offshore resources are never a quick fix, especially for testing. A 2005 report from AMR Research found that it took between fourteen months and three years before the offshore testers had sufficient familiarity to be effective in finding the root cause of problems. Others have found that the lack of domain knowledge resulted in spurious defects being reported that actually increased development overhead due to review and response times. In fact, one organization identified that as many as 33 percent of reported issues were traceable to tester error. In one of my previous company that I have worked, testing have been outsourced to a different company. And in that scenario, 45 percent of the reported issues found by the outsourced testing team was due to the testing team errors. This is a real time scenario that I have faced. So companies that choose offshore resources as a way to remedy a resource deficiency, or keeping cost cutting in mind, might find themselves with higher costs and resource demands for one or more than one year. Without careful planning and realistic expectations, this confluence may end up eroding quality and extending schedules.

One of the major aspects that is coming out, is the instability of test environments. In almost all testing projects, this is a concern, more so, if environment/development teams are a in different location other than test environment. We have to manage this by placing environment support in the time zone of the offshore test teams and ensure quick turn around time.For the same company that I have worked previously, the testing environment is in the different location from where the testing team was performing. And to resolve this issue, test team has to visit their premises on daily basis spending most of their time in travelling and settling down their. Again not a good practice of offshore testing.

Organizations should not look at offshoring activities from the cost perspective only. Rather, organizations should look at offshore activities where they could develop a 24 hours model of testing/development and other efforts. The change in the mindset would immensely help to decrease the time to release their products to market.For any off-shoring operation, there should be consistent and meticulous planning by the people who are involved. Off shoring cannot fix all the problems. However, with the right push from the management and the right attitude from the people involved, it would be a smashing hit. The first thing that needs to be developed for the people involved in such an operation should be trust and sincerity. The processes,timesheets, effective output, defects tracking index etc., comes next.

Sometimes testing team suggests some improvements to overcome these type of problems. But test process improvements are often rejected simply because the management believe the improvements are nothing more than a self-serving attempt by someone to make a name for himself. They don't believe that the improvements really are meant for the good of the project. I encountered this type of scenario. In one of my previous company, as a senior test team member, I suggested some of the facts that were really helpful to overcome some of the issues that we were facing while testing. And the funny thing was that most of the higher management agreed with my opinion but due to the same point that I have mentioned, (self-serving attempt by someone to make a name for himself) no one respond back for the same. After that, I tried to convince my views in front of the management in other way. I keep producing positive results (rather than focusing on improvements)) with the same processes/infrastructure that they were following. Producing positive results will always be more beneficial than focusing on improvements because it will eliminate the belief that you're offering a new process simply because you don't understand the existing one.With my 6 years of working experience in QA and Testing, I have been involved in QA efforts as part of the off-shore operations for various products. We (myself and my US/UK counterpart) overcame these as challenges and solved everything to a stage where the Off-Shore team could function seamlessly in conjunction with US/UK. The reason behind this is the commitment and sincerity from the both sides to resolve issues. To top all of this we developed mutual trust as the first thing. We did not see each other as problem creators or as competitors, working for the same company.

The outsourced testing team sometimes is not so efficient (The word "efficient" is usually just a code word for "Cheaper" in modern day business environment). The quality of (outsourced) work can also at times be very poor . Some of the reasons for this can be : no domain knowledge, reluctance of offshore team to ask questions (specially when given inadequate specifications), lack of documents, lack of documented (and established) procedures etc. I have worked with developers & testers who are the very best (overseas and in India) and others who can definitely be classified as the 'worst of the worst' (again,overseas and in India). Once I went onsite in one of my project. I met a testing engineer who was physically challenged but that guy was full of energy and a true testing specialist. I really appreciate that guy and his attitude and he is a legend for me in testing domain.

A smooth working relationship between developers and testers is essential to efficient testing. Too much valuable information is unwritten; the tester finds it by talking to developers. Developers and testers must often work together in debugging; that's much harder to do remotely. Developers often dismiss bug reports too readily, but it's harder to do that to a tester you eat lunch with. Remote testing can be made to work - I've done it - but you have to be careful. Budget money for frequent working visits, and pay attention to interpersonal issues.

Software specifications are usually not written to such detail as to someone outside the organization can follow. The resulting software usually falls short of satisfying the specifications. The results can be even more costly in time and money than having the software developed/tested onsite.Overall in my opinion, if the management cannot handle the things properly, offshoring resources is a bad idea.

-- Sanat Sharma