Documenting a project and Customer Satisfaction

The project is a carefully selected set of activities chosen to meet the predefined objectives by using the resources like time, money, people, materials, energy, space, provisions, communication, quality, risk etc. Projects need to be performed and delivered under certain constraints. These constraints are Quality, Resource, Scope and Time (QRST).

Project = Quality + Resource + Scope + Time

(Remember P=Q+R+S+T)

One constraint cannot be changed without impacting the others. i.e.

• Increased Scope typically means increased Time and increased Resources.

• Decreased Time means increased Resource and reduced Scope.

• Decreased Resource mean increased Time and reduced Scope.

The objective of a project is to deliver high performance, reliability and value to the end customer. The end result of a project should be more predictable, mature and defect free performance.

To achieve this, documenting a project is very important.

Presenting a seven step model which provides a good framework for documenting a project :

STEP 1 : DEFINE PROJECT, PURPOSE AND SCOPE

• List down customer needs and expectations (Requirements collection).

• Define the project mission (in not more that a single line).

• Identify and focus on strategically important areas/problems.

• Choose an appropriate project team (hardworking + talented).

• Choose a Team Leader who knows the processes.

• Determine how much progress can be expected with the time.

• Create a basic framework and execution plan for the project.

STEP 2 : MEASURE CURRENT SITUATION

• Understand the present process

• Flow chart the present process and identify the weak/strong points/links in it.

• Collect real time data to identify the real problems

• Standardize the process, if necessary

STEP 3 : CAUSE ANALYSIS

• Dig down for the root cause of the problem [RCA (Root Cause Analysis)]

• Identify the major potential causes and work against them immediately.

• Verify them with data, if possible.

STEP 4 : SOLUTIONS

• Choose between alternative solutions. Better to consult with the Customer at this point.

• Keep solutions simple, efficient and cheaper. The word "efficient" is usually just a code word

for "Cheaper" in modern day business environment.

• Identify barriers to implementing solutions and try to break those as soon as possible.

• Plan and make necessary changes. Use PDCA or Six Sigma techniques.

STEP 5 : RESULTS

• Evaluate the solutions and verify them.

• Collect data, to compare before and after improvement.

• Compare results with what we expected.

• Results should be approximately same as expected.

• If results are more or less than expectations, work on it.

STEP 6 : STANDARDIZATION

• Standardize the new process for future reference.

• Document carefully the changes made.

• Error proof the process.

STEP 7 : FUTURE PLANS

• Review what have been learned from this project

• Decide whether to continue with this project (maintaining part), or

• Close project and move on to a more pressing and challenging project

Quality of a project is calculated on the basis of customer's satisfaction.

Remember "CUSTOMER SATISFACTION IS QUALITY".

Now a question that comes into the picture is that "how do you know that your customer is satisfied ?". A general mindset to measure customer satisfaction is to examine the number of complaints that customer have raised. Now this is a reactive measure. It only responds after the event and it really doesn't measure satisfaction but only dissatisfaction. Financial complaints do not really tell if the customer are any more or less satisfied with the product or services. So how to measure ? Managers and directors often say "if our customers are unhappy, they soon tell us". Well do they ? If on a personal level you rarely write a complaint, what happens as a company level - is it different ?

I am just predicting the expected response of customer in this scenario. I think the customer could not afford the time to complain and they will definitely work for an alternative solution. So measuring levels of customer satisfaction is not only a key element for a project, it is also an important measure of the health of the organization. So action to establish, review and improve customer satisfaction is very important.

There are many modes of survey that can be used to measure the Customer satisfaction till some extent like :

On-line Survey : A link (URL) can be provided to the customer where they can give their views about the project. It should be objective type so that it can be filled in less time. By this way, the customer can complete the survey on-line and give their feedback.

Mail : Send the questionnaires and reply paid envelopes to the customers. Customers use freepost envelopes to return the completed questionnaires to you.

Tele-Interview : Conduct telephonic/video interviews with the customers.

So don't wait until its too late.

-- Sanat Sharma

Outsourcing Software Testing

I will start this blog with the Cem Kaner's explanation of testing which I recently encountered :

"Testing is a process of Technical, empirical investigation, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek."

From the last couple of years, software testing is moving towards a new direction - OUTSOURCING. A company might consider outsourcing testing for several reasons :

1. The company may feel that most testing tasks can be relegated to a contractor.

2. The company may not have the resources - people, skills, software, hardware or time - to perform the testing themselves.

3. The project to be tested may be of such a short life span that the company doesn't need any long-term investment in testing processes for the project.

4. The company might want an independent third party to perform the testing, in order to get a more objective view of quality.

5. The company is under the process of cost cutting.

Now the question is "What are some of the most common mistakes that companies make when outsourcing software testing ?"

Answer - 1 . The biggest mistake is that many times companies outsource both testing and development to the same vendor - thus they have a fox guarding a henhouse. These two activities need to be done by independent groups. The testing team must be able to provide independent , objective feedback on the development process and output.

Answer - 2 . If feedback is to be useful, it's morally like to be helpful if it's rapid. If there is a problem, testing team probably wants to find it and to start solving it as soon as possible. Separating testers and developers, tends lengthen the time in which a problem can be detected, in which it can be discussed, in which it can be demonstrated, in which it can be understood, and in which it can be fixed or retested.

I agree with Answer - 2.

A general tendency is -- What's faster: making a remark to a person next to you, or writing up an email and/or a bug report ? Moreover, feedback based on understanding is typically more useful than the feedback based on oblivion. Knowing how the product is intended to work and how it actually works helps me to find the problems in it more quickly. Knowing the developers and working with them helps immensely in learning about the product.

I worked a lot for product based companies. There, all the developers and testers are under same roof. So it is quite good for the testers to verify the bugs instantly with the help of developers. There as a part of testing team, I was responsible for preventing the bugs as well as finding the bugs. I think this kind of environment is very good for companies also as both teams (testing and development) are working for quality product. I also worked in a company where testing and development part have been outsourced by a different company. But there are some differences in this scenario. When a company outsourced development and testing part to a different company, they are interested only in number of bugs that have been found by testing team and number of bugs fixed by the development team. In this type of scenario, testers and developers are interested only to file the bugs and resolving them. I was in the testing team as one of the major resource of testing. I noticed at that time that management was interested only in number of bugs filed by testing team and number of bugs fixed by development team. This was because the company, that have outsourced the project, was analyzing the work of the team by the number of bugs. This view might be good in terms of finance/funding for both the companies but testing team is not doing their work properly. But anyhow, we have to work according to the processes set by the company. This is something called Voice Of Business.

In one of my previous company that I worked, only testing part have been outsourced by a big MNC. Now since that was the first time that some testing task have been outsourced by that MNC to some other company, nothing was defined as such. My team was responsible for only executing test cases that have been provided by that company. Moreover, we have to automate those test cases and report the test matrix to higher management. If we found any bug while testing, its a long process to justify that issue in front of the management of that company. Besides this, I have to arrange the sessions for all the modules/features that we have to test. This is definitely a time consuming activity. Later we decided to work in their office premise itself. My team started working at their office and this way we tried to resolve the maximum issues and we succeeded.

You may be wondering if outsourcing the testing is right for any organization or not? Well, I would say that it depends on how you are managing the processes. I am not against the outsourcing of testing. Some of the benefits of outsourcing testing include -

1. Lower cost

2. Technological advancement

3. Greater efficiency

4. Staffing Flexibility

5. Improved time-to-market

6. Skill generation and training

7. Focus on Core business

8. Enhanced coverage and quality of delivery.

As a summary, I would like to say that Outsourcing Software Testing is worth only when there are some processes that have been defined and most important the scope of the testing should be clear. This is quite a difficult task but if you want a successful software testing outsourcing, you should think twice. If not, better to perform in-house testing.

-- Sanat Sharma

Test Automation - Pros & Cons

WISHING YOU ALL A VERY HAPPY NEW YEAR 2007 Test Automation is the process to use the software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test controls and test reporting functions. In general, test automation involves automating a manual process already in place that uses a formalized testing process. Test automation is expensive and it is an addition, not a replacement, to manual testing. Test automation raises the hopes of higher management but often frustrates and disappoints the testing team. Although automation promises to deliver a high quality product, implementing automated tests can create as many problems as it solves. Automation needs some bandwidth in initial stage. This should be a full time activity. In one of my previous company, people were allowed to work on test automation on their spare time. And also there were lack of clear goals and experience in the team. And after four months, whatever they have worked on automation, the testing team used that only once. This was because that tool was not worth to test the particular module for that product for a long time. The tool was not robust and dynamic. Before going for Automation, testing team should be clear on what they have to AUTOMATE. Besides this, testing team should keep these points in mind - 1. What is the significance of re-running the test again and again. 2. Its not mandatory to automate each and every test case. 3. First we want to finalize the test cases to be automated. 4. The cost of going for automation. The cost of automating a test is best measured by the number of manual tests it prevents you from running and bugs it will there fore cause you to miss. 5. The life time of our automated test case and how many additional bugs will it be able to find. 6. GUI capture/relay tool like winrunner may be relatively cheaper option. I have observed some advantages of Automation testing. 1. Automated testing is very useful for load,stress or performance testing. 2. It is useful for testing applications in which the GUI is complex and is prone to human error. 3. It is useful for stable areas of the application than volatile areas. 4. It is useful for regression testing. 5. A good tool can fully document and replicate the defect for the convenience of developer. 6. It is a more consistent approach. 7. When we want to execute the test for repetitive time, these tests can be automated. 8. We can reach effective production. 9. Automation testing speeds up regression testing. 10. Automation also eliminates risks associated with human errors. Since advantages are there, so there must be disadvantages also. 1. Automated testing is found to be more time consuming than manual testing 2. Record/playback method is supposed to be the least cost effective method of automated (eg. if unexpectedly a popup window or a message appears then the whole process fails 3. The lifecycle of the test is not long enough.The test needs to be updated everytime there is a new release. 4. There needs to be modification everytime there is a bug fix. 5. Automated testing cannot detect color related issues (eg.mandatory fields) 6. We want to have sufficient time to automate the application I.e. Time consuming. 7. Automation testing is more expensive. 8. Functionality can never be automated. -- Sanat Sharma