Case Study: Third Party Software Implementation

Client: A large provider of residential high-speed Internet access

Application: A third-party software package to provide the client’s users with Internet security (anti-virus, firewall and parental control). The third party is a leading provider of security services technology to ISPs, broadband providers and telecom companies

Issues: Under an aggressive delivery schedule, the client required a prioritized report of defects identifiable in both a test environment and in the field. In addition, they sought information to assist in the evaluation of the business risk of launching the application on schedule. Finally, we were asked to provide the client's support group with a list of issues to be expected during deployment. At the start of the project the client had no formal test process in place.

ProtoTest’s Solution: ProtoTest quickly developed an approach to obtain the necessary information, identify issues and make a final recommendation. The test team consisted of a Project Lead, a Test Lead, and a Tester in addition to 85 volunteer end users (assembled by ProtoTest). A formal software test process was developed and documented. Elements of the process included Test Planning, Test Development, Test Execution, and Issue Management. Test cases were based on requirements derived by ProtoTest. A number of test cases were developed based upon questions we posed to the client (e.g., interoperability was not claimed with Peer-to-Peer (P2P) clients but we determined P2P should be tested as it represents approximately 40% of the client’s traffic).

Testing of the third party application was performed in two parts:

Controlled Test Group——This team developed the test plan and functional test requirements, then designed and executed test cases. The Controlled Test Group developed and managed the tests and reporting of the High Speed Data Test Group. The Controlled Test Group also worked alongside the third party provider's QA and Development teams to resolve issues found while testing.

High Speed Data Test Group——This team consisted of approximately 85 volunteer end-users on the client’s system. The High Speed Data Test Group was given a core set of tests to run and was expected to report results to the Controlled Test Group. The High Speed Data Test Group was encouraged, but not required, to run and report on tests beyond the core set provided.

Results:

• The testing project was completed on time and on budget.
• A larger than expected number of significant defects were uncovered.
• Recommendations were made based on risks and expected results of deployment.
• Cost savings for the client and the third-party provider were achieved by delaying the application launch until serious issues were resolved, finding and fixing defects in the application, and avoiding a potentially large number of support calls.