Case Study: QA Process Consulting

Client: An international manufacturer of industrial and automotive products

Application: An ERP System and underlying Oracle database management system

Issues: The QA team was operating in fire-fighting mode, with never enough time to do things "right." Relationships with Development and Customer Support had deteriorated. Too-frequent releases were inadequately tested, resulting in too many defects delivered to users. The QA process was in question, and management was not sure they had the right mix of skills on the team.

ProtoTest’s Solution: Within 8 weeks we performed a full QA process evaluation including staff assessment. We interviewed staff members in QA, Development and Customer Service to find out about the current process, current relationships and barriers to doing an effective job. A report was provided that detailed what was effective in the existing QA process, what was not working, and what changes were recommended. A roadmap was provided detailing steps to take to bridge the gaps. One recommendation was to reduce the frequency of releases to production gradually, while ramping up the quality of each release.

Each team member was given a software QA knowledge test. ProtoTest took the information gathered in the staff interviews and tests, plus information gathered from observing the team members in action, and created an assessment report. Each team member was assessed, with supporting documentation describing individual and team skill gaps and recommendations for training.

The QA team used the process recommendations and staff assessment to establish a team charter and meaningful quality goals. The charter and goals were shared with Development and Customer Support for review and approval.

Results:

• From "fire fighting" to "fire prevention." The QA team no longer feels they are in fire-fighting mode. Their customers (ERP software users) have not objected to the reduced frequency of production releases. Quality has improved, as evidenced by the fact that they now go 4 weeks at a time without having to do an emergency patch outside of the regular release cycle.
• Relationships improve. A process change was recommended regarding the way trouble tickets were handled between Support, Development, and QA. This change has been implemented, and now tickets are turned around faster and the quality of the work has improved. This has also helped improve the relationship between Development and Test because they cooperate directly when tickets come in.
• Risk reduced. Because of process improvements the team has implemented, they are better able to assess risks for each release and implement a testing strategy to mitigate them.
• Test planning accelerated. This was an area in the process that simply took way too long. ProtoTest provided a template to incorporate needed information and eliminate redundancy. A checklist is used in place of a full test plan on small testing efforts, which also saves time without increasing risk.
• Better skills balance. Our training and staffing recommendations helped the team survive when one of their most senior members shipped out with the Reserves to Iraq. They were without this individual for a few months, during which the other team members were able to cover effectively.