Upstream testing is a process where developers and testers work together from very early in the development cycle - even as features are being built. This is a flavor of an agile development methodology that can be embraced even in a waterfall process.
Introduction
Agile software development teams have reaped significant benefits by adopting the methodology over the waterfall process. However, there are significant challenges in moving from waterfall to agile. Most teams that make the change adopt a home grown hybrid model eventually leading to a complete agile methodology. If you are in a team that is practicing waterfall and you want to be more agile, consider adopting “Upstream Testing” in your current process.
How testing is usually done in waterfall
In ideal waterfall, features are delivered to QA after they are completely built - in one fell swoop. More commonly, features are delivered in interim builds to QA for testing. QA tests the interim build and reports defects on them. These defects are triaged, prioritized and assigned to developers for fixing in future builds. This is usually a slow and expensive process. Often times, the developers have moved on to other features building, sometimes, on the existing defects. Fixing the defect now is more expensive and risky. Additionally, in many situations QA is not able to complete the testing of features developed in the build. As additional builds are delivered, QA is often behind in testing. The later a defect is fixed, the higher the cost. Unfortunately, since the risk of fixing is also higher with the passage of time, many defects are deferred to a later release – which is effectively a “death sentence” for the defect. The reality is that it will never get fixed, resulting in an inferior product to the customer.
Embrace Upstream Testing
What can waterfall organizations to do in the short term to improve the quality of their deliverables? Adopt “Upstream Testing” in your process. “Upstream Testing” is a flavor of agile testing methodology that can be embraced without causing major disruption in existing processes. Very often it can be done without even the involvement of senior management.
In this process, QA works with software development even as features are being iteratively developed. As the developers are checking in code, the QA resources are testing the functionality on daily builds. The daily builds are not formal builds that are handed off to QA with all the bells and whistles. Instead, it is a build that is used by development to test the validity of their integration and check-ins. QA and development work off the same build. The real differentiator in this process is what happens when QA finds an issue. The tester simply reports this to the developer without registering the defect in the corporate defect tracking tool. The developer and tester agree on a fix and it is implemented right away. The tester tests the fix and the developer moves on to other features confident that not only has he unit tested it but that it has even been approved by a tester. When the formal build is done and handed off to QA for intensive and regression testing across many platforms, there are no surprises to QA and quality is already very high.
Note that there aren’t any significant deviations from how development and testing is done. The QA and development teams continue to report to their respective managers. The organization hierarchy is preserved while still injecting agility into the process. Apart from the development and QA teams, hardly any other organization is functionally affected – in contrast to a full agile methodology which has a ripple effect throughout the organization.
Benefits of Upstream Testing
Since the tester and developer are working together, the quality of the deliverable is being monitored continuously – almost on a daily basis. Defects are found and fixed early. It is extremely cost effective to the organization to have this done without going through defect reporting, triages, defect assignment, Change Control Board’s and such ceremonies. Defects that might get deferred are actually fixed – because it is possible to do so. The result is a higher quality product, delivered earlier in a more cost effective manner. The social engineering aspects of “Upstream Testing” cannot be underestimated. The tester gets to understand the complexity of development and the developer begins to appreciate the value of the tester in making the developer look good.
Development and QA start beginning to look like a team working towards a common goal.
Operational Challenges in Upstream Testing
Many organizations are set up functionally. Thus, the QA and development teams report to different directors who may ultimately report to the VP of Engineering. The very nature of this organization sets up different priorities for QA and development. Very often, one of the metrics used to measure the performance of QA teams is the number of defects filed per release of a product. It is not unusual to see colorful graphs of defects found outside the QA director’s office while the development director’s office proudly displays the defects fixed.
In “Upstream Testing” the defects filed are far fewer – because they are caught and fixed informally. The management of QA and development need to arrive at an understanding of a more mature and relevant measure of performance. This can often be challenging and needs good understanding and common goals between development and QA managers.
Conclusion
Upstream Testing is an agile practice that can be fairly easily introduced to teams following waterfall. It is not disruptive and fits in well with waterfall processes. It allows for defects to be caught earlier resulting in cost efficiencies. Apart from higher quality deliverables, the effect of the social engineering it introduces between development and test teams is high. If your team is following a waterfall process and looking to adopt more agile processes, consider “Upstream Testing”. It is an enabler for moving towards full fledged agile methodology in the future, while solving real problems today.
No comments:
Post a Comment