I suggest a different approach: analyze your risks
. All the risks in software development fall into two types: technical risks (the product will not work properly) and business risks (when a technical failure may influence the business). Common risk factors for software development projects are: 1. Business-related Quality risks:
2. Development-side Quality risks:
- Affect of the failure on company revenue
- Affect of the failure on the product churn rate
- Number of customers using the product day-to-day
- Number of other products using the output of the system
- Sophistication of end users
- Subject-area quality standards compliance (medical or government software)
- Costs associated with late delivery
- Visibility of the product quality by senior management
- Unexpected Requirements changes
- Misestimating Workload
- Turnover of project team members
- Engineering compromises
- Fresh, unproven technologies
- Bad project management
- Test environments instability
- Team experience (specific toolset)
- Lack of communication or documentation
- Changes in the functionality with new releases
If technical and business risks on the project are low
, there is no big need in structured testing at all. The best you can do is to conduct some manual exploratory testing and review with stakeholders, that's it.
If your analysis indicates medium
or above average risks it's time to focus on unit testing. Also add some manual black-box testing including test analysis, design, review and support of the tests.
But if your risks are high
, make sure you have the following areas covered: 1. Control your manual testing:
2. Use certain amount of automation on multiple layers:
- Each test case is detailed and stored in test management system, grouped in test suites.
- All test cases are reviewed and formally signed off by Development and Product Management team representatives.
- All the manual testing is accounted in TMS so you always control what, when and how is tested.Whenever the issue is missed, you can analyze the reasons and identify underperforming engineers.
- AdHoc testing is allowed only for product managers and developers. QA is not allowed to test adhoc.
- Whitebox tests (Unit, Integration, System level)
- Blackbox tests (UI and API logic)
- Performance test automation (Load and Volume tests)
3. Assess System Security on the regular basis.
Working approach there is the principle of diverse half-measures
: apply a bunch of methods instead of investing 100% of your budget in manual testing. This will save you much costs.
Understanding the risks not only helps you to mitigate them, you will be more confident in the budget to spend for a certain area of the project.