Handling Intermittent Bugs

The biggest challenge a QA Testing Engineer faces is to reproduce intermittent issues, also commonly called 'non-reproducible' bugs. In many cases, when we report an intermittent issue, the development team requests  further details as the information provided is not enough to reproduce them. The developers want us to find the exact scenario, environment and cause for their occurrence. There lies the biggest challenge which we, as testers, have to face.  Intermittent issues are usually complicated scenarios and require a lot of effort to precisely reproduce them. Thus, root cause analysis of such issues becomes more challenging. During Software Quality Assurance and Testing we report intermittent issues which only 1% of the users may face; however, their detection and fixing goes a long way in ensuring customer satisfaction and user experience for both novices and experts. Replicating intermittent issues is a daunting task which requires a lot of patience and a keen eye for details to prove that the bug is valid – even if under rare circumstances.

Whether we accept it or not, such errors reflect our inefficiency as testers. If the frequency of non reproducible intermittent bugs goes up, it may ruin the rapport between testers and developers. It may also impact business as there will be higher chances of escape issues that remain in the shipped product.  So, it is in the best interest of the product quality that we need to be extra cautious of such evasive bugs.

Reasons for intermittent behavior:
• Tester’s lack of attentiveness while testing programs/applications

• Build-specific bugs which would never re-occur once a stable build is generated

• Misinterpreting a particular observation by the tester

• Assuming to be insignificant, testers may neglect the subtle differences in the application’s environment which lead to  non reproducible bugs on specific OS or browser platform combinations
Thus, reasons for intermittent issues occurring or being reported can be many, and similarly, there are a lot of ways of reproducing such issues as well. Based on our QA and Testing experience at QA InfoTech, here are a few best practices which testers should follow in order to ensure high product quality, minimum escapes and healthy Cross Functional Team (CFT) relationship:

• One should take care of the environment while investigating intermittent problems. This consists of physical setup which includes hardware and logical setup comprising of server operating system, client operating system, database server, browser or other components required to run the application

• All  logs should be kept for reference so as to compare the new logs with old ones

• Seek some tool which can record the steps being performed on the machine. For example, on Windows 7, the in-built tool ‘Problem Steps Recorder’ can record each step performed on the system along with screenshots. This can be of great help in figuring out the steps performed earlier

• It is very important to have patience; investigate the issue as many times as possible because these issues cannot be manifested on demand

• As always, many problems can be solved by taking help from colleagues. Through proper commmunication with other team members, we can identify intermittent behaviour

• Try to find out more generalized conditions and do not simply rely on test cases

• For issues which show up for the first time after the product is installed, we need to be particular about using clean systems to replicate such scenarios

• In QA Testing some bugs are data specific; therefore, it is imperative to ensure that we use the same data comprehensively while trying to reproduce the bug

• Since intermittent issues might depend upon series of related tasks in specific order, we need to be always cautious about preconditions for the occurrence of such issues

• Sometimes intermittent issues might also crop up because of interaction of the application with another software running in the background that requires the same resources. Such co-existence issues might lead to bugs which otherwise would not occur when reproduced in isolation

• Develop a good rapport with your CFT. Often times talking to them to explain the intermittent issue you saw might throw open other ideas to help reproduce it

• Practice troubleshooting. When teams have some down times between projects, it would be a good challenge to have them take on some practical troubleshooting exercises which help develop the tester’s critical thinking and analytical capabilities.

• Lastly, for any Software Quality Assurance project it’s important to have an eye for detail; that is what differentiates the best tester from a good one

Additional articles about QA Testing

About the author

Knowledge Sharing Team at QA InfoTech is responsible for research on new tools & technologies for effective Software Testing services across all industry verticals. QA InfoTech (an ISO 9001:2008 & CMMI III certified) is one of the leading Software Quality Assurance and Testing Companies with five testing “Centers of Excellence” in the United States and India. It offers end to end Software QA Testing services.

Please Rate This Article

Select Rating Excellent Very Good Interesting Fair Could Use Improvement

Number of ratings: 4
Rating: 4