False Positives and False Negatives!

False Positives and False Negatives because of improper environment set up

Let’s begin with a real life scenario, I went to a doctor for a regular check up and I got to know that I had been suffering from ‘Cancer’. I had never imagined that I would be suffering from CANCER   I believed it just because the report said. Now my friend Mr. XXX consoled me and told me to visit the doctor again and tell him that I wanted a Recheck up. I found his idea a good way to confirm if I was really suffering from CANCER. I went to the doctor and after we were done with Recheck up, surprisingly but luckily I had no CANCER, actually there were some flaw in the instruments which generated a false result.

This is a false positive, because on account of some flaw in the instruments, I had been informed that I had CANCER.

Why called False Positive?

Expected Result as per me: I was totally fine and healthy.

Actual Result as per Doctor: I had CANCER.

As there is a deviation between the ‘Expected Result’ and ‘Actual Result’, it means there is a failure as per me.

When I asked the doctor what I would call such a fake report then he suggested me to call it  a ‘False Positive’. False because it was not as per my expectation and Positive because doctors are born to identify problems and symptoms in our body which my doctor has done by giving me a report of CANCER which he believed was true because he referred to a medical report.

Now what would be a False Negative??

A test which is marked as PASS but there still is a problem and under ideal conditions the test should FAIL.

It means if I had CANCER and the doctor left me by saying that I am fine just because of some instrumental flaw, then that would be called as a ‘False Negative’. 

Now let’s come to the Software’s False Positives and False Negatives scenarios.

We a group of software testers are testing an application ‘A’ under a set of environments say B, C, D etc.  We set up the environments properly and we started testing our AUT (Application under Test) under the desired environment. We started finding bugs and reported simultaneously. But over a week we noticed that the ‘Defect Rejection Rate’ was too high as per previous statistics. The development end people just updated their status as ‘Working Fine at Our End’. Then we planned to reset the environments and did it accordingly but with more cautions this time. Now all our own issues started to pass at our end also.

This was ‘False Positive’ in software testing. False because ‘Actual and Expected’ were not matching.  And Positive because we found the bugs at our end (Our job as a tester is to find bugs and we did it so a positive for our side).

But it is for granted that if we found some false positives then there must be some false negatives.  It means we passed some tests just because of environment set up issue. This is where the quality of software is affected. Because False positive when goes for fixing or debugging to the development guys they found it working and hence we can close the issue after proper environment set up and retesting.

But what will happen to the False Negatives that we didn’t find and hence didn’t report because it was working fine in our environment which was not properly set up?

 
                                We should always try to cross check the environment set up by us and the environment needed to test the application. Once we commit mistake at this point, it means we already degraded the quality of application because it would give rise to a lot of false positives that leads to wastage of time and money (resources)  for debugging and retesting and similar number of false negatives which are very difficult to find out, because no one notices it.