Test Oracles

To test any program, we need to have a description of its expected behavior and a method of determining whether the observed behavior conforms to the expected behavior. For this we need a test oracle.

A test oracle is a mechanism, different from the program itself, which can be used to check the correctness of the output of the program for the test cases. Conceptually, we can consider testing a process in which the test cases are given to the test oracle and the program under testing. The output of the two is then compared to determine if the program behaved correctly for the test cases, as shown in Figure 1.

Test oracles are human beings, so they may make mistakes when there is a discrepancy between the oracles and the results of a program. First we have to verify the result produced by the oracle before declaring that there is a fault in the program, that’s why testing is so cumbersome and expensive.

The human oracles generally use the specifications of the program to decide what the “correct” behavior of the program should be. To help the oracle to determine the correct behavior, it is important that the behavior of the system be unambiguously specified and the specification itself should be error-free.