Background. Mutation analysis is the premier technique to evaluate software test suite quality and estimating residual software defects. However, the reliability of mutation analysis is hampered by the so called equivalent mutants which are unkillable by any test case. Reliably detecting and eliminating killable mutants is difficult as it is highly program and location dependent. Hence, statistical estimation of killable mutants offers a promising approach.

Aims. Recently, frequency based species estimation methods has been proposed as a solution for related problems in software testing. While promising, there has been no comprehensive study on whether such frequency based species estimation methods can deliver an accurate estimate for killable mutants. Hence, this paper investigates the following: Can the frequency based estimation methods provide an accurate estimate for the number of killable mutants?

Method. In this paper, we report the results of a large-scale empirical study on the application of twelve widely known frequency based statistical models for estimating the number of killable mutants in ten mature software projects.

Result. Our investigation suggests that the considered statistical estimators lack sufficient predictive power and cannot produce reliable or useful estimates of killable mutants.