Changebased test selection an empirical evaluati

Authors: Quinten David Soetens Serge Demeyer Andy Zaidman Javier Pérez

Publish Date: 2015/11/11

Volume: 21, Issue: 5, Pages: 1990-2032

PDF Link

Abstract

Regression test selection ie selecting a subset of a given regression test suite is a problem that has been studied intensely over the last decade However with the increasing popularity of developer tests as the driver of the test process more finegrained solutions that work well within the context of the Integrated Development Environment IDE are in order Consequently we created two variants of a test selection heuristic which exploit finegrained changes recorded during actual development inside the IDE One variant only considers static binding of method invocations while the other variant takes dynamic binding into account This paper investigates the tradeoffs between these two variants in terms of the reduction ie How many tests could we omit from the test suite and how much did we gain in runtime execution as well as the fault detection ability of the reduced test suite ie Were tests omitted erroneously We used our approach on three distinct cases two open source cases —Cruisecontrol and PMD— and one industrial case — Historia Our results show that only considering static binding reduces the test suite significantly but occasionally omits a relevant test considering dynamic binding rarely misses a test yet often boils down to running the complete test suite Nevertheless our analysis provides indications on when a given variant is more appropriateThis work has been sponsored by i the Interuniversity Attraction Poles Programme Belgian State Belgian Science Policy project MoVES ii the Institute for the Promotion of Innovation through Science and Technology in Flanders IWTVlaanderen under project number 120028 entitled “Changecentric Quality Assurance CHAQ” iii the NWO TestRoots project the Netherlands

Keywords: