Automated Testing of Refactoring Engines Using Test Abstractions

Опубликовано 6 сентября 2016, 18:01

The standard approach to generating test suites, manual generation of the test inputs in the suite, is tedious and error-prone, especially for code that takes complex test inputs. This talk presents our approach that automates generation of test suites using test abstractions. Conceptually, test abstractions provide a high-level description of desired test suites; developers do not need to manually write large suites of individual tests but instead write test abstractions from which tools automatically generate individual tests. This approach has helped developers in both academia and industry to discover errors in several real applications. This talk illustrates the approach on testing refactoring engines using imperative test abstractions. Refactorings are behavior-preserving program transformations that improve program design, and refactoring engines are tools that automate the application of refactorings. Refactoring engines are a key component of modern IDEs such as Visual Studio, Eclipse, or NetBeans. A bug in a refactoring engine can have severe consequences as it can erroneously change large bodies of source code. Test inputs for refactoring engines are programs, and the core of our testing of refactoring engines is a framework for generation of abstract syntax trees that represent Java programs. Using this framework, we discovered 21 new bugs in Eclipse and 24 in NetBeans. We didn't test Visual Studio. This is joint work with Brett Daniel, Danny Dig, Kely Garcia, Vilas Jagannath, and Yun Young Lee