Formal Systems Laboratory

Improved Multithreaded Unit Testing

Abstract. Multithreaded code is notoriously hard to develop and test. A multithreaded test exercises the code under test with two or more threads. Each test execution follows some schedule/interleaving of the multiple threads, and different schedules can give different results. Developers often want to enforce a particular schedule for test execution, and to do so, they use time delays (sleep in Java). Unfortunately, this approach can produce false positives or negatives, and can result in unnecessarily long testing time. This paper presents IMUnit, a novel approach to specifying and executing schedules for multithreaded tests. A new language is introduced that allows explicitly specifying schedules as constraints on the events during test execution. A tool that automatically controls the code to execute the specified schedule is provided, as well as a tool that helps developers to migrate their legacy, sleep-based tests into event-based tests in IMUnit. The latter tool uses novel techniques for inferring events and schedules from the executions of sleep-based tests. Experience in migrating over 200 tests is described. The inference techniques have high precision and recall, of over 75%. IMUnit reduces testing time compared to sleep-based tests on average 3.41x.