We present progress on applying randomised testing techniques to the Dafny compiler, to automatically identify miscompilation bugs and compiler failures. We have built two complementary tools - fuzz-d, which is principally based on differential testing, and DafnyFuzz, which is principally based on metamorphic testing. These tools in turn complement an existing tool, XDsmith, that was previously used to successfully test the Dafny compiler. Together, these tools have been used to find and report 30 distinct bugs affecting the Dafny compiler, including a number of soundness bugs. Most of these bugs are beyond the reach of the existing XDsmith tool. We have also conducted experiments looking at statement and mutation coverage on the Dafny compiler source code. Our results show that fuzz-d yields complementary coverage compared with XDsmith and the Dafny compiler regression test suite.
Sun 14 JanDisplayed time zone: London change
11:00 - 12:30 | |||
11:00 18mTalk | Dafny Test Generationremote Dafny Aleksandr Fedchin Tufts University, Jeffrey S. Foster Tufts University, Zvonimir Rakamaric Amazon Web Services, Aaron Tomb Amazon Web Services | ||
11:18 18mTalk | Randomised Testing of the Dafny Compiler Dafny Alastair F. Donaldson Imperial College London, Dilan Sheth Imperial College London, Jean-Baptiste Tristan Amazon Web Services, Alex Usher Imperial College London | ||
11:36 18mTalk | Teaching Logic and Set Theory with Dafny Dafny Ran Ettinger NVIDIA and Ben-Gurion University of the Negev and Israel Academic College in Ramat Gan, Hezi Daniel Israel Academic College in Ramat Gan | ||
11:54 18mTalk | Learn 'em Dafny Dafny James Noble Creative Research & Programming Pre-print | ||
12:12 18mTalk | Colouring Flags with Dafny & Idris Dafny |