Functional Test Oracle Engineering in Practice: Challenges, Needs, and Practitioner Insights

Abstract

Functional test oracles are essential for determining whether a software system behaves as expected, yet building and maintaining them remains difficult in practice. Through insights from software practitioners, our work examines how functional test oracles are created, the challenges teams encounter, and the support they need. Our findings identify recurring obstacles in specifying expected behavior, maintaining tests as systems evolve, and handling complex user-interface interactions, and inform opportunities for more practical test-oracle engineering tools.