Property based testing has been used as a classical testing methodology, but it has promise within the context of quantum testing so long as expressive assertions are developed. The complexity of quantum algorithms poses a serious challenge when developing effective test cases. Property based testing can automate this complex task, only requiring the user to provide simple properties of the program, while the concrete test cases with inputs can be automatically generated. In our repository we produce a document containing properties of each program, which may be used when implementing property-based tests for the suite of benchmarks.
While working on the benchmark suite we gained insight into the programming languages used, in the last section of our paper, the findings and observations made relevant to testing are noted. Things such as the performance difference between the languages, carrying out different tasks in each language and how they affect testing quantum programs.
Looking ahead: the next steps
We plan to expand upon this work going forward in a few ways:
- Increasing the number of programs in the benchmark suite.
- Increasing the number of programming languages that the benchmarks cover.
- Develop a robust framework for benchmark program selection, test execution, mutant generation, and property-based testing.
- Port classical testing techniques, such as property-based testing, with the help of the benchmarks.
A good set of benchmarks will reduce the time it takes to develop and improve testing methodologies for quantum programs. Improving quantum software engineering practices such as testing will allow for more quantum programs to be developed, reduce the effort required to maintain quantum programs, and make it easier to teach and learn quantum programs.
To find out more about this exciting project you can read the full paper here.