Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov 7:9:e1625.
doi: 10.7717/peerj-cs.1625. eCollection 2023.

Automated black-box boundary value detection

Felix Dobslaw  1 , Robert Feldt  2 , Francisco Gomes de Oliveira Neto  3
Affiliations

Automated black-box boundary value detection

Felix Dobslaw et al. PeerJ Comput Sci. .

Abstract

Software systems typically have an input domain that can be subdivided into sub-domains, each of which generates similar or related outputs. Testing it on the boundaries between these sub-domains is critical to ensure high-quality software. Therefore, boundary value analysis and testing have been a fundamental part of the software testing toolbox for a long time and are typically taught early to software engineering students. Despite its many argued benefits, boundary value analysis for a given software specification or application is typically described in abstract terms. This allows for variation in how testers apply it and in the benefits they see. Additionally, its adoption has been limited since it requires a specification or model to be analysed. We propose an automated black-box boundary value detection method to support software testers in performing systematic boundary value analysis. This dynamic method can be utilized even without a specification or model. The proposed method is based on a metric referred to as the program derivative, which quantifies the level of boundariness of test inputs. By combining this metric with search algorithms, we can identify and rank pairs of inputs as good boundary candidates, i.e., inputs that are in close proximity to each other but with outputs that are far apart. We have implemented the AutoBVA approach and evaluated it on a curated dataset of example programs. Furthermore, we have applied the approach broadly to a sample of 613 functions from the base library of the Julia programming language. The approach could identify boundary candidates that highlight diverse boundary behaviours in over 70% of investigated systems under test. The results demonstrate that even a simple variant of the program derivative, combined with broad sampling and search over the input space, can identify interesting boundary candidates for a significant portion of the functions under investigation. In conclusion, we also discuss the future extension of the approach to encompass more complex systems under test cases and datatypes.

Keywords: Boundary value analysis; Boundary value detection; Boundary value exploration; Program derivative; Software testing.

PubMed Disclaimer

Conflict of interest statement

The authors declare there are no competing interests.

Figures

Figure 1
Figure 1. AutoBVA framework for automated boundary value analysis.
Figure A1
Figure A1. The type hierarchy for numbers in Julia shows the compatible types for Integer.
Bool is an Integer.
See this image and copyright information in PMC

References

    1. Afzal W, Torkar R, Feldt R. A systematic review of search-based testing for non-functional system properties. Information and Software Technology. 2009;51(6):957–976. doi: 10.1016/j.infsof.2008.12.005. - DOI
    1. Arcuri A, Iqbal MZ, Briand L. Random testing: theoretical results and practical implications. IEEE Transactions on Software Engineering. 2011;38(2):258–277.
    1. Baltes S, Diehl S. Usage and attribution of stack overflow code snippets in GitHub projects. Empirical Software Engineering. 2019;24(3):1259–1295. doi: 10.1007/s10664-018-9650-5. - DOI
    1. Bastani O, Sharma R, Aiken A, Liang P. Synthesizing program input grammars. ACM SIGPLAN Notices. 2017;52(6):95–110. doi: 10.1145/3140587.3062349. - DOI
    1. Borazjany MN, Ghandehari LS, Lei Y, Kacker R, Kuhn R. An input space modeling methodology for combinatorial testing. 2013 IEEE sixth international conference on software testing, verification and validation workshops; Piscataway. 2013. pp. 372–381.

LinkOut - more resources