Enriching automatic test case generation by extracting relevant test inputs from bug reports

Wendkûuni C. Ouédraogo, Laura Plein, Kader Kaboré, Andrew Habib, Jacques Klein, David Lo, Tegawendé François Bissyandé

Empirical Software Engineering 2025 article cited by 16

Source Open-access PDF More in Software & Coding

importance 4/10 partially solved Software Testing and Debugging Techniques software testing stage: research

Problem

Automated test generation tools struggle to produce relevant test inputs, limiting the bug-detection effectiveness of generated tests.

Why it matters

Weak automated tests let software bugs slip through, degrading software quality.

What the AI did

BRMiner combines Large Language Models with traditional techniques to extract relevant test inputs from bug reports and feed them to test generation tools such as EvoSuite and Randoop.

Results

On Defects4J benchmark: Relevant Input Rate 60.03% and Relevant Input Extraction Accuracy Rate 31.71%, significantly outperforming LLM-only methods.

Impact

Improved relevant input extraction over LLM-only baselines on a standard benchmark; no field adoption stated.

Record

Venue	Empirical Software Engineering
Published	2025
Work type	article
Authors	Wendkûuni C. Ouédraogo, Laura Plein, Kader Kaboré, Andrew Habib, Jacques Klein, David Lo, Tegawendé François Bissyandé
Citations	16 (OpenAlex, at import time)
References	64
Model or method	LLMs combined with traditional techniques (specific LLM not stated in source)
Organization	not stated in source
Primary source	https://doi.org/10.1007/s10664-025-10635-z
Open access	hybrid
Keywords	LLMs, test case generation, bug reports, Defects4J, EvoSuite

Importance rating

4 / 10

real but narrower advance

Improves automated test generation and bug detection on a benchmark; real but narrower software-quality advance, no field adoption stated.

Basis: inferred. Judged from the source text, which gives no explicit scale. Stated scale: not stated in source. Method and limits.

More in Software & Coding

#	Paper	Year	Citations	Importance
1	ImageNet classification with deep convolutional neural networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton	2017	106,345	8
2	Deep Residual Learning for Image Recognition Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun	2016	221,134	8
3	Faster sorting algorithms discovered using deep reinforcement learning Daniel J. Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco S...	2023	175	8
4	A deep learning‐based approach for software vulnerability detection using code metrics Fazli Subhan, Xiaoxue Wu, Lili Bo, Xiaobing Sun, Muhammad Rahman	2022	42	7
5	Deep Learning-Based Attack Detection and Classification in Android Devices Alfonso Gómez, Antonio Muñoz	2023	41	7