Challenges in Testing Large Language Model-Based Software: A Faceted Taxonomy

6/10 importance · partially solved · software testing · 2026 · validator confidence 70% · research

The problem

LLM-based software is non-deterministic, so traditional testing cannot verify correctness within individual test cases.

Why it matters

LLMs are becoming integral to software systems, making undetected output variability a correctness risk.

What the AI did

Provides a four-faceted taxonomy of LLM test case design, an LLM-assisted analysis of six open-source testing frameworks, and a sensitivity study of an agent-based system across model configurations.

Results

Taxonomy with four facets; analysis of six open-source testing frameworks; sensitivity study across model configurations

Impact

Organizes the field and identifies open challenges; does not solve LLM software testing.

How important is this problem?

6/10 — significant — thousands to millions affected

Correctness verification for LLM-based software is a foundational challenge as LLMs become integral to software systems; abstract frames it as a field-wide open problem but gives no scale.

inferred — an explicit judgement from the source text.

Scale stated in source: not stated in source

Ratings use a published rubric and are AI-assisted, not measured or peer-reviewed. Method

Sources & verification

Primary source: https://doi.org/10.1145/3806396
Model / method: not stated in source
Organization: not stated in source
Validator notes: glm-5.3: ok (confidence 70) | Credible venue; taxonomy paper, open challenges remain unsolved.
Keywords: LLM testing, non-determinism, test case design, taxonomy, multi-agent LLMs

A second-stage review judged this entry against the primary source; status “verified”. “Verified” means the evidence supported the claim — it does not mean the underlying result was independently reproduced.

ImageNet classification with deep convolutional neural networks
Accurately classifying 1.2 million high-resolution images into 1000 categories, a task previous methods handled poorly.

Software & Coding · 2017 · solved · 8/10 importance
Deep Residual Learning for Image Recognition
Very deep neural networks were difficult to train, blocking accuracy gains from increased depth in visual recognition.

Software & Coding · 2016 · solved · 8/10 importance
Faster sorting algorithms discovered using deep reinforcement learning
Improving the efficiency of fundamental sorting routines had stalled for both human scientists and computational approaches.

Software & Coding · 2023 · solved · 8/10 importance
A deep learning‐based approach for software vulnerability detection using code metrics
Detecting software vulnerabilities traditionally depends on human experts manually defining features, which is tedious and subjective.

Software & Coding · 2022 · ai assisted · 7/10 importance
Deep Learning-Based Attack Detection and Classification in Android Devices
Detecting and classifying malware attacks on Android devices, which dominate the global market, remains an ongoing challenge.

Software & Coding · 2023 · ai assisted · 7/10 importance