Impact of non-normal error distributions on the benchmarking and ranking of quantum machine learning models
Problem
Benchmarking quantum machine learning models with Mean Absolute Error learning curves can misrepresent true performance when error distributions are non-normal.
Why it matters
Misleading benchmarks can lead atomistic simulation communities to rank and adopt inferior models.
What the AI did
Analyzed prediction error distributions of a kernel ridge regression model on QM7b atomization energies and showed MAE poorly assesses its true performance, with notable improvement possible.
Results
Showed MAE poorly assesses true KRR-SLATM-L2 performance on QM7b (CCSD(T)/cc-pVDZ); assessment notably improvable (method truncated in abstract).
Impact
Identifies a concrete flaw in standard benchmarking practice for quantum ML models.
Record
| Venue | Machine Learning Science and Technology |
|---|---|
| Published | 2020 |
| Work type | article |
| Authors | Pascal Pernot, Bing Huang, Andreas Savin B |
| Citations | 26 (OpenAlex, at import time) |
| References | 36 |
| Model or method | KRR-SLATM-L2 (kernel ridge regression with SLATM representation, L2 metric) |
| Organization | not stated in source |
| Primary source | https://doi.org/10.1088/2632-2153/aba184 |
| Open access | gold |
| Keywords | quantum machine learning, benchmarking, mean absolute error, kernel ridge regression, QM7b |
Importance rating
incremental / niche
Methodological critique of MAE benchmarking for quantum ML models; relevant to atomistic simulation communities but a narrow, incremental advance with no stated broader impact.