Luciano Gerber

Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression Smooth-basis models such as Chebyshev polynomial regressors and radial basis function (RBF) networks are well established in numerical analysis. Their continuously differentiable prediction surfaces s...

Paper: arxiv.org/abs/2602.22422
Code: github.com/gerberl/poly...

GitHub - gerberl/poly-erbf-benchmark: Benchmark: Chebyshev polynomials and ellipsoidal RBF networks for tabular regression - results, figures, and reproducibility code Benchmark: Chebyshev polynomials and ellipsoidal RBF networks for tabular regression - results, figures, and reproducibility code - gerberl/poly-erbf-benchmark

Everything is open source and pip-installable:

pip install erbf # anisotropic RBF network
pip install poly-basis-ml # Chebyshev regressor + model tree

Both sklearn-compatible. Benchmark code and all results: github.com/gerberl/poly...

The practical message: put smooth models in the mix when you benchmark. The default "just use XGBoost" is reasonable but can leave desirable properties on the table - tighter generalisation, differentiable surfaces, inspectable coefficients.

The accuracy-gap Pareto front puts it visually. Smooth models occupy the best trade-off positions among CPU models.

The gap can signal how much of your training fit transfers. A tighter gap means less dependence on the specific training sample - relevant whenever stability matters. See Bousquet and Elisseeff 2002 (www.jmlr.org/papers/v2/bo...)

On generalisation gap: smooth models consistently show tighter gaps. When accuracy is matched (within 0.02 R^2), smooth models win on gap in 87% of pairwise comparisons with tree ensembles.

On predictive accuracy (adjusted R^2): the five competitive CPU models - ERBF, ChebyTree, XGBoost, ChebyPoly, RF - are statistically tied (Friedman/Nemenyi).

We benchmarked:
- Tree ensembles: XGBoost, Random Forest
- Smooth: anisotropic RBF network + Chebyshev polynomial regressor
- Hybrid: Chebyshev model tree
- Baselines: Ridge, Decision Tree
- Also TabPFN - thread for another day

Were we trading accuracy for these properties?

We'd noticed smooth models tend to show tighter generalisation gaps - smaller train-test differences. And you get a differentiable prediction surface, useful for surrogate optimisation, sensitivity analysis, and anywhere small input changes should produce proportionate outputs.

But we're also quite fond of smooth models - Chebyshev polynomials, RBF networks. Huw has used them in surrogate optimisation; I've relied on them for seismic and vibration signal modelling.

We're both fans of tree ensembles. XGBoost is a solid default - tunes fast, trains fast, performs well. We've used them extensively (e.g., in pricing).

Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression Smooth-basis models such as Chebyshev polynomial regressors and radial basis function (RBF) networks are well established in numerical analysis. Their continuously differentiable prediction surfaces s...

Smooth models over XGBoost/Random Forest for tabular regression - does it really mean trading accuracy for smoothness? My colleague @huwlloyd.bsky.social and I tested it: 55 datasets, 8 models, nested CV.

Preprint: arxiv.org/abs/2602.22422