Full Benchmark Report

Generated: 2026-01-05T20:18:21.110509+00:00

Environment

Property	Value
Machine	arm
Cores	10
Memory	32.0 GB
OS	Darwin 25.1.0
Git SHA	42da160

Library Versions

Library	Version
Python	3.12.9
boosters	0.1.0
xgboost	3.1.2
lightgbm	4.6.0
numpy	2.4.0

Configuration

• Suite: full

• Seeds: 5

• n_estimators: 100

• learning_rate: 0.1

• max_depth: 6

• growth_strategy: leafwise

• max_bins: 256

• min_samples_leaf: 1

• reg_lambda (L2): 0.0

• reg_alpha (L1): 0.0

• linear_l2: 0.01

• booster_types: gbdt, gblinear, linear_trees

Results

Regression

california (primary: rmse)

Booster	Library	rmse	train_time_s
gbdt	boosters	0.4071±0.0050	0.0901±0.0009
gbdt	lightgbm	0.4087±0.0061	0.1358±0.0014
gbdt	xgboost	0.4089±0.0057	0.1270±0.0016
gblinear	boosters	0.7370±0.0243	0.0297±0.0003
gblinear	xgboost	0.7370±0.0243	0.0454±0.0010
linear_trees	boosters	0.4573±0.0756	0.1695±0.0014
linear_trees	lightgbm	0.9466±0.5673	0.2170±0.0059

california_nan (primary: rmse)

Booster	Library	rmse	train_time_s
gbdt	boosters	0.4771±0.0058	0.1161±0.0011
gbdt	lightgbm	0.4767±0.0079	0.1844±0.0011
gbdt	xgboost	0.4742±0.0057	0.2402±0.0012
gblinear	boosters	0.9872±0.0145	0.0503±0.0014
gblinear	xgboost	0.9872±0.0145	0.0497±0.0022
linear_trees	boosters	0.4791±0.0067	0.2044±0.0009
linear_trees	lightgbm	0.8171±0.7022	0.2618±0.0015

california_weighted (primary: rmse)

Booster	Library	rmse	train_time_s
gbdt	boosters	0.5046±0.0092	0.0878±0.0007
gbdt	lightgbm	0.5022±0.0115	0.1351±0.0012
gbdt	xgboost	0.5052±0.0090	0.1261±0.0014
gblinear	boosters	0.9239±0.0748	0.0299±0.0002
gblinear	xgboost	0.9239±0.0748	0.0467±0.0010
linear_trees	boosters	0.5320±0.0452	0.1665±0.0018
linear_trees	lightgbm	0.9490±0.5352	0.2146±0.0015

synthetic_reg_medium (primary: rmse)

Booster	Library	rmse	train_time_s
gbdt	boosters	0.1710±0.0039	0.5810±0.0114
gbdt	lightgbm	0.2046±0.0029	0.8111±0.0068
gbdt	xgboost	0.2043±0.0036	1.6596±0.0165
gblinear	boosters	0.0005±0.0000	0.1497±0.0005
gblinear	xgboost	0.0005±0.0000	0.2285±0.0014
linear_trees	boosters	0.0909±0.0057	0.6808±0.0048
linear_trees	lightgbm	0.0927±0.0020	0.8977±0.0183

synthetic_reg_small (primary: rmse)

Booster	Library	rmse	train_time_s
gbdt	boosters	0.2352±0.0199	0.1771±0.0014
gbdt	lightgbm	0.2891±0.0161	0.2771±0.0018
gbdt	xgboost	0.2903±0.0163	0.6250±0.0188
gblinear	boosters	0.0006±0.0000	0.0152±0.0002
gblinear	xgboost	0.0006±0.0000	0.0231±0.0002
linear_trees	boosters	0.1722±0.0071	0.1916±0.0011
linear_trees	lightgbm	0.1891±0.0065	0.3208±0.0042

Quantile Regression

liander_energy_forecasting (primary: rcrps)

Booster	Library	rcrps	train_time_s
gbdt	boosters	0.0708±0.0134	1.1056±0.0187
gbdt	lightgbm	0.0652±0.0138	0.7538±0.0106
gbdt	xgboost	0.0604±0.0112	2.7766±0.0563
gblinear	boosters	0.0684±0.0072	0.1769±0.0009
gblinear	xgboost	0.0735±0.0063	0.3327±0.0373
linear_trees	boosters	0.0711±0.0141	1.2970±0.0051
linear_trees	lightgbm	0.0831±0.0133	0.9027±0.0089

Binary Classification

breast_cancer (primary: logloss)

Booster	Library	logloss	train_time_s
gbdt	boosters	0.0983±0.0307	0.0326±0.0010
gbdt	lightgbm	0.0866±0.0270	0.0199±0.0002
gbdt	xgboost	0.0978±0.0427	0.0444±0.0011
gblinear	boosters	0.1252±0.0397	0.0032±0.0001
gblinear	xgboost	0.1252±0.0397	0.0051±0.0002
linear_trees	boosters	0.0948±0.0319	0.0345±0.0010
linear_trees	lightgbm	0.0911±0.0207	0.0235±0.0005

synthetic_bin_medium (primary: logloss)

Booster	Library	logloss	train_time_s
gbdt	boosters	0.1736±0.0107	0.5471±0.0053
gbdt	lightgbm	0.1745±0.0125	0.6025±0.0422
gbdt	xgboost	0.1745±0.0130	1.0021±0.0187
gblinear	boosters	0.5063±0.0040	0.1623±0.0011
gblinear	xgboost	0.5063±0.0040	0.2262±0.0008
linear_trees	boosters	0.1787±0.0145	0.6598±0.0051
linear_trees	lightgbm	0.1781±0.0121	0.6371±0.0154

synthetic_bin_small (primary: logloss)

Booster	Library	logloss	train_time_s
gbdt	boosters	0.2910±0.0456	0.1520±0.0025
gbdt	lightgbm	0.2768±0.0405	0.1536±0.0056
gbdt	xgboost	0.2722±0.0349	0.2720±0.0075
gblinear	boosters	0.6017±0.0258	0.0170±0.0001
gblinear	xgboost	0.6017±0.0258	0.0233±0.0003
linear_trees	boosters	0.2926±0.0461	0.1599±0.0016
linear_trees	lightgbm	0.2850±0.0520	0.1630±0.0030

Multiclass Classification

covertype (primary: mlogloss)

Booster	Library	mlogloss	train_time_s
gbdt	boosters	0.4369±0.0066	2.8936±0.1540
gbdt	lightgbm	0.4285±0.0074	1.8948±0.0143
gbdt	xgboost	0.4698±0.0064	2.2280±0.0237
gblinear	boosters	0.7532±0.0117	3.0599±0.0405
gblinear	xgboost	0.7535±0.0084	4.6886±0.0374
linear_trees	boosters	0.4282±0.0070	5.1809±0.1419
linear_trees	lightgbm	0.4047±0.0084	2.6936±0.0192

iris (primary: mlogloss)

Booster	Library	mlogloss	train_time_s
gbdt	boosters	0.1738±0.1038	0.0016±0.0001
gbdt	lightgbm	0.1423±0.0872	0.0083±0.0006
gbdt	xgboost	0.1846±0.1158	0.0081±0.0003
gblinear	boosters	0.3106±0.0374	0.0008±0.0000
gblinear	xgboost	0.4261±0.0304	0.0021±0.0001
linear_trees	boosters	0.1770±0.0970	0.0020±0.0000
linear_trees	lightgbm	0.1063±0.0652	0.0130±0.0009

synthetic_multi_small (primary: mlogloss)

Booster	Library	mlogloss	train_time_s
gbdt	boosters	0.6771±0.0997	0.5393±0.0044
gbdt	lightgbm	0.6668±0.1138	0.6864±0.0115
gbdt	xgboost	0.6654±0.1010	1.7020±0.0328
gblinear	boosters	1.0560±0.0461	0.0831±0.0005
gblinear	xgboost	1.0546±0.0451	0.1121±0.0006
linear_trees	boosters	0.6781±0.1073	0.5680±0.0136
linear_trees	lightgbm	0.6876±0.1082	0.6941±0.0040

Reproducing

boosters-eval full

---

Best values per metric are bolded. Lower is better for loss/time metrics.