Table 11 MNIST results (LeNet)
From: Effective hyperparameter optimization using Nelder-Mead method in deep learning
Method | Mean loss | Min loss |
|---|---|---|
Random search | 0.005411 (±0.001413) | 0.002781 |
Bayesian optimization | 0.004217 (±0.002242) | 0.000089 |
CMA-ES | 0.000926 (±0.001420) | 0.000047 |
Coordinate-search method | 0.000052 (±0.000094) | 0.000002 |
Nelder-Mead method | 0.000029 (±0.000029) | 0.000004 |
Method | Mean accuracy (%) | Accuracy with min loss (%) |
Random search | 98.98 (±0.08) | 99.06 |
Bayesian optimization | 99.07 (±0.02) | 99.25 |
CMA-ES | 99.20 (±0.08) | 99.30 |
The coordinate-search method | 99.26 (±0.05) | 99.35 |
The Nelder-Mead method | 99.24 (±0.04) | 99.28 |