Table 7 Batch-Normalized Maxout Network in Network hyperparameters
From: Effective hyperparameter optimization using Nelder-Mead method in deep learning
Name | Description | Range |
|---|---|---|
x 1 | Learning rate (\(= 0.1^{x_{1}}\phantom {\dot {i}\!}\)) | [ 0.5,2] |
x 2 | Momentum (\(= 1 - 0.1^{x_{2}}\phantom {\dot {i}\!}\)) | [ 0.5,2] |
x 3 | L2 weight decay | [ 0.001,0.01] |
x 4 | Dropout 1 | [ 0.4,0.6] |
x 5 | Dropout 2 | [ 0.4,0.6] |
x 6 | Conv 1 initialization deviation | [ 0.01,0.05] |
x 7 | Conv 2 initialization deviation | [ 0.01,0.05] |
x 8 | Conv 3 initialization deviation | [ 0.01,0.05] |
x 9 | MMLP 1-1 initialization deviation | [ 0.01,0.05] |
x 10 | MMLP 1-2 initialization deviation | [ 0.01,0.05] |
x 11 | MMLP 2-1 initialization deviation | [ 0.01,0.05] |
x 12 | MMLP 2-2 initialization deviation | [ 0.01,0.05] |
x 13 | MMLP 3-1 initialization deviation | [ 0.01,0.05] |
x 14 | MMLP 3-2 initialization deviation | [ 0.01,0.05] |