1_1_2-全连接神经网络做mnist手写数字识别.md

[jurohd]

cross entropy的话最后一层后面还要加softmax吧

[jurohd]

不好意思我没看完 你的softmax在crossentropyloss里面了我没仔细看

[yangzhaonan18]

我在你代码的基础上，加了一层全连接之后，为什么会出现测试训练准确率一直10% 左右的情况呢，loss 也不下降，是因为你的代码没有加正则化的原因吗？谢谢
调整了学习率，也不收敛。

去掉一层，却可以收敛，97%在群里。为什么呢？

[yizt]

@yangzhaonan18 推测是梯度消失了，将weights_scale=1e-3改为1e-2就可以了.以下是我增加一个fc层后的测试的结果

 epoch:0 step:0 ; loss:2.3025979253964155
 train_acc:0.21875;  val_acc:0.0991

 epoch:0 step:500 ; loss:2.2866856309392967
 train_acc:0.15625;  val_acc:0.1064

 epoch:0 step:1000 ; loss:1.1675340534531413
 train_acc:0.59375;  val_acc:0.3591

 epoch:0 step:1500 ; loss:0.4990171329343591
 train_acc:0.90625;  val_acc:0.8664

 epoch:1 step:0 ; loss:0.359293357114892
 train_acc:0.96875;  val_acc:0.8764

 epoch:1 step:500 ; loss:0.19083221860863414
 train_acc:1.0;  val_acc:0.9237

 epoch:1 step:1000 ; loss:0.0747052528886187
 train_acc:1.0;  val_acc:0.9476

 epoch:1 step:1500 ; loss:0.08739012776302801
 train_acc:1.0;  val_acc:0.9513

 epoch:2 step:0 ; loss:0.03666094344819661
 train_acc:1.0;  val_acc:0.9543

 epoch:2 step:500 ; loss:0.26946484791939
 train_acc:0.96875;  val_acc:0.9538

 epoch:2 step:1000 ; loss:0.05818703805948156
 train_acc:1.0;  val_acc:0.961

[wangtao2668129173]

@yangzhaonan18 推测是梯度消失了，将weights_scale=1e-3改为1e-2就可以了.以下是我增加一个fc层后的测试的结果

 epoch:0 step:0 ; loss:2.3025979253964155
 train_acc:0.21875;  val_acc:0.0991

 epoch:0 step:500 ; loss:2.2866856309392967
 train_acc:0.15625;  val_acc:0.1064

 epoch:0 step:1000 ; loss:1.1675340534531413
 train_acc:0.59375;  val_acc:0.3591

 epoch:0 step:1500 ; loss:0.4990171329343591
 train_acc:0.90625;  val_acc:0.8664

 epoch:1 step:0 ; loss:0.359293357114892
 train_acc:0.96875;  val_acc:0.8764

 epoch:1 step:500 ; loss:0.19083221860863414
 train_acc:1.0;  val_acc:0.9237

 epoch:1 step:1000 ; loss:0.0747052528886187
 train_acc:1.0;  val_acc:0.9476

 epoch:1 step:1500 ; loss:0.08739012776302801
 train_acc:1.0;  val_acc:0.9513

 epoch:2 step:0 ; loss:0.03666094344819661
 train_acc:1.0;  val_acc:0.9543

 epoch:2 step:500 ; loss:0.26946484791939
 train_acc:0.96875;  val_acc:0.9538

 epoch:2 step:1000 ; loss:0.05818703805948156
 train_acc:1.0;  val_acc:0.961

我下载了代码按照步骤跑，发现有报错 尝试改了还是这个问题，请问还有这个代码方便发一份吗
x.shape:(16, 1, 28, 28),y.shape:(16, 10)
Traceback (most recent call last):
File "train.py", line 81, in
from nn.optimizers import SGD
File "/data/wangtao/numpy_neural_network/nn/optimizers.py", line 12, in
from modules import Model
File "/data/wangtao/numpy_neural_network/nn/modules.py", line 15, in
from optimizers import *
File "/data/wangtao/numpy_neural_network/nn/optimizers.py", line 12, in
from modules import Model
ImportError: cannot import name 'Model'

[yizt]

@yangzhaonan18 所有代码均已上传，请参照readme先编译

cd numpy_neuron_network
python setup.py build_ext -i