Experiments on CoNLL03NER

[SUDA-HLT-ywfang]

Hello!
I try to run your code on conll03-ner dataset. But, performance I get is not as good as bilstm-crf. Could you help me find the bug? Thanks.
Here is part of my experiment log.

True
Seed num: 42
MODEL: train
Load pretrained word embedding, norm: False, dir: ../Data/pretrain_emb/glove.6B.100d.txt
Embedding:
pretrain word:400000, prefect match:11415, case_match:11656, oov:2234, oov%:0.08827945941673912
Training model...
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
DATA SUMMARY START:
I/O:
Tag scheme: BMES
MAX SENTENCE LENGTH: 250
MAX WORD LENGTH: -1
Number normalized: True
Word alphabet size: 25306
Char alphabet size: 78
Label alphabet size: 18
Word embedding dir: ../Data/pretrain_emb/glove.6B.100d.txt
Char embedding dir: None
Word embedding size: 100
Char embedding size: 30
Norm word emb: False
Norm char emb: False
Train file directory: ../Data/conll03/conll03.train.bmes
Dev file directory: ../Data/conll03/conll03.dev.bmes
Test file directory: ../Data/conll03/conll03.test.bmes
Raw file directory: None
Dset file directory: None
Model file directory: save/label_embedding
Loadmodel directory: None
Decode file directory: None
Train instance number: 14987
Dev instance number: 3466
Test instance number: 3684
Raw instance number: 0
FEATURE num: 0
++++++++++++++++++++++++++++++++++++++++
Model Network:
Model use_crf: False
Model word extractor: LSTM
Model use_char: True
Model char extractor: LSTM
Model char_hidden_dim: 50
++++++++++++++++++++++++++++++++++++++++
Training:
Optimizer: SGD
Iteration: 100
BatchSize: 10
Average batch loss: False
++++++++++++++++++++++++++++++++++++++++
Hyperparameters:
Hyper lr: 0.01
Hyper lr_decay: 0.04
Hyper HP_clip: None
Hyper momentum: 0.9
Hyper l2: 1e-08
Hyper hidden_dim: 400
Hyper dropout: 0.5
Hyper lstm_layer: 4
Hyper bilstm: True
Hyper GPU: True
DATA SUMMARY END.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
build network...
use_char: True
char feature extractor: LSTM
word feature extractor: LSTM
build word sequence feature extractor: LSTM...
build word representation...
build char sequence feature extractor: LSTM ...
--------pytorch total params--------
9849140
Epoch: 0/100
Learning rate is set as: 0.01
Instance: 14987; Time: 125.29s; loss: 2452.2396; acc: 172887.0/204567.0=0.8451
Epoch: 0 training finished. Time: 125.29s, speed: 119.62st/s, total loss: 126550.64305019379
totalloss: 126550.64305019379
gold_num = 5942 pred_num = 6508 right_num = 2556
Dev: time: 11.00s, speed: 317.98st/s; acc: 0.9036, p: 0.3927, r: 0.4302, f: 0.4106
gold_num = 5648 pred_num = 6351 right_num = 2261
Test: time: 10.95s, speed: 339.54st/s; acc: 0.8919, p: 0.3560, r: 0.4003, f: 0.3769
Exceed previous best f score: -10

[TianlinZhang668]

uhnn, i got f1 which is 89, and i use the BIO format

[TianlinZhang668]

but in some chinese dataset . The results are also low