-
Notifications
You must be signed in to change notification settings - Fork 1
/
resutls_add_one_smoothing.txt
72 lines (68 loc) · 4.67 KB
/
resutls_add_one_smoothing.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
!!!!!!!!!!The test Sentence is!!!!!!!!!!
['upon this the captain started , and eagerly desired to know more .']
A table showing the bigram counts for test sentence.
smoothing =1
started desired more eagerly to captain this and know the upon </s>
17168 started 1 1 1 1 6 1 1 2 1 2 1 1
17152 desired 1 1 1 1 3 1 1 1 1 1 1 1
17649 more 2 1 1 1 12 1 1 25 1 15 3 26
17149 eagerly 1 2 1 1 1 1 1 2 1 1 1 1
21766 to 1 1 2 1 2 3 51 24 16 730 2 27
17470 captain 2 1 1 1 9 1 3 12 1 4 2 11
18535 this 1 1 1 1 8 2 2 10 1 25 1 26
23571 and 2 1 46 3 31 6 45 1 2 371 3 1
17293 know 1 1 2 1 2 2 1 3 1 9 1 8
31572 the 1 4 51 1 1 98 1 1 1 1 1 2
17707 upon 1 1 1 1 1 2 24 1 1 221 1 5
27200 <s> 2 1 8 1 61 23 116 409 3 630 27 27
A table showing the bigram probabilities for test sentence
smoothing =1
started desired more eagerly to captain this and know the upon </s>
started 0.00006 0.00006 0.00006 0.00006 0.00035 0.00006 0.00006 0.00012 0.00006 0.00012 0.00006 0.00006
desired 0.00006 0.00006 0.00006 0.00006 0.00017 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006
more 0.00011 0.00006 0.00006 0.00006 0.00068 0.00006 0.00006 0.00142 0.00006 0.00085 0.00017 0.00147
eagerly 0.00006 0.00012 0.00006 0.00006 0.00006 0.00006 0.00006 0.00012 0.00006 0.00006 0.00006 0.00006
to 0.00005 0.00005 0.00009 0.00005 0.00009 0.00014 0.00234 0.00110 0.00074 0.03354 0.00009 0.00124
captain 0.00011 0.00006 0.00006 0.00006 0.00052 0.00006 0.00017 0.00069 0.00006 0.00023 0.00011 0.00063
this 0.00005 0.00005 0.00005 0.00005 0.00043 0.00011 0.00011 0.00054 0.00005 0.00135 0.00005 0.00140
and 0.00008 0.00004 0.00195 0.00013 0.00132 0.00025 0.00191 0.00004 0.00008 0.01574 0.00013 0.00004
know 0.00006 0.00006 0.00012 0.00006 0.00012 0.00012 0.00006 0.00017 0.00006 0.00052 0.00006 0.00046
the 0.00003 0.00013 0.00162 0.00003 0.00003 0.00310 0.00003 0.00003 0.00003 0.00003 0.00003 0.00006
upon 0.00006 0.00006 0.00006 0.00006 0.00006 0.00011 0.00136 0.00006 0.00006 0.01248 0.00006 0.00028
<s> 0.00007 0.00004 0.00029 0.00004 0.00224 0.00085 0.00426 0.01504 0.00011 0.02316 0.00099 0.00099
The probability of the sentence under the trained model
smoothing =1
1.01571679763017e-23
!!!!!!!!!!The test Sentence is!!!!!!!!!!
['thus , because no man can follow another into these halls .']
A table showing the bigram counts for test sentence.
smoothing =1
thus because into these halls follow no can another man </s>
17274 thus 1 2 1 1 1 1 1 1 1 1 2
17233 because 1 1 1 2 1 1 3 1 1 1 1
17664 into 1 1 1 4 1 1 1 1 1 1 2
17544 these 1 1 1 1 2 1 1 2 1 1 3
17144 halls 1 1 1 1 1 1 1 1 1 1 2
17161 follow 1 1 1 1 1 1 1 1 2 1 6
17727 no 1 2 1 1 1 1 19 1 1 10 18
17377 can 1 1 1 1 1 3 1 1 1 1 10
17256 another 1 1 2 1 1 1 1 1 1 1 4
17668 man 2 1 3 3 1 1 1 6 1 2 63
27200 <s> 27 19 3 22 1 1 102 17 8 12 27
A table showing the bigram probabilities for test sentence
smoothing =1
thus because into these halls follow no can another man </s>
thus 0.00006 0.00012 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00012
because 0.00006 0.00006 0.00006 0.00012 0.00006 0.00006 0.00017 0.00006 0.00006 0.00006 0.00006
into 0.00006 0.00006 0.00006 0.00023 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00011
these 0.00006 0.00006 0.00006 0.00006 0.00011 0.00006 0.00006 0.00011 0.00006 0.00006 0.00017
halls 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00012
follow 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00012 0.00006 0.00035
no 0.00006 0.00011 0.00006 0.00006 0.00006 0.00006 0.00107 0.00006 0.00006 0.00056 0.00102
can 0.00006 0.00006 0.00006 0.00006 0.00006 0.00017 0.00006 0.00006 0.00006 0.00006 0.00058
another 0.00006 0.00006 0.00012 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00006 0.00023
man 0.00011 0.00006 0.00017 0.00017 0.00006 0.00006 0.00006 0.00034 0.00006 0.00011 0.00357
<s> 0.00099 0.00070 0.00011 0.00081 0.00004 0.00004 0.00375 0.00063 0.00029 0.00044 0.00099
The probability of the sentence under the trained model
smoothing =1
1.0762093761487005e-21