-
Notifications
You must be signed in to change notification settings - Fork 6
/
Copy pathREADME
92 lines (61 loc) · 3.55 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
Naive Bayes Classifier
----------------------
(c) Tim Nugent
Compile by running 'make'. Uses std=c++11 - on older compilers you may need to change this to 'std=c++0x' in the Makefile.
Run all tests with 'make test'
Run the train/classify tool as follows:
./nb_classify -d 1 training.data test.data
Training and test data should be in svm-light/libsvm format, e.g.
+1 1:4.12069 2:11.3896 3:18.5742 4:2.85764 5:53.4406
-1 1:3.14565 2:17.4338 3:19.3353 4:2.63431 5:56.4233
Sparse data is not (yet) supported. The -d flag is the decision rule, option 1 = Gaussian (default), 2 = Multinomial and 3 = Bernoulli. -v flag turns verbose mode on - use this to see the classification results. The -a flag passes the smoothing parameter used in the multinomial decision rule (default 1.0, Laplace smoothing).
Gaussian example
----------------
Generate Gaussian training and test data as follows:
./generate_gaussian_data 5000 +1 5.3 2.0 12.8 3.1 18.2 1.0 1.2 3.4 56.2 2.3 > training.data
./generate_gaussian_data 5000 -1 7.3 1.0 10.8 1.1 24.1 2.3 0.8 1.2 53.1 0.8 >> training.data
./generate_gaussian_data 1000 +1 5.3 5.0 12.8 5.1 18.2 4.0 1.2 5.4 56.2 4.3 > test.data
./generate_gaussian_data 1000 -1 7.3 4.0 10.8 4.1 24.1 5.3 0.8 4.2 53.1 2.8 >> test.data
Arguments are the number of examples, the class label, then pairs of means and standard deviations. Note that I bumped up the standard deviations in the test data to make classification harder.
Then train and classify as follows:
./nb_classify -d 1 training.data test.data
Multinomial example
-------------------
Example based on table 13.1 from here:
http://nlp.stanford.edu/IR-book/html/htmledition/naive-bayes-text-classification-1.html
echo "+1 1:2 2:1 3:0 4:0 5:0 6:0" > training.data
echo "+1 1:2 2:0 3:1 4:0 5:0 6:0" >> training.data
echo "+1 1:1 2:0 3:0 4:1 5:0 6:0" >> training.data
echo "-1 1:1 2:0 3:0 4:0 5:1 6:1" >> training.data
echo "+1 1:3 2:0 3:0 4:0 5:1 6:1" > test.data
Then train and classify as follows:
./nb_classify -d 2 training.data test.data
Bernoulli example
-----------------
Example taken from here:
http://www.inf.ed.ac.uk/teaching/courses/inf2b/learnnotes/inf2b-learn-note07-2up.pdf
echo "+1 1:1 2:0 3:0 4:0 5:1 6:1 7:1 8:1" > training.data
echo "+1 1:0 2:0 3:1 4:0 5:1 6:1 7:0 8:0" >> training.data
echo "+1 1:0 2:1 3:0 4:1 5:0 6:1 7:1 8:0" >> training.data
echo "+1 1:1 2:0 3:0 4:1 5:0 6:1 7:0 8:1" >> training.data
echo "+1 1:1 2:0 3:0 4:0 5:1 6:0 7:1 8:1" >> training.data
echo "+1 1:0 2:0 3:1 4:1 5:0 6:0 7:1 8:1" >> training.data
echo "-1 1:0 2:1 3:1 4:0 5:0 6:0 7:1 8:0" >> training.data
echo "-1 1:1 2:1 3:0 4:1 5:0 6:0 7:1 8:1" >> training.data
echo "-1 1:0 2:1 3:1 4:0 5:0 6:1 7:0 8:0" >> training.data
echo "-1 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0" >> training.data
echo "-1 1:0 2:0 3:1 4:0 5:1 6:0 7:1 8:0" >> training.data
echo "+1 1:1 2:0 3:0 4:1 5:1 6:1 7:0 8:1" > test.data
echo "-1 1:0 2:1 3:1 4:0 5:1 6:0 7:1 8:0" >> test.data
./nb_classify -d 3 training.data test.data
Generating dummy data
---------------------
See above for Gaussian data. For multinomial data:
./generate_multinomial_data 5000 +1 2 3 4 3 5 > training.data
./generate_multinomial_data 5000 -1 1 4 3 2 4 >> training.data
The five weights are used to generate integers with probability weight/sum of weights.
For Bernoulli data:
./generate_bernoulli_data 5000 +1 0.7 0.1 0.5 > training.data
./generate_bernoulli_data 5000 -1 0.4 0.2 0.6 >> training.data
The three probabilities are used to generate binary variables.
Note that the prior class probabilites are calculated based on their ratios in the training data.