Skip to content

Tried a CNN BLSTM CRF model

Jump to bottom
Ian Y. Ma edited this page Oct 24, 2019 · 12 revisions

My log file:

Ian Ma, Oct 18 - Oct 24, 2019:

--- To Do:

  1. Evaluate on our dataset using the same parameter. Precision and recall, F score for only person and organizations. Report cases: false positive, false negative. *2. Continue training on existing docs, also evaluate... *3. Automatic Content Extraction (ACE), inventory of name entities. https://www.ldc.upenn.edu/collaborations/past-projects/ace

--- Wrote: prep_eval.py eval.py $ pip3 install 'gast==0.2.2' $ python3 prep_eval.py $ python3 eval.py

--- Results: Precision: 0.25161290322580643 Recall: 0.24309884238646481 F1 score: 0.24728260869565213

Similar Precision: 0.5751152073732719 Similar Recall: 0.5842696629213483 Similar F1 score: 0.5796562935438923

--- Observations:

  1. Similar (model only catches a part of the name entity):

    Last week will be remembered for a long time as one of the saddest in this nation’s history with respect to road accidents. On Sunday last week Minister of State for Labour Mr. James Ocholi, SAN, as well as his wife and son died in an accident along the Kaduna-Abuja road. A day later, Chief of Training and Operations of the Nigeria Army Major General Yusha’u Mahmood Abubakar died in a crash on the Maiduguri-Damaturu highway. Abubakar was the former Theatre Commander of Operation Zaman Lafiya Dole, the military operation to banish insurgents from the North East. Under his command, spectacular successes were recorded in the war against Boko Haram. Acting General Officer Commanding [GOC] the Army’s 3 Armoured Division Brigadier M.S.A. Aliyu was also injured in the same accident. In the same week, 18 people died in an accident along the Bauchi-Jos highway when their commercial bus crashed. Ground Truth: M.S.A. Aliyu Predicted Label: MSA Aliyu

    Chukwu said: “Troops of 27 Task Force Brigade in conjunction with the Nigerian Air Force and Civilian Joint Task Force (CJTF) in the ongoing clearance operation destroyed another Boko Haram enclave. Ground Truth: 27 Task Force Brigade Predicted Label: Task Force Brigade

    General Officer Commanding Three Division, Major General Ebobowei Bona Awala, assured that the division will continue to work hand-in-hand with the various task forces in the areas of operations to ensure that piece is maintained and lives and property are secured in the region. Ground Truth: Ebobowei Bona Awala, Predicted Label: Bona Awala

    Last week will be remembered for a long time as one of the saddest in this nation’s history with respect to road accidents. On Sunday last week Minister of State for Labour Mr. James Ocholi, SAN, as well as his wife and son died in an accident along the Kaduna-Abuja road. A day later, Chief of Training and Operations of the Nigeria Army Major General Yusha’u Mahmood Abubakar died in a crash on the Maiduguri-Damaturu highway. Abubakar was the former Theatre Commander of Operation Zaman Lafiya Dole, the military operation to banish insurgents from the North East. Under his command, spectacular successes were recorded in the war against Boko Haram. Acting General Officer Commanding [GOC] the Army’s 3 Armoured Division Brigadier M.S.A. Aliyu was also injured in the same accident. In the same week, 18 people died in an accident along the Bauchi-Jos highway when their commercial bus crashed. Ground Truth: Theatre Commander Predicted Label: Theatre Commander of Operation Zaman Lafiya Dole

  2. False positive:

    In another development, the General Officer Commanding (GOC) 3 Division, Major-General Fatai Alli, said yesterday that Nigerian troops have recaptured almost all towns in Adamawa earlier held by the insurgents except Madagali, headquarters of Madagali Local Government Area. Predicted Label: Nigerian

    In another development, the General Officer Commanding (GOC) 3 Division, Major-General Fatai Alli, said yesterday that Nigerian troops have recaptured almost all towns in Adamawa earlier held by the insurgents except Madagali, headquarters of Madagali Local Government Area. Predicted Label: Adamawa

    In another development, the General Officer Commanding (GOC) 3 Division, Major-General Fatai Alli, said yesterday that Nigerian troops have recaptured almost all towns in Adamawa earlier held by the insurgents except Madagali, headquarters of Madagali Local Government Area. Predicted Label: Madagali

    In another development, the General Officer Commanding (GOC) 3 Division, Major-General Fatai Alli, said yesterday that Nigerian troops have recaptured almost all towns in Adamawa earlier held by the insurgents except Madagali, headquarters of Madagali Local Government Area. Predicted Label: Madagali Local Government Area

  3. False Negative:

    Acting G.O.C., 82 Division, Nigerian Army, Brig.-Gen. Ibrahim Attahiru, Igwe of Nike Enugu State, Igwe Julius Nnaji, and Commander, 82 Division, Nigerian Army Medical Services, Brig.-Gen. Austin Ogbe, during the provision of free medicare to the community in Enugu, recently. Ground Truth: Brig.-Gen.

    Radio Nigeria Defence Correspondent reports that before his appointment as Director, Army Public Relations, General Laleye served as Commander, One Division Garrison, Kaduna, while the outgoing Director resumes duty as the Commander of the Thirteenth Brigade. Ground Truth: General

    Similarly, the Chief of Training and Operations (Army), Major General DD Ahmadu led another team to Headquarters 27 Task Force Brigade, Buni Yadi, Yobe State, where they celebrated the Christmas with the troops. Ground Truth: Major General

Clone this wiki locally