Skip to content

Commit

Permalink
Merge pull request #264 from KGQA/v2.0-rana
Browse files Browse the repository at this point in the history 
Solve issues - #248, #249, #250, #251, #252
  • Loading branch information
@xixi019
xixi019 authored Jun 21, 2024
2 parents 6f15a71 + f2caad3 commit d4d4a04
Show file tree
Hide file tree
Showing 25 changed files with 433 additions and 275 deletions.
2 changes: 2 additions & 0 deletions dbpedia/LC-QuAD v2.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ datasetUrl: https://figshare.com/projects/LCQuAD_2_0/62270

| Model / System | Year | Precision | Recall | F1 | Hits@1 | Language | Reported by | Gold Entity |
| :-----------------------: | :--: | :-------: | :----: | :---: | :----: | :------: | :----------------------------------------------------------------------------------: | :---------: |
| TSET-Small | 2024 | - | - | 94.0 | - | EN | [Qi et al](https://www.mdpi.com/2076-3417/14/4/1521) | ✅ |
| TSET-Base | 2024 | - | - | 95.0 | - | EN | [Qi et al.](https://www.mdpi.com/2076-3417/14/4/1521) | ✅ |
| T5-Small | 2022 | - | - | 92 | - | EN | [Banerjee et al.](https://arxiv.org/pdf/2204.12793.pdf) | ✅ |
| T5-Base | 2022 | - | - | 91 | - | EN | [Banerjee et al.](https://arxiv.org/pdf/2204.12793.pdf) | ✅ |
| PGN-BERT-BERT | 2022 | - | - | 86 | - | EN | [Banerjee et al.](https://arxiv.org/pdf/2204.12793.pdf) | ✅ |
Expand Down
3 changes: 2 additions & 1 deletion dbpedia/QALD-3.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

| Model / System | Year |Precision|Recall| F1 |Language| Reported by |
|:---------------:|:----:|:-------:|:----:|:---:|:------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| Elahi et al. | 2024 | 75 | 83 | 78 | EN | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf) |
| virtual player | 2015 | - | - |64.29| EN |[Molino et al.](https://reader.elsevier.com/reader/sd/pii/S0004370215000259?token=FB393D21799A6B75BDC436414AE01B228DF054D86D53A35C538F9F7B859CBD11103353F39E7530607239E025589F7A18&originRegion=eu-west-1&originCreation=20220102201741)|
| virtual player | 2015 | - | - |59.47| EN |[Molino et al.](https://reader.elsevier.com/reader/sd/pii/S0004370215000259?token=FB393D21799A6B75BDC436414AE01B228DF054D86D53A35C538F9F7B859CBD11103353F39E7530607239E025589F7A18&originRegion=eu-west-1&originCreation=20220102201741)|
| WDAqua-core1 | 2020 | 64 | 42 | 51 | EN | [Diefenbach et al.](http://www.semantic-web-journal.net/system/files/swj2038.pdf) |
Expand All @@ -29,4 +30,4 @@
| AskNow | 2021 | - | - | 13 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
|WDAqua-core0 [2] | 2021 | - | - | 11 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| QASparql | 2021 | - | - | 6 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
|Qanary(TM+DP+QB) | 2021 | - | - | 2 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf)
|Qanary(TM+DP+QB) | 2021 | - | - | 2 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf)
3 changes: 2 additions & 1 deletion dbpedia/QALD-5.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

| Model / System |Year|Precision|Recall| F1 |Language| Reported by |
|:--------------:|:---:|:-------:|:----:|:----:|:------:|:-------------------------------------------------------------------------------:|
| Elahi et al. |2024| 76 | 75 | 75 | EN | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf) |
| Xser |2020| 74 | 72 | 73 | EN |[Diefenbach et al.](http://www.semantic-web-journal.net/system/files/swj2038.pdf)|
| UTQA |2016| - | - | 65.2 | EN | [Ben Veyseh](https://aclanthology.org/W16-1403.pdf) |
| UTQA |2020| - | - | 65 | EN |[Diefenbach et al.](http://www.semantic-web-journal.net/system/files/swj2038.pdf)|
Expand All @@ -24,4 +25,4 @@
| YodaQA |2020| 18 | 17 | 18 | EN |[Diefenbach et al.](http://www.semantic-web-journal.net/system/files/swj2038.pdf)|
| QASparql |2021| - | - | 12 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| AskNow |2021| - | - | 9 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
|Qanary(TM+DP+QB)|2021| - | - | 2 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
|Qanary(TM+DP+QB)|2021| - | - | 2 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
5 changes: 4 additions & 1 deletion dbpedia/QALD-6.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@

| Model / System |Year|Precision|Recall| F1 |Language| Reported by |
|:--------------:|:---:|:-------:|:----:|:---:|:------:|:-------------------------------------------------------:|
| Elahi et al. |2024| 78 | 88 | 83 | SP | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf)|
| Elahi et al. |2024| 79 | 83 | 82 | EN | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf)|
| gAnswer |2017| 70 | 89 | 78 | EN |[Hu et al.](https://ieeexplore.ieee.org/document/8085196)|
| Elahi et al. |2024| 77 | 73 | 75 | DE | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf)|
| gAnswer |2021| - | - | 25 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| WDAqua-core0 |2021| - | - | 24 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| QASparql |2021| - | - | 17 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| AskNow |2021| - | - | 9 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
|Qanary(TM+DP+QB)|2021| - | - | 2 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
|Qanary(TM+DP+QB)|2021| - | - | 2 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
1 change: 1 addition & 0 deletions dbpedia/QALD-7.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
| Model / System | Year |Precision | Recall | F1 |Accuracy |Language | Reported by |
|:----------------------------:|:----:|:--------:|:------:|:-----:|:-------:|:-------:|:-----------------------------------------------------------------------------:|
| LAMA | 2019 | - | - | 90.50 | - | EN | [Radoev et. al.](https://arxiv.org/pdf/2004.13843.pdf) |
| Elahi et al. | 2024 | 79 | 93 | 86 | | EN | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf) |
| LingTeQA [1] | 2020 | 63.40 | 73.50 | 64.20 | - | EN | [D. Nhuan et al](https://ieeexplore.ieee.org/abstract/document/9282949) |
| Liang et al. | 2021 | 81.30 | 52.70 | 63.90 | - | EN |[Liang et al.](https://assets.researchsquare.com/files/rs-70794/v1_stamped.pdf)|
| Ensemble BR framework | 2023 | 59.80 | 69.60 | 61.20 | - | EN | [Chen et al.](https://assets.researchsquare.com/files/rs-2676239/v1_covered.pdf?c=1680800823) |
Expand Down
3 changes: 2 additions & 1 deletion dbpedia/QALD-9.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

| Model / System | Year | Precision | Recall | F1 |Language| Reported by |
|:--------------------------:|:----:|:---------:|:------:|:-----:|:------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| Elahi et al. | 2024 | 95 | 76 | 85 | EN | [Elahi et al.](https://www.semantic-web-journal.net/system/files/swj3619.pdf) |
| SGPT_Q,K | 2022 | - | - | 67.82 | EN | [Al Hasan Rony et al](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9815253) |
| SGPT_Q | 2022 | - | - | 60.22 | EN | [Al Hasan Rony et al](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9815253) |
| Stage I No Noise [2] | 2022 | 80.40 | 42.10 | 55.30 | EN | [Purkayastha et al.](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9892263) |
Expand Down Expand Up @@ -55,4 +56,4 @@
| QASystem | 2019 | 9.70 | 11.60 | 9.80 | EN | [Zheng et. al.](https://arxiv.org/pdf/1910.09760.pdf) |
| AskNow | 2021 | - | - | 8 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| Qanary(TM+DP+QB) | 2021 | - | - | 7 | EN | [Orogat et al.](https://arxiv.org/pdf/2105.00811.pdf) |
| Elon | 2021 | 4.90 | 5.30 | 5 | EN | [Steinmetz et al.](https://link.springer.com/article/10.1007/s13740-021-00128-9)
| Elon | 2021 | 4.90 | 5.30 | 5 | EN | [Steinmetz et al.](https://link.springer.com/article/10.1007/s13740-021-00128-9)
Loading

0 comments on commit d4d4a04

Please sign in to comment.