Table 6 Comparison of evaluation measures on the Hypert and BERT models for each subtask
From: Hypert: hypernymy-aware BERT with Hearst pattern exploitation for hypernym discovery
Subtask | Evaluation measures | Hypert model | BERT model |
|---|---|---|---|
1A English | MRR | 38.68 ± 2.00 | 36.44 ± 2.12 |
MAP | 24.17 ± 1.26 | 23.29 ± 0.78 | |
P@1 | 29.57 ± 1.98 | 26.38 ± 2.93 | |
P@3 | 21.56 ± 1.38 | 20.90 ± 1.00 | |
P@5 | 21.27 ± 1.25 | 20.68 ± 0.79 | |
P@15 | 27.52 ± 1.35 | 26.63 ± 0.75 | |
2A Medical | MRR | 64.83 ± 3.32 | 62.62 ± 3.20 |
MAP | 50.24 ± 2.29 | 48.85 ± 1.57 | |
P@1 | 53.28 ± 3.86 | 49.94 ± 4.35 | |
P@3 | 46.69 ± 3.52 | 45.45 ± 2.27 | |
P@5 | 46.60 ± 2.91 | 45.66 ± 1.74 | |
P@15 | 54.69 ± 1.74 | 53.36 ± 0.97 | |
2B Music | MRR | 67.43 ± 2.37 | 63.19 ± 5.38 |
MAP | 55.03 ± 1.98 | 49.70 ± 3.37 | |
P@1 | 56.68 ± 2.98 | 50.92 ± 7.31 | |
P@3 | 52.94 ± 2.05 | 46.88 ± 4.28 | |
P@5 | 52.92 ± 2.37 | 47.27 ± 3.59 | |
P@15 | 58.59 ± 2.05 | 53.97 ± 2.48 |