Table 10 Model performance on two subgroups in the 1A dataset
From: Hypert: hypernymy-aware BERT with Hearst pattern exploitation for hypernym discovery
Subgroup | Evaluation measures | Proposed method | RMM | SPON | Prompting BERT (is a) | Prompting BERT (such as) |
|---|---|---|---|---|---|---|
Person | MRR | 84.78 ± 5.48 | 60.88 ± 17.54 | 66.83 ± 16.51 | 21.16 ± 1.22 | 17.08 ± 1.00 |
MAP | 45.78 ± 1.50 | 38.37 ± 5.31 | 38.82 ± 5.58 | 13.68 ± 0.47 | 11.01 ± 0.68 | |
P@1 | 79.34 ± 7.94 | 49.51 ± 21.90 | 55.01 ± 19.68 | 10.32 ± 1.23 | 8.50 ± 1.09 | |
P@3 | 44.82 ± 1.72 | 34.92 ± 8.61 | 36.53 ± 8.61 | 10.38 ± 0.58 | 8.24 ± 0.75 | |
P@5 | 40.88 ± 1.36 | 34.45 ± 5.77 | 34.92 ± 5.83 | 11.88 ± 0.42 | 8.60 ± 0.71 | |
P@15 | 45.04 ± 1.54 | 41.16 ± 2.79 | 40.33 ± 2.77 | 17.02 ± 0.58 | 15.34 ± 0.73 | |
Computer- Software | MRR | 36.69 ± 4.71 | 24.37 ± 3.49 | 17.01 ± 3.96 | 17.25 ± 3.19 | 19.52 ± 2.85 |
MAP | 21.75 ± 2.13 | 14.58 ± 3.01 | 8.36 ± 2.23 | 7.01 ± 1.46 | 8.31 ± 1.14 | |
P@1 | 22.53 ± 5.64 | 13.03 ± 3.13 | 7.94 ± 3.28 | 11.12 ± 2.35 | 11.59 ± 2.64 | |
P@3 | 19.44 ± 2.71 | 12.58 ± 2.72 | 6.89 ± 2.44 | 7.22 ± 1.39 | 8.68 ± 1.32 | |
P@5 | 20.09 ± 2.31 | 13.38 ± 3.29 | 6.89 ± 2.01 | 6.98 ± 1.44 | 7.97 ± 1.20 | |
P@15 | 25.55 ± 2.15 | 17.83 ± 4.44 | 10.91 ± 2.57 | 6.48 ± 1.42 | 7.97 ± 1.21 |