Skip to main content

Table 8 Comparison of MRR, MAP, and P@1 results by number of pretraining steps

From: Hypert: hypernymy-aware BERT with Hearst pattern exploitation for hypernym discovery

Subtask

Steps

Evaluation measures

MRR

MAP

P@1

1A English

0k

36.44 ± 2.12

23.29 ± 0.78

26.38 ± 2.93

1k

38.68 ± 2.00

24.17 ± 1.26

29.57 ± 1.98

2k

37.52 ± 2.02

23.59 ± 0.98

28.37 ± 2.64

3k

37.71 ± 1.08

23.76 ± 0.95

28.36 ± 1.39

4k

37.49 ± 1.05

23.69 ± 0.86

28.03 ± 1.33

5k

38.06 ± 2.04

24.51 ± 0.73

28.42 ± 2.53

6k

38.46 ± 2.33

24.36 ± 1.18

29.13 ± 2.62

7k

37.75 ± 1.49

23.89 ± 1.23

28.45 ± 1.52

8k

37.69 ± 3.08

23.89 ± 1.63

28.46 ± 3.43

9k

\(\underline{38.67 \pm 1.59}\)

\(\underline{24.46 \pm 0.89}\)

\(\underline{29.38 \pm 1.82}\)

10k

37.85 ± 1.94

24.40 ± 0.80

27.87 ± 2.78

2A

Medical

0k

62.62 ± 3.20

48.85 ± 1.57

49.94 ± 4.35

1k

64.83 ± 3.32

50.24 ± 2.29

53.28 ± 3.86

2k

66.52 ± 2.44

\(\underline{50.48 \pm 1.63}\)

55.64 ± 3.43

3k

60.91 ± 4.65

47.41 ± 2.60

49.40 ± 5.70

4k

65.10 ± 4.13

49.60 ± 2.50

54.62 ± 4.92

5k

64.04 ± 3.20

49.25 ± 1.38

53.34 ± 4.37

6k

66.21 ± 2.66

50.15 ± 2.23

\(\underline{55.68 \pm 2.69}\)

7k

64.74 ± 3.23

49.62 ± 2.04

54.28 ± 3.84

8k

\(\underline{66.34 \pm 3.81}\)

50.52 ± 2.15

56.16 ± 4.95

9k

64.42 ± 2.52

50.15 ± 1.52

53.18 ± 4.00

10k

63.44 ± 2.91

48.56 ± 1.34

52.62 ± 3.52

2B

Music

0k

63.19 ± 5.38

49.70 ± 3.37

50.92 ± 7.31

1k

67.43 ± 2.37

55.03 ± 1.98

56.68 ± 2.98

2k

65.71 ± 3.04

53.97 ± 2.24

53.84 ± 4.03

3k

62.93 ± 4.22

50.53 ± 3.75

50.42 ± 5.50

4k

65.63 ± 4.65

53.55 ± 3.27

53.86 ± 5.92

5k

66.89 ± 3.31

53.96 ± 2.33

55.62 ± 4.71

6k

\(\underline{67.40 \pm 3.41}\)

\(\underline{54.53 \pm 3.23}\)

\(\underline{56.42 \pm 4.46}\)

7k

66.09 ± 4.51

53.43 ± 3.02

54.84 ± 5.85

8k

67.21 ± 3.47

54.28 ± 1.93

56.18 ± 4.74

9k

65.25 ± 6.19

52.37 ± 4.57

53.48 ± 8.48

10k

64.54 ± 5.24

51.69 ± 3.77

52.92 ± 6.93

  1. 0k indicates the BERT model. Bold represents the best performance across the comparison models, and underline represents second place