Analysis of imprecise measurement data utilizing z-test for correlation

Abstract

The conventional Z-test for correlation, grounded in classical statistics, is typically employed in situations devoid of vague information. However, real-world data often comes with inherent uncertainty, necessitating an adaptation of the Z-test using neutrosophic statistics. This paper introduces a modified Z-test for correlation designed to explore correlations in the presence of imprecise data. We will present the simulation to check the effect of the measure of indeterminacy on the evolution of type-I error and the power of the test. The application of this modification is illustrated through an examination of heartbeat and temperature data. Upon analyzing the heartbeat and temperature data, it is determined that, in the face of indeterminacy, the correlation between heartbeat and temperature emerges as significant. This highlights the importance of accounting for imprecise data when investigating relationships between variables.

Introduction

In medical science, correlation analysis has been used to investigate the degree of dependence between two medical variables. The correlation analysis tells about the strength of the relationship between two variables. The Z-test of correlation has been applied to investigating the significance of the correlation between two variables. For example, in medical science, the decision-makers are interested to see the significance of the relationship between blood pressure and diet. Based on correlation analysis, the medical decision-makers can suggest a suitable medication. Therefore, the statistical test investigates the correlation between the significance of the two variables under study. The null hypothesis that the correlation between two variables is insignificant is tested versus the alternative hypothesis that two variables are significantly correlated. Statistical tests have been widely used in medical science for decision-making. Gordon Lan et al. [13] introduced the weighted Z-tests and applied them in medical science. Bellolio et al. [7] provided a detailed discussion on the suitability of the statistical tests for medical studies. Mukaka [19] discussed the suitability of correlation analysis for medical data. Pandey et al. [21] discussed the importance of t-tests in medical-related problems. Schober et al. [23] discussed the correlation analysis for anesthesia data. Kc [17] wrote a review on the applications of statistical methods for medical science. Janse et al. [15] discussed the limitations of correlation analysis using medical data. More applications of statistical methods in the medical field can be seen in [10, 18, 29, 32].

Statistical tests have been widely used for the analysis of measurement data. Grzesiek et al. [14] used the statistical test for the analysis of the temperature data. Avuçlu [6] presented the work on the detection covid-19 using statistical measurements. More applications of statistical analysis for the measurement data can be seen in [22, 27, 31].

The neutrosophic statistics were developed by [26] using the idea of neutrosophic logic developed by [25], and its efficiency of fuzzy logic and interval-analysis is shown by [9]. The applications of neutrosophic logic in medical science can be read in [8, 30]. Neutrosophic statistics are used for the collection of imprecise and interval data, analysis and interpretation of the imprecise data. The efficiency of neutrosophic statistics over classical statistics was discussed by [5, 11, 12]. Later on, the applications of neutrosophic statistics in the field of medical science were given by [1, 3, 5, 24].

The existing Z-test for correlation cannot be applied when the data is expressed in intervals or when uncertainty in parameters or level of significance is noted. To overcome this issue, in this paper, the Z-test for a single correlation coefficient using neutrosophic statistics will be presented. The test statistic for the proposed test will be developed and the application will be given using the heartbeat and body temperature data. It is expected that the proposed test will be efficient in investigating the significance of the correlation between variables expressed in intervals.

Method

Let ${X}_{N}={X}_{L}+{X}_{U}{I}_{{X}_{N}};{I}_{{X}_{N}}\epsilon \left[{I}_{{X}_{L}},{I}_{{X}_{U}}\right]$ and ${Y}_{N}={Y}_{L}+{Y}_{U}{I}_{{Y}_{N}};{I}_{{Y}_{N}}\epsilon \left[{I}_{{Y}_{L}},{I}_{{Y}_{U}}\right]$ be two neutrosophic random variables of size ${n}_{N}={n}_{L}+{n}_{U}{I}_{{n}_{N}};{I}_{{n}_{N}}\epsilon \left[{I}_{{n}_{L}},{I}_{{n}_{U}}\right]$ follow the neutrosophic normal distribution with the neutrosophic means ${\mu }_{XN}={\mu }_{XL}+{\mu }_{XU}{I}_{{\mu }_{XN}};{I}_{{\mu }_{XN}}\epsilon \left[{I}_{{\mu }_{XL}},{I}_{{\mu }_{XU}}\right]$ and ${\mu }_{YN}={\mu }_{YL}+{\mu }_{YU}{I}_{{\mu }_{YN}};{I}_{{\mu }_{YN}}\epsilon \left[{I}_{{\mu }_{YL}},{I}_{{\mu }_{YU}}\right]$, and neutrosophic standard deviation ${\sigma }_{XN}={\sigma }_{XL}+{\sigma }_{XU}{I}_{{\sigma }_{XN}};{I}_{{\sigma }_{XN}}\epsilon \left[{I}_{{\sigma }_{XL}},{I}_{{\sigma }_{XU}}\right]$ and ${\sigma }_{YN}={\sigma }_{YL}+{\sigma }_{YU}{I}_{{\sigma }_{YN}};{I}_{{\sigma }_{YN}}\epsilon \left[{I}_{{\sigma }_{YL}},{I}_{{\sigma }_{YU}}\right]$, respectively. Note that ${X}_{L},{Y}_{L},{n}_{L},{\mu }_{XL}$,${\mu }_{YL}$ are the determinate parts denote the classical statistics, ${X}_{U}{I}_{{X}_{N}},{Y}_{U}{I}_{{Y}_{N}},{I}_{{n}_{N}}\epsilon \left[{I}_{{n}_{L}},{I}_{{n}_{U}}\right], {n}_{U}{I}_{{n}_{N}},{\mu }_{XU}{I}_{{\mu }_{XN}},{\mu }_{YU}{I}_{{\mu }_{YN}}$ are indeterminate parts and ${I}_{{X}_{N}}\epsilon \left[{I}_{{X}_{L}},{I}_{{X}_{U}}\right],{I}_{{Y}_{N}}\epsilon \left[{I}_{{Y}_{L}},{I}_{{Y}_{U}}\right],{I}_{{\mu }_{XN}}\epsilon \left[{I}_{{\mu }_{XL}},{I}_{{\mu }_{XU}}\right],{I}_{{\mu }_{YN}}\epsilon \left[{I}_{{\mu }_{YL}},{I}_{{\mu }_{YU}}\right]$ are measures of indeterminacy. For designing of the proposed Z-test of a correlation coefficient, it is assumed that variance in ${X}_{N}={X}_{L}+{X}_{U}{I}_{{X}_{N}};{I}_{{X}_{N}}\epsilon \left[{I}_{{X}_{L}},{I}_{{X}_{U}}\right]$ should be independent from the variance in ${Y}_{N}={Y}_{L}+{Y}_{U}{I}_{{Y}_{N}};{I}_{{Y}_{N}}\epsilon \left[{I}_{{Y}_{L}},{I}_{{Y}_{U}}\right]$. Suppose that ${r}_{N}={r}_{L}+{r}_{U}{I}_{{r}_{N}};{I}_{{r}_{N}}\epsilon \left[{I}_{{r}_{L}},{I}_{{r}_{u}}\right]$ is neutrosophic correlation between ${X}_{N}={X}_{L}+{X}_{U}{I}_{{X}_{N}};{I}_{{X}_{N}}\epsilon \left[{I}_{{X}_{L}},{I}_{{X}_{U}}\right]$ and ${Y}_{N}={Y}_{L}+{Y}_{U}{I}_{{Y}_{N}};{I}_{{Y}_{N}}\epsilon \left[{I}_{{Y}_{L}},{I}_{{Y}_{U}}\right]$ that is given by the following [2] as

$$r_{N} \in \left\{ {\frac{{\mathop \sum \nolimits_{i = 1}^{{n_{L} }} \left( {X_{iL} - \overline{X}_{L} } \right)\left( {Y_{iL} - \overline{Y}_{L} } \right)}}{{\sqrt {\mathop \sum \nolimits_{i = 1}^{{n_{L} }} \left( {X_{iL} - \overline{X}_{L} } \right)^{2} \mathop \sum \nolimits_{i = 1}^{{n_{L} }} \left( {Y_{iL} - \overline{Y}_{L} } \right)^{2} } }},\frac{{\mathop \sum \nolimits_{i = 1}^{{n_{U} }} \left( {X_{iU} - \overline{X}_{U} } \right)\left( {Y_{iU} - \overline{Y}_{U} } \right)}}{{\sqrt {\mathop \sum \nolimits_{i = 1}^{{n_{U} }} \left( {X_{iU} - \overline{X}_{U} } \right)^{2} \mathop \sum \nolimits_{i = 1}^{{n_{U} }} \left( {Y_{iU} - \overline{Y}_{U} } \right)^{2} } }}} \right\};I_{{r_{N} }} \in \left[ {I_{{r_{L} }} ,I_{{r_{u} }} } \right]$$

(1)

where ${\overline{X} }_{L}$ and ${\overline{X} }_{U}$ are the lower and upper values of the neutrosophic sample average. To investigate either correlation coefficient ${r}_{N}\epsilon \left[{r}_{L},{r}_{U}\right]$ differs significantly from the specified correlation ${r}_{0N}\epsilon \left[{r}_{0L},{r}_{0U}\right]$, the null hypothesis that the correlation coefficient ${r}_{N}\epsilon \left[{r}_{L},{r}_{U}\right]$ is at least ${r}_{0N}\epsilon \left[{r}_{0L},{r}_{0U}\right]$ vs. the alternative hypothesis correlation coefficient ${r}_{N}\epsilon \left[{r}_{L},{r}_{U}\right]$ is at most ${r}_{0N}\epsilon \left[{r}_{0L},{r}_{0U}\right]$. Using the Fisher’s transformation, the value of quantity ${Z}_{1N}\epsilon \left[{Z}_{1L},{Z}_{1U}\right]$ is calculated as

$$Z_{1N} = Z_{1L} + Z_{1U} I_{{Z1_{N} }} ;I_{{Z1_{N} }} \in \left[ {I_{{Z1_{U} }} ,I_{{Z1_{U} }} } \right]$$

(2)

The quantity ${Z}_{1N}\epsilon \left[{Z}_{1L},{Z}_{1U}\right]$ can be written as

$$Z_{1N} \in \left\{ {\frac{1}{2}log_{e} \left( {\frac{{1 + r_{L} }}{{1 - r_{L} }}} \right),\frac{1}{2}log_{e} \left( {\frac{{1 + r_{U} }}{{1 - r_{U} }}} \right)} \right\}$$

(3)

Note that ${Z}_{1N}\epsilon \left[{Z}_{1L},{Z}_{1U}\right]$ follows the neutrosophic normal distribution. The mean, say ${\mu }_{{Z}_{1N}}$ of ${Z}_{1N}\epsilon \left[{Z}_{1L},{Z}_{1U}\right]$ is given by

$$\mu_{{Z_{1N} }} \in \left\{ {\frac{1}{2}log_{e} \left( {\frac{{1 + \rho_{0L} }}{{1 - \rho_{0L} }}} \right),\frac{1}{2}log_{e} \left( {\frac{{1 + \rho_{0U} }}{{1 - \rho_{0U} }}} \right)} \right\}$$

(4)

where ${\rho }_{0N}$ represents the specified value of the correlation coefficient.

The variance, say ${\sigma }_{{Z}_{1N}}$ of ${Z}_{1N}\epsilon \left[{Z}_{1L},{Z}_{1U}\right]$ is given by

$$\sigma_{{Z_{1N} }} \in \left\{ {\frac{1}{{\sqrt {n_{L} - 3} }},\frac{1}{{\sqrt {n_{U} - 3} }}} \right\}$$

(5)

The neutrosophic test statistic ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ is defined as

$$Z_{N} = Z_{L} + Z_{U} I_{{Z_{N} }} ;I_{{Z_{N} }} \in \left[ {I_{{Z_{L} }} ,I_{{Z_{U} }} } \right]$$

(6)

The statistic ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ can be written as

$$Z_{N} = \frac{{Z_{1L} - \mu_{{Z_{1L} }} }}{{\sigma_{{Z_{1L} }} }} + \frac{{Z_{1U} - \mu_{{Z_{1U} }} }}{{\sigma_{{Z_{1U} }} }}I_{{Z_{N} }} ;I_{{Z_{N} }} \in \left[ {I_{{Z_{L} }} ,I_{{Z_{U} }} } \right]$$

(7)

Note that the proposed statistic ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ is an extension of the existing $Z$-test. The proposed ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ reduces to the existing $Z$-test under classical statistics when ${I}_{{Z}_{L}}$=0.

Simulation studies

In this section, we will simulate the impact of the measure of indeterminacy on the type-I error, denoted by $\alpha$, which represents the probability of rejecting the null hypothesis when it is true. Additionally, we will investigate the way in which indeterminacy affects the power of the test $(1-\beta )$, with $\beta$ representing the probability of failing to reject the null hypothesis when it is false. Following the approach outlined by [28], we define the neutrosophic variable ${\rho }_{0N}$ as ${\rho }_{0L}+{\rho }_{0U}{I}_{N}$, where ${\rho }_{0L}$ represents the determinate value of the correlation coefficient, ${\rho }_{0U}{I}_{N}$ is the indeterminate part, and ${I}_{N}$ belongs to the interval $\left[{I}_{L},{I}_{U}\right]$, representing the degree of indeterminacy. Our analysis will initially focus on the impact of ${I}_{N}$ on the type-I error and subsequently examine its effect on the power of the test.

Effect of ${{\varvec{I}}}_{{\varvec{N}}}$ on type-I error

We will examine the impact of the measure of indeterminacy on the type-I error through a simulation conducted 10⁶ times. Following the approach outlined in [20], the type-I error is computed as the ratio of rejecting the null hypothesis to the total number of replicates. The type-I error values for both the classical statistics test and the proposed test, across various values of ${I}_{N}$, are depicted in Fig. 1. The lower curve in Fig. 1 represents the type-I error for the test using classical statistics, while the upper curve displays the values for the proposed test. The observation from Fig. 1 is that, for the classical statistics test, the type-I error remains consistent across all levels of indeterminacy. Conversely, the higher curve indicates an increase in the type-I error as the values of ${I}_{N}$ rise. This suggests a significant effect of the measure of indeterminacy on the evaluation of the type-I error, cautioning decision-makers to exercise care when making decisions regarding hypothesis testing in the presence of uncertainty.

Fig. 1

Graphs illustrating the type-I error for both tests

Effect of ${{\varvec{I}}}_{{\varvec{N}}}$ on type-II error

We will assess how the degree of indeterminacy influences the test's efficacy through a simulation conducted a million times. Following the methodology outlined by [20], the type-II error is calculated as the ratio of incorrect decisions to the total number of replicates. Table 1 presents the type-II error values for both the conventional statistical test and the proposed test across various levels of ${I}_{N}$. Figure 2 illustrates the trends in test power. In Fig. 2, the lower curve represents the power of the test using classical statistics, while the upper curve depicts the power of the test for the proposed method. Figure 2 reveals that, for the classical statistics test, the power remains consistent regardless of the level of indeterminacy. In contrast, the higher curve indicates a decline in test power as ${I}_{N}$ values increase. This implies a significant impact of the measure of indeterminacy on test performance. This study suggests that, unlike the classical statistics test, the proposed test's power is affected by the degree of indeterminacy. Consequently, it is concluded that relying on the existing test under classical statistics may lead decision-makers astray when making decisions in the presence of uncertainty.

Table 1 The power of the test

Fig. 2

Power of the test curves

Application

This section presents the application of the proposed $Z$-test for correlation using the heartbeat (HBT) and temperature (TMP) data. The medical decision-makers are interested in investigating the relationship between the HBT and TMP. The primary and secondary healthcare department is responsible for the delivery of essential and effective health services in the province of Punjab, Pakistan. Punjab Health Facilities Management Company (PHFMC) on behalf of the health department engages in providing the required services. Basic Health Unit (BHU) is the first level health care unit under the supervision of qualified doctors which usually covers around 10,000 to 25,000 population. Three months (June 2021 to August 2021) patients’ daily data who visited BHU with reporting gastritis (authenticated and reported by a qualified medical doctor). The minimum and maximum values of the patients visited in a day are recorded and the data of two variables is arranged in intervals. The schematic diagram is shown in Fig. 3. The interval data of HBT and TMP is recorded and reported in Table 2. From the data given in Table 2, it can be seen that the medical decision-makers cannot apply the existing Z-test to investigate the significance of the correlation between HBT and TMP. The use of the proposed Z-test for correlation seems suitable for analyzing HBT and TMP data.

Fig. 3

The schematic diagram

Table 2 The HBT and TEMP data

The proposed Z-test for correlation using the HBT and TMP data is carried out as: the neutrosophic correlation ${r}_{N}\epsilon \left[{r}_{L},{r}_{U}\right]$ is calculated as follows ${r}_{N}\epsilon \left[\mathrm{0.2024,0.4333}\right]$ and expressed in neutrosophic form as ${r}_{N}=0.2024+0.4333{I}_{{r}_{N}};{I}_{{r}_{N}}\epsilon \left[\mathrm{0,0.5329}\right]$. The quantity ${Z}_{1N}\epsilon \left[{Z}_{1L},{Z}_{1U}\right]$ is calculated as ${Z}_{1N}\epsilon \left[\mathrm{0.2052,0.4640}\right]$. The mean and standard deviation are calculated as ${\mu }_{{Z}_{1N}}\epsilon \left[\mathrm{0.5493,0.5493}\right]$ and ${\sigma }_{{Z}_{1N}}\epsilon \left[\mathrm{0.1222,0.1222}\right]$, respectively. The proposed test statistic $\left|{Z}_{N}\epsilon \left[{Z}_{L},{Z}_{U}\right]\right|$ is calculated as ${Z}_{N}=2.8162-0.6981{I}_{{Z}_{N}};{I}_{{Z}_{N}}\epsilon \left[\mathrm{0,3.0341}\right]$. Suppose that the value of the level of significance $\alpha$ = 0.05. The proposed test for investigating the relationship between HBT and TMP is implemented as.

• Step 1: State null hypothesis ${H}_{0N}:$ correlation between HBT and TMP is ${r}_{0N}=0.50$ vs. the alternative hypothesis ${H}_{1N}:$ correlation between HBT and TMP is less than ${r}_{0N}=0.50$.

• Step 2: For $\alpha$=0.10, the tabulated value from [16] is 1.64.

• Step 3: Compare ${Z}_{N}\epsilon [\mathrm{2.8162,0.6981}]$ with 1.64 and reject ${H}_{0N}$ if ${Z}_{N}\epsilon \left[\mathrm{2.8162,0.6981}\right]>1.64$.

By comparing the values of ${Z}_{N}\epsilon \left[\mathrm{2.8162,0.6981}\right]$ with 1.64, it is clear that the lower value of ${Z}_{L}$ is larger than 1.64, so, the null hypothesis ${H}_{0N}$ will rejected in favor of ${H}_{1N}$. On the other hand, the upper value of statistic ${Z}_{U}$ is smaller than 1.64 which leads to rejection of ${H}_{1N}$. From the analysis, it is clear that the determinate part which presents the statistic using classical statistics indicates that the correlation between HBT and TMP is less than 0.50. On the other hand, the indeterminate part shows that the correlation between HBT and TMP is 0.50. Under uncertainty, it is expected that there will be significant correlation between HBT and TMP.

Comparative studies based on HBT and TMP data

Based on the analysis of HBT and TMP data, the comparisons of the proposed Z-test for correlation are carried out with the existing Z-test for correlation using the classical statistics, fuzzy-based test and interval-statistics in terms of information and flexibility of the results. The neutrosophic forms of the correlation ${r}_{N}$ and the test statistic $\left|{Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]\right|$ are presented as ${r}_{N}=0.2024+0.4333{I}_{{r}_{N}};{I}_{{r}_{N}}\epsilon \left[\mathrm{0,0.5329}\right]$ and ${Z}_{N}=2.8162-0.6981{I}_{{Z}_{N}};{I}_{{Z}_{N}}\epsilon \left[\mathrm{0,3.0341}\right]$, respectively. From the results, it can be analyzed that under indeterminacy, the correlation between HBT and TMP may vary from 0.2024 to 0.4333. The values of statistic ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ may vary from $2.8162$ to $0.6981$. Note that as mentioned before the first values 0.2024, and $2.8162$ present the results of the Z-test for correlation using classical statistics. The statistical test using classical statistics states that the probability of rejecting ${H}_{0N}:$ the correlation between HBT and TMP is ${r}_{0N}=0.50$ when it is true is 0.05 and the probability of accepting ${H}_{0N}:$ a correlation between HBT and TMP is ${r}_{0N}=0.50$ is 0.95. On the other hand, the proposed test for correlation states that the probability of rejecting ${H}_{0N}:$ the correlation between HBT and TMP is ${r}_{0N}=0.50$ when it is true is 0.05, the probability of accepting ${H}_{0N}:$ correlation between HBT and TMP is ${r}_{0N}=0.50$ is 0.95 and the measure of indeterminacy/uncertainty associated with the decision is $3.0341$. Similarly, the Z-test using fuzzy-logic gives the information about the statistic ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ in intervals only. According to the fuzzy-based statistical test, it can be expected that the values of ${Z}_{N}\in \left[{Z}_{L},{Z}_{U}\right]$ may vary from $2.8162$ to $0.6981$. The fuzzy-based analysis and interval-analysis only give information in intervals and are unable to give any information about the measure of indeterminacy. From the comparative studies, it is concluded that the proposed Z-test for correlation is more informative than the test using classical statistics, fuzzy-based analysis and interval-based analysis.

Discussions based on HBT and TMP data

The main aim of the paper is to investigate the significance of the relationship between HBT and TMP. The neutrosophic form correlation analysis of HBT and TMP is ${r}_{N}=0.2024+0.4333{I}_{{r}_{N}};{I}_{{r}_{N}}\epsilon \left[\mathrm{0,0.5329}\right]$. The correlation analysis of HBT and TMP shows that the correlation between HBT and TMP may vary from 0.2024 to 0.4333. As mentioned earlier, the correlation value 0.2024 denotes the correlation using classical statistics. The value $0.4333{I}_{{r}_{N}}$ denotes the correlation related to the indeterminate part. From the correlation analysis of the determined part that is 0.2024, it can be seen that there is weak correlation between HBT and TMP. It means that the increase in TMP does not increase the HBT significantly. Under indeterminacy, the correlation of the indeterminate part is 0.4333. This means that there is a moderate correlation between HBT and TMP. It means that the increase in TMP may increase HBT. From the correlation analysis, it can be seen that although the correlation of the determinate part is insignificant as the measure of indeterminacy increases, it can increase the correlation between HBT and TMP. Therefore, the decision makers should be careful in dealing with patents having the diseases of HBT and TMP.

Concluding remarks

The paper discussed the adaptation of the Z-test of correlation through the application of neutrosophic statistics. It provided an explanation of the rationale behind employing the proposed Z-test and detailed the neutrosophic test statistic along with the corresponding implementation steps. The simulation study conducted in the paper led to the conclusion that there is a notable impact of indeterminacy on both the type-I error and the power of the test. The paper demonstrated the application of the proposed test using data from HBT and TMP intervals. The findings revealed that, in the presence of indeterminacy or when dealing with interval data, the correlation between HBT and TMP increases as the measure of indeterminacy rises. The analysis suggests that decision-makers can effectively use the proposed test to explore correlations between variables in diverse fields such as medical science, business, and industry. Additionally, the paper suggested avenues for future research, including the exploration of the proposed test using a resampling scheme. It also recommended further investigation into additional statistical properties as potential areas for future research.