Jump to ContentJump to Main Navigation
Show Summary Details
More options …

The International Journal of Biostatistics

Ed. by Chambaz, Antoine / Hubbard, Alan E. / van der Laan, Mark J.

2 Issues per year


IMPACT FACTOR 2016: 0.500
5-year IMPACT FACTOR: 0.862

CiteScore 2016: 0.42

SCImago Journal Rank (SJR) 2016: 0.488
Source Normalized Impact per Paper (SNIP) 2016: 0.467

Mathematical Citation Quotient (MCQ) 2016: 0.09

Online
ISSN
1557-4679
See all formats and pricing
More options …

Fuzzy Set Regression Method to Evaluate the Heterogeneity of Misclassifications in Disease Screening with Interval-Scaled Variables: Application to Osteoporosis (KCIS No. 26)

Li-Sheng Chen
  • School of Oral Hygiene, College of Oral Medicine, Taipei Medical University, No. 250 Wu-Hsing Street, Taipei, Taiwan
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Ming-Fang Yen
  • School of Oral Hygiene, College of Oral Medicine, Taipei Medical University, No. 250 Wu-Hsing Street, Taipei, Taiwan
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Yueh-Hsia Chiu
  • Department and Graduate Institute of Health Care Management, College of Management, Chang Gung University, 259 Wen-Hwa 1st Road, Kwei-Shan Tao-Yuan, Taiwan
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Hsiu-Hsi Chen
  • Corresponding author
  • Graduate Institute of Epidemiology and Preventive Medicine, College of Public Health, Nation Taiwan University, No. 17, Hsuchow Road, Taipei, Taiwan
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2014-08-05 | DOI: https://doi.org/10.1515/ijb-2014-0032

Abstract

Although the trade-off between the two misclassifications (false-positive fraction and false-negative fraction), corresponding to type I and type II error in statistical hypothesis testing based on Neyman–Pearson lemma, to determine the optimal cutoff in the province of evaluating the accuracy of medical diagnosis and disease screening using interval-scaled biomarkers has been attempted by the receiver operating characteristic (ROC) curve, the heterogeneity of the two misclassifications in relation to the utility or individual preference for relative weights between the two errors has been barely addressed and has increasingly gained attention in disease screening when the optimal subject-specific or subgroup-specific cutoff (the heterogeneity of ROC curve) is underscored. We proposed a fuzzy set regression method to achieve such a purpose. The proposed method was illustrated with data on screening for osteoporosis with bone mineral density.

Keywords: fuzzy set regression method; disease screening; utility ratio

Assuming the fuzziness of being classified as disease type can capture the heterogeneity of the two misclassifications that is related to the utility for the trade-off between the two errors. The fuzzy set regression model using the two logit membership functions was adopted to capture such heterogeneity that is explained by relevant individual-based covariates. The fuzzy utility ratio was further derived to assess the impact of individual heterogeneity on the intercept, slope, and quadratic parameters of the sensitivity function and the false-positive fraction (FPF) related to the receiver operating characteristics (ROC) curve. Regression coefficients of covariates encoded in the two membership functions were estimated by simultaneously solving the two equations related to sensitivity and FPF making allowance for inter-dependence of two sets of regression coefficients. Given the predetermined fuzzy utility ratio, the optimal subject-specific sensitivity and specificity for different risk profiles were determined.

When the proposed fuzzy set regression method was applied on data for screening of osteoporosis with bone marrow density, the significant risk factors responsible for the two error rates including age, obesity, and menopause for bone mineral density (BMD) was assessed. We present different estimated results for the impacts of age, obesity, and menopause for the BMD example on the higher FPF and intercept, slope, and quadratic component of the ROC curve. The heterogeneity of fuzzy utility ratios with different risk groups given different optimal subject-specific cutoffs is also presented.

The proposed fuzzy set regression method is useful for identifying the heterogeneity associated with the two misclassifications which has a significant implication for finding an individual-based optimal cutoff in disease screening using interval-scaled biomarkers.

1 Introduction

Minimizing a linear combination of type I and II errors for optimal test has been well documented in statistical hypothesis testing [1] following the Neyman–Pearson lemma [2]. Such a notion on the trade-off between type I and II errors, corresponding to FPF and false-negative fraction (FNF), is applied to selecting the optimal cutoff in evaluation of diagnosis accuracy of chronic disease or cancer particularly when a test is characterized by an interval-scaled variable. It is therefore of interest to assess the reciprocal relation between true-positive fraction (TPF, sensitivity) and true-negative fraction (TNF, specificity) (the more stringent the cutoff selected, the higher the specificity is and the lower the sensitivity, and vice versa) by minimizing both errors.

To this end, the most widely used method in previous literature is by ROC curve, which has long been developed and applied in assessment of medical diagnostic technology [3, 4]. The details and good interpretation of ROC for evaluating diagnostic markers of disease refers to the classical book written by Pepe et al. [5]. Pepe et al. further showed how to apply the ROC curve to evaluate the performance of biomarker that is used in population-based screening [6]. While the conventional ROC methodology is useful for selecting the optimal cutoff it is adequate for finding the optimal mean cutoff at population level but may not be able to explicitly identify the optimal cutoff at subject-specific or subgroup-specific level. To determine the optimal subject-specific or group-specific cutoff may capture individual (patient or physician such as radiologist) preference or utility for the trade-off between the two errors that has been demonstrated in the two previous studies [7, 8].

From statistical viewpoint, such a consideration is particularly indispensable while the optimal subgroup- or subject-specific cutoff should be determined for random samples selected from different subpopulations. For instance, subject-specific or subgroup-specific optimal cutoff has increasingly gained attention in several cases of population-based screening. The optimal cutoff of fecal hemoglobin concentration resulting from fecal immunological test occult for colorectal cancer screening has been suggested on the basis of age-gender-specific cutoff [9]. Age-specific cutoff of prostate specific antigen has been suggested for prostate cancer screening [10, 11]. The optimal cutoff for BMD score measured by quantitative ultrasound in relation to the risk of osteoporosis also varies with menopause status and other risk factor [12]. Subject-specific cutoff also plays a crucial role in individually tailored screening strategy on the choice of appropriate intervention or treatment modalities. To ascertain the optimal subgroup-specific or subject-specific cutoff is tantamount to evaluating the heterogeneity of misclassifications involved with the two errors. This thorny issue on the heterogeneity of misclassification has been considered by the derivation of covariate-specific ROC curves based on modeling the joint effect of relevant covariates on the survivor function of the test for disease and non-disease population [13]. In spite of its usefulness in clinical application, the ROC method cannot explicitly show how to minimize the two errors involved in determining the optimal cutoff in connection with the individual preference or utility of judging relative weight between type I and type II error. The covariate-specific ROC approach may also be difficult to accommodate the circumstance when empirical data on the measurement of individual preference or utility is provided. Here, we provide an alternative approach to solving this issue regarding the heterogeneity of misclassifications in the context of fuzzy set theory.

Under the semantics of fuzzy sets [14], the degree of similarity or uncertainty and the degree of preference or utility can be considered simultaneously. The degree to which a subject belongs to the disease group or the non-disease group in evaluation of diagnosis accuracy of chronic disease or cancer is related to the degree of uncertainty with respect to the two errors. The relative importance between FNF and FPF is a reflection of other degree of preference or utility standing for a set of more or less preferred objects [15], which is often captured by the membership function (tailored for measuring the degree of being classified) in fuzzy set theory. Assessment of the degree of being classified in the disease and non-disease group is measured by the degree of fuzziness, sometimes called fuzzy index that reflects individual utility or preference for the trade-off between the two error rates and can be approximated by defining appropriate membership function with the incorporation of factors affecting the degree of being classified as disease or non-disease so as to assess the heterogeneity of misclassifications mentioned above. The major aim of this study was to minimize the degree of fuzziness through the two membership functions in association with FPF and FNF in order to estimate the optimal subject-specific or subgroup-specific cutoff and solve the heterogeneity of misclassifications by developing the fuzzy set regression method.

2 An example of BMD screening for osteoporosis

Data on quantitative ultrasound for measurement of BMD as a screening tool for osteoporosis, a more common chronic disease in elderly women [16], is faced with the heterogeneity of misclassification mentioned above. In osteoporosis screening with quantitative ultrasound, quantitative ultrasound measurements were taken of the calcaneus for each participant by well-trained technicians for determining the individual bond mass. T-score regarded as an interval-scaled variable was recorded after quantitative ultrasound assessment. BMD values are expressed as absolute values in g/cm2 or as T-scores from the young adult. The T-score derived from reference data obtained in premenopausal women which is most widely used for diagnostic criteria of osteoporosis.

Researchers have suggested several different quantitative ultrasound cutoff thresholds, including quantitative ultrasound index T-scores of 0, –1, and –1.5 to determine which subjects should be considered for additional testing with dual-energy x-ray absorptiometry (DXA). A meta-analysis revealed the 79% of sensitivity and 58% of specificity when T-score cutoff threshold of –1 and the areas under the ROC curves ranged from 0.7 to 0.9 for osteoporosis by using quantitative ultrasound as BMD measurement tool from 25 studies [17]. For a T-score threshold of 0, sensitivity improved to 93% but specificity decreased to 24%. When lower quantitative ultrasound index T-score cutoff thresholds are used, the FNF increases and the FPF decreases. This systematic review also found that the sensitivity and specificity of quantitative ultrasound at commonly used cutoff thresholds seem to be too low to provide the additional DXA test for osteoporosis and the heterogeneity of areas under curves did exist for selecting optimal cutoffs depending on age, gender, and menopause status. This suggests the necessity of considering the optimal cutoff at subject or subgroup level. The fuzzy set regression method was therefore developed to optimize sensitivity and specificity considering the heterogeneity of misclassification with the incorporation of age, obesity, and menopause into the regression model defined by the two membership functions for assessing the degree of being classified as disease and non-disease group. The joint effects of these factors on sensitivity and specificity on the two membership functions were assessed when the degreed of fuzziness in relation to the two membership functions was minimized.

The rest of this manuscript is organized as follows. In the methodology of Section 3, we begin with the introduction of conceptual framework of fuzzy set regression method in Section 3.1 and we then define in Section 3.2 the basic concepts and notations of disease screening and also derive fuzzy index by measuring the degree of being classified as disease and non-disease group defined by the two membership functions with the incorporation of significant risk factors into the two corresponding logistic regression models underpinning the fuzzy set theory to account for the heterogeneity of both misclassifications. In Section 3.3, the fuzzy utility ratio obtained from the two membership functions was derived to link fuzziness with the two errors. The relationship between the fuzzy utility ratio and the slope of sensitive function using ROC analysis was derived by minimizing the degree of fuzziness through the two simultaneous equations (corresponding to the two membership functions) on the FNF and FPF to assess the significance of each covariate contributing to heterogeneity of misclassifications. Section 3.4 gives the derivation of optimal sensitivity and specificity given the assigned fuzzy utility ratio. Estimation and statistical testing of parameters are provided in Section 3.5. The proposed fuzzy regression model was illustrated by using the empirical data on BMD screening for osteoporosis from a Keelung community-based integrated screening (KCIS). Descriptions of target population, study design, contents of the KCIS program, and descriptive results are given in Section 4. Section 5 gives the results on the model selection of significant factors accounting for the fuzziness of the two membership functions, the estimated regression coefficients of three significant factors related to TPF and FNF, the plot of fuzzy index by different cutoffs of BMD, and how to use the fuzzy utility ratio to identify optimal sensitivity and specificity. Discussions on the heterogeneity of misclassification, the application to screening policy, and methodological consideration are given in Section 6.

3 Methodology

3.1 Conceptual framework of fuzzy set regression approach

Figure 1 outlines our conceptual framework for developing the fuzzy set regression method that applied to disease screening with an interval-scaled variable. The first step is to select a cutoff based on a biomarker with an interval-scaled variable. If the heterogeneity of both misclassifications is not considered, the optimal cutoff for the overall group can be determined by the conventional ROC method. If it is required, we assume such heterogeneity conceptually can capture the utility in association with the trade-off between FNF and FPF and the relative value between FNF and FPF can be measured by the degree of fuzziness. In the context of the fuzzy set, the two membership functions with the incorporation of covariates are therefore constructed to capture the fuzziness of being classified in the disease group or the non-disease group to reflect the heterogeneity of both misclassifications. The fuzzy utility ratio corresponding to the two membership functions was developed. Minimizing the fuzziness was done by using sensitivity (TPF) function as a function of FPF to link the prevalence of disease and the fuzzy utility ratio in the membership-function-based ROC analysis. To determine the optimal sensitivity and specificity, given the threshold of fuzzy utility, the two membership functions were used to assess how these covariates simultaneously affect the intercept and slope of the sensitivity (TPF) function and the FPF, which enables us to conduct formal statistical testing for the variables of interest encoded in the two membership functions. The following section gives the details of each step.

The framework of disease screening with interval-scaled variable using fuzzy set regression method
Figure 1

The framework of disease screening with interval-scaled variable using fuzzy set regression method

3.2 Derivation of membership functions and fuzzy index

We began with the definition of sensitivity and specificity in disease screening with interval-scaled biomarker by given the following notations.

Given a cutoff, say aˉ, the sensitivity is defined as the probability of a positive result given true disease status. The specificity is defined as the probability of a negative result given true negative disease status. Let disease status be denoted by two states, disease (D) and non-disease (D¯) and let X stand for the value of a certain biological marker, such as the BMD value. Given cutoff aˉ, the result of a test is either positive (xaˉ) or negative (x<aˉ).

3.3 Membership function underpinning fuzzy set

According to the basic fuzzy concept, for the ith individual, the degree of being classified in the disease group is denoted as giD giD={1ifXa¯π=11+e(α+β1χi1+β2χi2++βqχiq)ifX<a¯(1)and the degree of being classified in the non-disease group is denoted as giD¯ giD¯={1ifX<āπ=11+e(α+β1χi1+β2χi2++βqχiq)ifXā(2)In eq. (1), α,β1,β2,,βq are the regression coefficients of the logit membership function for the probability of x<aˉ in the disease group. In eq. (2), α,β1,β2,,βq are the regression coefficients of the logit membership function for the probability of xaˉ in the non-disease group.

In the fuzzy set, to measure the fuzziness of being classified in the disease group and non-disease group, we use the operator “min” to obtain the intersection of giD and giD¯. Following the notations from Cruz’s study [18], the fuzzy index (F) is given as follows: F=γ(t)min(giD,giD¯)dt=cγ(t)πidt+cγ(t)πidt=cγ(t)11+e(α+β1χi1+β2χi2++βqχiq)dt+cγ(t)11+e(α+β1χi1+β2χi2++βqχiq)dt=11+e(α+β1χi1+β2χi2++βqχiq)Φ(c)+11+e(α+β1χi1+β2χi2++βqχiq)(1Φ(c))=πcΦ(c)+πc(1Φ(c))(3)In eq. (3), t denotes the random variable with interval scale and r(t) represents the probability density function (p.d.f.) of t following a normal distribution. Here, πc and πc represent the predictive probability of disease and non-disease, respectively, under a cutoff of t, say c. If more covariates are included in the membership function, a higher value of πc or πc could be obtained. By aggregating the two distributions of πc and πc with Φ(c) together, it is possible to present the heterogeneity of the fuzzy index (F) using the percentile of risk scores derived from a set of significant covariates multiplied by the corresponding regression coefficients encoded in the two logit membership functions, as shown in eqs (1) and (2). Estimation and statistical testing of the inter-dependency of the two sets of parameters are detailed in the following section.

3.4 Fuzzy utility ratio associated with disease screening

As pointed out earlier, the misclassification of disease status in each subject consists of two parts, the FNF and FPF. In terms of a crisp set, this can be expressed as follows: Falsepositivefraction(FPF)=PXaˉ|DFalsenegativefraction(FPF)=PX<aˉ|D+(4)Eq. (4) captures the uncertainty of the two errors but does not capture the degree to which a subject belongs to the disease or non-disease group. To achieve the aim of the latter, we link the fuzzy index to the utility for the trade-off between the two errors. Let F1 and F2 be two membership scores (the range between 0 and 1) associated with the misclassifications due to FNF and FPF. Using the basic definition of expectation, we can derive the expected fuzziness according to the probabilities of the two misclassifications (FNF and FPF) weighted by F1 and F2, respectively. By treating TPF as function of FPF following the concept of the ROC analysis, we minimized the degree of fuzziness with respect to FPF. The fuzzy utility ratio F2/F1 was developed and equivalent to F2F1=P(D)P(D¯)×(m)(5)where m is the slope of fuzziness of FPF and P(D) represents the prevalence of disease and P(D¯) is the complementary component. The details of the derivation regarding the link between the fuzziness and the utility in association with the two errors and how to minimize the degree of fuzziness are given in the Appendix.

To estimate m, a polynomial linear regression model based on the unique shape of the ROC curve was adopted with the incorporation of a quadratic component to fit the relationship between FPF and FNF. To match the traditional ROC curve, which has the TPF on the y-axis and FPF in the x-axis, the FPF function is changed to a TPF function and written as follows: P+|D=γ0+γ1P+|D¯+γ2P+|D¯2+ε(6)We used an identical link by linking the intercept, slope, and quadratic parameters with a full or partial set of covariates under the logit membership functions of gD as follows: γ0=logitπs1=X0β0, γ1=logitπs2=X1β1, and γ2=logitπs3=X2β2, where s1, s2, and s3 stand for a subset of covariates (X0Xq) used in the membership function of gD in eq. (1).

To capture the relationship between the FPF and covariates as shown in the membership function of gD in eq. (2), we used a logit regression model with P(+|Dˉ)=11+eα+β1χi1+β2χi2++βqχiq+ξ(7)To assess the inter-dependency between the regression coefficients involved in the two equations, two simultaneous equations (6) and (7) for the FPF and TPF function corresponding to each membership function were developed and resolved to estimate the parameters of the regression coefficients related to the FPF and intercept, slopes, and quadratic component. Note that ξ and ε are error terms for the FPF and sensitivity, respectively, and are correlated with each other, rather than independent terms. So doing is more flexible than the conventional way of achieving the optimal procedure that the type II error is minimized given type I error is not more than pre-assigned significant level following the Neyman–Pearson lemma.

Eqs (6) and (7) can be extended to a generalized system for multiple markers, say s, as follows: {P1(+|D¯)=11+e(α1+β1.1xi1+β1.2xi2++β1.qxiq+ξ1)                      P1(+|D)=γ1.0+γ1.1P(+1|D¯)+γ1.2P(+1|D¯)2+ε1P2(+|D¯)=11+e(α2+β2.1xi1+β2.2xi2++β2.qxiq+ξ2)                    P2(+|D)=γ2.0+γ2.1P(+2|D¯)+γ2.2P(+2|D¯)2+ε2Ps(+|D¯)=11+e(αs+βs.1xi1+βs.2xi2++βs.qxiq+ξs)                    Ps(+|D)=γs.0+γs.1P(+s|D¯)+γs.2P(+s|D¯)2+εs(8)where Pj+|D and Pj+|Dˉ are the TPF and FPF, respectively, with the jth marker (1, 2,…, s) and a set of covariate X (X = 1,…, q). αj and βj.z are regression coefficients for the FPF with jth marker, and γj.l (l = 0, 1, 2) are defined in the same manner as above for a true positive with marker j. Basically, the development of multiple simultaneous equations for multiple markers evolving from one marker is like the extension of univariate analysis to multivariate analysis from a statistical viewpoint. Here, we only illustrate univariate analysis. The parameter estimation of regression coefficients related to the probability of a false-positive case, P+|Dˉ, the random intercept (γs0), slope (γs1), and quadratic parameter (γs2) is addressed in the following section of parameter estimation and statistical hypothesis testing.

3.5 Optimal specificity and sensitivity

The derivation of the first derivative of eq. (6) with respect to P(+|D¯) gives the slope m. m=(γ1+2γ2P(+|D¯))(9)Substituting eq. (9) into eq. (5) gives eq. (10). F2F1=P(D)P(D¯)×γ1+2γ2P(+|D)¯(10)Given the assigned fuzzy utility ratio (k = F2/F1) and the odds on the prevalence of disease, the optimal specificity (1 – FPF) obtained by combining eqs (5), (9), and (10) is Pˆ+|D¯=mˆ+γˆ12γˆ2.(11)The corresponding optimal sensitivity is Pˆ+|D=γˆ0+γˆ1Pˆ+|D¯+γˆ2Pˆ+|D¯2.(12)

3.6 Parameter estimation and statistical testing

To estimate the regression coefficients of the covariates of the two logit membership functions simultaneously by considering the inter-dependency of two sets of parameters, we combined the equations for gD and gD¯ illustrated in eqs (1) and (2). Applied to disease screening, the unified equation is written as follows: PXaˉ|D,X1,X2,...,Xq=α0Dˉ+β0D+β1DX1D+β2DX2D++βqDXqD+β1DˉX1Dˉ+β2DˉX2Dˉ++βqDˉXqDˉ(13)where D is the indicator variable for the presence of true disease and X1Xq are covariates related to the individual preference. X1D=X1×D,X1D¯=X1×(1D),X2D=X2×D,X2D¯=X2×(1D),,XqD=Xq×D,andXqD¯=Xq×(1D)The likelihood ratio test was used to assess whether a set of covariates or each covariate is statistically significant by assessing whether the difference of −2 log-likelihood between the two models under comparison given the degrees of freedom departs from the central chi-squared distribution.

Regarding the method for estimating parameters on the ROC curve using eqs (6) and (7) in univariate analysis (single marker) and eq. (8) in multivariate analysis (multiple markers), we considered a different estimation method. Because the FPF is a regressor for TPF, ordinary least-squares estimation may lead to a biased estimate. To consider two simultaneous equation biases and the cross-equation correction of error terms, as the two error terms are correlated through P+|D¯, a two-stage least-squares method was adopted [19]. The Wald test was used to assess the statistical significance using the criterion of whether the chi-squared value was greater than 3.84 given degree of freedom equal to 1.

4 Description on empirical data

To demonstrate how the fuzzy set regression method is applied to population-based disease screening, we illustrated its application using data derived from the Keelung community-based integrated screening (KCIS) program, which was launched in 1999. The details of KCIS program have been published elsewhere [20, 21]. Briefly, it is a population-based screening program that integrates five neoplastic diseases (cervix, breast, oral, liver, and colorectal cancer) and three chronic diseases (type 2 diabetes, hypertension, and hyperlipidemia). In addition, as the KCIS program also offers an opportunity to screen other chronic diseases for osteoporosis with BMD measurements was also included.

In the KCIS program, the BMD is measured in parallel with periodic screening for cancers and chronic disease as mentioned above. The unique character of the KCIS program is that it followed up the screenees over time to obtain a series of multiple outcomes for cancer and chronic diseases, the incident cases of osteoporosis can also be ascertained. Thus, it forms a population-based longitudinal follow-up dataset for the following analysis. The study design is based on a prospective community-based cohort study consisting of female who were aged 30 years or over and participated in the community-based integrated screening between 1999 and 2004. In order to ensure the temporal relationship between bone marrow density and incident cases of osteoporosis. We excluded prevalent cases who had been already diagnosed as osteoporosis before the date of first screen. A total number of 21,528 female were enrolled in the following analysis. Of 21,528, we identified 1,205 incident cases of osteoporosis. The KCIS program also provides comprehensive information on demographic factors, biochemical variables, anthropometric measurements, and reproductive factors. This enables the assessment of the relationship of BMD to osteoporosis making allowance for three covariates, age, menopause, and obesity are used here. Table 1 shows the distributions on the frequencies and mean values of three variables by osteoporosis and non-osteoporosis. The mean values and ranges of BMD between the osteoporosis group and non-osteoporosis group are also listed in Table 1.

Table 1

The distribution of selective factors in association with osteoporosis in the KCIS cohort

We applied eq. (13) as above to our data on BMD for osteoporosis screening to have the following equation: PBMD2|D,X1Age,X2Obesity,X3Menopause=α0Dˉ+β0D+β1DX1D+β2DX2D+β3DX3D+β1DˉX1Dˉ+β2DˉX2Dˉ+β3DˉX3Dˉ(14)where D is the indicator variable for the presence of true disease (osteoporosis = 1, non-osteoporosis = 0). X1D=X1(Age)×D,X1D¯=X1(Age)×(1D),X2D=X2(Obesity)×D,X2D¯=X2(Obesity)×(1D),X3D=X3(Menopause)×D,andX3D¯=X3(Menopause)×(1D)Age is a continuous variable. Obesity was defined as an indicator variable for the presence of obesity (obesity = 1, non-obesity = 0). Menopause was defined as an indicator variable for the menopause status (menopause = 1, non-menopause = 0).

5 Results

5.1 Fuzzy index with membership functions of the two error rates

To test whether a set of covariates or each covariate made a significant contribution to both the FNF and the FPF, defined by the two logit membership functions, eq. (16) was applied to different cutoffs. Given a cutoff of –1.5 (the cutoff a) for BMD, Table 2 shows the model selection and the assessment of statistical significance for three variables. We began with whether the set of three covariates (age, obesity, and menopause) was statistically significant (χ2(2) = 3,635.29; p <0.001) with the backward method. After retaining two variables, age and obesity (χ2(2) = 87.42; p <0.001) were significant risk factors, based on the comparison of the –2 log-likelihood values. Menopause was still significant for the membership functions when age and obesity were included in the model (χ2(2) = 3,761.50; p <0.001).

Table 2

Model selection and statistical testing for a set of covariates or each covariate in two logit membership functions

Figure 2 shows the fuzzy index with membership functions for different cutoffs of BMD. The fuzzy index was heterogeneous for the percentile of risk groups in light of the FNF (gD), increasing with the high-risk group regardless of the cutoff of BMD (Figure 2a). Both the FNF and the FPF were equally heterogeneous with the percentile of risk score category. Figure 2a and 2b shows the trade-off between the two misclassification rates when the cutoffs were altered. The figure indicates that the lowest fuzziness was between –1 and –2. Given a cutoff higher than –1.5, the low-risk groups showed greater fuzziness, whereas the opposite was noted for cutoffs lower than –1.5.

Fuzzy index by different cutoffs for percentile of risk score. (a) BMD with percentile of risk score determined by FNF. (b) BMD with percentile of risk score determined by FPF
Figure 2

Fuzzy index by different cutoffs for percentile of risk score. (a) BMD with percentile of risk score determined by FNF. (b) BMD with percentile of risk score determined by FPF

5.2 Fuzzy utility ratio

Table 3 shows the estimated results of the simultaneous equation systems for finding the optimal sensitivity and specificity given the fuzzy utility ratios. The results show that age, obesity, and menopause were positively associated with the FPF through eq. (7). The estimated regression coefficients (αs and βs) (see eq. (14)) linked through the logit membership functions of γ0, γ1, and γ2 mentioned in Section 3 are listed in Table 3. Regarding the effects of covariates on sensitivity function in eq. (6), obesity was significantly associated with the intercept (γ0) and slope (γ1). Age was a significant factor for the slope (γ1) and quadratic parameter (γ2).

Table 3

The estimated regression coefficients for γ0, γ1, and γ2 of sensitivity and FPF using two-stage least square simultaneous equations method

Figure 3 shows the heterogeneity of the fuzzy utility ratio for different risk groups given the cutoffs of BMD for osteoporosis. The fuzzy utility ratio decreased with the cutoff for BMD until it reached a plateau at a cutoff of around –2. The fuzzy utility ratio for a premenopausal obese 45-year-old woman was reduced from 186.1 at a cutoff of 1.5 to 17.2 at a cutoff of –2.0. A higher fuzzy utility ratio was also affected by the risk profiles, such as premenopausal status and younger age.

Heterogeneous results on fuzzy utility ratios (F1F2$${\raise0.7ex\hbox{${{F_1}}$} \!\mathord{\left/ {\vphantom {{{F_1}} {{F_2}}}}\right.\kern-\nulldelimiterspace}\!\lower0.7ex\hbox{${{F_2}}$}}$$) by different risk groups
Figure 3

Heterogeneous results on fuzzy utility ratios (F1F2) by different risk groups

Table 4 shows that the results of optimal sensitivity and specificity give the fuzzy utility ratio. With the fuzzy utility ratio set at 25, the optimal sensitivity and specificity for a 45-year-old obese premenopausal woman was 73.8% and 45.1%, respectively. So such finding is easily applied to the optimal subject-specific cutoff.

Table 4

Optimal sensitivity and specificity in the ROC curve by different covariates given predetermined fuzzy utility ratio

6 Discussion

The heterogeneity of misclassifications (FPF and FNF) in association with type I and type II errors under Neyman–Pearson lemma has increasingly gained attention in the field of evaluating the accuracy of medical diagnosis and disease screening based on a test characterized by a continuous biomarker. Assuming such heterogeneity is a reflection of utility or individual preference for the trade-off between the two errors, we make use of the fuzzy set theory to capture the fuzziness related to the utility or individual preference for such trade-off by the derivation of the two membership functions with the incorporation of significant factors that account for heterogeneity. The fuzzy set regression method was then built for solving the heterogeneity in association with the two misclassifications. The concept of linking fuzziness with sensitivity function as a function of FPF was envisaged to minimize the degree of fuzziness through the proposed unified fuzzy set regression so as to determine the optimal sensitivity and specificity. The novel estimation method using the two-stage least square method was also proposed to assess the inter-dependence of covariates affecting the fuzziness of the two misclassifications relaxing the traditional ROC analysis with independent assumption.

Although the trade-off between type I and type II errors has been optimized by Neyman–Pearson lemma and the corresponding optimal TPF and FPF has been solved by the ROC analysis, the issue on the heterogeneity of two misclassifications related to the utility of individual preference for minimizing the two errors still remained to be resolved. To the best of our knowledge, this is the first study to use the fuzzy set concept, together with the utility function related to FPF and FNF, to assess the heterogeneity of the misclassification of disease status with interval-scaled variables and to identify the optimal individual-based cutoff given different risk profiles. Our fuzzy set regression method has several unique characteristics. We introduced the two membership functions underpinning the fuzzy set to present the heterogeneity of the two misclassification rates. We proposed a unified regression model that simultaneously incorporates two logit regression models that include a series of significant risk factors to capture the degree of fuzziness of misclassification to which different individual characteristics contributed. This fuzzy set regression method is very flexible for handling interval-scaled biological markers affected by a mixture of other interval-scaled variables and categorical variables. The second is that because the fuzzy index does not present the relative weight of the utility associated with false-negative and false-positive cases, the fuzzy utility ratio approach was further developed to identify the subgroup optimal cutoff based on the trade-off between sensitivity and specificity given different risk profiles. Third, we proposed a novel statistical method for estimating parameters given a simultaneous equations system, which provides a very powerful approach for testing the statistical significance of putative factors affecting the degree of misclassification. The advance in making allowance for inter-dependence of the two simultaneous equations renders the minimization of the two errors more powerful and flexible compared with the optimal test based on the Neyman–Pearson lemma. Finally, our proposed fuzzy set regression method is very flexible to handling the circumstance when the fuzzy utility ratio (F2/F1) can be provided by empirical data on the survey of the measuring utility value in the association with FPF and FNF in the field of game theory.

The proposed fuzzy set regression method was illustrated with an example of osteoporosis screening with BMD. The fuzzy index score, based on the percentile of risk score derived from the two logit membership functions, gives a clear profile of the heterogeneity associated with the two misclassifications (Figure 2). The degree of fuzziness is determined not only by the percentile of the risk score but also by the relative contribution of the relevant covariates of fuzziness between FNF and FPF. It is obvious that the relevant covariates make equally significant contributions to the fuzziness in both error rates in the example of osteoporosis. Although the fuzzy index score can show the heterogeneity of the two misclassification rates, it is known that the utility ratio, calculated as the FPF divided by the FNF, indicates whether sensitivity or specificity is more important. The fuzzy utility ratio was used not only to show the heterogeneity of the two misclassification rates but also to identify the optimal sensitivity and specificity to determine the optimal subgroup cutoff, given different individual risk profiles.

It could be argued that one may consider the application of the conventional multiple logistic regression analysis without using information on fuzzy utility ratio to identify subgroup optimal cutoff. This can be achieved by first estimating clinical weights (regression coefficients), and 500 simulations were run based on this underlying model to generate the sensitivity and specificity given different cutoffs. The optimal sensitivity and specificity for each subgroup in combination with three covariates (age, obesity, and menopause) could be ascertained from 500 simulated estimates. The results show that the conventional logistic regression model is less discriminative to identify the optimal subgroup cutoff such as the covariate (obesity and menopause) in the illustration of 55 and 65 years of age in Table 4 (see the final column) than our proposed fuzzy regression model considering fuzzy utility ratio. The explanation for this disparity is that our proposed fuzzy regression model not only considers factors affecting the joint distribution of sensitivity and specificity but also considers the relative contribution between sensitivity and specificity using information on fuzzy utility ratio but the conventional multiple logistic regression failed to consider these two unique features.

Our fuzzy set regression method is also helpful for identifying individual cutoff values like an example of population-based osteoporosis screening with BMD. Using the predetermined fuzzy utility ratio, as shown in Figure 2, the optimal cutoff for different risk groups given the trade-off between sensitivity and specificity can be selected. The optimal cutoff value for different individual characteristics can also be identified by using the predetermined fuzzy utility ratio as mentioned above. The strategy developed here has significant clinical implications for individualized disease screening.

Although our method can accommodate the characteristics of screening with interval-scaled biological markers, this study raises two concerns. First, we only demonstrated how the proposed fuzzy set regression method was applied to a single marker versus multiple markers, although the proposed two-stage least-squares method can be extended to multiple markers, as in eq. (8). This should be tested with empirical data in the future. Second, we captured the heterogeneity of false-negative and false-positive cases using the two logit membership functions. It is necessary to extend other link functions under the framework of a generalized linear model in the future.

In conclusion, the fuzzy set regression method incorporating the membership functions encoded by a constellation of covariates was proposed to assess the heterogeneity of the two misclassifications (FPF and FNF) in disease screening with interval-scaled markers, which is related to the utility (measured by the fuzziness of being classified as disease or non-disease) for the trade-off between the two errors. To minimize the degree of fuzziness, the fuzzy set regression method in combination with the ROC optimization method given a fuzzy utility ratio was then applied to finding the optimal sensitivity and specificity, both of which, in turn, determine the optimal subgroup-specific or subject-specific cutoff for different risk groups. Our proposed fuzzy set regression method was demonstrated successfully with an example of screening for osteoporosis with bone marrow density.

Acknowledgments

We thank all of the staff who contributed to the KCIS program in the Keelung City Health Bureau and the Centre of Public Health. This research was supported by the National Science Council of Taiwan (NSC. 95-2314-B-002-019 and NSC. 100-2314-B-038-022) and Taipei Medical University (TMU99-AE1-B03).

Appendix

We link the fuzziness to the utility associated with false-negative and false-positive cases using the following expression: P(misclassification)=P(D)×P(negativetest()|D)+P(D¯)×P(postivietest(+)|D¯)Let F1 and F2 be two membership scores (the range between 0 and 1) associated with the misclassifications due to false-negative and false-positive cases, respectively. The expected fuzziness is expressed as follows: E(misclassification)=F1×P(D)×P(|D)+F2×P(D¯)×P(+|D¯)=F1×P(D)×fP(+|D¯)+F2×P(D¯)×P(+|D¯)(15)In eq. (15), F1 and F2 represent the fuzziness of false negative and false positive and fP(+|D¯) represents the function between false negative P|D and false positive P+|D¯.

To minimize the degree of fuzziness, let the first derivative of eq. (15) equal zero. dE(fuzziness)dP(+|D¯)=dfP(+|D¯)×F1×P(D)+F2×P(D¯)=m×F1×P(D)+F2×P(D¯)=set0(16)where m is the slope of fP(+|D¯).

This gives m×F1×P(D)=F2×P(D¯)(17)

References

  • 1. DeGroot MH, Schervish MJ. Probability and statistics, 4th ed. Boston, MA: Addison-Wesley, 2004. Google Scholar

  • 2. Neyman J, Pearson ES. On the problem of the most efficient tests of statistical hypotheses. Philos Trans R Soc A Math Phys Eng Sci 1933;231:289–337. Google Scholar

  • 3. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29–36. Google Scholar

  • 4. Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Crit Rev Diagn Imaging 1989;29:307–35. Google Scholar

  • 5. Pepe MS. The statistical evaluation of medical tests for classification and prediction. New York: Oxford University Press, 2003. Google Scholar

  • 6. Pepe MS, Etzioni R, Feng Z, Potter JD, Thompson ML, Thornquist M. Phases of biomarker development for early detection of cancer. J Natl Cancer Inst 2001;93:1054–61. Google Scholar

  • 7. Abbey CK, Eckstein MP, Boone JM. An equivalent relative utility metric for evaluating screening mammography. Med Decis Making 2010;30:113–22. Google Scholar

  • 8. Kuo SJ, Chen TH, Yen AM, Chen DR, Chen LS. Optimal two-stage breast cancer screening for countries with intermediate or low incidence of breast cancer. J Eval Clin Pract 2010;16:1345–52. Google Scholar

  • 9. McDonald PJ, Strachan JA, Digby J, Steele RJ, Fraser CG. Faecal haemoglobin concentrations by gender and age: implications for population-based screening for colorectal cancer. Clin Chem Lab Med 2011;50:935–40. Google Scholar

  • 10. Morgan TO, Jacobsen SJ, McCarthy WF, Jacobson DJ, McLeod DG, Moul JW. Age-specific reference ranges for prostate-specific antigen in black men. N Engl J Med 1996;335:304–10. Google Scholar

  • 11. Wu GH, Auvinen A, Yen AM, Hakama M, Tammela TL, Stenman UH, et al. The impact of interscreening interval and age on prostate cancer screening with prostate-specific antigen. Eur Urol 2012;61:1011–18. Google Scholar

  • 12. Hodson J, Marsh J. Quantitative ultrasound and risk factor enquiry as predictors of postmenopausal osteoporosis: comparative study in primary care. BMJ 2003;326:1250–1. Google Scholar

  • 13. Pepe MS. Receiver operating characteristic methodology. J Am Stat Assoc 2000;95:308–11. Google Scholar

  • 14. Dubois D, Prade H. An introduction to fuzzy systems. Clin Chim Acta 1998;270:3–29. Google Scholar

  • 15. Bellman R, Zadeh LA. Decision making in a fuzzy environment. Manage Sci 1970;17:B141–64. Google Scholar

  • 16. Nelson HD, Helfand M, Woolf SH, Allan JD. Screening for postmenopausal osteoporosis: a review of the evidence for the U.S. preventive services task force. Ann Intern Med 2002;137:529–41. Google Scholar

  • 17. Nayak S, Olkin I, Liu H, Grabe M, Gould MK, Allen IE. Meta-analysis: accuracy of quantitative ultrasound for identifying patients with osteoporosis. Ann Intern Med 2006;144:832–41. Google Scholar

  • 18. Amata Cruz GP, Beliakov G. Fuzzy gating and the problem of screening. Artif Intell Med 1996;8:377–85. Google Scholar

  • 19. Amemiya T. Advanced econometrics. Cambridge, MA: Harvard University Press, 1985. Google Scholar

  • 20. Chen TH, Chiu YC, Luh DL, Yen MF, Wu HM, Chen LS, et al. Taiwan community-based integrated screening group. Community-based multiple screening model: design, implementation, and analysis of 42,387 participants. Cancer 2004;100:1734–43. Google Scholar

  • 21. Chiu YH, Chen LS, Chan CC, Liou DM, Wu SC, Kuo HS, et al. Health information system for community-based multiple disease screening in Keelung, Taiwan (KCIS no 3). Int J Med Inform 2006;75:369–83. Google Scholar

Footnotes

    About the article

    Published Online: 2014-08-05

    Published in Print: 2014-11-01


    Citation Information: The International Journal of Biostatistics, ISSN (Online) 1557-4679, ISSN (Print) 2194-573X, DOI: https://doi.org/10.1515/ijb-2014-0032.

    Export Citation

    © 2014 by Walter de Gruyter Berlin / Boston. Copyright Clearance Center

    Comments (0)

    Please log in or register to comment.
    Log in