Skip to main content
Download PDF
Download ePub

Cross-cultural adaptation and psychometric properties of the Italian version of the Orthorexia Nervosa Inventory (ONI)

Journal of Eating Disorders volume 11, Article number: 144 (2023) Cite this article

A Correction to this article was published on 18 September 2023

This article has been updated

Abstract

Background

Orthorexia Nervosa (ON) is an emerging clinical condition characterized by a pathological fixation with healthy eating. Recently, the Orthorexia Nervosa Inventory (ONI) has emerged as a promising tool for assessing orthorexic tendencies and behaviours, aiming to overcome the well-established limitations of existing measures for ON. The present study aimed to examine the psychometric properties of the Italian version of the ONI.

Methods

A total of 879 participants (Mage = 33.22 years, SD = 9.19; 56.9% females) completed the ONI along with the Düsseldorf Orthorexia Scale (DOS), measures of disordered eating, obsessive-compulsive symptoms (OCD), and psychosocial impairment. To establish the factorial validity of the ONI, a competing measurement modeling approach was employed by comparing standard confirmatory factor analytic (CFA) with exploratory structural equation modeling (ESEM) solutions. Model-based omega coefficients were computed to examine the internal consistency of the scale. Factorial invariance tests across gender were conducted within a multi-group framework.

Results

A three-factor first-order ESEM solution provided the best and most parsimonious representation of the data: χ2(207) = 558.641, p < .001, RMSEA = 0.044 (90% CI 0.040–0.048), CFI = 0.976, TLI = 0.968, SRMR = 0.036. The three latent dimensions, labelled behaviors, emotions, and impairments, showed excellent internal consistency (ω > 0.88). Furthermore, ONI scores were found to be positively correlated with DOS scores, disordered eating, OCD symptoms, and psychosocial impairment, supporting its convergent and criterion validity. Eventually, the ONI was factorially invariant across gender.

Conclusions

Overall, the present study provides evidence for the satisfactory psychometric properties of the ONI in the Italian context, endorsing its use in both clinical and research settings.

Plain English summary

Orthorexia Nervosa (ON) is a clinical condition characterized by a pathological fixation with healthy eating. The Orthorexia Nervosa Inventory (ONI) is a recently developed instrument to assess ON behaviours and tendencies. However, the scale has not yet been adapted for the Italian cultural context. To fill this gap, the present investigation aimed to translate the ONI into Italian and examine its psychometric properties on a community sample. The study involved 879 participants (Mage = 33.22 years, SD = 9.19; 56.9% females) who completed a series of self-report questionnaires. Findings revealed that the ONI items reflect  three latent dimensions: behaviours and preoccupation with healthy eating; physical and psychosocial impairments; and emotional distress. Importantly, the present study accrued substantial evidence for the internal consistency and criterion validity of the scale, the latter supported by significant correlations with measures of disordered eating, obsessive-compulsive symptoms, and psychosocial impairment. Overall, these findings endorse the use of the Italian version of the ONI for clinical and research practices.

Introduction

The neologism Orthorexia Nervosa (ON) derives from the Greek “ortho” (right, proper) and “orexia” (appetite) and was first introduced by physician Steven Bratman in 1997 to describe a pathological fixation with healthy eating observed among his patients [1]. Despite the growing research interest, a consensus definition of ON has not been established yet [2]. To address this knowledge gap, a multidisciplinary cohort of 47 eating disorders experts from 14 different countries recently conducted a consensus conference, through which they converged to define ON as a mental health disorder that closely relates to the DSM-5 category of “Feeding and Eating Disorders” (F&ED) [3]. Individuals with ON exhibit a strong preoccupation with the quality and nutritional composition of the food, which leads them to impose rigid and inflexible dietary rules and spend an excessive amount of time planning, obtaining, preparing, and eating their food [3, 4]. Compared to anorexia nervosa (AN), where the focus is on the quantity of food intake [5], ON is primarily focused on the quality of food and is sustained by a desire to achieve a feeling of purity and healthiness and by overvalued ideas regarding the health benefits of food [5, 6]. Critically, ON may result in several negative consequences, such as marked emotional distress in relation to food deemed as unhealthy, impaired educational, work, or social life domains, reduced quality of life, and nutritional deficiencies [3, 7].

Despite the growing body of research, the assessment of ON remains challenging due to the limitations and poor psychometric properties of currently available assessment tools [6]. Recently, Opitz and colleagues [8] reviewed all ON-related questionnaires and identified several weaknesses (see also [6]). Notably, the internal consistency of these questionnaires sometimes falls below acceptable levels, discouraging their use. For instance, the reliability of the most employed measure assessing ON, the ORTO-15 [9], has ranged from unacceptable 0.14 to good 0.84 internal consistency values across 24 studies [8]. Moreover, the dimensionality of these questionnaires often remains matter of debate, exhibiting inadequate fit when attempting to reproduce the original factorial structure and showing items to be not relevant to the ON latent dimension [8, 10]. Finally, as noted by Oberle and colleagues [10], there is a lack of specific and comprehensive items assessing emotional distress due to violations of orthorexic dietary rules and physical impairment due to nutritional deficiencies, thereby hindering evaluation. For instance, both the Düsseldorf Orthorexia Scale [11] and the Teruel Orthorexia Scale [12] lack a latent dimension specifically designed for measuring the impairment due to orthorexic symptoms. In light of these challenges, the Orthorexia Nervosa Inventory (ONI) [10] has emerged as a promising new questionnaire for assessing ON, integrating items from established measures and novel items to capture a more comprehensive range of features characterising the clinical condition. The ONI is a 24-item self-report questionnaire composed of three latent dimensions: behaviours and preoccupation with healthy eating; physical and psychosocial impairments; and emotional distress. The original validation study demonstrated strong levels of internal consistency (Cronbach’s alpha of 0.94 for total score and > 0.88 for subscales) and test-retest reliability (r > .86) [10]. Moreover, ONI scores showed significant correlations with disordered eating and obsessive-compulsive symptoms, supporting its concurrent validity [10].

Given the promising evidence regarding the reliability and validity of the ONI, the present study aimed to adapt the instrument for use in the Italian context and investigate its psychometric properties. Specifically, the first aim was to examine the factorial structure of the ONI by comparing standard confirmatory factor analytic models with exploratory structural equation modeling solutions. The second aim was to examine the internal consistency of the scale through model-based omega coefficients. The third aim was to test the convergent and criterion validity of the ONI by examining its relationships with alternative measures of ON, disordered eating, obsessive-compulsive symptoms, and psychosocial impairment. The fourth aim was to test the factorial invariance of the scale across gender.

Methods

Procedure

Data were collected via a cross-sectional online survey hosted by Qualtrics (https://www.qualtrics.com). Participants were recruited via word of mouth and advertising on the main social network platforms in February and March 2023. After signing informed consent, participants voluntarily completed the survey without any incentive or compensation. A comprehensive overview of the study was provided on the first page of the survey, and respondents were informed that they could withdraw from the survey at any time by refraining from submitting the form. The online survey lasted about 15 min. The study ensured complete anonymity, with no personally identifiable information collected. The inclusion criteria were being of legal age (18 years or older), being a native Italian speaker, and having given informed consent. Exclusion criteria included participants who were unable to complete the questionnaire due to visual or cognitive impairments. The study design was approved by the Institutional Review Board of the Department of Psychology at Sapienza University of Rome (prot. 0000520; 18/03/2022).

Participants

The sample size calculation was performed based on three criteria. Firstly, we conducted an RMSEA-based power calculation for the test of close fit on the less parsimonious factorial structure [13]. For a power of 0.80, fixing the critical alpha to 0.05, the minimum sample size needed was 88 (df = 186). Secondly, to detect a practically significant effect size in correlation analyses (ρ = 0.2, [14]), the minimum sample size needed was 193 (power = 0.80; alpha = 0.05, two-tailed test). Thirdly, at least 5 observations per free parameter and 10 observations per each item of the scale were guaranteed according to common rule of thumbs [15, 16].

A total of 879 participants voluntarily completed the survey and were included in the present investigation. The sample comprised 56.9% women, 36.2% men, 4.9% individuals identifying as other or non-binary, and 2% who preferred not to disclose their gender. The mean age of the sample was 33.22 years (SD = 9.19), ranging from 18 to 68. In terms of marital status, 46% of participants were in a relationship, 31.5% were single, 20% were married, 2% were divorced, and 0.5% were widowed. Regarding education, 34% of participants held a master’s degree, 30% held a high-school diploma, 21.7% held a bachelor’s degree, 12.5% held a postgraduate degree, 1.6% held a middle-school diploma, and 0.1% held an elementary school diploma. The mean body mass index (BMI), calculated through self-reported height and weight, was 25.10 kg/m2 (SD = 5.37).

Measures

Orthorexia nervosa inventory (ONI)

The Orthorexia Nervosa Inventory (ONI) [10] is a self-report questionnaire that consists of 24 items measuring three latent dimensions: behaviours and preoccupation with healthy eating; physical and psychosocial impairments; and emotional distress. Items are rated on a 4-point scale, ranging from 1 (not at all) to 4 (very). Subscale scores can be computed by summing the items pertaining to each dimension, as well as a total score by adding up each item score. Total scores range from 24 to 96, with higher scores indicating greater orthorexic symptomatology. Table 1 provides an overview of the ONI along with item-level descriptive statistics in the current sample.

Table 1 Item-level descriptive statistics
Full size table

Cross-cultural adaptation of the Orthorexia Nervosa Inventory (ONI)

After obtaining permission from the authors of the ONI to translate and culturally adapt the questionnaire, we adhered to established guidelines and conducted a forward and back-translation procedure [17]. Firstly, the ONI was independently translated from English into Italian by two individuals with fluency in both languages and expertise in the field of orthorexia. The translations were then compared with each other and with the original English version. Any discrepancies were resolved through consensus, and the two versions were synthesised to produce one common Italian translation. Secondly, a bilingual individual, who was blinded to the original version, back-translated the Italian version of the ONI into English, and the back-translation was further compared with the original version to identify and resolve any semantic inconsistencies. The process was then reviewed by the principal investigator, and a prefinal version was obtained. As a last step, to ensure the clarity, comprehensibility, and lack of ambiguity of the items, the Italian version was administered in a pilot study on a sample of 20 participants from the target population. No further adjustments were deemed necessary. The Italian version of the ONI is available in Additional file 1: Document S1.

Düsseldorf Orthorexia Scale (DOS)

In order to assess the convergent validity of the ONI, the Düsseldorf Orthorexia Scale (DOS) [11] was administered. The DOS is a 10-item self-report questionnaire designed to measure orthorexic behaviours and attitudes. Items are rated on a 4-point scale, ranging from 1 (this does not apply to me) to 4 (this applies to me). Total scores range from 10 to 40, with higher scores indicating more severe orthorexic symptoms. The DOS showed satisfactory psychometric properties in the Italian context [18]. Cronbach’s alpha in the present sample was 0.858.

Obsessive-Compulsive Inventory-Revised (OCI-R)

In order to evaluate the criterion validity of the ONI, the Obsessive-Compulsive Inventory-Revised (OCI-R) [19] was employed. The OCI-R is a widely used 18-item self-report questionnaire assessing the level of distress stemming from obsessive-compulsive symptomatology. Each item is rated on a 5-point scale, ranging from 0 (not at all) to 4 (extremely). A total score can be calculated by summing the scores for all items (0–72), with higher scores indicating higher severity of obsessive-compulsive symptoms. The OCI-R demonstrated satisfactory psychometric properties in the Italian context [20]. Cronbach’s alpha in the present sample was 0.895.

SCOFF questionnaire

The SCOFF questionnaire [21] was employed to address the criterion validity of the ONI. The SCOFF is a brief tool for assessing the core features of Anorexia Nervosa (AN) and Bulimia Nervosa (BN). The questionnaire consists of five dichotomous questions (i.e., 0 = no, 1 = yes), with a total score ranging from 0 to 5 that can be computed based on the sum of affirmative responses. Previous studies (e.g., [22, 23]) have shown that the SCOFF questionnaire is a sensible and specific screening tool for detecting eating disorders (ED). Moreover, the SCOFF showed satisfactory psychometric properties in the Italian context [24].

Clinical Impairment Assessment (CIA)

The Clinical Impairment Assessment (CIA version 3.0) [25] was administered to evaluate the criterion validity of the ONI. The CIA is a self-report measure consisting of 16 items designed to evaluate the psychosocial impairment due to eating disorders features over the preceding 28-day period. Each item is rated on a 4-point scale ranging from 0 (not at all) to 3 (a lot). By summing the scores of all items, a total CIA score can be calculated, with a range from 0 to 48. The higher the score, the higher the severity of the impairment. The CIA demonstrated satisfactory psychometric properties in the Italian context [26]. Cronbach’s alpha in the present sample was 0.947.

Data analytic strategy

Data were analysed using IBM SPSS v. 25 (IBM Corporation, Armonk NY; USA), FACTOR [27], and Mplus v. 8.6 [28].

To examine the dimensionality of the ONI, a three-factor solution was suggested for investigation based on the results of a parallel analysis [29]. Therefore, a preliminary exploratory factor analysis (EFA) was fitted to the inter-item polychoric correlations using the robust weighted least squares—means and variance adjusted (WLSMV) estimator coupled with geomin rotation. The pattern of factor loadings identified in this EFA was consistent with the original validation study conducted by Oberle and colleagues [10], with one exception. Specifically, Item #20, which was originally associated with the “Impairments” latent dimension, exhibited a main loading on the “Emotions” factor.

Accordingly, to establish the factorial validity of the ONI, in line with recent recommendations for the assessment of construct-relevant psychometric multidimensionality, a competing measurement modelling approach was employed by comparing standard confirmatory factor analytic (CFA) with exploratory structural equation modelling (ESEM) solutions (see [30,31,32]). This approach allows accounting for two relevant sources of multidimensionality [30]: (1) the restrictive and fallible assumption that items are pure indicators of the intended constructs, as they can exhibit non-negligible associations with conceptually related dimensions other than the ones they are meant to measure; (2) the hierarchically-organized nature of the constructs.

To this end, five different models were tested [30]:

  1. 1.

    A unidimensional CFA model positing an overall ON latent factor.

  2. 2.

    A first-order CFA model, in line with the original validation study conducted by Oberle and colleagues [10], positing three correlated latent factors: behaviours, impairments, and emotions. Each item was specified to load on the factor it was supposed to measure, with all cross-loadings fixed at zero.

  3. 3.

    A bifactor CFA model positing a global ON factor and three specific factors: behaviours, impairments, and emotions. All items were allowed to load on the global factor, and each specific factor was reflected by the corresponding items with all cross-loadings fixed at zero. The general and specific factors were specified as orthogonal consistently with standard bifactor assumptions [30].

  4. 4.

    A first-order ESEM model positing the aforementioned three latent factors using a target oblique rotation. Unlike the restrictive independent cluster model of CFA, where cross-loadings are fixed at zero, ESEM with target rotation allows for the estimation of cross-loadings (i.e., non-target loadings) while constraining them to be as close to zero as possible. Through the pre-specification of target and non-target loadings, ESEM provides a more flexible confirmatory approach maintaning an a-priori control on the hypothesised factor structure (e.g., [30, 33])

  5. 5.

    A bifactor ESEM model positing a global ON factor and the aforementioned three specific factors via an orthogonal target rotation. Contrary to the bifactor CFA, cross-loadings on the specific factors were allowed but targeted to be as close to zero as possible.

It should be noted that second-order CFA and ESEM models were not estimated to avoid redundancy, as they are mathematically equivalent to the corresponding three-factor first-order CFA and ESEM solutions [34]. Moreover, since the ONI response format includes four ordered categories and the indicators showed non-negligible deviations from the univariate normal distribution (skewness and kurtosis > |1|), items were treated as ordinal and the factor models were fitted to the inter-item polychoric correlations using the robust weighted least squares—means and variance adjusted (WLSMV) estimator [28].

Given the sensitivity of the chi-square test to large sample sizes and in accordance with a multifaceted approach to the assessment of model fit [35], the following goodness-of-fit indices were reported [36,37,38]: Root Mean Square Error of Approximation (RMSEA; values ≤ 0.08 indicating reasonable fit and ≤ 0.05 indicating close fit), Comparative Fit Index and Tucker-Lewis Index (CFI and TLI, respectively; values ≥ 0.90 indicating acceptable fit and ≥ 0.95 indicating good fit), and Standardized Root Mean Squared Residual (SRMR; with values of ≤ 0.08 indicating good fit). To determine the best-fitting model, the competing CFA and ESEM models were compared by examining differences in goodness-of-fit indices (e.g., [30,31,32]). Specifically, differences in CFI and TLI of less than 0.010, in conjunction with differences in RMSEA of less than 0.015, suggest that the more parsimonious model should be preferred [32, 38]. Comparison of goodness-of-fit indices was integrated with a detailed examination of parameter estimates (e.g., [39]), such as target and non-target loadings, and factor correlations. With respect to the size of standardized factor loadings, a value of at least 0.32 (i.e., 10% of overlapping variance) was assumed as a reasonable threshold for interpretative purposes [40].

Internal consistency was assessed through model-based omega coefficients: ω = (Σ | λi |)2 / ([Σ|λi|]2 + Σδii), where λi denotes the factor loadings and δii denotes the uniquenesses [39, 41]. Compared with the traditional alpha coefficient, which relies on the strict assumption of essential tau-equivalence, omega takes into account the strength of association between the items and the latent factors and is suitable under the more realistic congeneric model (see [42]). Moreover, since the ONI was intended to yield a total score in addition to subscale scores, the global reliability index for multidimensional scales was calculated [43]. According to common recommendations, internal consistency values greater than 0.70 are considered acceptable in non-exploratory research [e.g., 44].

Furthermore, a zero-order correlation was calculated with DOS scores to assess the convergent validity of the ONI, while criterion validity was evaluated through zero-order correlations with SCOFF, OCI-R and CIA scores, and BMI. Following Cohen’s [45] benchmarks, correlation coefficients of 0.1 to 0.3 were considered small, 0.3 to 0.5 were considered moderate, and greater than 0.5 were considered large.

Finally, factorial invariance tests across gender were conducted within the stepwise framework proposed by Meredith [46] and adapted for ordered-categorical indicators [47, 48]. Namely, we specified a series of nested models through a multi-group approach: (1) configural invariance (i.e., the same number of factors and same pattern of loadings); (2) metric invariance (i.e., equivalence of factor loadings); (3) scalar invariance (i.e., equivalence of factor loadings and items’ thresholds). To compare these nested models and evaluate the feasibility of the invariance constraints, since χ2 was particularly sensitive to large sample sizes, differences in CFI (ΔCFI), TLI (ΔTLI) and RMSEA (ΔRMSEA) were calculated. Specifically, deteriorations greater than 0.010 in CFI and TLI, in conjunction with RMSEA changes higher than 0.015, were considered as a signal for lack of invariance [49, 50].

Results

Factor structure

Table 2 presents an overview of the goodness-of-fit indices for the five competing models. Findings revealed that the first-order ESEM provided the best and most parsimonious representation of the data. Although the bifactor ESEM exhibited a slightly better fit, the difference with the first-order ESEM was negligible and the latter was preferred for the sake of parsimony (ΔCFI = 0.005; ΔTLI = 0.004; ΔRMSEA = 0.003). However, as noted by Sánchez-Oliva and colleagues [51] (see also [39]), relying solely on the comparison of goodness-of-fit indices is not sufficient for model selection, and the choice should be complemented with a detailed examination of parameter estimates. By focusing on models’ parameters, two of the three specific factors in the bifactor ESEM were found to be poorly defined, with several target loadings falling below the thresholds for interpretability (i.e., at least 10% of overlapping variance; [40]). Moreover, the comparison of the first-order ESEM with the first-order CFA model revealed that the former exhibited markedly lower inter-factor correlations (r = .474 to 0.641) compared to the latter (r = .668 to 0.800), thus providing a better differentiation between conceptually related latent dimensions [30]. This was consistent with the presence of some non-target cross-loadings, further reinforcing the superiority of the ESEM approach, which acknowledges that items may not function as pure indicators of their respective latent dimensions, as assumed by the restrictive independent cluster model of CFA [30]. Under these conditions, ESEM can be especially advantageous in revealing sources of misfit that would otherwise remain concealed in traditional CFA (e.g., [52]).

Table 2 Goodness-of-fit statistics for the five competing models
Full size table

The retained first-order ESEM showed a great fit to the data: χ2(207) = 558.641, p < .001; RMSEA = 0.044 (90% CI 0.040–0.048), p(RMSEA < 0.05) = 0.988; CFI = 0.976, TLI = 0.968, SRMR = 0.036. Parameter estimates are reported in Table 3. Consistent with the hypothesised theoretical dimensions, all factors demonstrated substantial and significant target loadings, indicating that much of the variance was accounted for by the common factors (λ = 0.32–0.92; M = 0.65; ps < 0.001). Most of the cross-loadings were lower than 0.30, and the ratio between the primary loading and the cross-loading was higher than 2. The only exception was found for items #7 and #23, which displayed moderate cross-loadings. The occurrence and magnitude of the cross-loadings are in line with the findings from the original validation study of the scale [10] and are also consistent with theoretical reasons discussed in the following section. Moreover, the intercorrelations between the latent factors were found to be moderate-to-large in magnitude. Specifically, the correlation between behaviours and impairments was r = .613, p < .001, the correlation between behaviours and emotions was r = .471, p < .001, and the correlation between emotions and impairments was r = .641, p < .001.

Table 3 Standardized parameter estimates from the first-order ESEM model
Full size table

Eventually, the unidimensional CFA model revealed a poor fit to the data, χ2(252) = 2174.447, p < .001; RMSEA = 0.093 (90% CI 0.090–0.097), p(RMSEA < 0.05) < 0.001; CFI = 0.867, TLI = 0.855, SRMR = 0.087, indicating the absence of substantial common-method variance and supporting the discriminant validity of the posited factors.

Reliability and validity

Omega coefficients were excellent for all three latent dimensions: Behaviors (ω = 0.885); Impairments (ω = 0.884); Emotions (ω = 0.913). Moreover, a satisfactory global reliability index for multidimensional scales of 0.948 [43] provided psychometric justification for employing a total score to measure ON.

The pattern of zero-order correlations between ONI and the other theoretically related constructs supported the convergent and criterion validity of the scale (see Table 4). Specifically, ONI scores were significantly correlated with DOS scores (r range = 0.667 to 0.876, ps < 0.001), CIA scores (r range = 0.341 to 0.757, ps < 0.001), SCOFF scores (r range = 0.313 to 0.584, ps < 0.001) and OCI-R scores (r range = 0.256 to 0.423, ps < 0.001), with most of these associations showing moderate-to-large magnitudes. Regarding BMI, non-significant associations were found, except for the small correlation observed with the ONI Emotions subscale (r = .180, p < .001).

Table 4 Descriptive statistics and zero-order correlations for the ONI dimensions
Full size table

Factorial invariance tests

Factorial invariance tests across gender (males vs. females) were conducted on the three-factor first-order ESEM solution. Participants who identified as other/non-binary could not be included due to the limited sample size. The goodness-of-fit indices were excellent at each step of the analysis. Specifically, the model fitted well when simultaneously tested on both males and females (i.e., configural invariance): χ2(414) = 704.796, p < .001; RMSEA = 0.041 (90% CI 0.036–0.047), p(RMSEA < 0.05) = 0.997; CFI = 0.979, TLI = 0.972, SRMR = 0.045. Furthermore, when equality constraints were imposed on factor loadings (i.e., metric invariance) and items’ thresholds (i.e., scalar invariance), there was an improvement in CFI, TLI, and RMSEA. The three levels of gender invariance were thus established, as summarised in Table 5. After demonstrating the adequacy of the scalar invariance model, latent mean differences were examined. The latent means of the female group were freely estimated and scaled on the latent means of the male group. Scores on Behaviours (SMD = 0.059, p = .631) and Impairments (SMD = 0.047, p = .569) were not significantly different across gender. Nonetheless, the latent mean for Emotions was significantly higher in the female group (SMD = 0.414, p < .001).

Table 5 Tests of measurement invariance of the ONI across gender
Full size table

Discussion

The purpose of the present study was to explore the psychometric properties of the ONI in an Italian community sample, by offering valuable insights into its factorial structure, internal consistency, and convergent and criterion validity.

As a primary step, we investigated the factorial validity of the ONI by employing a competing measurement modelling approach, comparing standard confirmatory factor analytic (CFA) with exploratory structural equation modelling (ESEM) solutions. This allowed us to account for two important sources of psychometric multidimensionality, as highlighted by Morin and colleagues [30]. Firstly, the assumption underlying the traditional independent-clusters-CFA (ICM-CFA) that items serve as pure indicators of the intended constructs is questionable, as they may represent non-pure indicators that exhibit remarkable associations with conceptually related dimensions other than the ones they are primarily assumed to measure [30]. Under this view, ESEM allows for the estimation of non-target cross-loadings while maintaining a substantial confirmatory framework [33]. Secondly, when evaluating the latent structure of a multidimensional scale, the observed indicators might reflect hierarchically-organized constructs [30]. To address this complexity, a comparison of alternative models which includes second-order or bi-factor solutions is warranted in order to accurately reflect the true population model.

Findings revealed that the first-order three-factor ESEM solution offered a satisfactory and parsimonious representation of the underlying structure of the ONI. Moreover, the ESEM solution was factorially invariant across gender, confirming that males and females used a similar frame of reference and the same metric in responding to the ONI items. The three latent factors and the pattern of loadings were almost consistent with those found by Oberle and colleagues [10]. The first factor, named “behaviours”, measures behaviours and preoccupation related to healthy eating. The second factor, named “emotions”, measures emotional distress resulting from violations of imposed dietary rules. The third factor, named “impairments”, measures physical and psychosocial impairments caused by nutritional deficiencies and dietary restrictions. Importantly, all standardized factor loadings were equal to or higher than 0.32, suggesting that a substantial amount of variance in the observed indicators was explained by the latent dimensions [40]. The majority of cross-loadings were observed to be below the threshold of 0.30, and the primary loading-to-cross-loading ratio exceeded 2. However, items #7 and #23 exhibited a cross-loading higher than 0.30 representing an exception to the overall pattern. This may be explained by theoretical reasons. First, item #7 (“My healthy eating is a significant source of stress in my relationships”) demonstrated noteworthy loadings on both the “emotions” and “impairments” factors. This finding is consistent with the notion that stress, which may be defined as a negative emotional experience [53], can elicit a range of physiological, behavioural, and emotional responses [54] that may indeed result in both emotional distress and physical impairments. Second, item #23 (“Feeling good about my body is completely dependent on me strictly following my healthy diet”) showed meaningful loadings on both the “behaviours” and “emotions” factors. This may be attributed to the fact that the item formulation contains both a behavioural aspect, with the statement “strictly following my healthy diet”, and a focus on body dissatisfaction, which has been associated with orthorexic tendencies and experiencing emotional distress [55, 56].

The excellent omega coefficients obtained for the three dimensions of the ONI indicate a high degree of consistency among the items measuring each construct. This finding is further supported by the satisfactory global reliability index for multidimensional scales, which justifies the use of a total composite score to measure ON. These results are in line with the original validation study of the ONI [10], which also demonstrated strong internal consistency with Cronbach’s alpha values of 0.94 and greater than 0.88 for the total score and subscales, respectively. Nevertheless, compared to traditional Cronbach’s alpha, the use of model-based composite reliability coefficients provides several advantages. That is, alpha coefficients may lead to inconsistent estimates of the population internal consistency when the scales do not adhere to tau equivalence, relying on the stringent assumption that all indicators have equal true-score variance [57]. In contrast, composite reliability coefficients are calculated from factor-analytic parameters and are therefore unbiased with varying factor loadings. Furthermore, while Cronbach’s alpha assumes that items reflect a unidimensional structure, composite reliability coefficients can be extended to assess the reliability of multidimensional scales [42]. This flexibility is particularly advantageous as it allows for a more comprehensive assessment of the psychometric properties of a measure, accounting for the multidimensionality of the construct of interest. Consequently, several researchers have advocated for the abandonment of alpha as a reliability measure and for the adoption of alternative model-based estimates (e.g., [57, 58]).

Importantly, the ONI showed very large correlations with DOS scores (r > .667), providing reasonable evidence of convergent validity, i.e., the degree to which two different questionnaires developed to assess a common construct demonstrate substantial and significant correlations [59]. Moreover, the present study offers compelling evidence for the criterion validity of the scale, as demonstrated by the moderate to large zero-order correlations between ONI and measures of disordered eating, obsessive-compulsive symptoms, and impairment secondary to eating disorders features. Consistent with previous meta-analytic findings [60], the associations found between ON and disordered eating may suggest a partial overlap between ON and currently recognized F&ED. As Cena and colleagues [2] have suggested, ON shares core features with the broad field of F&ED, such as the prominent role of food in an individual’s life, heightened concerns about food, dietary restriction, and social and health consequences (e.g., social isolation and malnutrition). Similarly, zero-order correlations found with obsessive-compulsive symptoms are consistent with previous meta-analytic results [60]. These associations may reflect shared features between ON and such symptomatology, including the presence of obsessions and compulsions related to healthy food, as well as ritualistic behaviour related to meal purchase, preparation, and consumption [2]. However, the smaller magnitude of the correlation compared to that with eating disorder symptoms suggests that ON might be conceptualized as a distinct F&ED, rather than belonging to the OCD spectrum, as previously hypothesised [3, 60, 61]. Lastly, non-significant or small correlations were found with BMI, aligning with the majority of previous research in Italian [18], Hungarian [62], Turkish [63], Spanish [12], Greek [64], and Australian [65] samples, which suggested a lack of significant association between ON tendencies and BMI.

Several limitations of the present study should be acknowledged. Firstly, test-retest reliability was not assessed, which precluded us to draw conclusions about the stability of ONI scores over time. Secondly, the sampling method relied on snowball and social media advertising, which may have led to a non-representative sample of the general population. Thirdly, the questionnaires were administered using an online survey platform. Future studies are warranted to investigate the psychometric properties of the ONI using a paper-and-pencil administration. Fourthly, we enrolled a non-clinical sample. Future investigations are needed to administer the ONI in clinical populations with the aim of identifying an optimal cutoff score using receiver operating characteristic (ROC) analysis for discriminating between profiles at risk and not at risk of ON.

Conclusions

In conclusion, the present study shed light on the good psychometric properties of the ONI in the Italian context, endorsing its use in both clinical and research practices. As highlighted by a recent consensus conference involving a multidisciplinary cohort of 47 ON experts [3], the development of empirically supported assessment tools is imperative to investigate the prevalence of the condition, enhance identification and screening efforts, refine treatment protocols, and ensure appropriate care for affected individuals. In this view, ONI can support clinicians in the identification and treatment of individuals experiencing such conditions. Moreover, the application of the ONI in research settings can significantly contribute to the ongoing efforts to conceptualize this emerging clinical condition, particularly concerning its potential inclusion in the nosographic classifications.

Data Availability

The datasets used and/or analysed during the current study are available from the corresponding author upon reasonable request.

Change history

References

  1. Bratman S, Knight D. Health food junkies: overcoming the obsession with healthful eating. 1st ed. New York: Broadway Books; 2000.

    Google Scholar 

  2. Cena H, Barthels F, Cuzzolaro M, Bratman S, Brytek-Matera A, Dunn T, et al. Definition and diagnostic criteria for orthorexia nervosa: a narrative review of the literature. Eat Weight Disord. 2019;24:209–46.

    Article  PubMed  Google Scholar 

  3. Donini LM, Barrada JR, Barthels F, Dunn TM, Babeau C, Brytek-Matera A, et al. A consensus document on definition and diagnostic criteria for orthorexia nervosa. Eat Weight Disord. 2022;27:3695–711.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Moroze RM, Dunn TM, Craig Holland J, Yager J, Weintraub P. Microthinking about Micronutrients: a case of Transition from Obsessions about healthy eating to Near-Fatal “Orthorexia Nervosa” and proposed diagnostic criteria. Psychosomatics. 2015;56:397–403.

    Article  PubMed  Google Scholar 

  5. Brytek-Matera A. Orthorexia nervosa - an eating disorder, obsessive- -compulsive disorder or disturbed eating habit? Arch Psychiatry Psychother. 2015;27:55–60.

    Google Scholar 

  6. McComb SE, Mills JS. Orthorexia nervosa: a review of psychosocial risk factors. Appetite. 2019;140:50–75.

    Article  PubMed  Google Scholar 

  7. Dunn TM, Bratman S. On orthorexia nervosa: a review of the literature and proposed diagnostic criteria. Eat Behav. 2016;21:11–7.

    Article  PubMed  Google Scholar 

  8. Opitz M-C, Newman E, Alvarado Vázquez Mellado AS, Robertson MDA, Sharpe H. The psychometric properties of Orthorexia Nervosa assessment scales: a systematic review and reliability generalization. Appetite. 2020;155:104797.

    Article  PubMed  Google Scholar 

  9. Donini LM, Marsili D, Graziani MP, Imbriale M, Cannella C. Orthorexia nervosa: a preliminary study with a proposal for diagnosis and an attempt to measure the dimension of the phenomenon. Eat Weight Disord. 2004;9:151–7.

    Article  PubMed  Google Scholar 

  10. Oberle CD, De Nadai AS, Madrid AL. Orthorexia Nervosa Inventory (ONI): development and validation of a new measure of orthorexic symptomatology. Eat Weight Disord. 2021;26:609–22.

    Article  PubMed  Google Scholar 

  11. Barthels F, Meyer F, Pietrowsky R. Die Düsseldorfer Orthorexie Skala–Konstruktion und evaluation eines Fragebogens zur Erfassung ortho-rektischen Ernährungsverhaltens. Z für Klinische Psychologie und Psychother. 2015;44:97–105.

    Article  Google Scholar 

  12. Barrada JR, Roncero M. Bidimensional structure of the Orthorexia: Development and initial validation of a New Instrument. AN PSICOL-SPAIN. 2018;34:283.

    Article  Google Scholar 

  13. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychol Methods. 1996;1:130–49.

    Article  Google Scholar 

  14. Ferguson CJ. An effect size primer: a guide for clinicians and researchers. Prof Psychology: Res Pract. 2009;40:532–8.

    Article  Google Scholar 

  15. Bentler PM, Chou C-P. Practical issues in Structural modeling. Sociol Methods Res. 1987;16:78–117.

    Article  Google Scholar 

  16. Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1967.

    Google Scholar 

  17. Kuliś D, Arnott M, Greimel ER, Bottomley A, Koller M. Trends in translation requests and arising issues regarding cultural adaptation. Expert Rev PharmacoEcon Outcomes Res. 2011;11:307–14.

    Article  PubMed  Google Scholar 

  18. Cerolini S, Vacca M, Zagaria A, Donini LM, Barbaranelli C, Lombardo C. Italian adaptation of the Düsseldorf Orthorexia Scale (I-DOS): psychometric properties and prevalence of orthorexia nervosa among an italian sample. Eat Weight Disord. 2022;27:1405–13.

    Article  PubMed  Google Scholar 

  19. Foa EB, Huppert JD, Leiberg S, Langner R, Kichic R, Hajcak G, et al. The obsessive-compulsive inventory: development and validation of a short version. Psychol Assess. 2002;14:485–96.

    Article  PubMed  Google Scholar 

  20. Marchetti I, Chiri L, Ghisi M, Sica C. Obsessive-compulsive inventory- revised (OCI-R): presentazione e indicazioni di utilizzo nel contesto italiano. Psicoterapia Cognitiva e Comportamentale. 2010;16:69–84.

    Google Scholar 

  21. Morgan JF, Reid F, Lacey JH. The SCOFF questionnaire: assessment of a new screening tool for eating disorders. BMJ. 1999;319:1467–8.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Garcia-Campayo J, Sanz-Carrillo C, Ibañez JA, Lou S, Solano V, Alda M. Validation of the spanish version of the SCOFF questionnaire for the screening of eating disorders in primary care. J Psychosom Res. 2005;59:51–5.

    Article  PubMed  Google Scholar 

  23. Garcia FD, Grigioni S, Allais E, Houy-Durand E, Thibaut F, Déchelotte P. Detection of eating disorders in patients: validity and reliability of the french version of the SCOFF questionnaire. Clin Nutr. 2011;30:178–81.

    Article  PubMed  Google Scholar 

  24. Pannocchia L, Fiorino M, Giannini M, Vanderlinden J. A psychometric exploration of an italian translation of the SCOFF Questionnaire: SCOFF Questionnaire Psychometric Exploration. Eur Eat Disorders Rev. 2011;19:371–3.

    Article  Google Scholar 

  25. Bohn K, Doll HA, Cooper Z, O’Connor M, Palmer RL, Fairburn CG. The measurement of impairment due to eating disorder psychopathology. Behav Res Ther. 2008;46:1105–10.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Calugi S, Sartirana M, Milanese C, El Ghoch M, Riolfi F, Dalle Grave R. The clinical impairment assessment questionnaire: validation in italian patients with eating disorders. Eat Weight Disord. 2018;23:685–94.

    Article  PubMed  Google Scholar 

  27. Ferrando PJ, Lorenzo-Seva U. Program FACTOR at 10: Origins, development and future directions. Psicothema. 2017;236–40.

  28. Muthén LK, Muthén BO. Mplus user’s guide, 8th edn. LosAngeles: Muthén & Muthén; 1998–2017.

  29. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179–85.

    Article  PubMed  Google Scholar 

  30. Morin AJS, Arens AK, Marsh HW. A bifactor exploratory structural equation modeling Framework for the identification of distinct sources of construct-relevant Psychometric Multidimensionality. Struct Equation Modeling: Multidisciplinary J. 2016;23:116–39.

    Article  Google Scholar 

  31. Barbaranelli C, Fida R, Paciello M, Tramontano C. Possunt, quia posse videntur’: they can because they think they can. Development and validation of the work self-efficacy scale: evidence from two studies. J Vocat Behav. 2018;106:249–69.

    Article  Google Scholar 

  32. Van Zyl LE, Ten Klooster PM. Exploratory structural equation modeling: practical guidelines and Tutorial with a Convenient Online Tool for Mplus. Front Psychiatry. 2022;12:795672.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Marsh HW, Morin AJS, Parker PD, Kaur G. Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu Rev Clin Psychol. 2014;10:85–110.

    Article  PubMed  Google Scholar 

  34. Hershberger SL, Marcoulides GA. The problem of equivalent structural models. In: Hancock GR, Mueller RO, editors. Structural equation modeling: a second course. 2nd ed. Charlotte, NC: Information Age; 2013. p. 13–42.

    Google Scholar 

  35. Tanaka JS. Multifaceted conceptions of fit in structural equation models. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. p. 10–39.

    Google Scholar 

  36. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. p. 136–62.

    Google Scholar 

  37. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Modeling: Multidisciplinary J. 1999;6:1–55.

    Article  Google Scholar 

  38. Wang J, Wang X. Structural equation modeling: applications using Mplus. 2nd ed. Hoboken: Wiley; 2020.

    Google Scholar 

  39. Morin AJS, Arens AK, Tran A, Caci H. Exploring sources of construct-relevant multidimensionality in psychiatric measurement: a tutorial and illustration using the Composite Scale of Morningness: construct-relevant Multidimensionality. Int J Methods Psychiatr Res. 2016;25:277–88.

    Article  PubMed  Google Scholar 

  40. Tabachnick BG, Fidell LS. Using multivariate statistics. 5th ed. Boston: Pearson/Allyn & Bacon; 2007.

    Google Scholar 

  41. McDonald RP. The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. Br J Math Stat Psychol. 1970;23:1–21.

    Article  Google Scholar 

  42. Barbaranelli C, Lee CS, Vellone E, Riegel B. Dimensionality and reliability of the self-care of heart failure Index Scales: further evidence from confirmatory factor analysis. Res Nurs Health. 2014;37:524–37.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Raykov T. Scale construction and development using structural equation modeling. In: Hoyle RH, editor. Handbook of structural equation modeling. New York: The Guilford Press; 2012. p. 472–92.

    Google Scholar 

  44. Hair JF. Multivariate data analysis. Eighth edition. Andover, Hampshire: Cengage; 2019.

    Google Scholar 

  45. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, N.J: L. Erlbaum Associates; 1988.

    Google Scholar 

  46. Meredith W. Measurement invariance, factor analysis and factorial invariance. Psychometrika. 1993;58:525–43.

    Article  Google Scholar 

  47. Guay F, Morin AJS, Litalien D, Valois P, Vallerand RJ. Application of exploratory structural equation modeling to evaluate the academic motivation scale. J Experimental Educ. 2015;83:51–82.

    Article  Google Scholar 

  48. Morin AJS, Moullec G, Maïano C, Layet L, Just J-L, Ninot G. Psychometric properties of the Center for epidemiologic Studies Depression Scale (CES-D) in french clinical and nonclinical adults. Revue d’Épidémiologie et de Santé Publique. 2011;59:327–40.

    Article  PubMed  Google Scholar 

  49. Cheung GW, Rensvold RB. Evaluating goodness-of-fit indexes for Testing Measurement Invariance. Struct Equation Modeling: Multidisciplinary J. 2002;9:233–55.

    Article  Google Scholar 

  50. Chen FF. Sensitivity of goodness of fit indexes to lack of Measurement Invariance. Struct Equation Modeling: Multidisciplinary J. 2007;14:464–504.

    Article  Google Scholar 

  51. Sánchez-Oliva D, Morin AJS, Teixeira PJ, Carraça EV, Palmeira AL, Silva MN. A bifactor exploratory structural equation modeling representation of the structure of the basic psychological needs at work scale. J Vocat Behav. 2017;98:173–87.

    Article  Google Scholar 

  52. Morin AJS, Marsh HW, Nagengast B. (2013) Exploratory structural equation modeling. In: Hancock GR, Mueller RO, editors. Structural equation modeling: A second course. 2nd ed. Charlotte: Information Age; 2013. p. 395–436.

    Google Scholar 

  53. Baum A. Stress, intrusive imagery, and chronic distress. Health Psychol. 1990;9:653–75.

    Article  PubMed  Google Scholar 

  54. Tomiyama AJ. Weight stigma is stressful. A review of evidence for the cyclic Obesity/Weight-Based stigma model. Appetite. 2014;82:8–15.

    Article  PubMed  Google Scholar 

  55. Barnes MA, Caltabiano ML. The interrelationship between orthorexia nervosa, perfectionism, body image and attachment style. Eat Weight Disord. 2017;22:177–84.

    Article  PubMed  Google Scholar 

  56. Thompson JK. Curatore. Exacting beauty: theory, assessment, and treatment of body image disturbance. 1st ed. Washington, DC: American Psychological Association; 1999.

    Book  Google Scholar 

  57. McNeish D. Thanks coefficient alpha, we’ll take it from here. Psychol Methods. 2018;23:412–33.

    Article  PubMed  Google Scholar 

  58. Yanyun Yang, Green SB. Coefficient alpha: a reliability coefficient for the 21st Century? J Psychoeducational Assess. 2011;29:377–92.

    Article  Google Scholar 

  59. Duckworth AL, Kern ML. A meta-analysis of the convergent validity of self-control measures. J Res Pers. 2011;45:259–68.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Zagaria A, Vacca M, Cerolini S, Ballesio A, Lombardo C. Associations between orthorexia, disordered eating, and obsessive–compulsive symptoms: a systematic review and meta-analysis. Intl J Eating Disorders. 2022;55:295–312.

    Article  Google Scholar 

  61. Ryman FVM, Cesuroglu T, Bood ZM, Syurina EV. Orthorexia Nervosa: disorder or not? Opinions of Dutch Health Professionals. Front Psychol. 2019;10:555.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Varga M, Thege BK, Dukay-Szabó S, Túry F, Van Furth EF. When eating healthy is not healthy: orthorexia nervosa and its measurement with the ORTO-15 in Hungary. BMC Psychiatry. 2014;14:59.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Sanlier N, Yassibas E, Bilici S, Sahin G, Celik B. Does the rise in eating disorders lead to increasing risk of orthorexia nervosa? Correlations with gender, education, and body mass index. Ecol Food Nutr. 2016;55:266–78.

    Article  PubMed  Google Scholar 

  64. Grammatikopoulou MG, Gkiouras K, Markaki A, Theodoridis X, Tsakiri V, Mavridis P, et al. Food addiction, orthorexia, and food-related stress among dietetics students. Eat Weight Disord. 2018;23:459–67.

    Article  PubMed  Google Scholar 

  65. Reynolds R. Is the prevalence of orthorexia nervosa in an australian university population 6.5%? Eat Weight Disord. 2018;23:453–8.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The study was partially funded by the Sapienza University of Rome, grant “Avvio alla Ricerca” (AZ Grant).

Author information

Authors and Affiliations

  1. Department of Psychology, Sapienza University of Rome, Rome, Italy

    Andrea Zagaria, Claudio Barbaranelli & Caterina Lombardo

  2. Department of Experimental Medicine, Sapienza University of Rome, Rome, Italy

    Edoardo Mocini

Authors
  1. Andrea Zagaria
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudio Barbaranelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edoardo Mocini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Caterina Lombardo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AZ projected the study; AZ and EM collected the data. AZ analyzed the data. AZ wrote the first draft of the manuscript with contributions from the other authors. All authors reviewed and commented on subsequent drafts of the manuscript. CB and CL supervised the entire process.

Corresponding author

Correspondence to Andrea Zagaria.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Institutional Review Board of the Department of Psychology at Sapienza University of Rome (prot. 0000520; 18/03/2022).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original version of this article was revised: An error in the supplement has been corrected.

Supplementary Information

Additional file 1.

The Italian version of the Orthorexia Nervosa Inventory.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zagaria, A., Barbaranelli, C., Mocini, E. et al. Cross-cultural adaptation and psychometric properties of the Italian version of the Orthorexia Nervosa Inventory (ONI). J Eat Disord 11, 144 (2023). https://doi.org/10.1186/s40337-023-00858-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40337-023-00858-0

Keywords

Journal of Eating Disorders

ISSN: 2050-2974