Assessing eating disorder symptoms in low and middle-income countries: a systematic review of psychometric studies of commonly used instruments

Background Various well-validated interview and self-report instruments are available to assess eating disorder symptomatology. However, most psychometric studies have been conducted in high-income countries. The aim of the present study was to systematically review the available psychometric studies conducted in low- and middle-income countries on well-known measures for assessing eating disorder symptoms. Methods Psychometric studies with the following instruments were included: the Eating Disorder Examination (EDE), the Eating Disorder Examination Questionnaire (EDE-Q), the Eating Disorder Inventory (EDI), the Eating Attitudes Test (EAT), and the Children’s Eating Attitudes Test (ChEAT). Searches were conducted on August 30, 2021, in the following databases: MEDLINE, EMBASE, LILACS, Web of Science, PsycINFO, and CABI. The methodological quality of the studies was assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). The studies were considered to have conducted the minimum psychometric evaluation if they assessed at least the three types of validity (content, criteria, and construct) or diagnostic performance. The psychometric properties were also evaluated considering the cut-off points described in the literature for each of the analysis methods used to evaluate validity and reliability and two reviewers independently selected the studies and evaluated the quality criteria. Results A total of 28 studies were included. The studies were conducted in 13 countries (10 middle income and 3 low income). The instruments that were most used in the studies were the EAT and EDE-Q. According to the overall COSMIN assessment, in most (57%) of the studies the psychometric properties assessed were not described. Forty-three percent of the studies conducted the minimum psychometric evaluation. However, according to the described cut-off points, the results for the psychometric properties assessed showed, in general, acceptable validity and reliability. Conclusion The results of this review suggest a lack of studies with the recommended psychometric properties in low- and middle-income countries on these commonly used instruments. With the steady increase in the prevalence of eating disorders globally, psychometric investigations of instruments for measuring eating disorder symptoms in these countries should be encouraged to promote their early detection and treatment. Supplementary Information The online version contains supplementary material available at 10.1186/s40337-022-00649-z.


Background
Eating disorders (EDs) are psychiatric disorders characterized as disturbances in eating or eating-related behavior that lead to impaired consumption or absorption of food, which can compromise the physical and psychological health of the individuals affected [1][2][3]. EDs, particularly anorexia nervosa (AN), can in some cases lead to early mortality [4].
In a systematic review from 2019, the lifetime prevalence of EDs was estimated at 8.4% in women and 2.2% in men [5]. Although research has shown that the highest burden of EDs remains in high-income countries, prevalence studies have indicated that EDs have increased in other countries [6], specifically in East and South Asia [4,7]. According to the results from the Global Burden of Disease Study 2017, low-and middle-income countries have shown an important increase in the agestandardized rate of ED prevalence, rising from 116.74 per 100,000 population in 1990 (95% UI: 92.25-548) to 156.96 in 2017 (95% UI: 123. 42-194.26) [8]. However, the number of studies on the prevalence in low-and middleincome countries is limited [4]. Studies that help in identifying the local prevalence of eating disorders, such as in low-and middle-income countries, are very important; they can guide public health policies and help in implementing optimal and effective measures for the respective population to reduce the burden of this disease [8,9].
The tools commonly used to evaluate EDs are the Eating Disorder Examination (EDE) [10], the Eating Disorder Examination Questionnaire (EDE-Q) [11], the Eating Disorder Inventory (EDI) [12], the Eating Attitudes Test (EAT) [13], the Children's Eating Attitudes Test (ChEAT) [14], the Children's Eating Disorder Examination (ChEDE) [15], and the Children's Eating Disorder Examination Questionnaire (ChEDE-Q) [16]. Most studies that examine the psychometric properties of these instruments have been conducted in high-income countries and there is a scarcity of studies that show whether these instruments have been used in low-and middle-income countries. The use, in low-and middle-income countries, of instruments whose psychometric properties have not been evaluated prevents us from knowing whether the results of their measurements correspond to the actual state of the phenomenon being measured [17,18] Therefore, the objective of the present study was to systematically review psychometric studies on instruments that assess ED symptomatology in low-and middle-income countries.

Selection process
Two authors (COA and CMGD) independently scanned the abstract and title of each study from the search results. Next, all potentially relevant articles were read in full. In both phases, wherever there was a difference in opinion, a third author (RM), who did not initially evaluate the articles, reviewed them to reach a final decision.

Data collection process and data items
Two authors (COA and CMGD) independently extracted the data. Any disagreements were presented to the third reviewer (RM) to establish a consensus. The following information was extracted: first author; year of publication; country; questionnaire language; eligibility criteria for participants; number of participants included (sample size); sex and age; instrument names; mode of administration of the questionnaire (self-reported or interview); number of items; domains assessed; psychometric properties; type of study. In the original protocol, we planned to contact the authors if we could not identify any of the information that was to be collected from the articles. However, all necessary information was presented in the included articles.

Assessment of risk of bias in included studies
The methodological quality of individual studies was assessed according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) [22]. The COSMIN checklist for assessing the methodological quality of individual studies contains ten sections, including translation process, content validity, hypotheses testing (or convergent and discriminant validity), structural validity (or construct validity), criterion validity (or diagnostic process), internal consistency, test-retest, measurement invariance, responsiveness, and, finally, the overall score for the COSMIN assessment. To classify the psychometric properties, a fivepoint scale with the following descriptors was used: "very good, " "adequate, " "doubtful, " "inadequate, " "not reported, " or "not applicable. " The overall score for the methodological quality of each measurement tool was determined considering the lowest classification for any one of the items evaluated. For example, for the structural validity criterion, if a confirmatory factor analysis (CFA) was conducted (implying a "very good" classification), but the sample size was < 5 times the number of items (meaning an "inadequate" classification), the general quality classification considered for that methodological property was "inadequate" [22].
We also considered that the studies conducted the minimum psychometric evaluation, according to international guidelines, if they assessed at least the three types of validity (content, criteria, and construct) or diagnostic performance was evaluated with a gold standard tool [23]. Psychometric studies that did not include a description of the translation process were considered as lacking content validity.

Evaluation of psychometric properties evaluated in articles
The psychometric properties evaluated in the articles were assessed considering the cut-off points described in the literature for each of the analysis methods used to evaluate validity and reliability (Additional file 2).
The methodological quality of individual studies was visualized using the robvis web app, which depicts the plots obtained from these analyses [24]. The methodological quality of individual studies was assessed independently by two review authors (CA and CS). When there were differences in opinion, the third author (RM), who did not initially evaluate the articles, reviewed them to decide.
We assessed the performance of the original studies that developed and validated the questionnaires according to the COSMIN tool, minimum psychometric evaluation, and cut-off points.
We also described the methodological quality of individual studies according to the COSMIN tool of most validated questionaries' and compared the COSMIN results for the validated questionnaires that underwent the translation process with those that did not undergo the translation process.

Study selection
The search identified 4745 articles in the databases used, and a further 18 articles were identified in the gray literature. A total of 1719 duplicate articles were removed, leaving 3044 articles for the title and abstract evaluation. In this phase, 2901 articles were excluded. A further 1699 articles were excluded because they did not validate the instruments, 1026 used instruments other than the ones that are the focus of our study, 89 were studies conducted in populations from high-income countries, and 87 were review studies. Thus, 143 articles remained to be read in full. A further 115 were excluded because they did not meet the inclusion criteria: 76 studies did not validate the instruments, 28 used other instruments, nine were duplicates, and two were reviews. Thus, a total of 28 studies were included in this review. The systematic study selection is described in Fig. 1.

Results of individual studies Eating disorder examination (EDE)
The EDE is a semi-structured interview. There are three versions. The first version (12th edition) consisted of 62 items [10], the second version (16th edition) has 41 items [53], and the newest one (17th edition) was created based on the latest Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) and has 40 items [54]. All versions contemplate four domains: restraint, eating concern, shape concern, and weight concern. The global EDE score represents the average for the four domains.
The 12th edition of the EDE was translated into Mandarin and the psychometric properties were evaluated by one study in China. Studies included in review (n = 28)

Included
Reports sought for retrieval (n = 0) Reports not retrieved (n = 0)  According to the COSMIN methodological quality classification of individual studies the translation process was considered "doubtful"; the hypotheses testing was considered "adequate"; the validity criterion was considered "adequate"; the internal consistency was considered "very good"; and the test-retest assessment was evaluated as "doubtful. " The content validity, structural validity, measurement invariance, and responsiveness were not assessed (see Fig. 2 and Additional file 3) [29]. The EDE instrument met the minimum psychometric criteria considered in this review, through the diagnostic performance evaluation.
The psychometric property results according to the described cut-off points were as follows: discriminant validity: the EDE showed a significant difference between individuals with and without EDs; diagnostic performance: the area under the curve (AUC) was not evaluated; internal consistency: the Cronbach's alpha coefficient was good; and test-retest: the Spearman correlation coefficient showed a strong correlation [29] (see Table 2).

Eating disorder examination-questionnaire (EDE-Q)
The EDE-Q is a self-report version of the EDE. There are three versions of it: the EDE-Q 4.0 with 38 items [11], the EDE-Q 6.0 with 28 items [55], and the short version EDE-QS with 12 items [56]. The EDE-Q 4.0 and EDE-Q 6.0 have four domains: restraint, eating concern, shape concern, and weight concern. The global EDE-Q score represents the average of the four domains. Finally, the EDE-QS has 12 items and is the only version that is one-dimensional.
The included studies used the three versions of the EDI: the EDI [37], the EDI-2 [38], and the EDI-3 [39,40]. A translated version of the EDI was used by two studies [38,40]. These two studies did not carry out the translation process but instead used the version of the questionnaire already available in Spanish [37,39,60,61]. The studies were conducted in Mexico (in Spanish) [37,38], Iran (in Persian) [40], and Argentina (in Spanish) [39]. The number of participants in the studies varied considerably, from 47 [38] to 725 [39]. Three studies included only female participants [37][38][39] and one study included female and male participants [40]. The mean age of the participants varied from 15 [39] to 22 [40] years old. The participant inclusion criteria varied according to the target population chosen in the studies. All participants were recruited through convenience sampling. Two studies included patients diagnosed with an eating disorder according to the DSM-IV criteria, who were recruited in hospitals [37,38]; one study recruited participants in a secondary school setting [39]; and one study recruited participants at a university [40]. The latter study excluded participants who did not answer three or more items in the questionnaires [40], and another study excluded students who presented difficulties with communication and in understanding the questionnaire [39].

Eating attitudes test (EAT)
The EAT is a self-report questionnaire and has two versions. The first version has 40 items [13] and the second and most recent version (EAT-26) has 26 items [62]. Both versions contemplate three domains: dieting, bulimia and food preoccupation, and oral control [13,62].

Children's eating attitudes test (ChEAT)
The ChEAT is a self-report and 26-item instrument that contemplates three domains: dieting attitudes, oral control, and social pressure to restrict eating [14]. The ChEAT-26 was translated into Portuguese and its psychometric properties were evaluated in one study in Brazil. The instrument was validated in a sample of 346 female and male participants, with a mean age of 10 years old. The participant inclusion criteria was pre-adolescents from eight to 12 years old recruited from four private schools.
According to the COSMIN methodological quality assessment, the translation process was considered "doubtful"; the internal consistency assessment was classified as "very good"; and the structural validity examination was considered "adequate" [52]. The content validity, hypotheses testing, criterion validity, test-retest, measurement invariance, and responsiveness were "not reported" (see Fig. 2 and Additional file 3). This study did not contemplate the minimum psychometric assessment (content, criterion, and construct validity) or diagnostic performance.
Forty-three percent of the studies conducted the minimum psychometric evaluation. According to the described cut-off points, the psychometric properties assessed showed overall acceptable validity and reliability results (see Table 2).
Performance of original studies that developed and validated the questionnaires according to the COSMIN tool, minimum psychometric evaluation, and cut-off points.
Overall score, (40%) of the studies did not describe ("not reported") the psychometric properties assessed in this tool [10-12, 14, 62] (see Additional Files 4 and 5). None of the studies conducted the minimum psychometric evaluation.

Description of individual studies of most validated questionnaires according to the COSMIN tool
Since the EAT and the EDE-Q were the instruments with the most evaluations, they were compared with their original versions to compare the impact of psychometric properties. Forty percent of the original studies did not describe the psychometric properties (Additional files 7 and 8).

Comparison between studies of validated questionnaires with translation process and those without this process
According to the overall COSMIN score, in both cases, more than 45% of the studies did not describe most of the psychometric properties (see Additional file 9).

Discussion
Most of the studies conducted in low-and middleincome countries on measures for assessing well-known eating disorder symptoms did not described psychometric properties according to the COSMIN methodological quality classification of the individual studies and they did not conduct the minimum recommended assessment of these properties. However, according to the described cut-off points, the psychometric properties evaluated showed overall acceptable validity and reliability results. In addition, most studies were conducted in middleincome countries.
The overall score for the methodological quality of each measurement according to the COSMIN procedure is determined considering the lowest classification of any one of the items evaluated. While a strength of the COS-MIN procedure is that it has very rigid and specific criteria for evaluating psychometric properties, the result is that there is little flexibility in the tool. For example, in the test-retest evaluation, if researchers choose to report Pearson or Spearman correlations instead of the kappa coefficient or ICC, this property could be classified as doubtful [22]. The assessment of psychometric properties involves a vast field and there is still a lack of standardization in conceptual and methodological terms [23]. According to the general COSMIN assessment, most of the original studies also did not describe the psychometric properties evaluated. In addition, most of these studies did not meet the minimum criteria recommended for the evaluation of psychometric properties. These results reinforce the need to properly assess psychometric properties in the development of ED assessment tools.
We also considered the minimum criteria recommended for assessing psychometric properties, and most studies did not include these criteria either. For example, of the 17 studies that carried out the translation process, only two assessed the content validity for the cultural adaptation of the instrument [23]. However, the studies with and without the translation process showed great similarity in the evaluation of the psychometric properties according to the COSMIN score. All the validated tools were developed in English in high-income countries. Understanding the possible cultural difference between countries is crucial for an adequate assessment of the instruments [67,68].
The psychometric properties were most frequently assessed in the EAT and EDE-Q instruments. A possible explanation for the greater use of these questionnaires may be that both instruments include important domains for the assessment of EDs, and they are already widely used and recognized in clinical practice [69][70][71]. As with original studies, the majority of the most used questionnaires did not describe the psychometric properties evaluated in COSMIN.
The EDE, one of the most frequently used instruments for measuring EDs, was only validated in China [29,69]. The ChEDE and ChEDE-Q instruments were not validated in any of the low-and middle-income countries.
Of the 28 studies included in this review, only five received partial funding to develop their research. This emphasizes the scarcity of resources for developing ED research in low-and middle-income countries. A lack of resources can significantly compromise the feasibility of conducting studies according to methodological recommendations [72].
Regarding the limitations of this review, no countryspecific databases were accessed. However, the search for evidence included six databases and gray literature and did not include any language or publication date restrictions. Another limitation is that this review included only the most commonly used instruments and did not cover all instruments. In addition, we did not compare the diagnostic performance of questionnaires in studies that evaluated psychometric properties. However, only six studies evaluated the diagnostic performance of three instruments.

Conclusion
The results of this review suggest a lack of studies in low-and middle-income countries on psychometric properties in commonly used instruments for measuring EDs. With the steady increase in the prevalence of EDs globally, psychometric investigations of instruments for measuring eating disorder symptoms in these countries should be encouraged to promote their early detection and treatment.