Psychometric properties of instruments assessing exercise in patients with eating disorders: a systematic review

Background Research has identified factors specific to exercise in eating disorder patients such as affect regulation and compulsivity. Existing measures of exercise behaviour which were not originally designed for eating disorder patients may not adequately assess these factors. The aim of this systematic review is to identify and assess the psychometric properties of all self-report measures of exercise designed to be used with eating disorder patients. Method A systematic review was conducted following the PRISMA guidelines. MedLine, Scopus and PsycINFO were systematically searched. A total of 12 studies examining two measures, the Exercise and Eating Disorders and the Compulsive Exercise Test, met inclusion criteria. Results Validation studies showed promising results for both tests and established internal consistency, concurrent and convergent validity, and construct validity. The factor structure of the Compulsive Exercise Test was not confirmed in the majority of the studies included in this review, while there are only two studies conducting factor analysis on the Exercise and Eating Disorders. Conclusion The two measures identified by this systematic review represent the current research on measures of compulsive exercise for eating disorder patients. Further research is needed to confirm a factor structure and validate both the Compulsive Exercise Test and the Exercise and Eating Disorders in more diverse clinical samples.

consensus on what constitutes problematic exercise behaviours, and no widely accepted clinical criteria [3,4]. A number of terms describing problematic exercise behaviour can be found both within and outside of an eating disorder context: 'obligatory exercise' [5,6], 'compulsive exercise' [4,7,8], 'excessive exercise' [9], 'exercise addiction' [10,11] or 'exercise dependence' [12,13]. While some of these terms refer to the quantitative (frequency, intensity, duration) component of exercise, others address the qualitative dimension (motivation, psychological experience) [4,9]. Whether these terms capture the same construct remains unclear, and while they may overlap, there are clear distinctions between the various definitions [4]. There is evidence that the term 'compulsive exercise' may best describe exercise behaviour typically exhibited by eating disorder patients [14], hence this term is used throughout this paper unless when citing studies whose authors have explicitly used different terms. Research has found both anecdotal and empirical evidence supporting the term 'compulsive exercise' [4]. For example, eating disorder patients have described their exercise behaviour as 'obsessive', 'driven' and 'out of control' [15] and have indicated that they are unable to stop the behaviour even if they want to [16]. There are many operational definitions to be found in the literature, however the components they include appear to mostly overlap with the American Psychiatric Association's conceptualisation of compulsivity, which is defined as 'repetitive behaviours that the person feels driven to perform' that are 'aimed at preventing or reducing distress' [17].
While a widely accepted working definition and maintenance model of compulsive exercise in eating disorders is still lacking, recent research has identified multiple factors playing a role in its development and maintenance. Exercise in eating disorders has long been thought to be a unidimensional construct with weight and shape concerns at its core [8,16,18], but more recent research has identified additional relevant factors such as compulsivity and affect regulation [19][20][21].
Multiple contemporary studies suggest that there is a link between high levels of exercise and compulsivity [2,14,19]. Compulsive exercise behaviour is characterized by an internal drive to exercise, rigid and inflexible exercise schedules, favouring exercise over other activities, and an inability to reduce or stop exercising despite possible negative outcomes [22][23][24]. Compulsive exercise has been shown to be associated with eating disorder features such as shape and weight concerns and drive for thinness [5,14,25]. Furthermore, high-level exercisers both with and without eating disorders score higher on measures of compulsivity than normal level exercisers [19,26].
Exercising to avoid negative affect has consistently been shown to be a contributing factor to the maintenance of eating disorders [27,28], and managing negative affect has been identified as one of the major reasons for continuing to exercise among eating disorder patients [29,30]. There is also considerable evidence that exercise deprivation can lead to withdrawal symptoms [31][32][33][34]. Negative affective states when unable to exercise can include guilt, depression, irritability, restlessness, and anxiety [32,34], and eating disorder patients may exercise to avoid experiencing negative emotions caused by exercise withdrawal [23].
Prevalence rates of increased physical activity in adults with eating disorders were found to range from 39 to 45.5% across eating disorder diagnoses and from 37 to 80% in restrictive type anorexia nervosa patients. Similar ranges can be found for other eating disorder subtypes [8,35]. Among adolescent eating disorder patients, prevalence rates vary and may be as high as 85.3% [27]. While some studies have found differences in prevalence depending on eating disorder subtype [35], other studies have found no significant differences between eating disorder diagnoses [36]. This contradiction may be due to studies using a variety of different definitions and measures of physical activity [36]. For instance, the studies on prevalence rates used 'compulsive exercise' [8,27], 'excessive exercise' [35] and 'high-level exercise [36].
Compulsive exercise has consistently been found to be related to elevated eating psychopathology [5,30], and in particular, to weight and shape concerns [36], dietary restraint [1], drive for thinness [5] and body dissatisfaction [7]. Exercise in eating disorders is also associated with a variety of negative outcomes such as longer hospitalisation [36], higher risk of and earlier relapse [37,38], higher risk of a chronic outcome [38], suicidality [39], and treatment drop-out [40]. Exercise can precede the onset of an eating disorder [16] and is often one of the last remaining symptoms [41].
The findings mentioned above clearly highlight the multidimensional nature of exercise in eating disorders. Given the high prevalence rates of compulsive exercise and associated negative outcomes, reliable psychometric instruments are a necessity when trying to identify and treat these behaviours [23]. However, data on compulsive exercise has been inconsistent due to the use of different instruments assessing exercise behaviours [14,22], which all rely on different underlying definitions of the construct. Some examples include the Obligatory Exercise Questionnaire [42], a 20-item questionnaire which measures subjective need to exercise repetitively and assesses exercise frequency and intensity, feelings related to exercise and preoccupation with exercise [5]. The Compulsive Exercise Test, a 24-item questionnaire, was developed for use with eating disorder patients and addresses domains identified by research to play a role in exercise behaviours of these patients: compulsivity, affect regulation and shape and weight concerns [23]. The Commitment to Exercise Scale [9], an 8-item questionnaire, measures psychological commitment to exercise and addresses three components of exercising: negative affect when unable to exercise, exercising despite being unwell, and the degree to which exercise interferes with social commitments. The Exercise Addiction Inventory [11], a 6-item questionnaire developed for use as a screening tool, was designed to measure the degree of addiction to exercise by assessing for different components of addiction, including salience, mood modification, tolerance, withdrawal, conflict and relapse [10], while the Exercise Dependence Scale [12], a 29-item questionnaire, views exercise dependence as similar to substance dependence and is therefore based on the DSM-IV criteria of substance abuse. While these measures have been used with eating disorder patients and have been shown to distinguish between patients and controls [5,22,43,44], it remains unclear whether they capture the idiosyncrasies specific to exercise in eating disorder populations [3], particularly in light of research identifying components such as affect regulation and compulsivity, which may not be reflected in the measures available. Hence this review was designed to identify measures specifically developed for assessing exercise in eating disorder patients as a subgroup of exercisers.
Focusing on quantitative rather than qualitative aspects when measuring exercise may be inadequate to capture the features specific to exercise in eating disorders [4], as quantitative aspects of exercise appear not to be related to eating psychopathology in both clinical and non-clinical samples [2,14,45]. These results indicate that frequency, intensity and duration of exercise may be less problematic than other motives such as a compulsive drive to exercise [2,4,14,46]. In addition, researchers have been unable to agree on a quantitative threshold for problematic exercise behaviours [4]. Suggestions have included exercising at least five times a week for at least 1 h without stopping [47], exercising for more than 3 h on any given day [35], or exercising for at least 6 h a week [19]. It therefore appears that endeavouring to define problematic exercise behaviours in terms of quantitative factors may result in flawed definitions, particularly as not all quantitatively high amounts of exercise are compulsive, and not all compulsive exercisers show high frequency, intensity and duration of exercise [4].
It is therefore unlikely that the measures mentioned above all relate to the same underlying construct, and, with the exception of the Compulsive Exercise Test, none of them were designed specifically for use in an eating disorder context. The variety of definitions and measures has made it difficult to compare results across studies, and to judge whether results are relevant to eating disorder patients as a specific subgroup of exercisers [3]. Available measures may therefore not adequately capture problematic exercise behaviour as an eating disorder symptom. There is a clear need for measures of compulsive exercise that take into account eating disorder pathology and issues common to exercisers with eating disorders [23]. While a recent systematic review has highlighted the importance of affect regulation and compulsivity when studying exercise in eating disorder patients [20], no review has so far reported on available measures of compulsive exercise. The current review therefore aims at identifying and assessing all self-report measures of exercise designed to be used within an eating disorder context.

Method
A systematic review was conducted following the PRIS MA guidelines [48]. The search strategy was designed to find all studies which used a self-report measure of compulsive exercise developed to be used within an eating disorder context and assessed its psychometric properties. The inclusion criteria for studies used in this review were: (1) published in a peer reviewed journal (2) published in English, German or French, (3) studies used human subjects who either did not meet criteria for a clinical diagnosis or met criteria for an eating disorder as defined by DSM-5 and ICD-11 (4) studies used a selfreport measure for compulsive exercise designed to be used within an eating disorder population and (5) studies assessed psychometric properties of these measures. No restrictions were based on demographic data such as sex, age, BMI, age of onset or duration of the eating disorder or whether participants had received treatment. No limitations were placed on publication year.
A search of published studies was conducted in July and August 2019 using the following electronic databases: PsycINFO (1806present), MedLine (1946present) and Scopus (1966present). Search terms were grouped into three categories (Group 1: Exercise addiction OR exercise dependence OR obligatory exercise OR compulsive exercise OR excessive exercise OR driven exercise OR maladaptive exercise OR pathological exercise OR obsessive exercise OR over-exercise OR exercise OR physical activity OR physical fitness OR running AND Group 2: psychometric OR self-report OR questionnaire OR measurement OR interview OR inventory OR assessment AND Group 3: eating disorders OR anorexi* OR bulimi* or binge eating OR EDNOS). Search terms were mapped to subject headings where possible. All subject headings were exploded. In addition, reference lists of studies selected for full-text screening were searched manually to identify any studies that were missed by the search strategy. The initial search yielded 2673 studies. Based on the title, 2401 studies were excluded, and 209 were excluded based on the abstract. A total of 63 articles were retained for full-text screening. After removing duplicates, a total of 35 articles remained. Twenty four articles did not meet the inclusion criteria, therefore a total of 12 articles were included in the review. For details see Fig. 1. A second reviewer screened a proportion of the titles and abstracts to minimise selection bias. Any disagreement was resolved by consensus.
Study quality was assessed using the Modified Quality Index [49]. Changes to the original version included removing items that were only relevant to intervention studies such as blinding or randomisation. As no intervention studies were included in this review, the Modified Quality Index was deemed appropriate to assess study quality. The Modified Quality Index includes 15 items which can be scored either 1 (Yes) or 0 (No/ Fig. 1 Flow diagram of article retrieval process unable to determine). Items 5 and 14 from the Modified Quality Index were removed, as they were deemed irrelevant to the studies incuded in this paper. Results of the study quality assessment can be found in Table 1. Studies were generally found to be of good quality with low risk of bias excepting power, with no study using power calculations to determine sampling adequacy. Only one study reported exact probability values. In order to minimise meta-bias, measures were also assessed on whether they were tested beyond the research group in which they were initially developed.

Study description
The 12 studies included in this review comprise the research conducted on psychometric properties of self report measures of compulsive exercise within an eating disorder context up until 2018. Three studies focused on the Exercise and Eating Disorders (EED) [3], eight studies focused on the Compulsive Exercise Test (CET) [23], and one study included the CET and other self report measures. Of the 12 included studies, three used a nonclinical sample, six used a clinical sample as well as a non-clinical control group, and three used a clinical sample only. Diagnoses in the clinical samples included anorexia nervosa, bulimia nervosa, eating disorder not otherwise specified, binge eating disorder and other specified feeding or eating disorder. Six studies used only female participants, six studies used male and female participants, and one study focused on male participants only. Of the studies focusing on the CET, four focused on particular populations: two focused on adolescents and the other two focused on athletes. For sample characteristics and distribution of diagnoses of the individual studies as well as for relevant numeric values see Table 2.

The Exercise and Eating Disorders (EED) questionnaire
The Exercise and Eating Disorders (EED) is a self-report questionnaire designed to assess compulsive exercise as a symptom of eating disorders [3]. Three studies were found that address its development and validation. A pilot study was conducted in 2012 [3], followed by a second study in 2015 [50], which aimed to further test psychometric properties and the factor structure of the EED. A third study followed in 2018 [51], which aimed to validate the EED in a male sample. All three studies were conducted within the same research group. The EED was developed in a clinical eating disorders inpatient unit. Items and subscales were created based on clinical experience and issues typically voiced by patients in this setting. Items were initially grouped into three categories based on clinical experience without conducting factor analysis: intentions to exercise (subscale 1), which assesses reasons for exercising, consequences of not exercising (subscale 2), which assesses negative outcomes of not exercising, and bodily sensations, which assesses whether subjects notice physical sensations such as feeling hungry, thirsty or tired, and whether they take these sensations into consideration (subscale 3) [3].

Structure and psychometric properties
The EED is an 18 item self-report questionnaire scored on a 6-point Likert scale from 0 (never) to 5 (always). Two items were excluded after the pilot study, and three items were added in the revised version assessing quantitative aspects of exercise (frequency, intensity and duration) [50].
The EED's factor structure, Internal consistency, testretest-reliability, concurrent, convergent and discriminant validity and its ability to distinguish between patients and controls were tested across the three studies. Factor analysis revealed a four factor structure: Compulsive Exercise (factor 1), Positive and Healthy Exercise (factor 2), Awareness of Bodily Signals (factor 3) and Weight and Shape related Exercise (factor 4) [50]. The four factor structure was retained in the subsequent study [51]. In relation to the initial version of the EED [3], the items of the 'consequences of not exercising' subscale overlap with the items of the Compulsive Exercise factor on the revised version of the EED. The 'intentions to exercise factor' thematically overlaps with two factors on the revised version of the EED, Positive and Healthy Exercise' and Weight and Shape Exercise. The 'bodily sensations' subscale has remained largely unchanged in the revised version [50]. For more detailed information about the factor structure, see Table 2.
Results from all three studies indicate excellent internal consistency for the whole scale. Internal consistency for the individual subscales ranged from acceptable to excellent except for the Compulsive Exercise subscale in the pilot study [3]. Test-retest reliability was calculated in one study. Between test and retest, Pearson's r was .86 for global score and ranged between .68 and .90 for subscales. No significant differences were found between test and retest in global score and subscales [50].. Concurrent validity was assessed by correlation analysis between the EED and the Body Attitudes Test (BAT). Results show significant correlations between EED and BAT Total as well as subscales [3]. Convergent and discriminant validity were established by comparing EED and Eating Disorder Examination Questionnaire (EDE-Q) scores [50]. The EED distinguished successfully between patients and controls in all three studies. For details see Table 2.     Global CET-A score of 10 successfully discriminated female athletes with an eating disorder from those without. This cutoff score represented suitable levels of sensitivity (0.92) and specificity n/a n/a

The Compulsive Exercise Test (CET)
The Compulsive Exercise Test (CET) is a 24 item selfreport questionnaire designed to assess exercise within an eating disorder context [23]. Nine studies were found addressing its development and validation, both within and outside of the research group in which it was initially developed. The CET's intial development was informed by the existing literature on exercise in eating disorders, interviews with eating disorder specialists and patients and a critical appraisal of existing measures [23]. The CET adopts a multidimensional cognitivebehavioral approach, addressing concepts such as emotion regulation and compulsivity that existing research has identified to play a role in the development and maintenance of compulsive exercise [23].
Seven of the nine studies used Cronbach's α to measure internal consistency [22,23,53,54,[56][57][58]. Values ranged between questionable and excellent for the overall scale and subscales. Concurrent validity was assessed in three studies by comparing the CET to other measures of exercise behaviour, namely the Commitment to Exercise Scale (CES) [23,53,58], the Obligatory Exercise Questionnaire (OEQ) [23], the Reasons for Exercise Inventory (REI) [58], and the Exercise Beliefs Questionnaire (EBQ) [58]. Results show significant correlations between the measures, confirming concurrent validity of the CET. Convergent validity was assessed in seven studies [23,[52][53][54][56][57][58] by comparing the CET to the Eating Disorder Inventory (EDI) and the Eating Disorder Examination Questionnaire (EDE-Q). Results show significant correlations between the measures, confirming convergent validity of the CET. The ability of the CET to distinguish between patients and controls was assessed in three studies [22,55,56]. All three found significant differences between patients and controls when considering the total score and most subscales. One study established a cut-off value of 10 to distinguish female athletes with an eating disorder from those without [55]. One study established a cut-off value of 15 [22] to distinguish between eating disorder patients with and without compulsive exercise. None of the studies on the CET included test-retest-reliability. For details see Table  2.

Discussion
The aim of this systematic review was to examine, summarise and assess the existing research on measures of compulsive exercise which were designed to be used with eating disorder populations. Twelve studies examining two measures, the Exercise and Eating Disorders (EED) and the Compulsive Exercise Test (CET), were found. A number of psychometric parameters were investigated in these studies including construct validity, internal reliability, concurrent and convergent validity and the ability of the measures to distinguish between clinical samples and control groups.

Strenghts
This systematic review has a number of strengths in line with the recommendations of the PRISMA checklist [48]. First, several databases were searched in order to minimise the probability of overlooking any studies that fit the criteria of this review. Second, publications in three languages were included in the search. Third, two reviewers of quality were involved in the study selection process.
The CET and the EED have also demonstrated a number of strengths throughout the studies examined in this review. The CET is the first measure of exercise designed specifically for use with eating disorder patients [23]. Given that exercise is such a prominent feature of eating disorders, a measure tailored to this population was long overdue. Furthermore, the CET is based on empirical findings from the eating disorder literature as well as on interviews with eating disorder specialists and patients and a critical appraisal of existing measures, rather than deriving its rationale from other areas such as addiction theory or substance dependence [23].. These findings indicate that weight and shape concerns [18], compulsivity [14,19], and emotion regulation [28] play a role when eating disorder patients exercise, which is clearly reflected in the items and factor structure of the CET. The studies examined in this review have established the CET's concurrent and convergent validity with other well-established tests such as the Commitment to Exercise Scale, Obligatory Exercise Questionnaire, Reasons for Exercise Inventory, Eating Disorder Inventory and Eating Disorder Examination Questionnaire, as well as the CET's internal consistency. This makes the CET a promising start on the journey to establishing more specialised measures of exercise for eating disorder patients.
The EED was also designed to be used with eating disorder patients but was developed from a more practical standpoint. While both the CET and the EED took patient and clinician experiences into account, the developers of the EED did not rely on recent research or other measures of exercise to inform item development. Rather, the EED was developed by clinicians working in an inpatient eating disorder unit using a practical, patient-centered approach [3].. Items were designed based on clinical experience and on issues regarding exercise voiced by patients in the unit. Initial psychometric testing confirmed the validity of this approach [3,50]. The EED shows good concurrent and convergent validity, and good to excellent internal reliability. The initial factor structure [50] was retained in a further study [51].

Limitations
This systematic review has some limitations. First, grey literature was not searched, hence there may be unpublished measures that fit the search criteria applied to this review. Additionally, despite the fact that three languages were included in the search, there may be papers in other languages that were missed by the search strategy.
There were a number of limitations in the studies reviewed. First, the number of studies examining selfreport measures of exercise designed to be used within an eating disorder population is small. Only 12 studies were found that met inclusion criteria. In addition, this number was not distributed evenly between the tests, with only three studies examining the EED. Results pertaining to the quality of the CET and EED should therefore be interpreted with caution. Second, sample sizes varied significantly in the included studies. Some studies had small sample sizes and did therefore not meet the recommended criteria of 10 participants per item [59] or more than 1000 participants [60] for factor analysis. Third, no power calculations were conducted in any of the studies to assess whether the number of particpants was adequate, and to avoid type II errors.
Results of the nine studies evaluating the CET in different samples such as eating disorder patients, adolescents and athletes indicate that the factor structure is somewhat unstable, and changes depending on the sample used. The initial five factor structure could not be confirmed in the majority of the studies examined in this review. The Mood Improvement subscale did not distinguish between patients and controls in three studies [22,23,54], potentially rendering its usefulness questionable. Despite the CET having been designed for eating disorder samples, only two of the nine studies aimed to validate the CET in an adult clinical sample. Further research and item modification may be needed to confirm a factor structure and validate the CET in more diverse clinical samples.
An issue common to both the CET and the EED is that clinical samples were mostly recruited from either inpatient facilities or other specialized eating disorder services, which puts participants at the more severe end of the eating disorder spectrum. Additionally, issues voiced by patients were taken into account when developing the EED, but may not necessarily apply to less severe presentations. It can therefore not be assumed that the tests examined in this review can be used for people with eating disorders who do not currently seek treatment, people with past eating disorders, or eating disorder patients treated in private practice.
A general limitation of the studies reviewed is that the CET and the EED have different underlying operationalisations of exercise, thereby adding to the plethora of already existing approaches and definitions. In the pilot study [3], the EED used exercising more than five times a week combined with the EED score as criteria to identify compulsive exercisers, while the CET does not include items assessing exercise frequency. While some items and subscales appear similar, there are others that do not seem to overlap. For example, both the CET and the EED include a weight and shape component, and both contain items pertaining to mood regulation. However, the EED subscale awareness of bodily signals does not appear in the CET, while the CET subscale exercise rigidity does not appear in the EED.
Additionally, while both tests appear promising, an indication of which measure is best used for different patient groups is currently lacking. The amount of research investigating the psychometric properties of the EED and CET is small. There are only three studies investigating the EED, and only two studies assessed the CET with eating disorder subjects. Given the limited amount of research into the factor structure of the EED, the inconsistent support for the factor structure of the CET, the lack of studies assessing test-retest-reliability, and the lack of studies validating both the CET and the EED in diverse eating disorder samples, it is difficult to make any recommendations for clinicians as to which measure might be best to use. Future research should aim to further validate both measures and identify which test works best for different patient groups so that such a recommendation can be made.

Further research and outlook
Despite the progress that has been made in conceptualising compulsive exercise and identifying relevant correlates such as compulsivity and affect regulation, a widely used and accepted operational definition is still missing [4]. This may make it difficult to compare results between studies, as studies tend to differ in their definitions of compulsive exercise as well as in behaviours described to operationalise the construct. It may therefore be beneficial to conduct more research on an empirically sound working definition of compulsive exercise.
While all measures identified in this systematic review were constructed using classical test theory, there are other measurement models that might be more suitable to measuring compulsive exercise such as item response theory and generalizability theory, which would allow for separation of systematic from unsystematic errors, and for both norm-referenced and criterion-referenced outcomes [61,62]. Future instruments may benefit from a different theoretical underpinning to explore alternative measurement models.
Additionally, while available measures define compulsive exercise as a state, it is possible to theorize that it is actually a state, or a state-trait-combination whereby the state component is subject to time or situation specific variation and is activated when an eating disorder is developed [63]. Future instruments may benefit assessing state as well as trait, allowing for a clearer conceptualisation of compulsive exercise, and adding to existing knowledge to inform assessment and treatment.
On a more practical note, recent research has indicated that carefully designed exercise interventions may be beneficial for eating disorder patients [64]. In order to be able to design and implement these interventions, it is necessary to gain understanding of the pathogenesis and maintenance factors of compulsive exercise in eating disorders. Psychometrically sound measures assessing compulsive exercise could help facilitate designing exercise interventions that are beneficial for patients' recovery. In particular, measures could potentially be used as guidelines for recommending exercise regimes to eating disorder patients.

Conclusion
Research has identified factors specific to exercise in eating disorders such as weight and shape concerns, affect regulation and compulsivity. Most available measures of exercise are not designed to be used specifically with eating disorder patients and therefore may not adequately measure exercise behaviours in this group. Tests specific to eating disorder patients are however essential in diagnosing and treating compulsive exercise behaviours. The CET and the EED are the only known measures of exercise designed to be used with eating disorder patients, providing researchers and clinicians with more tailored instruments. While initial validation studies showed promising results, more research is needed to further establish validity of these measures. Validating the CET and EED in more diverse eating disorder groups may help to establish them as routinely used clinical measures.