Allen Cognitive Level Screen

Inpatient Psychiatric Patients: (David & Riley, 1990; n = 71 patients admitted to general hospital psychiatric unit during a 13-month period; Mean Age = 36.99 (12.18); mood disorder, 54%; schizophrenia, 13%; adjustment disorder, 11%; anxiety disorder, 6%; impulse control disorder, 4%; somatoform disorder, 4%; psychoactive substance use disorder, 4%; organic mental disorder, 1%; various other diagnoses, 3%)

• Mean ACL score = 4.9 (0.85)

Schizophrenia: (Ozturk et al., 2022; n = 120 (60 with schizophrenia, mean age = 41.58 (10.66), male = 61.7% and 60 healthy subjects, mean age = 39.05 (9.58), male = 56.7%; Turkish translation of ACLS-5)

• Excellent test-retest reliability: (ICC = 0.93)

Predictive validity:

Schizophrenia: (Stewart et al., 2019; n = 150; mean age = 40.59 (11.05), age range = 18-64, male = 95 (63%); Australian sample)

• Poor predictive validity: Lower ACLS scores prior to discharge predicted faster representation at the hospital (rehospitalization) within a 12 month period (r = 0.17, p = .05)

Schizophrenia (Chan & Yeung, 2008; n = 201; Mean Age = 43.14 (9.9); Chinese sample)

• Poor predictive validity of the ACLS-2000 predicting community and social functioning assessed by the Chinese version of the Multnomah Community Ability Scale (r = 0.11).

Schizophrenia (Velligan et al., 1998; n = 110; Mean Age = 35.7 (9.7); subjects discharged from state hospital over 3 year period, follow-ups 1.5-3 years post-discharge)

• Excellent predictive validity of the ACLS-90 for global functioning 1 to 3.5 years later as assessed by the Social and Occupational Function Scale (r = 0.60)
• Adequate predictive validity of the ACLS-90 for level of social effectiveness, occupational function, and productive activity as assessed by the Multinomah Community Ability Scale, Levels of Functioning Scale, and Lehman work and Productive Activity Scale, respectively. (r = 0.40, 0.37, 0.46, in the sequence presented)

Schizophrenia (Secrest, et al., 2000; n = 33 adult men with schizophrenia or schizoaffective disorder; Mean Age = 47.94)

• Excellent predictive validity of ACLS with Routine Task Inventory (r = 0.67) predicting task performance
• Poor predictive validity of ACLS with the number of preservative errors on Wisconsin Card Sorting Test (WCST) (r = -0.47)
• Adequate predictive validity of ACLS with number of categories obtained on WCST (r = 0.57) predicting executive function, cognitive dysfunction, and community functioning in patients with chronic schizophrenia
• Poor predictive validity of ACLS with number of hours worked per week (r = 0.21)

Inpatient Psychiatric Patients: (Henry et al., 1998, n = 100 inpatients consecutively admitted to acute mental health unit of hospital during a 6-week period; Mean Age = 39.57 (14.75); schizophrenia, bipolar disorder n = 39; major depression, PTSD/dissociative disorder, anxiety disorder n = 61)

• Adequate predictive validity of the ACLS-90: patients with mean score of 4.92 discharged to independent living situation, patients mean score of 4.50 discharged to supported living situation (r = 0.34)

Concurrent validity:

Inpatient Psychiatric Patients: (Scanlan & Still, 2013; n = 225 individuals from eight inpatient psychiatric units; Mean Age = 39.6 (13.2); schizophrenia n = 154; schizoaffective disorder n = 20; other psychoses n =14; bipolar disorder/mania n = 12; depression n = 11; “other” n = 14)

• Adequate concurrent validity of the ACL with functional performance in personal care/basic ADLs, telephone use, travel/transport, shopping, cooking, washing/laundry, housework (r = 0.48, 0.35, 0.33, 0.44, 0.45, 0.41, 0.48 respectively)
• Poor concurrent validity of the ACL with functional performance in money management (r = 0.29) and medication management (r = 0.19)

Schizophrenia (Leung & Man, 2007; n = 61 chronic schizophrenic patients,  Mean Age = 45.07; n = 61 more than 6 months stay in existing psychiatric setting, Mean Age = 29.84; Chinese sample)

• Excellent concurrent validity of the Chinese version of ACLS (CACLS) and Chinese version of Mini Mental State Examination (CMMSE) total as well as CACLS and Chinese version of Functional Needs Assessment (CFNA) total (r = 0.609, 0.714 respectively)
• Adequate to excellent concurrent validity of CACLS and CFNA’s subsets (r = 0.456-0.737)

Schizophrenia (Velligan et al., 1998)

• Excellent concurrent validity of the ACLS-90 to the Functional Needs Assessment (FNA). (rho = ~0.56)
• Adequate concurrent validity was found between ACLS-90 and multiple neuropsychological tests, including Hopkins Verbal Learning, Digits Backward, Finger Tapping, Simple Reaction Time, Inhibition, Continuous Performance Test, and Hooper Visual Organization assessment. (rho = 0.30 to 0.62)
• Poor concurrent validity found between ACLS-90 and Digits Forward and Choice Reaction time. (rho = 0.28, 0.16, respectively)

Schizophrenia (Velligan et al., 1995; n = 110; Mean Age = 34.6 (7.5); subjects experienced consecutive admissions to inpatient hospital over 18 months)

• Excellent convergent validity demonstrated between ACLS-90 and FNA was demonstrated across the whole study, among participants identified as non-Hispanic whites, and participants identified as Mexican-Americans. (r = 0.66, 0.67, 0.60, respectively)
• Adequate convergent validity was found between FNA and ACLS-90 among study participants that identified as African-American. The authors noted that unlike the other groups represented, there were no occupational therapists or researchers in the study that were from their own identified ethnic group. (r = 0.46)

Schizophrenia (Keller & Hayes, 1997; n = 58; n = 41 living in community, n = 17 living in long-term psychiatric hospital; Mean Age = 35.50 (9.85))

• Adequate convergent validity of adaptive functioning was found between ACLS-90 and the Life Skills Profile (LSP) among total score and subtests for communication, nonturbulence, and self-care. (r = 0.54, 0.37, 0.40, 0.53, respectively)
• Poor convergent validity of adaptive functioning was found between ACLS-90 and the LSP subtests for responsibility and social contact. (r = 0.26, 0.28, respectively)

Discriminant validity:

Schizophrenia: (Ozturk et al., 2022)

• Significant ability of the ACLS-5 to discriminate between subjects with schizophrenia and healthy controls (z = 7.065, p < 0.001)

Schizophrenia (Leung & Man, 2007)

• Construct validity supported since the mean rank of the control group (87.75) was significantly higher than the schizophrenia group (33.8).

Schizophrenia (Su et al., 2011; n = 76; n = 35 scored 4 on ACLS, Mean Age = 37.83 (10.25); n = 41 scored 5 on ACLS, Mean Age=37.73 (9.28); Taiwanese Sample)

• Construct validity supported: the mean rank of participants that scored a 5 on ACLS compared to participants that scored a 4 on ACLS was significantly higher for processing speed, immediate verbal recall, delayed verbal recall, and working memory (p≤0.006)

Schizophrenia (Chan & Yeung, 2008)

• Construct validity supported: ACLS scores were significantly different by living situations (long-stay care homes, halfway houses, supported hostel/housing, living with family, and living alone). (p=0.000)

Convergent Validity:

Schizophrenia: (Ozturk et al., 2022; n = 120)

• Adequate correlations between ACLS-5 and Social Functioning Scale (SFS) Total and sub-parameters:
  • SFS Total (r = 0.426, p < 0.001)
  • Interpersonal behavior (r = 0.392, p = 0.00)
  • Recreation (r = 0.384, p = 0.02)
  • Independence-competence (r = 0.487, p < 0.001)
  • Independence-performance (r = 0.382, p = 0.03)
  • Employment/occupation (r = 0.441, p < 0.001)
• Poor to adequate correlations between ACLS-5 and Wisconsin Card Sorting Test Totals and sub-scores (p < 0.001):
  • Total correct (r = 0.429)
  • Total errors (r = -0.475)
  • Percentage of total error (r = -0.484)
  • Perseverative responses (r = -0.537)
  • Percentage of perseverative responses (r = -0.540)
  • Perseverative errors (r = -0.557)
  • Percentage of perseverative errors (r = -0.555)
  • Percentage of non-perseverative errors (r = 0.257)
  • Categories (r = 0.437)

Substance Use: (Rojo-Mota et al., 2017; n = 232 participants with addictions currently undergoing addiction rehabilitation treatment; Mean Age = 38.26 (11.66); Spanish sample)

• Adequate convergent validity of the ACLS-5 with the visuospatial test of the Montreal Cognitive Assessment (MoCA) (r = 0.43)
• Poor convergent validity of the ACLS-5 with the attentional and abstraction tests of the MoCA (r = 0.18, 0.18)
• Poor convergent validity of the ACLS-5 with the Prefrontal Syndrome Index  (r = –0.19 to 0.07)

Inpatient Psychiatric Patients: (Schubmehl et al., 2018; n = 193 inpatients from a psychiatric unit in California in acute phase of a psychiatric illness diagnosis; Mean Age = 37.2 (14.4); schizophrenia, n = 47; schizoaffective disorder, n = 45; psychotic disorder NOS, n = 23; bipolar disorder, n = 53; major depressive disorder, n = 19)

• Adequate convergent validity of the ACLS-5 with Trail Making Test Part A and Part B (r = 0.46 and 0.45, respectively)
• Adequate convergent validity of the ACLS-5 with Controlled Word Association Test (r = .047)

Inpatient Psychiatric Patients: (David & Riley, 1990

• Adequate convergent validity of the ACL with the Symbol-Digit Modalities Test (r = 0.521)
• Adequate convergent validity of the ACL with the Shipley Institute of Living Scale Abstraction and IQ (r = 0.355 and 0.311, respectively)
• Poor convergent validity of the ACL with the Shipley Institute of Living Scale: Vocabulary (r = 0.252)

Inpatient Psychiatric Patients: (Mayer, 1988; n = 40 adult acute psychiatric inpatients; Mean Age = 33.0 (17.9); Mean Chronicity = 7.5 (6.9))

• Adequate convergent validity of the ACL with the Wechsler Adult Intelligence Scale-Revised edition (WAIS-R): FSIQ, Digit Symbol, Digit Span, Object Assembly, and Picture Arrangement (r = 0.46, 0.59, 0.54, 0.55, and 0.59, respectively)
• Excellent convergent validity of the ACL with the WAIS-R: Block Design (r = 0.618)

Convergent Validity:

Pediatric Mental Health: (Shapiro, 1992; n = 24 males with an emotional disturbance attending a private school in New York; Mean Age = 12.5(Range = 8-15))

• Poor convergent validity of the ACL with Perceptual Memory Task full scale and subscales (r = -0.04 to 0.19)
• Adequate convergent validity of the ACL with the raw and age equivalent Beery VMI scores (r = 0.42, 0.36 respectively)

Known Groups Method:

Pediatric Mental Health: (Lee et al., 2003; n = 61 consisting of 32 adolescents living in the community and 28 adolescents residing in residential mental health facilities; Mean Age = 14.8 (1.5))

• Construct validity supported: Statistically significant difference in scores on the ACL-90 between groups (p<0.05)

Older Adults and Dementia (Wesson et al., 2017; n = 87 older adults with no cognitive impairment, Mean Age = 82.5; n = 43 older adults with mild cognitive impairment (MCI), Mean Age = 83.1; n = 30 older adults with dementia; Mean Age = 79.2)

Diagnosis

Cut-off Scores

Sensitivity

Specificity

No cognitive impairment

4.5

76.7%

90.8%

   vs Dementia

4.7

86.7%

81.6%

MCI

4.5

76.7%

74.4%

   vs Dementia

4.7

86.7%

60.5%

No dementia (No cognitive impairment & MCI)

4.5

76.7%

85.4%

   vs Dementia

4.7

86.7%

74.6%

No cognitive impairment vs MCI

5.1

67.4%

52.9%

Older Adults and Dementia (Wesson et al., 2017)

No cognitive impairment

MCI

Dementia

Mean (SD)

5.13(0.41)

4.94(0.47)

4.45(0.40)

Range

4.4–5.8

4.2–5.8

3.4–5.8

Discriminant Validity

Older Adults and Dementia (Wesson et al., 2017)

• Discriminant validity of LACLS-5 supported: Older adults with no cognitive impairment performed better than older adults with MCI  and older adults with dementia  performed the worst. The Assessment of Motor and Process Skills (AMPS) Motor scores for clinic participants were worse than the Sydney Memory and Ageing Study (MAS) participants in the dementia group (U = 36.5). Lower LACLS-5 scores were associated with increased age (F(1,148) = 5.95).

Convergent Validity

Older Adults and Dementia (Wesson et al., 2017)

• Poor to adequate convergent validity of LACLS-5 with functional variables of AMPS Motor and Process (r = 0.29-0.53)
• Poor convergent validity of LACLS-5 with Disability Assessment for Dementia BADL and IADL subscales (r = 0.29, 0.37 respectively)
• Poor to adequate convergent validity of LACLS-5 with cognitive variables of Mini Mental State Examination, Trail Making Test Part A and Part B, Rey Auditory Verbal Learning Test, Boston Naming Test, and Controlled Oral Word Association Test (r = -0.33 - 0.46)

Convergent validity:

Acquired Brain Injury: (Huertas-Hoyas et al., 2022; n = 80, mean age = 52.20 (10.9), age range = 19-74, male = 44 (55%); Spanish sample)

• Adequate convergent validity between ACLS-5 scores and age (ρ = –0.35)
• Adequate convergent validity between the ACLS-5 and Montreal Cognitive Assessment (MoCA) (r = 0.36)
• Adequate convergent validity between the ACLS-5 and Barthel Index (BI) (r = 0.53)
• Adequate convergent validity between the ACLS-5 and Functional Independence Measure and Functional Assessment Measure (FIM-FAM) (r = 0.54)

Acquired Brain Injury: (Park & Lee, 2020; n = 34, mean age = 56.23 (1.45), age range = 13-76, male = 20 (58.8%); Korean sample)

• Excellent convergent validity between the ACLS-5 and Korean Mini-Mental Status Examination (K-MMSE) (r = 0.778)
• Excellent convergent validity between the ACLS-5 and Cartoon Intention Inference Test (CIIT) (r = 0.817)
• Excellent convergent validity between the ACLS-5 and Lowenstein Occupational Therapy Cognitive Assessment-II (LOTCA-II) total score (r = 0.627)
• Adequate convergent validity between the ACLS-5 and LOTCA- orientation sub-item (r = 0.470)
• Adequate convergent validity between the ACLS-5 and LOTCA-thinking operation sub-item (r = 0.341)
• Adequate convergent validity between the ACLS-5 and Social Behavior Sequence Task (SBST) (r = 0.376)