Occupational Self-Assessment

Turkish, Nondisabled Older Adults (Pekçetin, Salar, Inal, Kavihal, 2018; n = 117; age > 65; nondisabled; Turkish sample)

• Results of exploratory and confirmatory factor analysis were within acceptable ranges.

• Exploratory factor analysis: OSA value item 5 and OSA competence item 16 had the lowest factor loadings.

• Confirmatory factor analysis (CFA): The model fit the data according to indices of relative fit (Root Mean Square Error of Approximation (RMSEA) = 0.99, Chi square ratio of the degrees of freedom (CMIN/DF) = 2.1), but Comparative Fit Index (CFI) value was 0.594 indicating poor fit of this model.

• Convergent validity: Study examined only the competence scores to determine convergent validity.

• OSA competence was evaluated with the functional autonomy measurement system (SMAF) and World Health Organization Quality of Life - Old (WHOQOL-OLD) scales; both scales showed a moderately significant relationship with OSA competence total scores (p < 0.01). Table 5 summarizes correlational data (out of 11 correlations, 7 were adequate, 1 was poor; and 3 were not significant)

• Authors state that the Turkish OSA is a valid and reliable instrument for determining occupational competence and values of nondisabled, elderly Turkish individuals. Normal 0 false false false EN-US ZH-CN X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:Calibri; mso-fareast-language:EN-US;}

Disabled, Inpatient Older Adults: (Stuber & Nelson, 2010; n = 30; mean age = 74.0 (9.9) years)

• Adequate correlation between COPM (performance) and OSA (competence) (r = .51).

• Adequate correlation between COPM (performance) and SIGA (overall) (r = .58).

• Adequate correlation between OSA (performance) and SIGA (overall).

• Excellent correlation between the mean of SIGA self-identified goals and COPM performance scores (r = .76).

• Excellent correlation between the COPM (satisfaction) and COPM (performance) (r = .76).

• Adequate correlation between the COPM (satisfaction) and OSA (competence) (r = .54).

• Excellent correlation between the COPM (satisfaction) and SIGA (overall) (r = .62).

• Excellent correlation between the COPM (satisfaction) and SIGA (self-identified goals) (r = .81).

Turkish, Nondisabled Older Adults (Pekçetin et al., 2018)

• Following the translation of the OSA from English to Turkish, a pilot study was conducted to ensure there was no loss in intended meaning and to guide adaptations.

• Items not fully understood were replaced until all items were understood by all participants.

• Alterations included larger font size to accommodate for visual acuity among the elderly population, as well as separating the competence and value ratings to help respondents differentiate between the two types of ratings.

• Response options were not abbreviated (e.g., instead of “well”, the response choice was, “I do this well.”)

Musculoskeletal Disorders (Murad, Farnworth, & O’Brien, 2011; n = 35; Mean Age = 40.8 (11.6) years; Injured workers with musculoskeletal disorders; Not enrolled in SOCSO RTW program; Multicultural, multi-ethnic, Malaysian sample)

• SEM for entire group (n = 35): 3.74; Calculated from 10.38 x (square root of (1 – 0.87)).

Musculoskeletal Disorders (Murad et al., 2011)

• MDC for entire group (n = 35): 10.37; Calculated from 1.96 x 3.74 x square root of 2.

Musculoskeletal Disorders (Murad, Farnworth, & O’Brien, 2012; n = 35; mean age = 40.8; injured workers with musculoskeletal disorders; Not enrolled in a RTW program; multicultural, multi-ethnic, Malaysian sample)

• Total Score ICC not provided.

• Adequate-Excellent test-retest reliability: (ICC= 0.564-0.844) for 20 out of 21 items.

• Adequate test-retest reliability: (ICC= 0.413) for item #8, “managing my basic needs."

Musculoskeletal Disorders (Murad et al., 2011)

• Excellent test-retest reliability: (ICC = .87).

• Exceeded minimal acceptable level of 0.60 and was significant (p < 0.01).

Note: Researchers examined only the competency scale of the OSA.

Musculoskeletal Disorders (Murad et al., 2012)

• Excellent: Cronbach's alpha = 0.91 for items that make up the instrument.

• Adequate: Cronbach's alpha = 0.79 for subscale domain items that make up the instrument (Volition, Habituation, Skills/Occupational Performance).

Musculoskeletal Disorders (Murad et al., 2011)

• Excellent: Cronbach's alpha = 0.91.

Musculoskeletal Disorders (Murad et al., 2012)

Convergent validity:

• Volition: Poor convergent validity with stress, anxiety, and depression subscales of DASS-21 (r = -0.412, -0.467, -0.403, respectively).

• Volition: Adequate convergent validity with vitality, physical functioning, role emotional, and role physical subscale of SF-36 (r = 0.370, 0.519, 0.398, 0.402, respectively).

• Habituation: Poor convergent validity with stress, anxiety, and depression subscales of DASS-21 (r = -0.491, -0.399, -0.455, respectively).

• Habituation: Adequate convergent validity with role emotional, and role physical, and mental health subscale of SF-36 (r = 0.499, 0.364, 0.515, respectively).

• Skills/Occupational Performance: Adequate convergent validity with physical functioning, mental health, and role physical subscale of SF-36 (r = 0.513, 0.353, 0.407, respectively).

Musculoskeletal Disorders (Murad et al., 2011)

Convergent validity:

• Poor convergent validity with stress, anxiety, and depression subscales of DASS-21 (r = -0.426, r = -0.398, r = -0.398, respectively).

• Adequate convergent validity with the physical functioning subscale of SF-36 (r = 0.552).

• Weak, but significant convergent validity with role physical and mental health subscales of SF-36 (r = 0.461, r = 0.451, respectively).

• Weak and non-significant convergent validity with vitality and role emotional subscales of SF-36 (r = 0.314, r = 0.331, respectively).

Discriminant validity:

• Very low and non-significant discriminant validity with pain intensity subscale of VAS (r = -0.19).

Musculoskeletal Disorders (Murad et al., 2011)

• Forward and back translation was completed by two translation experts from MARA University of Technology, Malaysia.

• Six occupational therapists (mean work experience = 5.1 (5.9) years; fluent in English and Malaysian) rated each item on a 5-point Likert scale for grammar and accuracy in meaning and content.

• Malaysian OSA was piloted with five workers with MSDs.

Iranian, Nondisabled Children (Asagari & Kramer, 2008; n = 336; mean age = 13 (1.2); nondisabled; Persian sample)

• Persian OSA has acceptable construct validity.

• The model fit the data according to indices of relative fit for competence and values scales, respectively (chi-square = 1.26, 1.25; RMSEA = 0.29. 0.29; GFI = .91, .90; AGFI = .89, .88; CFI = .90, .90; ECVI = 1.02 for both scales).

• Correlation coefficients between first order factors (skills, habituation, volition) and the constructs of competence and values were statistically significant (p < .01), and justified this second-order model.

• All standardized factor loading were significant (t > 2.0); items were reliable and correlated to first-order factors.

• Two sets of squared multiple correlations were calculated for the second-order model.

• Squared multiple correlations of items 4 and 5 on both competence and values scales, item 14 in the competence scale, and item 9 in the values scale are low (these items were not effective measures of underlying first-order factors)

• Squared multiple correlations for the second set: skills (.98, .99), habituation (.88, .98), and volition (.99, .99) on competence and values scales, respectively.

• Persian OSA items showed good fit to Rasch model (all competence items and 19 of 21 values items met requirements for fit).

• Competence item hierarchy did not replicate item hierarchy found in international OSA analysis (Kielhofner et al., 2007). OSA competence and values item reliability = .95 and .96, respectively; separation indexes = 4.58 and 5.21, respectively (indicated 6.44 and 7.29 statistically different levels of competence and value for occupations distinguished by Persian OSA items).

• 307 students were measured on competence scale of Persian OSA; 306 students were measured on values scale of Persian OSA; misfitting students = 8.6%, 8.9%, respectively; Client separation reliability on both scales = .75; Student separation index = 1.74, 1.73, respectively (2.6 and 2.64 statistically different levels of student competence and value measured).

• Persian OSA items effectively “capture” competence, but not value.

• Persian OSA is aligned with theoretical relationship proposed by MOHO: occupational competence and value for occupation are impacted by volition, habituation, and skills.

• CFA and Rasch results provided strong evidence for construct validity of the Persian OSA. OSA items measured latent traits of occupational competence and value fairly well. Factor loadings indicated that most items were highly correlated with latent traits.

• All competence items fit the Rasch model; only two values items exceeded fit requirements. High Rasch item separation reliability, separation index, and strata statistics for both scales provided evidence that items are reliable.

Adolescents with Infectious Mononucleosis (Taylor, Lee, Kramer, Shirashi, & Kielhofner, 2011; n = 296; age range = 12-18)

• At baseline, 90.48% of the occupational competence scale items and 90.48% of the values scale items met fit requirements (MNSQ statistic < 1.4 associated with a ZSTD < 2). Scale items would coalesce to capture the intended constructs.

• At baseline, the occupational competence rating scale was used as intended as average client measures and thresholds were ordered and all rating scales had acceptable fit statistics. The values rating scale was also used as intended at baseline. Average client measures associated with each rating category and threshold difficulties increase incrementally, and outfit MnSq fit statistics for each category are less than 2.0 (Linacre, 2002).

• Fit statistics could not be calculated on small numbers of participants who used the highest ratings for all items; percentages are based on the total participants for whom fit statistics could be calculated. At baseline, 7.12% misfit on the competence scale and 8.87% misfit on the values scale. At 12 months clients, 8.23%, misfit on the competence scale and 6.74% misfit on the values scale. Thus, in all instances over 90% met criteria for being validly measured (MnSq statistic < 1.4 associated with a z-std < 2).

• At baseline, the OSA competence items measured almost four significantly different levels (3.99 strata) of competence among adolescents (person separation index = 2.74). The OSA value items also measured four significantly different levels (4.03 strata) of value among adolescents (person separation index = 2.77).

• OSA had evidence of minimal variance between baseline and 12 months. Competence item calibrations remained stable over time (Baseline, N = 296; 12-month, N = 90). Only one competence item became significantly more challenging at 12 months. The OSA competence rating scale thresholds were not all stable over time; adolescents felt more competent and had an easier time using the highest rating category of “extremely well” at baseline (N = 296; 12-month, N = 90). However, the estimates were only separated by 0.44 logits and the associated errors were small. Difference was significant, yet small.

• Majority of the items on the values item hierarchy were the same over time (Baseline, N = 296; 12-month, N = 90). Three OSA values items were significantly different at 12 months. The OSA value rating scale thresholds were not all stable over time; adolescents indicated more value for more items and had an easier time using the highest rating category of “most important” at baseline (Baseline, N = 296; 12-month, N = 90 ). However, as with the competence rating scale, the baseline and 12-month parameter estimates differed by a small amount (0.29 logits) and the associated errors were small. Difference was significant, yet small.

• To account for changes in the rating scale over time, initial and 12 month OSA client measures were obtained using a common rating scale structure derived from a combined, simultaneous analysis of both time points. Although four items varied over time, no item varied over time on both scales. Therefore, the OSA items were considered invariant over time and considered to also have a shared item estimate at both time points. Deriving person measures from this common rating scale places the baseline and 12-month assessment on a shared and common metric, or frame of reference, and derives a more conservative but more rigorous estimates of change (Wolfe & Chiu, 1999).

• The OSA competence scale detected significant change in 34% of the participants while the values scale detected significant change in 38%. The OSA competence score detected significant increases in 34% of adolescents who were recovered and in 19% of those with ongoing fatigue. The OSA competence score detected significant decreases in 3% of those recovered and 10% of those with ongoing fatigue. The OSA values scale detected similar rates of significant increases and decreases in both groups. Twenty-four percent of recovered and 26% of those with ongoing fatigue showed increases, while 14% of those recovered and 13% of those with ongoing fatigue showed decreases.

• There were no significant associations between the adjusted OSA competence and OSA value measure with gender and ethnicity at baseline (N = 296). There were weak or no associations between OSA value baseline measure and age (N = 296).

• OSA competence scale was moderately correlated with infectious symptoms, fatigue, health status, and perceived stress. As expected, the OSA value measures were not correlated with infectious symptoms or fatigue severity. Significant but minimal correlations were found with stress and global health.

• T-tests comparing adolescents who were non-recovered at 12 months follow-up with matched controls who were recovered adolescents revealed that non-recovered adolescents had lower adjusted competence measures (M = 54.98, SD = 8.79) than those who were recovered (M = 62.21, SD = 8.33), t (88) = −3.84, p = 0.00. There was no significant difference between these groups in the reported values/value measures.

Iranian, Nondisabled Children (Asagari & Kramer, 2008)

• Following the translation of the OSA from English to Persian, a pilot study was conducted to ensure there was no loss in intended meaning and to guide revisions. The Persian OSA was back translated to English by a team of bilingual experts.

Arabic, Nondisabled University Students (Yazdani et al., 2008; n = 50; Jordanian sample)

• A preliminary test-retest study indicated the Arabic translated instrument was reliable and acceptably stable over time.

• ICC not provided.

Arabic, Nondisabled University Students (Yazdani et al., 2008)

• Habituation (Competency):

  • Adequate correlation with the Afectometer 2 (r = .497)

  • Adequate correlation with Satisfaction with Life (SWL) (r = .391)

• Habituation (Values):

  • Poor correlation with the Afectometer 2 (r = .167)

  • Poor correlation with the SWL (r = .112)

• Volition (Competency):

  • Adequate correlation with the Afectometer 2 (r = .383)

  • Poor correlation with the SWL (r = .282)

• Volition (Values):

  • Poor correlation with the Afectometer 2 (r = .185)

  • Poor correlation with the SWL (r = .107)

• Performance Capacity and Skills (Competency):

  • Adequate correlation with the Afectometer 2 (r = .470)

  • Adequate correlation with the SWL (r = .366)

• Performance Capacity and Skills (Values):

  • Poor correlation with the Afectometer 2 (r = .227)

  • Poor correlation with the SWL (r = .156)

• Environmental Support (Competency):

  • Adequate correlation with the Afectometer 2 (r = .459)

  • Adequate correlation with the SWL (r = .469)

• Environmental Support (Values):

  • Poor correlation with the Afectometer 2 (r = .203)

  • Poor correlation with the SWL (r = .081)

Physically Disabled, Psychiatrically Disabled, and Non-disabled Adults (Kielhofner & Forsyth, 2001; n = 302; Mean age = 45.45 (19.38) years)

• The competence scale revealed 14 subjects (4.6%) misfit the Rasch model which is evidence that subjects were validly measured by the OSA.

• Person separation was 2.80. The mean of person measures was 54.86 +- 8.01 demonstrating that the difficulty of the items (item measure mean = 50.00) fit the mean of the sample.

• The 8 environment items did an adequate job measuring the construct of environmental support. There was no item misfit. 14 subjects (4.6%) misfit, reflecting that the scale was valid for a large proportion of the subjects. The misfitting subjects didn’t share any similar characteristics. Subject separation was 1.76. The mean of person measures was 56.09 +- 9.97. This signifies that the items in this section were targeted below the subjects.

• The items in the value scale did an adequate job defining occupational competence. No items misfit. The scale is valid due to the calibration of the items from most important to least important. 15 subjects (5%) misfit meaning there’s no significant misfitting. Person separation was 1.84. The mean of person measures was 58.99 +- 6.81 demonstrating that the subjects were calibrated higher than items (item measure mean = 50.00).

• The values of environmental support did an adequate job measuring the construct of environmental impact. No item misfit. 5 subjects (1.6%) misfit demonstrating its validity for all the participants. The mean of person measures was 40.14 +- 6.17 demonstrating that the subjects were calibrated lower than items (item measure mean = 50.00).

Disabled and Non-disabled Adults: (Kielhofner et al., 2009; n = 515; Mean age = 54.71 (18.22) years; English and non-English speaking sample)

• The items in the Occupational Competence scale had acceptable fit statistics and adequately defined the construct of Occupational Competence. In the occupational competence scale, 41 % of the participants had a measure higher than the highest item measure and there was a ceiling effect. 94% of the participants demonstrated good fit statistics according to their pattern of responses to the OSA Occupational Competence scale. Of the 32 participants who exceeded the fit requirements, there were no correlation to diagnosis or country where data was collected from. The Occupational Competence scale demonstrated evidence of being a valid scale for a heterogeneous sample.

• The items in the Value scale adequately defined the construct of Values. The item ‘Taking care of others for whom I am responsible’ exceeded fit requirements.18% of participants had measures higher than the highest Values item measure and there was a ceiling effect. 91.4% of individuals met the fit criteria and consistently demonstrated similar responses. The 44 individuals who exceeded fit requirements had no patterns diagnosis or nationality. This is evidence that the Values scale is a valid scale for majority of individuals.

• The environmental scaled revealed inadequate separation statistics with 1.62 as the Environmental Impact scale and 1.45 as the Environmental Values scale. This may signify that the environmental scale needs additional items to expand the sequence of environmental constructs.

Non-disabled Adults and Adults with Multiple Sclerosis (Kielhofner et al., 2009; n = 86; Mean age = 47.94 (12.72) years; English and non-English speaking sample)

• In the Occupational Competence scale, 18 items had adequate fit statistics and acceptably defined the construct of Occupational Competence. Only 2 individuals had measures higher than the highest Occupational Competence item measure indicating that there was little demonstration of a ceiling effect. The participant separation and reliability increased. The category ‘I have a lot of problem doing this’ was collapsed with ‘I have some difficulty’ due to the fact it was only use 5% of the time and the data was analyzed as a three point scale. All the items met fit requirements which indicated that the reworded 4 point rating categories may have caused the individuals to use the OSA rating scale in a way that was not intended.
• In the values scale, 19 items with the reworded 4 point rating categories had acceptable fit statistics. The items of ‘Doing activities I like’ and ‘Taking care of the place where I live’ exceeded fit criteria. The average measures for participants did not match well with the average of the item measures. The person separation index on the values scale decreased. The 4 point values rating scale was not equally used. The rating category ‘Not so important’ was used 1% of the time and ‘Somewhat important’ was used 7% of the time. When this data was reanalyzed with these 2 categories collapsed, only 1 item exceeded the fit requirements.

Disabled and non-disabled Adults (Kielhofner et al., 2009; n = 542; Mean age= 47.71 (16.75) years; English and non-English speaking sample)

• In the Occupational Competence scale, 20 items had adequately defined the construct of Occupational Competence. The item of ‘Managing my finances’ exceeded fit. 7.2% of the individuals had measures above the highest item measure with a small ceiling effect. 90.2% of individuals demonstrated adequate fit statistic with 53 participants exceeding the fit requirement.

• In the Values scale, the items had adequately defined the construct of Values because each item demonstrated acceptable fit. 7% of the measures of individuals were above the highest item measure indicating a minor ceiling effect. The person separation index increased demonstrating that the revised scale increased the scale’s ability to discriminate among individuals. 90% of individuals had valid measures and 55 individuals exceeded fit requirements.

Adults with Disabilities (Kielhofner et al., 2010; n = 112, Fatigue following cancer treatment (54.5%), multiple sclerosis (34.8%), no impairment (9.8%), bipolar disorder (0.9%); Mean age = 52.6 years)

• The OSA was administered twice to 112 participants, 58 to 650 days between administrations.

• Separate Rasch Rating Scale Model (RSM) analysis was used for each OSA scale, and for each administration.

• Large differences are indicators of instability (i.e., I z I > 1.96).

• Of the 21 items on Competence scale, only one item ("Taking care of others for whom I am responsible") had a significantly different estimate over time (standardized difference < 1.96 logits, item difficulty difference > .50 logit) indicating an increase in difficulty.

• Overall, the items on the Competence scale functioned as the same measurement instrument across each of the two administrations, and was used as intended, as indicated by incremental increases in step calibrations.

• The standardized difference for the third step calibration (between the "Well" and "Extremely Well" categories) was significantly different (z < -1.96).

  • However, minimal difference between the two estimates and small errors (of the order of .06 to .14 logit). Thus, the difference between the third step calibrations can be overlooked. The Competence scale could be considered sufficiently stable over the two time points.

• Of the 112 participants included in this analysis, 32% showed significant change in the level of Competence reported for everyday activities; 17 participants showed a significant decrease in person measures (z > 1.96) with less Competence reported at time 2, and 19 participants showed a significant increase in person measures ( z < -1.96) indicating higher Competence reported at time 2.

• Only one item ("Identifying and solving problems") showed a significant difference in difficulty between time 1 and time 2 administrations.

• The Values rating scale was used as intended at both time 1 and time 2 as step calibrations increased incrementally. However, two of the three Values step calibrations had significant standardized differences, indicating that the rating scale was not used in the same manner at both administrations.

• Time 1 Values data were re-analyzed using the step calibrations obtained from the combined analysis (Fjc). That is, the Values steps calibrations were “anchored” to these estimates. Then corrected estimates were obtained for item difficulties (Di1c) and person measures (Bn1c) at time 1.

• Time 2 Values responses were re-analyzed. The Values rating scales step calibrations were anchored to the calibrations obtained during the combined analysis (Fjc).

  • However, for time 2, the 20 items that were stable (all but the item “Identifying and solving problems”) were anchored to the item difficulties obtained during the corrected time 1 Values analysis (Di1c).

• Time 1 became the common estimate for the 20 stable items because they were considered to be used in the same way by participants over time.

• The resulting time 2 person measures (Bn2c) are along a construct of Value that is common to both assessment administrations.

• The comparison between the corrected time 2 person measures (Bn2c) and the corrected time 1 person measures (Bn1c) resulted in significant standardized differences (| z | > 1.96) for 55 participants (49% of the data). Of these, 40 (36%) reported more value for everyday activities at time 1 and had higher person measures at time 1 (z > 1.96). Fifteen participants (13%) reported more value for everyday activities at time 2 and had higher person measures at time 2 (z < -1.96).

Danish, Psychiatric Clients (Peterson & Hartvig, 2008; n = 18; Danish sample)

• Researchers developed a pilot version of the OSA in Danish based on theoretical background of MOHO, content of the OSA, and previous Danish translations of MOHO concepts to ensure conceptual equivalence. The pilot version was tested by eight occupational therapists with 18 clients in two different psychiatric settings.

• Following two semi-structured qualitative group interviews with the eight occupational therapists who used the pilot version, researchers made changes to assure the translation used words and expressions common in Danish, as well as to make sure the concepts were accurately represented in Danish.

• The second translation was peer-reviewed by Danish- and English-speaking occupational therapists with expertise in MOHO assessments. The researchers made further changes using the therapists’ recommendations for making the translation more readily understandable and for more accurately representing MOHO concepts.

• An authorized translator who was not an occupational therapist made a back-translation from Danish to English, which did not reveal problems or need for further changes.