Modified Fresno Test

Last Updated

November 14, 2018

Purpose

The Modified Fresno Test (MFT) was created utilizing information from the original FT and the AFT to make modifications to measure the skills of evidence-based practice (EBP) for physical therapists using the 5-step EBP model (ask, acquire, appraise, apply, and assess).

Please see the original Fresno Test and Adapted Fresno Test.

Link to Instrument

Instrument Details

Acronym MFT

Area of Assessment

Reasoning/Problem Solving

Assessment Type

Other

Administration Mode

Computer

Cost

Free

Actual Cost

$0.00

Allied Health Care Professions

Utilizes the entire five-step evidence-based practice model (ask, acquire, appraise, apply, and assess)
Eight short answer questions that include two case studies, two questions that require a series of mathematical calculations, and three fill-in-the-blank questions
Total score 0-232; A higher score indicates a better understanding of EBP
Standardized grading rubric assigns points to all answers
Assessor training required on the use of the grading rubric and to be content experts on teaching EBP

Number of Items

13 questions

Electronic access with no external resources allowed except for a calculator, pen/pencil, and note paper

24.5-56 minutes

Up to 60 minutes (24.5-56 minutes)
Time to grade: 10-20 minutes (Miller, Cummings, & Tomlinson, 2013; Tilson, 2010)

Required Training

No Training

Instrument Reviewers

Kathy Sanders, MS, OTR/L and Andrea Vassev, MPT

Considerations

Please see the original Fresno Test and Adapted Fresno Test.
The FT, AFT, and MFT are best utilized for assessing participants’ knowledge of EBM or EBP. It is important to note that they may not test an individual’s ability to successfully apply EBM or EBP in the clinical setting (Ramos et al., 2003).
It is essential to select the correct Fresno Test for your discipline because the case scenarios and test questions are geared toward specific disciplines. For example, an OT or PT would not be appropriate for the FT as its clinical case scenarios are specific to medical practice.
The FT, AFT, and MFT can be used when measuring a change in knowledge after EBP instruction or to determine areas of weakness before an instruction or practice is implemented (Argimon-Pallàs et al., 2011). It may be helpful to compare groups based on pretest upper and lower (25%) quartiles if making comparisons utilizing a pre and posttest design (McCluskey & Bishop, 2009).
Consider giving the pre and posttest at least four weeks apart, to help prevent bias based on test recall (Argimon-Pallàs et al., 2011).
MFT raters participated in three different training sessions on three different days; 2-hour introduction on the test, scoring rubric, and standardized data collection form; 2.5 hours of practice with the scoring rubric; and 1.5 hours for questions and discussion (Tilson, 2010).
Information concerning rater training in the clinical setting is lacking, therefore it is unclear how long the training process would take and also what support and resources may be needed when utilizing novice raters.
There are two additional studies that are adaptations of the original Fresno Test, specific to pharmacy students (Coppenrath, Filosa, Akselrod, & Carey, 2017), and to pediatric bedside nurses (Laibhen-Parkes, Kimble, Melnyk, Sudia, & Codone, 2018), that are not included in this review.

Standard Error of Measurement (SEM)

PT students without EBP training: (Miller, Cummings, & Tomlinson, 2013; n = 38; mean age = 23, standard deviation (SD) = 8 years; 1^st year PT students; tests taken over 2-week period)

SEM = 11 points

Minimal Detectable Change (MDC)

MODIFIED FRESNO TEST

PT students without EBP training: (Miller, Cummings, & Tomlinson, 2013)

MDC₉₀ = 25.7 points

Cut-Off Scores

PT students without EBP training: (Miller, Cummings, & Tomlinson, 2013)

Based on the authors’ expert opinion, passing is defined as achieving at least 50% of the possible point value on an individual item.

PT students with and without EBP training and PT faculty with expertise in EBP: (Tilson, 2010; total n = 108; 1^st year PT students without EBP training, n = 31; 3^rd year PT students with EBP training, n = 50; PT faculty who are EBP experts, n = 27)

Based on authors’ expert opinion, a passing score is defined as > 50% of available points for individual items.

Normative Data

PT students without EBP training (Tilson, 2010)

Mean score = 92.8 (40.0% of total)

PT students with EBP training (Tilson, 2010)

Mean score = 118.5 (51.1% of total)

PT faculty with expertise in EBP (Tilson, 2010)

Mean score = 149.0 (64.2% of total)

Test/Retest Reliability

MODIFIED FRESNO TEST

PT students without EBP training: (Miller, Cummings, & Tomlinson, 2013; n = 35)

Adequate test-retest reliability (ICC = 0.46)

Interrater/Intrarater Reliability

PT students without EBP training: (Miller, Cummings, & Tomlinson, 2013; two raters who teach EBP to DPT students)

Excellent inter-rater reliability (n = 20) ICC = 0.83
Excellent intra-rater reliability (n = 11) ICC = 0.85 and 0.94

PT students with and without EBP training and PT faculty with expertise in EBP: (Tilson, 2010; n = 22; two raters who teach EBP to DPT students)

Excellent inter-rater reliability (ICC = 0.92)
Excellent intra-rater reliability (ICC = 0.96 rater 1 and 2)

Internal Consistency

PT students with and without EBP training and PT faculty with expertise in EBP: (Tilson, 2010)

Adequate internal consistency (Cronbach’s alpha = 0.79)

Construct Validity

PT students with and without EBP training and PT faculty with expertise in EBP: (Tilson, 2010; total n = 108; novice DPT 1st year students without EBP training, n = 31; trained DPT 3rd year students with EBP training, n = 50; DPT faculty who are EBP experts, n = 27)

Significant linear trend (p < 0.001) for sequentially improved mean score by group corresponding to level of EBP training

Content Validity

Novice DPT Students Year 1, Trained DPT Students Year 3, and Physical Therapy Faculty Experts in EBP: (Tilson, 2010)

The MFT was assessed by four physical therapy EBP experts from diverse academic and geographic settings.

Face Validity

Experts in PT EBP: (Tilson, 2010; n = 4)

Experts agreed that the MFT is a comprehensive assessment of EBP knowledge and skills for physical therapists which is shown by score improvement with higher levels of training.
Lack of ceiling effects showed that scores on FT correspond to mastery of material rather than years of clinical experience.

Floor/Ceiling Effects

PT students with and without prior training in EBP and PT faculty with expertise in EBP: (Tilson, 2010)

Excellent; scores ranged 50-200 on a scale of 0 to 220.
Floor effects were midigated by setting a passing score that is lower than “mastery of material”, and none were observed
No ceiling effects were observed

Bibliography

Miller, A. H., Cummings, N., et al. (2013). “Measurement error and detectable change for the Modified Fresno Test in first-year entry-level physical therapy students.” J Allied Health 42(3): 169-174. Find it on PubMed

Tilson, J. K. (2010). “Validation of the Modified Fresno Test: Assessing physical therapists' evidence based practice knowledge and skills.” BMC Med Educ 10(38): 1-9. Find it on PubMed