Welcome to week 12 of Strength & Conditioning for Therapists. Last week we looked at assessments of muscle strength and the differences in reliability and precision between them. This week we’re going to look at how to calculate your own test reliability. Well, actually I’m going to show you how to work out the precision, or variability of a test so that you can confidently identify whether or not an individual’s performance has actually changed following your rehabilitation/conditioning, or if differences in scores are due to the variability of the test. The more variable a test, the less precise you can be about concluding whether or not your patent’s performance has changed.
The Coefficient Of Variation
Or V% for short, is a measure of the variability of a test. The lower the V%, the more precise the measurement.
For example a coefficient of variation of 8% associated with a test means that the individual’s score recorded is accurate to within plus or minus 8%. I.e. If you record a score of 200 Newtons on a strength test, the actual score will lie somewhere between 208N – 192N, similarly, an 8% coefficient of variation on a Timed Up And Go test of 10s, the actual score would be between 10.8 – 9.2s
This becomes important when you’re trying to detect change in an individual’s performance between tests. In a rehabilitation setting, assessments of performance/function are often weeks apart.
To exemplify, let’s say you conducted assessments on a group of patients before (Test 1) and 6-weeks after (Test 2) a strength-focussed rehabilitation programme. You want to know if your programme has been successful in increasing the muscle strength in these patients. The table below illustrates your findings.
If you look at the Difference column first (Test 2 minus Test 1), it shows that the strength scores increased in each patient. Hurrah, the programme worked! However, can we be certain that these changes are real and not due to the variability of the test?
Let’s assume that this particular test of strength has an 8% coefficient of variation for inter-day (between-day) assessments. What does this mean?? Well, if we see a difference in individual scores from Test 1 to Test 2 that is less than 8%, we conclude that there is NO CHANGE in performance. The precision of this particular test is only capable of detecting changes that exceed 8%. Therefore, Person 4’s performance hasn’t actually changed, the difference observed does not exceed the variability of the test …. Have they been doing their rehab???? 😉
Calculating Coefficient of Variation
So how can you work out the precision of the tests that you use? The formula is:
(Standard Deviation / Mean)*100SD and Means for the population
Here’s what you need to do to get the data:
- Select a sample of people who are representative of the population you want to assess
- Select the test that you want to evaluate
- Standardise the test conditions (see last week’s post) – i.e. make sure the test is carried out in the exact same way for each person on each occasion
- Familiarise each person to the test to eliminate learning effects & let them have a few practice attempts
- After a rest, conduct the test a few times to obtain and average of more than 1 measurement if possible (I typically take 3 MVCs when assessing strength), and record the data as Test 1
- Repeat this process for each patient so you have Test 1 data for multiple people
- Arrange to reassess each patient under exactly the same conditions about a week later. Record the data as Test 2
- Enter your data into a spreadsheet, download the template below
Calculate the Mean of all the values (250.45 in this example) and the SD (Standard Deviation) of the Mean (26.4. ) You can use the ready-made formulas in Excel or Numbers to do this. Now you have all the data you need to calculate V% and your test’s precision for inter-day assessments. Enter them into the formula (SD/Mean)*100) and hey presto!
In this example the precision of this test in detecting change between days is 10.6%. This means individual performance scores recorded between days needs to differ by around 11% to conclude change has occured
You’ve probably seen V% reported in the literature for many standardised tests of function/performance, so why bother? Well, the calculated precision is specific to the population, testing conditions, type of test, etc. Take hand-held dynamometry (HHD) for example, you can substantially improve the precision of the test by securing the HHD to an immovable object so the tester doesn’t have to apply the resistance to the patient’s movement. If you do this, it’d be good to know how precise your test is. Also, if a test’s precision has been evaluated in elite athletes, scores are likely to be much less variable between days and thus the test precision higher compared to if your population is a non-exercising patient, whose score are likely to be more variable over time.
What’s the point of all of this? It’s SO important to take measurements and assess outcomes to help with clinical decision making. But it’s also important to know that the decisions you’re making are based on robust data. If you see a ‘change’, or indeed no change, in scores of a person’s function, performance, pain etc. etc. then make sure that you know that it’s real, and not an artefact of the variability of the test.
IT’S HERE! Download your Free 14-page guide: Strength & Conditioning for Therapists & this weekly blog straight into your inbox!