Don't get caught Franken-testing!

[fa icon="calendar'] Oct 30, 2018 10:24:18 AM / by Erin Norton

When doing Functional Capacity Evaluations (FCE) or Pre-Hire Physical Abilities Testing (PAT) some clinicians and FCE vendors pull a bit of a testing protocol from here and a bit of another from there, slap it together and call it a defensible, functional test.  While that might help get the job done, it won’t be anything close to defensible – unless you’re talking about the martial arts maneuvers that you, the physician, insurance carrier or employer will be doing - when the test is legally challenged. 

While nobody wants to find themselves in court – there are many other reasons to care about defensible FCEs and PATs. Here are just a few:

  • A defensible FCE/PAT is an accurate FCE/PAT. First and foremost, at the end of the day, you want to go home feeling like you’ve conducted a fair and objective test that truly determines a person’s physical abilities as they relate to work.  An accurate post-injury FCE or Return-to-Work test helps to determine when – or whether - someone can go back to work in their own – or any job.  Pre-Hire tests determine whether someone gets the job – or not.  There’s a lot at stake riding on these assessments.  They not only affect employee/employer pocket books but deeply affect the social fabric of the worker’s life.  When people can’t or don’t work, their personal lives often unravel.  So, you want a test with proven validity: one supported by peer-reviewed, published research. Don’t accept anything less.
  • A defensible FCE/PAT is a reliable FCE/PAT. Secondly, you want to know that your test results on a patient or job applicant would be the same if performed by any other therapist.  If you and another therapist administer the same test to the same person and each gets a different test result, which one is correct?  Your results?  Their results?  Neither? In order to have full confidence in your test, it should have peer-reviewed published research with demonstrated reliability. 
  • Why is defensibility so important? If you perform a test that isn’t backed by reliability and validity research, it won’t hold up in court when challenged. And you can be sure that if there’s even a hint you used a test that isn’t backed by evidence, it will be challenged. How can you be so sure? Because Uncle Sam says so: the use of testing protocols not supported by research is essentially illegal under the ADA and EEOC guidelines.  Read Part 1607 in the EEOC’s Uniform Guidelines on Employee Selection Procedures, sometime when you can’t sleep.   https://www.gpo.gov/fdsys/pkg/CFR-2017-title29-vol4/xml/CFR-2017-title29-vol4-part1607.xml. If you’ve been using a test that doesn’t meet these standards, it’s going to be a long night.

Section 1607.9 of this document clearly states what won’t pass as validity:

Frankenstein_1Unacceptable substitutes for evidence of validity. Under no circumstances will the general reputation of a test or other selection procedures, its author or its publisher, or casual reports of its validity be accepted in lieu of evidence of validity. Specifically ruled out are: assumptions of validity based on a procedure's name or descriptive labels; all forms of promotional literature; data bearing on the frequency of a procedure's usage; testimonial statements and credentials of sellers, users, or consultants; and other nonempirical or anecdotal accounts of selection practices or selection outcomes.”  

In other words – you can’t fake it.  Or, at least you’re not supposed to.

There’s research and then there’s marketing.  So, here’s where Franken-testing comes in.  If you ask a functional testing vendor whether their test is backed by research or if it’s defensible, you’ll hear a resounding “Why, YES!!!!  OF COURSE!!”  But if you actually look at the studies they cite, you’ll too often find they’ve borrowed a lifting protocol from here and a sincerity of effort protocol from there.  Then they snagged the references behind those protocols for their “reference list” (many of which don’t address reliability and validity) And where protocols don’t exist (for example, when testing tolerance of work positions and repetitive movements or mobility), these vendors will just use a bit of creativity to substitute for actual evidence and problem solved. Right? Not so fast...

Frankenstein_2Here’s the dirty little secret: There never was any reliability or validity research backing their “test.” It’s a little reminiscent of how Dr. Frankenstein worked – a little bit of this and a little bit of that and voila! – you have a monster!  Unfortunately, Dr. Frankenstein’s monster came back to endanger not only his life but the lives of his family, leading him to regret his creation.  If you

Franken-test and you may be creating some profound consequences for your patients, employers and job applicants; it’s a safe bet that you’ll regret that, too.

  • Objectivity, Reliability and Validity are expensive. When you’re choosing a functional testing system for FCEs and PATs, take time to read the studies.  Real research is hard, complicated and expensive to conduct – and usually happens in a university-based setting with external funding. 

The ErgoScience Difference.  The ErgoScience FCE/PAT was developed through university-based research and has been replicated by outside investigators.  From the beginning, our test developers set out to systematically and objectively develop a reliable and valid protocol, and then to examine that protocol through prospective research.  We spent 5 years developing and validating a testing protocol – one with a scoring system that makes it possible to objectively rate observations of materials handling, position and movements.  Our research has been published in the peer-reviewed Journal of Occupational Medicine and Work.  We didn’t just throw something together and then try to justify it.   

If Franken-testing scares you, ErgoScience won’t. If you’d like to learn more or read our research articles, call us (866) 779-6447 today for a no-obligation assessment of your FCE testing needs. Or visit us on the web at https://www.ergoscience.com. You’ll be glad you did.

Topics: Clinicians

Erin Norton

Written by Erin Norton

Erin help clinics find the best solution to fit their needs from functional capacity evaluations, to job analysis, pre-employment screens or impairment ratings.