Managing Statistical Illusions When Interpreting Psychometric Instruments


When a coach or selection specialist begins to use psychometric tools, there is a tendency to over-value the statistical outputs and results, and undervalue the self-reports or other data about the person being assessed.

Put simply, the most predictive elements of future performance are (in order of importance):
1. Current performance and preference patterns (the most predictive indicator)
2. Bio-Data - the actual history of the person (the next most reliable indicator.)
3. Consistent and unbiased input into behaviorally based and incident specific performance measurement systems (the next most reliable indicator)

It is also important to note that an unstressed self-report of preferences and interests may be as good an indicator of future performance as a psychometric instrument.

The advantage of the use of an instrument comes into play when there is a known and sound baseline of data against which the person is being compared for the same interest or preference. The preferences and interests, if they are clearly defined and reasonably well measured by the instrument, can provide a common language and foundation for comparison.

Human Patterns® uses a data set of 5000 employed American workers taken from multiple SEC codes in which all the members of the entity are included -- from the CEO to the person who holds the lowest level job in the organization.

Even with a good data set, intervening variables can intrude on the reliability of the results. Many factors can have an impact on the choices people make during an administration, including fluency with English, current job stresses and satisfaction, chemical consumption, a desire to map responses to an imagined ideal - especially if the psychometric is part of a selection or promotion process - and even prior sleep or exercise.

The rule of thumb used by the responsible coach administering Human Patterns® is to trust the input from their coachee during an administration, and as reported in the Human Patterns output, but only WITHIN LIMITS! When the person completing the instrument objects to their results, the coach or administrator needs to double-check the scores and make sure the language the coach is using reflects the actual standard deviation number and does not over or underweight the score.

On the other hand, scores greater than a single standard deviation - even if qualified or even objected to by the person who completed the instrument - do present a likelihood of reliability and replicability. After all, the person who made the choices that led to the statistical output was not induced, seduced, or coerced in any way. The choices made were their own and - unless they were being intentionally arbitrary or misleading - reflect a "real" difference with others who completed the instrument.

When all is said and done, the purpose and best use of the administration of a psychometric tool is to offer a COMMON CONTEXT for exploration of preferences and interests. The real advantage of the tool may well be the common context - even if there are legitimate differences of opinion about the ability of the instrument to capture a specific pattern from a specific person.

Testimonials

A. Laffoley

Academic Program Director Raleigh- Durham, NC

I used the Human Patterns Inventory in the development program for high potential senior leaders and recommend it as an effective tool as part of any comprehensive employee development program. In my opinion an important differentiator of this tool is the light it shines on the switches that may occur in our behavior when we are in reaction mode (e.g. in a stressful situation). Bringing awareness to where this occurs is invaluable to an individual’s personal development.

K. Jobe

Executive Recruiter Charlotte, North Carolina Area

I have used the Human Patterns as an internal recruiter as well as during client “coaching” engagements. It is the most comprehensive psychometric test that I have ever worked with. I highly recommend this tool to any organization that is committed to talent optimization.

F. Christian

Managing Director Chicago, IL

Human Patterns is a rare exception among assessment tools. Most are simplistic and slipshod, more mirrors of their creators' craniums than windows into one's own. Human Patterns has a richness that allows me to start meaningful conversations with the hidden high potentials I work with, who after years of severe underemployment have lost sight of themselves and their unique ways of working with the world. I'm so enthusiastic I now require it for new clients to shortcut to solutions.