The Unintended Consequences of Mis-Managing Metrics.

Some time ago, a large agency said to me that they were changing their outcome measures. I was somewhat surprised, given that they have been collecting their outpatient outcomes for some years. 

The reason? Because the existing measure was not showing that they are improving over time. 

I said, “What?”

They wanted to move from a global wellbeing measure to a symptom-specific measure.

Why? Because it is more sensitive to change i.e., make the data look better.

Before we judge this agency, think about why they might want to do this.

After all, these are well-intentioned managers trying to lead a large team of therapists, who are also good people trying to be of help to those in need. 

But these clinical leaders are sandwiched between upper management and clinicians at the ground level.

If they picked a measure that was more sensitive to change, they are more likely to show improvement, which the Finance department would be pleased, so as to justify their funding. 

But is this actually reflective of real improvement for each person? 

Besides, can a client improve in symptoms, but not improve in their overall life, i.e., sense of personal wellbeing, improvement in relationships, engagement in meaningful work? 

And vice versa?

Dignity First

Don’t assume that all kinds of measurement is useful and good. 

We can assume that since global wellbeing, symptom-specific and goal-specific measures are useful, let’s get everyone to do that. 

Not only therapists get burdened, the load gets passed onto their clients. 

Recall: 

FIT is not an evaluative tool.

It is a conversational instrument.

FIT is not just about looking evidence to support your approach.

It is a way to let your dataset change your mindset.

What use would it be if the measures are treated like just another piece of paperwork, primarily for the bean counters?

We devalue the human dignity of the therapist and client when we lose sight of doing meaningful things. This leads to a degradation of the therapeutic work. 

Yet, the use of measures can be meaningful, only if you treat it so. 

​Kim de Jong and colleagues​ found that not all therapists benefited from feedback. They found that commitment and an open attitude to the use of feedback by the therapist moderated the effects of feedback.

Your family physician does not take your temperature and put that number aside for the ministry of health and safety. She uses it to guide her clinical treatment. 

The instruments are only as good as the one who is using it. 

Take the time to focus on what’s most important to the people that you are serving. Then find a way to measure that. 

Finally, as noted by Jerry Muller in ​The Tyranny of Metrics​, even the best measures are subject to “corruption or goal diversion.”

Here are two fundamental questions we must get clear before we proceed with the use of measures:

1. Who is it for?
   
2. What is it for?

Don’t get tricked by the simplistic nature of these two questions. Take the time to discern this. Protect and reinforce this once it is clear.

Don’t get sidetracked. 

Our highest priority is to protect our ability to prioritise. 

— Greg Mckeown

Further Readings

Here are three relevant readings on:

If you would like to learn more on how to improve your work, subscribe to the Frontiers of Psychotherapist Development (FPD). Each week, we serve you highly curated recommendations and insights straight into your Inbox.