The Croftlands Trust first introduced the Outcomes Star nearly 2 years ago.
The organisation works with people across social care, including people with mental health and/or drug and alcohol issues, brain injured adults and older adults.
The star is used consistently in two substance misuse services and the mental health crisis service.
Here, Mark Barrett describes the process of implementing the Star and a difference he has seen it making:
We were motivated by wanting to learn as an organisation
I have had a personal curiosity about how to measure and demonstrate outcomes for service users since working in the substance misuse field in the early 1990s.
My main motivation was to learn more as an organisation about where we were and were not having an effect and use that learning to improve services.
We went in search of a tool, came across the Outcomes Star and were really impressed.
The first step was getting people to try it and integrating both the star and motivational interviewing theory and practice into our ways of working.
We let people just have a go, in their own way
Workers were unsure at first but my approach was to give the Star to those workers likely to be most open and encourage them to just have a go with it in their own way.
Almost immediately, positive feedback from those workers and their service users encouraged others to try it too.
Now I can guarantee that whenever I visit a project, one or more of the service users will pull out their completed star to show me the progress they have made.
We noticed a measureable increase in client engagement
One of the first effects we saw was a measurable increase in client engagement and length of stay.
Since we introduced the star and motivational interviewing at the same time, it is impossible to say which is responsible for increasing engagement or whether it is a combination of the two.
However, it is clear that the Star helps staff focus on the agreed support plans so they don't get too entrenched in the day-to-day issues brought in by their clients and lose sight of the bigger picture. Further, it also shows both improvement and where people need more support; it reinforces to staff the progress made and helps engage clients in their own process.
It helped us develop and improve our needs assessment and reinforced to staff the importance of really engaging with service users.
Project level outcomes data is absolutely brilliant
Now we are able to show improvement going back 18 months or two years.
There has been a clear improvement for the majority of service users across all the Outcomes Star areas. It's also interesting how this varies between services and analysis of the data has raised questions that we have looked into.
The Star data has confirmed much of what we suspected and proved that our work is strong in some areas and weak in others.
For example, one of the areas in which there were least positive outcomes in the substance misuse services was in effective use of time, with 25% of clients in one service going downhill and a further 38% showing no improvement. We decided this was not acceptable. As a result, we asked service managers to think more about the link between their service and structured day services and back to work services, to be more proactive and make those links stronger to ensure service users were constructively engaged during the day.
73% of clients improved their management of their mental health
And a further 17% remained stable. However, we were initially concerned about the dip in mental health for 10% of service users.
The Star data enabled us to drill down and look at those people that had slipped backwards.
This service primarily works with people in transition and we saw from the data that those that had dipped were then readmitted to acute mental health services.
This reassured us that we were assessing people correctly and that some people coming to us needed a higher level of support than we can provide, so slipped back and were readmitted elsewhere.
Training and time is needed to ensure consistency in scoring
Early on we found that some service users had very high scores on the Star and realised that this was because some service users were not making very realistic assessments of where they were and staff were just agreeing with the service user assessment rather than entering into a discussion and sharing their own perceptions. However, training has helped considerably, as has the new version of the Outcomes Star which is much easier to use. Staff now find it much easier to understand the underlying journey of change and scoring.
It has also been important to ensure that the Star is discussed regularly within teams. Now we find that scoring on the Star is more accurate, consistent and realistic.
Our commissioners want this kind of data for all projects
We have started to share outcomes data with commissioners including supporting people, social services and the primary care trust and have had very positive responses. For example, one said they wished they had that kind of information for other services they commissioned as well. This reflects well on us as a voluntary organisation. We anticipate project level outcomes data being increasingly useful in discussion and negotiation locally.
For example we are in a process of change here in Cumbria; there will be a reduction in inpatient beds and discussions are taking place about opening more projects similar to our mental health crisis projects in other parts of the county.
We are feeding our outcomes data into these discussions.
Now we really can demonstrate that what we do works
I have always felt under some pressure to demonstrate that what we do works; we are constantly asked why someone should use our service rather than another and it has been very hard to demonstrate our effectiveness apart from using discharge information.
Now we can, plus the star data we feel comes directly from service users, which gives it more gravitas; it demonstrates where they are and the progress they have made.