Evaluation Exchange: Supporting Vulnerable Residents in Diverse Communities

The Evaluation Exchange is a partnership between UCL and voluntary and community sector organisations in Camden and East London that aims to improve evaluation practice. This blog reflects on the work done by the Subco Trust for vulnerable adults in Newham.

With experience of providing culturally and linguistically appropriate services for vulnerable Asians elders and carers in Newham since 1993, Subco Trust were able to recognise an unmet need in the community created by the COVID-19 pandemic. In Autumn 2021, they launched a Mental Wellbeing project to enable people to develop coping strategies, including support groups covering bereavement, reducing stress, and healthy living.

As with any new project, the Wellbeing project needs evaluation: to learn what service users have liked and what can be improved; to meet the reporting needs of funders; and to provide evidence for the next steps. Our team of researchers are working closely with Subco Trust to develop tools which can be used to measure two of the key outcomes: changes in mood and well-being; and service user empowerment. We know that our evaluation approach needs to be sensitive to the diverse communication needs of Asian individuals living in Newham. As part of this, we are testing short forms, which can be easily translated or completed with the support of staff, and include icons as well as text. 

Capitalising on the digital inclusion workshops being run as part of the Mental Wellbeing project, evaluation data will be collected via tablet during group sessions. This aims to meet the monitoring and evaluation needs of the organisation, by reducing the administrative burden on staff, and making it easier to analyse and present the data. When considering how we measure other outcomes of the project, such as satisfaction with the service, we have learnt that service users are happy to share their experiences and feedback in conversation with staff. We are thinking about how Subco can build an evaluation framework that allows them to easily collate and reflect on this rich and complex information, alongside more traditional feedback.

Working within COVID-19 restrictions also raises interesting challenges. Unable to observe the services first-hand, we have relied on the valuable insight of Taskin from Subco Trust into the work that the organisation does, and the needs of their services users and staff. We liked the idea of giving service users a chance to reflect on previous activities using photos and videos as a prompt; but how would you do this online with uncertain digital access, as well as maintain a natural flow of conversation? There are no easy answers, but we will continue to think creatively about it.
 

Looking to the future, we want this to be sustainable. One outcome that we identified was in supporting staff with the skills needed to be part of ongoing monitoring and evaluation. We hope that the work we do together on the Mental Wellbeing project can be used as a template by Subco Trust in the future to streamline the support evaluation and reporting on other programmes of work.

The team: Taskin, Chief Executive of Subco Trust; Giulia, PhD student in Mental Health Science; Sethu, Senior Lecturer in Nursing and currently undertaking an Educational Doctorate; Anthony, PhD student at the Social Research Institute; and Mel, Post Doctoral Researcher at the Institute of Health Informatics.

Find out more about the Evaluation Exchange

---