King's College London

Young people not in employment, education or training (NEET) face many complex barriers to engaging with the labour market. Across the UK, there are many schemes, run by charities and local authorities, that try to address these challenges and support young people NEET into sustained, meaningful employment. These schemes have the potential to be immensely important and impactful, but they are also costly, and approaches are fragmented across providers. There is a need for more evaluation of these programmes, to understand what works and for whom, but such evaluations are often complex to set up and face a number of barriers to successful implementation.

The Evidence Development and Incubation Team (EDIT) in the Policy Institute was funded by the Youth Futures Foundation to conduct pilot evaluations of two such initiatives: Liverpool Talent Match, delivered by Merseyside Youth Association and DurhamWorks Futures, delivered by Durham County Council. These mixed-methods evaluations aimed to understand the way the programmes were run, how young people interacted with them, and whether there was evidence that they were improving outcomes. Importantly, these studies also aimed to assess whether the programmes were suitable to progress to a robust impact evaluation such as a randomised controlled trial (RCT) or quasi-experimental evaluation (QED), either separately or as a comparative exercise to understand different models of support for young people NEET.

The reports of these two studies have now been published and can be found here.

Overall, we found that:

• For both programmes, there were associations between level of engagement a young person had in the programme and higher levels of a range of outcomes including uptake of EET opportunities, self-esteem, and wellbeing.
• Qualitative work suggested that for both programmes positive outcomes would likely be driven by the personalised and holistic approach of the intervention, which could help overcome barriers such as lack of motivation, family breakdown and bereavement, disruptive peer groups, and poor mental health.
• Both programmes were well-understood by those delivering them and were largely delivered as intended.

From a methodological and delivery capability standpoint, robust impact evaluations were possible for both programmes; however, for both, the optimal design would have been an RCT, and both organisations expressed concerns with this approach due to the necessity of withholding the programme from participants allocated to the control group. For DurhamWorks Futures, we were not able to identify a suitably robust alternative approach, while for Liverpool Talent Match, we believed it would be possible to conduct a quasi-experimental design called a matched difference-in-differences. For various reasons, YFF determined not to take these evaluations forward.

However, these studies demonstrate the value of pilot and feasibility studies in their own right. Through our fieldwork and analysis, we were able to provide useful insights to the two programmes to understand the mechanisms by which they may achieve change, and to inform their own process improvement. In addition, it is appropriate that some pilot studies do not progress further; if all feasibility studies progressed to full impact evaluations, it would suggest that the feasibility studies themselves were unnecessary or pointless. Instead, we were able to provide YFF with good information about their options, on the basis of which they determined that further evaluation was not suitable, and the funding that would have gone to these evaluations could be directed somewhere else. Finally, the development work we conducted has helped YFF to refine its approach to scoping and designing impact evaluations in youth employment, shedding light on some of the challenges and opportunities that are faced. This helps set YFF, and EDIT, up for success in future projects of this nature.