Prevention and Predictive Analytics
I was at a What Works Scotland seminar this morning, the latest in our joint events with NHS Health Scotland on the Economics of Prevention. Papers and slides and a summary of discussion groups will be posted at the WWS website. We heard papers from Heather McCauley on the use of predictive analytics in New Zealand, on modelling the burden of disease by Diane Stockton and using agent-based models to consider informal care and obesity by Eric Silverman. They were followed by Ian Marr who summed up, drawing on his first-hand knowledge of social impact bonds and the social impact partnership model he has been developing.
A key aspect of preventative thinking, from Derek Wanless to Campbell Christie and beyond, is the issue of understanding where the most public service spending goes and therefore targeting spending, as far as one can, to those people and needs that will otherwise generate disproportionate public cost e.g. early year intervention to prevent what would otherwise lead to, in high likelihood, negative future outcomes such as less good education and employment outcomes, poorer health and or episodes involving the justice system. A key issue is also how to manage the disinvestment that goes with a shift to prevention.
While it was fascinating to hear Eric Silverman tall about these simulation model as safe playgrounds of policy experimentation without consequences (unlike piloting, for instance), I want to talk primarily about Heather’s exposition of preventative predictive analytics in New Zealand. She told us about the evolution of the programme, how it works and provided detail in terms of policy spheres such as welfare benefits and children in care.
The three big lessons and challenges that arose for me were as follows:
- Moving government to think and act in terms of the lifetime costs (on an actuarial basis) rather than the annual cash costs of a high need individual, household or client;
- Using statistical/econometric methods to uncover the probabilities that signify the high need households and individuals – the diagnosis of where lifetime costs are very high and therefore where large potential savings can be made; and
- Designing the optimal mix of practice and policies that allow case managers to maximise the effectiveness of intensive interventions (what works?).
All three are difficult – the third, perhaps the most challenging. Let’s look at each in a little more detail.
Heather described the need for culture change to take on the lifetime cost approach. She pointed out that New Zealand has a culture of seeking the best possible value for the public dollar and so the shift from short term to a longer, multi-parliamentary term perspective, can be made and perhaps done so more readily than in the UK or Scotland. Many of us might be comfortable with the idea of focusing on the lifetime savings made by preventing someone falling into the negative outcomes suggested above – but it does require current governments spending money now and postponing benefits to future governments. Heather provided the example of using a helicopter to transfer a spinal injuries patient from an accident site immediately to hospital with potential long term savings in reduced future health care costs. Lifetime benefits considerably outweigh upfront (helicopter usage) costs.
Second, the New Zealand benefit figures suggest that much of their employability spend goes to job seekers who are a small proportion of the total client group compared to the higher and persistent incidence of for example those on disability benefits and lone parent benefits. They cost more in lifetime terms and represent longer term need. Modelling under certain conditions offers, to different degrees in different policy areas, a reasonable basis to diagnose where highest need is concentrated and where benefits might be maximised by effective targeted interventions. But as was stressed in the presentation, these models produce probabilities and associations; they are not causal and indeed there is a fascinating question about understanding why some highly at risk groups remain resiliently unaffected in future years – what can we learn from their resilience?
Heather rightly recognises the suspicions and criticisms open to these sorts of approaches (often relating to big data and predictive algorithms): bias, non-discretionary model creating discriminatory or arbitrary outcomes, perverse incentives, moral hazard and discrimination like cream-skimming of the cheapest easiest candidates in areas like the work programme. Transparent models (all on line from the New Zealand government) and independent scrutiny of the models, their assumptions and how they work ‘under the hood’, is essential, as is always seeking to improve the model and to reduce negative aspects of models.
Finally, there is the classic what works question – assuming that the modelling has indicated who and where the highest need target group resides, what are the suite of policy tools and interventions that best reduce the lifetime cost and make those savings because negative future outcomes are significantly reduced? How do we assemble good practice, policies, and effective case management in the variety of policy areas likely to be developed? A sector by sector repository and on-going discussion about these tailored responses is essential.
Predictive analytics has well founded criticisms but as in so many areas, this is one where continued independent scrutiny, a commitment to transparency and a willingness to continuously improve modelling, can provide valuable prevention benefits but there I can be no guarantee that this will be so. Furthermore, there is the small question of then designing the appropriate mix of policy responses aimed at those in most need