Podcast Episode: Risk assessment and risk management in youth justice
Category: Criminal justice
What follows is a transcription of the audio recording. Due to differences between spoken and written English, the transcript may contain quirks of grammar and syntax.
LJ - Lorraine Johnstone
DO - David Orr
SA - Stuart Allardyce
NV - Nina Vaswan
MM - Mark McSherry
On 9th December 2013, Mark McSherry, Head of Development with the Risk Management Authority, hosted a discussion at the Centre for Youth and Criminal Justice about risk assessment and risk management in Youth Justice. Taking part, Lorraine Johnstone, Consultant, Clinical and Forensic Psychologist, David Orr, a Social Worker with Edinburgh Youth Offending Service, currently seconded to the Centre for Youth and Criminal Justice, Stuart Allardyce, also a Social Worker and seconded from Barnardo’s to the Centre for Youth and Criminal Justice, and Nina Vaswani, Research Fellow at the Centre for Youth and Criminal Justice.
The focus for today’s discussion is risk assessment and risk management in relation to youth and youth justice.
LJ My name is Lorraine Johnstone, I am a consultant clinical and forensic psychologist. I have a long standing interest in risk assessment in general terms across mental health offending and young people.
DO My name is David Orr, I am a Social Worker with Edinburgh Youth Offending Service, but I am currently seconded to the Centre for Youth and Criminal Justice, where my role for Chris is particularly in managing the high risk agenda and particularly youth violence, and my interest in the field is kind of long standing from working with young people who present with significant violent behaviour and how they offend, being managed both by myself and others through different systems, Children’s Hearing System and Adult Criminal Justice System, in trying to establish what works and what doesn’t.
SA My name is Stuart Allardyce, I am a Social Worker as well, I am seconded from Barnardo’s to the Centre for Youth and Criminal Justice, and I suppose one of my long standing practice interests is working with young people with sexually harmful and sexual offending behaviour, including risk assessment, it’s a kind of core principle of that.
NV My name is Nina Vaswani, I am a Research Fellow at the Centre for Youth and Criminal Justice, and one of my research interests has been about risk assessment with children and young people and how professionals apply their own decision making to that risk assessment.
MM My name is Mark McSherry, I am the Head of Development Authority and my role today really is to focus on having a lively and stimulating debate in relation to the topics that are under discussion. So to set the context of today’s discussion, I would be interested in exploring why the risk assessment is such a significant issue in both youth and criminal justice and maybe just ask yourself, Stuart, you could present your argument in relation to that position.
SA Absolutely, I mean clearly risk assessment is something that is critically important in a number of different fields, not just in social work and criminal justice, but also in politics and business and indeed, all aspects of social care, and we could maybe speculate for about what some of the social worker drivers are that kind of underpin that, but I think also there’s a specific narrative that some people might already be aware of that’s very important for Youth and Criminal Justice, and that’s a narrative that kind of really goes back to the 1970’s, back in 1974, a Sociologist called Robert Martinson produced a very important paper called “What Works?’ with a question mark after it, and in that paper he argued actually there was very little evidence of anything being particularly affective in working with either young people or adults that are involved with offending behaviour. And on the back of that paper, which actually really struck a cord at that time, and was widely circulated and had a real kind of resonance, and number of academics and practitioners started doing what they could to build an evidence base about what is affective in working with young people and adults involved with offending behaviour, and some of the most interesting work that was done in North America, particularly by people like Don Andrews, involved meta analysis, where a number of different studies to show what’s seen to be effective in working with young people and adults involved with offending. And one of the things that they found was that services that took on board risk and used that in their work seemed to be more effective. So services that provided some kind of risk assessment and then targeted most high levels of resources to kind of high tariff cases, and then provided less resources to lower tariff cases tended to be more effective, and this is what we call the Risks Needs Paradigm, which has become, I suppose a core principle in what we do in working with both young people and adults. So the idea of risk assessment, really from the 1980’s onwards, has been a real fundamental aspect of our work. And on the back of that, a number of different risk assessment tools have been developed for use by practitioners in both the adult and adolescent field, so I think that’s probably one of the kind of key reasons.
MM Thank you Stuart, and I am conscious, David, that you have some views in relation to the progress on the risk assessment instruments that have been mentioned by your colleague.
DO I mean I think what Stuart’s highlighted feeds quite well into the development of the risk assessment tools, it kind of maps that debate the way that the sociological processes have shaped things, so to go back to Martinson, I mean Martinson tragically killed himself in 1990’s, you know something along those lines, you know threw himself off a building, having been I guess disillusioned at the manner on which his research had been interpreted and the kind of co-opting of the Northern Rocks agenda by the political right, particularly in the States, so you have a system whereby nothing works, therefore let’s lock up individuals, where I belief in rehabilitation is essentially out the window. Yes, and Stuart mentioned what works and when it appeared, initially it was “What Works’ with a question mark, because people were uncertain and the media analysis of the areas of different academics fed into kind of the core principles about managing risk and impact dosage, high risk, high levels and intervention, the lower risk, lower levels of intervention, so I suppose one of the things that’s interesting is that over time the question marks would have disappeared, so it went from “What Works?’ to “This is What Works’ and that becoming almost accepted tablets of stone. So with respect to the risk assessment tools in Youth and Criminal Justice, Social Work and so forth, you kind of start off with first generation people talk about, which I guess is the clinical judgement on its own and you think Freud, come into my room, lie down, let’s talk, let’s hear about your stories, I’ll then come to a position on what’s contributed to your current position. Understandably that was open to criticism as being insufficiently ligneous scientifically, so you have a response against that and then the shift of the kind of actuarial risk assessment methods coming anywhere from insurance brokers to military strategy planning working out the questions you need to ask, basing that on all sort of statistical and group evidence bases. So we know certain factors contribute to risk, or to offending, therefore are they there or are they not, and if not, you know we are not concerned about them. So that’s a crude kind of summary of actuarial methods. The third generation then begins to become a little bit more sophisticated, taking both static and dynamic risk factors, static being essentially those that are not open to change, number of previous offences or has committed a previous offence and the dynamic factors from substance misuse, alcohol consumption and also patterns of thinking and behaviour, and then finally and then it’s easy to lose track of generations, I think we are up to 4 now, but those that combine essentially clinical and actuarial, static and dynamic risk factors, but also pull in some kind of risk management planning as part of that so that it’s not a one off discreet process, you know managing difficult behaviour should be part of a long standing plan where progress can be assessed, changes can be identified. So I guess that’s maybe how the history and the sociology sits with what it looks like in the frontline, but I am hoping to be corrected.
MM Thanks very much, David. And currently in Scotland in 2013, in relation to that progress between first generation instruments up to fourth generation instruments, before we begin to look at some of the significant critiques both of the risk paradigm and also the resistance paradigm, what would the view of the group in relation to where we currently are in relation to the application of the different generations of instruments in the group, I am open to anyone to take a response on that.
LJ I think it’s quite an interesting question because I think different systems operate different models of methodologies at the minute, so if you were to look across adult forensic mental health, I think you would see almost exclusively the application of the structure of professional judgement paradigm, whereas I think Criminal Justice Agencies, Police and Social Work still use actuarial or adjusted actuarial, but are moving towards trying to, I think, fit that into a broader paradigm, so I think it’s quite an interesting picture nationally where you do have systems who should always work together in a multi agency way, but a somewhat different approach. But I do think it is important to acknowledge and recognise that we do have the Risk Management Authority in Scotland and certainly it is recognised internationally as a guiding light for us, whereby we can benefit from expert knowledge and people trying to tell us what we should be doing, when … you know with what populations, so I think we are very fortune and certainly compared to other jurisdictions across the world, whereby there is a very open, explicit and visible attempt to try and take us forward into best practice, irrespective of the agency that we are in.
DO I mean I suppose I could kind of summarise where things are at from a youth justice perspective in terms of the actual training that people undertake and the kind of core work that they deliver which tends to be people being trained in the use of risk of reoffending assessments, so … you know as in YLSCMI looking at risk of violence during the structural assessment of violence risk in young people, and then into sexually harmful behaviours with the aim of the risk assessment tool and some people using the Juvenile Sex Offender Assessment Protocol. I think the more interesting question for me is what best meets the needs of young people, the critical few, the high risk individuals who present serious challenges to systems, risks to themselves and to others and also unless a young person you work with has some kind of understanding of what you mean when you talk about risk, you know to sit in a meeting room with a young person who is awaiting a judgement in the Sheriff’s Court for a serious violent offence to say, I reckon you are a high risk and we will need to plan accordingly, it’s absolutely meaningless, so how do you make those systems and instruments come to life so that they have value in keeping people safe, the young person included. I think that there’s been a movement, probably over the last number of years, whereby what Lorraine has been talking about, about structured professional judgement and some of the influences of psychology and some of the really positive aspects of that filtering over into social work, I also think that the 2 professions continue to have differing roles and one of the challenges that we face at the minute is, as a social worker, how do you make some of those principles come alive in the meat and 2 veg of your work, the reports for children’s hearing systems, the reports for the courts, because sometimes there’s a wee bit of limit to how much one can innovate in the context of fairly structured reports. So that would be my take on things, rather than that.
MM We know from some of the literature, particularly in Loan Camp, going back to your point earlier on, Stuart, in relation to the implementation of R&R, approaches that some of the critiques of the use of risk instruments or the risk needs of R&R quite often are focused on the degree to which implementation has been with integrity, I am just wondering in the Youth Justice context, what your views would be in relation to how we have implemented national initiatives or work in relation to risk instruments and how that may support the continued professional development?
SA Yes, I think something that’s interesting at the moment is, as David has already said, there are 2 mainstream risk assessment tools for Youth offending that are generally used in social work settings in Scotland, YLSCMI and Asset, and what’s interesting is that both of those assessment tools are going through a process of redevelopment to the model, so there’s a new version of YLS that’s currently available, and one of the interesting things about the new development in YLS, is it has some differential scoring for both males and females, so there’s a thinking within that tool now and by the authors of that tool about actually did this assessment tool overestimate risk for young women and do we need to actually do something different in relation to this? So we see a, if you like a refinement of the actuarial approach with respect to YLSCMI. With Asset, although we don’t have a new version of Asset available yet, the Youth Justice Board within Wales is currently working on what’s called “Asset Plus’, and for what we know about Asset Plus, it seems to be the case that some of the criticisms of the risk needs paradigm that have been there on the academic literature, and in particular some of the stuff that’s been written around desistance, what the processes are that come into play when young people move on from offending behaviour are increasingly being recognised in the development of that tool, so one of the things that seems to be there with Asset Plus is that it seems to be being developed as something that looks much more like a structural professional judgement tool, which means that there’s a high level of formulation required in all of that, and although there are certain aspects that will be scored, you are not asked to put together a final scoring in relation to the young person that would tell you something about risk, actually you are invited to begin to develop a narrative about why the young person committed the offence in that particular context that they were in, and what might that tell us about the future with respect to all the other things that we know about the young person and the environment that they are in. So you can see where we are at the moment, which is kind of at that stage of third and fourth generation actuarial orientated tools, but there were different paths out of this at the moment, some of which are about kind of refined actuarial methods, but some of which are about actually taking us in a much more structured professional judgement to make a decision.
DO I think just, to pick up on how good are we at assessing … how good we are at assessing, I don’t think that we necessarily have enough information at present saying, actually this is a cohort of the critical few for the last 10 years who we have caught up with a result of longitudinal survey data, looking at what has happened in the 10 years since those assessments and the outcomes for them. So it’s difficult to over-acclaim for how good or not practitioners are in the use of these tools, I mean I think it’s ultimately, like I said, about the relationships built with the young people who assessments are being undertaken for because in the absence of that kind of interaction and dynamic, a lot of the practical completion of assessments and so forth is just that, it’s just an assessment exercise, there’s got to be an investment from the worker, from the family around the young person, from the young person themselves, from all of the different professionals in order to kind of, to manage risk.
LJ I think it does trigger a broader question, those actually how you evaluate the success of the risk assessment, I think your question is about integrity monitoring and I think in a sense it’s very easy to monitor the integrity of a risk factor rating because it’s something … it’s a construct that’s defined, you are taught how to rate it, taught how to find it, so in a way that that can be a fairly straightforward thing to do, but I think it is important to understand … that doesn’t necessarily mean that the risk assessment is doing it’s job, it just means that people are rating a construct in the same way, so I think it’s maybe a bit further on than this discussion but I think it is a really important thing for us to think about how we actually assess the impact of our assessments, and certainly from my own experience, I would argue that the impact is that you are more confident in assessing and managing your risk, you feel like you understand the person, it doesn’t necessarily mean you will negate that risk, you will just manage it effectively, for some people, the critical few, and then maybe for other people you will be able to put them into the correct programmes and treatments. So I think, for me, there are a lot of methodological flaws, both in a construct and the evaluation of risk assessment practice per se, and I think in a broader .. I suppose a broader process, I think one of the reasons we are here today is there is very much importation of adult models, there’s very much importation of scientific methodologies to look at how well we are doing something without a fully developmentally informed or a methodologically sophisticated enough approach. So I think it is beyond the scope of this discussion, but I think how we actually … how will we ever know that our risk assessments are doing the right thing, because on your predictor and you say someone is really high risk, it’s not a satisfying outcome, per se to say, well you are a high risk …
NV I think that’s a really important point and I think there are kind of limited research out there anyway, you talked about importing things, we also import lots of things from North America and there’s very little research about risk assessment in Scotland or in the UK, and also in the kind of real world situation, a lot of research is undertaken in artificial conditions or retrospectively, so there’s a lot of things that we don’t know I think about how people make decisions about what’s given.
DO And I suppose, I mean to go back to what works and the evidence about programme interventions working effectively and programme integrity and fidelity from one to the next, whenever the individuals who wrote the programme, or the individuals who delivered the programme, outcomes tended to be better than whenever the individuals who wrote the programme taught others and others administered it, and risk assessments likewise, there’s a huge business certainly in risk assessment for better or for worse and there are individuals who have an investment in a product which is a factor and there is research indicating that outcomes will be, some of the publications are more likely whenever it’s relating to risk assessments that are opened and created by a series of high profile bit hitters, whether that be in more actuarial fields or more clinical fields, so my view is there is no perfect model, whether it be a clinical, actuarial, third, fourth, firth generation if there is going to be one and you have to be a critical consumer of those products.
LJ I think it is important, I mean I wondered just for the benefit of the listen around, mindful that we sort of went into quite complex constructs and defining instruments and methodologies, but I think some people won’t know what actuarial tools actually are, and I think it is important to emphasise that generally they are designed from robust research, they are designed from a sample or a population where commonalities, common risk factors are extracted, put into a scale and they are weighted to give some sort of indication of the likelihood of an event, and I think for some, back to Stuart’s earlier point, one of the other reasons I think for risk assessment, per se, is about offender rehabilitation but it’s also to help courts, it’s to help other people make very, very serious decisions, where actually the question is, are they more likely than not to do something bad again, and these tools provide a way or have been marketed and promoted as providing a way to give an estimate whether it’s above or below chance, and I think it’s naive to think we don’t live in a … you know, we are in a political climate, we are in an economic climate, decisions need to be made, they need to be made in a time efficient way and where we have got something that looks like it’s developed in the research that does that job then that is an appealing process, but I think the question we are asking, “where are we just now’, I think what we are realising is that there isn’t a panacea out there, risk assessment is extremely complex, absolutely whichever tool you choose to use, whichever research base you choose to form your opinion and I think one thing we are doing here, nationally, is trying to embrace that complexity, have much more open dialogue about what we should do, when, with whom, and I think sometimes it’s very polarised and I think there is a commercial element to it, but I think there is also particularly with youth, a more honest acceptance that actually we need to be doing a wee bit better than we are, it’s not simply importing models, it’s not looking at the research uncritically, and I think that’s really important to emphasise.
MM I think Lorraine’s point is really important because if you look at the literature around risk assessment in Youth and Criminal Justice, actually most of the studies around risk assessment tools have been about their predicted validity, do they tell us something meaningful about whether people are going to commit an offence or not in the future, and clearly that’s an extremely important question, and if risk assessment tools don’t help us with that then actually they don’t get us into the ball park, but there are lots of other things that are important that we also need to evaluate. We also need to know whether the risk assessment tool will help inform what we do with the young person, actually what kind of intervention would be helpful, we need to have risk assessment tools that can help us and make decisions about risk management, actually around particular things like safety planning that needs to be in place, are they wide enough in their range to help us understand the impact of families on young people and allow us to provide the right kinds of intervention for families, and I mean wide enough so that we will be holistic enough to be aware of other risks that a young person might be subject to in various way. A young person might be at risk of child sex exploitation for instance or they might be at risk in terms of issues from self harm or suicidal behaviour … if we do a risk assessment that is just about offending behaviour and doesn’t capture some of those other issues that might need to be attended to in some way. So we need to have a kind of broad concept of risk assessment in all of that, and also where’s the research in terms of what the experience is of young people and indeed families, in terms of risk assessment. So there are lots of things that are lacking but we are beginning to see a bit of literature about some of these things that opens out that question of “actually, do these tools work or not?'
DO It doesn’t take you to have read that many articles in this field before you come across Neil’s quote about predicting is very difficult, especially about the future which kind of captures, I think, the challenge, it’s an inexact science and the points that Stuart has made about the big picture of where a young person sits in their family, their community, their society is a really big one, but equally big is what’s the likelihood of this young person going out and causing serious harm to themselves or others, so it’s finding a balance between a whole range of competing priorities, which I guess is what the nature of this work is.
SA A few of you have mentioned the lack of evaluation or the importation of evaluation from abroad, particularly North America, I am conscious that there are some individual and local pieces of work that may challenge that in terms of what is being evaluated in relation to the application risk instruments in Scotland, Nina, I know you were involved in evaluating the use of YLSCMI, or …
NV yes, I was
SA … in Scotland and I am just wondering what learning, from a practitioner perspective, there was from that particular study?
NV I think the big thing, I think for me about learning from it was that actually we don’t know enough yet about how people do handle risk assessment instruments and how they do make decisions. I mean my study looked at over 1000 YLSCMI’s conducted over a 2 year period and they were undertaken as part of standard social work practice, they weren’t done for the research, so there’s lots of flaws in that, in that the quality of the assessments weren’t adjusted or monitored, but it’s also quite helpful in that it does actually give us an idea of how risk assessment does work in practice under kind of typical social work conditions. I think in terms of the YLSCMI in this context, I think it was quite a fair role in that it did find that it was quite a good predictor of the future recidivism, but I do think it raised quite a number of other questions that we didn’t really explore in that research and we haven’t yet got to the bottom of, about how the professional older age function, that is in the YLSCMI, which allows the practitioner to adjust the actuarial output to take into account context, take into account other factors and I think that really kind of muddied the waters somewhat in that we don’t really know the best way in which practitioners can do that, at least in this particular instance anyway.
MM I am conscious of something from Jackie, because that’s work she is particularly interested in relation to whether or not professional overrides should be allowed in relation to the application of particular actuarial instruments, and I wonder, Lorraine, from your perspective what your critique would be of that particular argument.
LJ I think there are lots of different challenges for the actuarial approach per se, but just on that specific question, I think where you engage in the process of professional override, you are essentially dismissing what otherwise the evidence base is telling you to do, so therefore you have to be extremely confident, extremely well read, have a very sound rationale for doing that to ensure that you don’t fall into the trap of either some sort of ratter by us, whether that’s a ceiling effect or a floor effect if you like, and I think what we don’t know and certainly in some of the judgement research, is that we actually don’t know how we make decisions very well as humans, you know there’s all sorts of heuristics which we are aware of and some of which we are not, that will influence whether or not we will decide, you know, make a particular decision. So I think it’s a very … personally it’s something that I find quite a difficult concept to get my head around, because if your argument is you are engaging an evidence based practice and you are basing an opinion and you calculate a score on what most people who are the same way, behaved like, and then you override that, then where is your evidence, how robust is that, how defensible is that, what have you used, is it just that you have seen someone like this person before and they didn’t do it, is that a sound rationale? So I think it falls within the same scope of criticisms as the unstructured professional judgement approach, it lacks transparency, there isn’t a systematic way of demonstrating what you have done, why you have done it and how you have done it, and I think essentially too, I would imagine it’s a very difficult thing to do, particularly if you have got someone where there is a high risk, so if your actuarial score is saying high risk and then you as a professional say, well actually I don’t think they are, I think they are low risk and then they go off and offend, you know, who takes the responsibility for that? So I think it’s a lot to ask practitioners to do, professionally as well as give them a defensible structure to do that.
NV I think I would probably agree with you there, I mean I think practitioners do bring a lot of wisdom, a lot of experience, a lot of professional expertise, but they also bring with them a value base, a history, a certain way of thinking about the heuristics, so I don’t think you would want to take that information away, people need that opportunity to apply their kind of knowledge, but it’s a bit more I think how do they do that and what circumstances does that work and certainly in my research, in this case the actuarial output was actually the most accurate predictor of future reoffending followed by the kind of risk categories and then followed by the professional override category, which was the kind of lowest predictor, so I think while you wouldn’t want to take any decision making away from practitioners, I think we could all do with better understanding on how people go about that and I think it does leave people open to quite a lot of risk to themselves, although in my research I think as well, the actuarial output and the professional judgement did agree in most of the situations and whether that is something that is about confidence or lack of confidence in overriding things or that there was that agreement, and certainly it was in almost 90% of actuarial outputs and professional judgements seemed to be the same.
MM Nina, can I just ask, when professional override was done in the recent tool, did that improve outcomes or did it get in the way of outcomes in the study, did it improve the capacity of the workers involved with the case being able to predict future offending behaviour or did it go in the other direction?
NV In this particular study it did reduce the accuracy of the output, in some circumstances it reduced it to chance levels, whereas it had been quite a big predictor. In terms of outcomes, it is difficult to say, as Lorraine mentioned earlier it’s kind of what you do with that risk assessment that is getting influenced, so you don’t know, they didn’t look at it and study about what services they got, what other factors in their life have changed since that risk assessment happened, so we are just going on kind of real world scenario. And other studies have equally found that that kind of adjusted actuarial output generally tends to weaken the predicted accuracy of these tools, but as I said, I think if we continue to develop a knowledge and understanding of how people make those decisions and they could be made with the best of intentions, if resources are allocated on whether people are high risk or low risk or moderate risk, that sort of thing is going to influence peoples decision making, whether it’s conscious or not, so I think we need to better understand why and how and when people should apply that judgement.
DO I am kind of surprised by the development of the override because it does seem to fly in the face a bit of a real commitment to evidence based statistically derived instruments, it’s almost like there’s a kind of have your cake and eat it, but I get the bit that they are responding to critics saying you know, this is too crude, we need to have something that captures the complexity of human life and so forth, but actually when you throw something like that in, it doesn’t improve things based on the statistical evidence. I think that it reflects the, probably the pressures coming from practice in part where people are saying sometimes this is the “score”, but it just doesn’t feel right, and that goes back probably to what Lorraine said about that is probably quite risky because it is getting back into unstructured professional judgement, so I would be interested to see more research in that are, I am not saying it’s the wrong thing to have, I think it’s an interesting area, but we probably don’t have … in the UK anyway, your kind of sufficient evidence based to be saying that it’s a good thing or a bad thing specifically.
LJ I wonder as well though, I mean I think the structured professional judgement paradigm offers a very good methodology and a very honest methodology towards risk assessment, because it does say it is actually extremely complex, there’s a minimum set of risk factors that you consider, but if you, for example, see something odd or unusual or it’s very low base rate, or there’s a particular form of violence and you are required, as an assessor, to consider that as well in the context of the bigger picture, so you are almost building on the existing evidence base, whereas I think the override principle is almost dismissing the evidence base. So I do think there are important differences there and I think the construction of the structured professional judgement paradigm trying to keep what was good about the actuarial approach in as much as it’s research based is evidence based, you know, in general terms these factors have good utility, however human beings are very complex, sometimes very unusual, sometimes present with behaviours that will never be in high enough frequency to sit a Met analysis, but as a practitioner you should recognise this, so I think that approach is a much more sensible approach as to looking at the override, because I don’t know that you can ever … I would really struggle to think about a methodology that would capture all the variables that went into an override decision, whereas I think the structured professional judgement paradigm has tried to account for that in a much more individualised way, so it’s that professional addition as opposed to instead of what the evidence us. But I think it is important to acknowledge that there are quite polarised views about methodologies that can be quite a heated topic and debate and I think it’s important as well to acknowledge that all of these, when you look at research, research does look at predicted validity and utility, however it talks in averages and actually the average of finding there is somebody that you’ll have seen before, you know the story is quite familiar, the cases are extremely taxing and challenging and your unusual ones as well. So I think there’s something much more need around the risk specificity and risk sensitivity and complex co-morbid risks, and I think in a way that some of our activity should be directed in that direction.
MM Could you maybe explain a wee bit more, I am conscious our listeners might not understand some of the terms that …
LJ Well I suppose thinking about … often we take a risk assessment tool and we say let’s administer this risk assessment tool, so even if it’s for example, sexual offending. Sexual offending can be anything from touching someone inappropriately in their arm or their back or their buttocks, all the way through to a sexual homicide, you know it’s a whole range of things, and we will look at risk assessment protocols and we will have a set of risk factors that are relevant to outcome, but in something like indecent exposure for example, is a high frequency offence, whereas sexual homicide is a very low frequency event, so the risk sensitivity and risk specificity effects if you like are how sensitive are the risk factors to each different outcome, and we don’t really have … you know, we have general risk factors and themes and areas that we know are relevant or we think are relevant, but we don’t really very, very robust research that says there’s a definite cause and effect between variables and outcomes, you know Stuart’s point about formulation earlier, as practitioners we will take a very holistic view, but empirically there might not be a statistical association between some of the risk factors, so I think the only way really to do a good risk assessment is to acknowledge and work and be honest about the complexity, work with the literature as it stands just now but keep driving forward a research agenda that helps us answer some of this.
MM Considering what helps practitioners, I am conscious Stuart, that you mentioned earlier on in terms of that individual approach in Scotland that we would have to have been asleep over the last 5 to 10 years and either Youth and Criminal Justice not to have been involved in some of the debates and discussions about the risk paradigm and the desistance paradigm. What would you say, Stuart, are some of the significant critiques of the desistance paradigm?
SA The critiques of the desistance paradigm? Well I mean I suppose I would say, I mean the desistance paradigm which really looks at the factors that seem to be there or for individuals who move on from offending behaviour which often capture some of those factors around human and social capital, by which I mean actually how integrated are people into communities, how embedded are they in the kind of social systems and structures who have them, do we create systems that mean that young people that are involved with offending behaviour are discriminated against or excluded from social opportunity or do we provide them with opportunities to keep on developing in a healthy way. And also the other kind of key thing that comes from the desistance literature is the importance of identity, so I suppose this notion that we get from a vast range of literature now that connotes that people who move on from offending behaviour will often look back at the period of time in their life where they were involved with the offending behaviour and they will talk about themselves as … you know that’s how I was then, I am very different now, you know I am a person who now has a job, I am a person who now has a family, you know there are things that have happened in terms of development and change that have actually allowed some kind of change in identity for the person to take place. Now risk assessment tools as we have don’t really capture the issues around identity, don’t really capture these issues around kind of social ownership of issues around offending behaviour, but one of the critiques I think that’s sometimes made by practitioners is actually although this is really important, actually how do we apply it and use it in practice? So we may we know that for instance a young person who is involved with sexual offending behaviour might have lots of issues around social skills, relationship skills, little social opportunities, they may be quite socially isolated, they maybe don’t have an emotional confidant in their life, these may be some of the factors that might explain why they have gravitated towards a behaviour of sexual offending in a sexual harmful nature. And clearly we can do some work around kind of social skills, but do we begin to set up dating agencies for young people, you know I suspect we don’t, but actually some of the things that probably need to change for young people are beyond the kind of scope and range of what we do as practitioners, they’re actually bigger social issues that need to be addressed. They might be issues around how included marginalised people can be in our societies, so I think practitioners can see the kind of conceptual ideas that are in the desistance literature but they can sometimes be a little bit frustrated about, how do we actually use that in practice?
DO To expand on that point, I want to look at Barry’s work in this area, I think it’s been really insightful, with the emphasis on kind of the responsibilisation of young people, rational chose theory, the idea that the way in which we conceptualise offending behaviour is ultimately about individuals making choices based on the weighing up of gains and losses and what is not captured is effectively the structural argument, you know and I don’t think any risk assessment tools or instruments that we are utilising address that satisfactorily, you know the unspoken reality of a lot of our social work, of a lot of psychology is that not only are we working with a critical few in terms of behaviours, but we are also working with individuals whose experience of exclusion, social and economic deprivation, is not representative of anywhere near the whole or the average experience, and I think that with desistance literature is very positive in terms of drawing attention to that. Talking about a belief in redemption scripts, you know Shadd Maruna stuff about a different life and a different narrative … I also think it’s limited in terms of what it’s called, desistance, because it’s desistance from what, it’s desistance from offending, therefore individuals continue to be defined by their offending behaviour, so that may be one to look at in the future. In terms of practical application, I think one of the big areas where there has been a big attraction is looking at the consultation and the rehabilitation of offenders at the minute, you know, a big bit of the desistance literature is focused on how damaging it is not having access to employment opportunities as a result of previous offending and that’s led to a kind of a policy emphasis on revisiting what are the kinds of offences that should exclude you from work, if any, what are the comparators in Europe and internationally about how individuals and purpatory offences are committed and what are the implications of that for their future lives in building pro social lives, full of protective factors and social opportunities. So it has it’s strengths and weaknesses, I think, but for me how we capture the social experience of young people and how we deliver both individually, professionally and as communities and society, how do we address those needs more effectively, because I don’t think we are doing that, there’s plenty of room for improvement.
MM I think we capture that really well because there is that tension between desistance from offending behaviour, but actually what are people desisting to and if they are desisting to a life where actually there are few social opportunities or poor social opportunities, then actually how helpful is that, so I think there’s a real kind of conflict there.
LJ I think there’s something really important to think about took about almost some of our assessments have a paradoxical affect that, or an adolescent will present with a problematic behaviour, then there’s a very large intensive resource around them to confirm yes, that they did present with an adolescent behaviour and then we will delineate all of their problems and difficulties, and almost sometimes press a pause button on their development while we get an understanding, and I always think for lots of people this won’t apply but for some I am sure it does, that the adolescent, or the age crime curve is 15 to 17, a little bit older now, but there is a natural desistance and I do worry sometimes that this anxiety that we have and almost enthusiasm we have to try and help, we actually hinder natural processes and natural resolution and strengths building, and I don’t know that we fully understand that. Either we are a very problem focused risk averse nation, society and I think in certainly in some cases that’s justifiable, but I do have lots of anxieties about how much we interfere with adolescent development, in particular where we often … I can think of assessments where it’s led to people being detained or whatever and they are really prevented from the relationship building, the relationships with their family are impeded, they are not going to get employment because they are not getting the skills, and I do worry sometimes about the perpetuation of actually the very thing that we set out to try and avoid. So I think it’s challenging.
MM Yes, interestingly in terms of the literature in Scotland to risk management, Asset are involved with all 32 local authorities in examining the profile of the 13,000 individuals who have had a full LSCMI undertaking, and you look at the proportion of those who are on that system, over 50% are over 30+, which turns upside down some of the evidence we know from historically … I suppose as a practitioner, trying to understand the critiques of those who would put forward the actuarial approaches are over descriptive and not individualised, those who would criticise desistance for being woolly and unfocused, not getting particular clarity in relation to methods, as a practitioner or listener to this Iriss.fm debate, what guidance can we offer in relation to the integration or the relationship between those models in Scotland and what work has been taken forward?
LJ As a psychologist, so I suppose I should declare that, I think for me the bridging gap between theory and practice is always formulation and I think that that is really where we need to get to and the formulation is understanding the evidence, the research but actually tying it into the individual and making sense of their individual circumstances. And I think actually it’s a fairly straight forward bridge to build between practice in terms of theory and the practice, because typically most people have that knowledge base anyway, it’s just a different way of expressing it, it’s not numbers any more, I think as Stuart said earlier, it’s a narrative, it’s multi theoretical, it’s multi systems and it’s highly individualised, and I think also that the beauty of that is that there is no limitations into the theoretical models that you can draw from, so it can be much more case specific, so that would be my sort of suggestion.
NV I think I would just be keen that practitioners are aware of looking at the benefits and limitations to details that they use and that’s not to suggest that they have to conduct their own mental analysis or have a degree in statistics, but there are benefits and drawbacks to any of the tools and to any of the social, actuarial approach or adjust actuarial or professional judgement, and I think it’s just important to be aware of what you are choosing and why you are choosing it and what the benefits and the limitations are of that, particularly how we report that risk and how we kind of manage that risk as well, that’s what I would think.
SA Yes, I would probably say the same as some of the things that Lorraine was saying, that we sometimes get into this habit in meeting criminal justice or thinking that our assessment is the useful assessment tool, and it’s not, an assessment tool is something that can contribute to the process that is in assessment, but it is something that is just an element of it, because actually an assessment takes place in the context of a relationship or a set of relationships, it’s an encounter that takes place, so I think we need to move to a much more nuanced understanding of what assessment is, which actually to be honest, speaking as a social worker, I think in some ways we’ve always had as a kind of fundamental of good assessment practice, it’s always been something that’s been individualised, that’s been holistic in nature, that’s been ecologically orientated, these are the kind of principles of assessment in social work that we all subscribe to, and clearly assessment tools, when we are working with young people and adults who are involved with offending behaviour, can help us address specific areas and needs that might underpin the offending behaviour and help us think about what would be most useful for the individual to help that desistance process, but they are just one aspect of what we do, I think that’s the important bit.
DO Quite often when I hear practitioners talk about research base, one of the criticisms that’s levied is “I don’t have time to read that research’ or the research is presented in such a way that it’s unfeasible for me to spend that time … I mean fundamentally that is our reason for being, you know, I mean if we are not making it easier in some way for practitioners to do their job in an informed way, then there’s no point in us being here … and I think, you know, I am not going to go into the ins and outs of our action plan, what we have done or not done this year, but engaging with practitioners, learning from their experiences and undertaking things like risk formulation workshops that we did over the last number of months has proven valuable. I mean on the point about where do practitioners need to be in this debate … I think there’s something about … where people are in their careers as well in that, if you are a student social worker in your first criminal justice … youth justice placement, you know, that initial encounter when you walk through a door and there’s some 15 year old sat in front of you who has just stolen a car and you have never encountered that situation before, you know, you want some kind of tool to assist with that and one of the things that the likes of Asset offers is 12 domains, a checklist to give yourself, you know, confidence that haven’t … there aren’t any yawning gaps missed and I think that after going through a number of those kinds of assessment processes, you begin, if you are a critical practitioner, to question, is there more to it than this, and that’s where some of the more nuanced SPGA stuff comes in about it can’t just be these domains, maybe there’s something more to it. I think that there’s also an issue about the training and development of … when we speak for social workers, you know, how well are people prepared both to enter into practice and on qualifying, continued professional development that takes in some of these roles, because I think that it’s critical that we have in our universities where we train the social workers of the future, individuals who can speak with authority about practice, but can also speak credibly about theory, whether that be desistance or actuarial tools, and one of the things that has frustrated me through the years is, there was a period when there were social workers who said, oh, I don’t do theory, it’s just all gut instinct, anybody can work with people … I can’t be doing with that, likewise whenever you have ivory tower academics who left practice many moons ago and the idea of sitting there in front of somebody that they would be working with on a day in day out basis would terrify them, that’s not good enough, we need to try and carve out a way where … I suppose one of the things was activism … action without thinking is activism, thinking without action is mentalism and you need to find some kind of combination of those both and to me, that’s kind of what you want to be in social work, you want to be somebody who can embrace theory, you want to be somebody who can connect with people without alienating them the first night you sit down with them and especially when working with the kind of groups that we are talking about where the window of opportunity to be deemed all right or not is pretty small, you know that’s the first battle before you get into even using a risk assessment tool or approach
MM And there are similar challenges within the forensic setting …?
LJ Absolutely, I think in a way, certainly in health, forensic mental health, I think there is some more discretion in terms of the methodologies that are used, there’s a lot of clinical discretion that we have where I think it’s absolutely the same in as much as the resources, time and priorities and a lot of the time to sit down and actually read in-depth about the practice that you are actually required to engage in is just time that people don’t have, they just simply don’t have it, so we are able to access the work of the centre as well but we also have the work of the School of Forensic Mental Health, and I think similarly there’s scrutiny put on us through mental health tribunals etc, but I do think there is a real challenge for practitioners, frontline practitioners to keep abreast of the literature and similarly I think there’s a real challenge for academics to embrace complexities of real people, and actually help us with research that actually reflect the cases that we see.
MM You know if we are fast forwarding 12 months from here, what one area of work do you think would progress that particular agenda to support either challenges of social work or within the forensic setting?
DO I would like to see … I mean knowledge exchange is kind of buzz terminology within what we are doing, which is bringing practice experience, taking academia and trying to pull something together, but more approaches, whether they be small projects, not, like Nina said, some vast mental analysis, but individual practice experiences around use of a particular methodology or particular assessment, whereby practitioners share with academics and they produce an output throughout, which may be about their individual experience but it may inform others as well, you know creating a momentum around evidence based practice but also practice that’s informed by the experiences of young people because I think its one of the things if you went into Polmont today and said to a bunch of guys, you know what’s better Asset or YLS, or what was your experience of being … you know it would be ridiculous, and they probably, of you said to them, “can you name the risk assessment tools that were used in your Criminal Justice Social Work report that ultimately led to you being sentenced to 8 years in prison?”, you know there’s a huge disconnect there and I think again that’s where Monica Bryant’s work is really interesting because it’s just so clear how irrelevant almost some of this is to a young persons lived experience, maybe somewhere down the line they come to appreciate it’s impact, so engaging with young people, knowledge exchange.
MM Okay, you are slightly cheating the …
DO Yes, I have taken 2, I am sorry
MM Yourself, Stuart?
SA If you look at the frame guidance that we currently have, one of the things that it recommends is that we move away from trying to band people in terms of high, medium and low risk and we move to a much more kind of nuanced conceptualisation of risk assessment where we are making some calls about what somebody might be at risk of doing in the future, who they might be a risk to, what might be the level of harm that’s caused in all of that and what do we need to do to actually protect the public in relation to all of this, and indeed reduce risk. So I think you can only do that by having a kind of critical idea of what your role is and actually being able to move towards a kind of formulation idea of what risk assessment is. And for me, I have to say although I think there’s a lot of value in structured professional judgement approaches, I think it’s also sometimes about just having a kind of idea or formulation that you bring to cases that can inform any kind of style or mode of risk assessment that you are involved with. So if in a years time we can have a few more practitioners that are a little bit more conversant with the idea of formulation in the work, which I think actually they are probably using anyway but probably don’t use that language, but I think we would be getting somewhere.
MM Importantly in terms of language, when we say formulation within the framework for risk assessment management evaluation, we mean the understanding of the case ..
SA Yes
MM … quite often we use the terms between professions that each profession may not understand, so for you it would be on that understanding of the case based on the particular profession?
SA Absolutely, yes
DO I think linked to that as well, if you ask me … say I am back in practice, allocated what is deemed a high risk case and a box load of files come in, the one thing that I most want there to be in there is a detailed analytical chronology starting pre-birth, working to current day and that feeds into everything, you know the lived experience from start to finish, whether it be biological, sociological, psychological factors, you know, that’s where I would want you people to start.
MM So, acknowledging that risk assessment process of analysing, evaluating and then importantly communicating that information …
DO Yes
MM … in a meaningful way. Thank you David, and Lorraine, you have one wish list …
LJ Well I am going to be slightly naughty because what I am going to say is, I think … I am going to try and provide some data from our managing high risk youth project that says that although this is a very resource intensive procedure, it is actually achievable in a much shorter timeframe when you have a skilled staff group. So we are piloting quite an interesting approach at the minute, whereby social workers and multi disciplinary colleagues will come along to our clinic and will sit with a panel of people who are pretty skilled in this approach, skilled in risk assessment, and within 2 or 3 hours we are helping them produce that formulation, those risk scenarios, because I think one of the major obstacles is this notion that a barrier of resources, “we don’t have time, it’s difficult to learn something new …” but I think hopefully in 12 months time, that was when you gave us our deadline … we should have some data that says this is actually achievable, you know, it’s not so frightening and intimidating that people will revert back to the habitual models that they have used, it’s actually something that can be done so long as you invest in your staff group, you contain them, you support them and you give them a system to progress. So that’s my wish list, is a decent paper from that clinic.
MM For information on that project, Lorraine, where can listeners find out more?
LJ It’s on the website, isn’t it, which is
SA The CYCJ website, Centre for Youth and Criminal Justice …
MM And since we began this discussion in relation to research and evaluation, can I end by asking yourself, Nina, in 12 months time, one particular area of work in relation to the structure of evaluation do you think would progress?
NV Yes, I think I would … you know, what David and Lorraine said, I think anything that we can do to contribute to the evidence base of assessment and risk for how practitioners make decisions would be really useful. I think there is a lot of research there that says that kind of professional judgement isn’t applied in the right way, whether it’s through actuarial adjustment, you know it does reduce the acts of our risk assessment, but I think that the work that’s going on in Lorraine’s clinic would be really interesting in how people really do come to decisions about risk, and certainly I would like to be looking at more research about what factors can influence practitioners decision making. I think it’s an area that we definitely don’t know enough about at the moment.
MM Thank you Nina and thank you everyone for contributing to today’s discussion.
Transcript Copyright:
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License