Algorithmic injustice: an interview with Rick Burgess from GMCDP

Greater Manchester Coalition of Disabled People (GMCDP) are currently taking legal action against the Department for Work and Pensions (DWP) to highlight the potential discrimination in their algorithm for detecting fraud. They have paired up with Foxglove, who specialise in challenging discriminatory and unjust algorithms, to begin a Judicial Review process if the DWP refuse to reveal their process for selecting claimants for fraud investigations. DWP fraud investigations can be traumatic and stressful, involving intense surveillance and even the threat of criminal conviction. If built-in bias means the system is selecting disabled people more regularly for these investigations, then it could be unlawful.

We spoke to Rick Burgess from GMCDP about the challenges disabled people face when claiming disability-related benefits, how fraud investigations harm disabled benefits claimants, and why GMCDP have brought this legal action.

Could you tell me a bit about Greater Manchester Coalition of Disabled People?

I work for the Greater Manchester Coalition of Disabled People (GMCDP), and within that, my main job is the GM Disabled People’s Panel. We have a partnership agreement with them and so we’re one of the equality panels for Greater Manchester. We play a critical friend role – we advise and challenge at the highest strategic level to try and make things work better for disabled people.

GMCDP came into existence in 1985. Right now we are in the process of changing structure and turning into a Community Interest Company (CIC). We wouldn’t be a charity when we formed, because charity has a certain kind of approach to disabled people that we’re not fans of. We did notice over Covid that there were lots of pots of money that we couldn’t apply for because they weren’t for ‘companies’ – but we’re only a company because we’re disabled people that don’t want to be treated like a charity. We work from the social model of disability, not the charity or medical model. Charity is an ideological straitjacket.

How’s it going?

Andy Burnham has always been really good with us, but we’ve been treated terribly by the government for the last 12 years. Greater Manchester Combined Authority gets it now, but whenever we try to change things here, there’s sabotage by central government.

The shortest version is: there’s a million things to do. There’s a constant background fact that the whole benefits system is a nightmare. It’s been turned from a system which wasn’t great but which wasn’t actively trying to destroy people into frankly, now, a system that does feel like it’s trying to destroy you. Social care is in ongoing collapse. Housing remains hopelessly inaccessible. Transport has had some improvements here and there sometimes, but fundamentally there’s still a lot to do to make it accessible.

We work with a social model perspective: there’s barriers in society – physical, mental and ideological barriers that disable us – and we’re nowhere near the point where a disabled person can just normally go about their life and not meet anything that bars them from stuff. We’re nowhere near that. In fact we’ve gone backwards.

We ran what we called a benefits self-defence workshop about 3 years ago and Greater Manchester Law Centre was part of that. That was great. The idea was to upskill people so they could not only deal with their own case, but teach others how to deal with theirs, because we recognised that there is simply no prospect of there being enough capacity in professional advice services to meet the demand. You’re looking at several million people on disability benefits in the country. You can’t become a welfare rights expert overnight, but you can show that there’s probably always something you can do to appeal a decision. It’s never the end of the line, even though it’s hugely frustrating and dispiriting.

So we’re trying to prioritise where we can have some effect, but it feels like going up the down escalator.

Moving onto the challenge you’re running with Foxglove, what made you want to make the challenge? What inspired it?

As disabled people, we have a long-term exasperation with how the DWP do things, but there’s rarely something you can grasp onto and have a real chance of affecting something. GMCDP hasn’t done anything like this before.

The way it started was that, a while ago, Privacy International did a big piece of work about the surveillance the DWP do. It was really good but terrifying, and I don’t advise people to read it if they’re prone to become paranoid. If you need the detail it’s there, but I wouldn’t say to go and read it, because it will make you want to hide in a cave somewhere. Russian spies would get less surveillance than people on benefits.

When that came out, Foxglove got in contact, and we said we think they are using an algorithm in how they’re choosing who gets investigated for fraud. It seemed very possible based on experiences our members have had. It’s called ‘compliance’, and you get either an email, a letter or a phone call, and you never know why, they just ask you to prove you’re not a crook.

I was talking to an Outreach worker for Foxglove, and they were talking about algorithms. The fundamental thing with algorithms is that they’re created by humans, humans have biases, and unless you do it really carefully and properly co-produce it, your algorithm is just going to formalise existing prejudice, automate it and then accelerate it. Intersectionally – not just for disabled people – it will pick out people who have traditionally faced disadvantage and exaggerate that disadvantage. That’s what badly created algorithms do.

We began the process around September last year. As part of that we’ve got a Crowdfunder, and we have now reached about £15,000 to cover potential costs. We’ve done some extra stuff around algorithms too. It’s become quite a good strand of work because Foxglove like to campaign around their cases, and they’ve got a Press Officer. The case is a lynchpin of a whole bunch of other activity about algorithmic justice and transparency.

I hope, because Foxglove are really good at this, and have won a bunch of stuff before, we have a chance. Foxglove’s expertise is absolutely essential and brilliant on this.

We’re still in a position where you have to explain what an algorithm is. The easiest way to explain them is that it’s a set of instructions that tell a computer to do something. In this case we think it’s trying to detect unusual economic activity – so, how much money have they got coming in, how much is going out, how many credit cards have they got. All these things start to factor in. But it doesn’t take into account the different ways that disabled people may live. The additional costs of having an impairment, on average, are £400-500 a month (Scope research) which you don’t get enough additional money for, so disabled people are often even poorer. We doubt they will factor those sort of things in. For example – why are they always buying more expensive food? Well it may be that you have to buy pre-prepared food because you cannot manipulate cooking devices. If these things aren’t factored in, then the algorithm could be selecting people unfairly.

We’re just asking a question about how the algorithm works. They’re the ones refusing to answer.

‘Fear of the Brown Envelope’ from Foxglove’s website, 2022.

What does being investigated for fraud do to people, psychologically, when they know they’re being investigated for fraud but don’t know why?

We have talked about the fear of the brown envelope. It drops through your letterbox and you don’t necessarily open it immediately – your heart sinks and it takes a few days to get round to opening it. My experience was that I was a full-time carer for my mum, and I looked at it like this: I have a human being relying on me, I haven’t got time for this bullshit. It didn’t intimidate me as much as it might have if I’d been on my own, but I do know it does hit other people extremely hard and they can shut down, not even answer the letter. If people fail to respond – which can happen for many reasons – their benefits can be cut. We know of cases where, sometime later, they’re found dead in their flat. That absolutely does happen. It’s extremely distressing and tough.

When I had free time in the evenings, I would put together the papers and send them back, but you’re never really told when they’ve stopped looking into you. Eventually you get notified, but without much detail. It’s really unsatisfactory how you’re treated, to put it mildly.

For me, it never progressed to an interview, though I’ve had friends who faced that. When it does progress to interview, a lot of people don’t understand that they should really have an advocate at that interview. It’s the beginning of a potential criminal prosecution – you should not be walking into it on your own. I know people who have walked into it on their own and they’ve made things worse because they didn’t know what was going on. It’s appalling.

Our main point about the algorithm is: if you’re going to investigate someone for fraud, that’s quite a powerful weapon and so you should be really sure about who you’re using it on. Because otherwise you are causing immense distress to probably innocent people for no good reason. That’s not only immoral and unreasonable, we believe it may be unlawful.

Do you think the use of algorithms makes it harder to hold authorities accountable?

Yes, because the DWP has already made the excuse that it’s not the algorithm – that it’s data matching and a human checks it. What they’re effectively saying is that they are using an automated system, but that a human checks it, so that makes it okay. But by just combing through everyone and cross-checking with credit data and bank data, the DWP are doing something invasive and disproportionate to the threat faced. They’re applying a level of surveillance you would apply to serious organised crime to individuals claiming some benefits. We suspect, in how it picks people, it will be disproportionately picking disabled people, because there’s always a suspicion in the department that you’re faking it.

Do you think part of the suspicion is because benefits are too low, so there’s always suspicion you’ll be faking it to increase your income to actually liveable levels?

Yes – and I think people actually become ill or impaired because they are on Universal Credit. You can be on Universal Credit because of unemployment, and it can trigger a depressive episode. The system can damage you. The DWP are becoming authors of their own doom. If you don’t want lots of people on disability benefits, you have to have a society that isn’t disabling people, where we have accessible housing, equal employment and pay, for example.