Next Steps on a Cumulative Impact Assessment (CIA)

Simon Duffy explores the possibility of developing an even more rigorous assessment of how cuts have targeted disabled people.

Author: Simon Duffy

Since the Spending Review of 2010 The Centre for Welfare Reform has tried to understand the overall impact of the cuts, changes to the benefit system and other social policy impacts on disabled people and people in poverty. Given a basic understanding of Government finances it was obvious that these groups were under attack. This then led to the creation of the Campaign for a Fair Society and to my various efforts to estimate the likely impact of the cuts.

What has been most surprising has been the lack of interest in the issue from newspapers and TV and the decision by the Labour Party (so far) to avoid drawing attention to what is one the greatest social injustices of our generation. The policies being introduced today are far worse than anything developed by Mrs Thatcher and they put the UK on track for being one of the most unfair and unequal societies in the developed world. When I speak to international audiences and explain recent changes they are shocked and surprised by what they discover. 

The Government's success in targeting these disadvantaged groups for cuts can partly be explained by their effectiveness in turning disabled people and people in poverty into scapegoats for wider financial problems that they clearly did not create. Nevertheless, in a country that prides itself on its sense of fairness, the acceptance of these policies marks a serious deterioration in moral integrity.

Many of the organisations that one would have expected to speak out about these issues, especially national charities, and QUANGOs, have also been very quiet. This may be because so many organisations have become very dependent upon funding by Government. 

However, not just the Campaign for a Fair Society, but many other new voices have arisen to challenge this injustice:

These new-wave campaigning groups have all had some limited success in challenging the Government and the Opposition and Pat's Petition and the WOW Petition gained mass support (over 104,000 signatures) for the very reasonable suggestion that the Government might at least do its sums, and that it should calculate the 'cumulative impact' of its policy changes. This idea has become known as the Cumulative Impact Assessment or CIA.

Like these other groups, The Centre for Welfare Reform has had no funding to work on this; but we have so far published four estimates of the cumulative impact of the Government's cuts. In 2011 these calculations were set out on webpages within The Centre's library and in the following three years they were published as part of three distinct publications:

Manifesto for a Fair Society (2012)

A Fair Society? (2013)

Counting the Cuts (2014)

As time moves on I have tried to improve both the methodology and the data that I have used, and in the last iteration of this research I decided to publish all the figures online via Google Spreadsheets.

Thanks to the WOW Petition there was a debate on this matter in the Commons on the 27th February. Much to my surprise my last report was cited in the debate and the Minister of State for Disabled People, Rt. Hon Mike Penning MP said that he would contact me and my team. [By the way, there is no team, just me, a computer and the internet.]

Then on 12th March I received the following by email:

Letter from ODI - 12th March

Dear Dr Duffy

During the recent Commons backbench debate on welfare reform on Thursday 27 February, Ian Mearns MP raised the issue of an assessment of the impact of the Government’s welfare reforms on disabled people, which included references to your work at the Centre for Welfare Reform.

The Minister of State for Disabled People, Rt. Hon Mike Penning MP, is familiar with the work that you have done in this area, and has asked me to contact you.

We would welcome the opportunity to meet at a mutually convenient date and for you to present your analysis and findings to a group of poverty and disability analysts in DWP. If this is acceptable, then please contact Alex Barton [email] to make the necessary arrangements to take this forward.

Yours sincerely

Hugh Pullinger - Deputy Director, Office for Disability Issues

After discussions with our allies and in response to this letter I sent the following reply to the ODI:

Letter to ODI - 11th April 2014

Dear Mr Pullinger

I would be very pleased to meet with you or your officials to discuss my findings and analysis. However, would it be possible to clarify a few matters first:

1. Current methodology and data

As it stands, all the data, the workings and the analytical assumptions that we've made are in the public domain and published on the Google spreadsheet referenced within our last report. Given that your researchers can take their time to review this without my presence could I ask that, before we meet, you provide me with your critique, setting out:

  1. Any methodological mistakes you think I have made, with potential corrections
  2. Any data errors, with alternative sources of data
  3. Any different analytical assumptions you would like to propose

It would make our time much more productive if this could be produced in writing before the meeting.

2. Queries

The published spreadsheets also set out a range of issues where I have made reasonable simplifying assumptions, for lack of relevant data. Given the vast resources of Whitehall perhaps I could outline these assumptions and my uncertainties here and you could also ask your researchers to look into this and offer me better founded assumptions or data in the following area:

  1. Local government spending - I have not tried to fully integrate local government and Government spending and I have not found data for local government spending that allows for integration of data with PESA 2103. Do you have access to the relevant data (some of which will obviously need to be based on forward predictions based on reasonable assumptions)? It would be much better to develop an integrated account of Government spending to avoid double-counting or other errors.
  2. Social Care spending - As you are probably aware the number of people supported by social care has fallen by 25% in the past few years. This seems to be higher than the financial cut and implies that the marginal impact of cuts is disproportionate and is actually radically increasing the inefficiency of social care. Given that it is the level of service that is the key outcome we are interested in have you any alternative suggestion as to how best to measure the radical reduction in social care?
  3. Benefit spending - As you are probably aware, my last analysis used the figures set out by the DWP for benefit spending. This seemed to be much lower than the deep cuts announced by the Chancellor in previous Autumn Statements. Can you provide an up to date account of the savings you expect to make across all benefits; but especially with respect to DLA/PIP, ESA/IB and other disability related benefits?
  4. Social Care and poverty - Given the poverty forced on disabled people by the high level of means-testing in social care I assume that a disproportionate number of those entitled to social care are poor. However I could find no data on this and have had to make a reasonable assumption. Does this data exist and what assumption would the ODI make about poverty for social care recipients?
  5. ONS analysis - The ONS household income analysis - which sets out benefits, taxes and benefits in kind would be an ideal foundation for a Cumulative Impact Assessment given that we already possess data for the distribution of disabled people across the income deciles. If we could add in social care to this framework we would have a very strong foundation for further work. In addition, if we could estimate cuts in benefits against this framework then we would have effectively achieved the desired goal. Do you possess either (a) a breakdown of social care use against income deciles or (b) an analysis of the relative distribution of cuts in benefits against income deciles? If not what would it take for the DWP to provide this rather important data?

Again, I would be most grateful if you would provide information about these matters in writing before our meeting.

3 Who will be present?

As you know the challenge of a Cumulative Impact Assessment is to draw together information from various different Government departments, or other bodies, including:

It would be most useful to know who would be present in advance of the meeting and clearly it will be important that appropriate officials from all the relevant departments are involved in the meeting.

Yours sincerely

Simon Duffy

Letter from ODI - 15th May 2014

Dear Dr Duffy

Many thanks for your response on 11th April 2014. I am sorry for the delay in getting back to you. Hugh Pullinger has recently moved to another post within DWP and I have only just taken over as head of the Office of Disability Issues.

As you are based in Sheffield, I wondered if it would be more convenient for you to meet with our officials in our Sheffield offices.

Please could you contact Alex Barton to make the necessary arrangements to take this forward.

Yours sincerely

Pat Russell, Deputy Director, Office of Disability Issues.

Letter to ODI - 7th July 2014

Dear Ms Russell

Thanks you for replying to my earlier letter. I am very happy to meet in Sheffield, rather than London, as you suggest. This would be much less costly for me.

However could you also address the issues I raised in my earlier letter, namely:

[Here I repeated the requests for information from my first letter]

I received an email in response to this on 11th July saying that Alex Barton would not be available until the end of the month.

Email from ODI - 18th August 2014

Thank you for your letter, as discussed me and my colleague Peter Matejic would be very happy to travel to Sheffield to meet with you, In terms of cementing a date how does Wednesday the 10th of September suit? If convenient then we would propose a meeting around 1pm at our offices at Rockingham House, 123 West Street, Sheffield S1 4ER,  If not then are any dates w/c 8th of September more convenient? Or we can move further back into September if that’s more practical

Email to ODI - 18th August 2014

Dear Alex

Thanks for your message. Can you confirm that you have seen my original letter and the questions set out within it?

Email from Alex Barton, 19th August 2014

Dear Simon,

I can confirm I have seen your original letter, we feel it is best to cover the content of your requests in sections 1 and 2 within this meeting,

In relation to section 3, invitees, as discussed attendance will consist of myself and my colleague Peter Matejic we regret we are unable to discuss matters beyond the methodological approach you have adopted. We hope this will still prove useful?

Kind regards, Alex Barton

Email to Alex Barton, 5th September 2014

Dear Alex

I am still happy to meet with you and your colleagues, even without colleagues from other departments, as long as you address the central issue that I set out in my original letter - that you should provide your own critique - in writing - in advance of our meeting.

As I made clear in my previous letters, The Centre for Welfare Reform has made its own analysis completely transparent - both in terms of data and methodology. I am also aware that, as with all statistical methodologies, there are many questions one could raise about any particular methodology used. [I also set out many of the ways in which I myself would like to improve the model.]

But, as it stands, The Centre for Welfare Reform and the ODI/DWP differ on a rather basic point:

Is it or is it not possible to make a reasonable estimate of the degree to which cuts in spending (and other financial and service impacts of Government policy) target disabled people?

The Centre for Welfare Reform is confident that a reasonable estimate can be made of the cumulative impact - and it has made such an estimate. The ODI/DWP seems to believe that such an estimate cannot be made. I find this position hard to believe for several reasons:

  1. The DWP has vast resources at its disposal in order to make such an estimate
  2. The DWP recruits some of the most able people in the country to understand the impacts of Government policy
  3. If the DWP cannot calculate the impact its own policies then how can it in all good faith carry out such policies, especially when it has a duty to protect the rights of disabled people

For all these reasons I find it hard to believe that you cannot make a reasonable estimate of the impact and relative impact of Government policy (and again this should be all of Government policy - not just DWP policy).

If you do not offer me your own critique in writing, and if you fail to answer any of the questions in my letter in writing, in advance of the meeting I am afraid that I would feel that there was a significant risk that the meeting was not being held in good faith. It seems unreasonable and unjust for me to make my own own case utterly transparent, while the DWP does not set out its own reasoning or demonstrate that my own methodology is deeply flawed.

Again, to repeat, I am happy to meet with you and much happier that this should be in Sheffield, rather than in London, but that meeting needs be set up on some kind of reasonable basis.

Best wishes, Simon Duffy

Email from Alex Barton, 9th September 2015

Dear Dr Duffy,

We are more than happy to discuss with you the key issues you have raised in your correspondence, however, whilst we are mindful of your request, in order for us to all move forward it would be best for us all to meet and have a discussion where you talk us through the analysis you have carried out at a mutually convenient date. Given the proximity to our proposed Wednesday slot, if you do still wish to meet, we will need to arrange another convenient date to enable us to book the most cost effective travel.

Government regularly produces analysis of the cumulative impact of all Coalition changes at fiscal events, not only to welfare and personal tax policy but also to public spending, on households across the income distribution, and we recognise (and promote) this type of analysis as a valuable tool in policy development and evaluation. However, we also believe it should be treated with some caution. In particular, the impacts of policy changes are compared with the continuation of the previous Government’s policies, which were unaffordable, rather than with an alternative approach to fiscal consolidation.

Of course, DWP carefully considers the equality impact of the individual policy changes on those with protected characteristics including gender, race and disability – in line with both its legal obligations and its strong commitment to equality issues, but we must ensure methodological credibility in order to publish a robust cumulative impact separately for disabled people. This is a very complex issue involving a series of judgements about the robustness of any methodology or dataset, which we believe is more appropriate for discussion.

However, the main reasons why we do not believe that such analysis can be conducted accurately are that:

  • our survey data is limited, particularly in terms of the capturing of the severity of disability;
  • there are a number of overlapping reforms coming in at different points up to 2017/18 and that the order of reforms is important to capture;
  • the caseloads are dynamic, so changes in one benefit will affect eligibility to others and this is necessary to capture in any analysis;
  • and restrictive assumptions are needed around how income is shared within households.

I would just like to reiterate that the Government expects its reforms to taxes, welfare, and public service spending to have a long term positive impact, in particular helping to get more people into work by making work pay and providing greater support for those who cannot work by targeting resources more effectively. We can expect to see people moving into work and taking more hours.

Once again, I sincerely hope you will carefully consider and accept our request to meet with you very soon. 

Kind regards, Alex Barton

Email to Alex Barton, 7th October 2015

Dear Alex

Thank you for your email. It is helpful that you set out your reasons for the Government refusing to carry out a cumulative impact assessment of the impact of its policies on disabled people. On one matter we can agree - the Government has certainly introduced a wide range of changes which will all impact on the lives of disabled people and their families (although I cannot agree to call them 'reforms'). 

These include:

There are two critical issues where we do not agree; for:

  1. It is not too difficult to estimate the cumulative impact of these policy changes. As you are aware The Centre for Welfare Reform, Demos and recently the National Institute of Economic and Social Research have all carried out such research, and we have all concluded that disabled people are severely targeted by Government policy.
  2. It is irresponsible to bring in a whole range of policies that impact on the lives and rights of disabled people at the same time, without knowing their likely impact. In fact I do not understand how it is possible to justify the Government's policies in terms of their "long term positive impact" if you can't estimate that impact.

In fact the Government's position is self-contradictory. You claim that it is too difficult to estimate the impact of your policies, while claiming that they will be positive. I am afraid I therefore have to conclude that you are not being entirely honest with me.

What is more, you are seeking to meet me to discuss a methodology which I have made utterly transparent. Any problems you have identified can be stated publicly and in this way the methodology can be improved. The fact that you do not want to offer public or constructive criticisms of the methodology, or to offer better data, suggests that you are not acting in good faith. I am afraid all our correspondence to date suggests that the purpose of any meeting with me would be merely to satisfy the Department desire to have carried out, to the letter, a promise made by the Minister in Parliament, but only for the sake of form, and with no real interest in developing a cumulative impact assessment.

Until you can make a proposal that suggests that the Government is acting in good faith I will have to conclude that any meeting would be pointless.

Best wishes, Simon Duffy

Further steps

I do hope that there is progress on this issue. If I have been able to develop a reasonable methodology and carry out all the relevant research with no funding or support then I assume that the Government, the Labour Party or a major Foundation could do a much better job. If the Government does get back to me and provide some or all of the relevant data then I will at least be able to build on our current state of knowledge.

Currently I will continue to offer what support I can to any disability group campaigning against this gross injustice and, with the Campaign for a Fair Society and Learning Disability Alliance England, I will work to identify some suitable body who might hold and take forward this work.

Recent correspondence between Ian Jones of the WOW Campaign and David Phillips of the Institute of Fiscal Studies (IFS) confirms the fact that the IFS also believes that a CIA is quite possible.

The publisher is The Centre for Welfare Reform.

Next Steps on a Cumulative Impact Assessment (CIA) © Simon Duffy 2015

All Rights Reserved. No part of this paper may be reproduced in any form without permission from the publisher except for the quotation of brief passages in reviews.

Article | 14.04.14

disability, social justice, tax and benefits, Article

Also see