Official Report: Minutes of Evidence

The Chairperson (Mr McKay): I welcome to the Committee Ms Jennifer Wallace, head of policy at the Carnegie Trust; Dr Peter Doran, associate of the Carnegie Trust and member of Queen’s University's school of law; Ms Michelle Furphy from NISRA; and Dr Bernie Stuart from the Department of Finance and Personnel. You are all very welcome. Before we go to questions, I invite the witnesses to make a brief opening statement. It may be helpful to first receive a brief recap of the Carnegie recommendations for the benefit of members.

Dr Bernie Stuart (Department of Finance and Personnel): Would you like us to start with a summary of the recommendations or the summary from the Northern Ireland Statistics and Research Agency (NISRA)?

The Chairperson (Mr McKay): The recommendations would perhaps be best.

Dr Stuart: Maybe if I tell you what we had planned to tell you about before the questions — [Laughter.]

We had thought about telling you about the well-being work, which ties in with how outcomes are measured, of which Michelle is going to give you a brief summary. Jen would then give you a summary of the recommendations and Peter had planned to tell you about how the report has been taken forward so far since it was completed.

The Chairperson (Mr McKay): That is fine.

Dr Stuart: I will put it into a little bit of context. As you know, the report was completed in March and has been taken around a number of bodies, such as the Chief Executives' Forum, and is being considered by the Minister of Finance at the moment. After he has considered the report in a bit more depth, the plan is that he will bring it to the Executive for further consideration.

The thinking is that the report ties in quite well with the outcomes-based approach for the next Programme for Government (PFG). As you know, there is quite a lot of other work under way on an outcomes-based whole-of-government approach; so the report, which, as you know is an independent report, ties in very well with current thinking and the timing is also quite good as we move into the next Programme for Government. It also ties in well with the community planning functions of the new councils. They now have to come up with plans for their way forward and their thinking through an outcomes-based approach. The idea is that an outcomes-based whole-of-government approach is the way forward and will be taken forward to the Programme for Government. That is the context. If you are happy with that, Jen can start with the summary of the recommendations.

Ms Jennifer Wallace (Carnegie UK Trust): Thank you and good morning. On behalf of the Carnegie UK Trust, I am delighted to be here with you this morning and by your interest in our work on this matter.

We started as a roundtable in Scotland on well-being in 2010, and some recommendations from that were taken forward by the Scottish Government. We are keen on sharing between the jurisdictions of the UK, so, when there was interest in Northern Ireland, we were happy to help. We are not a shoot-and-run policy think-tank organisation. We are in it for the long haul, so, if there are things that we can do as an organisation or with our associates, we would be pleased to help you in the work that you might be interested in on well-being.

I will not rehearse the process of the Roundtable because you are probably aware of the journey that we went on. In the end, 10 recommendations were made. They are a package rather than a pick-and-mix selection. They are about trying to move forward, as Bernie said, a whole-of-government approach to improving outcomes for communities.

We started with the idea of integrating well-being as a collective purpose through the Programme for Government. That is the first recommendation. The international research and our background work shows quite clearly that having a sense of purpose, a vision for government and a shared narrative are helpful in the complex policy environment that we all work in, where so many things are interrelated. The second one is about a society-wide conversation on well-being. Again, from the international evidence, the importance of discussing with the community about what they feel well-being is comes out time and again. We put that at number two because we think it is critical to have that towards the front of a framework. Number three is about agreeing a set of strategic commitments and outcomes, which we referred to throughout as the well-being framework. In our minds, there is an overarching purpose of government and then a series of agreed outcomes that form a framework. We hope that that would be something that the Government would be interested in in terms of the Programme for Government.

We hope in recommendation four that future PFGs look at the changes that would need to happen in public services and other areas of government activity to create greater well-being. You are then moving into government activities and policy changes that you might want to implement as a direct result of the well-being approach.

Number five is about training and capacity building. We found throughout our work and the international work the difficulty that some people have in understanding what well-being is and what outcomes are. Therefore, there is a process of change management that has to be gone through and invested in.

There is a recommendation for an annual report before the Northern Ireland Assembly. You might be particularly interested in that because there are roles here for Assembly scrutiny of government and social progress. That is something we see as being very much at the heart of it. There is an additional recommendation for a standing advisory group because there are areas of significant expertise, perhaps some of them technocratic issues about indicators, others about community issues, that we hope would be able to help to develop this process. Such a group would also keep that eye outside and provide a little bit of external scrutiny in terms of keeping things on track.

Those are the recommendations. If you are comfortable with that, we will move on to Peter and tell you what has happened since the launch.

Dr Peter Doran (Queen's University Belfast): We had a launch on 12 March. We handed our recommendations over to Minister Hamilton, who was very supportive and, as you heard, undertook to take the recommendations forward to the Executive. Our chairs of the Roundtable, Martyn Evans and Aideen McGinley, chaired the launch and had an opportunity to present our findings to your Chair on another occasion, and we were delighted to be able to do that.

We have been very taken by the response to the report. The Minister has been very supportive. Just yesterday, we met the partnership panel. Part of the ambition is to align the work of local government and community planning with some of the outcomes in future PFGs.

Yesterday's event with the partnership panel was excellent. We had unanimous support for considering the content of the Carnegie recommendations. Minister Durkan and Minister Hamilton spoke very supportively, and, indeed, a number of the representatives of local government indicated that they would like to take the recommendations forward for further consideration and may well undertake a visit to Scotland to meet some of their counterparts. Your Chair will remember that, at our first conference in 2013, he was among the first to recognise the importance of the agenda to have the new dispensation for local government fully reflected and integrated in a future PFG. So, thank you for that.

There has been excellent support from the Chief Executives' Forum and from parts of civil society, and you will know that the recommendations talk about a new, compelling narrative. In many ways, this is not a new narrative. We are increasingly finding that it is picking up on conversations that are happening at the highest level in the public sector, in the Civil Service and in civil society organisations as well, especially around outcomes and the possibilities of building a new common language for governance around well-being. Those are some of the responses that we have had. We are having a conference on 10 June, when we will provide an opportunity for further responses to the recommendations. I know that your Chair has agreed to address the conference, and you are all very welcome to come to that. That is my contribution for now.

Dr Stuart: If you are happy, Michelle will talk a little bit about how well-being is measured. I emphasise that this is not specifically about measuring the particular outcomes that will be in the well-being framework that Peter referred to, because it is a new way of measuring things and a longer-term measure, and, rather than measuring outputs, it is about measuring outcomes. Michelle can give an update on what is available in Northern Ireland and what type of approach might be used, and we can move on from there.

Ms Michelle Furphy (Northern Ireland Statistics and Research Agency): Thank you, Bernie. Thank you to the Committee for the opportunity to come back again. My colleague Kevin Sweeney and I were here last year to speak to you about this topic. You will have a slide show presentation in your pack. I have been asked to run through it in summary, so I will not go through it on a slide-by-slide basis. If I refer to a specific slide, you can look at it and keep track with me.

There is some background information in there about why we are measuring well-being, and that was to give us a bit of a recap as to where this has all come from. It is really about that broader context and the international call for Governments to measure all of what matters and to get away from focusing solely on the likes of GDP. It is really important to note that I am not saying that you do not use GDP; you still do, as GDP is still very important. It is about trying to get that wider dimension, and that is what we mean by well-being. It encompasses all the economic, social and environmental dimensions that influence our economic lives, social lives and so on.

I will outline what has happened recently with our near neighbours. In the UK, in November 2010, David Cameron launched a debate on measuring national well-being. That was not a particularly political step, because the previous Labour Government had already taken steps to have those developments in place to measure well-being. Reflecting that non-political nature, David Cameron asked the Office for National Statistics (ONS) to take the work forward. On slide 4, that is obviously David Cameron, and the other individual is Jil Matheson, who was the National Statistician at the time.

ONS then undertook a programme of trying to identify what well-being is all about in the UK and did a widespread consultation asking about the sort of things that Jen and Peter talked about. It involved the citizen and asked people what it means to try to get those measures identified in that way. There were also a couple of events here in Northern Ireland feeding into that. The outcome is the wheel that you see in slide 6, which, hopefully, you are familiar with. You have seen that from us in the past. It includes 10 domains; that is the terminology used by ONS. It is 10 collective areas that the consultation showed were important to UK well-being: personal well-being; our relationships; health; what we do; those sorts of things. Within the 10 domains, there are 41 individual indicators that are used to measure UK national well-being.

I will turn to Scotland. We heard from Jen and Carnegie, whose involvement in Scotland was obviously pivotal in all this. Scotland was actually a step ahead in that it had undertaken its developments in Scotland Performs even before the UK national debate was launched. I know that the Committee has received information in the past on Scotland and that it is familiar with the national performance framework. Slides 8 and 9 show the snapshot of that and how it works at that high level. There are a set of strategic objectives and a series of national outcomes. In slide 9, there is the measurement set, which is where NISRA has come in. We focused on the measurements that have been used in Scotland and in ONS.

Slides 10 and 11 are just a snapshot of something called a Committee scorecard. That is what the statisticians in Scotland use. As a part of the budgetary review process, each of the Scottish Committees receives one of the scorecards, which has on it all the indicators that are relevant to their remit. As you can see, it is very pictorial; it is just a case of "performance improving", "performance maintaining" or "performance worsening". So, the Committees can see at a glance how the indicators that they have an interest in have been performing over recent times. We are aware that the Committees have found that to be very useful in planning and developing their work programmes as they can see the areas that they may need to look at in more detail, given performance over the previous year.

What have we done in Northern Ireland about all that? You know that we have been involved — both NISRA and our public sector reform division colleagues — with the Carnegie Roundtable. Alongside that, we in NISRA have also been looking at trying to replicate the UK's Office for National Statistics' wheel of the 41 indicators because the focus of the ONS work is very much on UK measures and UK indicators. So, in March 2014, we produced the Northern Ireland data for those same measures. We updated that since, in January last year and, most recently, in March of this year. Slides 13 through to 16 give a couple of snapshots and excerpts from that report that we published recently.

We developed the most recent report to try to incorporate more infographics and make it more user-friendly and also to include some international comparisons. That was at the request of the Minister. On these particular 41 indicators we could say where Northern Ireland is compared with the UK, but he asked whether we could say where we are compared with other countries. There are obviously limitations and the report documents them, but we did as much as we could. At least it provides some context for the Northern Ireland data.

Looking at those 10 domains and 41 indicators, slides 17 and 18 show you the extent of coverage of Northern Ireland data for those particular indicators. You can see that, in the personal well-being domain, all five of the UK indicators are available for Northern Ireland. In the "our relationships" domain, there is currently one indicator, but the additional two will be available. We have added relevant questions to household surveys to be taken in Northern Ireland, which will make that information available. They relate to satisfaction with social life and satisfaction with family life.

We have all the same information available on health indicators; all the same information or very similar information available on what we do; and four out of the six are the same or similar on where we live. Two that are not currently available relate to accessing the natural environment and households with good transport access.

In the personal finance domain, four out of five are the same or similar and one is not available. That is the indicator on wealth. We do not have the same survey here in Northern Ireland that gives information on household wealth.

On the economy, obviously, if you look at the wheel and the report, it looks as though that information is not available. That is because there is a national set of indicators in the UK economy domain. It focuses on things like net public sector debt, net national income, inflation and so on. All those things are collected for the UK but not for the regions. However, alternative measures are available for Northern Ireland; things that are specific to our economy, such as the Northern Ireland composite economic index, the number of businesses we have and business survival rates. There is a whole raft of potential economic measures that we could use for Northern Ireland.

On education and skills, information is available for two out of the three UK ones that are the same or similar in Northern Ireland. On governance, one is available and the other is forthcoming. The one that is available relates to voter turnout, and the one that is forthcoming is about trust in the national Government. We added a question on trust in the Northern Ireland Assembly to one of our household surveys.

On the natural environment, our final domain, we have the same or similar Northern Ireland data on three out of the four indicators. The final one, at UK level, focuses on energy that is consumed from renewables. We do not have that here in Northern Ireland, but we do have data on electricity consumed from renewables, so there is something even though it is not the same as that particular one.

Looking at the Scottish measures, you can see that we have done a bit of analysis on what we would have if we were to apply that same set of 50 national indicators here in Northern Ireland. We have the same or similar Northern Ireland data for over two thirds of those 50 national indicators. A lot of the ones that we do not have are things that are specific to Scotland, for example, the Scottish Funding Council's knowledge transfer grant metrics. Obviously, we would produce something akin to that ourselves. For the 11 purpose targets that are in Scotland, we have the same or similar Northern Ireland data for all of them.

The final thing that I want to say is about international developments in general; going back to the start, if you like. Similar measurement frameworks to the Scotland Performs and ONS national well-being measures are in place in a lot of other countries internationally. Carnegie UK has looked at this in the past and has done an analysis of similarities and differences across the areas that have well-being measurements in place. You can see that there is an awful lot of commonality. The things that come out as being important to well-being are income, employment, housing, health, work-life balance, education, social connections, environment, personal security, civic engagement and subjective well-being. That is not to pre-empt any sort of engagement with the public or the wider sectors, but they are likely to be the sort of things that will come out of that.

I hope that I have given you an indication of the range of data that is already available that could be used to measure Northern Ireland-specific outcomes on the things that matter to Northern Ireland, but, in line with the findings of the Carnegie report, it is the outcomes that should drive the measurements, not the availability of the data.

The Chairperson (Mr McKay): Thank you very much. Thank you all for the work that you have done. This is some excellent work from both the Department and the Carnegie Trust on well-being.

How important is scrutiny in this? The set-up that they have in Scotland is very interesting, with the Committees taking a hands-on role with regard to well-being. Also, are you all singing off the same hymn sheet? What is the Department's view on what Carnegie has brought forward in its report? Does Carnegie feel that the Department's pace is adequate to move well-being forward on the Department's agenda?

Dr Stuart: Maybe I will answer the second bit of the question first, and then Peter might like to comment on the scrutiny.

It is fair to say that the outcomes-based approach is the widespread government approach to measuring things, so there is general acceptance of the direction of travel. It is not just for the Finance Minister to take the detail forward; it is a whole-of-government approach. The Minister is considering the recommendations in detail and will bring his thinking to the Executive, but really an Executive approach for the whole of the Programme for Government is being recommended, not an individual Department approach. So, yes, on the direction of travel, the detail has yet to be spelt out by the Executive.

Peter, do you want to say how Carnegie feels about the scrutiny role?

Dr Doran: I know that Jen has a few things to say. It is core to the shift that we are anticipating. If we are to shift towards a learning policy environment and a more supportive scrutiny function, both across the Assembly Committees and on the part of the Audit Office — I am thinking about the role of the PAC — the scrutiny culture is important and is one that we reflected on at the Roundtable. We suggest that it is an area that we could give consideration to. There is a high-level piece about how we bring the Programme for Government to life. I think about it as creating a 3D production rather than the 2D production. It is about a participatory, ongoing conversation with opportunities throughout the mandate for the Assembly and also wider society to engage in the conversation — a fully informed conversation — if we have some new data linked to the outcomes and aspirations for government.

At a more specific level, we have the recommendations of the report cards for the Committees. The Committees, for example, would be considering indicators and information about outcomes and aspects that would not be reflective simply of their departmental-derived responsibility. There would be a wider brief within the context of those reporting cards; that is part of the idea. Jen has a few more comments to make on that.

Ms Wallace: The Carnegie UK Trust has been influenced by some of the international work. We came into this following the Stiglitz, Sen and Fitoussi report, and we have been working reasonably closely with the OECD. The international trends come, in part, from an understanding of a difficulty in trust in government, which is shared across advanced democracies and is not specific to us. It is an issue that our public feel very strongly about and one that our parliamentarians feel very strongly about.

A large number of statistics are available to us in the modern age, particularly in the UK where we are very well served by statistics. However, we have to make sure that we are giving people consistent messages, so there is an issue about the extent to which we provide information to the public in a way that they can scrutinise the performance of government, but there is a separate issue about ensuring that parliamentarians can hold the Government to account on the basis of the promises that they make to the electorate. So, there are two slightly separate issues around scrutiny and trust.

We have said that, internationally, there is a split between the extent to which the type of well-being frameworks that we are talking about exist as instruments of Parliaments or Assemblies or as instruments of government. For us, it is critical that it is an instrument of government, because we would like to see it influencing policy development. If you remove it from direct policy development, it becomes a useful scrutiny tool, but it becomes disconnected from the direction of any policy.

That said, it is important that a shared narrative is a shared narrative, so we would expect parliamentarians to be involved in the setting of the outcomes of the well-being framework and to be consulted on it, involved in it and informed about it. Hopefully, in the way that is beginning to develop in Scotland, they will be able to use that information to hold the Government of the day to account more effectively for what they are delivering.

The Chairperson (Mr McKay): Obviously, there will be engagement with all the respective Departments with regard to well-being, but how do you ensure that the targets will not be skewed? Obviously, there are a number of targets for Departments anyway, and some Departments would arguably set targets that they are going to reach, as opposed to making themselves go that extra mile. How do you guard against the process being skewed by politicians?

Ms Wallace: There is an interesting debate about what we mean by outcomes, and there are different ways of interpreting outcomes. If you look at the international work and some of the work in Scotland and Wales, you see that there are differences in what we think about it. On one level, you have some very high-level outcomes — we want to be safer, stronger, healthier and happier — and those outcomes are, at one level, macro issues. We are proposing a more detailed set of outcomes but are still focused on outcomes.

Inputs are very open to gaming, and activities and outputs are open to gaming. The closer that you get to the high-level outcomes, the harder that it is to game on them. I will give you a concrete example. It is relatively easy to game on crime rates, because you can control what goes into the data. If you are talking about an outcome that is related to people's fear of walking alone in their neighbourhood, which is one of the statistics collected, it is very difficult to game on that, because that is how someone feels. So, it is harder for them to influence that directly without making some of the positive changes that we would hope government and other agencies would be making in that direction.

Mr McQuillan: Peter, in your opening remarks, you said that you had engaged with and listened to conversations from the community and the like. When I look at who has been involved in this whole thing, I do not see many from the community and voluntary sector. How can you convince me that those voices are being heard?

Reading through the outcomes, there is nothing that I would not agree with, but the people I deal with day and daily are not really worried about increasing energy from renewable sources. They are worried about the cost of energy and how they will pay for it. Things like that are important to people, but I cannot see anywhere in the report that those voices are being heard. It looks to be more of an academic study rather than actual people's voices feeding into it.

Dr Doran: I think that you are looking at the membership of the Roundtable.

Mr McQuillan: I looked at that, yes.

Dr Doran: A couple of NGOs were represented on that. More to the point, we commissioned a number of focus groups. We went into the community, and we also spoke to the rural sector and the youth sector. We were very careful to pick up on views and reflect those in the report. We also did a series of stakeholder interviews that were conducted by John Woods. While the members of the Roundtable processed the information, the information did not simply come from within the Roundtable. It had a richer set of inputs. There was also a consultation period, during which we received contributions from community-based organisations, including voluntary and arts organisations and the rural community. There were also high-level inputs, for example from the OECD.

On the tension between the renewables and the cost consideration, I suppose that we are talking about presenting that very tension, that trade-off. We want to allow people to be involved and help them to appreciate that there are always trade-offs, be it costs and policy preferences. We also want them to appreciate the context and maybe be alongside the political decision-makers a bit more in making those judgements and understanding why those judgements have to be made at times. We are not advocating particular policy points in our document; we are trying to suggest that it is more effective in the long term and good for society in the long term if everybody is in on the conversation and not just at the end points of the policy consultation.

Mr McQuillan: I agree that it is more beneficial if everybody is involved in the conversation. I have not seen that in the report, but I take what you are saying. I would not mind seeing the report from the rural community. I represent a rural community and would be very interested in that.

Dr Doran: Absolutely, we would be delighted to pass on the results of the focus groups to you.

Mr McQuillan: I would enjoy seeing them.

Ms Wallace: May I make an additional comment on that? It is important to stress that we thought long and hard about whether we were going to print the report and present it. It is also important to stress that the report is a draft. We have talked about the difficulties with the words "outcomes" and "well-being", and we wanted to show people what we meant by those. However, the activities that we undertook were not meant to be a large-scale community consultation. We were interested in presenting an idea and a starter. We hope that other partners, if they wish to do so, will be able to engage more directly with community groups on taking that forward as a finalised version.

The report is in no way a final version; it is a draft trying to communicate some of our concepts in a visual way. It is quite a tricky and complex area.

Mr McQuillan: I did not realise that it was just a draft either.

Ms Boyle: Thanks you for attending. It is a very good report framework. Peter, you spoke about scrutiny. In paragraph 8.2, you suggested that you see the Audit Office and the Public Accounts Committee having a role to play. As the Chair of the Public Accounts Committee, I want to ask about your about your point that the approach to the process and the work of the PAC can, at times, compound the very nature of the job that you have set out to do. The work of the Committee is all about value for money, and we focus a lot on the outcomes as opposed to inputs, like you. I would maybe take a slight exception to your point, but I understand where you are coming from.

I am the Chair of the PAC, and there are other members of that Committee on this Committee. I would be delighted to have you with us at any time to discuss the framework and how we can help to alleviate any issues that may arise from the process of the work. It certainly would not be our intention to compound any work or any framework. As I said, we are always interested in the outcomes.

The framework is about changing minds and lives. My opinion is that changing lives starts in the womb and in improving the life chances for our children, addressing inequalities and improving educational attainment, as set out in the framework. Where does that sit in the recommendations? At what stage would you be likely to see that work being done under the recommendations: under recommendation 1 or recommendation 2?

Dr Stuart: Jen made the point that the chart is illustrative and does not pre-empt what the Executive would agree to be the outcomes. We have described a framework and what the outcomes might be like. That is largely based on the Scottish model and other measures, and Michelle has explained how you would go about measuring them. However, the Executive could, for example, say that a particular form of words is not what we want to focus on and that they want to focus on a different outcome.

Ms Boyle: And on early intervention?

Dr Stuart: That is something that the Committee has talked about many times. Some outcomes might be based on that focus. Those are just illustrative, and a process would have to happen for the Executive to agree what the Northern Ireland well-being framework would look like

Ms Boyle: The convergence of councils and different Departments, which will apply here in the future, will help the framework.

Dr Stuart: Yes, one of the recommendations was that it is a whole-of-government approach. The outcomes are focused on what it looks like from the citizen’s perspective. Different Committees might see the same dashboard indicators, because they have a contribution to make. As you said, early intervention in children can, for example, affect justice outcomes many years later, so you might invest in that. So, the budgets have to follow the outcome as opposed to departmental lines. It is a new way of thinking really.

Ms Boyle: It is a new way of thinking, but this is how it should have been in the first instance; there should have been cross-departmental joined-upness. We are delighted to have you with PAC at any time to see how we can assist in the process.

Mr McCallister: Jennifer, when you talked in your opening remarks about the vision and shared narrative for government, I wondered whether you had actually met the Northern Ireland Executive. Vision and shared narrative would not sum up my thoughts on it. I suppose that I am allowed to say that; you are not allowed to comment on it. At the most basic level it is about how you make government relevant to people’s lives. As politicians, that is good for us in that it engages the public. We could be looking at an election on Thursday week with 50% turnouts in many constituencies. How do you make it relevant?

Bernie’s point about budgets chasing outcomes or following outcomes has to be key if you are changing government. In your experience of being in different Departments, you will know that there is far too much silo mentality in government. It is about protecting budget lines rather than looking at where the outcomes are going. Your example of early intervention is well made. The benefits will come not only in education; they will be in health, justice and right across Departments.

It will be a huge shift in culture for a Northern Ireland Executive and Assembly to have the confidence to start down that road. How do you see us taking the steps to get to where you think we should be? We are not going to do it just in one go; there will be a process of beginning that type of engagement and change in culture.

Dr Stuart: You are right. Changes in other countries have taken some years to come to full implementation. When it goes to the Executive, as Michelle said, the general domains will be common to most civilised cultures, but it is what you focus on and what changes you have to make. It will be a gradual process.

One of the interesting things to come out of the work, as far as I have looked at it, is the focus on keeping the long-term outcome measure constant. In the past, and Daithí mentioned gaming in the past, sometimes if you measure an output and it is not looking very good, you will say well, "That does not measure what we want to do; we will measure something else instead", and it shows a better picture and it goes down and you change it to something else.

The kind of open framework that the well-being framework has is that you have the measures set and people can see the direction of travel, and it might have a downward trend for a bit or it might go up, but you keep the focus on the long-term aim. It is more difficult if you have proper measures in place to change it.

You mentioned scrutiny. Scrutiny has to take a different form because you are not saying, "Why did that go down last month and it is going up this month?"; it is more, "Are we still achieving the long-term objective and are the interventions that we are doing putting us in the direction that we want to go?".

Mr McCallister: It is more looking at the trend rather —

Dr Stuart: It is looking at the trend rather than the individual numbers from month to month.

The other thing is that if you do a certain intervention over a period of two years and you are having absolutely no impact on the direction of travel, you need to stop doing it and do something else.

Mr McCallister: One of the comments that I heard recently at an event was that we are not good at "decommissioning"; that is, we are not always good at decommissioning services that are proving ineffective, so we stick with them. It ties in with how we get better at fessing up and saying, "This isn't working; let's change it". The Government as whole are not good at doing that.

Ms Wallace: On that point, there are examples of appraisals for new policies using the frameworks. One of our arguments for having a well-being framework at the heart of government is to tackle that point because, as new initiatives and new legislation come forward, you can appraise them on what impact they would have on well-being; but the vast majority of our public services would deliver the same thing in the same way continuing within that. So, we need to go back and look at what we are doing, how we are delivering it, and whether they are meeting needs. That will mean stopping some activities, but it also requires quite a strong evidence base, which is why some of this work is important. Those of us engaged in this work have to have confidence that those decisions are correct, so there are layers of evidence gathering and appraisal that go on underneath this rather than a framework that has no depth to it.

Ms Furphy: I can add a couple of examples of the sort of appraisal that Jen mentioned of how some UK Departments have used the UK measures. The Department for Transport has used the domains and the well-being framework in a tool to help policymakers to assess the social impacts of major transport investment decisions. The UK Airports Commission has also used it to inform its quality-of-life assessment for a third London runway. Those are just some of the practical examples of how Departments use the frameworks in decision-making processes.

Dr Doran: The response to your question lies in your phrase "whole of government". Our co-chair, Aideen McGinley, is very fond of the phrase "a systems approach". There are two parts to that. If you are taking an incremental approach rather than a mid- to long-term direction of travel, if you have not set yourself mid- to long-term goals, incrementalism can be quite dysfunctional. Moreover, if you are working in Departments, how you imagine the challenge can be quite limited if you are not working alongside people from other disciplines, other Departments or even people outside the Department.

Part of the culture shift is to look at how some of what are regarded as entirely rational decisions at the micro level in the system can be dysfunctional when they are looked at from the system-wide perspective. This is a developmental conversation; it will not happen overnight. It is about communication, embedded practices and the cues that the system gives to people in the system that might not serve the system well. That is the point that we are making about scrutiny.

There was a sense that, at certain points in the scrutiny process, the culture tends to reinforce risk aversion rather than innovation. It is just an invitation to reflect on that and to begin to look from a system perspective rather than simply allowing what we have to continue without deep reflection happening in the system.

Ms Wallace: If I may build on that, I did the international work for Carnegie a couple of years ago and travelled to Canada and the US. In Virginia, which has had a system very like this for a long time, they talk about the pursuit of "Why". This data, in itself, does not give them an answer; it is not a magic bullet. However, it helps them to answer the questions, "Why is that happening? Why is that going up and that going down? Are these things related? Is there something else happening in society that we need to be concerned about?". It helps with that conversation, that pursuit of "why" and their experience and also in the Scottish experience.

Mr McCallister: I take Peter's point about risk. As in all things, you have to find a balance. Everyone is frightened of ending up before Michaela at the Public Accounts Committee. [Laughter.]

That can lead to questions about whether we have got the balance right, and, maybe, we should keep asking why.

Mr Girvan: Thank you for the paper that you presented here this morning. I take on board the comment that if we do something for two years and find that it is not working, we need to revisit it. That must be a novel idea because we have fired millions into so-called deprived areas that are still deprived after millions have been put in. There is no more money going into other areas. In certain communities it would not matter how much you threw in because you will never get the outcomes. We continue to replicate that in our European funding priorities and we fire money in. It is not just small money that we are talking about; a large percentage of our European money has gone into certain communities that have always been classed as deprived, and they are still deprived. If you go to those people they will say that nothing is being done for them and they get nothing. I do not know whether it is a culture or whatever.

I want to go back to paragraph 8.2. I looked at the names of the Roundtable panellists, some of whom have appeared before the PAC for very good reasons. I wonder whether that has had some influence on the outcome. They were before the PAC for very good reasons, not just because of wrongful decisions but because of the waste and wrongful spending of millions of pounds of public money — not hundreds of thousands, but millions. It would sometimes be pertinent to look at those reports and find out why they were put out in relation to some of those bodies as opposed to accepting the view of certain individuals who have a personal axe to grind or a chip on their shoulder — both shoulders — because somebody has challenged them.

This country has a process- as opposed to an outcome-driven system: as long as you follow the process, you can do whatever you want. It does not necessarily mean that you have to have the outcomes, and that is where I have a big difficulty. We do not have an outcome-focused approach. I hope that by going through this mechanism we will end up taking away some of that approach of having a process with no outcomes. We have created an industry of people who spend their time analysing and coming up with ideas about where our shortfalls are. Everyone knows where our shortfalls are. If you read this report, it identifies that you can nearly lift from what has been done in other regions. You can say that health is a very big area in which there are major shortfalls, but it does not mean that we are not throwing enough money at it; it just means that we are not using our money correctly or that we are going to have to redo certain things.

I welcome part of the report, but I would like feedback on some of the content, which I believe has been based on personal prejudice. Is there any way that that can be screened out of a report?

Dr Doran: Your conclusions are entirely in line with our own thinking. We want to shift from a process-fixated culture to one that is more respectful of the need to learn from experience and make the hard decisions. Experience does not suggest that we are not doing the right thing. Your comments remind me of a conversation that we had with John Swinney when he gave us the history of the Scottish framework. It was not just about the Scotland performance framework but the deep changes that they made to their departmental structure. They deconstructed Departments and installed one single budgeting officer; they changed practices and got people to collaborate and think differently about policy design.

Mr Girvan: That is not even easy to do in our culture. Once you introduce change, we have all sorts of problems. I am not just talking about it from a workforce point of view; the unions and everyone else are involved. As soon as you say that we are going to deconstruct, that what you see is not what we will end up with and we are going to have complete change, you will end up with placards and protests at Belfast City Hall every Saturday morning because you want to change certain things. That is a culture that has to be embraced and worked with to ensure that we all have the right reasons for going forward.

We are not here as an employment agency; we are here to deliver services. We are to ensure that that service is delivered effectively and efficiently, and we are also trying to use whatever public money we have to identify the social areas that need help and ensure that outcome. I would like to do what you have just outlined; unfortunately, I know that it would come up against extreme public opposition, primarily from within the industry.

Dr Doran: It was not so much what they did, but what triggered the process in Scotland was the very point that you were making: they realised that they were investing a lot of resources in policy aspirations that were simply not being realised and that they could not continue. I think that there is some public sympathy at least for the need to shift resources to where they are most effective and where there are visible outcomes. If the public are more engaged in the processes of deliberation on those decisions, that is part of the well-being piece as we see it. When people have more control over their lives and more control over decision-making, they have a greater sense of where decisions are coming from, and they will be much more tolerant of the decisions if they are participatory.

Part of the piece as we see it is that, if local government and community planning aspirations are more fully reflected in the process and the PFG, you develop a more sympathetic and literate public. I guess that you develop a learning society around the learning organisation that we want government to become. You are right that none of this will happen overnight, but it is about different expectations from us as well as from the public, who you might imagine at this stage, if we were to clear some of these changes, might see them as a diktat from on high. We are talking about a more collaborative and horizontal model of governance with greater insight all the way from design to delivery.

Ms Wallace: Our Enabling State programme at Carnegie covers a lot of very similar ground; in fact well-being is one of the key recommendations from that work as well as from this. If you take a slightly longer-term view of what we are delivering with public services, you can see that, in the post-war era, we were able to improve people's well-being by giving them something. We gave them education and health services, and that had a transformative effect on our societies. In the 21st century, we cannot do the same thing; we have to work in partnership with people. Their well-being outcomes, which we have been talking about, are interrelated. We can no longer deliver something to somebody and expect them to have a radical change in their well-being, so all we see are small, incremental increases. We increase budgets all the time, when we can, but we do not see the corresponding transformative effect on people's well-being.

Mr Girvan: Well-being can sometimes be a perception among individuals. Some people feel that they are perfectly happy and they might have absolutely nothing. Your interpretation of their well-being and their interpretation of their well-being are two totally different things. I know some people who have absolutely nothing of this world going for them, yet they are happy and content. They do not want any change; they just want it that way.

Dr Stuart: One of the things that we need to be clear about in measuring the wheel of well-being and so on is a common definition agreed with everybody from the general public to politicians to government on what is important to people. Those measures are very important in making a decision on a future policy, because, if you are contributing to the agreed definition of well-being, it is more visible to everybody. Jen mentioned the visibility to people of the whole picture. If you can see that although education provision in your local area may be moving to somewhere else it will be better overall for children, you will be happier with it, even though it might have a negative impact on, say, your daily journey to work.

However, if you are getting something that contributes positively to your life overall as measured, you will be happier with it as long as it is visible to you. I think that it will be very important for the community to see what the whole picture is like.

Mr Girvan: I am always very conscious that a great majority of individuals out there do not engage with anybody; therefore, their voices are rarely, if ever, heard. That is why a majority of people who do not engage a lot tend to live their own life. However, if you have a very strong and militant lobby that actively goes out to engage and to thereby change not only government policy but legislation to protect itself, how is protection given to the weight of others' views? I do not believe that that is properly measured. You could have 1,000 people who wish to write in about one issue, but you might get only 10 writing in about another, yet the 10 are probably more important than the big lobby group for the 1,000. How is that measured and equated? You cannot just take the blanket view that says, "We received so much communication about this issue that we had to change the policy to ensure that that was dealt with".

Ms Furphy: The Office for National Statistics' well-being framework means that we have a series of those very questions, such as the individual's own perception of their well-being, happiness or satisfaction with life, their anxiety states and so on. We have those questions in our Northern Ireland surveys. They are for the general population, so hopefully in our household surveys we will pick up some of those who do not engage with government per se. This is obviously a household survey, but we will merge the years, so when we have enough data, we will find that those are the sorts of issues —

Mr Girvan: How are those households identified?

Ms Furphy: They are randomly selected by address. When we get to the stage where we have enough data, we want to be able to look at what makes some people happier than others and at the sort of things that influence them. You may find the case to be the very point you made, in that those individuals who, from an objective perspective, seem to have very little can be higher up that happiness chain.

Mr Girvan: It is a state of mind.

Ms Furphy: Across the UK, those questions have been added to over 20 government surveys of policy by looking at areas such as health, crime, housing and taking part in sport and culture. That is so that they can look at this question: what is the impact of our service on individuals?

Mr Girvan: Do you ask how many happy pills they have taken that day and whether that gives you a good response? Honestly, I am a sceptic about some of these things. If you look at statistics as they are presented, you would believe that we are in an area that is deprived of all sorts of functions and everything else, and that, health-wise, we are probably on the back foot in contrast with the rest of the United Kingdom. Yet when you look at the statistics that are used to measure the calculation that doctors and doctors' surgeries use for their drawdown for their area and the quota required — that was something that we studied — you see that, in many areas, we score much higher for health than any other part of the United Kingdom. We sat back and said, "But that's not the story we have been told. We have been told that we've been suffering because of the Troubles. We have more people on antidepressants and this, that and the other". The statistics did not bear that out. Yes, the prescription writing did, but the statistics did not. That is the point.

I know that we have to engage in this process, and we have to get it, but I still think that we also have to be conscious of those who maybe try to skew the outcomes and outputs through a lobby campaign or by putting forward a group to drive government policy down a route that means that we end up with us prioritising things that are maybe not necessarily the most important things in a community.

Ms Wallace: Where the community consultations that we are recommending have happened internationally, they have been done to engage the views of thousands of people. That is quite a different model from a paper-based consultation exercise where you might get responses from umbrella bodies and special interest groups to try to access the views of some of the people who you are talking about. The difficulty is that you then have to triangulate that with some of the expert views of statisticians on what is measurable. So, there is job of work to do to bring that together into a coherent framework.

Mr Girvan: I always have a fear of academics trying to tell us what is important, because sometimes their view of what is important is about self-protection as well.

Ms Wallace: An additional point is that, where this is done around the world, most of the indicators blend objective data like exam results with perception data, such as how you feel about your own levels of literacy or education, with the subjective well-being data you are talking about, such as happiness. It is a whole block, rather than just one area.

Mr Girvan: Can I ask about those who had input to this? I believe that they possibly had a chip on both shoulders about the outcome.

Ms Boyle: You are not going to name names.

Mr Girvan: I am happy to. There are three of them on the front page.

This approach is compounded somewhat by some of the work of the PAC and associated media coverage that is often adversarial rather than focusing on the shared endeavour. It has a specific role and a job to do, and it must do it. If some of those people felt that that was vital to include in the report, I think that their priorities are far from right. I am saying that because I do not think that it should be.

Ms Wallace: I will make two points. First, the long document was the document by the secretariat, so John, Peter and I wrote that. It is not necessarily a consensus document by the group. The short report with the recommendations is the consensus and the group's agreed view. We took evidence. I do not know where that one came from; if it came from evidence, I could not tell you. My reading of it and my view when we were writing it was that that adversarial comment was more about the media coverage that it generated, so I would just temper that. I appreciate that perhaps we could have written it slightly more carefully, if that were the case.

Do you want to add anything?

Dr Doran: It is about the way in which the media amplify some of these bodies' findings.

Mr Girvan: The media, by nature, do not like good news stories; therefore, they focus on everything. Unfortunately, something may come out that we do not necessarily want to be made public; we want to ensure that lessons are learned from it. That is one thing, but the media jumps on it because it likes to say, "See those guys up on the hill there; they haven't a clue what they are doing. This is what is happening, and they are allowing it to happen. Look at what happened on their watch". Some of those reports are not always good for us, and they show flaws in governance issues associated with us. That is fine, but it is right to be highlighted. The point is that we deal with a media that is very much not our friend; it is actually trying to put us down on every occasion. If you wanted a profession or a calling, you would not want to be a politician, because you leave yourself open to all sorts of abuse day and daily. Therefore, it is not necessarily the sort of thing that the media wants to portray as positive; it wants to put that across. I think that it should be geared more towards the media's perception. It never sells many good news stories about Northern Ireland. You have to go out and try to ensure that it covers those. Even then, you might just get a small mention and a focus on something irrelevant.

Dr Doran: Yes. We were making a fairly modest point. We would never dream of questioning the need for rigorous scrutiny and for naming the issues when they are identified. Our point to the Audit Office was that it would be useful to find ways to balance that part of the role with institutionalising the learning that needs to be captured when you have these reports and processes. How do you disseminate the learning and ensure that it is not going to happen again?

We talk about a learning organisation so that you do not have cues that might even be unintentional, whereby the functions and practices of the auditors signal that they want to talk to you only when things go wrong. It should be a continuous conversation that encourages capacity building and disseminating good practice and that helps those who get it wrong to get it right next time and to ensure that they do. It is about more continuous engagement with the organisation as a whole. It was a very modest point that we were making, and we certainly were not questioning the need for rigorous scrutiny of the issues where they needed to be scrutinised. This is about effectiveness, as well as well-being.

Ms Boyle: Following on from that, it is about having the confidence to take risks and to manage them. Paul made a point about policy change. I just want to comment on that; you do not have to respond to it. Where policy change is concerned, we can look at Transforming Your Care (TYC) as an example — love or loathe it, but we have it. My opinion is that there are aspects of it that are very good. However, the funding to follow TYC has, in my opinion, not really happened, and communities are well tuned in to that subject. I have been canvassing on the doors for the last few weeks, and I have heard that from the individuals concerned. The funding aspect of policy change has to happen, and, in my opinion, it has not happened with TYC for care in the community. So, funding is also a big aspect. I just wanted to comment on that.

Mr Cree: I was not going to speak at all. I appreciate the good work that has been done, but my big difficulty is in identifying what is process driven and what are outcomes. We have a major problem, because process wins on most occasions. The thing that concerns me is how you apply objectivity to the whole exercise. Obviously, you have to be objective. In my opinion, the trouble is that the responses that you get are, in fact, subjective. It is about what people feel. You then have the dilemma, which Paul mentioned and which I identify with as well, of loud voices, organised society and insatiable demand. For example, Jennifer mentioned that they are rewarding people. That is OK, but if you live in a material society now with an insatiable demand, how can you reconcile all those things, deliver the greater good where it is needed and not simply be part of a process that you will still be doing in 25 years?

I have travelled a fair bit in my time so far, but one of the most remarkable things that I learned was in Malawi. There are not too many people in Malawi, but I have never known people as happy and content. They have nothing — absolutely nothing. This material thing concerns me. I do not know how you can see the wood for the trees in this one. Are all the responses going to be from people who want more of this, more of that and more of the other? How will you be objective? How are you going to appreciate that these things coming to you really should be subjective?

Dr Stuart: I think that some people maybe interpret the word "well-being" as being just for subjective measures. Michelle's presentation illustrates that once the Executive, local government and the citizens have agreed the outcomes that they want, measures are pinned down. Some of those are objective measures through surveys, but others are tangible measures. We will never be doing away with the measure of GDP, which is factual. There are factual measures as well — Jennifer mentioned one — that indicate —

Mr Cree: May I interrupt you there? Why GDP? Why not GVA?

Dr Stuart: GVA or GDP; it is a tangible measure whichever way —

Mr Cree: Which one?

Dr Stuart: GVA is used now, but people still refer to it as GDP. GVA is the more modern measure, but it is a tangible number that you can get to. If you combine a number of subjective measures with objective measures but know up front what you are going to be measuring, you can get an indication of whether you are going in the direction of travel of the outcome that you require.

It is not just about how people feel about it. It will be difficult to do — there is no doubt about it — because it is, as many of you said, a big change and a more holistic look at things. However, if you move in that direction, it is clear to people what they are trying to get and it allows you to stop things. Michaela mentioned Transforming Your Care, for example, but its objective is to improve people's longer-term health. If some of the measures that are put in place are not working, it is all about having a measurement base that says, "Actually, we are not improving it by doing that, so we can change it". It is all based on evidence, not subjective feeling. I think it will take a lot of time to bed in. It will take a cultural change. People are not going to accept it easily, but if they are involved in the process from the beginning, I think that it should end up with a better outcome. Do you want to add anything, Jennifer?

Ms Wallace: We talked a bit about domains of well-being, education, housing and so on. In my view, it is almost impossible to conceive of a well-being framework that does not have outcomes in each of those domains. Those are internationally recognised and are well known. The local level comes in as the level underneath that. What is a good education in Northern Ireland? What is the level that people are comfortable with? What is safety? Feeling safe in Scotland, for example, is quite different from feeling safe in India. You might want to use different outcomes and different measures for that. That is where the community consultation comes in. What does the community understand those outcomes to be in their local area? It is not just about saying that we want only education or health but about how you understand those things in your local context. When we talk in Scotland about doing one of those large-scale consultation exercises, we are talking about starting off from the base of where people are, because that is what they understand. What do you need to live a good life? Moving up a rung to —

Mr Cree: Just on that, when you ask, "What do people need to lead a good life?", is that a purely material measure again?

Ms Wallace: No.

Mr Cree: How do you measure the psychological?

Ms Wallace: There are well-known measures. In Scotland, they use the Warwick-Edinburgh mental health and well-being survey, which is also known as WEMWBS for short. That short survey comprises 10 questions that ask about your anxiety levels, your satisfaction and your social networks and then gives one score. Then you aggregate that up to society level and say that, for Scotland, we have a level of well-being of — I cannot remember the figure, but it gives you a figure.

Mr Cree: How do you weight that? It cannot be the same in all parts.

Ms Wallace: You aggregate it; you do not weight it. It is just a simple index, not a weighted index.

Mr Cree: It is just an index.

Ms Wallace: That is for WEMWBS, but that is only one example. The ONS's four questions are a different example of how to do it. There are different ways of getting at the subjective, but, again, the subjective is only one element of all the different indicators that you would want to put into a well-being framework.

Mr Cree: You said:

You have to watch that you do not end up superimposing your own views. That would worry me as well. Maybe it is a discussion for another day, because we are getting into the abstract, but it is very important.

Ms Wallace: It is.

Mr Cree: It has to be got right, or it is meaningless.

Ms Wallace: Another two levels of questions were under consideration, although one was for individuals. Largely, that is about understanding and helping people to get into a conversation about well-being. The second-level questions are about what Scotland needs to flourish or thrive. "Flourish" is language that we have been using in Scotland for quite a while. You are asking people to move slightly further away from themselves and into a Scotland-wide setting. We have been trialling that in Wester Hailes, which is an area of reasonable deprivation outside Edinburgh. We have found that, actually, people can make that jump pretty well. Thy can move from saying "what I need" to saying "what we need as a society" in a managed and facilitated session. That is what we recommend in moving forward and in having some of those conversations so that we can avoid some of the pitfalls that you are raising about how we can move that forward in a meaningful way.

Ms Furphy: I come at it from the perspective of the next stage. Once the outcomes are identified, how do you identify how to measure them? If you look at how ONS approached that, you will see that it developed and published a set of criteria on how it would select the measurement tools. That is the sort of model that could be interpreted where it is transparent what the criteria have been for inclusion.

Mr Cree: You need to continuously validate that response.

Ms Furphy: Do you mean the criteria themselves? One of the criteria was whether the statistic is reliable and collected over a period of time, as well as where it was available. There was a whole range of different criteria, including the geographical levels that it was available for and so on.

Ms Wallace: More recently, the OECD has published criteria that are similar to the ONS criteria but slightly more up to date.

The Chairperson (Mr McKay): Obviously the Minister is to bring this to the Executive for consideration. Do we have any indication of the timetable for that? Does it have to be done before the summer? What is the timetable for the Programme for Government?

Dr Stuart: It is not set yet, but the Programme for Government timetable is imminent, as you know. It will be as soon as possible, but we have not got a date yet. He hopes to bring it before the summer, timetables being agreeable to that.

The Chairperson (Mr McKay): What are the next steps for Carnegie with this process in the months ahead?

Ms Wallace: Peter mentioned the conference, which we are hoping will be a quite a large and public event in which we will try to gather some more views. We will take stock after that. Peter Doran and John Woods, who are both Carnegie associates, have agreed to stay with the programme until the end of this calendar year. We will continue with stakeholder meetings, presentations, advocacy events and engagement and see where we are and take stock closer to the end of the year. As I said in my introductory remarks, the Carnegie UK Trust is committed to this and to continuing its relationships and partnerships in Northern Ireland on well-being. If there are things that, you feel, we can do to assist, please come and have a conversation with Peter, John or me.

The Chairperson (Mr McKay): OK. Excellent work. Thank you very much for that.