Promises to Keep

Syed Akhtar Mahmood

The political parties announced their election manifestos with much fanfare. One wanted to change the country, another to save it. When the manifestos were announced, there was one question on most people's minds: Will the parties really deliver on their promises if they are elected?

We do not know the answer but this much we know: if we want them to deliver, we have to hold them accountable. And, for this we need a system for monitoring and evaluation -- monitoring the implementation of the promised actions and evaluating the impact of these.

There is another reason why we need such a system. Many of the promises are expressed in general terms, such as "freeing the education sector of political influence" (BNP) or "creation of a digital Bangladesh" (AL). Noble goals, no doubt. But how will one achieve these?

Ministers often get away with broken promises because their commitments are expressed in vague and rhetorical terms. Accountability thus requires a system for converting vague, general promises to precise indicators that can be monitored. What are the specific actions that will need to be taken, and in what sequence? A good monitoring system helps unbundle the general objectives into specific monitorable actions.

Monitorable indicators could be used for more than just checking on government promises. These could be used more generally to evaluate progress in implementing government programs and projects and their development impact.

Q Zaman/ Driknews

Consider, for example, a think-tank or other academic institution funded by public money. Rarely do we ask if these institutions are giving society good value for its money and doing what they are supposed to do.

Of course, monitoring project implementation is not easy; even more difficult is evaluating project impact. Nonetheless, it is worthwhile to develop monitorable indicators because the exercise itself could help clarify thinking on how projects may be designed. This means monitoring is not a thing you leave for the future. You think about monitoring right at the beginning, when you design a project or a program.

Organisations who regularly use performance indicators have found it useful to distinguish between four types of indicators: input, output, outcome and impact indicators. An example will clarify the relevance of each indicator and the distinction between them. Consider the yearly budgetary exercise. Across the globe, finance ministers often point to budgeted expenditures on the "socially-important" items, such as education and health, as indicative of their government's commitment to social causes. More sophisticated finance ministers go further and focus on particular categories of social sector expenditures, such as primary education and public health. They then try to show that these expenditures constitute a significant portion of the budget and/or are rising over time.

Do all these necessarily mean that the government has attached top priority to education and health? No, it does not. A finance minister who makes such a claim is only partially correct. There is more to this than just putting aside money in the budget. This is better understood if we apply the four indicators mentioned above.

Let us begin by asking the all-important question: Why do we argue for an increase in expenditures in the social sectors? It is not the expenditures per se that is important but what we hope to achieve through these expenditures. This brings us to a critical, but oft-neglected, dimension of any government program or project; i.e. its objectives. Why are we doing what we are doing? For example, why do we argue for financial sector reforms? What is wrong with the financial sector as it is and what do we intend to achieve through the reforms? Or, to take another example, if we are putting public money into an institution, such as the Bangladesh Institute for Development Studies (BIDS), the relevant questions could be: Why do we need a BIDS? What is its mandate? What is it supposed to do? Clarifying the objectives of a project is so critical to identifying monitorable indicators that half the task is done once the project or program goals have been spelt out.

Let us go back to the example of public expenditures on education and health. The objective here is not spending the money per se but something else, such as improved access of the poor to education and health, and better quality of education and health services.

Why do we want these? Firstly, because these are worthwhile by themselves; an well-educated and healthy population is itself an indicator of development. Secondly, because human development, in turn, facilitates other dimensions of development. In order to achieve our goal of wider access to, and better quality of, social services, we may need to build more schools (or hospitals or clinics), provide them with better teachers (or doctors or paramedics) and equip them with the necessary supplies. And these, in turn, may require greater allocations in the budget.

Although we did not put it in those terms, the above paragraph actually provides a real-life example of the input-output-outcome-impact indicators framework I had alluded to earlier. Increased budgetary allocations to education or health are like inputs. They are intended to produce some outputs, such as more schools, better teachers or greater supply of the necessary equipment, books, medicines etc. Some of the output, such as number of schools, can be measured quantitatively; others, such as quality of schools, may have to be evaluated qualitatively.

However, achieving these outputs is not the end-objective. We want these because we believe that these will lead to better access of the poor to social services and improve the quality of these services. These are the outcomes we expect from the increased budgetary spending on education and health. And we want all these because we expect these to have an impact on development. The distinction between output, outcome and impact is important. Failure to comprehend this could be the cause of much confusion in the design and evaluation of many a project. Real world experience tells us that such confusions are common.

In real-life projects, the inputs may not necessarily lead to the outputs, the outputs to the outcome and so on. Thus, the mere allocation of additional money in the budget may not lead to more schools and clinics, more books and medicines, and better teachers and doctors. Money may be spent inefficiently, e.g. siphoned off by corrupt contractors and government officials. When the inputs do not lead to the desired outputs, you have a deficiency in project implementation, which is different from poor project design.

But even when the money is well-spent, we may not see the desired outcome. We may build more schools, and provide them with better teachers and good supplies, but discover that all these made little difference to the poor. This may suggest that it is not the availability of schools, but other factors, which are the real hurdles to the poor's access to education. If the latter is indeed our goal, then these factors should be addressed in addition to, or instead of, building more schools. This is an issue of project design. A project may be efficiently implemented and the project outputs realized, but if the design was deficient, the desired outcomes will elude us. The output of a project is thus not the same as its outcome.

Nor is the impact of a project the same as its outcome. The output and outcome indicators tell us what is expected from the project, the first in a narrower sense and the second in broader terms. The impact indicators place the project or program within the bigger scheme of things, relating it to the ultimate development or poverty alleviation goals. In our example, expanded access of the poor to social services is believed to have an impact on development. There is little controversy on this.

Shafiqul Islam Kajol/ Driknews

However, one can think of projects where the link between the outcome and the expected impact on development is less clear-cut. For example, a project may involve the hiring of consultants to write a draft privatisation law. Here the draft law is the expected output. An acceleration of privatisation, resulting from the adoption of the privatization law, could be the expected outcome. Proponents of privatisation would argue that an acceleration of privatisation would have a significant development impact by increasing efficiency in the economy. Opponents of privatisation are likely to disagree. In many cases, project designers need to think through clearly the link between the expected outcome and the expected impact of the project.

To conclude, proper accountability requires that we convert promises to precise monitorable indicators. But it also means that we clearly distinguish between the input, output, outcome and impact indicators. If we do so, we will be better at evaluating government projects and programs. We will be less impressed, for example, with mere increases in budgetary allocations to a particular sector, however socially important that sector might be. We will want to know if this additional money is efficiently spent and, more importantly, whether it really makes any difference to our developmental goals. A government will no longer get good marks just by showing that it has spent more money, irrespective of the impact on growth or welfare. This, in turn, will force governments to think more carefully about the objectives of any government program or project.

Munem Wasif

In this article, I have largely used one example of government programs, i.e. budgetary expenditures on education and health, to illustrate my arguments. However, much of what I had to say applies to other programs or projects, be it a highway project, a bridge, a financial sector reform program, a fund set aside in the budget for sick industries, or a think-tank funded by public money. And it applies to loftier goals I had alluded to earlier, such as freeing the education sector of political influence or creating a Digital Bangladesh.

The bottom-line is clear. Until we develop a culture of insisting on precise monitorable indicators, we will not be able to ensure accountability on the part of anyone who uses public resources.

And we will not be able to know if a minister has indeed kept his, or her, promise!

Syed Akhtar Mahmood is an economist.