My keynote at the 6th Beijing Forum on ESD was about the problem of identifying sustainable development indicators that focus on the contribution of education; that is, ESD. This is the text of the talk ...
The Development of ESD indicators – exploring frameworks and criteria
The issue of writing indicators for both sustainable development and for ESD is one that UN agencies and national governments have struggled over for some time. In terms of sustainable development, this is particularly important if we are to know whether we’re becoming more sustainable – or at least less unsustainable, and whether policy and strategy need to be changed. This is a significant challenge, but a necessary one.
The use of indicators is now widespread, but it carries considerable risk. Both the rationale, that is, the need for good information, and the risk that we will measure the wrong thing, are noted here:
Where there is no reliable accounting and therefore no competent knowledge of the economic and ecological effects of our lives, we cannot live lives that are economically and ecologically responsible. Wendell Berry
That which is good and helpful ought to be growing and that which is bad and hindering ought to be diminishing. ... We therefore need, above all else, ... concepts that enable us to choose the right direction of our movement and not merely to measure its speed. EF Schumacher
We try to measure what we value. We come to value what we measure. Donella Meadows
The challenge is just as great in relation to ESD, as it is for sustainable development itself, if we’re serious in thinking that ESD can make a contribution to sustainable development, as well as to student learning. Conceptually, this is all complex. However, it is something that practitioners, school leaders, and policy-makers need to consider as they promote ESD beyond the Decade in the global action programme for educational reform and development. I shall explore these issues and the criteria for an ESD school indicator framework based on both theoretical and pragmatic perspectives.
I’ll begin by saying something about indicators in general before moving on to sustainability and ESD as, what you can say in general must also apply in particular to ESD indicators. Indicators can …
- simplify or distil complex information
- monitor or account for the performance of systems
- measure the state, direction and rate of change
- act as a ‘warning’ system
- raise awareness and communicate information
- stabilise processes – particularly implementation
- aid accountability, governance, decision-making, dialogue
- address stakeholder interests about the state of the system
Indicators summarise information, measure change, raise awareness, warn of problems, aid communication, and help decision-making, and need to be written with clarity, and to be developed systematically. This is the logic to their generation ...
- Indicators have to be linked to desired outcomes (which may be short- or longer-term). They indicate the extent to which outcomes are met
- Such outcomes have to be linked to particular purposes (objectives)
- Purposes (objectives) have to be linked to overall organisational aims / vision / mission
- Key evaluation questions have to be posed, where indicators answer such questions.
To evaluate is to form a judgement about progress made towards purposes, and indicators help us understand the value of such judgements. We use indicators to show the extent to which desired outcomes are met. In our professional practice, the idea and use of key performance indicators is now commonplace. But this does not make their use easy, or uncontroversial.
While it’s important to have indicators, it’s also important not to accept uncritically what they appear to tell us, particularly in contexts where complexity and uncertainty are the norm. Let me illustrate this. The UK had 68 sustainable development indicators before they were changed 6 months ago. The indicator for education was the proportion of 19 year-olds with Level 2 qualifications and above.
This is easy to understand, and to measure, but what does it really tell us? Probably, only the proportion of 19 year-olds with particular qualifications, as there’s no obvious link to whether we’re developing on a sustainable pathway. The new indicators remove this problem as they don’t mention education directly at all.
These are the 12 headline indicators that are high-level outcome measures which capture priority issues in line with the ‘guiding principles’ of sustainable development. These are organised in terms of economy, society, and environment, with 4 headline indicators each.
Economy
Economic prosperity / Long term unemployment / Poverty / Knowledge & skills
Society
Healthy life expectancy / Social Capital / Social mobility in adulthood / Housing provision
Environment
Greenhouse gas emissions / Natural resource use / Wildlife: bird population indices / Water use
There is an implicit reference to education as part of the knowledge and skills indicator under “Economy”.
Human capital stock & Human capital per head
Where human capital is defined as “the knowledge, skills, competencies and attributes embodied in individuals that facilitate the creation of personal, social and economic well-being” (OECD, 2001).
This is not straightforward. There’s no clear link to sustainable development, and no mention of ESD. It’s much the same as the old indicator. Although it is more sophisticated, it’s now much harder to make sense of – and to measure.
Because of this limited view of human capital, we find that a lifetime of education and training will count as a contribution to sustainable development, whether or not there’s any focus on sustainability – that is, any ESD. This seems strange. However, it also means that, even if there is ESD, it will not be acknowledged. This seems even stranger.
There is, of course, a much bigger problem which applies to all indicators. It’s been suggested that such headline indicators represent a sort of barometer or compass bearing. For example, if all the indicators can be lined up so that they’re all pointing in the right direction, then we can conclude that we’re making progress along the sustainable development pathway. It’s a pleasing, and reassuring, metaphor.
However, there’s a major problem with this view:
- If the headline indicators are broadly negative, we can tell that the overall position is not sustainable.
- Unfortunately, this does not mean that when they are all positive, the position necessarily is sustainable.
- Indeed, it is even possible that positive indicator results will operate perversely to move us off a sustainable pathway.
In other words, if you fail to meet an indicator, then you know there’s work to do; if you do meet it, however, there’s always uncertainty about how appropriate the indicator was in the first place. And there’s the risk that positive results will send the wrong message. For example, if we were to succeed in keeping the global temperature rise below, say, 3o C, we’d still have to wait for years to see whether that had been enough.
In this sense, using indicators is rather like looking in a car’s rear-view mirror. It enables us to see where we’ve been – to see how well we’re driving – to see something of the progress we’ve made. On a good journey, there will be a clear road behind us. On a bad day, however, we will see the crashed cars and the bodies in the road. The mirror alerts us to how we’ve been doing so far, but tells us nothing about what’s ahead. The point here is that though mirrors – and indicators – are useful, they are an imperfect guide to the future. You need to keep your eye on rear-view mirrors – to ensure that they don’t distort what has happened. The same applies to indicators.
Accuracy, timeliness, clarity, field of view, and ease of access, are all crucial qualities in an indicator (and a mirror), and the view you have should be both valid and meaningful. A constant danger is that you produce indicators that are easy to measure, but which lack intellectual and policy coherence. This is particularly true of ESD. The potential benefits and risks of ESD indicators are probably the same as for any indicator. These are:
Benefit:
- You can get (useful) information on steps taken, and progress made, against aims and targets
- You can then change policy / practice in the light of information
Risk:
- You may be measuring the wrong thing
- The information you get may be misleading
- It may take considerable time to know whether this is so
So much is straightforward, and applies to ESD as it does to, say, biodiversity.
Both of these need a conceptual frame if indicators are to be developed. With biodiversity, the link to sustainable development seems clear, but to what extent is this the case with ESD? This is the UK’s new headline indicator measure for bird populations:
Indicator measure: Wildlife: Bird population indices - farmland birds, (b) woodland birds, (c) seabirds and (d) water and wetland birds
Rationale: Natural capital includes those elements of the environment that yield resources and ecosystem services, but we cannot determine our entire capital of natural resources and instead have to focus on selected aspects of the natural environment and changes in its state.
The rationale here is that populations of key species of birds are a good indicator of the broad state of wildlife and the countryside because they occupy a wide range of habitats and have key positions in the food chain. This seems sound because the conceptual framing of biodiversity is robust. We’re using bird population measures to infer information about much broader issues – which is sound indicator practice we should bear in mind for ESD.
There is, for example, little ambiguity about bird populations, though there will be methodological issues in the estimation of their numbers. We are, of course, really interested in how these indicator values change over time, with upward trends being desired.
So what can we say about an indicator for ESD? We might start by asking about the relationship between ESD and sustainable development, and whether it is sufficiently conceptually robust? You will have your own views on this.
Here are two frameworks from the literature illustrating how we might both think about ESD, and research its effectiveness:
ESD is … Teaching which develops in students an understanding of the nature of society and its relationship with the environment, together with the capabilities and potential to promote justice in the distribution of economic, social and environmental assets, now and for the future.
ESD is … Teaching which contains a significant element of work related to either or both of the natural environment and natural resources, plus a significant element of work related to either economic or social issues (or to both).
The first of these is about students developing an understanding of the nature of society and its relationship with the environment, and sees sustainable development primarily in terms of increasing equity and global social justice. The second has a prime focus on the natural environment and natural resources, and sees sustainable development more in terms of reducing the environmentally unsustainable impacts of economic and social development. Are both these ESD? Can both be ESD? If so, are they poles on a continuum of ideas, or mutually inconsistent and competing views? Either way, any attempt to generate indicators for these two frames would lead to different outcomes.
A lot of the thinking about ESD indicators has been at the system level, and has been about structures and quality assurance. And there’s always a temptation to cover a lot of ground – to try to identify every aspect of what is being considered. But this is always a mistake as it results in so many indicators that they obscure rather than illuminate, as any meaningful signals are lost in the noise of communication.
We can think of ESD indicators at different levels. Three possible levels are: an institution, a course of study, and the actual learning that results.
The following sets out a number of attributes that might characterise an educational institution that takes sustainability seriously; that is, issues around sustainability …
- have a high profile across the work of the institution, and in its community links
- are fundamental to, and integral across, the institution’s work rather than being add-on or fragmented
- are raised in different settings, as appropriate, and treated as holistically as possible
- are focused on building students’ capability for critical and independent thinking for the future
As possible precursors of indicators, these seem fine, up to a point. But they are generic. They mention, sustainability, but are not obviously about it, and other ideas could be substituted: health, or enterprise are both possibilities. For example, if you replace sustainability by health & well being in the above, it still makes sense, and so it seems that this cannot be specific enough.
The risk in all this is that we have an approach to ESD indicators where they don’t really relate to sustainable development. So, perhaps more detail will help. The following attributes seem to be characteristic of an institution that is orientated towards sustainability.
A social learning community with a systemic view of the world and a heightened sense of place that …
- has a growing awareness of its environmental impact, and has a strategy for optimising the efficiency of its buildings and steadily reducing its need for natural resources and its creation of waste
- uses these as foci for learning and skills acquisition and to enhance social cohesion
- values outdoor, environmental, experiential and exploratory learning as a means of effectively engaging with real-world issues in authentic settings
- is outward-looking, where work in embedded in the local context (socially, economically, environmentally, and culturally), with tangible links to real communities across the world
Here we see a focus on environmental impact, on reducing natural resource use and waste, and on how facilities are used for learning and social cohesion. There’s a focus on outdoor and experiential learning, and on engagement with real-world issues in authentic settings. Here, even though sustainable development is not mentioned, these seem more obviously about it that the previous set.
So, what about the next level: courses or programmes of study? What can we look for in these? This is more tricky, although you might think it should be straightforward. Can’t we just look at what curricula and schemes of work say? This is an approach that is used in some parts of Europe. But is it really enough to see whether terms such as ‘sustainability’ are mentioned, or whether there’s a focus on the interplay of social, economic and environmental issues?
But just relying on counting things does not seem enough. After all, it’s what teachers do with such programmes that tells you whether the experience has a sustainability focus, and, whether the learner experience is any good. In a similar way, as well as estimating numbers, you have to ask how healthy and viable bird populations are. This brings us to the question of what people actually learn from all this.
So, can we say anything about how that might be framed? Here’s one possible frame for ESD learning that sees such outcomes in terms of Knowledge and Understanding, Skills, and Attributes:
Knowledge and understanding
- Understanding interconnections
- Natural systems and their limits
- Structures and societies
Skills
- Theoretical skills
- Putting theory into action
- Persuasion and conflict resolution
Attributes
- Attitudes & Behaviours
This is a conventional way of viewing such things although the sub-headings used here show a particular emphasis on understanding systems and interconnections, and on bringing about change.
The following is another way of thinking about knowledge and understanding that seems to show a different emphasis …
- Interdependence – of society, economy and the natural environment from local to global
- Citizenship & stewardship – rights, responsibilities, participation and co-operation
- Needs and rights of future generations
- Diversity – cultural, social, economic and biological
- Quality – of life, equity and justice
- Sustainable change – development and carrying capacity
- Uncertainty, and precaution in action
Whilst it seems possible to develop a large number of indicators based on all these, is it sensible? We’ve seen the problems with that sort of approach: you get a lot of information, but not much clarity.
There’s also another issue. Stephen Sterling warns us about being too focused on determining or predicting the outcome of an action or intervention, because of complexity and uncertainty.
"In complex situations, it is simply impossible to determine or predict the outcome of an action or intervention. We live in a culture interested in certainty, prediction, and control through setting measures such as 'performance indicators', and 'specific learning outcomes'. Yet increasing uncertainty means that we need to become more comfortable with ambiguity and approximation."
Sterling says that we need to become more comfortable with ambiguity and approximation. And learning is always complex, and its outcomes are not only uncertain, but largely indeterminate. Thinking about birds again, we might remember that a few key species were a good indicator because they occupy a range of habitats and have key positions in the food chain. Here, bird population measures allow us to infer information about much broader issues. Can we use this approach for education and ESD?
Could we agree on a few key aspects of an educational experience that would allow us to infer information about its effectiveness in relation to sustainability? If so, what should we look for?
Stephen Gough has argued that, because sustainable development necessarily implies a change in the models we use to live by, then any ESD programme should challenge these models by asking questions about how we live – and might live. He said that we should try to ‘catch this change’ – the dynamic in the educational process, and focus on the choices that have been made in relation to change and to learning. I wonder, do such choices and change offer the possibilities of being a useful focus for an indicator for education and ESD?
This brings me back to the question of what’s to count as sustainable development in the first place, and therefore what’s to count as ESD. As this is where I usually end up, it seems a good place to stop.
Respond