{"id":2297,"date":"2012-09-02T16:35:06","date_gmt":"2012-09-02T15:35:06","guid":{"rendered":"http:\/\/blogs.bath.ac.uk\/edswahs\/?p=2297"},"modified":"2012-09-02T16:35:06","modified_gmt":"2012-09-02T15:35:06","slug":"exploring-the-potential-benefits-and-risks-of-esd-indicators","status":"publish","type":"post","link":"https:\/\/blogs.bath.ac.uk\/edswahs\/2012\/09\/02\/exploring-the-potential-benefits-and-risks-of-esd-indicators\/","title":{"rendered":"Exploring the potential benefits and risks of ESD indicators"},"content":{"rendered":"<p>As I noted <a href=\"http:\/\/blogs.bath.ac.uk\/edswahs\/2012\/08\/19\/education-on-the-path-to-sustainability-\u2013-how-shall-we-know\/\">earlier<\/a>, on Monday, I'm giving a talk on\u00a0the potential benefits and risks of ESD indicators at a symposium at the University of Bern which explores an ESD indicator set \u00a0that has been generated by researchers and practitioners in Austria, Germany and Switzerland. \u00a0The outline of my talk is set out here:<\/p>\n<p><strong>Exploring the potential benefits and risks of ESD indicators <\/strong><\/p>\n<p>I\u2019ve been asked to give a general introduction to the potential benefits and risks of ESD indicators.\u00a0 I\u2019ll begin by saying something about indicators in general, and about sustainable development indicators, before moving on to ESD as it seems to me that what you can say about indicators applies more narrowly to sustainable development and ESD indicators as well. \u00a0In doing this I\u2019ll be drawing on the work that a number of us here did through Anglo-German Foundation grants a few years ago, and also on some more recent sustainable development indicator work in the UK. \u00a0First, some words of warning from three people smarter than me:<\/p>\n<blockquote><p><em>Where there is no reliable accounting and therefore no competent knowledge\u00a0of the economic and ecological effects of our lives, we cannot live lives that are\u00a0economically and ecologically responsible<\/em><em>.\u00a0 Wendell Berry<\/em><\/p>\n<p><em>That which is good and helpful ought to be<\/em><em> <\/em><em>growing and that which is bad and hindering<\/em><em> <\/em><em>ought to be diminishing. ... We therefore need,<\/em><em> <\/em><em>above all else, ... concepts that enable us to<\/em><em> <\/em><em>choose the right direction of our movement and<\/em><em> <\/em><em>not merely to measure its speed.\u00a0 EF Schumacher<\/em><em> <\/em><\/p>\n<p><em>We try to measure what we value. We come to value what we measure<\/em><em>.\u00a0\u00a0 Donella Meadows<\/em><\/p><\/blockquote>\n<p>I think we all understand this.\u00a0 The idea and use of indicators \u2013 in a general sense \u2013 is now widespread, and probably has been for a very long time, and we all experience such indicators everyday.\u00a0 This shows a range of indicator uses \u2026<\/p>\n<blockquote><p><em>Indicators can \u2026<\/em><\/p>\n<ul>\n<li><em>simplify or distil complex information<\/em><\/li>\n<li><em>monitor or account for the performance of systems <\/em><\/li>\n<li><em>measure the state, direction and rate of change<\/em><\/li>\n<li><em>act as a \u2018warning\u2019 system <\/em><\/li>\n<li><em>raise awareness and communicate information <\/em><\/li>\n<li><em>stabilise processes \u2013 particularly implementation <\/em><\/li>\n<li><em>aid accountability, governance, decision-making, dialogue<\/em><\/li>\n<li><em>address stakeholder interests about the state of the system<\/em><\/li>\n<\/ul>\n<\/blockquote>\n<p>They find particular uses in quality assurance systems. \u00a0Today, we are interested in the idea of indicators in the context of the evaluation of professional practice in the sense of a measure of <span style=\"text-decoration: underline\">progress<\/span> towards a social goal.\u00a0 Indicators need to be both written with clarity, and systematically developed, and there is a logic to their generation that goes something like this ...<\/p>\n<ul>\n<blockquote>\n<li><em>Indicators have to be linked to desired outcomes (which may be short- or longer\u2013term).\u00a0 They indicate the extent to which outcomes are met<\/em><\/li>\n<li><em>Such outcomes have to be linked to particular purposes (objectives)<\/em><\/li>\n<li><em>Purposes (objectives) have to be linked to overall organisational aims \/ vision \/ mission<\/em><\/li>\n<li><em>Key evaluation questions have to be posed, where indicators answer such questions.<\/em><\/li>\n<\/blockquote>\n<\/ul>\n<p>To evaluate is to form a judgement about such progress towards a social goal, and indicators help us understand the worth of such judgements.\u00a0 In our professional practice, the idea and use of <span style=\"text-decoration: underline\">key<\/span> performance indicators is now commonplace.\u00a0 But this does not make their use easy, or <span style=\"text-decoration: underline\">un<\/span>controversial.\u00a0 Whilst it is important to have indicators, it\u2019s also important not to accept uncritically what they appear to tell us.<\/p>\n<p>The UK government has had 68 sustainable development indicators in place for a long while now, and these are being reviewed this year. \u00a0The responsible ministry, Defra, says this:<\/p>\n<blockquote><p><em>The Government has committed to measure and report our progress through a new set of sustainable development indicators.\u00a0 The intention is that these will provide a high level transparent overview as to whether the UK is developing on a sustainable path [and] support our evidence base for policy development across Government.<\/em><\/p><\/blockquote>\n<p>All very laudable, but I think that this confidence is misplaced.\u00a0 However, what we\u2019re interested in seems quite similar, in principle, to this. \u00a0Looking at the existing sustainable development indicators shows the difficulties.\u00a0 This is the current UK indicator for <em>education \u2026<\/em><\/p>\n<blockquote><p><em>The proportion of <\/em><em>19 year-olds with Level 2 qualifications and above <\/em><\/p><\/blockquote>\n<p>This has the considerable merit of being easy to understand and measure, but what does it <em>really<\/em> tell us?\u00a0 Probably, only the number of 19 year-olds with Level 2 qualifications.\u00a0 This is because the link to sustainable development is tenuous at best, and it\u2019s an act of some faith that this provides <span style=\"text-decoration: underline\">any<\/span> measure that we\u2019re developing on a sustainable pathway.\u00a0 Maybe the indicator should be the proportion of PhD theses that address sustainable development \u2013 except we can recall David Orr telling us that sustainability problems were caused by people with PhDs.\u00a0\u00a0 But Orr\u2019s rhetoric is prone to over-simplification.<\/p>\n<p>We have another education indicator in the current UK set.\u00a0 This one is focused on ESD, and has been under development for about 10 years \u2013 and still is.\u00a0 Is this blank just because we weren\u2019t any good at thinking about it?\u00a0 Or because it\u2019s impossible to create anything valid and meaningful? \u00a0 I lean to the latter view, which is why I am so interested in what has been written in this report. \u00a0The UK\u2019s new proposals sidestep all these problems quite brilliantly.\u00a0 There is no mention of ESD, and even education just gets a passing glance.\u00a0 It\u2019s proposed that we should have headline, and supplementary indicators \u2026<\/p>\n<blockquote><p><em>The 12 provisional headline indicators are high-level outcome measures, and capture priority issues for making economic, environmental and social progress \u2026 in line with the UK Sustainable Development Strategy\u2019s \u2018guiding principles\u2019 of sustainable development: <\/em><\/p>\n<ul>\n<li><em>living within the planet\u2019s environmental limits<\/em><\/li>\n<li><em>ensuring a strong, healthy and just society<\/em><\/li>\n<li><em>achieving a sustainable economy<\/em><\/li>\n<li><em>promoting good governance; and <\/em><\/li>\n<li><em>using sound science responsibly.<\/em><\/li>\n<\/ul>\n<\/blockquote>\n<p>The headline indicators are:<\/p>\n<blockquote><p><strong><em>Economy<\/em><em> <\/em><em>Society<\/em><em> <\/em><em>Environment<\/em><\/strong><em> <\/em><\/p>\n<p><em>Economic prosperity \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Healthy life expectancy \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Greenhouse gas emissions<\/em><\/p>\n<p><em>Long term unemployment \u00a0 \u00a0 \u00a0 \u00a0Social Capital \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Natural resource use <\/em><\/p>\n<p><em>Poverty \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0Social mobility in adulthood \u00a0 \u00a0Wildlife &amp; biodiversity <\/em><\/p>\n<p><em>Knowledge &amp; skills \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Housing provision \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 Water availability <\/em><\/p><\/blockquote>\n<p>Those who think these things matter will have noticed that sustainable development has been split into what many see as its component parts. \u00a0Sadly, it never gets put back together. \u00a0The only reference to education is within the <em>knowledge and skills<\/em> headline indicator within the \u201cEconomy\u201d section.\u00a0 The documentation says this:<\/p>\n<blockquote><p><em>The value of knowledge and skills (as a proxy for human capital) per person of working age), where <\/em><em>h<\/em><em>uman capital is defined as \u201cthe knowledge, skills, competencies and attributes embodied in individuals that facilitate the creation of personal, social and economic well-being\u201d (OECD, 2001). <\/em><\/p><\/blockquote>\n<p>Rather ironically, OECD omits any reference to the environmental well-being which is at the heart of the issues we face.\u00a0 The UK measure is based on gender, age and the level of qualification acquired during participation in compulsory and post-compulsory education, in vocational and tertiary education, etc. None of this is straightforward, and there is no mention of ESD. \u00a0And you can see the problem immediately \u2013 it\u2019s really just the same as the old indicator \u2013 the proportion of the population with level 2 qualifications \u2013 though it <em>is<\/em> somewhat more sophisticated.\u00a0 This, when taken with the limited view of human capital, means that a lifetime of education and training will count whether or not there is any focus on sustainable development \u2013 any ESD if you like.\u00a0 So, it seems that ESD, if it happens at all, will count for nothing.<\/p>\n<p>However, there\u2019s a much bigger problem.\u00a0 It has been suggested that such headline indicators represent a sort of barometer or compass bearing.\u00a0 For example, if all the indicators can be lined up so that they are all pointing in the right direction, then we can conclude that we\u2019re making progress along the sustainable development pathway. \u00a0It\u2019s a pleasing, and reassuring, metaphor.\u00a0 However, there is a major problem with this view.\u00a0 Reseach that we were involved in, in 2004, noted this:<\/p>\n<ul>\n<blockquote>\n<li><em>If the headline indicators are broadly negative, we can tell that the overall position is not sustainable. <\/em><\/li>\n<li><em>Unfortunately, this does not mean that when they are all positive, the position necessarily is sustainable. <\/em><\/li>\n<li><em>Indeed, it is even possible that positive indicator results will operate perversely to move us off a sustainable pathway. <\/em><\/li>\n<\/blockquote>\n<\/ul>\n<p>In other words, if you fail to meet an indicator, then you know there\u2019s work to do; if you do meet it, however, there\u2019s always uncertainty about how appropriate the indicator was. \u00a0For example, if we were to succeed in keeping the global temperature rise below, say, 3<sup>o <\/sup>C, we\u2019d still have to wait for years to see whether that was enough.\u00a0 As the old clich\u00e9 has it: only time will tell. \u00a0In this sense, using indicators is rather like looking in a car\u2019s rear-view mirror. This enables us to see where we\u2019ve been \u2013 to see how well we\u2019re driving \u2013 the sort of social and temporal progress we are making.\u00a0 On a bad day we can see the mayhem we\u2019ve caused \u2013 the crashed cars and bodies in the road.\u00a0 It alerts us to how we\u2019ve been doing, but is an imperfect guide to the future. \u00a0The point about rear-view mirrors is to keep your eye on them \u2013 and to try to ensure that they don\u2019t distort what has happened.\u00a0 These are all important qualities needed in a mirror and an indicator:<\/p>\n<blockquote><p><em>accuracy, timeliness, clarity, field of view, ease of access, etc are all crucial in a mirror and indicator <\/em><em>and the view you have should be both valid and meaningful<\/em><em> <\/em><\/p><\/blockquote>\n<p>A real danger is that you produce indicators that are easy to measure, but which lack intellectual and policy coherence.<\/p>\n<p>Turning now to ESD, what are the potential benefits and risks of ESD indicators.\u00a0 Well, they are probably the same as for any other indicator \u2026<\/p>\n<blockquote><p><em>Benefit: <\/em><\/p>\n<p><em>You can get (useful) information on steps taken, and progress made, against aims and targets<\/em><\/p>\n<p><em> You can then change policy \/ practice in the light of information<\/em><\/p>\n<p><em>Risk: <\/em><\/p>\n<p><em>You may be measuring the wrong thing<\/em><\/p>\n<p><em>The information you get may be misleading<\/em><\/p>\n<p><em> It may take considerable time to know whether this is so<\/em><\/p><\/blockquote>\n<p>So much is straightforward, and applies to ESD as it does to, say, biodiversity.\u00a0 Both, of course, need a conceptual frame if indicators are to be developed.\u00a0 Actually, they need this, if <em>goals<\/em> are to be set.\u00a0 But, with biodiversity the link to sustainable development seems reasonably clear, whereas, with ESD, this never seems to be the case \u2013 at least to me.\u00a0 This is probably because the conceptual framing of biodiversity is more robust than for ESD.<\/p>\n<p>This is the UK\u2019s proposed new headline indicator measure for wildlife &amp; biodiversity \u2026<\/p>\n<blockquote><p><em>Indicator measure:<br \/>\nWildlife: Bird population indices - farmland birds, (b) woodland birds, (c) seabirds and (d) water and wetland birds <\/em><\/p>\n<p><em>Rationale:<br \/>\nNatural capital includes those elements of the environment that yield resources and ecosystem services, but we cannot determine our entire capital of natural resources and instead have to focus on selected aspects of the natural environment and changes in its state. <\/em><\/p><\/blockquote>\n<p>The rationale here is that populations of key species of birds are a good indicator of the broad state of wildlife and countryside because they occupy a wide range of habitats and key positions in the food chain. \u00a0In addition to this, there are 3 related supplementary indicators that focus on river water quality, fish stocks, the status of species &amp; habitats, and the UK\u2019s biodiversity impacts overseas.<\/p>\n<p>Here, the number of indicators is relatively few, and there is a sound conceptual rationale for the headline, keystone indicator.\u00a0 <em>Prima facie<\/em>, this seems good practice.\u00a0 There is little ambiguity, for example, about bird populations, though there will be methodological issues in the estimation of their numbers.\u00a0 Of course, these are not actually indicators; rather, they are indicator measures.\u00a0 The indicator will be how these measures change over time, with upward trends being desired.<\/p>\n<p>But,<em> is<\/em> the relationship between ESD and sustainable development robust?\u00a0 On past experience, I\u2019d say not enough, and the difference between practitioners on this matter can be considerable.\u00a0 Here are two frameworks that have been put forward in the UK higher education context to delineate how we might both think about ESD, and research its effectiveness:<\/p>\n<blockquote><p><em>ESD is \u2026<\/em><\/p>\n<p><em>Teaching which develops in students an understanding of the nature of society and its relationship with the environment, together with the capabilities and potential to promote justice in the distribution of economic, social and environmental assets, now and for the future.<\/em><\/p>\n<p><em>ESD is \u2026<\/em><\/p>\n<p><em>Teaching which contains a significant element of work related to either or both of the natural environment and natural resources, plus a significant element of work related to either economic or social issues (or to both).<\/em><\/p><\/blockquote>\n<p>The first sees sustainable development primarily in terms of <em>increasing<\/em> equity and global social justice whereas the second sees it more in terms of <em>reducing<\/em> the environmentally unsustainable impacts of economic and social development. \u00a0Are both these ESD?\u00a0 Can both <em>be<\/em> ESD?\u00a0 If so, are these poles on a continuum of ideas, or mutually inconsistent views?\u00a0 Either way, it\u2019s not useful to try to arbitrate between them.\u00a0 Except to say, of course, that any attempt to generate indicators for these approaches would lead to quite different outcomes.<\/p>\n<p>Looking at past practice, a lot of the effort that has gone into such indicators has been at the system level and has been about structures and inputs, and quality, rather than about outputs \u2013 that is, about what people know, can do, etc.\u00a0 There has also been the problem of saturation \u2013 that is, having an indicator for all occasions.\u00a0 UNECE couldn\u2019t resist this temptation, trying to cover, with its 50 plus indicators, just about everything. \u00a0And there has been what I call the substitution problem.\u00a0 Here are a number of attributes that might characterise an educational institution that took sustainability seriously, and which could be precursors of indicators.<\/p>\n<blockquote><p><em>This is an organisation where issues around sustainability \u2026<\/em><\/p>\n<ul>\n<li><em>have a high profile across the work of the institution, and in its community links<\/em><\/li>\n<li><em>are fundamental to, and integral across, the institution\u2019s work rather than being add-on or fragmented<\/em><\/li>\n<li><em>are raised in different settings, as appropriate, and treated as holistically as possible <\/em><\/li>\n<li><em>represent one of the institution\u2019s key ethical stances<\/em><\/li>\n<li><em>are focused on building students\u2019 capability for critical and independent thinking for the future<\/em><\/li>\n<\/ul>\n<\/blockquote>\n<p>These are somewhat generic attributes in that they are not obviously about \u2018sustainability\u2019, and other ideas could be substituted: <em>health<\/em>, for example, <em>faith<\/em>, or <em>enterprise<\/em> are all possibilities.\u00a0 This is not an isolated problem.\u00a0 I remember that my admiration for the logical consistency of an early UNECE indicator document was tempered by the fact that if you replaced ESD by <em>health education<\/em> (and sustainable development by, say, <em>well-being<\/em> \u2013 or even <em>road safety<\/em>) it still made remarkable sense. \u00a0It seems to me that this just won\u2019t do, as validity questions loom large.\u00a0 The risk in all this is that we have an approach to ESD indicators where they don\u2019t relate to sustainable development at all; rather they relate just to themselves.\u00a0 As my colleague (and UNECE indicator group member) Paul Vare has noted, many indicators obscure rather than illuminate, and any meaningful signal is lost in the noise of communication. \u00a0But, the following attributes about an educational institution that takes sustainability seriously are, I think, rather different.<\/p>\n<blockquote><p><em>A social learning community with a systemic view of the world and a heightened sense of place \u2026<\/em><\/p>\n<ul>\n<li><em>has a growing awareness of its environmental impact (footprint), has a strategy for steadily reducing it, and uses these as foci for learning<\/em><\/li>\n<li><em>values outdoor, environmental, experiential and exploratory learning as a means of effectively engaging with real-world issues in authentic settings<\/em><\/li>\n<li><em>is outward-looking, and whose work in embedded not only in its local context (socially, economically, environmentally, and culturally), but which has tangible links to real communities in other parts of the world <\/em><\/li>\n<li><em>recognises that place is now a global phenomenon that raises moral issues of inter-dependence and shared responsibility, in relation to social and environmental justice<\/em><\/li>\n<li><em>understands that [1] it can, and should, contribute not just to maximising learning and skills acquisition (its tradition role), but also to enhancing social cohesion, as well as [2] lessening its need for natural resources and its creation of waste, and maximising the efficiency of its buildings<\/em><\/li>\n<\/ul>\n<\/blockquote>\n<p>These seem obviously <em>about<\/em> sustainability, though there is no mention of this \u2013 or ESD. Stephen Gough has argued that, because sustainable development necessarily implies a change in the models we use to live by, then any ESD programme worth its salt would challenge these models by asking questions about how we live \u2013 and might live.\u00a0 He said that we should try to \u2018catch the change\u2019 \u2013 the dynamic in the educational process, and focus on the <span style=\"text-decoration: underline\">choices<\/span> that have been made in relation to change and to learning.\u00a0 Such choices and change offer the possibilities of being a useful focus for an indicator of ESD. \u00a0With ESD, however, the challenges seem considerable, particularly where materials and courses are concerned.\u00a0 I say this because it\u2019s not just a question of listing or counting, materials or study programmes.\u00a0 First, there are issues of quality to be taken into account \u2013 are the materials and programmes any good?\u00a0 In a similar way, one has to ask how healthy bird populations are.<\/p>\n<p>More importantly, perhaps, there seem also to be validity issues.\u00a0 How will we know whether materials or study programmes are actually about sustainability, or, perhaps, merely only about its component parts?\u00a0 In other words, it doesn\u2019t seem enough to count a study programme just because it says it\u2019s about sustainability or sustainable development, or because it focuses on component parts.\u00a0 Rather, don't we need greater conceptual coherence around ESD, and more consensus around\u00a0criteria as to what\u2019s to count?<\/p>\n<p style=\"margin: 0px 0px 18px;padding: 0px;border: 0px;font-size: 12px;vertical-align: baseline;color: #51555c;font-family: helvetica, arial, sans-serif;line-height: 18px\">\n","protected":false},"excerpt":{"rendered":"<p>As I noted earlier, on Monday, I'm giving a talk on\u00a0the potential benefits and risks of ESD indicators at a symposium at the University of Bern which explores an ESD indicator set \u00a0that has been generated by researchers and practitioners...<\/p>\n","protected":false},"author":237,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[5],"tags":[],"class_list":["post-2297","post","type-post","status-publish","format-standard","hentry","category-talks-and-presentations"],"acf":[],"jetpack_featured_media_url":"","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/posts\/2297","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/users\/237"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/comments?post=2297"}],"version-history":[{"count":0,"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/posts\/2297\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/media?parent=2297"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/categories?post=2297"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.bath.ac.uk\/edswahs\/wp-json\/wp\/v2\/tags?post=2297"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}